Symbolic Regression and Sequence Modelling with Conditional and Dynamic Language Models
| dc.contributor.author | Valipour, Mojtaba | |
| dc.date.accessioned | 2024-05-30T17:33:34Z | |
| dc.date.available | 2024-05-30T17:33:34Z | |
| dc.date.issued | 2024-05-30 | |
| dc.date.submitted | 2024-05-13 | |
| dc.description.abstract | In an era where the boundaries of machine learning are continuously being pushed, this thesis presents two more advancements in the field of deep learning and artificial intelligence, with a focus on symbolic regression and dynamic training methodologies for neural networks. The first major contribution, SymbolicGPT, introduces a novel approach to symbolic regression using a transformer-based language model. This model significantly outperforms traditional methods by leveraging the strengths of probabilistic language models for improved accuracy and efficiency. The second theme of this thesis revolves around dynamic training methodologies, aimed at enhancing the adaptability and computational efficiency of neural networks under varying constraints. Within this framework, we introduce DyLoRA and SortedNet as key innovations. DyLoRA offers a dynamic, search-free low-rank adaptation technique, enabling models to adjust their complexity on-the-fly without extensive retraining. SortedNet proposes a generalized framework for embedding multiple neural network architectures within a single model, facilitating efficient model selection and adaptation. Extending SortedNet, SortedLLama applies these principles to large language models, demonstrating efficient dynamic inference capabilities. | en |
| dc.identifier.uri | http://hdl.handle.net/10012/20630 | |
| dc.language.iso | en | en |
| dc.pending | false | |
| dc.publisher | University of Waterloo | en |
| dc.subject | Deep Learning | en |
| dc.subject | Natural Language Processing | en |
| dc.subject | Large Language Models | en |
| dc.subject | Symbolic Regression | en |
| dc.subject | Dynamic Inference | en |
| dc.subject | Modular Neural Networks | en |
| dc.subject | Anytime Inference | en |
| dc.subject | Low-Rank Adaptation | en |
| dc.title | Symbolic Regression and Sequence Modelling with Conditional and Dynamic Language Models | en |
| dc.type | Doctoral Thesis | en |
| uws-etd.degree | Doctor of Philosophy | en |
| uws-etd.degree.department | David R. Cheriton School of Computer Science | en |
| uws-etd.degree.discipline | Computer Science | en |
| uws-etd.degree.grantor | University of Waterloo | en |
| uws-etd.embargo.terms | 0 | en |
| uws.contributor.advisor | Ghodsi, Ali | |
| uws.contributor.affiliation1 | Faculty of Mathematics | en |
| uws.peerReviewStatus | Unreviewed | en |
| uws.published.city | Waterloo | en |
| uws.published.country | Canada | en |
| uws.published.province | Ontario | en |
| uws.scholarLevel | Graduate | en |
| uws.typeOfResource | Text | en |
Files
Original bundle
1 - 1 of 1
Loading...
- Name:
- Valipour_Mojtaba.pdf
- Size:
- 3.73 MB
- Format:
- Adobe Portable Document Format
- Description:
- Mojtaba Valipour PhD Thesis
License bundle
1 - 1 of 1
No Thumbnail Available
- Name:
- license.txt
- Size:
- 6.4 KB
- Format:
- Item-specific license agreed upon to submission
- Description: