Variational Inference for Text Generation: Improving the Posterior
| dc.contributor.author | Balasubramanian, Vikash | |
| dc.date.accessioned | 2020-08-10T20:17:45Z | |
| dc.date.available | 2020-08-10T20:17:45Z | |
| dc.date.issued | 2020-08-10 | |
| dc.date.submitted | 2020-08-05 | |
| dc.description.abstract | Learning useful representations of data is a crucial task in machine learning with wide ranging applications. In this thesis we explore improving representations of models based on variational inference by improving the posterior. We explore two approaches towards this goal: 1) auxiliary losses to regularize the latent space and enforcing desired properties and 2) normalizing flows to develop more flexible posteriors to be used during variational inference. We propose a proximity based loss function that helps in disentanglement by regularizing the latent space based on similarity according to a criterion. We evaluate our model on a task of disentangling semantics and syntax in sentences and empirically show that our model successfully manages to learn independent subspaces that learn semantics and syntax respectively. We compare our model to existing approaches using automated metrics and human evaluation to show that our model is competitive. We also explore the effectiveness of normalizing flows for representation learning and generative modeling. We perform experiments that empirically show that variational inference with normalizing flows beats standard approaches based on simple posteriors across various metrics in text generation and language modeling. We also propose a variant of planar normalizing flows called block planar normalizing flows for use in disentanglement tasks. We perform ablation experiments to empirically show that our proposed block planar flows help in improving disentanglement. | en |
| dc.identifier.uri | http://hdl.handle.net/10012/16107 | |
| dc.language.iso | en | en |
| dc.pending | false | |
| dc.publisher | University of Waterloo | en |
| dc.subject | Machine Learning | en |
| dc.subject | Natural Language Processing | en |
| dc.subject | Natural Language Generation | en |
| dc.subject | Artificial Intelligence | en |
| dc.subject | Normalizing Flows | en |
| dc.subject | Disentanglement | en |
| dc.subject | Variational Inference | en |
| dc.title | Variational Inference for Text Generation: Improving the Posterior | en |
| dc.type | Master Thesis | en |
| uws-etd.degree | Master of Mathematics | en |
| uws-etd.degree.department | David R. Cheriton School of Computer Science | en |
| uws-etd.degree.discipline | Computer Science | en |
| uws-etd.degree.grantor | University of Waterloo | en |
| uws.contributor.advisor | Vechtomova, Olga | |
| uws.contributor.affiliation1 | Faculty of Mathematics | en |
| uws.peerReviewStatus | Unreviewed | en |
| uws.published.city | Waterloo | en |
| uws.published.country | Canada | en |
| uws.published.province | Ontario | en |
| uws.scholarLevel | Graduate | en |
| uws.typeOfResource | Text | en |
Files
Original bundle
1 - 1 of 1
Loading...
- Name:
- Balasubramanian_Vikash.pdf
- Size:
- 2.11 MB
- Format:
- Adobe Portable Document Format
- Description:
- Main thesis document
License bundle
1 - 1 of 1
No Thumbnail Available
- Name:
- license.txt
- Size:
- 6.4 KB
- Format:
- Item-specific license agreed upon to submission
- Description: