Contributions to Unsupervised and Semi-Supervised Learning
MetadataShow full item record
This thesis studies two problems in theoretical machine learning. The first part of the thesis investigates the statistical stability of clustering algorithms. In the second part, we study the relative advantage of having unlabeled data in classification problems. Clustering stability was proposed and used as a model selection method in clustering tasks. The main idea of the method is that from a given data set two independent samples are taken. Each sample individually is clustered with the same clustering algorithm, with the same setting of its parameters. If the two resulting clusterings turn out to be close in some metric, it is concluded that the clustering algorithm and the setting of its parameters match the data set, and that clusterings obtained are meaningful. We study asymptotic properties of this method for certain types of cost minimizing clustering algorithms and relate their asymptotic stability to the number of optimal solutions of the underlying optimization problem. In classification problems, it is often expensive to obtain labeled data, but on the other hand, unlabeled data are often plentiful and cheap. We study how the access to unlabeled data can decrease the amount of labeled data needed in the worst-case sense. We propose an extension of the probably approximately correct (PAC) model in which this question can be naturally studied. We show that for certain basic tasks the access to unlabeled data might, at best, halve the amount of labeled data needed.
Cite this work
David Pal (2009). Contributions to Unsupervised and Semi-Supervised Learning. UWSpace. http://hdl.handle.net/10012/4445
Showing items related by title, author, creator and subject.
Lu, Tyler (Tian) (University of Waterloo, 2009-05-05)The emergence of a new paradigm in machine learning known as semi-supervised learning (SSL) has seen benefits to many applications where labeled data is expensive to obtain. However, unlike supervised learning (SL), which ...
Filipowicz, Alexandre (University of Waterloo, 2013-05-22)The following two studies examined the influence of spatial regularities on our ability to learn and predict frequencies and sequences of events. Research into statistical and sequence learning has demonstrated that we can ...
Learning from Partially Labeled Data: Unsupervised and Semi-supervised Learning on Graphs and Learning with Distribution Shifting Huang, Jiayuan (University of Waterloo, 2007-08-20)This thesis focuses on two fundamental machine learning problems:unsupervised learning, where no label information is available, and semi-supervised learning, where a small amount of labels are given in addition to unlabeled ...