Test-Time Training for Image Inpainting

dc.contributor.advisorVeksler, Olga
dc.contributor.authorGhiro, Genseric
dc.date.accessioned2022-06-21T12:46:58Z
dc.date.available2022-06-21T12:46:58Z
dc.date.issued2022-06-21
dc.date.submitted2022-06-07
dc.description.abstractImage inpainting is the task of filling missing regions in images with plausible and coherent content. The usual process involves training a CNN on a large collection of examples that it can learn from, to later apply this knowledge on new, unseen images with areas to complete. One important consideration is that each training dataset has its own domain, and domains in the image space are extremely diverse. For optimal results, the user image at inference time should belong to the same domain as the training examples, which greatly limits the range of inputs a trained network can be used for. Moreover, collecting new data is hard, and training on different datasets requires a great amount of computation power and time. In this thesis, we propose a test-time learning approach for inpainting. More specifically, we train a CNN like described above, but in addition, we use the user image with holes to build a new dataset, on which we continue training the network for a small number of iterations during test-time training. To the best of our knowledge, test-time training has never been done for inpainting. With this technique, our hope is that the model will learn to fill holes that are specific to the user image better. It also facilitates domain adaptation, which means that a wider range of input domains at inference time can produce acceptable results when using the same pretrained model. We obtain results demonstrating that even for a state-of-the-art model, our method can achieve significant improvements, in particular for perceptual scores. Moreover, if used in a software, it has the advantage of being optional, meaning that the user can choose to keep the original result if there is no significant improvement, which makes our framework beneficial in all situations.en
dc.identifier.urihttp://hdl.handle.net/10012/18414
dc.language.isoenen
dc.pendingfalse
dc.publisherUniversity of Waterlooen
dc.subjectinpaintingen
dc.subjectonline learningen
dc.titleTest-Time Training for Image Inpaintingen
dc.typeMaster Thesisen
uws-etd.degreeMaster of Mathematicsen
uws-etd.degree.departmentDavid R. Cheriton School of Computer Scienceen
uws-etd.degree.disciplineComputer Scienceen
uws-etd.degree.grantorUniversity of Waterlooen
uws-etd.embargo.terms0en
uws.contributor.advisorVeksler, Olga
uws.contributor.affiliation1Faculty of Mathematicsen
uws.peerReviewStatusUnrevieweden
uws.published.cityWaterlooen
uws.published.countryCanadaen
uws.published.provinceOntarioen
uws.scholarLevelGraduateen
uws.typeOfResourceTexten

Files

Original bundle

Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
Ghiro_Genseric.pdf
Size:
13.53 MB
Format:
Adobe Portable Document Format
Description:

License bundle

Now showing 1 - 1 of 1
No Thumbnail Available
Name:
license.txt
Size:
6.4 KB
Format:
Item-specific license agreed upon to submission
Description: