Continual learning-based Video Object Segmentation

Abstract

Machine learning models, specifically deep convolutional neural networks, have exceeded human-level performance in many research areas, such as object classification and voice recognition. However, they are not comparable to humans in real-world learning scenarios when the training data is non-i.i.d. infinite streaming data. An example of those real-world scenarios is continual learning. Continual learning, as a new area of research in the field of machine learning, has become quite popular. It is the process of learning sequential data that comprises different domains and tasks. The main feature of a continual leaning problem is that the learning model does not have access to previously trained data. The main challenge of training a machine learning model on sequential data is catastrophic forgetting, which happens when a model forgets the previously learned tasks after being trained on new ones. There are three different solutions for the problems of continual learning: prior-focused (regularization-based) solutions, likelihood-focused (rehearsal-based) solutions, and hybrid (ensemble) approaches. In this thesis, semi-supervised video object segmentation (VOS) is addressed as a continual learning problem specifically for long video sequences, and three solutions are proposed. The first solution is Gated-Regularizer Continual Learning (GRCL) which is a prior-focused solution. The second proposed solution is aligned with likelihood-focused solutions and is Reconstruction-based Memory Selection Continual Learning (RMSCL). The third proposed solution is a hybrid solution (Hybrid) that benefits from GRCL and RMSCL.

All of the proposed solutions improve the performance of two baseline Online VOS methods (LWL and JOINT) but they can augment any online VOS and improve its performance on long videos.