Some Mathematical Perspectives of Graph Neural Networks

Abstract

Many real-world entities can be modelled as graphs, such as molecular structures, social networks, or images. Despite coming with such a great expressive power, the complex structure of graphs poses significant challenges to traditional deep learning methods, which have been extremely successful in many machine learning tasks on other input data structures, such as texts and images data. Recently, there have been many attempts in developing neural network architectures on graphical data, namely graph neural networks (GNNs). In this thesis, we first introduce some mathematical notations for graphs and different aspects of training a feedforward neural network. We then discuss several notable GNN architectures including Graph Convolutional Neural Networks, Graph Attention Networks, GraphSAGE, and PinSAGE. Some special aspects of GNN training are also presented. Finally, we investigate a neighborhood sampling approach on PinSAGE to a product-user recommendation problem.