Simple Convolutional Neural Networks with Linguistically-Annotated Input for Answer Selection in Question Answering

Abstract

With the advent of deep learning methods, researchers have been increasingly preferring deep learning methods over decades-old feature-engineering-inspired work in Natural Language Processing (NLP). The research community has been moving away from otherwise dominant feature engineering approaches; rather, is gravitating towards more complicated neural architectures. Highly competitive tools like part-of-speech taggers that exhibit human-like accuracy are traded off for complex networks, with the hope that the neural network will learn the features needed. In fact, there have been efforts to do NLP "from scratch" with neural networks that altogether eschew featuring engineering based tools (Collobert et al, 2011). In our research, we modify the input that is fed to neural networks by annotating the input with linguistic information: POS tags, Named Entity Recognition output, linguistic relations, etc. With just the addition of these linguistic features on a simple Siamese convolutional neural network, we are able to achieve state-of-the-art results. We argue that this strikes a better balance between feature vs. network engineering.