Math Information Retrieval using a Text Search Engine

Abstract

Combining text and mathematics when searching in a corpus with extensive mathematical notation remains an open problem. Recent results for math information retrieval systems on the math and text retrieval task at NTCIR-12, for example, show room for improvement, even though formula retrieval appears to be fairly successful. This thesis explores how to adapt the state-of-the-art BM25 text ranking method to work well when searching for math and text together. Symbol layout trees are used to represent math formulas, and features are extracted from the trees, which are then used as search terms for BM25. This thesis explores various features of symbol layout trees and explores their effects on retrieval performance. Based on the results, a set of features are recommended that can be used effectively in a conventional text-based retrieval engine. The feature set is validated using various NTCIR math only benchmarks. Various proximity measures show math and text are closer in documents deemed rel- evant than documents deemed non-relevant for NTCIR queries. Therefore it would seem that proximity could improve ranking for math information retrieval systems when search- ing for both math and text. Nevertheless, two attempts to include proximity when scoring matches were unsuccessful in improving retrieval effectiveness. Finally, the BM25 ranking of both math and text using the feature set designed for formula retrieval is validated by various NTCIR math and text benchmarks.