Integrating Column-Oriented Storage and Query Processing Techniques into Graph Database Management Systems
MetadataShow full item record
Column-oriented RDBMSs, which support traditional read-heavy analytics workloads, employ a specific set of storage and query processing techniques for scalability and performance, such as positional tuple IDs, column-specific compression, and block-oriented processing. We revisit these techniques in the context of contemporary graph database management systems (GDBMSs). GDBMSs support a new set of analytics workloads, such as fraud detection in financial transaction networks or recommendations in social networks, that are also read-heavy but have fundamentally different access patterns than traditional analytics workloads. We first review the data characteristics and query access patterns in GDBMS to identify components of GDBMSs where existing columnar techniques can and cannot directly be used. We then present the physical data layout of columnar data structures, new columnar compression, and query-processing techniques that are optimized for GDBMSs. Our techniques include a new compact vertex and edge ID scheme, a new null and empty list compression scheme based on prefix-sums, and list-based query processing. We have integrated our techniques into GraphflowDB, an in-memory GDBMS. Compared to uncompressed storage, our compression techniques has scaled the system by 3.55x with minimal performance overheads. Our null compression scheme outperforms existing columnar schemes in query performance, with minor loss in compression rate and achieves both higher compression rate and better query performance as compared to row-oriented storage techniques adopted by existing GDBMSs. Finally, our list-based query processor techniques improve query performance by 2.7x on a variety of path queries and significantly outperform their corresponding conventional versions.
Cite this version of the work
Pranjal Gupta (2020). Integrating Column-Oriented Storage and Query Processing Techniques into Graph Database Management Systems. UWSpace. http://hdl.handle.net/10012/16122
Showing items related by title, author, creator and subject.
El Gebaly, Kareem (University of Waterloo, 2007-08-17)Automatic physical database design tools rely on ``what-if'' interfaces to the query optimizer to estimate the execution time of the training query workload under different candidate physical designs. The tools use these ...
Mirsafian, Melicaalsadat (University of Waterloo, 2019-09-20)Ordered domains such as numbers and dates are common in real-life datasets. The SQL standard includes an ORDER BY clause to sort the results, and there has been research work on formalizing, reasoning about, and automatically ...
Anzum, Nafisa (University of Waterloo, 2020-09-28)Connections amongst real-world entities provide significant insights for numerous real-life applications in social networks, semantic web, road maps, finance, among others. Graphs are perhaps the most natural way to model ...