Integrating Column-Oriented Storage and Query Processing Techniques into Graph Database Management Systems
MetadataShow full item record
Column-oriented RDBMSs, which support traditional read-heavy analytics workloads, employ a specific set of storage and query processing techniques for scalability and performance, such as positional tuple IDs, column-specific compression, and block-oriented processing. We revisit these techniques in the context of contemporary graph database management systems (GDBMSs). GDBMSs support a new set of analytics workloads, such as fraud detection in financial transaction networks or recommendations in social networks, that are also read-heavy but have fundamentally different access patterns than traditional analytics workloads. We first review the data characteristics and query access patterns in GDBMS to identify components of GDBMSs where existing columnar techniques can and cannot directly be used. We then present the physical data layout of columnar data structures, new columnar compression, and query-processing techniques that are optimized for GDBMSs. Our techniques include a new compact vertex and edge ID scheme, a new null and empty list compression scheme based on prefix-sums, and list-based query processing. We have integrated our techniques into GraphflowDB, an in-memory GDBMS. Compared to uncompressed storage, our compression techniques has scaled the system by 3.55x with minimal performance overheads. Our null compression scheme outperforms existing columnar schemes in query performance, with minor loss in compression rate and achieves both higher compression rate and better query performance as compared to row-oriented storage techniques adopted by existing GDBMSs. Finally, our list-based query processor techniques improve query performance by 2.7x on a variety of path queries and significantly outperform their corresponding conventional versions.
Cite this version of the work
Pranjal Gupta (2020). Integrating Column-Oriented Storage and Query Processing Techniques into Graph Database Management Systems. UWSpace. http://hdl.handle.net/10012/16122
Showing items related by title, author, creator and subject.
El Gebaly, Kareem (University of Waterloo, 2007-08-17)Automatic physical database design tools rely on ``what-if'' interfaces to the query optimizer to estimate the execution time of the training query workload under different candidate physical designs. The tools use these ...
Krastnikov, Simeon (University of Waterloo, 2020-12-10)A major algorithmic challenge in designing applications intended for secure remote execution is ensuring that their execution is oblivious to their inputs, in the sense that their memory access patterns do not leak sensitive ...
Khaliq, Shahid (University of Waterloo, 2019-09-17)Adjacency lists are the most fundamental storage structure in existing graph database management systems (GDBMSs) to index input graphs. Adjacency lists are universally linked-list like per-vertex structures that allow ...