Semantic Order Compatibilities and Their Discovery
Loading...
Date
2019-09-20
Authors
Mirsafian, Melicaalsadat
Journal Title
Journal ISSN
Volume Title
Publisher
University of Waterloo
Abstract
Ordered domains such as numbers and dates are common in real-life datasets. The SQL standard includes an ORDER BY clause to sort the results, and there has been research work on formalizing, reasoning about, and automatically discovering order dependencies among columns in a table. However, a crucial assumption made in research and practice is that the order over a column is syntactic: numbers are ordered numerically, strings lexicographically and dates chronologically. To the best of our knowledge, this work is the first to relax this assumption. We present a generalized definition of order compatibilities that allows semantic orders such as (low, medium, high) or (excellent, very good, good, average, poor). We show that in general, validating whether there exists a semantic order relationship between columns is NP-complete, with some tractable special cases. We give an algorithm to automatically discover semantic order relationships in the data, we provide examples of interesting orders found by our algorithm that were missed by existing algorithms, and we show that the NP-complete validation cases do not occur frequently in practice.
Description
Keywords
databases, order dependencies, data profiling, order in databases