Event level pattern discovery in multivariate continuous data

Abstract

Patterns naturally arise in all types of data. The universal drive to uncover and understand these patterns has generated a wide range of pattern discovery tools and algorithms. The result is that patterns are sought after in many different forms, including rules, network weights, topologies, hierarchical trees, hypergraphs, membership functions, probability density functions and functional relationships. This diversity of pattern instantiations raises the question as to what is the fundamental information in the data.

In this thesis, the event is promoted as the fundamental information bearing entity in continuous data. Events, event associations and patterns are defined for the continuous sample space. From the event perspective, pattern discovery is viewed as the search for statistically significant events, where significance is judged according to the objective of the discovery. Hence, event-based pattern discovery is formulated as a mathematical optimization problem with statistical objective functions. A novel sequential and recursive methodology is proposed as the solution technique to the optimization task.

For two or three dimensional data, an approximation based on selective recursive partitioning is developed. The application of the discovered events to multivariate density estimation, smoothing and classification demonstrate the versatility of the event framework. An event-based measure of significant temporal change forms the basis for a time-dependent discovery algorithm. A new event synthesis procedure facilitates the analysis of high-dimensional data by selectively constructing high dimensional events. Parallel event plots serve as an interpretative visualization tool.

Experiments illustrate that on a classical front, event-based classification can be comparable to existing methodologies. From a discovery standpoint, the event approach offers unprecedented interpretability of complicated multivariate continuous data. Local dependencies are easily revealed and locally significant features are immediately identified. Temporal changes can be objectively assessed and elusive high-dimensional outliers can be detected. Subdimensional clusters, while traditionally challenging to unravel, are handled confidently with event-based pattern discovery.