A distance-based neural network model for sequence processing - Calvert, David (University of Waterloo, 1998)
This work describes the development of an artificial neural network architecture for sequence modeling and recall. The system described learns from an initial data set which it treats as an exemplar for all later comparisons. One of the principles of this work is to design a system that takes advantage of the architectural features common to neural networks. These features are many simple storage locations (weights) and a collection of simple processing elements. Although many methods currently exist for processing sequences, and the behavior of this network bears some similarity to these methods, it was always the intent to develop a system which operates in a unique manner. To that end, this system does not attempt to perform an existing sequence processing algorithm and simply cast the algorithm into a neural network. It instead tries to store a great deal of sequence information verbatim and then develop generalizations based upon this previously learned material. The network developed uses two classification networks, either Fuzzy ART or Self Organizing Maps, to contrast enhance the input and output sequence elements. An internal sequencing network builds an association between these classifications during training. The sequence layer also generates a distance value which indicates the similarity between these stored patterns and testing sequences. The system is tested upon several data sets which have results generated using other neural network techniques. Data sets include the NETtalk word-phoneme matchings and the Sante Fe time series competition data. Results from the testing show that the network generally performs as well or better than many neural techniques. However, this technique is much faster than all other neural methods which it is compared against. Several characteristics of the network are examined. These include the number of input presentations required for the network to focus on a solution and the use of the vigilance parameter in the classification networks as a distance measure.
Keywords: Harvested from Collections Canada