Show simple item record

dc.contributor.authorSnow, Mallory 21:23:24 (GMT) 21:23:24 (GMT)
dc.description.abstractModern Hopfield Networks (HNs) have the ability to store a large number of target memories (e.g. binary patterns) and then recall a memory in its entirety when prompted by a sub-set or perturbed version of it; in this sense, these networks demonstrate properties of a Content Addressable Memory (CAM). Associative memory operates in a similar manner, with memories being elicited by cues of partial or noisy information. Hence, these models provide a basis for modelling associative memory in humans. However, these modern HNs often use functions that rely on many-body synapses and are thus not accurate to neurobiology. More biologically realistic versions of modern HNs have been proposed, although these implementations often still utilize the softmax function. Computing the softmax for a single node in a layer requires the knowledge of all other nodes in that layer, resulting in non-local computations. The softmax function also causes each target memory stored in the network to correspond to the activation of a single hidden node. Such one- hot representations are not robust, and not observed in biological neural networks. In this thesis, we investigate more biologically plausible versions of a modern HN that uses the softmax function. Specifically, we introduce a more biological version of this network by using a local softmax function – the locLSE network. To ensure each target memory has a distributed hidden representation, we propose a network that applies population coding to the hidden layer via the Neural Engineering Framework (NEF) – the Distributed locLSE network. Both proposed networks can learn the connection weights using a local learning rule derived from gradient descent on the energy function. It is found that the proposed, more biologically accurate, versions of the original modern HN still demonstrate capabilities of a CAM at similar rates to the original. Lastly, the NEF is applied to the entire locLSE network by implementing it using a software tool called Nengo. The locLSE network in Nengo is found to behave as a CAM for small datasets of target memories, and provided there are enough neurons in the network.en
dc.publisherUniversity of Waterlooen
dc.subjectassociative memoryen
dc.subjectHopfield networksen
dc.subjectbiological plausibilityen
dc.subjectdynamical systemsen
dc.titleBiological Plausibility in Modern Hopfield Networksen
dc.typeMaster Thesisen
dc.pendingfalse R. Cheriton School of Computer Scienceen Scienceen of Waterlooen
uws-etd.degreeMaster of Mathematicsen
uws.contributor.advisorOrchard, Jeff
uws.contributor.affiliation1Faculty of Mathematicsen

Files in this item


This item appears in the following Collection(s)

Show simple item record


University of Waterloo Library
200 University Avenue West
Waterloo, Ontario, Canada N2L 3G1
519 888 4883

All items in UWSpace are protected by copyright, with all rights reserved.

DSpace software

Service outages