SHARP neural network model

The SHARP neural network model is a mathematical model of a cortical area network that can be easily implemented on a computer with Von Neumann architecture. The acronym stands for Systolic Hebb Agnostic Resonance Perceptron.

The novel neural network model was presented the first time in 2015, by Luca Marchese, at WIRN – 25th Italian Workshop on Neural Networks – Societa Italiana Reti Neuroniche (SIREN) – International Institute for Advanced Scientific Studies (IIASS) [1][2].

The SHARP model is neural fuzzy classifier inspired by the architecture of the cerebral cortex.

The first part of the acronym “SHARP” stands for “Systolic Hebb” and refers to the learning rule applied to synapses on resonating neurons in a sequential flow. The second part stands for “Agnostic Resonance Perceptron” and refers to the perceptron-like multilayer architecture, where the capability of the neurons in each layer to resonate to a specific value of a feature is “agnostic” because it is not related to previously learned patterns.

This page does not deep the mathematical part of the algorithm but describes the characteristics of the network and its basic architecture.

Introduction

The most important characteristics of the neural network are:

• instantaneous learning;
• recognition time, in the software simulation on a Von Neumann computer, that is independent of the number of the learned patterns (i.e. compared with Fuzzy-ART and RCE);
• an architecture that can be mapped on a pulsed neural network (the specifications for a digital pulsed realization are not yet defined);
• simulation with basic math operations
• formal description with simple set theory; and
• rule extraction.

Architecture

The architecture of the network is a three-dimensional matrix (FIG.1) for which the three dimensions represent, respectively, a mini-column, a macro-column and a cortical area network. Each minicolumn is linked with a specific feature of the input stimulus and contains neurons resonating with specific values of the feature. The RF (Resonate and Fire) neurons in the mini-column are competitive modules (CM) working with a controlled WTA (Winner Takes All) mechanism. A macro-column is composed of mini-columns linked through synaptic connections that build paths representing configurations of the learned input patterns. The input vector distance from the stored prototype is measured using the BOX-distance with extreme flexibility because the influence field of a single vector component (one side of the Rectangle in Fig.2) can be different for any component (FIG.2). Fig.3 shows a prototype vector with different generalization ranges for each feature of the vector. The black internal lines represent the learned vector. The yellow area represents the generalization capability of the prototype.

Fig.4 in the right side shows a simple SHARP network with three categories (category neurons on the top) and prototypes composed of three components or features (the cilinders that represent the minicolumns). The delays imposed by the synapses that connect the neurons between minicolumns associated with the same category determine the final delay of the impulse emitted by the category neuron. If the neurons in the category layer are connected between themselves with inhibitory synapses, the consequence is that the first firing neuron inhibits all the other neurons.

The model can be viewed without the WTA connections on the category layer. In this case the recognition of the input pattern is "fuzzy", with the first firing category neuron representing the most probable catgorization. The amount of the delays of the following firing category neurons represents a "weighted view" of the categorization of the input pattern.

The network could be realized on hardware in a digital pulsed version, but the very interesting capability is related to the performance of the network when simulated on a Von Neuman computer. Indeed the netrork does not require long training algorithms and can read the data in one single step. Also fuzzy-ARTMAP or RCE requires two or three cycles on data in order to recover problems related to mismatch events in the previous steps that could have determined, respectively, an increment of the Vigilance (fuzzy-ARTMAP) or reduction of Neuron Influence Field: due to these reductions some previously learned patterns could result not identified. This problem cannot happen in the SHARP but on the counterpart the "plasticity vs stability" problem is not completely resolved in this network because the number of prototypes grows like a web of wires in a fixed structure with a fixed number of neurons (in Fuzzy-ARTMAP and RCE the prototypes are added to the network when needed). The problem of Fuzzy-ARTMAP and RCE is that they require the serial scanning of all the prototypes on a Von Neumann computer. In the SHARP model the recognition is performed in one step simply by addressing the neurons with the values of the input pattern and checking the values of the synapses between them.

An analogic version of the model is described in [3] but should be considered an exercise with the goal to demonstrate the possibility of using analogic neurons. Actually the paper is quite complicated and the model uses neurons that are not biologically plausible (the author admits the limit of this study)[3].

Instantaneous learning and recognition without scanning prototypes

The SHARP neural network algorithm learns in a single step and recognizes a pattern in a timeframe that is independent of the number of learned patterns. Therefore, the algorithm can be executed on a Von-Neumann computer with constant performance as an RBF (radial basis function) network executed on a correctly sized SIMD (single instruction multiple data) computer. The learning process can be described with the set theory [4]. The formulas presented in [4] are simplified for the software emulation and are not completely compliant with the pulsed realization (the total delay of the macro-column is approximated to the maximum single delay).

References

• [1] Luca Marchese – "Spatial-Temporal Entangled Sparse Distributed Storage and Sparse Distributed Code in the Systolic Hebb Agnostic Resonance Perceptron proposed as Hypothetical Model linking Mini and Macro-Column Scale Functionality in the Cerebral Cortex" –May 20, 2015 – WIRN – 25th Italian Workshop on Neural Networks – Societa Italiana Reti Neuroniche (SIREN) – International Institute for Advanced Scientific Studies (IIASS) – SPRINGER Edition
• [2] Luca Marchese – Advances in Neural Networks – SPRINGER – CHAPTERS 153–160: "Spatial-Temporal Entangled Sparse Distributed Storage (STE-SDS) and Sparse Distributed Code (SDC) in the Systolic Hebb Agnostic Resonance Perceptron (SHARP) Proposed as Hypothetical Model Linking Mini and Macro-Column Scale Functionality in the Cerebral Cortex".
• [3] Luca Marchese - "SHARP (Systolic Hebb – Agnostic Resonance – Perceptron): A Bio-Inspired Spiking Neural Network Model that can be simulated Very Efficiently on Von Neumann Architectures" – 2014 – American Journal of Intelligent Systems – 2014; 4(5): 159–195 – SAP ( Scientific & Academic Publishing)
• [4] Luca Marchese - "Systolic Hebb Agnostic Resonance Perceptron (SHARP): a Neural Network Model Inspired By the Topological Organization of the Cerebral Cortex, which Implements Virtual Parallelism on Von Neumann Computers" – Researchgate –