Researchers design a new hardware-based artificial neural network capable of learning and process sensory data as the human brain does

June 5, 2018 | Author: neuromimeTICs

The electronic circuits of the future will be able to self-organize in order to classify the information as the human brain does. They are neuromorphic electronic systems, so called because they emulate the architecture and behavior of the human brain. Now, researchers of neuromimeTICs group have proposed a new artificial neural network of this type based on hardware, with physical components such as the memristors, capable of emulating the way that the brain processes sensory stimuli.


Implementing an electronic system of neural networks that imitates the human brain, by using electronic components, capable of self-organizing and learning to perform tasks such as classifying colours or obtaining a differential medical diagnosis, is feasible according to the proof of concept obtained by the Marta Pedró and Javier Martín, researchers of the neuromimeTICs group, of the Universitat Autònoma de Barcelona (UAB), with the support of the National Centre for Microelectronic of Barcelona, which has provided the necessary devices for this research. The conclusion is part of the presentation "Proposal of a Self-Organizing neural network based on memristors" presented at a meeting held by the neuromimeTICs group on May 10th of this year 2018 in the same School of Engineering of the UAB

The talk brings together the results of the research carried out in the first year and a half of the doctoral thesis by Marta Pedró, principal author of the work, electronic engineer and PhD student at the School of Engineering of the UAB, psychologist, also by the UAB, researcher of the aforementioned group of Reliability of Electron Devices and Circuits Group, as well as a member of the neuromimeTICs group. It has been presented for the first time at the 2018 IEEE International Reliability Physics Symposium (IRPS) conference on March 13th, 2018, in San Francisco, California.

"We have been able to provide a proof of concept, at the simulation level, so that an adaptive hardware system can be implemented, that is a physical neural network with the capacity to self-organize itself, that manages to emulate the behavior of sensory cortex of the brain: similar signals or input data causes a similar response in neurons that are located close together, while very different input data activate neurons that are distant", says Pedró.

A neuromorphic electronic system

The neuromorphic electronic systems imitate the human brain and synapses or neural connections that are generated within different parts of this organ (such as lateral sulcus, temporal lobe, primary auditory cortex or secondary auditory cortex). In the cortical areas that processes sensory stimuli, the neurons are organized topologically, so that those that are closest to each other, in the same cerebral region, respond similarly to similar stimuli.

The neuromorphic circuit designed by UAB researchers, in which a preexisting learning algorithm has been adapted, has succeeded in reconstructing this topological organization of neurons. It is an architecture that consists of a synaptic matrix that connects a series of components that simulate the neurons of the input and output layers of the neural network. Through the learning algorithm, this system is capable of self organizing, learning and functioning, without any need for a mathematical computation. The learning is produced by means of Spike-timing-dependent plasticity (STDP) or plasticity depending on the time of firing between the neurons, which consists in emulating a biological process according to which the connection between two neurons is adjusted according to the temporal correlation between the electric impulses that emit the own neurons "Neurons that fire together, wire together," summarizes Pedró.

Source: Neural Networks Research Centre, Helsinki University of Technology. (1997). World Poverty Map.

Once the system has been configured, a simulation has been performed to test its operation. For this, a set of 10,000 data has been provided. Input neurons have received as an incentive the RGB primary colours (red, green and blue) and the response of the system has been that neurons have organized in groups (clusters), each of which is sensitive to a single stimulus (or colour) or a combination of them.

The size of the obtained clusters is related to the number of times that each of the three colours has appeared. "The result is that the system has been able to learn to classify colours, so that colours that it has never met before can be assigned to the colour cluster that is most similar to it. The key to this is that the algorithm is able to extract relevant statistical information on the input data it receives, without knowing a model that can predict an output to a particular entry", explains Pedró.

Source: Neural Networks Research Centre, Helsinki University of Technology. (1997). World Poverty Map.

Experiment and simulation

The work has consisted of an experimental part and another of simulation. "We wanted to verify that it is possible to implement a system based on memristors, whose architecture and behavior resembles that of the biological brain. The experimental work was necessary to determine what type of electrical signal and waveforms are required to be able to excite our devices, so that they behave like a synapse, inducing small changes controlled in their conductivity", explains Pedró.

Subsequently, it was necessary to perform a simulation to verify three aspects: "Firstly, we could reproduce the results of the experiment, that is, we modelled the electrical behavior of the devices; secondly, we could develop a simulation in the system, to be able to work and adapt the learning algorithm to the circuitry or architecture that today can be manufactured with these devices; and, thirdly, that we could imitate the learning algorithms that have been proposed to model the sensory cortical areas of the brain. For us, it has been a key to take inspiration from the actual biology instead of opting for more abstract algorithms that are used in software-based artificial intelligence", adds Pedró.

The contribution: the "proof of concept"

Thanks to the simulation carried out, Pedró emphasizes that "we have been able to provide a proof of concept that an adaptive hardware system can be implemented, that means a physical neural network with the ability of self-organize, that manages to emulate the behavior of the sensory cortical areas of the brain. In practice, this means that similar signals or input data between themselves cause a similar response in neurons that are close together, while very different input data activate distant neurons."

After this research, this engineer states that the next logical step would be to verify the concept test at the experimental level, implementing the learning algorithm in a "physical" chip, although she warns that in order to carry out this experiment, funds are needed, since this type of systems must be manufactured.

Prospects for the future

We are investigating how to do medical diagnoses when the relationship between the risk factors and the prevalence of the disease is not clear or is not known

— Marta Pedró

Based on these results, another perspective that opens up is that "new architectures can be investigated to replicate more complex human learning behaviors, such as learning to associate concepts such as the red word with the red colour; to perform certain actions based on external stimuli; or to order the ignition of a red LED when the red word is entered into the chip by means of electrical signals", says Pedró as different examples. But, without any doubt, the greatest benefit of the research in this field could come from its medical applications. "We are investigating how to perform medical diagnoses when the relationship between the risk factors and the prevalence of the disease is not clear or is not known", emphasizes Pedró.

"It is important to point out that the main difference of this proposal to conventional electronics is that in conventional electronics there is an external user that programs the function that the chip will execute, while here the actions of the chip are defined through external stimuli, that is, through the experience”, highlights the engineer.