Brief Overview of Artificial Neural Network Systems

Last week I gave myself a crash course on Neural Networks and Machine Learning in an attempt to disseminate the concepts behind the core algorithms and processes behind artificial neural networks, machine learning and agents of artificial intelligence. The topics are often discussed from a mathematical perspective; I wanted to review a set of readings and make my own interpretation of what I found in terms of programming terminology for the sake of applying the concepts in my research.

Artificial Neural Networks (ANN) are assemblies of artificial neurons, sometimes called units, which are not designed to imitate actual neurons but modeled after biological findings of how the brain works. We are not quite at a stage of scientific discovery where we can say with certainty how neurons work individually or within a network in the brain but scientists have nonetheless created digital models of their theories of how neurons communicate. These ideas have carried over to computer science as inspiration for how to develop and model algorithms that seemingly have the capacity to perceive, learn, and make decisions based off of sensory data.

neuronAbove: a model of the neuron (Cichocki)

There are many artificial models of the neuron, but the general idea is that neurons take in a numerical array of information (often binary) and pass the array through a weighted threshold  in order to determine an output value. The output is also represented numerically, often as a value between 0 and 1 or -1 and 1, either in binary and/or analog format. The output is dependent on the weighted value threshold, which determines how ‘confident’ the neuron is that the input passes or fails its threshold (e.g. if at least 60% of the inputs are 1’s, output a 1 signal). In some models, the output can be fed back to a weighting algorithm within the neuron to determine whether the weighting threshold should be modified and to what degree.

nn
Above: ANN arrangements: a) FeedForward b) FeedBack c) cellular arrangement (Cichocki)

Neurons can be arranged in a number of ways within an Artificial Neural Network. Their outputs are fed to the inputs of other neurons which can arrangements such as the examples above.  FeedForward is essentially a chain of neurons, while FeedBack incorporates neurons dedicated to returning incoming information back into the chain. (Artificial) cellular arrangements involve neurons with multiple, nonlinear connections with each other.

perceptron

In many neural networks, these chains occur in layers such as described in the feedforward multi-layer Perceptron model above, one of many models used for designing neural networks. Multiple neurons with very similar input domains and tasks are arranged to communicate with neighboring layers but not neurons on the same layer. These networks can have multiple layers, but apparently run most effectively with three layers for most purposes. The layers themselves do not have to have the same number of neurons as any of the other layers. This arrangement of neurons allows the Perceptron to analyze data chunks instead of individually at a ‘pixel’ scale.

For example, in an image recognition process a specialized neural network might be tasked to analyze the direct capture of data from a camera in one layer, (possibly split across red, green, and blue channels), determine small clusters where brightness values form discrete lines or patterns in a second layer, then determine if these clusters form a particular shape in a third, then finally feed this information to a single neuron to determine how closeley the collective analysis from the third layer matches a ‘learned’ model for a cat (many machine learning algorithms undergo a ‘training process’ to prepare a neural network model to recognize certain features; those that do not are considered ‘unsupervised machine learning algorithms’).

In essence, ANN’s use the collective processing power of many smaller units in order to solve higher level problems. The fact that these machine learning algorithms are being brought up in our Body-Centric class is rather eye-opening with regards to how such technologies might be integrated with bodily technologies. A friend of mine referred me to an example of neural networks being integrated with prosthetic, where the prosthetic limb interpreted incoming electrical signals and gradually learned to perform actions reflective of its user’s intentions. If I remember correctly, I believe he was describing imitative machine learning, where the machine tries to imitate particular ‘memories’ that are implanted in it during its training process; perhaps such a device could imitate human hand motions. I wonder if and how some of these concepts could be carried forward to some of the sensory technologies we experimented with in the past, including vision, the EMG sensor, etc.

Sources:

Cichocki, Andrezej. “Neural Networks for Optimization and Signal Processing” Chchester, NY: J. Wiley, 1993. Print.

Buduma, Nikhil. “Fundamentals of Deep Learning” Sebastopol, CA: O’Reilly Media, Inc., 2017. Print.

Castano, Arnaldo Perez. “Practical Artificial Intelligence”. Apress, 2018. Print.