The impact of noise in analog deep and echo state neural networks

Abstract

Nowadays, neural networks are successfully used for solving many tasks, including image recognition and classification, improvement of audio recordings, speech recognition. The bottleneck of modern parallel neural networks is the speed of memory access and data processing. This problem can be solved if the network is fully hardware-implemented, when all mathematical operations are implemented at the physical level. The physical implementation of neural networks fundamentally changes the peculiarities of noise impact, since there are many internal noise sources with different properties in hardware networks.

The influence of analog neuron noise, its propagation and accumulation threaten to make this approach inefficient. In our previous works we have studied the impact of noise on deep neural networks. Here, the effects of correlated and uncorrelated additive noise are studied for recurrent network based on echo network. We show that the activation function affects noise accumulation in the same way as in deep networks. Uncorrelated noise can be suppressed by the system itself, while correlated noise accumulates in the system and can disrupt the propagation of some intermediate and output signals.

Moreover, we study the impact of additive and multiplicative noise which properties come from photonic neural network experimental setup.

This work was supported by the Russian Science Foundation (project no. 21-72-00002).

Speaker

Nadezhda Semenova
Saratov State University
Russia

Discussion

Ask question