The influence of noise on neural networks

Abstract

Neural networks are a powerful tool for data processing and demonstrate high performance compared to traditional machine learning methods, making them increasingly sought after in various fields. This paper examines how different types of noise affect the training of neural networks, which plays a crucial role in enhancing their performance and robustness.
There are hardware neural networks whose operation is implemented not through computer simulation but physically as a separate device. Such neural networks often encounter noise from internal components, which can significantly reduce their efficiency and reliability. We focused on the main types of noise: additive and multiplicative.
The model under investigation was trained using the gradient descent method. This allows for weight adjustments based on randomly selected subsets of data, helping to avoid local minima and making the training more robust. During the training process, the model adjusts the weights and parameters of the network in such a way as to minimize error and increase the accuracy of handwritten digit recognition.
In the course of the experiments, it was found that different types of noise, both in already trained networks and those arising during training, affect the accuracy of neural networks differently in hidden and output layers. In the hidden layer, the impact of additive noise leads to a rapid decrease in accuracy, while multiplicative noise proves to be more resilient to negative effects.

This work was supported by the Russian Science Foundation (project No. 23-72-01094) https://rscf.ru/project/23-72-01094/

Keywords: neural networks, noise, machine learning

Speaker

Ivan Kolesnikov
Saratov State University
Russia

Discussion

Ask question