Iterative Feature Selection with Redundancy Accounting for the Neural Network Solution of Inverse Problems of Spectroscopy

Abstract

In neural network solution of inverse problems of spectroscopy, there is a need to reduce the dimension of the input data in order to achieve a more accurate and stable solution while reducing computational complexity. Since the intensities of the channels of the spectrum are most often used as input features, a large part of these features may be excessive: on the one hand, some of them may not carry useful information in relation to the problem being solved, and on the other hand, some subsets of the features (for example, adjacent channels) may carry similar information. The method used in this study is based on the iterative selection of features with the highest relevance with respect to the target variable, and on the elimination of redundant features with high statistical dependency among themselves.
In this study, we consider the physical problem of determining the concentration of heavy metal ions in water by Raman and absorption spectroscopy data, as well as by integration of data from both types of spectroscopy. We compare the quality of a neural network solution of the problem on the complete set of input features and on its subsets compiled using the selection method under consideration, as well as using traditional feature selection methods. Also, for the proposed feature selection method, various metrics for determining relevance and redundancy are considered.

Speaker

Nickolay O. Shchurov
Faculty of Physics, M.V. Lomonosov Moscow State University, Moscow, Russia
Russia

Discussion

Ask question