Using Method Integration Transfer Learning for Neural Network Solution of an Inverse Problem in Optical Specroscopy

Abstract

In various fields of industry and ecology there is an urgent need for simple, non-contact, highly selective express methods to determine concentrations of dissolved ions. Traditional chemical and analytical methods provide a high accuracy of determining the concentrations, but these methods are contact, their implementation requires a long time, good sample preparation and consumption of expensive reagents, and they are individual for each ion [1,2]. Optical spectroscopy methods have no such drawbacks: spectra of solutions can be obtained quickly, remotely, without special preparation of samples (in vitro) [3]. However, for many methods of optical spectroscopy no analytical and/or direct numerical solution for the problem of determination of concentrations of each component in multi-component solutions by spectra is available. Therefore, the application of machine learning (ML) methods to solve these problems is being actively investigated [4,5]. In addition, this problem belongs to the class of inverse problems, which are often ill-posed or ill-conditioned. This causes high sensitivity of the solution to noise in the data, and, as a result, reduction in the accuracy of the solution.

In the preceding study [6, 7] it has been suggested to use integration of optical spectroscopy methods to increase the accuracy of the solution obtained by machine learning methods. We consider joint use of Raman spectroscopy and optical absorption spectroscopy methods to determine the concentrations of heavy metal ions in water. This complex IP is solved by artificial neural networks (ANN) as an ML method. It is demonstrated that when one of the methods is strong by its results (absorption spectroscopy), and the other one is weak (Raman spectroscopy), their joint application does not allow one to improve the results of the strong method.

The reason for this effect may be related to the ANN training procedure used, which consists in searching the error minimum using gradient optimization methods. The disadvantage of gradient optimization methods is that the result depends on the initial point of the search, which may lead to finding a local error minimum, rather than a global one. In the case of the integration of spectroscopic methods, this can be described in such a way that it is “easier” for the neural network to pay attention to the data of the “stronger” method only in the training process, ignoring the “weak” one. To reduce the influence of this factor, it is possible to use other training methods that are less susceptible to it, for example, genetic algorithms [8]. Another way to reduce the influence of this factor is to change the training procedure so that it starts from an initial point close to the global minimum. This approach is implemented in such methods as pre-training [9] and transfer training [10]. In this paper, we consider the adjustment of this approach to the integration of optical spectroscopy methods, which consists in initial training neural networks on the data of only the weaker method (Raman spectroscopy), followed by additional training on the data of two methods (Raman and absorption spectroscopy).

This study has been performed at the expense of the grant of the Russian Science Foundation (project no. 19-11-00333).

1. Crompton T.R. Determination of anions in natural and treated waters. Taylor&Francis, 2002, 828 p.

2. Michalski R., Jablonska M., Szopa S., Łyko A. Application of Ion Chromatography with ICP-MS or MS Detection to the Determination of Selected Halides and Metal. Metalloids Species, Critical Reviews in Analytical Chemistry, 2011, V. 41, pp. 133-150

3. Rudolph W.W., Irmer G. Raman and Infrared Spectroscopic Investigation on Aqueous Alkali Metal Phosphate Solutions and Density Functional Theory Calculations of Phosphate-Water Clusters. Applied Spectroscopy, 2007, V.61 (12), pp.274A–292A

4. Isaev I., Burikov S., Dolenko T., Laptinskiy K., Vervald A., Dolenko S. Joint application of group determination of parameters and of training with noise addition to improve the resilience of the neural network solution of the inverse problem in spectroscopy to noise in data. Lecture Notes in Computer Science, 2018, V. 11139, pp. 435–444.

5. Isaev I., Burikov S., Dolenko T., Laptinskiy K., Dolenko S. Artificial neural networks for diagnostics of water-ethanol solutions by Raman spectra. Studies in Computational Intelligence, 2019, V. 799, pp. 167–175.

6. Isaev, I., Trifonov, N., Sarmanova, O., Burikov, S., Dolenko, T., Laptinskiy, K., Dolenko, S. Joint application of raman and optical absorption spectroscopy to determine concentrations of heavy metal ions in water using artificial neural networks. Proceedings of SPIE, Saratov Fall Meeting 2019: Laser Physics, Photonic Technologies, and Molecular Modeling, 2020, V. 11458, pp. 114580R.

7. Isaev, I., Sarmanova, O., Burikov, S., Dolenko, T., Laptinskiy, K., Dolenko, S. Study of resilience of neural network solution of inverse problem based on integration of optical spectroscopic methods to noise in data. Proceedings of SPIE, Saratov Fall Meeting 2020: Laser Physics, Photonic Technologies, and Molecular Modeling, 2021, V. 11846, pp. 1184611.

8. Montana, D. J., Davis, L. Training feedforward neural networks using genetic algorithms.
Proceedings of the 11th International Joint Conference on Artificial Intelligence (IJCAI-89), 1989, V. 1, pp. 762-767.

9. Erhan, D., Bengio, Y., Courville, A., Manzagol, P. A., Vincent, P., Bengio, S. Why Does Unsupervised Pre-training Help Deep Learning?. Journal of Machine Learning Research, 2010, V. 11, pp. 625-660.

10. Pan S. J., Yang Q. A survey on transfer learning. IEEE Transactions on knowledge and data engineering, 2009, V. 22, № 10, pp. 1345-1359.

Speaker

Igor Isaev
D.V. Skobeltsyn Institute of Nuclear Physics, M.V. Lomonosov Moscow State University, Moscow, Russia
Russia

Discussion

Ask question