Services on Demand
Journal
Article
English (pdf)
Article in xml format
Article references
Automatic translation
Send this article by e-mailIndicators
-
Cited by SciELO -
Access statistics
Related links
-
Similars in
SciELO
Share
Permalink
Revista de Matemática Teoría y Aplicaciones
Print version ISSN 1409-2433
Rev. Mat vol.29 n.2 San José Jul./Dec. 2022
http://dx.doi.org/10.15517/rmta.v29i2.48885
Artículo
Deep gaussian processes and infinite neural networks for the analysis of EEG signals in Alzheimer's diseases
Procesos gausianos profundos y redes neuronales infinitas para el análisis de señales EEG en la enfermedad de Alzheimer
1Yachay Tech University, School of Mathematical and Computational Sciences, Urcuquí, Ecuador; krishna.roman@yachaytech.edu.ec
2Yachay Tech University, School of Mathematical and Computational Sciences, Urcuquí, Ecuador; andy.cumbicus@yachaytech.edu.ec
3Yachay Tech University, School of Mathematical and Computational Sciences, Urcuquí, Ecuador; sinfante@yachaytech.edu.ec
4Yachay Tech University, School of Mathematical and Computational Sciences, Urcuquí, Ecuador; rfonseca@yachaytech.edu.ec
Deep neural network models (DGPs) can be represented hierarchically by a sequential composition of layers. When the prior distribution over the weights and biases are independently identically distributed, there is an equivalence with Gaussian processes (GP) in the limit of an infinite network width. DGPs are non-parametric statistical models used to characterize patterns of complex non-linear systems due to their flexibility, greater generalization capacity, and a natural way of making inferences about the parameters and states of the system. This article proposes a hierarchical Bayesian structure to model the weights and biases of a deep neural network. We deduce a general formula to calculate the integrals of Gaussian processes with non-linear transfer densities and obtain a kernel to estimate the covariance functions. In the methodology, we conduct an empirical study analyzing an electroencephalogram (EEG) database for diagnosing Alzheimer's disease. Additionally, the DGPs models are estimated and compared with the NN models for 5, 10, 50, 100, 500, and 1000 neurons in the hidden layer, considering two transfer functions: Rectified Linear Unit (ReLU) and hyperbolic Tangent (Tanh). The results show good performance in the classification of the signals. Finally, we use the mean square error as a goodness of fit measure to validate the proposed models, obtaining low estimation errors.
Keywords: deep Gaussian process; Alzheimer disease; electroencephalogram.
Los modelos de redes neuronales profundos (DGPs) se pueden representar jerárquicamente mediante una composición secuencial de capas. Cuando la distribución prior sobre los pesos y sesgos son independientes idénticamente distribuidos, existe una equivalencia con los procesos Gaussiano (GP), en el límite de una anchura de red infinita. Los DGPs son modelos estadísticos no paramétricos y se utilizan para caracterizar los patrones de sistema no lineales complejos, por su flexibilidad, mayor capacidad de generalización, y porque proporcionan una forma natural para hacer inferencia sobre los parámetros y estados del sistema. En este artículo se propone una estructura Bayesiana jerárquica para modelar los pesos y sesgos de la red neuronal profunda, se deduce una formula general para calcular las integrales de procesos Gaussianos con funciones de transferencias no lineles, y se obtiene un núcleo para estimar las funciones de covarianzas. Para ilustrar la metodología se realiza un estudio empírico analizando una base de datos de electroencefalogramas (EEG) para el diagnóstico de la enfermedad de Alzheimer. Adicionalmente, se estiman los modelos DGPs, y se comparan con los modelos de NN para 5, 10, 50, 100, 500 y 1000 neuronas en la capa oculta, considerando dos funciones de transferencia: Unidad Lineal Rectificada (ReLU) y tangenge hiperbólica (Tanh). Los resultados demuestran buen desempeño en la clasificación de las señales. Finalmente, utilizó como medida de bondad de ajuste el error cuadrático medio para validar los modelos propuestos, obteniéndose errores de estimación bajos.
Palabras clave: procesos gausianos profundos; enfermedad de Alzheimer; electroencefalogramas.
Acknowledgements
We want to express our special thanks to the journal's anonymous reviewers for their suggestions to improve the manuscript. In the same way, we thank the authors of the previous works who provided us with the data for our research.
References
D, Abásolo; J, Escudero; R, Hornero; C, Gómez; P, Espino. Approximate entropy and auto mutual information analysis of the electroencephalogram in Alzheimer's disease patients, Medical & Biological Engineering & Computing 46 (2008), 1019-1028. Doi: 10.1007/s11517-008-0392-1 [ Links ]
D, Agrawal; T, Papamarkou; J, Hinkle. Wide neural networks with bottlenecks are deep Gaussian processes, Journal of Machine Learning Research 21 (2020), no. 175, 1-66. Available from: Link [ Links ]
N, Cedeño; G, Carillo; M,J, Ayala; S, Lalvay; S, Infante. Analysis of chaos and predicting the price of crude oil in Ecuador using deep learning models, in: T, Guarda; F, Portela & M,F, Santos. (Eds.) Advanced Research in Technologies, Information, Innovation and Sustainability, part of Communications in Computer and Information Science series 1485, Springer, Cham, 2021, 318-332. Doi: 10.1007/978-3-030-90241-4_25 [ Links ]
N, Cedeño; S, Infante. Estimation of ordinary differential equations solutions with Gaussian processes and polynomial chaos expansion, in: J,P, Salgado Guerrero; J, Chicaiza Espinosa; M, Cerrada Lozada; S, Berrezueta-Guzman. (Eds.) Information and Communication Technologies, part of Communications in Computer and Information Science series 1456, Springer, Cham , 2021, pp. 3-17. Doi: 10.1007/978-3-030-89941-7_1 [ Links ]
Y, Cho; L, Saul; L, Kernel. methods for deep learning, in: Y, Bengio; D, Schuurmans; J, Lafferty; C, Williams & A. Culotta. (Eds.) Advances in Neural Information Processing Systems 22 (2009), 342-350. Available from: Link [ Links ]
A, Damianou; N, Lawrence. Deep Gaussian processes, Proceedings of the 16th International Conference on Artificial Intelligence and Statistics, Scottsdale AZ, U.S.A. (2013), 207-215. Available from: Link [ Links ]
A, Garriga-Alonso; C,E, Rasmussen; L, Aitchison. Deep convolutional networks as shallow Gaussian procesesses, arXiv, 2019. 1808.05587 (stat.ML). Doi: 10.48550/arXiv.1808.05587 [ Links ]
I, Goodfellow; Y, Bengio; A, Courville. (2016). Deep Learning. The MIT Press, Cambridge MA, U.S.A. http://www.deeplearningbook.org [ Links ]
T, Hazan; T, Jaakkola. Steps toward deep kernel methods from infinite neural networks, arXiv, 2015. 1508.05133 (cs.LG). Doi: 10.48550/arXiv .1508.05133 [ Links ]
S, Infante; J, Ortega; F, Cedeño. Estimación de datos faltantes en estaciones meteorológicas de Venezuela vía un modelo de redes neuronales, Revista de Climatología 8, 51-70. Available from: Link [ Links ]
M,E, Khan; A, Immer; E, Abedi; M, Korzepa. Approximate inference turns deep networks into Gaussian processes, arXiv, 2019. 1906.01930 (stat.ML). Doi: 10.48550/arXiv .1906.01930 [ Links ]
A, Krizhevsky; I, Sutskever; G, Hinton. Image net classification with Deep convolutional neural networks, in: F, Pereira; C,J, Burges; L, Bottou & K,Q, Weinberger. (Eds.) Advances in Neural Information Processing Systems 25 (2012), 1097-1105. Available from: Link [ Links ]
Y, LeCun; Y, Bengio; G, Hinton. Deep learning, Nature 521 (2015), no. 7553, 436-444. Doi: 10.1038/nature14539 [ Links ]
J, Lee; Y, Bahri; R, Novak; S,S, Schoenholz; J, Pennington; J, Sohl-Dickstein, Deep neural networks as Gaussian processes, arXiv, 2018.1711.00165v3 (stat.ML). Doi: 10.48550/arXiv.1711.00165 [ Links ]
M, Luca; G, Barlacchi; B, Lepri; L, Pappalardo. Deep learning for human mobility: a survey on data and models, arXiv, 2020. 2012.02825 (cs.LG). Doi: 10.48550/arXiv .2012.02825 [ Links ]
D,J,C,MacKay. Probable networks and plausible predictions. A review of practical Bayesian methods for supervised neural networks, Network: Computation in Neural Systems, 6 (1992), no. 3, 469-505. Doi: 10.1088/0954-898X_6_3_011 [ Links ]
P, Martínez-Arias; R, Fonseca-Delgado; R, Salum; I, Amaro-Martín. Alzheimer's disease diagnosis system using electroencephalograms and machine learning models, Iberian Journal of Information Systems and Technologies. (2020) pp. 275-288. Available from: Link [ Links ]
A,G, Matthews; M, Rowland; J, Hron; R,E, Turner; Z, Ghahramani. Gaussian process behaviour in wide deep neural networks, arXiv, 2018.1804.11271v2 (stat.ML). Doi: 10.48550/arXiv.1804.11271 [ Links ]
D, Ming; D, Williamson; S, Guillas. Deep Gaussian process emulation using stochastic imputation, arXiv, 2021. 2107.01590 (stat.ML) Doi: 10.48550/arXiv.2107.01590 [ Links ]
A, Mosavi; S, Ardabili; A,R, Varkonyi-Koczy. (2020) List of deep learning models, in: A, R,Várkonyi-Kóczy. (Ed.) Engineering for Sustainable Future, Springer Nature, Cham Switzerland, 2020, pp. 202-214. Doi: 10.1007/978-3-030-36841-8_20 [ Links ]
R, Neal. Bayesian Learning for Neural Networks, Lecture Notes in Statistics, Springer Verlag, New York, 1996. Doi: 10.1007/978-1-4612-0745-0 [ Links ]
R, Novak; L, Xiao; J, Lee; Y, Bahri; G, Yang; J, Hron; . . . J, Sohl-Dickstein. Bayesian deep convolutional networks with many channels are Gaussian processes, arXiv, 2019. 1810.05148v4 (stat.ML). Doi: 10.48550/arXiv.1810.05148 [ Links ]
C,E, Rasmussen; C,K,I, Williams. Gaussian Processes for Machine Learning, The MIT Press, Cambridge MA, U.S.A. , 2006. Available from: http://gaussianprocess.org/gpml/ [ Links ]
H, Salimbeni; M, Deisenroth. Doubly stochastic variational inference for deep Gaussian processes, arXiv, 2017. 1705.08933v2 (stat.ML). Doi: 10.48550/arXiv.1705.08933 [ Links ]
A, Sauer; R,B, Gramacy; D, Higdon. Active learning for deep Gaussian process surrogates, arXiv, 2021. 2012.08015 (stat.ME). Accepted in Technometrics. Doi: 10.48550/arXiv.2012.08015 [ Links ]
S, Schoenholz; J, Gilmer; S, Ganguli; J, Sohl-Dickstein. Deep information propagation, arXiv, 2017. 1611.01232v2 (stat.ML). Doi: 10.48550/arXiv.1611.01232 [ Links ]
C,K,I,Williams. Computing with infinite networks, in: M,C, Mozer and M, Jordan & T, Petsche. (Eds.) Advances in Neural Information Processing Systems 9 (1997), 295-301. Available from: Link [ Links ]
A,G, Wilson; P, Izmailov. Bayesian deep learning and a probabilistic perspective of generalization, arXiv, 2020. 2002.08791v3 (cs.LG). Doi: 10.48550/arXiv.2002.08791 [ Links ]
I,H, Witten; E, Frank; M,A, Hall; C,J, Pal. Data Mining: Practical Machine Learning Tools and Techniques, 4th Edition. (2016) [ Links ]
Z, Zhao; M, Emzir; S, Särkkä. Deep state-space Gaussian processes Statistics and Computing. (2021), 31-75. Doi: 10.1007/s11222-021-10050-6 [ Links ]
Received: November 01, 2021; Revised: June 23, 2022; Accepted: June 28, 2022
This is an open-access article distributed under the terms of the Creative Commons Attribution License
