Keywords:-
Article Content:-
Abstract
Developing neural network models is a significant research area, which involves different optimization problems. For instance, feature set optimization, searching of the best neural network architecture, hyper-parameters and weights optimization. All these problems do not have one solution. The most popular approaches here are Gradient-based methods and Evolutionary methods. Gradient-based methods are quite useful for neural connections weights optimization and for Neural Architecture Search. Evolutionary algorithms (NEAT, HyperNEAT, CoDeepNEAT etc.) are state-of-the-art methods for neural networks topology optimization. There are numerous heuristic methods like Particle Swarm Optimization that also could be used in the neural networks structural tuning.
This paper discusses the neural networks models and evolutionary methods in environmental sound classification systems. We consider Snapshot algorithm also in order to build ensemble of solutions.
References:-
References
Molnar C., König G., Herbinger J., Freiesleben T., Dandl S., Scholbeck C.A., Casalicchio G., Grosse-Wentrup M. & Bischl B. (2020). Pitfalls to Avoid when Interpreting Machine Learning Models. Preprint arXiv.org, 10 p, URL:
https://arxiv.org/abs/2007.04131
Ravichandiran S. (2018). Hands-On Meta Learning with Python. Birmingham: Packt Publishing.
Bohrer J.S, Grisci B.I. & Dorn M. (2020). Neuroevolution of Neural Network Architectures Using CoDeepNEAT and Keras. Preprint arXiv.org, 29 p.,
URL: https://arxiv.org/abs/2002.04634
Sarmah D.K. (2020) A Survey on the Latest Development of Machine Learning in Genetic Algorithm and Particle Swarm Optimization. In: Kulkarni A., Satapathy S. & etc. Optimization in Machine Learning and Applications. Algorithms for Intelligent Systems. Springer, Singapore. DOI: https://doi.org/10.1007/978-981-15-0994-0_6.
Salamon J., Jacoby C. & Bello J.P. (2014). A Dataset and Taxonomy for Urban Sound Research. 2014 ACM Multimedia Conference (MM '14): Proceedings of the 22nd ACM international conference on Multimedia. (Orlando, 3-7 November 2014). Orlando, Florida, pp. 1041-1044,
DOI: https://doi.org/10.1145/2647868.2655045
Elsken T., Metzen J.H. & Hutter F. (2019). Neural Architecture Search: A Survey. Journal of Machine Learning Research Vol. 20., pp. 1-21.
Hospedales T., Antoniou A., Micaelli P. & Storkey A. (2020). Meta-Learning in Neural Networks: A Survey. Preprint arXiv.org, 23 p,
URL: https://arxiv.org/abs/2004.05439
Sun Y., Xue B., Zhang M., Yen G.G. & Lv J. (2020). Automatically Designing CNN Architectures Using the Genetic Algorithm for Image Classification. Preprint arXiv.org, 14 p, URL: https://arxiv.org/abs/1808.03818
Bakhshi A., Noman N., Chen Z., Zamani M. & Chalup S. (2019). Fast Automatic Optimisation of CNN Architectures for Image Classification Using Genetic Algorithm. Congress on Evolutionary Computation (CEC 2019): Proceedings of the 2019 IEEE Congress on Evolutionary Computation. (Wellington, 10-13 June 2019). Wellington, New Zealand, pp. 1283-1290,
DOI: https://doi.org/10.1109/CEC.2019.8790197
Kwasigroch A., Grochowski M. & Mikołajczyk M. (2019). Deep neural network architecture search using network morphism. Methods and Models in Automation and Robotics (MMAR): Proceedings of the 2019 24th International Conference. Międzyzdroje, Poland, pp. 30-35,
DOI:https://doi.org/10.1109/MMAR.2019.8864624
Radiuk P. & Kutucu H. (2020). Heuristic Architecture Search Using Network Morphism for Chest X-Ray Classification. Proceedings of the 1st International Workshop. (Khmelnitskyi, 10-12 June 2020). Khmelnitskyi, Ukraine, pp. 107-121.
URL: http://ceur-ws.org/Vol-2623/paper11.pdf.
Tirumala S.S. (2020). Evolving deep neural networks using coevolutionary algorithms with multi-population strategy. Neural Computing and Applications. DOI: https://doi.org/10.1007/s00521-020-04749-2
Liu H., Simonyan K. & Yang Y. (2019). DARTS: Differentiable Architecture Search. Preprint arXiv.org, 13 p,
URL: https://arxiv.org/abs/1806.09055/
Zoph B. & Le Q.V. (2017). Neural Architecture Search With Reinforcement Learning. Preprint arXiv.org, 16 p,
URL: https://arxiv.org/abs/1611.01578.
Baker B., Gupta O., Raskar R., Naik N. (2017) Accelerating Neural Architecture Search using Performance Prediction. Preprint arXiv.org, 14 p, URL: https://arxiv.org/abs/1705.10823.
Bender G., Kindermans P.-J., Zoph B., Vasudevan V. & Le Q. (2018). Understanding and Simplifying One-Shot Architecture Search. PMLR 80: Proceedings of the 35th International Conference on Machine Learning. (Stockholm, 10-15 July 2018). Stockholm, Sweden, pp. 550-559.
Real E., Liang C., So D.R. & Le Q.V. (2020). AutoML-Zero: Evolving Machine Learning Algorithms From Scratch. Preprint arXiv.org, 23 p, URL: https://arxiv.org/abs/2003.03384
Gottapu R.D. & Dagli C.H. (2020). Efficient Architecture Search for Deep Neural Networks. Procedia Computer Science, Vol. 168 (2020), pp. 19-25,
DOI: https://doi.org/10.1016/j.procs.2020.02.246
Lopes V. & Fazendeiro P. (2020). A Hybrid Method for Training Convolutional Neural Networks. Preprint arXiv.org, 6 p,
URL: https://arxiv.org/abs/2005.04153
Lyu Z., Karns J., ElSaid A. & Desell T. (2020). Improving Neuroevolution using Island Extinction and Repopulation. Preprint arXiv.org, 11 p, URL: https://arxiv.org/abs/2005.07376
Gülcü Y. & Kuş Z. (2020). Hyper-Parameter Selection in Convolutional Neural Networks Using Microcanonical Optimization Algorithm. IEEE Access, Vol. 8, pp. 52528-52540. DOI: https://doi.org/10.1109/ACCESS.2020.2981141
Doncieux S., Paolo G., Laflaquière A. & Coninx A. (2020). Novelty Search makes Evolvability Inevitable. Preprint arXiv.org, 9 p,
URL: https://arxiv.org/abs/2005.06224
Alet F., Schneider M.F., Lozano-Pérez T. & Kaelbling L.P. (2020). Meta-learning curiosity algorithms. Preprint arXiv.org, 22 p., URL: https://arxiv.org/abs/2003.05325.
Huang G., Li Y., Pleiss G., Liu Z., Hopcroft J.E. & Weinberger K.Q. (2017). Snapshot Ensembles: Train 1, Get M for Free. International Conference on Learning Representations (ICLR 2017): Proceedings of the 5th international conference. (Toulon, 24-26 April 2017). Toulon, France, pp. 1-14.
URL : https://openreview.net/pdf?id=BJYwwY9ll
Kenneth O. Stanley, Risto Miikkulainen (2002). Evolving Neural Networks through Augmenting Topologies. Evolutionary Computation. Vol. 10(2), pp. 99-127. URL:http://nn.cs.utexas.edu/downloads/papers/stanley.ec02.pdf