DGA domain detection and botnet prevention using Q-learning for POMDP
https://doi.org/10.35596/1729-7648-2021-19-2-91-99
Abstract
An effective method for preventing the operation of computer network nodes for organizing a botnet is proposed. A botnet is a collection of devices connected via the Internet for the purpose of organizing DDoS attacks, stealing data, sending spam and other malicious actions. The described method implies the detection of generated domain names in DNS queries using a neural network with parallel organization of convolutional and bidirectional recurrent layers. The effectiveness of the method is based on the assumption that generated domain names are used to create a botnet for merging. Experiments confirm that the proposed neural network is superior to the accuracy of existing counterparts on the UMUDGA dataset. The estimation of the quality of recognition of generated domain names using ROC analysis is calculated for a trained neural network. The article also formulates a model for controlling detectors using a partially observable Markov decisionmaking process to block infected nodes of a computer network. The search for the optimal policy for the formulated model by means of Q-learning of value agents is proposed. A comparative analysis of the average, minimum and maximum value of actions taken by agents in the process of interacting with the environment is carried out.
About the Authors
Y. V. Bubnov
Belarusian State University of Informatics and Radioelectronics
Belarus
Belarus
Bubnov Yakov Vasil’evich - M.Sc., Postgraduate student at the Electronic Computing Machines Department
220013, Minsk, P. Brovka str., 6
tel. +375-29-757-28-23
N. N. Ivanov
Belarusian State University of Informatics and Radioelectronics
Belarus
Belarus
PhD, Associate Professor, Associate Professor at the Electronic Computing Machines Department
References
1. Kolias C., Kambourakis G., Stavrou A., Voas J. DDoS in the IoT: Mirai and Other Botnets. Computer. 2017;50:80-84.
2. Patsakis C., Casino F., Katos V. Encrypted and covert DNS queries for botnets: Challenges and countermeasures. Computers & Security. 2020;88:101614.
3. Watson D., Zalmout N., Habash N. Utilizing Character and Word Embeddings for Text Normalization with Sequence-to-Sequence Models. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels, Belgium: Association for Computational Linguistics; 2018: 837-843 DOI:10.18653/v1/D18-1097.
4. Basaldella M., Antolli E., Serra G., Tasso C. Bidirectional LSTM Recurrent Neural Network for Keyphrase Extraction. Digital Libraries and Multimedia Archives (eds. Serra, G. & Tasso, C.) Cham: Springer International Publishing; 2018: 180-187. DOI:10.1007/978-3-319-73165-0_18.
5. Mnih V., Kavukcuoglu K., Silver D., Rusu A.A., Veness J., Bellemare M.G., Graves A., Riedmiller M., Fidjeland A.K., Ostrovski G., Petersen S., Beattie C., Sadik A., Antonoglou I., King H., Kumaran D., Wierstra D., Legg S., Hassabis D. Human-level control through deep reinforcement learning. Nature. 2015;518:529-533.
6. Wang Z., Schaul T., Hessel M., Van Hasselt H., Lanctot M., De Freitas N. Dueling network architectures for deep reinforcement learning. Proceedings of the 33rd International Conference on International Conference on Machine Learning. Vol. 48. New York, NY, USA: JMLR.org; 2016: 1995-2003.
7. Lillicrap T.P., Hunt J.J., Pritzel A., Heess N., Erez T., Tassa Y., Silver D., Wierstra D. Continuous control with deep reinforcement learning. Proceedings of the 4th International Conference on Learning Representations. San Juan, Puerto Rico; 2016.
8. Zago M., Gil Pérez M., Martínez Pérez G. UMUDGA: A dataset for profiling DGA-based botnet. Computers & Security. 2020;92:101719.
9. Woodbridge J., Anderson H.S., Ahuja A., Grant D. Predicting Domain Generation Algorithms with Long Short-Term Memory Networks. Computing Research Repository. 2016; 1-13 arXiv:1611.00791.
10. Vosoughi S., Vijayaraghavan P., Roy D. Tweet2Vec: Learning Tweet Embeddings Using Character-level CNN-LSTM Encoder-Decoder. Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval. New York, NY, USA: Association for Computing Machinery; 2016: 1041-1044. DOI:10.1145/2911451.2914762.
11. Highnam K., Puzio D., Luo S., Jennings N.R. Real-Time Detection of Dictionary DGA Network Traffic using Deep Learning. Computing Research Repository. 2020; 1-12 arXiv:2003.12805.
Review
For citations:
Bubnov Y.V., Ivanov N.N. DGA domain detection and botnet prevention using Q-learning for POMDP. Doklady BGUIR. 2021;19(2):91-99. (In Russ.) https://doi.org/10.35596/1729-7648-2021-19-2-91-99
Views: 663
Email this article 