Architecture of a Prototype System for Network Traffic Anomaly Detection Based on Machine Learning and Visual Analytics

The paper presents the architecture of a prototype system for detecting network traffic anomalies. The system is based on a three-tier architecture using the Flask web framework to create a RESTful API. Anomaly detection is implemented using the Isolation Forest unsupervised machine learning algorithm (100 estimators, contamination factor 0.05) from the scikit-learn library, which processes data pre-normalized using StandardScaler in one-hour windows. The analysis results, including a multi-level classification of anomaly severity (with norma­ lized scores in the range of 0–1, where values greater than 0.8 correspond to the critical level) and ensuring compatibility with SIEM systems, are interactively visualized using Chart.js. Key theoretical and practical challenges, such as data quality, feature selection, scalability (algorithmic complexity O(n log n)), parameter optimization, and interpretability of results, are discussed.

1. Sommer R., Paxson V. (2010) Outside the Closed World: On Using Machine Learning for Network Intrusion Detection. IEEE Symposium on Security and Privacy. 305–316.

2. GarcΊa-Teodoro P., DΊaz-Verdejo J., Maciá-Fernández G., Vázquez E. (2009) Anomaly-Based Network Intrusion Detection: Techniques, Systems and Challenges. Computers & Security. 28 (1–2), 18–28.

3. Buczak A. L., Guven E. (2016) A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection. IEEE Communications Surveys & Tutorials. 18 (2), 1153–1176.

4. Bhuyan M. H., Bhattacharyya D. K., Kalita J. K. (2014) Network Anomaly Detection: Methods, Systems and Tools. IEEE Communications Surveys & Tutorials. 16 (1), 303–336.

5. D’Amico A., Whitley K. (2008) The Real Work of Computer Network Defense Analysts. VizSEC 2007 (Mathe­ matics and Visualization). 19–37.

6. Ring M., Wunderlich S., Grüdl D., Landes D., Hotho A. (2017) Flow-Based Benchmark Data Sets for Intrusion Detection. Proc. of the 16th European Conf. on Cyber Warfare and Security. 361–369.

7. Sharafaldin I., Lashkari A. H., Ghorbani A. A. (2018) Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization. Proc. of the 4th International Conf. on Information Systems Security and Privacy. 108–116.

8. Cordero C. G., Vasilomanolakis E., Milanov N., Koch C., Hausheer D., Mühlhäuser M. (2015) An Overview of the Botnet Simulation Framework. IEEE Conf. on Communications and Network Security (CNS). 739–740.

9. Liu F. T., Ting K. M., Zhou Z. H. (2012) Isolation-Based Anomaly Detection. ACM Transactions on Know­ ledge Discovery from Data. 6 (1), 1–39.

10. Ding Z., Fei M. (2013) An Anomaly Detection Approach Based on Isolation Forest Algorithm for Streaming Data Using Sliding Window. IFAC Proceedings Volumes. 46 (20), 12–17.

11. Apruzzese G., Colajanni M., Ferretti L., Guido A., Marchetti M. (2018) On the Effectiveness of Machine and Deep Learning for Cyber Security. 2018 10th International Conference on Cyber Conflict (CyCon). 371–390. IEEE.

12. Wang H., Bah M. J., Hammad M. (2019) Progress in Outlier Detection Techniques: A Survey. IEEE Access. 7, 107964–108000.

13. Ribeiro M. T., Singh S., Guestrin C. (2016) “Why Should I Trust You?”: Explaining the Predictions of Any Classifier. Proc. of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mi­ ning. 1135–1144.

14. Perdisci R., Ariu D., Fogla P., Giacinto G., Lee W. (2009) McPAD: A Multiple Classifier System for Accurate Payload-Based Anomaly Detection. Computer Networks. 53 (6), 864–881.

15. Staheli D., Yu T., Crouser R. J., Damodaran S., Nam K., O’Gwynn D., et al. (2014) Visualization Evaluation for Cyber Security: Trends and Future Directions. VizSec ‘14: Proceedings of the Eleventh Workshop on Visua­ lization for Cyber Security. 49–56.

16. Perdisci R., Lee W., Feamster N. (2010) Behavioral Clustering of HTTP-Based Malware and Signature Generation Using Malicious Network Traces. NSDI’10: Proceedings of the 7th USENIX Conference on Networked Systems Design and Implementation. 12.