Posted
Here is the link to my undergraduate dissertation, which I wrote from December 2023 to April 2024. This report is the culmination of seven months of research, programming and writing, and I am pretty proud of it. My endlessly patient and supportive supervisor was Fabio Pierazzi. If you are a computer science student at King’s, I strongly recommend him as a dissertation supervisor and a lecturer. For my report, I received an 80%, which is a First Class Honours mark in the United Kingdom.
To sum up my project, I created a new method for building machine learning models to be specifically used for detecting attacks on a computer network. The main area where my method differed from other previous designs is that I focused heavily on improving explainability – how easy it is to figure out why a machine learning model classified a particular flow of network traffic as either malicious or benign – and the machine learning model’s ability to detect network attacks which it had not seen before. These two concentrations, explainability and detection of novel attacks, were chosen because they are both extremely necessary for the commercial viability of any network intrusion detection model that relies on machine learning. If an intrusion detection system can’t adapt to hackers innovating new attacks, it will quickly become useless; if an operator can’t figure out why their tool classified traffic as malicious or benign, how can they trust it?
I then implemented my new design in Python to determine its viability. On the explainability front, I succeeded, but my model failed to detect reconnaissance attacks (specifically port scanning and FTP/SSH brute forcing attacks) when it had not seen them previously. I believe that my model failed to detect novel attacks because my classification algorithm was too simple, reconnaissance attacks present extremely similarly to regular network traffic, and my dataset wasn’t high enough quality (there’s a lack of quality datasets that can be used to train machine learning models to detect network attacks).
The main third-party tools that I used for my implementation were the Apache Pyspark API, which increased the efficiency of training and testing my machine learning models by a significant degree, and the SHAP library, which uses game theory (Shapley values) to explain how much different attributes of a network flow contribute to my machine learning model classifying the flow as an attack or as normal traffic. The two datasets that I used for training and testing my model were the CICIDS2017 dataset by Sharafaldin, et al. and its 'improved' counterpart by Engelen, et al., described in their paper "Troubleshooting an intrusion detection dataset: the CICIDS2017 case study". I think that given a more complex classification algorithm and more real-world data, my methodology could still be successful as a basis for building high-performance, explainable network intrusion detection systems.
If you have any questions, comments or feedback, I am eager to hear it all! Email me at ccarrieasmith at gmail dot com :)