Soma Halder - Hands-on Machine Learning for Cyber Security

Chapter 1, Basics of Machine Learning in Cyber Security

The key focus of the chapter is allow readers to get familiarized with machine learning, how is it practiced and its need in Cyber Security domain. This chapter builds on using machine learning instead of conventional rule based engines while allowing the readers to tackle challenges in the cyber security domain. Further, this chapter allow readers to get hands-on knowledge on python, MongoDB and various libraries for machine learning and concepts related to supervised and unsupervised learning.

Chapter 2, Time series Analysis and Ensemble Modeling

The first phase of the threat lifecycle deals with Reconnaissance where malwares/APTS passively engages the target by searching through public information, penetrating confidential corporate documents and so on. Time series Analysis helps detect packets exchanged at odd hours or identify spikes seen during holidays or odd hours in the corporate network.Given a sequence numbers for time series dataset we can restructure the data model to look like a supervised learning by using the values at previous point in time to predict the value at next point. Time
Similarly Ensembles methods are techniques that give boost in accuracy on predictions made by our model. Meta learning from the ensemble algorithm helps identify the resources which are getting exploited to continue the reconnaissance activity.

In this chapter we will implement two examples on ensemble modeling and time series forecasting each.

Chapter 3, Segregating Legitimate and Lousyc

In the threat kill chain the initial attack often starts with URL Injection in emails, attachments etcetera. Detecting bad urls in the initial stages of the attack help security professionals to combat them early on. Thus, Having learnt the base concepts of machine learning this chapter will focus on building practical skills and will discuss examples of machine learning Implementation on URLs, identifying the good, bad and the worst of URLs through an intelligent machine learning based python driven example application. Additionally, this chapter will provide hands-on knowledge to the concepts learnt in the first chapter as well as testing the accuracy of result generated by our program.

Chapter 4, Knocking Down Captchas

In the previous chapter, we learnt about defensive mechanisms like URL analysis and building intelligent programs that can detect anomalies in the web request/response. In this chapter, we will learn how attackers can effectively bypass defensive technologies using machine learning. This chapter will discuss bypassing Captcha mechanism through machine learning and will build on hands-on knowledge of image processing, voice and text processing in order to bypass Captcha. At the end of this chapter, the readers will have hands-on knowledge on defeating protection mechanisms while learning how they can build effective countermeasures.

Chapter 5, Using Data Science to Catch Email Frauds and Spams

In the last three chapters, we have covered the basics of ML while putting it use in cyber security. This chapter will focus on real world scenarios and case studies. A steep rise in the complexity of cyber attacks has made it very difficult for the blue teams to develop a proper countermeasure to fraudulent emails and spam. In this chapter, we will try to build a program that will solve the problem of detecting email frauds and spam using supervised learning with the Naive Bayes algorithm. The program also enables learning the insights of Email, its headers and distinct information for fraud and spam classification.

Chapter 6, Efficient Network Anomaly detection using K Means

Network Anomalies could be pointers to malwares connectivity with the command and control, the malwares lateral movement and exfiltration from with the network. Thus accurately detecting network anomalies can help the network administrator to prevent a cyber attack. We will cover alert and exploration based network security monitoring in this chapter. Making effective use of K Means algorithm, we will try developing a solution which will have the ability to detect anomalies and alert any malicious activity going on the network. However, searching for anomalous network flows is a challenging task due to the wide statistical variety of network behavior. The chapter will also allow readers to gain knowledge on Pythons SK-Learn and NumPy set of libraries and modules.

Chapter 7, Decision Tree and Context Based Malicious Event Detection

Making decisions are hard, whether it's life changing decisions or the decisions on types of events occurring in a system. This chapter develop readers skill on defining better decision making using decision tree. Decision trees are predictive models designed to go from observation to conclusion. We know that each event or incident in computer systems security consists of number of independent series of actions. However, using only machine learning and processing each individual action on a standalone basis, we can not detect if it was malicious or not. Therefore, we will learn about preserving context of the previous events that can help us to take decision about the credibility of actions in particular event.

Chapter 8, Catching Impersonators and Hackers Red Handed

Each user has their different digital fingerprint that includes how we interact with some application, our typing speed, way we move our hand on touch screen or how we scroll on a web page etc. Machine learning gives us the ability to transparently produce a digital fingerprint from users interaction with system and validating whether the user is impersonator or a legitimate user. The beauty of this technology is that behavioural fingerprints are highly detailed and virtually impossible to imitate. This chapter will include the methodology to implement such systems.

Chapter 9, Speeding Things up with GPU

The machine learning includes data intensive computation tasks as training the system or a neural net takes massive amount of computer processing that spans upto days or weeks depending upon its complexity. Using GPU, developers can build more sophisticated solutions which in turn leads to intelligent next generation systems.
Speed comparison of vector multiplication example with and without GPU

Chapter 10, Change the Game with TensoFlow

TensorFlow is open source software library by google that can make use of either CPU's or GPUs, or a mix of them. It allows us to express arbitrary computations as data flows or graphs, it can be used to design artificial neural network to detect patterns and correlations just like humans learns and reasons things.
TensorFlow provides high level python interface and efficiency of C in an easy to understand framework.
We will use TensorFlow to re implement some of the above examples to compare and contrast its results.

Chapter 11, Financial Frauds and How Deep Learning can Mitigate Them

Financial frauds like credit card fraud, mortgage or loan fraud are taking places everywhere and with the access to internet based financial services the rate is increasing tremendously. Fraud in this aspect not necessarily mean people being devious like huge credit card frauds on payment gateways, it also includes where somebody is lying bits about their previous salary or employment history. So how do we catch this kind of frauds ?Thats where deep learning jumps in, we can create models and train the system from previous fraudulent events so that they can be mitigated in future.