Machine Learning in Cyber Security

The cyber threat landscape today forces organizations to constantly track and correlate millions of external and internal data points across a number of endpoints.  It simply is not feasible to manage this volume of information on an ongoing basis with a team of people.

Machine learning shines here because it can recognize patterns and predict threats in massive data sets, all at machine speed. By automating the analysis, cyber teams can rapidly detect threats and isolate situations that need deeper human analysis.  Machine learning techniques can better protect organizations in a number of ways:

1. Detecting surreptitious attackers on networks:   Machine learning can detect behavioral anomalies to find attackers on the inside or logged in with stolen credentials.
2. Predicting “bad neighborhoods” online: By learning from internet activity patterns, machine learning can automatically identify attacker infrastructure being staged to launch the next threat.
3. Attack detection through novelty and outliers:  Machine Learning finds attack patterns humans cannot readily detect, like a new peer relationship on the network with hosts communicating that can’t or shouldn’t be.  
4. Find suspicious cloud user behavior: Analytical techniques uncover suspicious user behavior indicative of cloud account compromise to extract data or perform malicious operations.
5. Modern malware detection:  Machine learning is valuable in detecting polymorphic malware, breaking down threat attributes to better stop new and reengineered polymorphic threats.

Beware the pitfalls

While machine learning offers tremendous promise for cyber security, it has its share of shortcomings that need to be acknowledged in order to use it appropriately.

1. Dealing with bad recommendations: If an application using machine learning suggests an incorrect movie recommendation, it is typically ignored. However, if machine learning incorrectly misses a threat or falsely convicts a good file, that could potentially interrupt business operations.   Machine learning attempts to account for the real-world cost of mistakes, but it shows how challenging security is for machine learning, so it neither misses threats nor blocks legitimate business.
2. Accounting for change:  How can machine learning account for changes occurring in the world around it? For example, if it operates in an environment in which two countries are foes, how can it account for a peace treaty struck between the former adversaries? This makes periodic retraining vital so it remains accurate as the world evolves.
3. Dealing with “explainability”:  When machine learning detects something bad, it often explains itself with math logic, instead of relevant security context. For example, say a machine learning system detected an infected device in the finance office.  Prior to potentially yanking the CFO off the network, a security practitioner must confirm the relevant security event details of the infection – how the computer was infected, if there is a vulnerable application on the laptop, what file turned malicious etc. to better understand how to respond.  Math logic won’t help here, but rather the related security event information we note that machine learning doesn’t always share. This “explainability” problem is a real challenge.

Making machine learning work for your organization

Machine learning is not a panacea for increasing cyber resilience. Rather, it is a helpful, additional security layer to augment other techniques in place.  Rather than being used in isolation, it needs to be combined with other cyber security techniques from intrusion prevention rules and anti-virus signatures, to whitelists, to sandboxing to behavioral techniques. Specific to machine learning, no one single technique or method will suffice, rather we must call on a pipeline of hundreds of algorithms working together for successful outcomes.

Second, no security approach is effective without a team of humans carrying out threat intelligence research, confirming all is working as it should and addressing changes in context (remember that peace treaty?)  

Last but not least, machine learning has many technical measures of success, but not all are helpful for a security professional. For machine learning to be most successful and embraced wholeheartedly, it must generate understandable outputs and generally “show its work” with security context.