
What are the Key Machine Learning Challenges for Threat Detection?
ML algorithms learn about their environments and create baseline norms before searching for anomalous events that can suggest a compromise.
By
CIO Applications Europe | Thursday, March 18, 2021

ML algorithms learn about their environments and create baseline norms before searching for anomalous events that can suggest a compromise.
Fremont, CA: The advancement of machine learning and its ability to provide deep insights based on big data remains a hot topic. Many C-level executives are launching deliberate ML programs to see if their businesses will benefit from them, and cybersecurity is no exception. Most information security vendors have implemented some sort of machine learning, but it is clear that it is not the panacea that some have made it out to be.
Stay ahead of the industry with exclusive feature stories on the top companies, expert insights and the latest news delivered straight to your inbox. Subscribe today.
Although machine learning solutions for cybersecurity can and will provide a substantial return on investment, they are not without challenges. To realize ML's full potential, organizations should be mindful of a few potential losses and set reasonable targets.
False Positives and Alert Fatigue
The most common criticism leveled at ML-detection software is that it produces a "impossible" number of alerts every day – consider millions of alerts every day, effectively performing a denial-of-service assault against analysts. This is especially true for "static analysis" methods, which depend heavily on how threats appear.
Even a 97 percent accurate ML-based detection approach can be ineffective because simply put, the math is not in favor.
Assume an enterprise has one threat among 10,000 users on its network. One can determine whether a warning is actually a positive attack using Bayes' law by multiplying 0.97 (for 97 percent accuracy) by the probability of an actual threat among all users, or 1/10,000. This means that even though an alarm is 97 percent accurate, the actual probability of a real attack is 0.0097 percent!
Since improving beyond 97 percent may be impossible, the best solution is to restrict the population under evaluation by whitelisting or prior filtering with domain expertise. This could imply concentrating on highly credentialed, privileged users or a particular critical business unit.
Dynamic environments
ML algorithms learn about their environments and create baseline norms before searching for anomalous events that can suggest a compromise. Furthermore, if the IT enterprise is continuously reinventing itself to meet business agility requirements and the dynamic environment does not have a steady baseline, the algorithm cannot efficiently determine what is normal as well as will issue alerts on completely benign events.
See also: Top Cyber Security Solution Companies
More in News
Weekly Brief
I agree We use cookies on this website to enhance your user experience. By clicking any link on this page you are giving your consent for us to set cookies. More info
Be first to read the latest tech news, Industry Leader's Insights, and CIO interviews of medium and large enterprises exclusively from CIO Applications Europe
THANK YOU FOR SUBSCRIBING


