Hardware/Network
From Trapping to Hunting: Intelligently Analyzing Anomalies to Detect Network Compromises
Reported by lastline.com
By Dr. Giovanni Vigna, Co-Founder and CTO at Lastline ,
Breach Detection Systems (BDSs) identify patterns of events in order to determine if a network has been compromised. Event streams include network activity (e.g., DNS activity and HTTP requests), host activity (e.g, user logins and the invocation of programs), and the analysis of various artifacts that are observed in the network (e.g., the reports produced by sandboxing technology that characterize the behavior of programs and documents).
One of the goals of BDSs is to provide the most effective automated detection with minimal false positives, because excessive false positives cause “fatigue” in the incident responder. This means that the sensitivity threshold of a BDS system must be set so that an alert is generated only when a substantial amount of supporting evidence is gathered.
For activity that falls below the established threshold, the information gathered by a BDS system can be used to detect attacks that cannot be automatically identified with a high degree of confidence. These attacks, instead of having a clear pattern of malicious behavior, are identifiable because their actions are anomalous when compared to normal network traffic.
Anomaly detection works under the assumption that malicious activity will result in anomalies in some event stream, and, at the same time, anomalies in an event stream are caused by malicious activity. Unfortunately, in the real world, both assumptions are sometimes incorrect, and anomaly detection has been riddled by both false negatives (because malicious activity does not generate anomalies) and false positives (because benign activity generates anomalies).
Even though pure anomaly detection might not work in the general case, it can provide hints for where an analyst might look more deeply to make connections between seemingly unrelated events. This is a new approach that instead of providing machine-based, automated detection, supports human-centered analysis of interesting events. In a nutshell, the analyst moves from being a trapper to being a hunter .
A hunting system provides a series of observations that are either anomalous per se (according to a pre-established model), or anomalous when put in the context of the historical behavior of a network. The following examples illustrate the observations produced by a hunting system:
The important aspect of these examples is that none of the resulting observations is necessarily the result of a compromise. A user might have been tasked to perform a remote backup with a cloud-based service, resulting in a large amount of data being transferred to an outside host, could be under pressure for a deadline and working off-hours, or might start using new tools to improve productivity.
However, a hunting system would be able to provide a human analyst with the ability to analyze, sort, connect, correlate, and expand these observations. A human analyst might be able to recognize that the sudden spike in upload traffic, correlated with an unusual session time for a user in a department that is notorious for not working late hours is enough to warrant an in-depth investigation. Or, the appearance of chains of remote desktop connections (from host A to host B to host C), a pattern that has never been observed before, together with an anomalous number of failed accesses to a shared file system might be worth the attention of the hunter, as it could be evidence that an intruder has gained access to the system and is poking around, trying to get a larger foothold in the network.
Fundamentally, the hunting tool does five things:
It is clear that while the “Collects” and “Presents” functionality is less difficult to design, the “Models” and “Reports” components are the ones that require the development of novel approaches in order to produce relevant observations that contain sufficient explanatory power (just observing that something is weird is often not enough: the system must explain why something is weird).
Also, the target user of this tool is a sophisticated user who is able to use his own domain knowledge about the network being protected in order to go beyond just passively absorbing the output of BDS systems, into the realm of investigation. Many enterprises lack the resources to dedicate to this task, and, therefore, Managed Security Service Providers (MSSPs) could provide “hunting services” to their clients. Opening a new approach to managed security that is not only reactive, but also proactive.
Author BIO
Dr. Giovanni Vigna has been researching and developing security technology for more than 20 years, working on malware analysis, web security, vulnerability assessment, and intrusion detection. He is a Professor in the Department of Computer Science and the director of the Center for CyberSecurity at the University of California in Santa Barbara, and is co-founder and CTO at Lastline.