A botnet is a network of compromised hosts that fulfills the malicious intents of an attacker. Once installed, a bot is typically used to steal sensitive information, send SPAM, perform DDoS attacks, and other illegal activities. Research in botnet detection has been quite prolific in the past years, producing detection mechanisms that focus on specific command and control structures, or on the correlation between the activities of the bots and the communication patterns shared by multiple infected machines.
We present an approach that aims to detect bot-infected hosts. Our approach (I) is independent on the underlying botnet structure, (II) is able to detect individually infected hosts, (III) deals with encrypted communication, (IV) does not rely on the presence of noisy malicious activities and can thus detect legitimate-resembling communication patterns, and (V) has a low false positive rate.
Our technique starts by monitoring a network trace produced by a bot sample B, which is summarized into a set of network flows. Similar flows are then grouped together by relying on a hierarchical clustering algorithm. The resulting clusters are analyzed for evidence of periodic behaviors. If no periodic behaviors are found, an output-based system selects those clusters that recur the most across different network traces obtained by running the sample B multiple times. Finally, our analysis automatically produces a network behavior model of B, which is deployed on a Bro NIDS sensor, that operates on real-time and realistic settings, raising few false positives.