Information-based Dependency Matching for Behavioral Malware Analysis

Malicious software (malware) has been a constant threat to computer environments. Every year malware inflict staggering amount of damage and incur vast financial losses worldwide. Malware has changed drastically and its purpose, attack vectors and methods are no longer simple. Furthermore the attackers often utilize unknown vulnerability, evasion techniques and generator algorithms which drastically increase the impact, effectiveness and quantity of malware. Thus the task falls to security experts to develop tools and techniques to thwart this ever expanding threat. The challenge is to detect all attacks, regardless of evasion techniques, while keeping false alarms to a minimum. This thesis seeks to analyze the application of function call-based malware detection. More specifically function calls with their inter-dependencies, extracted by use of information-based dependency matching. Analysis will be performed to research whether this method is reliable and improve obfuscation resilience. The thesis discusses the difference of per­forming detection at library call, system call or function call(hybrid) layer, and how well detection can be performed at these layers.