Track 4: Scalable Detection infrastructure - Knowledge base for the Linux kernel

An operating system kernel is typically the most trusted software component of a computer system because it runs with the highest privileges. A kernel vulnerability therefore poses a dangerous threat to system security. For example, a buffer overflow exploit can be used to tamper with kernel functionality to launch various attacks such as escalating the privilege of malicious programs, opening backdoors, stealing information and disabling OS-level defenses. These problems are compounded by kernel extensions such as device drivers that run with the same privileges as the core kernel of the operating system but are generally third-party, untrusted or possibly malicious code. Furthermore, an kernel-level vulnerability or a kernel extension can be used to install kernel rootkit software. Such a rootkit poses a serious risk because it is specifically designed to hide its presence, making it hard to detect. For example, a rootkit may subvert the software that aims to detect it.

Today, security technologies such as firewalls and virus and worm scanners primarily focus on detecting attacks on application code. Typically, they use a blacklist of malware signatures found in the binary of the application code to detect known attacks. This approach will not work well for attacks on the operating system kernel because kernel rootkits can be very sophisticated. Furthermore, they are hard to characterize because they function similar to benign kernel code. Modern intrusion detections systems can also use anomaly detection methods, but these methods are typically limited to the system-call (user-kernel) interface. Unfortunately, kernel rootkits can execute entirely in the kernel context without issuing any system calls, and thus any system-call based monitoring methods are insufficient for detecting these rootkits.

The objective of this thread is to build a security infrastructure and a knowledge base for detecting attacks on the operating system kernel. The security infrastructure will ensure that our security mechanisms cannot be easily evaded and that kernel extensions invoke core kernel functionality at well-defined extension entry points, similar to applications invoking kernel code at well-defined system-call entry points. The knowledge base will be used to detect 1) kernel integrity violations indicative of attacks, and 2) malicious code posing as a benign extension.


Team members

Peter Goodman University of Toronto PhD Student