DynamoRIO and similar dynamic binary instrumentation (DBI) systems are used for program analysis, profiling, and comprehensive manipulation of binary applications. These DBI tools are critical for malware analysis, program feature collections, and virtual machine binary translations. An important aspect of these DBI tools is the transparent feature, i.e. the binary application (such as malware) being analyzed is not modified and is not aware of the runtime code manipulation.
This presentation shows techniques that break the transparency feature of popular DBI tools (such as DynamoRIO and PIN). We will provide code that presents different behaviors when running on native hosts vs. running with DBI and vs. running on VM. The detection is based on specially crafted X86 instruction sequences that expose the fundamental limitation of binary instrument and translation. In this talk, we will also present position independent NOP sequences that can be used to help evade detections and differentiate different types of X86 decoders.
Kang Li is an Associate Professor at the University of Georgia. He graduated with his PhD from Oregon Graduate Institute. Before he joined the University of Georgia, he was a research scientist at Georgia Tech. His research interests are in the areas of computer security and operating systems.
Xiaoning Li is a security researcher for a Fortune 50 company. For the past 10 years, his work has been focusing on vulnerability research, new exploit development, malware analysis, and reverse engineering.