Analyzing a binary program is generally viewed as a difficult task. This is
because much information has been lost during compilation, especially when
optimization techniques are applied. The bar of analyzing binaries is so
high that many people believe it is safe to put secrets in binary programs,
as few people can analyze the binaries and obtain the secrets. Driven by
this misconception, programmers either put secret data in source code,
believing no one will be able to read them, or invest little in the security
of their programs, believing no one can analyze the binary and discover
vulnerabilities. Nonetheless, in the past decade, the technical progress in
the field of binary analysis has been continuously lowering the bar.
Nowadays, attackers can understand binary programs, recover secrets, and
discover vulnerabilities faster than ever. The obscurity of binary programs,
if it ever existed, is fading away, and thus yields a great impact in
software security of both newly developed and legacy binary programs.
In this talk, I will present some approaches to facilitate the reverse
engineering of binary programs. First, we observe that dynamic analysis,
especially dynamic binary instrumentation (DBI), is a powerful technique for
monitoring behaviors of a binary program while treating it as a gray box or
a black box. As an application, I will show that many software-based DRM
solutions can be attacked at a low cost, rendering them less useful than
what is commonly expected. Second, although static analysis techniques are
usually seen as inaccurate, I will demonstrate that combining them with
carefully selected domain-specific heuristics and more expensive analysis,
like symbolic execution, yields much more accurate output. On this aspect, I
will present our work of binary reassembling using static analysis, which is
a vital improvement for binary patching, enhancing, and binary code reuse.
Finally, I will present my on-going work of integrating static analysis and
symbolic execution to greatly improve its scalability in vulnerability
excavation. Our research has advanced or has the potential to advance the
state-of-the-art of binary analysis, showing that it is feasible to solve
many tasks that were once believed to be extremely difficult.