Skip to Main content Skip to Navigation
Preprints, Working Papers, ...

Efficient Extraction of Malware Signatures Through System Calls and Symbolic Execution: An Experience Report

Abstract : The ramping up use of network connected devices is providing hackers more incentives and opportunities to design and spread new security threats. Usually, malware analysts employ a mix of automated tools and human expertise to study the behavior of suspicious binaries and design suitable countermeasures. The analysis techniques adopted by automated tools include symbolic execution. Symbolic execution envisages the exploration of all the possible execution paths of the binary without neither concretizing the values of the variables nor dynamically executing the code (i.e., the binary is analyzed statically). Instead, all the values are represented symbolically. Progressing in the code exploration, constraints on symbolic variables are built and system calls tracked. A satisfiability-modulo-theory (SMT) checker is in charge of verifying the satisfiability of the collected symbolic constraints and thus the validity of an execution path. Unfortunately, while widely considered promising, this approach suffers from high resource consumption. Therefore, optimizing the constraint solver and tuning the features controlling symbolic execution is of fundamental importance to effectively adopting the technique. In this article, we identify the metrics characterizing the quality of binary signatures expressed as system call dependency graphs extracted from a malware database. Then, we pinpoint some optimizations allowing to extract better binary signatures and thus to outperform the vanilla version of symbolic analysis tools in terms of malware classification and exploitation of the available resources. CCS CONCEPTS • Security and privacy → Malware and its mitigation; • General and reference → Empirical studies; Evaluation; • Software and its engineering → Constraint and logic languages;
Complete list of metadata

Cited literature [59 references]  Display  Hide  Download
Contributor : Stefano Sebastio <>
Submitted on : Friday, December 14, 2018 - 2:38:11 PM
Last modification on : Friday, October 23, 2020 - 4:38:04 PM
Long-term archiving on: : Friday, March 15, 2019 - 12:51:11 PM


Files produced by the author(s)



Eduard Baranov, Fabrizio Biondi, Olivier Decourbe, Thomas Given-Wilson, Axel Legay, et al.. Efficient Extraction of Malware Signatures Through System Calls and Symbolic Execution: An Experience Report. 2018. ⟨hal-01954483⟩



Record views


Files downloads