AIFORE: Smart Fuzzing Based on Automatic Input Format Reverse Engineering

Ji Shi1,2,3,4 , Zhun Wang∗ 2 , Zhiyao Feng2,6 , Yang Lan2 , Shisong Qin2 , Wei You5 , Wei Zou1,4 ,
Mathias Payer6 , and Chao Zhang†2
1 {CAS-KLONAT‡, BKLONSPT§}, Institute of Information Engineering, Chinese Academy of Sciences
2 Institute for Network Science and Cyberspace & BNRist, Tsinghua University; Zhongguancun Lab
3 Singular Security Lab, Huawei Technologies
4 School of Cyber Security, University of Chinese Academy of Sciences
5 Renmin University of China
6 EPFL

Abstract Knowledge of the input format is essential to generat-

ing high-quality inputs. The input format describes how the
Knowledge of a program’s input format is essential for ef-
program expects the input bytes to be organized. Ideally,
fective input generation in fuzzing. Automated input format
well-formatted inputs will be parsed and processed properly,
reverse engineering represents an attractive but challenging
achieving the desired results. Ill-formed inputs will be filtered
approach to learning the format. In this paper, we address
out by sanity checkers in the program and discarded early.
several challenges of automated input format reverse engi-
Therefore, the fuzzer can generate inputs following the for-
neering, and present a smart fuzzing solution AIFORE which
mat specification, which will help bypass some sanity checks
makes full use of the reversed format and benefits from it.
in shallow code, and finally, reach and test deeper and more
The structures and semantics of input fields are determined
complex code. Besides, because the sanity mechanisms in
by the basic blocks (BBs) that process them rather than the
programs are not always sound and complete, it is highly pos-
input specification. Therefore, we first utilize byte-level taint
sible that some inputs against the format specification are not
analysis to recognize the input bytes processed by each BB,
filtered out but passed to deeper code [1]. These inputs usually
then identify indivisible input fields that are always processed
have a greater chance of triggering unexpected behavior or
together with a minimum cluster algorithm, and learn their
bugs, and therefore should be favored by the fuzzer. Thus, the
types with a neural network model that characterizes the be-
fuzzer can produce valuable inputs for both exploration and
havior of BBs. Lastly, we design a new power scheduling
exploitation purposes with the guidance of format knowledge.
algorithm based on the inferred format knowledge to guide
There are abundant works on input format inference and
smart fuzzing. We implement a prototype of AIFORE and
format-guided smart fuzzing. In general, such solutions try to
evaluate both the accuracy of format inference and the per-
answer three core questions regarding the input format: (1)
formance of fuzzing against state-of-the-art (SOTA) format
where the boundaries of different input fields are, (2) which
reversing solutions and fuzzers. AIFORE significantly outper-
types these fields belong to, and (3) how to utilize the knowl-
forms SOTA baselines on the accuracy of field boundary and
edge of input format to guide fuzzing. However, existing
type recognition. With AIFORE, we uncovered 20 bugs in 15
solutions so far only address parts of these problems or have
programs that were missed by other fuzzers.
limitations.
Regarding the input field boundary recognition, existing
1 Introduction studies [2–4] mainly rely on statistical analysis or dynamic
taint analysis to group bytes processed by the same instruction
Fuzzing is an efficient technique for finding vulnerabilities into a unique field. Such solutions have several limitations.
in programs. Typically, fuzzing produces a large number of First, an instruction may process multiple input fields, e.g.,
inputs and feeds them to programs under test, detecting secu- in a loop, and thus these fields will be erroneously merged
rity violations. A smart and effective fuzzer relies on a high into one field. Further, a long field would be processed by
quality of inputs, which enables efficient exploration of the different instructions and be erroneously split into multiple
program state space and increases the chance of triggering fields, because one instruction, in general, cannot process data
vulnerabilities. beyond the machine word size (e.g., 4 bytes for a 32bit CPU).
∗ Zhun
Lastly, identifying fields through statistical analysis requires
Wang is the co-first author.
† Chao lots of different inputs, which are often not available.
Zhang is the corresponding author.
‡ Key Laboratory of Network Assessment Technology, CAS Regarding the input field type identification, existing solu-
§ Beijing Key Laboratory of Network Security and Protection Technology tions [5–7] in general rely on prior knowledge (e.g., parameter
types of some standard library calls like strcpy) to extract the present a two-step strategy to utilize the format knowledge,
type of fields being passed in. Nevertheless, such prior knowl- i.e., recognizing new variants and prioritizing infrequently
edge is scattered and requires intensive engineering efforts tested formats. First, we pick test cases that have significantly
to convert to heuristic rules and apply them, which will also different code coverage and re-analyze their input formats,
inevitably introduce false positives and false negatives. More- since they are likely to have different formats. Second, we
over, such solutions can generally only recognize program prioritize the seeds whose formats are less frequently tested
variable types (e.g., int, array, and string) rather than se- during fuzzing, and increase their mutation power. (§3.3)
mantic types (e.g., magic number, size, or checksum), since Results: We compare AIFORE prototype with SOTA
they do not model how the input fields affect the program’s format-aware fuzzing solutions and input reversing works,
behaviors from the semantic level. including ProFuzzer [8], TIFF-fuzzer1 [6], WEIZZ [9], Eco-
Regarding the utilization of input format, existing fuzzing Fuzz [13], AFL-Analyze [14], and AFLFast [15]. We conduct
solutions [2,8–10] in general use the input format to guide test experiments on 15 different types of formats and 15 well-
case generation or mutation to reach deeper code in the target tested real-world programs, including document readers, mul-
program. Besides, some solutions also use format knowledge timedia processors, packet parsers, compression tools, and
to determine if a seed is good or not. For example, AFLS- network protocol analyzers. The results show that AIFORE’s
mart [10] assigns more power (i.e., performing more muta- format reverse engineering module achieves high accuracy for
tions) to well-formed seeds, which can be parsed success- different seed sizes and different compiler optimization levels
fully by Peach [11] (a generation-based fuzzer with manually (§5.1). AIFORE’s average accuracy of field boundary recogni-
crafted input specification templates). However, it cannot dis- tion is 84.06%,2 compared to the corrected ground truth from
tinguish the quality of seeds if given a simple template without 010 Editor3 , while the accuracy of ProFuzzer, TIFF-fuzzer,
field relation constraints (Table 1). and AFL-Analyze are 36.27%, 63.14%, and 23.73%, respec-
Our approach: To address these challenges, we propose tively (§5.2). Regarding the field type identification, AIFORE
AIFORE which automatically reverses input formats, later correctly predicts the type with an accuracy of 84.26% in
using them to smartly guide fuzzing. The core insight is that, untrained formats and programs, higher than ProFuzzer’s
since input fields are interpreted by BBs, the structural and 56.60% and AFL-Analyze’s 36.76% (§5.2). At last, we apply
semantic information of inputs can be inferred from them, no the extracted format knowledge and the power scheduling
matter what the input format specification is like. Thus, we can algorithm to fuzzing. In the testing of 24 hours, AIFORE out-
learn the structures and semantics of input fields by analyzing performs other fuzzers in BB coverage (6% higher than Pro-
BBs that process them, and then utilize the knowledge to Fuzzer, 26% higher than WEIZZ) and finds 20 bugs missed
conduct smart fuzzing. by other fuzzers (§5.3).
First, AIFORE utilizes dynamic taint analysis to learn In summary, we make the following contributions:
which input bytes are processed by each BB. Note that, a • We propose a field boundary recognition method, which
single indivisible field may be partially processed by multiple utilizes taint analysis to identify relationships between
instructions in one BB, but is unlikely to get partially pro- input bytes and BBs, as well as a minimum cluster algo-
cessed by multiple BBs, since these BBs will not always get rithm to split indivisible input fields.
executed together but an indivisible field should get analyzed • We present a novel deep learning–based solution to pre-
together. As a result, we assume that BBs can be used as a dict the type of input fields processed by BBs.
basis to identify field boundaries. Given the taint analysis • We present a novel format-based power scheduling algo-
results, we split field boundaries by recognizing indivisible rithm to explore infrequent types of inputs.
fields via a minimum cluster algorithm. (§3.1) • We implement a novel smart fuzzing solution AIFORE
Second, AIFORE builds a deep learning model to com- and systematically evaluated it on a wide range of in-
prehend BBs, i.e., to predicate the type of input fields pro- put formats and programs. Results show that it has a
cessed by them. Note that, BBs process different types of much better performance on input format recognition
input fields (e.g., size, offset, enum, string, checksum, or and format-aware fuzzing than SOTA solutions.
magic number) differently. Therefore, we train a Convolu-
tional Neural Network (CNN) model [12] to learn the patterns 2 Motivational Example
of how BBs process different types of input fields and then
predict the semantic type of input fields. (§3.2) In this section, we take readelf as an example to illustrate
Third, we design a novel power scheduling algorithm based how it parses the ELF file format and summarize the limita-
on the format dynamically extracted during fuzzing to im- tions of existing approaches.
prove fuzzing efficiency. Note that, a program may still accept 1Weuse TIFF-fuzzer to distinguish it from the file format TIFF.
inputs that do not satisfy the specification [1], as well as inputs 2Wecalculate the arithmetic mean of the accuracy in Table 6.
of different format variants. Different variants of input formats 3 A popular commercial editor which can resolve different file formats.

have different impacts on program behaviors. Therefore, we Homepage: https://www.sweetscape.com/010editor/

 0 1 2 3 4 5 6 7 8 9 A B C D E F

0x00h: 7F 45
Magic Number
4C 46 01
Class
01
Data
01
Version
00
Osabi
00
Abiver.
00 00 00
Ei_pad
00 00 00 00
E_ident
Listing 1: Source code taken from readelf, which is respon-
0x10h: 02 00
E_type
03
E_machine
00 01 00
E_version
00 00 74 80 04
Address
08 34 00 00 00
Address of Program Header sible for parsing ELF format input.
0x20h: A4 00 00 00 00 00 00 00 34 00 20 00 02 00 28 00
Address of Section Header E_flags ELF Header Size Header Size Header Number Sec Header Size
1 # define BYTE_GET ( field ) byte_get ( field , sizeof ( field ))
0x30h: 04 00 03 00
Sec Header Num Sec Header Index 2 static bfd_boolean get_file_header ( Filedata * filedata
String Magic Enumeration Offset Size Checksum ){
3 ...
(a) ELF file header structure of 32bit. 4 filedata -> file_header . e_machine = BYTE_GET ( ehdr32 .
e_machine );
0 1 2 3 4 5 6 7 8 9 A B C D E F
5 filedata -> file_header . e_shnum = BYTE_GET ( ehdr32 .
0x00h: 7F 45 4C 46 02 01 01 00 00 00 00 00 00 00 00 00
Magic Number Class Data Version Osabi Abiver. Ei_pad E_ident e_shnum );
0x10h: 01 00 3E 00 01 00 00 00 00 00 00 00 00 00 00 00 6 ...
E_type E_machine E_version Address

0x20h: 00 00 00 00 00 00 00 00 58 D0 03 00 00 00 00 00 7 }
Address of Program Header Address of Section Header
00 00 00 00 40 00 00 00 00 00 40 00 3D 00 10 00
8 void init_dwarf_regnames_by_elf_machine_code ( unsigned
0x30h: E_flags ELF Header Size Header Size Header Number Sec Header Size Sec Header Num Sec Header Index int e_machine ){
String Magic Enumeration Offset Size Checksum 9 dwarf_regnames_lookup_func = NULL ;
10 switch ( e_machine ){
(b) ELF file header structure of 64bit. 11 case EM_386 :
Figure 1: ELF file header definition. 12 init_dwarf_regnames_i386 () ;
13 break ;
14 case EM_X86_64 :
An ELF file consists of several data structures (e.g., file 15 init_dwarf_regnames_x86_64 () ;
16 break ;
header, program header, and section header), each composed 17 ...
of several fields. For the file header structure as shown in 18 }
19 }
Figure 1, it has some fields consisting of consecutive bytes 20 static bfd_boolean process_file_header ( Filedata *
(e.g., magic number from offset 0x00 to 0x03) and some filedata ){
21 if ( header -> e_ident [ EI_MAG0 ] != ELFMAG0 ||...||
fields with single-byte values (e.g., class at offset 0x04). header -> e_ident [ EI_MAG3 ] != ELFMAG3 ){
Take the e_machine at offset 0x12 as an example, it indicates 22 return error ;
23 }
the machine architecture (e.g., AArch64, i386, or x86_64), 24 ...
which changes the structure of ELF (e.g., the length of address 25 init_dwarf_regnames_by_elf_machine_code ( filedata ->
file_header . e_machine )
fields can be four or eight bytes, marked with the red box). 26 ...
27 if ( header -> e_shstrndx != SHN_UNDEF && header ->
Listing 1 shows the code snippet of readelf, which e_shstrndx >= header -> e_shnum ){
parses inputs of ELF format. We can see that there are 28 printf (" corrupt ");
29 }
two main steps to parse the input file. The first step is to 30 ...
read the input and initialize certain variables correspond- 31 }
ing to the input fields. For example, line 3 to line 6 read
the fields from the input and initialize corresponding vari-
ables (e.g., file_header.e_machine). The next step is to Observation 2: The program code processing fields of
further process these initialized variables in the function different types shows different patterns. For example, an enu-
process_file_header. Different BBs are used to process meration variable (e.g., e_machine) is very likely to be pro-
different types of variables (i.e., input fields). For example, cessed by a switch-case statement, and a size variable (e.g.,
the four-byte magic number is compared one byte at a time e_shnum) may be processed by mathematical operations.
on line 21, while the e_machine is handled as an enumeration Observation 3: The structure of the input may differ, and
with a switch-case statement. the program will dispatch distinct code to parse the input. For
From this example, we observe the following: example, the variation of e_machine may indicate a different
Observation 1: In most cases, bytes in an indivisible field data structure, like the length of address fields marked with
are parsed together in one BB. For example, the two-byte the red box in Figure 1. If a new structure is found during
field e_machine is processed as a whole in the block starting fuzzing, the fuzzer should re-assign the power and use the
at line 4, and another block starting at line 10. For the field corresponding format knowledge to get more coverage.
e_shstrndx, it is parsed together in BBs at line 27. However, From these observations, we summarize that the existing so-
there are corner cases in which parts of one field (e.g., magic lutions have the following limitations. First, splitting fields at
number field parsed at line 21) can be parsed in different BBs. the instruction level is not generally appropriate. For example,
Developers’ code style and compiler optimizations would instructions in BYTE_GET may parse multiple fields and will
affect whether to parse them as one entity or separately. We cause different fields to be merged erroneously. Second, most
will illustrate this further in §5.1.1. existing solutions of field type identification rely on human-
Besides, one BB may process multiple fields and get ex- extracted code patterns , which are labor-intensive and cannot
ecuted multiple times at runtime. For example, BBs in the cover complicated cases. For instance, there could be multiple
BYTE_GET function are executed multiple times to read from BB patterns given one input field type. Third, existing format-
the input buffer and extract different fields to assign different aware fuzzing solutions have limitations when considering the
variables. power scheduling during fuzzing. For example, ProFuzzer [8]
 Fuzzing with Power Scheduling
(§3.3)
AFLSmart with well-prepared pit files, the distributions vary
for different formats and programs. For PNG, the average of
Seed Format-Aware Power
Valuable Seed
Analysis Mutation Reassign v(s) decreases to 35.23%, and 29.90% of them are no less
than 50%, which is the key threshold to judge the quality of a
Field Type Classification
(§3.2) seed file in AFLSmart. In AIFORE, we design a new power
Binary
Taint (insn, offset) Field Feature Field Type
Analysis Extraction
CNN scheduling algorithm to re-assign the energy to those formats
Format Template that are seldom fuzzed.
Field Boundary Analysis
Initial Seed
(§3.1)
BB Level
Merge
Minimum
Cluster Field Boundary
3 Design

Figure 2: Architecture of AIFORE. The architecture of AIFORE is shown in Figure 2. During

fuzzing, if the coverage increment a seed brings is distinguish-
able (i.e., increasing 3% of the average coverage), we then
only analyzes the seeds it considers valuable to extract format
mark it as a valuable seed. For each valuable seed, we use
knowledge and does not re-assign the fuzzing power to these
AIFORE to analyze its format and fuzz the file with a format
valuable seeds. The validity-based power scheduling highly
model including the extracted knowledge of field boundary
depends on Peach [11] and the constraints (e.g., checksum or
and field type. The core insight of AIFORE is that the in-
size) in the pit file. Peach is a fuzzing framework that can
put format knowledge can be inferred from patterns of BBs
generate program input with a given format model called a
processing the input file. Therefore, AIFORE utilizes taint
pit file. AFLSmart defines the degree of validity v(s) of a
analysis to construct a map between input bytes and the BBs
seed s based on how many input bytes can be parsed by Peach
processing them. Figure 3 demonstrates two BBs taken from
and assigns more power to the seeds with higher v(s). In the
readelf which parse the input format in the motivation ex-
extreme case, if there is no constraint in the pit file, Peach
ample. The annotation at the end of each instruction lists the
will parse the input successfully in most cases and assign a
offsets of input bytes processed by it, traced by the taint
100% degree of validity to it. To verify this, we choose seven
analysis engine.
formats and target programs to observe to which degree the
Given the knowledge of input bytes processed by each in-
validity changes during fuzzing in AFLSmart. We choose
struction, the following three modules jointly reverse the input
these targets since AFLSmart provides the corresponding pit
format and balance the fuzzing power to the seeds. The first
files, and they contain a few constraints, like checksum, size,
module analyzes the field boundary and splits the input into
and others.
fields, i.e., consecutive bytes which have the same semantics
We divide the experiments into two groups. For the first concerning the impact on program behaviors. The second
group, we launch AFLSmart with its original, provided pit module predicts the field type information via a CNN-based
files, containing the description of the constraints between model, which is trained to comprehend how a program parses
fields. For the second group, we feed AFLSmart with cus- different types of fields. The field boundary and field type
tomized pit files. These pit files only describe field bound- consist of the format template (i.e., a peach pit file). For each
aries and types, without complex constraints. We use AFLS- field, we record its boundary with its start position and size
mart to fuzz for 12 hours and collect thousands of seeds and its type. The last module will decide which seed is worth
for both groups. We collect the number of seeds considered format extraction and will re-assign more fuzzing power to
"valid" and assigned a high degree of validity by AFLSmart. those less mutated formats.
From the result (Table 1), we conclude that v(s) is almost
100% when we feed AFLSmart with pit files that lack con-
straints. Almost all the seeds have the same degree of validity, 3.1 Field Boundary Analysis
which makes the power scheduling fall back to AFL, despite Identifying the boundary of different fields is a fundamental
the format quality of the seeds varies. However, if we feed task for reverse engineering of input formats. Given Observa-
tion 1, a field consisting of consecutive input bytes is usually
Table 1: AFLSmart’s degree of validity v(s), average processed as a whole in each BB. We thus propose to split
v(s) / ratio of (v(s) ⩾ 50%) fields with a minimum cluster (MC) method from the block
Format Program1 pit w/ Constraints pit w/o Constraints level rather than the instruction level as Tupni [4] and Poly-
ELF readelf 0.15%/0.14% 100.00%/100.00% glot [3] did. The overall process is shown in Algorithm 1.
GIF gif2tiff 0.58%/0.59% 100.00%/100.00%
JPG jhead 0.06%/0.06% 100.00%/100.00% First, we split the binary code into BBs. As shown in Figure 3,
PCAP tcpdump 3.48%/3.73% 100.00%/100.00% we will get two BBs from line 4 in Listing 1. Second, we
PNG pngtest 35.23%/29.90% 100.00%/100.00%
WAV sfinfo 52.86%/54.27% 100.00%/100.00% collect and merge the input bytes processed in one BB. For
ZIP 7za 0.13%/0.13% 100.00%/100.00% instance, the first BB in Figure 3 processes two input bytes
1 Parameters are shown in Table 2 at offsets 18 and 19, and the second BB (BYTE_GET at line
 1 ; get_file_header loc_4069E0:
2 . text : 0x4405B8 mov edx , eax ; (18 , 19) movzx edi, word ptr[ole+22h] ; size
3 . text : 0x4405BA mov rax , [rbp + filedata]
4 . text : 0x4405BE mov [rax + 52h] , dx ; (18 , 19) xor eax, eax
5 . text : 0x4405C2 mov rax , cs : get_byte mov byte ptr [olest+3Dh], 1
6 . text : 0x4405C9 lea rdx , [rbp + ehdr64] lea rdx, [rdi-1] ; size-1
7 . text : 0x4405CD add rdx , 14h
8 . text : 0x4405D1 mov esi , 4 mov [olest+38h], edi
9 . text : 0x4405D6 mov rdi , rdx cmp rdx, 0FFFFFFh ; size-1
10 . text : 0x4405D9 call rax ; get_byte ja short cleanup

11 ; get_byte loc_4069FE: cleanup: ; ptr

12 .text : 0x46DCF2 mov rax , [rbp + field] call _malloc mov rdi, rax
13 .text : 0x46DCF6 movzx eax , byteptr[rax] ; (16 , 18 , 40 , 42 , 44 , 46 , 48 , 50)
14 .text : 0x46DCF9 movzx eax , al ; (16 , 18 , 40 , 42 , 44 , 46 , 48 , 50) call _free
15 .text : 0x46DCFC mov rdx , [rbp + field] mov rdi, olest ; ptr
16 .text : 0x46DD00 add rdx , 1
17 .text : 0x46DD04 movzx edx , byteptr[rdx] ; (17 , 19 , 41 , 43 , 45 , 47 , 49 , 51)
xor r12d, r12d
18 .text : 0x46DD07 movzx edx , dl ; (17 , 19 , 41 , 43 , 45 , 47 , 49 , 51) olest = rdi ; OLE2Stream_0 *
19 .text : 0x46DD0A shl edx , 8 ; (17 , 19 , 41 , 43 , 45 , 47 , 49 , 51) call _free
20 .text : 0x46DD0D or eax , edx ; (16 , 17 , 18 , 19 , 40 , 41 , 42 , 43 ,
44 , 45 , 46 , 47 , 48 , 49 , 50 , 51)
21 .text : 0x46DD0F mov eax , eax ; (16 , 17 , 18 , 19 , 40 , 41 , 42 , 43 ,
44 , 45 , 46 , 47 , 48 , 49 , 50 , 51)
22 .text : 0x46DD11 jmp locret_46DFC8
Figure 4: Forward slicing example.

Figure 3: Basic blocks to parse e_machine.

parse and use the fields of this type, we build a CNN model
Algorithm 1 Field Boundary Analysis to map the latter to the former. We, therefore, collect a large
Input: BinaryProgram, TaintTrace number of training data consisting of code snippets and the
Output: FieldBoundaries ▷ A list of field intervals without overlapping
1: BBs = E X T R A C T BB(BinaryProgram) types of fields processed by them. Then we train a CNN
2: BB_to_taint = Dict[BB, Set] ▷ Map a BB to a set of tainted offsets model to predicate the field type from a target code snippet.
3: foreach inst ∈ TaintTrace do ▷ Merge inst-level taint to BB-level
4: BB = A D D R T O BB(inst.addr, BBs)
5: taint_offset = E X T R A C T T A I N T O F F S E T F R O M T R A C E (inst) 3.2.1 Field Types
6: BB_to_taint[BB].udpate(taint_offset)
7: FieldBoundaries = []
8: foreach taint_offset ∈ BB_to_taint.values() do
We consider the semantic types (e.g., offset or checksum)
9: interval_lst = S P L I T T O C O N S E C U T I V E (taint_offset) of fields rather than their program variable type [18, 19] (e.g.,
10: foreach interval ∈ interval_lst do
11: FieldBoundaries.append(interval)
int or string). Such semantic types are more challenging to
12: S P L I T O V E R L A P P E D I N T E R V A L S (FieldBoundaries) identify than variable types but are more beneficial to fuzzing.
▷ Split overlapped intervals with the sorted end-points
In our prototype, we support six semantic types which can
13: return S O R T E D (FieldBoundaries)
help the fuzzer mutate the field more efficiently for most
real-world programs.
1 in Listing 1) processes bytes at offsets 16–19 and 40–51. • Size. This type of field represents the length of a data chunk
The second BB is common in programs since they would consisting of one or multiple fields in the input file. For
first read different fields in the input as a whole to a buffer in example, the e_ehsize field at offset 0x28 in Figure 1 in-
the memory, then the program will parse them accordingly. dicates the header size is 0x34 bytes.
Third, we split fields based on each block’s taint attributes • Enumeration. This type of field can only take a limited set
both in general BBs (e.g., BYTE_GET) and field-specific ones. of valid values. For example, the e_type field indicates the
For each MC of consecutive bytes, which is processed as a binary type, taking one of the seven valid values defined in
whole in all BBs, e.g., the bytes at (18, 19) in Figure 3, will the ELF format [20].
be recognized as a field. • Magic number. This type of field, in general, serves as the
The MC method can find most input fields correctly. How- signature of a file. For example, the first four bytes of an
ever, there are some exceptions that the field boundaries are ELF file is a magic number with the value "\x7fELF".
different from the specification. This is because the program • String. This type of field indicates a string literal, either in
parses the field in its way. For example, the program may ASCII, Unicode, or other encodings.
check the magic number byte by byte rather than as a whole. • Checksum. This type of field is used to verify the integrity
In such cases, MC may fail to group the bytes into a single of a part of data in the input and is common in media files,
field. We will explain it with an actual example in §5.1.1. compressed files, and font formats (e.g., PNG, ZIP, or TTF).
• Offset. This type of field indicates the location of another
chunk of data in the input. For example, the e_shoff field
3.2 Field Type Classification
at offset 0x20 in Figure 1 indicates the offset of the section
Based on Observation 2, we propose to classify the pattern of header.
BBs to infer the type of input fields processed by them. Previ-
ous works [16, 17] have proved that neural network models 3.2.2 Training Data Collection
are effective at extracting hidden features and outperforming
humans in several classification tasks. Thus, here we utilize We use the input format knowledge provided by the 010 Editor
neural networks to model BBs that process input fields. tool as the ground truth. We pick some well-known formats
Specifically, given a field type and the code snippets that and programs that process them, then perform the taint anal-
Algorithm 2 Data Vectorization Algorithm 3 Power Scheduling Algorithm
Input: FieldOffset, BinaryProgram, TaintTrace Input: Seeds
Output: FeatureOffset 1: AvgFormatExecCnt = 0
1: IRFEATURE = [‘CmpNEZ’,‘MSubF’, ... ] 2: foreach seed ∈ Seeds do
2: LIBCALL = [‘memcpy’,‘strcpy’, ... ] 3: if I S I N I T I A L S E E D () then
3: FORMATSTR = [‘%s’,‘%d’, ... ] 4: seed.format = E X T R A C T F O R M A T (seed)
4: BBs = E X T R A C T BB(BinaryProgram) 5: seed.format.exec_cnt = 0
5: FeatureOffset = [0]*(len(IRFEATURE)+len(LIBCALL)+len(FORMATSTR)) 6: if not F U Z Z E R I S S T U C K () then
6: foreach BB ∈ BBs do 7: M U T A T E A N D C H E C K (seed)
7: if I S BBP R O C E S S E D (BB) then ▷ Skip if BB has been processed be- 8: else
cause of forward slicing 9: if seed.format.exec_cnt > AvgFormatExecCnt then
8: continue 10: continue ▷ Skip the seed whose format has been mutated ade-
9: if I S BBT A I N T E D (FieldOffset,TaintTrace,BB) then quately
10: BB = F O R W A R D S L I C I N G (BB) 11: else
11: ir_lst = E X T R A C T IR(BB) ▷ Vectorize IR features 12: M U T A T E A N D C H E C K (seed) ▷ Re-assign power to the seed with a
12: ir_feature = [0]*len(IRFEATURE) less mutated format
13: foreach operation ∈ ir_lst do 13: procedure M U T A T E A N D C H E C K (seed):
14: index = IRFEATURE.index(operation) 14: new_seed = M U T A T E W I T H F O R M A T (seed)
15: ir_feature[index] += 1 15: E X E C U T E A N D C H E C K (new_seed) ▷ Execute ’new_seed’ and add to
16: str_lst = E X T R A C T FS(BB) ▷ Vectorize format string features queue if it brings new coverage
17: str_feature = [0]*len(FORMATSTR) 16: new_seed.format = seed.format
18: foreach str ∈ str_lst do 17: new_seed.format.exec_cnt = U P D A T E E X E C C N T ()
19: index = FORMATSTR.index(str) 18: AvgFormatExecCnt = U P D A T E A V G C N T ()
20: str_feature[index] += 1 19: if I S V A L U A B L E (new_seed) then
21: libcall_lst = E X T R A C T CALL(BB) ▷ Vectorize libcall features 20: new_seed.format = E X T R A C T F O R M A T (new_seed)
22: call_feature = [0]*len(LIBCALL) 21: new_seed.format.exec_cnt = 0
23: foreach libcall ∈ libcall_lst do
24: index = LIBCALL.index(libcall)
25: call_feature[index] += 1
26: FeatureOffset += (ir_feature,call_feature,str_feature) will record it in the one-hot encoding.
27: return N O R M A L I Z E (FeatureOffset)
Lastly, we record the format strings used in the BB and
count them into the feature of the semantic information. For
example, %x indicates an integer, while %s may imply a field of
ysis. Given an input field, we determine its type from the a string. This feature is also presented with one-hot encoding.
ground truth, locate its offset, and then filter relevant BBs These three parts of the features above are concate-
that have processed this field from the taint analysis results. nated (right side of line 26 in Algorithm 2) to get the feature
For each relevant BB, we perform a forward slicing (i.e., line vector of the slice. If a field is processed by multiple code
10 in Algorithm 2) to merge consecutive child BBs with an slices, then these slices’ vectors will be added together (left
important semantic dependency on the given field. side of line 26) to get the overall feature vector of this field.
Take the code snippet in Figure 4 as an example. The size As a result, we use the (feature vector, field type)
field is loaded in the first block and is compared against a pairs to train the neural network models.
threshold, then used by a malloc call in the consecutive child
block. We tend to slice the code forward to merge these two 3.2.4 CNN Model Building
blocks so that we can learn this field will be used by malloc
and collect more meaningful features. We choose the CNN model to infer the field type for two
reasons. First, CNN models are proven effective in many clas-
3.2.3 Data Vectorization sification tasks and binary analysis tasks [17, 22, 23]. In our
task, we intend to classify the field types into different cat-
Algorithm 2 shows how AIFORE vectorizes the training data egories based on the extracted code features. Second, CNN
consisting of pairs of code slices. Several features are consid- models can naturally filter out the background information
ered when vectorizing the code slices. and effectively capture valuable features. In our task, the
First, the semantic information of instructions in the BB is background information can be those features in the com-
essential for determining how the program processes the field. mon instructions that are activated by multiple types of fields.
To simplify the analysis, we transform instructions in BBs In our prototype, we build a CNN model with six embed-
to Intermediate Representation (IR, e.g., VEX [21]). We vec- ded hidden layers. We use adam as the optimizer, and use
torize the corresponding IR operation with one-hot encoding categorical_crossentropy as the loss function.
for each instruction that processes the given input field. Since
the number of IR operations in VEX is too large for one-hot
3.3 Fuzzing with Power Scheduling
encoding, we select about 100 commonly used IR operations
(line 1 in Algorithm 2). Through the previous steps, we can identify field boundaries
Besides, we also consider the use of standard library calls and field types. However, we still have the following two
as vital semantic information. We first choose a list of com- problems when using the knowledge to help fuzz. First, on
mon standard library functions such as memcpy, strcpy, and which seed should we perform format extraction (i.e., power
malloc. For each invocation of such a library function, we scheduling for format analysis)? Second, how can we bal-
ance the fuzzing power for different format variants produced Table 2: (format, program) pairs used in our experiments.
during mutation (i.e., power scheduling for mutation)? We Program&Parameter Format Boundary Type1 Fuzz
7Z ✓ ⊙
design a power scheduling algorithm as shown in Algorithm 3 7za t @@
ZIP ✓ X* ✓
, which is built on top of AFL, to solve both problems. First, readelf -a @@ ELF ✓ ⊙ ✓
elfutils-readelf -a @@ ELF X ✓
AIFORE will try to analyze the initial seed to build its format GIF X* ✓
model (Line 3 to 5). Then the fuzzer will mutate the seed exiv2 pr @@ JPG ✓ X*
with the guidance of format knowledge (Line 7). If the newly TIFF ⊙
tiffdump @@ TIFF ✓ ⊙ ✓
mutated seed is valuable, then we re-analyze its format (Line magick identify @@ GIF ⊙ ✓
19 to 21). When the fuzzer gets stuck (i.e., the fuzzer fails to gif2tiff @@ GIF ⊙ ✓
gifsicle @@ -o out.gif GIF ✓ ⊙ ✓
get new coverage in a given time), we will assign the fuzzing jasper –input @@ –output
BMP ✓ X ✓
power to those formats which are not fully mutated yet (Line test.bmp
jhead -v -exifmap @@ JPG X ✓
8 to 12). pngtest @@ PNG ✓ X ✓
Power scheduling for format analysis. The fuzzer gener- freetype_parser @@
OTF ✓ X*
TTF ✓ ⊙ ✓
ates numerous seeds during fuzzing. Identifying field bound- sfinfo @@ WAV ✓ X ✓
aries and types of each seed is unnecessary and impractical, tcpdump -nr @@ PCAP ✓ X ✓
since not all the seeds are valuable, and analyzing all the xls2csv @@ XLS ✓ X ✓
arping 192.168.1.1 -I eno2 -c 1 ARP2 ✓ X
seeds will cost too much power . Several works [13, 15, 24] nslookup example.com DNS2 ✓ X
have proved that not all seeds are valuable during fuzzing. 1 ⊙: the (format, program) pair is in the training set; X: the program
So we extract the format knowledge (i.e., field boundary and is not in the training set; X*: the program is in the training set but
field type) only for the initial seed and the seeds which we the (format, program) pair is not.
2 We show the evaluation results of ARP and DNS in §Appendix A.
consider valuable. "Valuable" seeds in our design mean the
input can reach more new BBs and likely belong to an un-
seen format variant (e.g., a new type of elf architecture in
and get the map of instructions and the input bytes processed
Figure 1). If a seed is "valuable", we extract its field bound-
by them. We develop two format analysis modules with 7K
aries and field types through the function E X T R A C T F O R M A T
lines of Python code. We use Angr [27] as the backend to
in Algorithm 3.
support static analysis. To train the model, we implement the
Power scheduling for mutation. Not all the formats are
machine learning component with Keras 2.2.4. To collect the
mutated equally during fuzzing. The fuzzer mutates the seed
ground truth data, we write an automation script to export the
one by one with M U T A T E W I T H F O R M A T function. Specifi-
template result from 010 Editor with the help of AutoIt [28].
cally, it will mutate the bytes of each field as a whole , and
We convert the extracted format knowledge into a Peach pit
choose a proper mutator based on the field type. For example,
file.
the mutator will try interesting size values instead of bitflip
for a size type field and insert or delete bytes for a string
type field. Besides, for those bytes without format knowledge,
the fuzzer will use the default mutation methods like those
in AFL. We will further analyze how the adaptive mutator 5 Evaluation
benefits fuzzing in §5.3.2.
During mutation, whenever a valuable seed occurs, the
fuzzer starts mutating the seed based on the re-analyzed for- We evaluate AIFORE to answer four research questions:
mat. However, if two "valuable" seeds show up close , the • RQ1: What is the performance of each format extraction
previous one is mutated inadequately. We design a power module of AIFORE? (§5.1)
re-assign mechanism to balance the fuzzing power among the • RQ2: What is the format analysis performance of
format variants. When the fuzzer gets stuck, we re-assign the AIFORE, compared with other SOTA format reverse
fuzzing power to those seeds bound with less mutated format engineering works? (§5.2)
variants and skip those seeds bound with the fully mutated • RQ3: What is the performance of AIFORE in terms of
format variants. code coverage and bug detection compared with other
SOTA fuzzers? (§5.3)
• RQ4: How does each module of AIFORE contribute to
4 Implementation fuzzing efficiency? (§5.4)
We build the taint analysis engine in C++ based on We run all experiments on a machine with a 24-core CPU
VUzzer [25] and libdft [26] to support 64-bit programs and (Intel(R) Xeon(R) CPU E5-2650) and 128GB memory. We
byte-level taint analysis. We re-write the taint propagation train the model with a Tesla P100 GPU. After the model is
component for each kind of instruction with about 5k lines of trained, AIFORE only uses the GPU when predicting field
code to check if the operands are tainted by the input bytes types, and it is used infrequently.
5.1 RQ1: Performance of Format Extraction Table 3: Signature field of ELF from 010 Editor.
Name Value Start Size
In this section, we evaluate each of AIFORE’s format extrac- char file_identification[4] ‘\x7fELF’ 0 4
char file_identification[0] 127‘\x7f’ 0 1
tion modules to see how well it extracts the format. char file_identification[1] 69‘E’ 1 1
Target File Formats and Programs. According to Klees char file_identification[2] 76‘L’ 2 1
char file_identification[3] 70‘F’ 3 1
et al. [29], there are on average 7 real-world programs eval-
uated by 32 unique papers in recent years. In this paper, we
collect 17 programs to evaluate AIFORE, as well as 15 kinds
of formats, of which 13 are file inputs (including image, exe- the test suite of the target programs which have various fea-
cutable, compression file, or compound file) and 2 are network tures like different file sizes, different compression levels, or
protocols (§Appendix A). Table 2 shows all formats and pro- different enumerations in specific formats. For each sample,
grams (with parameters) we used to evaluate each module of we manually craft the ground truth of boundary information as
AIFORE. described in Ground Truth. For each format, we choose one
We choose target inputs and programs based on the follow- program which parses input files most completely as shown
ing considerations. First, these formats are diverse and widely in column "Boundary" of Table 2. Then we use AIFORE to
used. Second, the program could parse as many fields as pos- infer their field boundaries (denoted as BoundaryA ).
sible when given one type of input, since AIFORE analyzes
We use accuracy as the metric, i.e., the number of fields
input format based on the dynamic taint trace, and cannot
correctly identified by AIFORE divided by the total number
recognize input fields that are not processed4 . We compile
of fields in the ground truth.
these programs with different optimization levels (from -O0
to -Os) to assess the robustness of AIFORE. Note that BoundaryT might be coarse-grained, i.e., the tem-
Ground Truth. First, given an input to test, we use the plate from 010 Editor misses some fine-grained fields. For
public format template from the 010 Editor to parse the file example, the template for PCAP only describes fields from the
and export all field records with boundary and type informa- data link layer to the transport layer (i.e., TCP/UDP), without
tion. These templates are written by experts and verified by knowledge of fields in the application layer (e.g., HTTP/DNS).
the community. Nevertheless, the target program tcpdump parses the packet
Then, we manually preprocess the records to get the ground in more detail, making AIFORE produce more fine-grained
truth of field boundaries in two steps: (1) Delete redun- boundary information. We skip these fields when calculating
dant definitions from the ground truth. For example, Table 3 the accuracy. To ensure the fairness of the experiment, we
shows the signature field of the ELF format. Field records also remove them when counting the accuracy for all other
file_identification[0] to [3] should be removed since solutions in §5.2. Although AIFORE may perform worse (or
file_identification[4] exists as a whole field. (2) Re- even better) in the fields we manually removed, we consider
move field records that are not parsed by the target program. the results will only be affected slightly since such cases are
For example, readelf does not process the ei_pad field of rare during our analysis. Further, our result in §5.2 shows the
the ELF format in Figure 1. We skip these fields because all accuracy of AIFORE is much higher than other solutions.
approaches that reverse the input format based on the behavior
The results are shown in Table 4 and we draw 2 conclusions.
of the program cannot figure them out.
First, the accuracy is irrelevant to the compiler optimization
Finally, we manually check the records to correct am-
level. That is because the MC method is considered from the
biguous semantic types of fields. For example, the char
view of semantic block, which is seldom affected by compiler
cname[4] field in the template for the PNG file indicates the
optimizations. For the target of 7z and zip, the total number
type is string. However, it is used as a magic number type
of fields in the input is small, and thus even one incorrect
in the program pngtest. Another example, some formats
field may cause a large accuracy fluctuation (about 10%).
may contain a version field, which could be confusingly
Second, we manually investigated the targets which are with
marked as a string or an integer. To solve the problem,
low accuracy and found the reason is that the program parses
we manually analyze the semantic type of every field in the
the fields in its way. For example, readelf compares the
template, and remove those with confusing semantic type
first four bytes (i.e., magic number) in 4 BBs respectively,
labels, increasing the accuracy of model training.
as shown in Figure 5. With the MC approach, we will split
the magic number into four single-byte fields, as they are
5.1.1 Field Boundary Accuracy parsed in different BBs. However, they are defined as a whole
in the specification. Although the result is different from
We denote the ground truth of boundary information as
the specification, we consider this will not affect (or even
BoundaryT . We choose 5 samples from the Internet or from
benefit) the fuzzing efficiency since fuzzing is to test the
4 This limitation is shared among all solutions that utilize dynamic taint implementation of the program rather than extract accurate
analysis [4, 8]. field boundaries compared to the specification.
Table 4: Accuracy of field boundary analysis. The table shows the average number of fields and file size in BoundaryT , as well
as the accuracy for different optimization levels of target programs.
Optimation Levels
Format Program Avg. Field Count Avg. Size (bytes)
-O0 -O1 -O2 -O3 -Os
7Z 7za 9 425 55.56% 55.56% 55.56% 55.56% 55.56%
BMP jasper 1,243 1,702 88.81% 89.38% 88.58% 89.14% 89.14%
ELF readelf 77 324 96.10% 93.51% 93.51% 93.51% 93.51%
GIF gifsicle 217 821 96.77% 96.77% 97.23% 97.23% 94.00%
JPG exiv2 24 424 50.00% 50.00% 55.00% 53.58% 50.00%
OTF freetype_parser 258 226,772 75.44% 76.64% 61.06% 61.06% 58.58%
PCAP tcpdump 22 114 90.90% 90.90% 90.90% 88.64% 86.36%
PNG pngtest 42 239 63.12% 64.31% 63.12% 63.12% 63.12%
TIFF tiffdump 73 22,700 82.19% 82.19% 76.71% 76.71% 82.19%
TTF freetype_parser 110 7,876 76.36% 76.36% 76.36% 76.36% 76.36%
WAV sfinfo 18 37,524 77.77% 72.22% 72.22% 72.22% 72.22%
XLS xls2csv 288 19,968 41.67% 41.67% 36.81% 36.81% 36.81%
ZIP 7za 12 906 58.33% 58.33% 58.33% 58.33% 58.33%
Average 184 24,600 73.31% 72.91% 71.18% 70.94% 70.48%

mov rax, [rbp+header]

movzx eax, byte ptr [rax] ; [0]； 0.95 0.25
cmp al, 7Fh；[0] ；[0]
jnz short loc_41856C
0.90 0.20

mov rax, [rbp+header]

Growth Rate
movzx eax, byte ptr [rax+1]；[1] 0.85 accuracy 0.15

Accuracy
cmp al, 45h；‘E’；[1] 1.0
jnz short loc_41856C
growth rate
0.80 0.10

mov rax, [rbp+header]

movzx eax, byte ptr [rax+2]；[2] 0.75 0.05
cmp al, 4Ch；‘L’；[2]
jnz short loc_41856C
0.70 0.00
0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 2.00
mov rax, [rbp+header] Scale
movzx eax, byte ptr [rax+3]；[3]
EXIT(0)
cmp al, 46h；‘F’；[3]
jz short loc_41858D Figure 6: Mean validation accuracy (with 95% confidence
Figure 5: Basic block to parse magic number. interval error band) and growth rate among different scales of
training set.

5.1.2 Field Type Accuracy

Figure 14 shows these results. We conclude that (1) the
We now evaluate the performance of the CNN model based performance with different compiler optimization levels (i.e.,
on 4 experiments. -O0 to -Os) behave differently, and the average accuracy is
• Experiment 1: Can the CNN model perform well on over 85% ; (2) the accuracy results are stable even with mixed
the training set and the validation set? We collect 10,582 compiler optimizations, which means the model applies to
fields from 8 formats, and we choose corresponding programs real-world cases.
to parse these inputs to train the model. The training set is • Experiment 2: How much data do we need to train
marked as ⊙ in column "Type" of Table 2. The ratio between a sufficiently reliable CNN model? To further analyze how
the training set and the validation set we use is 4:1. Then we much data is needed to train the CNN model, we also calcu-
mark the six semantic types as labels and the corresponding late the accuracy achieved by the model trained on different
(vectorized) taint traces as input data to train the CNN model. amounts of data. We set 10,582 fields as the unit (i.e., 1.0)
It takes, on average, 2-3 hours of manual effort to label the scale and use different scales (i.e., 0.2 to 2.0) to train the
training data for one format. This only needs to be done once model accordingly. For each group, we randomly choose the
for each format and we consider this reasonable. training data and repeat the experiment 5 times. The result
To validate whether the model can perform well under dif- is shown in Figure 6. From the result, we can conclude that
ferent compiler optimizations, we trained the model from -O0 a larger amount of training data results in higher model ac-
to -Os as training data respectively. To further investigate if curacy. However, the growth rate stagnates quickly when the
the model applies to real-world cases (i.e., where the selected scale is larger than 1.0.
compiler optimization of the target is not known), we also • Experiment 3: For trained programs, can the model
evaluate the accuracy of the model with mixed optimizations. predict field types of unseen formats? We define a format
We spend 12 hours collecting the taint traces for all samples is unseen by a program if the pair of (program, format) is not
in the training set, and the training time for the models is on in the training set of AIFORE. In this experiment, we apply
average 20 minutes. After training the model, we evaluate AIFORE to programs that have been trained with certain for-
their Top-1 accuracy, i.e., whether the top-score prediction mats, and verify whether AIFORE could recognize unseen
result is correct. formats for these programs (marked as X* in Table 2). Specif-
Table 5: Field type classification accuracy. The results represent the Top-1 / Top-2 accuracy respectively, where the latter indicates
whether the type of either the top two results is correct.
Optimization Levels
Program Format Samples
-O0 -O1 -O2 -O3 -Os Mixed
exiv2 GIF 202 73.20%/80.00% 84.62%/100.00% 100.00%/100.00% 94.12%/100.00% 100.00%/100.00% 95.14%/100.00%
exiv2 JPG 109 77.78%/88.89% 90.00%/100.00% 66.67%/77.78% 80.00%/80.00% 72.73%/81.82% 76.66%/85.28%
freetype_parser OTF 361 74.13%/86.42% 71.30%/85.19% 76.47%/87.50% 80.00%/90.00% 67.86%/89.29% 70.56%/92.23%
7za ZIP 364 89.01%/96.98% 93.37%/97.79% 94.90%/99.06% 91.23%/98.25% 92.99%/97.19% 91.70%/98.18%
Average 78.53%/88.07% 84.82%/95.75% 84.51%/91.09% 84.51%/92.06% 83.40%/92.07% 83.52%/93.92%
elfutils-readelf ELF 5385 62.66%/74.59% 73.93%/78.67% 61.11%/87.50% 54.99%/64.33% 71.69%/72.21% 67.33%/75.77%
jasper BMP 1068 83.11%/97.30% 92.00%/99.43% 90.06%/99.42% 91.98%/100.00% 90.48%/98.10% 93.46%/100.00%
jhead JPG 913 84.00%/88.00% 86.36%/86.36% 86.36%/90.91% 86.96%/86.96% 84.00%/92.00% 85.92%/86.92%
tcpdump PCAP 1412 74.70%/91.42% 84.13%/85.72% 78.04%/91.77% 89.81%/91.40% 75.44%/81.43% 73.04%/85.83%
pngtest PNG 1004 80.12%/87.77% 86.88%/91.10% 82.35%/85.88% 88.34%/89.11% 83.31%/88.96% 80.11%/82.31%
sfinfo WAV 434 88.89%/100.00% 97.91%/100.00% 97.37%/100.00% 98.80%/98.80% 98.99%/100.00% 97.92%/99.74%
xls2csv XLS 1339 71.00%/84.93% 75.36%/81.88% 68.00%/71.20% 71.94%/71.94% 65.56%/77.46% 66.95%/71.76%
Average 77.78%/89.14% 85.22%/89.02% 80.47%/89.53% 83.26%/86.08% 81.35%/87.17% 80.68%/86.05%

ically, for programs in the training set, we collected some new -Os) against existing input format reverse engineering works,
file formats supported by them, and apply our trained model like Polyglot [3], Protocol Informatics (PI) project [30], Pro-
to predict the field types of the new formats. Fuzzer [8], AFL-Analyze [14], and TIFF-fuzzer [6] to mea-
The upper part of Table 5 shows the evaluation result. We sure the format reversing performance.
consider Top-2 accuracy because the Top-K suggestions from We collect the field boundary and the type results of
the model are still valuable for fuzzing. To be concrete, the AIFORE as described in §5.1.1 and §5.1.2, and compare the
Top-K field type knowledge can help the fuzzer reduce the results with ProFuzzer, AFL-Analyze, and TIFF-fuzzer. There
mutation space and find high-quality test cases faster. are also other tools like WEIZZ [9] that can extract input
We have two findings from this table: (1) AIFORE can formats. However, we did not compare the field boundary
predict the field type in unseen formats with high accuracy, accuracy of AIFORE with them since they cannot extract all
while the Top-1 accuracy is over 80% on average and the the field boundaries even though the target program parses
Top-2 accuracy is over 90%; (2) the model performance is the fields already. We analyze concrete cases to explain the
irrelevant to the programs’ optimization levels. Although reason for such false negatives in §Appendix B.
compiler optimizations might change the features of the code, Metrics. For the field boundary analysis, we calculate dif-
our model learns stable patterns across optimizations. ferent solutions’ accuracy as described in §5.1.1, i.e., the num-
• Experiment 4: For untrained programs, can the model ber of correctly identified fields divided by the total number
predict their (unseen) input formats? We apply the afore- of fields in the ground truth.
mentioned trained model to analyze untrained programs We delicately process the results for the field type analysis
(marked as X in Table 2) and predict their input formats. We since different solutions focus on different field types. For
choose 7 untrained programs (and 2 protocols shown in Ap- example, ProFuzzer classifies the field types into 6 categories
pendix) and corresponding formats to test the accuracy of while AFL-Analyze recognizes only 3 semantic types. We
AIFORE. These untrained programs are chosen based on two manually check the result produced by different solutions
criteria: (1) they should be able to parse the chosen formats, and calculate the accuracy accordingly. For AFL-Analyze, we
and (2) they do not share libraries, which process the chosen only check if its results match the 3 semantic types they define
formats, with programs in the training set. The latter require- (i.e., raw data, magic number, and length). For ProFuzzer,
ment is applied for a fair comparison. Note that, the chosen we similarly check its results. For AIFORE, we manually
formats (e.g., ELF) may be processed by other trained pro- check if the Top-1 result matches the 6 semantic types we
grams (e.g., readelf) in the training set. But still, they are define in §3.2.1.
unseen to the untrained program (e.g., elfutils-readelf). In addition, we also measure the average time cost of ana-
The bottom of Table 5 shows the evaluation result. We lyzing one input format for each solution.
learn that AIFORE can predict the field type with a Top-1 Test Targets. To be fair, we choose 4 formats and programs
accuracy of 81% and a Top-2 accuracy of 88% on average. with different accuracy levels from the training set (Table 4)
Even when we use mixed test data from different compiler and from unseen formats and programs (Table 5) respectively.
optimizations, AIFORE can also predict the field type based We intend to investigate what is the performance of other
on how the program parses the field. solutions regarding the targets with different accuracy levels
under AIFORE. The chosen targets are shown in Table 6. For
5.2 RQ2: Comparison of Format Extraction the programs, we compile all the targets with their default
compiler optimizations. Regarding input samples, we choose
In this section, we compare AIFORE with the CNN model the ones that are not in the training set for the trained programs
trained from programs with mixed optimizations (i.e., -O0 to and randomly choose some input samples for the untrained
 Table 6: Field boundary/type accuracy comparison between different input format reverse engineering solutions.
Large Seeds Small Seeds
Format Program
Size(bytes) AIFORE ProFuzzer AFL-Analyze TIFF-fuzzer Size(bytes) AIFORE ProFuzzer AFL-Analyze TIFF-fuzzer
ELF readelf 808 96.43%/87.73% 37.40%66.25% 43.73%/40.00% 94.30%/(N/A) 324 98.91%/91.23% 74.26%/66.67% 55.70%/38.60% 97.40%/(N/A)
GIF gifsicle 695 97.64%/87.31% 12.59%/52.94% 6.44%/11.76% 71.96%/(N/A) 198 97.64%/72.23% 50.29%/60.00% 31.96%/12.00% 65.30%/(N/A)
Trained
TIFF tiffdump 448 82.23%/89.33% 27.13%/42.11% 13.19%/21.05% 81.30%/(N/A) 166 84.33%/88.45% 37.01%/40.00% 21.05%/21.67% 82.33%/(N/A)
TTF freetype 542 67.43%/83.22% 30.28%/65.69% 9.93%/20.44% 5.56%/(N/A) 148 72.23%/85.32% 2.20%/0.00% 1.35%/21.21% 40.00%/(N/A)
PCAP tcpdump 894 88.64%/82.34% 28.84%/71.14% 0.00%/39.04% 81.20%/(N/A) 114 93.18%/84.23% 73.18%/77.78% 39.66%/44.44% 85.60%/(N/A)
WAV sfinfo 572 100.00%/95.55% 63.85%/66.67% 48.77%/57.69% 100.00%/(N/A) 44 100.00%/91.22% 56.25%/63.64% 56.25%/45.45% 100.00%/(N/A)
Untrained
BMP jasper 630 45.34%/83.54% 12.11%/91.18% 1.35%/32.35% 24.24%/(N/A) 58 45.34%/84.33% 75.00%/82.35% 28.12%/23.53% 22.22%/(N/A)
XLS xls2csv 6656 81.25%/74.56% (N/A)/(N/A)* 22.16%/0.56% 25.50%/(N/A) 5632 94.40%/78.32% (N/A)/(N/A)* 0.00%/51.02% 33.33%/(N/A)
Average 1406 82.37%/85.45% 26.53%/57.00% 18.20%/27.86% 60.51%/(N/A) 836 85.75%/84.42% 46.02%/48.81% 29.26%/32.24% 65.77%/(N/A)
* The tool fails to get the result within 24 hours

Table 7: Average time to parse a file (seconds).

Large Seeds Small Seeds
Input Program
Size(bytes) AIFORE (B/T) ProFuzzer AFL-Analyze TIFF-fuzzer2 Size(bytes) AIFORE (B/T) ProFuzzer AFL-Analyze TIFF-fuzzer2
ELF readelf 808 29(24/5) 14,224 14 8 324 28(22/6) 977 6 7
GIF gifsicle 695 24(19/5) 84,123 42 91 198 23(17/6) 11,392 5 18
Trained
TIFF tiffdump 448 43(39/4) 1,529 9 13 166 44(40/4) 145 4 11
TTF freetype 542 19(12/7) 2,967 8 13 148 17(11/6) 100 2 6
PCAP tcpdump 894 24(19/5) 26,241 18 8 114 20(14/6) 95 2 8
WAV sfinfo 572 21(17/4) 2,993 11 8 44 23(19/4) 27 1 8
Untrained
BMP jasper 630 21(15/6) 2,004 10 6 58 9(3/6) 110 0.1 6
XLS xls2csv 6,656 61(56/5) (N/A)1 255 22 5,632 52(47/5) (N/A)1 17 94
1 The tool fails to get the result within 24 hours
2 The time only includes field boundary extraction

programs. The details of the training set and validation set for we have a trained model with a stable prediction time. We
field type prediction are described in the first part of §5.1.2. find that in Profuzzer and AFL-Analyze, the profiling phase
Input Size. The input size will affect the run-time perfor- consumes most of the time, which is strongly related to the
mance of format reverse engineering. AIFORE, ProFuzzer, input size. However, the taint analysis in AIFORE (and TIFF-
and AFL-Analyze all rely on dynamic analysis to predict the fuzzer) is not very sensitive to the input size. Since AFL-
field type, but with different methods. Both ProFuzzer and Analyze and TIFF-fuzzer perform some rough analysis, they
AFL-Analyze mutate each input byte and rerun the program to have better execution time but achieve lower accuracy than
get the coverage bitmap as the profile of the current execution. AIFORE on average.
Based on the variation of this profile, they can analyze the Moreover, we also conduct reverse engineering on two
type features of each byte and combine consecutive bytes of protocols. The detailed results are presented in Appendix
similar features as a field with the corresponding type. Thus, a A. Compared with other tools [3, 30, 31], AIFORE can give
larger file may consume more time to get the result. AIFORE not only more accurate format knowledge in terms of field
is based on the taint trace and the CNN model to infer the boundary and type but also with more details.
field boundaries and field types. Although the model is only
trained once, the taint analysis requires a time-consuming 5.3 RQ3: Comparison of Fuzzing Performance
analysis. To better understand how the input size affects the
performance of each work, we split the input files into dif- There are already some fuzzers [2, 6, 8, 13, 15, 32] that try to
ferent groups according to their size and observe the time to extract format knowledge and perform power scheduling to
complete the analysis in each group. optimize the fuzzing process. We compare AIFORE against
them to investigate how much our format analysis and power
Results. Table 6 demonstrates the accuracy of field bound-
scheduling can improve fuzzing efficiency. Note that for the
ary and type analysis, and Table 7 shows the average cost of
field type classification, once the model has been built, we do
time to parse an input, respectively.
not need to retrain the model during the fuzzing process.
From Table 6, we can learn that AIFORE achieves higher Target Programs and Seeds. For the programs, we use 15
accuracy both in field boundary recognition and field type programs to parse files as shown in Table 2, of which 6 are
prediction. Besides, AIFORE performs better in trained pro- trained and 9 have not been seen by AIFORE. For each file
grams than those in untrained. This is consistent with our type, we randomly choose one input file as the initial seed for
experience since the model has learned the pattern of how all fuzzers.
those programs parse different field types accurately. Fuzzers. We compare AIFORE with 6 fuzzers, includ-
The average time to parse a file with AIFORE (and TIFF- ing format-aware fuzzers, format-unware but popular fuzzers,
fuzzer) does not differ significantly for different size classes, and power scheduling optimization fuzzers, i,e, AFL [33],
while ProFuzzer and AFL-Analyze spend much more time AFLFast [15], ProFuzzer [8], TIFF-fuzzer [6], WEIZZ [9],
parsing larger files. In Table 7, B represents boundary identifi- and EcoFuzz [13]. AFL is one of the most popular grey-box
cation, the count includes the required time for taint analysis fuzzing tools, and there are many fuzzers built on top of AFL.
(which requires most of the total time). T represents type AFLFast and EcoFuzz optimize AFL by prioritizing the seeds
prediction, its time cost remains stable during the test since that may lead to new coverage. ProFuzzer has a dynamic
 Table 8: Basic block coverage after 24 hours.
Format Program1 AIFORE (B)2 AIFORE (B+T)2 AIFORE (B+T+P)2 AFL AFLFast EcoFuzz ProFuzzer TIFF-fuzzer WEIZZ
BMP jasper 7123 7705 7887 6694 5926 7777 7437 5331 6344
readelf* 9880 13798 17985 9652 12020 13791 17872 2426 15720
ELF
elfutils-readelf 7213 7541 8308 6995 5873 6608 7978 2180 6519
gifsicle* 4702 5062 5435 4528 4634 4675 5315 4146 4578
magick* 14221 14414 16685 13529 13684 12335 14512 8351 10002
GIF
gif2tiff* 2458 2451 2576 2372 2466 2500 2580 2014 2383
exiv2 16529 17500 19245 14117 14417 16602 18952 7637 11437
JPG jhead 1228 1258 1281 1244 1244 1244 1253 856 1246
PCAP tcpdump 16772 19201 22963 14535 13743 19823 22930 1561 11837
PNG pngtest 3219 3223 3629 3217 3239 3268 3617 2846 4802
TIFF tiffdump* 1145 1155 1177 1054 1073 1104 1135 522 1128
TTF freetype_parser* 9893 10520 10370 6362 6309 10030 7278 4513 7316
WAV sfinfo 2477 2501 2632 2326 2326 2379 2498 2019 2566
XLS xls2csv 2476 2699 2706 2512 2510 2619 2424 1841 2470
ZIP 7za 24696 25266 28159 21429 22089 25226 27979 13299 21833
1* for trained pairs of formats and programs.
2 B for Field Boundary; T for Field Type; P for Power Scheduling Algorithm.

probing stage to infer the field boundary and field type for
improving fuzzing efficiency. TIFF-fuzzer and WEIZZ use mean
format knowledge to increase fuzzing efficiency. 80%

60%

Relative Increment
5.3.1 Code Coverage Result
40%

For each target program, we run all fuzzers for 24 hours with 20%
5 repetitions. Then we measure the average BB coverage
0%
instead of path coverage since not all the fuzzers use the same
metric to calculate the path. −20%

We can draw the following conclusions from the result (BTP) (BTP) (B) (BT) (BTP)
in Table 8. First, AIFORE increases the coverage signifi- ProFuzzer WEIZZ AFL (B) (BT)

cantly for most targets except pngtest for which WEIZZ Figure 7: Coverage comparison between AIFORE and other
performs the best. The reason is that WEIZZ can not only SOTA fuzzers, and coverage increment by each module of
detect the checksum field in the file, but it can also correct AIFORE.
the checksum values. However, AIFORE does not support
checksum value correction even though it can be aware that
the field is a checksum field. For all the targets, AIFORE programs from Table 4 for 7 days. With the help of format
(B+T+P) (i.e., enable the field boundary, type analysis, and knowledge, AIFORE finds 34 bugs (20 are uncovered by
utilize the power scheduling algorithm) has an average 6% other fuzzers) in total after manual deduplication, including
and 26% increment on average compared with ProFuzzer and 10 buffer overflow bugs (CWE-122), 18 NULL pointer deref-
WEIZZ respectively, as shown in Figure 7. erence bugs (CWE-476), and 6 double-free bugs (CWE-617).
Second, the coverage increment with AIFORE is signifi- Among the 20 bugs that are uncovered by other fuzzers, 18
cant, even though the target program and the file format are of them have been known to vendors before. AIFORE finds
unseen. 2 vulnerabilities in the newest version of xls2csv, and we
Third, AIFORE achieves the best performance. For TIFF- have responsibly reported both bugs to the Launchpad, and
fuzzer, since it aims at maximizing the likelihood of triggering the track IDs are 1901462 and 19014635 . Here we analyze 2
a bug, we find it is not good at increasing the code coverage. of 34 bugs to illustrate how AIFORE finds them.
Although ProFuzzer also performs better than format-unaware Sfinfo. Sfinfo is a program for parsing WAV files and show-
fuzzers like AFL and AFLFast in most cases, its analysis time ing their properties or metadata to users. AIFORE reports a
is proportional to the input size, which makes it not scaleable heap overflow bug during the testing. The bug locates in a
to large inputs. Observe that ProFuzzer performs the worst function for parsing an enumeration field called FormatTag.
in XLS except TIFF-fuzzer. The reason is that the minimum The field represents the way the wave data is stored and affects
size of the XLS seed is above 1k bytes which is too large for how the following bytes will be processed. If the FormatTag
ProFuzzer. field in the input file is mutated to WAVE_FORMAT_ADPCM, then
the length data might be calculated incorrectly, which then
5.3.2 Bugs Found by AIFORE causes a heap overflow. The key to triggering this bug is

Coordinated Vulnerability Disclosure. To explore 5 https://launchpad.net/ The bugs have not been opened to the pub-
AIFORE’s ability of bug finding, we fuzz the real-world lic when we submit the paper.
setting FormatTag equal to WAVE_FORMAT_ADPCM, which is Steelix [34] identifies magic numbers in the input to pass
easier to achieve if the fuzzer is aware of the boundary and value validation. But it does not analyze fields of other types.
type of this field. The FormatTag is an enumeration field and Intriguer [2] utilizes light-weighted taint analysis to find multi-
AIFORE identifies the boundary and the type of this field byte fields processed by instructions in the program, then uses
correctly. Then it mutates the field accordingly and triggers the field-level knowledge to optimize symbolic execution.
this bug. From the perspective of file format, it can only extract a small
Readelf. During the testing of readelf, AIFORE finds a part of the fields because of its incomplete traces. WEIZZ [9]
stack overflow bug. The bug can be triggered when a 4-bytes splits the input into fields according to dependencies between
offset field is mutated to an invalid value. The offset field input bytes and comparison instructions, which ignores the
serves as indicating the location of a string. When the string’s bytes not affecting the control flow of the program.
size is bigger than 256 bytes. a call of sprintf will cause AIFORE extracts more accurate, concrete, and complete
the stack buffer overflow. The root cause of this bug can be format knowledge of field boundaries and semantic types,
concluded as (1) the offset field must be set to point to an which improves fuzzing. Besides, AIFORE utilizes a novel
invalid string region, and (2) the string field pointed to must power scheduling algorithm to balance the power for different
be long enough to trigger the stack overflow. We backtrack formats.
the log and find that AIFORE identifies the 4-bytes offset
field correctly, and mutates the field as a whole to an invalid
6.2 Input Format Reverse Engineering
value. The invalid offset points to a string field and AIFORE
mutates the string field with a longer length. So it can trigger Input format reverse engineering works can be classified into
the bug. two main categories regarding the problem they solve.
Field Boundary Identification. Identifying the boundary
5.4 RQ4: Contribution of Each Module of different fields in the input is fundamental to reversing the
format. Several works try to split the input into fields based on
In this section, we analyze how each module of AIFORE con- taint analysis [3, 4, 35–38] or trace analysis of a few network
tributes to the fuzzing performance. We investigate the BBs messages [5, 30, 39–41]. The closest work to AIFORE is
covered by different components during fuzzing in §5.3. The Tupni [4], which uses the weighted taint information in the
result is shown from column 3 to column 5 in Table 8, and instruction, as long as a greedy algorithm to identify different
we draw a boxplot in Figure 7. We have the following con- fields. However, the instruction-level taint information may
clusions. First, the module of boundary identification helps produce false positives since it does not consider the semantic.
AIFORE to mutate the bytes which have the same semantic Besides, Tupni [4] is a coarse-grained method in which the
and belongs to one field, and it boosts AIFORE to cover 9.3% record it identifies may contain several fields, rather than
more BBs than AFL. Besides, AIFORE uses the AI model a single field. AIFORE considers the BB as the minimum
to predict the field type, which helps the fuzzer know how functional unit instead of the instruction. AutoFormat [35]
to mutate. Benefiting from the field type prediction module, combines dynamic taint analysis and call stack to build a
AIFORE increases another 6.9% code coverage on average field tree. It relies much on the tokenization and the operation
for all the targets. At last, AIFORE utilizes a novel power of the tree itself, rather than on how the program processes
scheduling algorithm to help the fuzzer assign more energy different fields. MIMID [42] and AUTOGRAM [43] also rely
to those formats which are not mutated adequately. It brings on dynamic taint analysis and call stack analysis. They are
an 8.8% increment in coverage. Different from AFLSmart, used to extract context-free grammar for text-based inputs
our approach does not rely on the constraints in the pit file. rather than context-sensitive ones like binary-based inputs
We further analyze how it works with a case study in Ap- which are more complex. Reverx [44] splits the input into
pendix §B.3. fields by predefined delimiters and it cannot be used for binary
messages. There are some other works [30, 45] that try to
6 Related Work identify the field by analyzing a large number of high-quality
inputs. Nevertheless, AIFORE can extract the field knowledge
with only one input.
6.1 Format-Aware Fuzzing
Field Type Identification. Identifying the type of distinct
Format-aware fuzzers try to understand the format of inputs fields is also an important problem. Current works [3, 34, 46,
to increase the fuzzing efficiency. TIFF-fuzzer [6] performs 47] mainly utilize dynamic program analysis and predefined
bug-directed mutation by inferring some program variable rules to classify the input field into types. Dispatcher [48] iden-
types (e.g., int, char*) of input fields. ProFuzzer [8] uses tifies the field type by leveraging taint analysis and heuristics
predefined rules to deduce fields and corresponding types. rules similar to TIFF-fuzzer. Polyglot [3] tries to identify the
However, adding more data types is labor-intensive. Besides, keywords and separators in the protocol message. It also tries
the rules may not be accurate enough to cover all the cases. to identify the length field by heuristics rules. However, the
identified field types are limited, and the heuristic rules have 8 Conclusion
limitations when the message is not strict. For example, a
protocol may allow multiple delimiters. Input format knowledge is useful for fuzzing to discover vul-
Different from the existing works, AIFORE utilizes a nerabilities in programs. Existing approaches struggle at cor-
machine-learning model to extract the feature and predicts the rectly recognizing or applying the input format. We introduce
field types automatically. We feed multi-dimension seman- AIFORE to automatically reverse engineer input format and
tic features from the tainted instructions, format strings, and later guide the fuzzing process. Specifically, we propose to
library calls in related BBs to train the model. In this way, utilize taint analysis to infer basic blocks responsible for pro-
AIFORE does not require extra manually-defined rules and cessing each input byte, and group input bytes with a mini-
therefore it is more general. mum cluster algorithm. Further, we utilize a neural network
to infer the type of input fields based on the behavior of basic
blocks. Based on the input knowledge, we present a novel
6.3 Binary Analysis with AI power scheduling algorithm for fuzzers. A systematic eval-
uation shows that this solution has better effectiveness and
The closest works in AI-based binary analysis include bi- efficiency than existing baselines.
nary similarity detection [23, 49] and semantic information
recovery [50, 51]. For the binary similarity detection, [23]
utilizes BERT and CNN to find similar code. [52] uses Struc- Acknowledgements
ture2vec to vectorize the CFGs. Different from existing works,
This work was supported by the National Key Research and
AIFORE aims at classifying the input fields into different
Development Program of China (2021YFB2701000), Na-
categories, which is a classification problem instead of an
tional Natural Science Foundation of China (61972224), Bei-
embedding problem. There are also some works [50, 51] that
jing National Research Center for Information Science and
try to recover the semantic information (e.g., function names,
Technology (BNRist) under Grant BNR2022RC01006.
variable types) from the binary program with AI. However,
the program variable type is simpler than the field type. For
example, a variable of int type may represent an offset or References
a size field. AIFORE aims to recover the semantic type of [1] J. Chen, J. Jiang, H. Duan, N. Weaver, T. Wan, and V. Paxson, “Host
an input field, which is harder but more useful. of Troubles: Multiple Host Ambiguities in HTTP Implementations,”
in Proceedings of the 2016 ACM SIGSAC Conference on Computer
and Communications Security. Vienna Austria: ACM, Oct. 2016, pp.
1516–1527.
7 Limitation
[2] M. Cho, S. Kim, and T. Kwon, “Intriguer: Field-level constraint solving
for hybrid fuzzing,” in Proceedings of the 2019 ACM SIGSAC Confer-
While AIFORE has good accuracy for field boundary detec- ence on Computer and Communications Security, 2019, pp. 515–530.
tion and type analysis, some limitations remain. As described, [3] J. Caballero, H. Yin, Z. Liang, and D. Song, “Polyglot: Automatic
our method fundamentally relies on dynamic taint analysis. extraction of protocol message format using dynamic binary analy-
sis,” in Proceedings of the 14th ACM conference on Computer and
Thus, the key limitation of AIFORE is that how the program
communications security, 2007, pp. 317–329.
parses the input highly affects the result. For example, if a
[4] W. Cui, M. Peinado, K. Chen, H. J. Wang, and L. Irun-Briz, “Tupni:
program does not parse some fields, AIFORE cannot extract Automatic reverse engineering of input formats,” in Proceedings of
the format knowledge. Further, if a program parses individ- the 15th ACM conference on Computer and communications security,
ual bytes of a field separately, AIFORE may produce false 2008, pp. 391–402.
positives. However, we can overcome this issue by feeding [5] W. Cui, J. Kannan, and H. J. Wang, “Discoverer: Automatic proto-
the input to several programs that can parse this format. The col reverse engineering from network traces.” in USENIX Security
Symposium, 2007, pp. 1–14.
second limitation is that our analysis runs at byte granularity,
[6] V. Jain, S. Rawat, C. Giuffrida, and H. Bos, “Tiff: using input type
which means bit-level fields cannot be analyzed. Supporting inference to improve fuzzing,” in Proceedings of the 34th Annual
bit-level analysis is technically possible but requires further Computer Security Applications Conference, 2018, pp. 505–517.
engineering and optimization. The byte-level analysis also [7] Z. Lin, X. Zhang, and D. Xu, “Automatic reverse engineering of data
indicates that AIFORE does not support text-based input. The structures from binary execution,” in Proceedings of the 11th Annual
minimum unit of text-based input is a keyword rather than Information Security Symposium, 2010, pp. 1–1.
a byte. Third, it is hard for AIFORE to analyze the cases [8] W. You, X. Wang, S. Ma, J. Huang, X. Zhang, X. Wang, and B. Liang,
of input encryption or code obfuscation (e.g., in malware or “Profuzzer: On-the-fly input type probing for better zero-day vulnera-
bility discovery,” in 2019 IEEE Symposium on Security and Privacy
ransomware). There is no obvious format information in en- (SP). IEEE, 2019, pp. 769–786.
crypted input, and the developer may also use obfuscation [9] A. Fioraldi, D. C. D’Elia, and E. Coppa, “Weizz: Automatic grey-box
code to hide the operation pattern to parse the file. There are fuzzing for structured binary formats,” in Proceedings of the 29th ACM
several orthogonal works, for example, Reformat [38] that try SIGSOFT International Symposium on Software Testing and Analysis,
to reverse the format of encrypted input. 2020, pp. 1–13.
[10] V. Pham, M. Böhme, A. E. Santosa, A. R. Caciulescu, and A. Roy- [31] A. Orebaugh, G. Ramirez, and J. Beale, Wireshark & Ethereal network
choudhury, “Smart greybox fuzzing,” IEEE Transactions on Software protocol analyzer toolkit. Elsevier, 2006.
Engineering, 2019.
[32] D. She, A. Shah, and S. Jana, “Effective Seed Scheduling for Fuzzing
[11] “The peach project,” https://www.peach.tech/. with Graph Centrality Analysis,” in 2022 IEEE Symposium on Security
[12] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classifica- and Privacy (SP). IEEE Computer Society, Apr. 2022, pp. 1558–1558.
tion with deep convolutional neural networks,” Advances in neural [33] M. Zalewski, “American fuzzy lop,” http://lcamtuf.coredump.cx/afl/,
information processing systems, vol. 25, pp. 1097–1105, 2012. 2014.
[13] T. Yue, P. Wang, Y. Tang, E. Wang, B. Yu, K. Lu, and X. Zhou, [34] Y. Li, B. Chen, M. Chandramohan, S.-W. Lin, Y. Liu, and A. Tiu,
“{EcoFuzz}: Adaptive {Energy-Saving} greybox fuzzing as a vari- “Steelix: program-state based binary fuzzing,” in Proceedings of the
ant of the adversarial {Multi-Armed} bandit,” in 29th USENIX Security 2017 11th Joint Meeting on Foundations of Software Engineering, 2017,
Symposium (USENIX Security 20), 2020, pp. 2307–2324. pp. 627–637.
[14] “Automatically inferring file syntax with afl-analyze,” 2016, https:// [35] Z. Lin, X. Jiang, D. Xu, and X. Zhang, “Automatic protocol format
lcamtuf.blogspot.com/2016/02/say-hello-to-afl-analyze.html. reverse engineering through context-aware monitored execution.” in
[15] M. Böhme, V.-T. Pham, and A. Roychoudhury, “Coverage-based grey- NDSS, vol. 8. Citeseer, 2008, pp. 1–15.
box fuzzing as markov chain,” IEEE Transactions on Software Engi- [36] P. M. Comparetti, G. Wondracek, C. Kruegel, and E. Kirda, “Prospex:
neering, vol. 45, no. 5, pp. 489–506, 2017. Protocol specification extraction,” in 2009 30th IEEE Symposium on
[16] M. I. Jordan and T. M. Mitchell, “Machine learning: Trends, perspec- Security and Privacy. IEEE, 2009, pp. 110–125.
tives, and prospects,” Science, vol. 349, no. 6245, pp. 255–260, 2015.
[37] B. Cui, F. Wang, T. Guo, G. Dong, and B. Zhao, “Flowwalker: a fast
[17] J. Wang, Y. Yang, J. Mao, Z. Huang, C. Huang, and W. Xu, “Cnn-rnn: A and precise off-line taint analysis framework,” in 2013 Fourth Interna-
unified framework for multi-label image classification,” in Proceedings tional Conference on Emerging Intelligent Data and Web Technologies.
of the IEEE conference on computer vision and pattern recognition, IEEE, 2013, pp. 583–588.
2016, pp. 2285–2294.
[38] Z. Wang, X. Jiang, W. Cui, X. Wang, and M. Grace, “Reformat: Auto-
[18] J. Lee, T. Avgerinos, and D. Brumley, “Tie: Principled reverse engi- matic reverse engineering of encrypted messages,” in European Sympo-
neering of types in binary programs,” 2011. sium on Research in Computer Security. Springer, 2009, pp. 200–215.
[19] A. Slowinska, T. Stancescu, and H. Bos, “Howard: A dynamic excavator [39] S. Kleber, H. Kopp, and F. Kargl, “{NEMESYS}: Network message
for reverse engineering data structures.” in NDSS, 2011. syntax reverse engineering by analysis of the intrinsic structure of
[20] “Tool interface standard (tis) executable and linking format (elf) speci- individual messages,” in 12th {USENIX} Workshop on Offensive Tech-
ficatione,” 1995, https://refspecs.linuxfoundation.org/elf/elf.pdf. nologies ({WOOT} 18), 2018.
[21] Y. Shoshitaishvili, R. Wang, C. Hauser, C. Kruegel, and G. Vigna, “Vex,” [40] I. Bermudez, A. Tongaonkar, M. Iliofotou, M. Mellia, and M. M. Mu-
https://github.com/angr/pyvex. nafò, “Towards automatic protocol field inference,” Computer Commu-
nications, vol. 84, pp. 40–51, 2016.
[22] H. Lee and H. Kwon, “Going deeper with contextual cnn for hyper-
spectral image classification,” IEEE Transactions on Image Processing, [41] J. Kannan, J. Jung, V. Paxson, and C. E. Koksal, “Semi-automated
vol. 26, no. 10, pp. 4843–4855, 2017. discovery of application session structure,” in Proceedings of the 6th
ACM SIGCOMM conference on Internet measurement, 2006, pp. 119–
[23] Z. Yu, R. Cao, Q. Tang, S. Nie, J. Huang, and S. Wu, “Order mat-
132.
ters: Semantic-aware neural networks for binary code similarity detec-
tion,” in Proceedings of the AAAI Conference on Artificial Intelligence, [42] R. Gopinath, B. Mathis, and A. Zeller, “Mining input grammars from
vol. 34, no. 01, 2020, pp. 1145–1152. dynamic control flow,” in Proceedings of the 28th ACM Joint Meeting
on European Software Engineering Conference and Symposium on the
[24] A. Rebert, S. K. Cha, T. Avgerinos, J. Foote, D. Warren,
Foundations of Software Engineering, ser. ESEC/FSE 2020. New York,
G. Grieco, and D. Brumley, “Optimizing seed selection for
NY, USA: Association for Computing Machinery, 2020, pp. 172–183.
fuzzing,” in 23rd USENIX Security Symposium (USENIX Security
14). San Diego, CA: USENIX Association, Aug. 2014, pp. [43] M. Höschele and A. Zeller, “Mining input grammars with AUTO-
861–875. [Online]. Available: https://www.usenix.org/conference/ GRAM,” in 2017 IEEE/ACM 39th International Conference on Soft-
usenixsecurity14/technical-sessions/presentation/rebert ware Engineering Companion (ICSE-C), 2017, pp. 31–34.
[25] S. Rawat, V. Jain, A. Kumar, L. Cojocar, C. Giuffrida, and H. Bos, [44] J. Antunes, N. Neves, and P. Verissimo, “Reverx: Reverse engineering
“Vuzzer: Application-aware evolutionary fuzzing.” in NDSS, vol. 17, of protocols,” 2011.
2017, pp. 1–14.
[45] I. Bermudez, A. Tongaonkar, M. Iliofotou, M. Mellia, and M. M. Mu-
[26] V. P. Kemerlis, G. Portokalidis, K. Jee, and A. D. Keromytis, “libdft: nafo, “Automatic protocol field inference for deeper protocol under-
Practical dynamic data flow tracking for commodity systems,” in Pro- standing,” in 2015 IFIP Networking Conference (IFIP Networking).
ceedings of the 8th ACM SIGPLAN/SIGOPS conference on Virtual IEEE, 2015, pp. 1–9.
Execution Environments, 2012, pp. 121–132.
[46] T. Wang, T. Wei, G. Gu, and W. Zou, “Taintscope: A checksum-aware
[27] F. Wang and Y. Shoshitaishvili, “Angr-the next generation of binary directed fuzzing tool for automatic software vulnerability detection,”
analysis,” in 2017 IEEE Cybersecurity Development (SecDev). IEEE, in 2010 IEEE Symposium on Security and Privacy. IEEE, 2010, pp.
2017, pp. 8–9. 497–512.
[28] Lonami, “Autoit scripting language,” 2018, https://www.autoitscript. [47] Z. Lin and X. Zhang, “Deriving input syntactic structure from ex-
com/site/. ecution,” in Proceedings of the 16th ACM SIGSOFT International
[29] G. Klees, A. Ruef, B. Cooper, S. Wei, and M. Hicks, “Evaluating fuzz Symposium on Foundations of software engineering, 2008, pp. 83–93.
testing,” in Proceedings of the 2018 ACM SIGSAC Conference on [48] J. Caballero, P. Poosankam, C. Kreibich, and D. Song, “Dispatcher:
Computer and Communications Security, 2018, pp. 2123–2138. Enabling active botnet infiltration using automatic protocol reverse-
[30] M. A. Beddoe, “Network protocol analysis using bioinformatics algo- engineering,” in Proceedings of the 16th ACM conference on Computer
rithms,” Toorcon, 2004. and communications security, 2009, pp. 621–634.
[49] F. Zuo, X. Li, P. Young, L. Luo, Q. Zeng, and Z. Zhang, “Neural ma- The results of DNS and ARP format reverse engineering are
chine translation inspired binary code similarity comparison beyond shown in Figure 8 and Figure 9 respectively. The orange parts
function pairs,” arXiv preprint arXiv:1808.04706, 2018.
represent the wrong results. From the result, we can learn that
[50] J. He, P. Ivanov, P. Tsankov, V. Raychev, and M. Vechev, “Debin: Pre- AIFORE extracts more detailed and correct information than
dicting debug information in stripped binaries,” in Proceedings of the
2018 ACM SIGSAC Conference on Computer and Communications Wireshark in the green parts of DNS. From the RFC speci-
Security, 2018, pp. 1667–1680. fication [53] of DNS, we know that the QNAME field contains
[51] Y. David, U. Alon, and E. Yahav, “Neural reverse engineering of several labels, and each label consists of a byte representing
stripped binaries using augmented control flow graphs,” Proceedings of the length followed by a number of octets representing data.
the ACM on Programming Languages, vol. 4, no. OOPSLA, pp. 1–28, Wireshark marks the QNAME as a whole and is not able to split
2020.
it into individual labels. AIFORE extracts more detailed field
[52] X. Xu, C. Liu, Q. Feng, H. Yin, L. Song, and D. Song, “Neural network- boundaries than Wireshark, as shown in the result. AIFORE
based graph embedding for cross-platform binary code similarity de-
tection,” in Proceedings of the 2017 ACM SIGSAC Conference on also identifies the field type correctly.
Computer and Communications Security, 2017, pp. 363–376.
[53] P. V. Mockapetris, “Rfc1035: Domain names-implementation and spec-
ification,” 1987. B Case Study
[54] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and
D. Batra, “Grad-cam: Visual explanations from deep networks via In this section, we analyze some concrete examples to help
gradient-based localization,” in Proceedings of the IEEE international understand how AIFORE outperforms other state-of-art works
conference on computer vision, 2017, pp. 618–626. from the view of field boundary recognition and field type
classification. We take the ELF as an example to illustrate.

Appendices B.1 Field Boundary Case

A Comparison of Protocol Reverse Engineer- We take 2 format-aware fuzzing tools that can identify fields
in the input as examples. The field extraction result during
ing fuzzing is shown in Figure 10.
Beyond the file inputs, we also choose two protocols, ARP From the result, TIFF-fuzzer and AIFORE split the first
and DNS, to test and compare the result with Wireshark [31], four bytes (i.e., magic_number field) into single-byte fields.
Polyglot [3], and PI [30]. The program we choose to parse However, since the program parses the bytes one by one, then
ARP is arping which is provided by the system of Ubuntu it is better to fuzz each of the bytes rather than as a whole.
18.04, and the parameter is arping 192.168.1.1 -I eno2 For WEIZZ, there are a few false negatives. The reason is that
-c 1. This command produces an ARP request and then WEIZZ relies on cmp instruction when extracting the fields,
parses an ARP reply packet. The program to parse DNS which is not sufficient.
is nslookup from BIND 9, and the parameter we use is In AIFORE, we perform a complete taint analysis on valu-
nslookup example.com. able inputs, rather than WEIZZ, which achieves a higher accu-
In this experiment, we only focus on reversing the format racy on field identification and thus can increase the fuzzing
of the response part (i.e., ARP reply and DNS response packet). efficiency better than other fuzzers, as shown in Table 8.
Given only one response packet, AIFORE can successfully
reverse the format via taint analysis and related analysis. How- B.2 Field Type Case
ever, during the test against PI, it fails to reverse the format
given only one response packet, so we change the testing For field type identification, state-of-art works generally de-
command from -c 1 to -c 10 to collect 10 response packets pend on specific rules, and the field types they identified are
for PI to analyze. During the execution of the command, we usually program types rather than the semantic type of a
capture the packets so that we can later compare the results. field. For example, TIFF-fuzzer infers field types based on
We use the format knowledge taken from Wireshark as the APIs (strcmp,strcpy) in libraries and then splits the field
ground truth. Wireshark identifies protocol formats by tem- into several program variable types such as char* and int.
plate scripts written carefully by community experts. How- However, such rules and types may not be sufficient. Take
ever, we find AIFORE can extract more detailed format knowl- the section name, s_name, in the ELF file as an example.
edge in some cases, like the DNS target. Thus, we also use Such fields are string type. However, TIFF-fuzzer considers
the RFC 1035 [53], which defines the DNS protocol format them as consecutive int bytes. The reason is that readelf
specification as complementary to the ground truth. The or- uses repe cmpsb instruction rather than strcmp call to parse
ange parts represent the wrong results. AIFORE can not only s_name. In AIFORE, it marks this field as a magic number,
identify the field boundaries but also identify the field types. which is also reasonable, since the program tries to compare
AIFORE predicts the field type correctly for 4 fields out of 5. the section name with hardcoded strings. We then investigate
 Polyglot AIFORE PI 0.0.2
Wireshark 3.4.0 AIFORE PI 0.0.2 Bind 9.3.4 Bind 9.16.16 Bind 9.16.16
RFC 1035 Wireshark 3.4.0
0 0
Zeroed data
Hardware type: Enumeration Enum ID : Integer ID : Integer Unused N/A for type Binary data
2 Binary data 2
Protocol type: Enumeration Enum Flags : Size Flags : Size Fixed Size Binary data
Zeroed data 4
4
Hardware size: Size Size Zeroed data
Binary data QDCOUNT : Size Questions RR : Size Fixed Size
5 Binary data
Protocol size: Size Size 6
6 Header Zeroed data
Zeroed data ANCOUNT : Size Size
Opcode: Enumeration String Answer RR : Size Fixed
Binary data Binary data
8
8
Binary data NSCOUNT : Size Authority RR : Size Fixed Size
ASCII data 10 Zeroed data
ARCOUNT : Size Additional RR : Size Fixed Size
Sender MAC address: Address N/A for type Binary data
12
Direction Size Ascii data
Zeroed data

ARP Binary data Ascii data

14
Binary data
Sender IP address: Address N/A for type Variable String G data
Zeroed data
Binary data QNAME: Binary data
18 QNAME(String)
Binary data Labels(Length+String)
Queries
Direction Size
G data
Target MAC address: Address N/A for type ASCII data
Variable String
Ascii data
24 G data 0 N/A for type G data
25
QTYPE : Enumeration QTYPE : Enumeration Fixed Enum G data
Target IP address: Address N/A for type Binary data 27
Ascii data
QCLASS : Enumeration QCLASS : Enumeration Fixed Enum
G data G data
28 29

Figure 8: Results of the ARP reverse engineering. Figure 9: Results of the DNS format reverse engineering.

0 1 2 3 4 5 6 7 8 9 A B C D E F
0x00h: [7F] [45] [4C] [46] [01] [01] [01] [00] [00] [00] [00] [00] [00] [00] [00] [00] ; char *__cdecl get_file_type(unsigned int e_type)
…
0x10h: [02 00] [03 00] [01 00 00 00] [74 80 04 08] [34 00 00 00] …
0x20h: [A4 00 00 00] [00 00 00 00] [34 00] [20 00] [02 00] [28 00] mov rcx, 0F3DDh
call __afl_maybe_log ; bitmap changes here
0x30h: [04 00] [03 00] mov rax, [rsp + 98h + var_88]
mov rcx, [rsp + 98h + var_90]
(a) TIFF-fuzzer mov rdx, [rsp + 98h + var_98]
lea rsp, [rsp + 98h]
mov edi, edi
0 1 2 3 4 5 6 7 8 9 A B C D E F mov edx, 5 ; e_type
jmp ds:off_77DA00[e_type*8] ; switch jump
0x00h: [7F 45 4C 46] [01 01 01 00] 00 00 00 00 00 00 00 00
0x10h: [02 00] [03 00] 01 00 00 00 74 80 04 08 [34 00] [00 00
0x20h: A4 00 00 00] 00 00 00 00 34 00 [20 00] [02 00] [28 00]
loc_40DFF0: ; jumptable case 0 loc_40DFE0: ; jumptable case 4
0x30h: [04 00] [03 00] mov esi, offset aNoneNone mov esi, offset aCoreCoreFile
… case 2 …
xor edi, edi ; domainname
(b) WEIZZ xor
jmp
edi, edi
_dcgettext
; domainname
jmp _dcgettext

0 1 2 3 4 5 6 7 8 9 A B C D E F
0x00h: [7F] [45] [4C] [46] [01 01 01 00] 00 00 00 00 00 00 00 00 Figure 11: ProFuzzer failure case.
0x10h: [02 00] [03 00] 01 00 00 00 74 80 04 08 [34 00] [00 00
0x20h: A4 00 00 00] 00 00 00 00 34 00 [20 00] [02 00] [28 00]
0x30h: [04 00] [03 00] Another work that can identify the semantic type of a field
(c) AIFORE
is ProFuzzer. We take the field e_type locates at offset 0x10
Figure 10: Field identification results. Bytes in red brackets in Figure 10 as an example. It represents the file type (e.g.,
indicate they are different than the specification, while the ET_EXEC), which is an enumeration. During the probing
shallow parts indicate the target fails to extract the fields. stage, ProFuzzer considers it an incorrect offset type. We
then investigate the reason and find this is due to the limitation
of code coverage bitmap. ProFuzzer mutates each of the bytes
in e_type to observe the similarity of the bitmaps. But from
Figure 11, we can learn that different switches share the same
bitmap, which leads ProFuzzer to make the wrong decision.
why the model in AIFORE predicts the field as a magic num- However, in AIFORE, it predicts the field type based on how
ber. We leverage the Grad-cam [54], which is used to explain the program parses the specific field, and we also use forward
why a model makes a specific decision. It helps humans to slicing to merge BB to get more code features, which makes
understand the internal working principle of a classification it able to recognize this field correctly.
model.

B.3 Power Scheduling Case Study

To explain the decision of the model, we feed the vector-
ized semantic feature (IR operation, library call, and format To better understand how the power scheduling works in
strings) of the magic field to Grad-cam. Then we observe AIFORE, we choose tcpdump as a target to answer two ques-
which feature plays the most important role in the decision. tions. First, do valuable seeds really bring new formats? Sec-
We choose the top 5 features of Grad-cam to observe: [‘Cm- ond, can AIFORE assign more power to those formats which
pLT32U’, ‘128to64’, ‘CmpLE64S’, ‘And8’, ‘CmpLE32U’]. are mutated less?
As the result shows, the cmp feature plays the most important tcpdump is a program to parse pcap files. The pcap file
role when the model predicts the field as a magic number, consists of several data structures (e.g., pcap header, frame)
which is reasonable. which are composed of several fields. Before analyzing the
result, let us illustrate the frame structure first. As shown in 50%
80%

Figure 12, the frame has some fields with consecutive bytes 40%
70%

Rel. Exec. Times

Rel. Exec. Times

60%

(e.g., Incl Len consisting of bytes at offsets from 0x08 to 30% 50%

40%

0x0B) and some fields with single-byte (e.g., L4 Prot ) val- 20%
30%

ues. Take L4 Prot at offset 0x27 as an example, it indicates 10%

20%

10%

the data type (TCP, UDP, etc.) of Layer 4, which is high- 0%

0 1 2 3 4 5 6 7 8 9 10 11
0%
0 1 2 3 4 5 6 7 8
Structure Index Structure Index
lighted in yellow. tcpdump will dispatch different codes to (a) Power scheduling equipped. (b) Power scheduling unequipped.
parse various kinds of L4 layer data.
Figure 13: Execution times analysis for power scheduling
algorithm.

0 1 2 3 4 5 6 7 8 9 A B C D E F Table 9: New formats produced by valuable seeds.

0x00h: FF E6
Ts Sec
B9 60 01 35 08
Ts Usec
00 2A 00
Incl Len
00 00 6A 00 00
Orig Len
00 Layer2 Layer3 Layer4 No Variant
0x10h: 02 42 38 48 79 F0 02 42 AC 11 00 0D 08 00 45 08 88, 267, 287, 27, 51, 300,
Destination MAC Source MAC Layer3 Protocol Ver Len DiffServ 15, 207, 713,
00 5C 91 77 40 00 40 06 C3 E4 AC 11 00 0D 73 1B Origin 296, 310, 360, 420, 526, 551,
0x20h: Total Length Identification Flags TTL L4 Prot Checksum Source IP 728, 759, 784, 306, 593
Seed Id 368, 549, 618, 594, 924, 937,
0x30h: C5 FE 00 16 76 98 7D 55
SEQ
2E F4 FF E6 B9 60 5F 1928
Dest IP Source Port Dest Port ACK Tcp Len
725, 765 1785
Total 11 7 2 10
(a) Frame with TCP packet.
0 1 2 3 4 5 6 7 8 9 A B C D E F

0x00h: 36 00 00 00 96 D3 09 00 BD 01 00 00 BD 01 00 00
Ts Sec Ts Usec Incl Len Orig Len

0x10h: FF FF FF FF FF FF 10 A1 CD 11 00 0D 08 00 45 00
Destination MAC Source MAC Layer3 Protocol Ver Len DiffServ

0x20h: 01 AF
Total Length
00 00
Identification
00
Flags
00 40
TTL
11
L4 Prot.
79 3F
Checksum
00 00
Source IP
00 0D FF FF C Other Tables and Figures
0x30h: FF FF 00 44 00 43 01 9B CA 88
Dest IP Source Port Dest Port Data Length Checksum

(b) Frame with UDP packet. Figure 14 shows the field type accuracy results of models
trained on programs with different compiler optimization lev-
Figure 12: Frame structure definition.
els. The detailed explanation is in Question 1 of §5.1.2.

To answer the first question, we investigate how many valu-

able seeds bring new structures. We consider the seeds valu-
able if they can reach more BBs (e.g., 3%) than others. We
run AIFORE to fuzz tcpdump for 5 hours and investigate how (a) O0 Performance (b) O1 Performance
many new structures are produced. The result is shown in
Table 9. From the result, we have 30 valuable seeds in total,
and for each of the seeds, if it brings new structures, we then
count the number. For example, there are 11 seeds bring new
format variants of Layer 2 data. In summary, there are 20
valuable seeds that produce new structures out of 30.
(c) O2 Performance (d) O3 Performance
To further understand the performance of the power
scheduling in AIFORE, we also record the execution times for
each of the new structures during fuzzing. We have two groups
of experiments. The only difference is that the first group is
equipped with the power scheduling algorithm, while the other
is not. We feed tcpdump with the same seeds, and we fuzz
them for 3 hours with AIFORE. The result is shown in Fig- (e) Os Performance (f) Mixed Performance
ure 13. As it can be seen, if AIFORE is equipped with power
Figure 14: Field type accuracy of models trained on different
scheduling, some of the fuzzing power can be re-assigned
programs (red/blue lines: accuracy on training/validation set).
to those new structures, which are mutated less. However,
the fuzzer focus most of its power on one structure without
the power scheduling. Noted that there is a structure that is
mutated with large execution times in both of the groups. The
reason is that the power scheduling starts to work after the
fuzzer gets to run for a while, rather than at the beginning.