x86 Assembly

Language
Debugging Survival Guide
(x86-DSG)
Divyanand Kutagulla
Objective
 Understand common IA32 assembly
language constructs used by VS.NET while
it generates code targeting x86 CPUs.
Topic Scope
 x86 architecture registers
 IA32 instruction set
 Used in both Intel and AMD CPUs
 Instruction types covered
 Basic Integer type instructions
 No Floating point, MMX/3DNow! instructions

 Illustrate VS.NET code generation using these

instructions
Contents…
 CPU Registers
 IA32 Instruction Format
 x86 Assembly Instructions
 Function Calling Conventions
 Code generation examples using above
instructions
CPU Registers
 General purpose registers
 Segment registers
 Instruction Pointer
 Flags register
 Special purpose registers
 Debug registers
 Machine Control registers
General purpose registers
 8 32-bit registers

EAX Integer Function return values

EBX General

ECX Loop counter values

EDX High 32 bits of a 64 bit value

General purpose registers (cont’d)

ESI Source address for memory moves /

compare instructions
EDI Destination address for memory
moves / compare instructions
ESP Stack pointer
EBP Base frame pointer (used to access
function parameters, local variables)
Register accesses
 IA32 supports accessing portions of the CPU
registers: EAX, EBX, ECX and EDX through
special mnemonics.

EAX Full 32 bit access

AX 16 bit access (bits 15:0)
AH High byte access (15:8)
AL Low byte access (7:0)
The EFLAGS Register
 Contains the status and control flags that
are set by the instructions executed.
These indicate the result of the instruction
execution.
The EFLAGS Register flags
Flag VS.NET Intel Notes
Mnemonic Mnemonic

Overflow OV OF
Direction UP DF Indicates the direction of string
processing. 1 means highest address to
lowest. 0 means lowest address to highest
address
Interrupt EI IF Set to 1 if interrupts are enabled. This is
Enable always set to 1 by a user mode debugger
Sign PL SF
Zero ZR ZF
Auxiliary Carry AC AF Indicates a carry/borrow in BCD arithmetic
Parity PE PF
Carry CY CF
IA32 Instruction Format
 General format:
 [prefix] instruction operands
 Prefix used only in String Functions
 Operands represent the direction of operands
 Single
operand instruction: XXX src
 Two operand instruction :XXX dest src
 XXX represents the instruction opcode
 src & dest represent the source and destination operands
respectively
IA32 Instruction Format (cont’d)
 Source operands can be:
 Register/Memory reference/Immediate value
 Destination operands can be:
 Register/Memory reference
 Note:
 The Intel CPU does NOT allow both source
and destination operands to be memory
references
Memory References
 Same as a C/C++ pointer access
 Pointer operands appear within square
brackets e.g. Contents of
memory location
 MOV EAX, [0x00040222h] 0x00040222
 Can also have register names instead of hex
addresses e.g. MOV EAX, [EBX]
Refers to the contents
of register EBX
Memory References (cont’d)
 Control the size of memory accessed by
preceding the memory reference with a
size:
 BYTE PTR: byte access
 WORD PTR two byte access
 DWORD PTR four byte access
E.g. MOV EAX, BYTE PTR [0x00001234]
Memory References (cont’d)
 Invalid Memory Accesses
 Accessing illegal memory: CPU generates a
general protection fault: (GPF)
 Access a memory location that does not exist:
CPU generates a page fault
The x86 Stack
 Starts from high memory and “grows”
towards low memory
 Used by the CPU to:
 Store return addresses
 Pass parameters to functions
 Store local variables
 ESP indicates current top of stack
VS.NET Inline Assembler
 Can embed x86 assembly language in C/C++ source
E.g.
void foo (void)
{
__asm
{
//x86 assembly
}

//each instruction can occur separately on each line

preceded //by the __asm directive like below:
__asm //x86 assembly
__asm //x86 assembly
}
x86 Assembly Instructions

“Instructions You Need to Know”

-John Robbins
NOP
 NOP – No operation
 Takes no arguments
 Commonly used by the compiler as padding INSIDE
functions so as keep them properly aligned
e.g.
void NOPFuncTwo ( void )
{
__asm
{
NOP
NOP
}
}
Stack Manipulation Instructions
 PUSH <argument>
 Pushes a word/double word on the stack
 argument can be a register/memory
location/immediate value (hardcoded number)
 POP <argument>
 Pop a word/double word from the stack
 Note:
 PUSH decrements the ESP while POP increments the
ESP
Stack Manipulation Instructions
(cont’d)
 PUSHAD
 Push (save) all general purpose registers
 POPAD
 Pop (restore) all general purpose registers
 Avoids long sequence of PUSH/POP
instructions to save/restore the registers
 Used mainly in system code
Example
void SwapRegistersWithPushAndPop ( void )
{
__asm
{
//Swap the EAX and EBX values. The sequence
//gives you an idea of how it could be done.
PUSH EAX
PUSH EBX

POP EAX
POP EBX
}
}

(Example taken from Debugging Applications by John

Robbins)
Arithmetic Instructions
 ADD <dest> <src>
 SUB <dest> <src>
 The result is stored in the destination and
the original value in destination is
overwritten.
Arithmetic Instructions (cont’d)
 DIV/MUL: Unsigned Division/Multiplication
 Uses the EDX register to store the high bytes
of double-word and higher (64 bit) results.
EAX stores the low bytes
 IDIV/IMUL: Signed Division/Multiplication
 IMUL sometimes has 3 operands:
 IMUL <dest> <src1> <src2>
CPU Breakpoint Instruction
 INT 3 (OxCC)
 Mainly used as padding by compilers as
padding BETWEEN functions in a file
 Keeps portable executable sections aligned on
the linker’s /ALIGN switch (defaults to 4 KB)
 Allows for misaligned memory accesses to be
easily detected
 Executing this instruction stops CPU execution and
automatically invokes the debugger
CPU State Instructions
 ENTER: save CPU state
 LEAVE: restore CPU state
 Mainly used in interrupt processing
Pointer Manipulation Instructions

 LEA: Load Effective Address

 LEA <dest> <src>
 Loads the destination register with the
address of the source operand
 Used to emulate pointer access
Example
//...
int * pInt ;
int iVal ;

// The following instruction sequence is identical to the

// C code: pInt = &iVal ;
__asm
{
LEA EAX , iVal
MOV [pInt] , EAX
}
//..
(Example from Debugging Applications by John Robbins)
Another example
//....
char szBuff [ MAX_PATH ] ;

// Another example of accessing a pointer through LEA.

// This is identical to the C code:
// GetWindowsDirectory ( szBuff , MAX_PATH ) ;
__asm
{
PUSH 104h // Push MAX_PATH as the second parameter.
LEA ECX, szBuff // Get the address of szBuff.
PUSH ECX // Push the address of szBuff as the first
// parameter.
CALL DWORD PTR [GetWindowsDirectory]
}
//....
(Example from Debugging Applications by John Robbins)
Function Call Instruction
 CALL <argument>
 argument can be a register \ memory
reference \ parameter \ global offset
 Automatically pushes the return address on
the stack and decrements ESP
Example
__asm
{
// Call a function inside this file.
CALL NOPFuncOne

// If symbols are loaded, the Disassembly window

// will show:
// CALL NOPFuncOne (00401000)

// If symbols are NOT loaded, the Disassembly

// window shows:
// CALL 00401000
}

(Example from Debugging Applications by John Robbins)

Another Example
__asm
{
// Call the imported function, GetLastError, which
// takes no parameters. EAX will hold the return
// value. This is a call through the IAT so it is a
// call through a pointer.

CALL DWORD PTR [GetLastError]

// If symbols are loaded, the Disassembly window will

// show:
// CALL DWORD PTR [__imp__GetLastError@0 (00402004)]

// If symbols are NOT loaded, the Disassembly window

// shows:
// CALL DWORD PTR [00402004]
}
(Example from Debugging Applications by John Robbins)
Yet another example…
//....
__asm
Set up the stack before
{
calling the
PUSH 104h
LEA ECX, szBuff GetWindowsDirectory
PUSH ECX function

CALL DWORD PTR [GetWindowsDirectory]

// If symbols are loaded, the Disassembly window
// will show:
// CALL DWORD PTR [__imp__GetWindowsDirectory@4 (00401000)]

// If symbols are NOT loaded, the Disassembly

// window shows:
// CALL DWORD PTR [00401000]
}
//....

(modified e.g. from Debugging Applications by John Robbins)

Function Return Instruction
 RET <optional argument>
 Argument says how many bytes to pop off the
stack (to account for parameters passed to
the function)
 Pops the callers return address off the top of
stack and put it in the instruction pointer
 Return address validity is NOT checked!!!:
potential security hazard
Data Manipulation Instructions
 AND <dest> <src> : logical AND
 OR <dest> <src> : logical OR
 NOT <arg>: logical NOT
 One’s complement negation (Bit Flipping)
 NEG <arg>:
 Two’s complement negation
Data Manipulation Instructions
(cont’d)
 XOR <dest> <src>: logical XOR
 Fastest way to zero out a register!!!
 INC/DEC <arg> : increment/decrement
 Often used in speed optimized code (executes
in single clock cycle)
 Directly maps to the C++ operators:
 ++ : INC
 -- : DEC
Data Manipulation Instructions
(cont’d)
 SHL/SHR <arg> : shift left and Shift right
 SHL: fastest way to multiply by 2 (C++: <<)
 SHR: fastest way to divide by 2 (C++: >>)
 MOVZX <dest> <src>: move with zero
extend
 MOVSX <dest> <src>: Move with sign
extend
Compare Instruction
 CMP <arg1> <arg2>: compare arg1 and
arg2 and set the appropriate conditional
flags in the EFLAGS register
 The conditional flags can viewed in the
Register window in VS.NET
(Debug->Windows->Registers)
Test Instruction
 TEST <arg1> <arg2> : Bitwise AND of
both arguments and sets the appropriate
conditional flags
 PL (SF)
 ZR (ZF)
 PE (PF)
Jump Instructions
 JE <label> : Jump if equal
 JL <label> : Jump if less than
 JG <label> : Jump if greater than
 JNE <label> : Jump if not equal to
 JGE <label> : Jump if greater than or equal to
 JLE <label> : Jump if Less than or equal to
Jump Instructions (Cont’d)
 Always follow a CMP/TEST instruction
 JMP condition is always the opposite of
the original conditional
Loop Instruction
 Loop <label>: Decrement ECX and if ECX
isn’t 0, go and re-execute the code
sequence marked by <label>
 Rarely used by the VS.NET compiler
CPU Atomic Operation Prefix
 LOCK
 Prefix to any of the IA32 instructions
 Directs the CPU that the memory access by the
prefixed instruction will be an atomic operation
 Other CPUs in the system can’t access the memory
 Can be used in conjunction with the CMP and TEST to
implement semaphores
 Used to multithreaded code running on
multiprocessor machines
Function Calling Conventions
 Specifies how parameters are passed to a
function
 Passed in through stack/registers
 Specifies how stack cleanup occurs upon
function return
 Who performs the cleanup, the caller or the
callee?
(Supplied handout has table summarizing the various calling conventions)
Instruction usage examples
 Discuss usage of the previously mentioned
instructions
 Generated by VS.NET during compilation
 Examples discussed:
 Function Entry and Exit
 Global variable, Local variable and Function
parameter access
Function Entry (Prolog)
 Compiler generated at the beginning of a
function (before the actual processing
code of the function)
 This code sets up the stack for access to
the function’s local variables and
parameters (the Stack Frame)
Prolog Example
__asm
{
// Standard prolog setup
PUSH EBP // Save the stack frame register.
MOV EBP, ESP // Set the local function stack //
frame to ESP.
SUB ESP , 20h // Make room on the stack for 0x20
// bytes of local variables. The // SUB instruction
appears only if // the function has local // variables.
}
(Example from Debugging Applications by John Robbins)
Function Exit (Epilog)
 Compiler generated (after the end of the
processing code of the function)
 Undoes the operations of the prolog
 Stack cleanup can be performed here
e.g.
__asm
{
// Standard epilog teardown
MOV ESP , EBP // Restore the stack value.
POP EBP // Restore the saved stack frame register.
}
(Example from Debugging Applications by John Robbins)
Global Variable Access
 Memory References with a fixed address
e.g.

int g_iVal = 0 ;
void AccessGlobalMemory ( void )
{
__asm
{
// Set the global variable to 48,059.
MOV g_iVal , 0BBBBh

// If symbols are loaded, the Disassembly window will show:

// MOV DWORD PTR [g_iVal (4030B4h)],0BBBBh

// If symbols are NOT loaded, the Disassembly window shows:

// MOV DWORD PTR [4030B4h],0BBBBh
}
}
(Example from Debugging Applications by John Robbins)
Function Parameter Access
 Function Parameters are positive offsets
from EBP (stack frame register)
 Callerpushes the parameters before calling
the function
 Rule of thumb: “Parameters are positive”
-Robbins
Example
void AccessParameter ( int iParam )
{
__asm
{
// Move the value if iParam into EAX.
MOV EAX , iParam

// If symbols are loaded, the Disassembly window will show:

// MOV EAX,DWORD PTR [iParam]

// If symbols are NOT loaded, the Disassembly window shows:

// MOV EAX,DWORD PTR [EBP+8]
}
}

Caller code pushes iParam onto the stack before calling the function code above:
// AccessParameter(0x42);
push 66 ; 00000042H
call ?AccessParameter@@YAXH@Z ; AccessParameter
Local Variable Access
 Local Variables occur as negative offsets from the EBP
(stack frame pointer) register
e.g.
void AccessLocalVariable ( void ) {
int iLocal ;
__asm {
// Set the local variable to 23.
MOV iLocal , 017h

// If symbols are loaded, the Disassembly window will show:

// MOV DWORD PTR [iLocal],017h

// If symbols are NOT loaded, the Disassembly window shows:

// MOV [EBP-4],017h
}
}
(Example from Debugging Applications By John Robbins)