Function

 Conventions  

!  Standard  Entry  Sequence  (cdecl)  

"  Save  the  old  base  pointer  
"  Set  the  new  stack  base  pointer  
"  Allocate  space  for  variables  
__function:
push ebp ; 55
mov ebp, esp ; 8BEC
sub esp, x ; Not always present

__function:
enter ; C8
sub esp, x ; Not always present
2
Function  Conventions  

!  Standard  Exit  Sequence  (cdecl)  

"  Reload  old  stack  pointer  
"  Reload  old  stack  base  
"  Deallocate  space  for  variables  
...
mov esp, ebp ; 8BE5
pop ebp ; 5D
ret ; C3 near, CB far

...
leave ; C9
ret ; C3 near, CB far
3
Function  Call  Conventions  

!  cdecl  
"  Used  by  GCC  and  GNU  libraries  
!  stdcall  
"  Used  by  Win32  API  
"  Sometimes  incorrectly  called  “pascal”  
!  fastcall  
"  Many  different  implementations  
"  Not  standardized  

4
Function  Call  Conventions  

!  cdecl  
"  Parameters  pushed  right  to  left  
"  EAX,  ECX,  EDX  not  preserved  
"  Return  values  are  returned  in  EAX  
!  Floating  point  returns  in  ST0  
"  Caller  performs  clean-­‐up  
!  stdcall  
"  Same  as  cdecl,  except  callee  cleans-­‐up  
!  RET imm  is  a  sign  of  this  
!  fastcall  
"  One  or  more  parameters  passed  in  registers  
"  MS  VC++,  GCC  
!  First  arg  #  ECX,  second  arg  #  EDX,  remainder  right  #  left  

5
cdecl  Function  Call  
Convention  
!  Push  Parameters  on  Stack  
!  Call  the  Function  
!  Save  and  Update  EBP  
!  Save  Registers  that  Will  Be  Overwritten  
!  Allocate  Local  Variables  
!  Execute  Function  
!  Release  Local  Storage  

6
cdecl  Function  Call  
Convention  
!  Restore  Saved  Registers  
!  Restore  EBP  
!  Return  
!  Clean  Up  Parameters  

7
stdcall  Function  Call  
Convention  
!  Push  Parameters  on  Stack  
!  Call  the  Function  
!  Save  and  Update  EBP  
!  Save  Registers  that  Will  Be  Overwritten  
!  Allocate  Local  Variables  
!  Execute  Function  
!  Release  Local  Storage  

8
stdcall  Function  Call  
Convention  
!  Restore  Saved  Registers  
!  Restore  EBP  
!  Clean  Up  Parameters  
!  Return  

9
Function  Call  Conventions  

!  Others  
"  pascal  
!  Parameters  pushed  left  to  right  
!  Windows  3.*  
"  syscall  
!  Parameter  size  passed  in  AL  
"  safecall  
!  Encapsulated  COM  error  handling  
"  thiscall  
!  Either  caller  or  callee  clean-­‐up  
"  …  

10
Control  Statements  

!  If-­‐Else  
!  Switch  
!  For  
!  While  

11
If-­‐Else  Statement  

12
If-­‐Else  Statement  

13
Switch  Statement  

14
Switch  Statement  

15
For  Statement  

16
For  Statement  

17
While  Statement  

18
While  Statement  

19
Determining  Signed-­‐ness  

!  Signed  and  Unsigned  Variables  

"  Operations  on  signed/unsigned  variables  use  
different  instructions  
"  IMUL/MUL  
"  IDIV/DIV  
"  Jcc  

20
Determining  Signed-­‐ness  

21
Tools  of  the  Trade  

!  Disassembler  
"  Machine  code  to  instructions  
!  Decompiler  
"  Instructions  to  code  (often  to  C  code)    
!  Debugger  
"  Real-­‐time,  step-­‐thru-­‐code  debugging  

22
Disassemblers  

!  Disassemblers  
"  Converts  machine  code  to  instructions  

23
 Decompilers  

!  Decompilers  
"  Attempt  to  convert  instructions  or  byte  codes  to  
higher-­‐level  languages    
"  Good  decompilers  are  implemented  via  p-­‐code  
analysis  
!  Allows  decompiler  code  to  be  applied  to  various  
architectures  as  long  as  a  p-­‐code  translation  exists  

24
Debuggers  

!  Debuggers  
"  Modes  
!  User-­‐mode  
!  Kernel-­‐mode  
"  Common  features  
!  Create/attach  to  a  process  
!  Set/clear  breakpoint  
!  Step  into/over  
!  Trace  into/over  

25
Debuggers  

!  Breakpoints  
"  Software  breakpoints  
!  INT  3h  (\xCC)  
"  Memory  breakpoints  
"  Hardware  breakpoints  
!  Intel  Dr0-­‐Dr7  registers  
!  Traces  
"  Records  instructions  and  execution  contexts  
!  Stepping  
"  Step  into/over  
26
GNU  Debugger  (gdb)  

!  Disassembler,  Debugger  
"  Command-­‐line  
!  Insight  is  a  GUI  wrapper  for  gdb  
"  Not  just  for  Linux  
!  Native  x86  Windows  support  
!  Special  versions  for  various  architectures  

27
GNU  Debugger  (gdb)  
Breakpoint  Tutorial  

28
GNU  Debugger  (gdb)  
Breakpoint  Tutorial  

29
OllyDbg  

!  Disassembler  
!  Debugger  
"  Open  
!  Creates  a  process  with  debug  privileges  
"  Attach  
!  Attach  to  a  process  already  running  
"  Detach  (version  2.*)  
!  Detaches  the  debugger  and  allows  the  process  to  
continue  
"  Terminate  
!  Kills  the  debuggee  

30
OllyDbg  

!  Views  

31
OllyDbg  2.0  

!  Views  

32
OllyDbg  

!  Code  Analysis  
"  Right-­‐click#Analysis#Analyse  code  (Ctrl  +  A)  
"  Static  code  analysis  
!  Argument  labeling  
!  Function  address  name  resolution  
!  Control  logic  labeling  
!  …  

33
OllyDbg  

!  Just-­‐in-­‐time  Debugger  
"  Options  #  Just-­‐in-­‐time  debugging  
"  Runs  Olly  when  a  fatal  error  occurs  
!  Plugins  
"  Great  feature  
"  Well  used  by  the  reverse  engineering  community  

34
OllyDbg  
Debugging  Tutorial  
!  Breakpoints  
"  Set  a  breakpoint  (F2)  

35
OllyDbg  
Debugging  Tutorial  
!  Breakpoints  
"  Resume  execution  (F9)  

36
OllyDbg  
Debugging  Tutorial  
!  Breakpoints  
"  Resume  execution  (F9)  

37
OllyDbg  
Debugging  Tutorial  
!  Breakpoints  
"  Stack  view  

38
OllyDbg  
Debugging  Tutorial  
!  Stepping  
"  Step  into  (F7)  

39
OllyDbg  
Debugging  Tutorial  
!  Stepping  
"  Step  into  (F7)  

40
OllyDbg  
Debugging  Tutorial  
!  Stepping  
"  Step  into  (F7)  

41
OllyDbg  
Debugging  Tutorial  
!  Stepping  
"  Let’s  say  we  step  into  (F7)  

42
OllyDbg  
Debugging  Tutorial  
!  Stepping  
"  Let’s  say  we  step  over  (F8)  

43
OllyDbg  
Assembly  Patching  Tutorial  

44
OllyDbg  
Assembly  Patching  Tutorial  
!  Assembly  Patching  
"  Change  the  jump  from  “invalid”  code  to  “valid”  
code  

45
OllyDbg  
Assembly  Patching  Tutorial  
!  Assembly  Patching  
"  Double-­‐click  on  the  short  jump  

46
OllyDbg  
Assembly  Patching  Tutorial  
!  Assembly  Patching  
"  Change  the  jump  address  

47
OllyDbg  
Assembly  Patching  Tutorial  
!  Assembly  Patching  
"  Hit  assemble  
"  Check  that  the  size  of  the  code  hasn’t  changed  

48
OllyDbg  
Assembly  Patching  Tutorial  
!  Assembly  Patching  
"  View  patches  
!  Click  on  the  “/”  toolbar  button  or  hit  Ctrl+P  
!  Right  click  on  an  entry  and  click  “Follow  in  
Disassembler”  to  return  to  the  disassembler  at  the  
target  address  
!  Note:  this  view  does  not  currently  exist  in  OllyDbg  
2.0    

49
OllyDbg  
Assembly  Patching  Tutorial  
!  Assembly  Patching  
"  Right  click  on  the  patch  
"  “Copy  to  executable”  #  “All  Modifications”  
"  OllyDbg  2.0:  “Edit”  #  “Copy  to  executable”  

50
OllyDbg  
Assembly  Patching  Tutorial  
!  Assembly  Patching  
"  Select  “Copy  all”  (OllyDbg  1.*  only)  

51
OllyDbg  
Assembly  Patching  Tutorial  
!  Assembly  Patching  
"  Right  click  in  the  newly  generated  executable  
"  Select  “Save  file”  

52
OllyDbg  
Assembly  Patching  Tutorial  
!  Assembly  Patching  
"  Save  the  new  executable  

53
IDA  

!  Disassembler,  Decompiler*,  Debugger  

"  Commercial  debugger  
!  With  freeware  and  demo  versions  
"  Now  a  Hex-­‐Rays  product  
!  Formerly  Datarescue  
"  *Decompilers  sold  seperately  (and  is  expensive)  

54
IDA  

55
 IDA  

!  Shortcuts  
"  Run  (F9),  step  into  (F7),  step  over  (F8)  
"  Set/clear  breakpoint  (F2)  
"  Apply  name  to  an  address  (N)  
"  Comment  (:),  repeatable  comment  (;)  
"  Toggle  graph  view/assembly  view  (space)  
"  Jump  to  name/address  (G)  
"  Follow  reference  (enter)  
"  Display/jump  to  cross-­‐references  (X)  
"  Return  to  previous  location  (esc)  
56
 IDA  Patching  

!  Patching  
"  Edit  “cfg/idagui.cfg”  
"  Change  “DISPLAY_PATCH_SUBMENU”  to  “YES”  

57
Hex-­‐Rays  Decompilers  

58
WinDbg  

!  Disassembler,  Debugger  
"  User/kernel-­‐mode  debugger  

59
Questions/Comments?  

60