Assembly functions and stacks

Assembly functions and stacks

Prerequisites :
– Knowledge of stack data structure
– Basics of assembly language

At the end of this tutorial, you should be able better understand how the stack works with different operands in assembly, and their typical use case.

The stack is a place to store temporary data, such as local variables or passing of parameters from one function to another. Keeping in mind that a stack follows a last-in first-out(LIFO) order, lets proceed to see how each operand work with the stack in the scenarios below

Scenario 1 : A typical function prologue

In this scenario, a new stack frame for the current function is created

 push ebp      // save the value of ebp on the stack.
 mov ebp, esp  // copy esp into ebp
 sub esp, 0x10 // size of local variables

Scenario 2 : A typical function epilogue

In this scenario, the stack frame is restored as the function before returning to the caller function

 mov esp, ebp // Copy ebp into esp
 pop ebp      // Restore ebp (earlier we pushed it)
 ret          // Return to caller function

Scenario 3 : A function call with no args

In this scenario, we will unpack the call operand in assembly

call foobar // foobar();

A call operand does two things –

push return_address_after_callee_is_done // esp - 4
jmp foobar

As you can see, the return address is saved so the called function knows where to return to in the program’s execution flow, before actually jumping to it.

ret foobar_caller // esp + 4

A ret instruction does two things

pop eip // return_address_after_callee_is_done is pop into eip; esp + 4
jmp eip 

Sometimes, you will see a ‘ret 0x8’ or ‘ret 8h’

ret 0x8 
a normal ret increases esp by 4, this further adds 8 to esp, so in total esp + 0xC. So if its 'ret 0x4' - it increases by esp by 8

Scenario 4 : A function call with args

In this scenario, we will see how the argument of a function call interacts with the stack.

Take arg1 to be a 4-byte value

  push arg1   // save arg1 on the stack . esp-4 after this operation
  call foobar // foobar(arg1);            esp-4 (see scenario 3)

foobar:
  push ebp
  mov ebp, esp
  // [ebp+0x4] will contain our return address
  // [ebp+0x8] will contain arg 1
  mov eax, [ebp+0x8]   
  pop ebp
  ret 0x4 // earlier esp-4 twice, so here esp+4 twice also

Scenario 5 : A function call with multiple args

In this scenario, we will see how the arguments of a function call interacts with the stack. (note: the calling convention assumed here is _stdcall)

Take arg1, arg2 to be a 4-byte value

push arg2   // save arg2 on the stack. esp-4 after this operation
push arg1   // save arg1 on the stack. esp-4 after this operation
call foobar // foobar(arg1, arg2);     esp-4 (see scenario 3)

foobar:
  push ebp
  mov ebp, esp
  // [ebp+0x4] will contain our return address
  // [ebp+0x8] will contain arg 1
  // [ebp+0xC] will contain arg 4
  mov eax, [ebp+0x8] 
  pop ebp
  ret 0x8 // Earlier esp-4 thrice, so now esp+8+4

CONCLUSION
Of course, there are a million and one ways the stack is used by different operands and in different situation, so this list is definitely not exhaustive. Nonetheless, you should be more familiar with the typical use cases of the stack now