How To: Security-Oriented C Tutorial 0x0F - Functions Part III: The Stack

Security-Oriented C Tutorial 0x0F - Functions Part III: The Stack

Security-Oriented C Tutorial 0x0F - Functions Part III: The Stack

Welcome back to another tutorial on functions. Last time, we looked inside memory to see what functions looked like in the flesh and all that Assembly was pretty nasty. At least you have some exposure to it. This time, we will be examining how functions work with the stack.

Example Code

Let's bring back the code we used in the previous tutorial.

Image via wonderhowto.com

The Return Location

So I've explained a few times about the return. We know that it passes control back to the caller of the function and we also know it can give back a value but we're still unsure about how it knows where it needs to return. Let's compile it and debug to see how it all works.

Compiling and Debugging

First of all, we require breakpoints to pause the program so that we can see what is happening at those times. We break at line 15 before we call the function and at line 7 inside the function. We will then compare the differences in the stack to see if anything changed.

From the previous tutorial, we saw that two values were moved onto the stack during the function analysis of main and as we can see, they're both there. Let's continue into the add function and print the stack again.

Matching the addresses, we can see our variables num1 and num2 again. But what's this in the red? It looks strangely familiar... Oh, I know where that came from! Let's bring the assembler dump image of main from last tutorial.

We have two matching addresses, one in the stack and one inside the main function of the code segment! Back in the introduction to functions, I said It will then proceed to jump execution back to the saved location of the next instruction in the function which called it. Well, now we know its magic trick. During the call of a function, the caller pushes its next instruction location onto the stack so when the callee finishes, ESP will pop (remove from the top of the stack) off all of its data and when the ret instruction is called, execution flow will use that address to find where it needs to go. But hang on... We saw another pair of our variables num1 and num2 again didn't we? The two values just before the return address...

Function Arguments

Back in Tutorial 0x0D, I had explained about functions and their arguments. When a function is called, the caller passes its own variables and/or values into the callee by copying them and placing them on top of the stack so that the callee now has its own variables which is named whatever you named them in the parenthesis of the function definition. In this case, num1 and num2 were copied, pushed (to place on top of the stack) and stored as variables a and b in add.

Debugging

Let's see that stack again...

Right there, in the blue! So we weren't hallucinating, it's really there! Before the return address is pushed, we require the parameters of the callee function (a and b), so as I have already stated a couple of times, the caller pushes its values onto the stack so it has a copy of its own. Notice how I've been bolding the word copy every time I've typed in for the past few tutorials. It is a copy, not the actual original value. If you change the variables in the callee function, it will not change the values in the caller unless you explicitly return and save the value. What's also interesting is in what order the values are pushed. If you take a look, our first parameter is num1 and our second is num2 but on the stack, it's the other way around! The reason for this is so functions (like printf) with something called variable arguments can be supported*. Let's try experimenting.

*From Subroutines Continued: Passing Arguments, Returning Values and Allocating Local Variables, 2015, Prof. Moshovos, A., Computer and Electrical Engineering at the University of Toronto

Example Code

This time, we have given the swap function a void return value, which means it does not return anything. If this is the case, no return call is required. num1 and num2 are passed into the swap function again, just like the add function in the previous code only now, we have set the two values to different numbers. Obviously, since I have already explained this, we know that the values will not change.

Compiling and Running

As expected.

Conclusion

We've almost finished on functions, there are probably going to be two more tutorials on this topic but I will give it a rest now because there are other things we need to know before getting there. Remember to revise the content, try to fully understand what is happening and maybe we can bring up another exploitation method with the buffer overflow vulnerability. Thanks for reading!

dtm.

3 Comments

I'm sorry if I sound like a noob here,

but why is it "int main()" and not "void main()"?
what happens if I put void instead of int?

reason I ask is because in Java, we write
public static void main(String[] args)

Hey there, Voidx!

When you define main in C it is standard to have it return an int. The program is expected to give a return value back to the operating system (or whatever ran your program) so that it knows how the program was terminated whether it was successful or some error occurred. As for Java, I'm not sure of why main is declared as having a void return type. For more information, please refer to this Stack Overflow question.

dtm.

thanks for the reply
in Java, we just don't return anything, but use "System.out.println()" to print out strings/words/sentences

Share Your Thoughts

  • Hot
  • Latest