I'm playing around with memory and trying to get a deeper understanding of how things function under the hood, I attempted this about a year ago but I decided to give it another shot.
So the problem:
I'm trying to overflow the buffer and overwrite the return address, my goal first of all (before attempting to execute my shell from the buffer overflow) is to overwrite the return address,
I manage to do this with a python script, I run the program in gdb and specify 500 A's to be printed, this seems to work as I then inspect the registers and the instruction pointer is 0x41414141 meaning it's overwritten it with 'A's
so next my shellcode takes up 53 bytes and I need some padding from what I heard to make sure the address I want to return to does so ( following this video - https://www.youtube.com/watch?v=1S0aBV-Waeo
computerphile ), this padding will be 40 bytes,just an address 10 times.
so it will be 500(this is where the seg fault occurs) - 53 - 40 = 407 no ops,
in other words 407 no ops + 53 bytes of shellcode + 40 bytes of memory addresses = 500 bytes
when I run this in gdb I seem to get a seg fault but this time I get a seg fault and the return address is overwritten with no ops or /x90's.
I can't seem to figure out why the return address is being overwritten with no ops,
here are the the videos illustrating what is actually happening:
1) video of actual error - https://www.youtube.com/watch?v=XO7BmWOTqwU&feature=youtu.be
2) register info - https://www.youtube.com/watch?v=rcHndTwVEUk&feature=youtu.be
if anybody could give me some pointers (no pun) or some help getting to the bottom of this, that would be great, I'm seriously bored and decided to give this a shot again one year later.
*note all done on a virtual machine that I own and this thread is only for educational purposes (in reality a simple buffer overflow attack like this is extremely rare as protections are in place)