How to recover from stack-corruption with reversible debugging

Published in

Time Travel Debugging

3 min readNov 2, 2020

If a program overwrites its own program counter register, it is almost impossible to recover using a conventional debugger — without the program counter, the debugger cannot figure out which function the program was running, and so cannot even give any useful information about what is on the stack or where the code was immediately before the stack was corrupted. This makes debugging pretty much impossible.

With a reverse debugger however, recovery is almost comically simple. You can simply do:

reverse-step

- to rewind one instruction, and the state of the program will move back to the instruction that corrupted the program counter, allowing you to see what’s gone wrong. This will also allow the debugger to know what function was running and so be able to interpret the stack and display it to you in a useful way. You can replay your code and subsequently find the issue in order to then debug and fix it quickly.

For example, in this program, the function foo overwrites its stack with zeros and then attempts to return, which results in the program counter registers being set to zero.

#include <strings.h> static void 
foo( void) 
{ 
    int b[1]; 
    bzero( b, 100); /* Overwrite our own stack. */ 
    return; 
} int 
main( void) 
{ 
     foo(); 
     return 0; 
}

The program crashes when run:

> gcc -g foo.c 
> ./a.out 
Segmentation fault (core dumped)

Looking at the core file with gdb doesn’t give us much information because the program counter register has been trashed, so there’s no usable backtrace:

> gdb -q a.out core 
Reading symbols from .../a.out...done. 
[New LWP 30704] Core was generated by `./a.out'. 
Program terminated with signal 11, Segmentation fault. 
#0 0x0000000000000000 in ?? () 
(gdb) backtrace 
#0 0x0000000000000000 in ?? () 
#1 0x0000000000000000 in ?? () 
(gdb) info reg pc pc 
0x0 0x0 
(gdb)

Running under gdb doesn’t help either — there’s still no backtrace after the program crashes:

> gdb -q a.out 
Reading symbols from .../a.out...done. 
(gdb) run 
Starting program: .../a.out Program received signal SIGSEGV, Segmentation fault. 0x0000000000000000 in ?? () 
(gdb) backtrace 
#0 0x0000000000000000 in ?? () 
#1 0x0000000000000000 in ?? () 
(gdb) info reg pc pc 
0x0 0x0 
(gdb)

However, with UDB (formerly known as UndoDB), we can quickly figure out what has gone wrong:

> udb a.out 
UndoDB reversible debugger 6.1.4 (build 25). Copyright 2020 Undo Ltd. 
Reading symbols from a.out... 
Using GNU GDB 9.2: 
  Copyright (C) 2020 Free Software Foundation, Inc. 
  License GPLv3+: GNU GPL version 3 or later <http://gnu.org  /licenses/gpl.html> 
  This is free software: you are free to change and redistribute it.     
  There is NO WARRANTY, to the extent permitted by law. 
  Type "show copying" and "show warranty" for details. For help, type "help". 
For quick-start help on UndoDB, type "help udb". 
(udb) run 
Remote debugging using :51747 
Starting program: /home/chris/Working/foo/a.out 
Program received signal SIGSEGV, Segmentation fault. 0x0000000000000000 in ?? () 
(udb) backtrace 
#0 0x0000000000000000 in ?? () 
#1 0x0000000000000000 in ?? () 
(udb) info reg pc
 pc 0x0 0x0

Do the reverse-stepi trick to recover from the program counter corruption:

(udb) reverse-stepi 
0x0000555555555171 9 } 
(udb) backtrace 
#0 0x0000555555555171 in foo () at foo.c:9 
#1 0x0000000000000000 in ?? ()

Now we know where we are, we can step back and figure out what went wrong:

(udb) reverse-next 
8 return; 
(udb) backtrace 
#0 foo () at foo.c:8 
#1 0x0000000000000000 in ?? () 
(udb) reverse-next 
7     bzero( b, 100); /* Overwrite our own stack. */ 
(udb) backtrace 
#0 foo () at foo.c:7 
#1 0x000055555555517f in main () at foo.c:14 
(udb)

So UDB has enabled us to figure out exactly what has gone wrong in seconds.

We hope you found this example useful. And if you’re interested in trying UDB, help yourself to a free trial.

Also visit the UDB technical documentation pages.

RELATED CONTENT

You may also be interested in reading this post on Why it’s time to debug different with time travel debugging.

Debug Different with UDB the more powerful GDB alternative that lets you debug backward and forward with great performance.

Originally published at https://undo.io.

How to recover from stack-corruption with reversible debugging

Written by Undo Bytes