Finding out Python stack trace via GDB

When working as a system admin I often had a problem where the python application got stuck but was unsure where. To handle this case I created a GDB script that retrieved that backtrace.

These scripts refere to older versions of Python and Linux. It may not work on newer version because of compiler/python struct differences. If somebody needs them updated, I can update the scripts for other os/python version.

Python 2.7 on Ubuntu 14.04

macro define get_function_name(ptr) (*((char**)(*((char**)((char*)ptr + 0x20)) + 0x68)) + 0x30)
macro define get_filename(ptr) (*((char**)(*((char**)((char*)ptr + 0x20)) + 0x60)) + 0x30)
macro define get_firstlineno(ptr) ((int)*((char**)(*((char**)((char*)ptr + 0x20)) + 0x70)))
 
define frame_print
p get_function_name($r15)
p get_filename($r15)
p get_firstlineno($r15)
end
 
define py-bt
python
import re
frame = gdb.execute("bt", False, True)
for id in map(lambda x: re.match("#(\d+).*",x).group(1), filter(lambda x: "PyEval_EvalFrameEx" in x, frame.split("\n"))):
    try:
        print("frame %s" % id)
        gdb.execute("frame %s" % id, False, True)
        gdb.execute("frame_print")
    except Exception as e:
        print(e)
end
end
py-bt
set confirm off
quit

Python 3-5 on Ubuntu 14.04

macro define get_function_name(ptr) (*((char**)(*((char**)((char*)ptr + 0x10)) + 0x58)) + 0x24)
macro define get_filename(ptr) (*((char**)(*((char**)((char*)ptr + 0x10)) + 0x50)) + 0x24)
macro define get_firstlineno(ptr) ((int)*((char**)(*((char**)((char*)ptr + 0x10)) + 0x60)))
 
define frame_print
p get_function_name($r12)
p get_filename($r12)
p get_firstlineno($r12)
end
 
define py-bt
python
import re
frame = gdb.execute("bt", False, True)
for id in map(lambda x: re.match("#(\d+).*",x).group(1), filter(lambda x: "PyEval_EvalFrameEx" in x, frame.split("\n"))):
    try:
        print("frame %s" % id)
        gdb.execute("frame %s" % id, False, True)
        gdb.execute("frame_print")
    except Exception as e:
        print(e)
end
end
py-bt
set confirm off
quit

How it actually works

Basic idea is that Python uses c structs to capture python frames. The exact information about frames is contained in the certain fields of these structs. Each field of a struct is basically a offset from the struct base and this is what I did here. I have converted all pointer field member references to offsets and just loaded the values they pointed to. I basically navigated over the memory until I found out what I needed.

The remaining part was a global variable (i think it is a thread local) which is provided by the Python and can be used to navigate all the frames.