Working with pdb in Python

Yep, this is another introductory-level article about pdb— sue me! The goal is to teach the essentials (and to give a few helpful tips) to those who don’t feel like reading the docs (or get heebie-jeebies when thinking about the console). It is intended for Python 3.

You can watch recorded demonstrations for selected commands on asciinema (links are in the text).

Why bother?

Skip to Getting started if you get bored easily.

As with every console tool, pdb may seem a bit intimidating. It doesn’t have a nice GUI to hold your hand, and it can feel very cryptic at times.

What’s this all about?

However, you probably shouldn’t use print to debug your programs, and if you do, it won’t hurt to know what to do when good ol’ print doesn’t cut it. So, unless you can spare a few bucks on a fancy IDE (like PyCharm) there is no avoiding it — you’re going to need pdb sooner or later. However, that’s not necessarily a bad thing; knowledge of the “lightweight” solution can turn out to be very handy. First of all, pdb is textual, and in the long run, typing is a lot faster than clicking. Secondly, it’s optimized for speed — minimal, textual interface doesn’t get in your way. This makes for a slightly steep learning curve, but with a bit of practice, you will start saving on those precious keystrokes in no time. Finally, pdb is a standard module, which means that — regardless of a Python distribution — it is always available (it’s also free and open source, so no vendor locking — yay!). You just need to import it:

import pdb

In CPython, reference Python implementation, the source code for standard modules resides in the Lib catalog. Therefore, the pdb’s source code can be found in Lib/pdb.py. If you have an evening to spare, I recommend taking a look at the source code. There is no better way to understand software than by reading its code. It’s very short too — last time I checked the main file was around 1700 lines.

Lib/pdb.py

A bit of jargon

pdb is a debugger, which means that it allows you to “spy” on a program while it executes — by inspecting or modifying its runtime — or to see what the program did when it crashed. The latter technique is called post-mortem debugging.

pdb offers four basic features to help to find bugs:

• Execute the program step by step, under the direct control and supervision of the programmer (which is called single-stepping or program animation).
• Make the program stop on specified lines (so-called breakpoints). pdb also supports conditional breaking, i.e., breakpoints with corresponding Boolean expressions which determine whether debugger should stop upon reaching the breakpoint.
• Examine why or what had happened, when the program had stopped (via symbol resolver or expression interpreter).
• Change the runtime of the program, by modifying the variables, to experiment with the effects it has on the program behavior.

The program might be executing on the same machine as pdb, on another machine (e.g., with remote-pdb), might be called by the programmer, or third-party (e.g., when debugging with pytest).

*There is also this thing called reverse debugging, but we will talk about it later.

Getting started

With the introduction out of the way, it’s time to get started with pdb. There are a couple of options to run it.

Here is the documentation for the commands. It’s good to keep it at hand, as we go through the tutorial.

Starting from the script

Firstly, because the debugger is included in Python’s standard library, we can import it the same way we would any library. Then, we can call pdb.set_trace() to hard-code a breakpoint (place in which program should halt the execution and yield the control to the programmer). The typical way to do it is:

import pdb; pdb.set_trace()

However, it implies editing the source code itself. This is invasive and may be considered awkward, but for most of the time, it’s good enough. I highly recommend adding a shortcut to your text-editor for inserting the setup line.

Starting pdb via set_trace

New in version 3.7: You can use built-in breakpoint() instead of import pdb; pdb.set_trace().

Starting from pdb

Secondly, we can reverse the flow of control and run pdb before a single line of code is executed. When invoking Python interpreter with the module option (of the form -m <module-name>), the specified module (given it is located on the Python module path) is executed as a script. By calling python -m pdb <script>, then, we call pdb as a script to debug other scripts. This will result in breaking on the first line in the file:

$ python3 -m pdb script.py
> /home/siegmeyer/script.py(1)<module>()
-> x = 1 # First line in the file.
(Pdb)

Here <script.py>(1)<module>() means that we are at the first line in the file script.py and no code has been executed yet. If you’re executing from a string, the first part will read <string> (e.g., when starting via pdb.run).

Starting pdb as a script

Oh, I almost forgot — to exit the debugger type quit or exit. If you’ve started with set_trace , quit/exit will raise bdb.BdbQuit exception, and crash the program (it will not be resumed).

Side-note: pdb offers three functions (provided by the library, not the prompt) — run, runeval and runcall which allow you to execute a statement, expression and a function under the debugger control, respectively. One can use them directly from the code or Python interpreter. In my estimation they are pretty much useless, so I’m not covering them. Here is an example usage (straight out of the documentation):

>>> import pdb
>>> import mymodule
>>> pdb.run('mymodule.test()')
> <string>(0)?()
(Pdb) continue
> <string>(1)?()
(Pdb) continue
NameError: 'spam'
> <string>(1)?()
(Pdb)

Getting help

The first and the last thing one needs when learning a new tool is a good help-system. Like every decent piece of software, pdb has a built-in one. We can access it via h(elp) command. To get help on help type: h help:

Without arguments, h(elp) prints available commands (see above).

Shortening names

Please note the brackets around the last part of the command. The brackets indicate that the rest of the word is optional. This can save up a lot of keystrokes. However, you need to remember that commands take precedence over names. In effect, short cryptic variable like b will issue a command (break in this case) instead. Of course, this should not be a problem if your naming hygiene is well-maintained.

Command signatures

help is here to tell us about commands; but how can we know which commands accept which arguments? What about commands that accept multiple arguments in different forms? What if they are optional? To specify that pdb is using quasi-EBNF (it’s not really it, but somewhat similar).

In short, arguments to commands must be separated by whitespace. Optional arguments are enclosed in square brackets: []. Alternatives in the command syntax are enclosed by parenthesis and separated by a vertical bar:(|).

Let’s look at the example (nevermind the actual command; we will focus on the notation for now):

What this means is that:

• break can be called with or without any argument — [] means optional.
• break can be given [filename:]lineno or function —( | ) means alternative (here, again, first argument form has an optional part).
• break always accepts zero or more conditions, separated by a comma.

Getting things done

Here are the absolute basics.

Inspection

Debugging is all about spying on a running program. For that, we need to able to inspect variables. To evaluate a name, just type it in. If the naming is poor, and the variable clashes with pdb commands, you can use p(rint) to explicitly print it. For example:

(Pdb) p a_list
['a', 'list', 'of', 'strings']

There is also pp (for pretty print) which takes care of a bit more sane formatting in some cases — for example when printing lists (however, to be honest, I do not remember using it once).

If you want to quickly print all the arguments passed to the currently inspected function, args is your friend:

(Pdb) list
  1     def foo(spam, ham, eggs)
  2         import pdb
  3  ->     pdb.set_trace()
  4
  5
  6 foo(1, 2, 3)
[EOF]
(Pdb) args
spam = 1
ham = 2
eggs = 3

*list displays source code. We will talk about it in more depth a bit later.

Inspecting variables

Interpretation

Before we go into topics such as single-stepping or breakpoints, let’s cover features not specific to debuggers.pdb supports expression interpretation (with some restrictions), so any valid Python expression is executable in pdb’s prompt. This allows us to modify the runtime (e.g., by introducing variables or calling functions) and inspecting what’s already available. Please note, however, the usage of the word “expression”. pdb is not a full interpreter, and so multiline statements are out the window. This includes things like class or function declarations (lambdas are ok because they are expressions).

(Pdb) 1 / 2 * 3
1.5
(Pdb) foo = lambda bar: bar * 2
(Pdb) foo('bzz')
bzzbzz

If for some reason pdb doesn't recognize command as a valid Python expression, you can explicitly execute it as such using the bang ! operator:

(Pdb) !x, y = 1, 2
(Pdb) !x, y = y, x + y

*Note the lack of the whitespace after the exclamation mark.

Navigation

Next, we need to be able to navigate the source code. For that, we will use list. Without arguments, list displays 11 lines around the current position in the file or continues going through the previous listing.

Seems straightforward, except for that last part. It turns out that default list behavior is somewhat counterintuitive — list displays code at the pointer which is shifted down every time the command is called. When executed for the first time, the pointer is taken to be the current position.

Therefore, if we keep repeating the command, we keep going down through the code. To list the same chunk of code, use l . (there is another fix for this which we will discuss later). You can also specify lines to print, for example list 9 11. Since Python 3.2 we also have ll (or longlist), which displays the whole source code for the current function or a lexical block. When debugging module-level code, this will print the entire file!

Next up, we have n(ext) — execute until the next line is reached or the function returns. This is self-explanatory. However, one big gotcha is that next refers to the next logical line. This means that if we are at the end of the loop body, we will jump to its beginning (given that loop condition holds).

Stepping with next

Stepping through the loop with n(ext) gets tiresome pretty quickly. Luckily, we have unt(il) — without argument, it continues the execution until the line with a greater number than the current one is reached. If a line number is specified, continue until that line. This means that when at the end of the loop body, until will not loop around, but instead break out of the loop.

Stepping in and out of the loop with “until”

If you want to break out of the function, use return. It allows following the execution and breaking at the last (logical) statement in the function. If there are multiple return statements, return will stop at the one determined by the control flow. If you want to check the value of the last returned function, use retval.

Skipping the function body with “return”

To continue execution of the program (i.e., to give up the control and effectively end the debugging), use continue.

Pro-tip: If you started your session via set_trace, and the setup line is placed in a loop (or any block of code being called repeatedly), you might easily find yourself caught in a seemingly endless spiral of pdb's prompts. In such case continue won’t help you (since it can only refer to the current session, not the future ones). To break out of it, use this hack: pdb.set_trace = None

Jumping around

Reversing time and stepping back through program state is called reverse debugging. It’s a very advanced technique and pdb does not support it at the time (although there are some other experimental implementations for Python). Nonetheless, in order to jump back and forth between/through/around executable lines, without resetting the program state, we can use jump. To jump forth, simply type jump specifying the next line that should be executed. Please note that all the lines between the current and the target are not going to be executed. To jump back, use the same format. This time we will move up to line 100. Please note that every line between the current one and the target (which has smaller line number this time around) still remains executed.

Moving up and down with “jump”

To summarize:

• list displays code;
• next continues execution until the next logical step;
• until continues execution until the next physical line;
• continue continues execution indefinitely;
• return jumps to the end of the current function;
• jump allows moving up and down along the executable lines without changing the program state.

Breakpoints

Before we can really start getting things done, we need one last component — dynamic breakpoints. Until now, we’ve used set_trace to specify where debugger should stop the execution. With break we can create breakpoints dynamically, during an active session. Let’s look at the signature one more time:

That’s a mouthful, so let’s unpack it. Simplest usage (what you will do most of the time) is just ordering pdb to break at a line. To set a breakpoint, type break <lineno>.

Additionally, we can specify a filename in which interesting line is located by prepending it to the invocation:

(Pdb) break some_file:123

If you don’t feel like counting lines, you can use function name:

(Pdb) break some_function

In such a case, the debugger will stop at the first line of the function body.

To clear a breakpoint, use clear lineno (clear without any arguments will delete all breakpoints).

To list all existing breakpoints, simply type break (without any arguments):

(Pdb) break
Num Type         Disp Enb   Where
1   breakpoint   keep yes   /script.py:3
2   breakpoint   keep yes   /script.py:5

Conditional breakpoints

If you’re interested only in certain conditions when setting up a breakpoint (i.e., conditions under which the debugger should stop), you can specify them as the last argument, condition:

(Pdb) b 15, x == 7

The above command says that the debugger should stop whenever line 15 is reached and x is equal to 7. In general, all valid expressions are allowed, and all expressions are to be evaluated in the scope in which the breakpoint was defined.

Setting up breakpoints

Advanced topics

Post-mortem debugging

Post-mortem debugging is a technique which allows us to debug a crashed program, at the time it died, using its traceback object. Naturally, because it is no longer running, we cannot step through it and are restricted to inspecting the last state of its execution context (which is represented by the traceback).

A traceback is a stack trace (a report on the state of the call stack) from the point of an exception handler (code catching the error) down the call chain to the origin of the error (where the exception was thrown).

The usual way of entering post-mortem is via pdb.pm() or pdb.post_mortem(). Those methods are available when sys.last_traceback is not null:

>>> import script
Traceback (most recent call last):
  File "script.py", line 1, in <module>
    0/0
ZeroDivisionError: division by zero
>>> pdb.pm()
> <...>/script.py(1)<module>()
-> 0/0
(Pdb)

The catch is, this will only work when Python interpreter is handling the error. (so that last_traceback is properly set up). The following code:

try:
    0/0
except:
    import pdb; pdb.pm()

will throw an error:

Traceback (most recent call last):
  File "exc.py", line 2, in <module>
    0/0
ZeroDivisionError: division by zero
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "exc.py", line 4, in <module>
    import pdb; pdb.pm()
  File "/usr/lib/python3.7/pdb.py", line 1625, in pm
    post_mortem(sys.last_traceback)
AttributeError: module 'sys' has no attribute 'last_traceback'

Of course, it doesn’t make much sense to enter pm in the except clause anyway.

Additionally, you can call post_mortem with explicit traceback:

pdb.post_mortem(exception_obj.__traceback__)

Traversing the stack

So far we’ve only seen navigation around the code and stepping inside functions. However, pdb allows traversing the call stack (chain of function calls in the running program). This is very handy when you want to know what function called what, and when. It is achieved with three commands: where, up and down.

• where — print the stack trace, with the most recent frame at the bottom.
• up — move to the previous frame in the stack trace.
• down — move to next frame in the stack trace (if applicable).

Moving up and down the stack trace with “up” and “down”

Remote debugging

If you are trying to debug a program running on a distant machine (or one that you have no direct access to), it is often useful to use remote debugging. pdb does not support it out-of-the-box, but luckily we have remote-pdb package. Setting up a remote session is very simple:

>>> import remote_pdb as rpdb                                                                                                                                                                                      
>>> rpdb.set_trace('0.0.0.0', 46485)                                                                                                                                                                                               
CRITICAL:root:RemotePdb session open at 0.0.0.0:46485, waiting for connection ...                                                                                                                                
RemotePdb session open at 0.0.0.0:46485, waiting for connection ...

As you can see, this opens up a connection on the displayed port (46485). We can connect to it using Telnet:

telnet 0.0.0.0 46485
Trying 0.0.0.0...
Connected to 0.0.0.0.
Escape character is '^]'.
--Return--
> <stdin>(1)<module>()->None
(Pdb)

or, from another host:

telnet 10.3.39.124 46485

Configuration

pdb can be configured via .pdbrc file:

If a file .pdbrc exists in the user’s home directory or in the current directory, it is read in and executed as if it had been typed at the debugger prompt. This is particularly useful for aliases. If both files exist, the one in the home directory is read first and aliases defined there can be overridden by the local file.

For details, consult the documentation.

Extensions

As with most open-source software, pdb has community extensions. Their description is beyond the scope of this article, but take a look at ipdb and pdb++. One of the best additions to look out for is sticky mode.

Summary

In conclusion, we’ve looked at all most-used pdb features. We learned:

• different ways of starting a pdb session;
• how to get help;
• how to read command signatures;
• how to inspect variables;
• how to use pdb as an interpreter;
• how to navigate through the code;
• how to inspect the stack;
• how to use breakpoints;
• what is post-mortem debugging;
• how pdb can be utilized to debug remotely;
• how pdb can be configured.

This should give you a good start in incorporating pdb in your usual Python workflow.

Thank you for reading! You can find me on GitHub or here.