Building REPLs for Fun and Profit

Photo by Kevin on Unsplash — Cropped from original

This article has a companion repository that you can clone to try out the examples yourself. Grab it from GitHub!

REPL stands for “Read-Eval-Print Loop”, and it is one of the most useful tools in a software engineer’s arsenal.

In fact, you’re already very familiar with Python REPLs — you interact with one every time you type python (or ipython, or bpython, or ptpython, or…) into your command line:

But, did you know that you can build your very own Python REPL, with just a few lines of code?

Automation for Developers

Suppose you just wrote a client library for a web service, and you want to test it out to make sure it works correctly. What’s the first thing you do?

I bet your answer is, “Drop into a Python shell [aka REPL] and test it out.”

How cool would it be to have a script that initialises a client for you and then drops you straight into that REPL?

Better yet, how awesome would it be to ship your library with its own REPL?

In this article, we’re going to build a REPL template that you can use in any project, and then we’ll enhance it with some extra tricks that will make it super useful for you, your team, and your users.

The REPL Script

Dropping into a REPL from any Python script is actually very simple:

# deep_thought/repl_basic.py
from code import InteractiveConsole
header = "Welcome to REPL! We hope you enjoy your stay!"
footer = "Thanks for visiting the REPL today!"
scope_vars = {"answer": 42}
InteractiveConsole(locals=scope_vars).interact(header, footer)

You can see repl_basic.py in its entirety on GitHub.

Note the locals argument that we passed to InteractiveConsole.__init__(). This defines the local variables that are accessible from within the REPL. You can use this to inject objects into the REPL environment (for example, pre-configured web service client instances… hint, hint).

Anything not defined in locals will not be accessible in the REPL:

> python deep_thought/repl_basic.py
Welcome to REPL! We hope you enjoy your stay!
>>> answer
42
>>> header
Traceback (most recent call last):
File "<console>", line 1, in <module>
NameError: name 'header' is not defined
>>> footer
Traceback (most recent call last):
File "<console>", line 1, in <module>
NameError: name 'footer' is not defined
>>> quit()
Thanks for visiting the REPL today!

What if you use IPython?

IPython is a popular replacement for the Python shell that provides features such as tab-completion, history navigation, and more.

And, it can be embedded easily into your REPL script!

# deep_thought/repl_ipython.py
import IPython
header = "Welcome to REPL! We hope you enjoy your stay!"
footer = "Thanks for visiting the REPL today!"
scope_vars = {"answer": 42}
print(header)
IPython.start_ipython(argv=[], user_ns=scope_vars)
print(footer)

You can see repl_ipython.py in its entirety on GitHub.

Aside from a few things named differently, and the fact that we have to print the banners ourselves, it’s basically the same as dropping into a vanilla Python REPL.

Note one important difference, however: We have to pass argv=[] to IPython.start_ipython(). Otherwise, IPython will try to interpret any command-line arguments that were passed to our script, as if we were executing the ipython command in a terminal!

Here’s what the end result looks like:

> python deep_thought/repl_ipython.py
Welcome to REPL! We hope you enjoy your stay!
In [1]: answer
Out[1]: 42
In [2]: header
 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — -
NameError Traceback (most recent call last)
<ipython-input-2-f9d9cfc153ed> in <module>()
 — → 1 header
NameError: name 'header' is not defined
In [3]: footer
 — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — — -
NameError Traceback (most recent call last)
<ipython-input-3–1d4a2811ae3d> in <module>()
 — → 1 footer
NameError: name 'footer' is not defined
In [4]: quit()
Thanks for visiting the REPL today!

Making IPython Optional

Of course, if you want to distribute your REPL script as part of a package, or if not everyone on your team uses IPython, you’ll want to make this optional.

Using a bit of exception catching, you can default to IPython for your REPL, falling back to the vanilla one if IPython is not installed:

# deep_thought/repl_compat.py
header = "Welcome to REPL! We hope you enjoy your stay!"
footer = "Thanks for visiting the REPL today!"
scope_vars = {"answer": 42}
try:
    import IPython
except ImportError:
    from code import InteractiveConsole
    InteractiveConsole(locals=scope_vars).interact(header, footer)
else:
    print(header)
    IPython.start_ipython(argv=[], user_ns=scope_vars)
    print(footer)

You can see repl_compat.py in its entirety on GitHub.

Adding Command-Line Options

Often it is useful to specify options (such as a host or URL to connect to) at runtime. For a REPL script, that means accepting command-line arguments.

To build our REPL executable, we will use the click library. Click handles many details for us, such as parsing arguments, creating command interfaces, and more.

Here’s what our application looks like:

# deep_thought/repl_cli.py
import click
class DeepThought:
    def __init__(self, host):
        super().__init__()
        self.host = host
    @property
    def answer(self):
        print(f"Connecting to {self.host}...")
        return 42
@click.command()
@click.option("--host", default="localhost", help="Host URI")
def main(host):
    header = "Deep Thought initialised as `cpu`. " \
             "Type `help(cpu)` for assistance."
    footer = ""
    scope_vars = {"cpu": DeepThought(host)}
    try:
        import IPython
    except ImportError:
        from code import InteractiveConsole
        InteractiveConsole(locals=scope_vars).interact(header, footer)
    else:
        print(header)
        IPython.start_ipython(argv=[], user_ns=scope_vars)
        print(footer)
if __name__ == “__main__”:
    main()

You can see repl_cli.py in its entirety on GitHub.

When the script is executed (with optional --host argument), it creates an instance of our DeepThought class and configures it. It then drops the user into a REPL to do whatever they’d like.

> python deep_thought/cli.py --host=example.com
Deep Thought initialised as `cpu`. Type `help(cpu)` for assistance.
In [1]: cpu.answer
Connecting to example.com
Out[1]: 42

Entry Points

For the icing on the cake, let’s install our REPL script as an executable, so that the user can invoke it by typing a custom command into their terminal:

> deep-thought --host=example.com
Deep Thought initialised as `cpu`. Type `help(cpu)` for assistance.
...

To make this work, we will take advantage of an oft-overlooked feature of Python’s setuptools library, called entry points.

First, our project must have a setup.py file. If you’ve never created one before, don’t panic; they are quite straightforward.

For this project, we will stick to the bare minimum:

# setup.py
from setuptools import setup
setup(
    name="repl_demo",
    version="1.0.0",
    packages=["deep_thought"],
    install_requires=["click"],
    entry_points={
        "console_scripts": [
            "deep-thought = deep_thought.repl_cli:main",
        ],
    },
)

You can see setup.py in its entirety on GitHub.

Let’s break this down:

name is the name of our project, or “distribution”, as it’s known in setuptools parlance. The exact value doesn’t matter, so long as it’s unique; this is how tools like pip know which distributions are installed.

version is the version number of our distribution.

packages tells setuptools which Python packages to include in our distribution. For this project, we only need to declare the deep_thought package, but more complex projects might include several packages here.

install_requires tells setuptools which other distributions need to be installed alongside ours (it serves the same purpose as a requirements.txt file). Since we’re using the click library in our script, we need to add it to install_requires.

entry_points defines the entry points for our distribution. We’ll focus on these in a moment.

There’s a lot more that you can do with your project’s setup.py file to customise the way it is built, packaged, deployed and installed. If you are interested in learning more, check out the Python packaging documentation.

Let’s focus on the entry_points declaration for now:

entry_points={
    "console_scripts": [
        "deep-thought = deep_thought.repl_cli:main",
    ],
}

Entry points provide a mechanism by which your project can inject its own functionality into other distributions. It can be very powerful, but also very complex. We won’t got into too much detail about how entry points work; if you are interested, there’s a comprehensive explanation on the setuptools documentation.

For now, we will do a bit of hand-waving and say, “Just make it look like the example code.”

Install and Test

So far, all we’ve done is define some metadata about our distribution, but we haven’t actually installed anything yet. If we try to execute our new command-line application, it won’t work:

> deep-thought --host=example.com
command not found: deep-thought

Fortunately, this is an easy problem to solve; all we have to do is reinstall our distribution:

> pip install -e .
...
> deep-thought — host=example.com
Deep Thought initialised as `cpu`. Type `help(cpu)` for assistance.

The -e flag installs our distribution in development (or “editable”) mode. It’s not necessary to install a command-line application; it’s just considered a best-practise when you are installing a library that you are actively developing. See pip documentation for more information.

Conclusion

The REPL is one of the most useful gifts that you can give to your power users (and yourself!). API clients, mathematical models, database interfaces, and more… REPLs add value to all kinds of applications.

Did you build a REPL into your Python project? Tell us about it in the comments!