Type hinting, Easy coding (Add types info in your Python codes)

Why you should add types in Python codes and how to get started.

Although dynamically typed, Python allows users to provide type information for all variables (arguments, return values, local and global variables). (screenshot with Visual Studio Code)

Python is a dynamically typed language, meaning you don’t need to declare to Python the types of the variables you use. Nevertheless, since v3.5, Python allows users to provide type hints to variables (arguments, return values, local and global variables) in the way shown in the image above. While praised by many, a lot other developers feel this feature redundant, strange and ugly.

“Types? Come on! I chose Python to make my life easier. Now you ask me to declare types?”

“What if my function works for integers as well as for strings? Are you asking me to write some Python function templates? Does such a thing even exist?”

“I actually tried type hinting using “typing” module, but it was too much hassle as soon as I started using it seriously.”

In this post, I’ll start by giving examples of adding simple type hints and show how your text editors can serve you better with those hints. I’ll then dive a bit deeper into type hints and introduce some slightly more advanced but very useful typing concepts. In the end, I’ll give my own advice about how to start with type hinting which I think might be useful to beginners.

Why adding types can make coding easier?

The main reason why I’m advertising Python hinting is that by providing type hints, your text editors have more information to help you.

Nowadays many people use quite smart text editors (such as PyCharm, Visual Studio Code, Atom, Sublime Text, vim, etc) to improve coding efficiency. Those text editors have nice features such as syntax highlight, auto-complete suggestions and help info display in tooltip boxes while you are typing (see the image below for an example).

Whenever possible, modern text editors automatically display help information in a tooltip box while you type. (screenshot of an example in Visual Studio Code)

However, due to Python’s dynamically typed nature, your text editor usually won’t have enough information to decide how to help you. While it doesn’t seem like a big favor when the variables you use are mostly of primitive types (str, int, etc.), your coding experience can improve a lot when working with classes with complex internal structures.

Let’s give a real-world example. Imagine we are writing a function which converts some simple XML to a Python dict object. To try the following experiment yourself on your favorite text editor, you need to pip install lxml. Note that no prior knowledge about this package or XML is required. We only use it to illustrate type hinting. (You can find a nice tutorial about the lxml package and the element tree (etree) API at https://lxml.de/tutorial.html if you are interested, but you really don’t have to for reading this article. None of the details about this package actually concerns us.)

Now let’s type the following function header to the text editor (I’m using Visual Studio Code here, but feel free to use your favorite).

def simple_xml_to_json(root_node):

Note that although we, as the author of the function, know that the argument root_node is an instance of the etree element class in lxml, Python doesn’t really care when you are writing this function. In fact, we don’t even need to import this package to use it in the function. Interestingly, many developers feel this is a blessing of Python rather than a curse. Well, let’s go ahead and try to use this root_node object in the function body.

Indulged by my text editor, I usually just type root_node. (with the “dot”) and expect a list of attributes and methods to pop up for me. Here instead (and not surprisingly), nothing happened:

Without information about the type of the “root_node” argument, your text editor has no idea how to help you. (screenshot with Visual Studio Code)

Again, you don’t get any help information about root_node (e.g.: auto-completion suggestions) because your text editor doesn’t know what it actually is. How can we give the text editor this information? By providing type hints!

Now let’s add in the type hints and see what will happen.

With some type hints, your text editor can immediately provide you with autocomplete suggestions. (screenshot with Visual Studio Code)

Now we are able to harness the autocomplete functionality and quickly find the attributes or methods we are looking for. Even better, after we choose a method and type (, we see the documentation of that method popped up and greeting us. Without need to go to any documentation website, we can quickly understand all about the method and how to use it.

Method docstring displayed while you are coding. (screenshot with Visual Studio Code)

Dive a bit deeper into type hints

Now that we’ve seen how type hints helps us code more efficiently, let’s introduce some more advanced and useful concepts. A warning here: you can go very deep and abstract into the topic of type hinting with complicate technicality. Here, I’ll only introduce concepts which are commonly used as soon as you start adding type hints into your codes.

Again, nothing is better than examples. Take a look at the following function, which performs a linear search and tells whether a target element exists in a collection or not. Just for sake of illustration, I added a very contrived optional integer argument called upper_limit with the default value to be None. If a positive integer is given to this argument, the function only search the collection l up to this limit. (Yes, I know, it’s a contrived keyword argument. Bear with me for now.)

An example function to demonstrate some adcanced type hints.

The type hint for return value is easy and is left as an exercise for now (see the next snippet). When we type hint the arguments, we immediately have the following questions:

1. I have been using the name "collection" for the first argument l but it can be a list, a tuple, or anything which supports iteration with the for loop. What type should we give to it?
2. In order for the comparison == between target and elements in l to make sense, they must be of the same (or comparable) type. However, we don’t really restrict what this type really is. They can be all integers, strings, etc. All we require is that the type of target and elements in l is the same. Can we do that without specifying what the type actually is?
3. upper_limit variable is definitely an integer, but it can take None value, which is of the type NoneType. How can we allow one variable to be either of int type of of NoneType?

Let’s address these questions one by one.

Iterable type

Experienced readers may have recognized what l really is from the description of Question 1 above. It’s a Python iterable. Technically, a Python iterable is not really a type, but a protocol. Any class which supports the iteration methods (some special methods in the class definition) is qualified as an iterable type. Talking about Python iterables deserves a separate article. Here all you need to know is that an object obj is an iterable if you can do for x in obj. For example, lists, tuples are all iterables. A file handler (the object returned by the open method) is an iterable because you can do for line in fhand to iterate lines of the file.

How do we give hints that l is an iterable object? Good news is that, iterables are so ubiquitous in Python that its type hint has already been created. All you need to do is to import Iterable from typing module.

Now our function becomes the following (I’m omit the function docstring from now on. I also added return type here, which is bool).

An example using Iterable type from typing module

Note here we only said the type of l is an iterable, but didn’t provide any hint to the type of elements of l. The element can be of any type. We can improve this by hinting the type of the elements of the iterable. For example, to claim that all elements in l are integers, simply use Iterable[int] for l.

Here we encounter Question 2. Again, we only want to claim that target and elements in l are of the same type but don’t restrict what this type actually is. They can be integers, strings, or any other types which support ==.

The idea of “type variables” comes to save the day.

Type variables

For Question 2, we need some generic type (or a type placeholder). You can define such a placeholder using TypeVar from typing module. See below on how the function looks like now.

Here we claim that l is an iterable whose elements are of type T (which can be any concrete type) and target is a variable of the same type T. Great!

Now let’s come to the last argument upper_limit. We tend to do upper_limit: int=None since it’s an integer. However, this line doesn’t make sense because its default value None itself is not of type int. (Note that if you do this and call the function, you codes will run! Python itself doesn’t really check the types for you. As the name suggests, they are just type hints, not type declaration. However, upper_limit: int=None is simply confusing.)

Union and Optional

There are at least two ways to solve this problem. First, we can claim that upper_limit is either of type int or NoneType. The Union type in typing module is introduced to address this “either … or …” situation. Using Union, we can write upper_limit: Union[int, NoneType].

Since None is a very popular default value for many optional variables, typing module also introduced another guy called Optional to cater our needs. For example, Optional[int] is essentially the same as Union[int, NoneType].

It’s time to finalize our function

Some final words for beginners

As shown in the last section, you can dive quite deep into Python’s type hinting system, and soon things can go nasty. While mastering type hinting can be quite valuable especially to some big organizations and big projects, it can be unnecessarily complicated for beginner developers. Luckily, Python allows you to gradually adopt the type hinting knowledge and practices. Python doesn’t refuse to run you codes if your codes have no type hints, or even wrong type hints (again, type hinting doesn’t change Python’s dynamically typed nature and will be ignored by the interpreter at runtime). My personal suggestion is, make use of this gradual feature. When you add type information, make sure you add correct information, but there’s no need to type hint everything.

Also note that there are type checking tools (e.g: mypy) which checks your codes with the type hints (like what the compiler does when compiling source codes). However, it may cause some initial frustration to beginners and drag you into the mud of more advanced type hinting technicality earlier than necessary. If this causes frustration to you, feel free to not to do that first and only come back to it after you become more comfortable with type hinting.

Here are some cases in which adding type hints will help you or the users of your codes in my opinion:

1. Functions whose arguments are instance of some classes. As we demonstrated above, if the class itself is well documented and type hinted, you can immediately get informative help from your text editor when you use this argument.
2. If you are writing a module (to be used by yourself or others), try to add type hints to every function (arguments and return value) or class (class variables and methods). This will make sure that the users of your module can get help. In fact, in this case, I recommend to write well-formatted docstrings, but this will deserve another article.

I really hope this article will become an invitation to those who started writing wonderful Python projects and are looking for simple tips to improve code quality. Please do let me your thoughts in the comments, and let me know what other coding topics you may find interesting. This will encourage me to write more useful articles.

Happy coding!