🐍 Type checking your Python code! 🐍
Using Type hints (PEP 484) and mypy for static type checking!
One of the first things that come to a developer’s mind when someone begins talking about a programming language is whether that language is statically or dynamically typed!
Python is known for its duck typing system which is very flexible but often lacks the safety which a static type analysis can offer and is, usually, desired for larger software projects.
In this post, we'll learn the following topics:
- Dynamic vs Static typing;
- Python type hints syntax and usage;
- Using mypy type checker.
[ ! ]: This is long post, but most of it is composed of easy-to-follow code snippets! Use it as a reference to know how to use some of the most common type hints of the language in many situations!
Shall we get started?
1. Dynamically Typed Languages
Python is a dynamically typed language which basically means that the type of a variable is checked only during runtime. The following example illustrates a variable called my_var which, during the program’s execution, is given many different types. For example, in Python code:
This also means that possible type errors may be discovered only when the code is executed. This would, in many situations, be avoided if a statically typed language were used.
PEP 484 has introduced a way of performing static type checking of Python code by using type hints and a static type checker such as mypy! But don’t worry: Python will remain as a dynamically typed language! The static type checking and the type hints can be introduced at specific parts of the code where such typing safety is desired! Therefore, the addition of type hints in Python code can and should be a gradual process.
2. Statically Typed Languages
Languages such as Java or C are statically typed. Essentially this means that type checking will take place at compile time based on the source code. Hence, in most cases, once a variable is bound to a given type in the source code, it’ll remain with that type. As an example, in Java code:
String my_var = "igor" // my_var will only accept String types
3. Duck typing and Dynamic Typing
I’d say duck typing is a style of Dynamic typing. However, in Duck typing the type checking during runtime is not performed at all! What is checked during runtime is whether the variable possesses the requested method or not!
Here goes a silly example using some animals:
Type hints are a great way of documenting your code! By using such feature, it’s very easy for another developer to understand what is going on! Before such feature, one could use docstrings inside functions, for example, to describe the expected types for that block of code!
Most Common Type Hints
How about we start giving some types to our Python code?
For simple cases such as the function above, we can use the same name of the built-in classes, like str, int, float, bool, etc., for specifying the type of a variable:
The syntax is as follows for type hints of function parameters and the function's return type:
def fn(arg1: type1, arg2: type2, ...) -> ReturnType:
Here we can notice that by using type hints we achieved the following things:
- Documented the code and allowed the docstring to be shorter;
- Allowed static type checkers, such as mypy, to check for type inconsistencies.
This syntax can applied to Python 2 in the following way using a comment-based approach (well, it’s just a comment so it cannot break the execution):
However, this article will focus on Python 3 syntax! Time to move on to new stuff, right folks?
For variables, the type hints syntax is as follows:
x: int = 2
Let’s just be careful when trying to type hint everything, okay? It’s not always necessary since static type checkers can infer types too!
x = 2 # will also be inferred as an int type
Now, let’s move on to some compound types which are a bit more complicated. For those, we’ll resort to the typing module which includes a bunch of type definitions that we can start using right away!
Let's begin with a simple silly function that returns a doubled list:
We could start with the list class type which we are used to since it’s a base class:
However, this is not as specific as we would like it to be. Sometimes, we want to specify the types of the list members! Thanks to the typing module we can use the compound type List:
Nice! Now we can tell that this function expects a list of integers! But, what if we wanted to return tuples with the original value along its doubled value?
Again, just saying we are returning the base type tuple isn't that helpful! Instead, let's use the compound type Tuple!
Nice! Now we know that the function double_together returns a list of 2-tuples with integer elements.
On the other hand, as one can observe, compound types can easily become cumbersome to use because we keep adding more and more types! This is when type aliases can be very useful:
Here we have defined our custom type: IntPair. We also defined that a list of IntPair will be returned from the function! This is often easier to read and makes using compound types a breeze!
How about the types of a dictionary? Such a common data structure! Here’s one whose keys are strings and its values are integers:
But how about dictionaries that have key values of different types? For that we have TypedDict which is still an experimental feature but already an officially supported one. However, to use it, we must install the module mypy_extensions:
pip install mypy_extensions
Even though using type hints allow us to write more "robust" code, we may come to situations where we decide to give up any static analysis. For such situations we can use the Any type:
Naturally this will work and static type checkers won’t complain anymore about incoherent types! So, beware: no more static typing safety for you!
Oh, and just to keep in mind: List and Tuple or Dict (without more typing specifications) are the same as declaring the types:
List[Any], Tuple[Any], Dict[Any, Any]
Moreover, one can also define functions that have no return with None:
Another very common function is design in Python is to return a value or None. For such cases the typing module has the Optional compound type:
Here, Optional[str] means: this function will try its best to return a str but, in some cases, it may return None. Hence, any consumer of this function must be prepared to handle None values! And hey, remember: I warned you!!
Another neat feature: Optional[T] is an alias for Union[T, None]! Union is a compound type that means either one type OR another one(s)!
x : Union[str, int, bool] # x can be a str, an int or a bool!
Now, let's use some type hints for something that the Java fans out there are definitely waiting for: using custom classes as types! Yay, this can be done!
Let's begin with a simple class which represents Mario in a 2-D game board:
Notice that the update_position method, rather than mutating Mario’s instance state, returns a new instance with the updated position! Let’s give some hints for the handle_player_events function first!
The first thing we can notice is that the class name, Mario, also defines a type that can be used throughout the code! Hence we see that the mario type parameter means: hey! here a Mario instance class can come in anytime!
Now, typing the Mario class involves some trickery, specially when it comes down to the method update_position:
Since the Mario class returns an instance of itself, we cannot declare the return type of update_position as Mario since we are still defining what is this class and therefore what is the type Mario!
For such cases one must resort to string type annotations which will be type checked later and by later we can think after the class is defined! Hence the method returns "Mario" and not just Mario!
We can declare our own custom type variables by using the TypeVar class imported from the typing module!
And still for the Java fans out there, we can define generic types! A generic type is typically creating a class which inherits from the Generic class with one or more type variables!
The type variables serve as the parameters for generic types!
With those we can create our own simplified but typed version of Python dictionaries:
Another type which is quite common due to Python's first class functions is the Callable type which should be used to define function types: both the type of its parameters and also its return type! The syntax is as follows:
Callable[[Param1, Param2, ...], ReturnType]
Oh and speaking of strings, there's also the type AnyStr which can be both string and bytes! In fact, it's defined as:
AnyStr = TypeVar('AnyStr', str, bytes)
3. Using Mypy as the static type checker
One of the benefits ofadding type hints is that we can improve our code readability! However, once we get an interpreter to run the code, those types will not be enforced!
In order to do so, one must use a static type checker! My favorite one is the type checker which is being developed by Guido: mypy! To install it activate your venv and install it:
pip install mypy
Then, it's pretty much like using the black formatter or any other linting tools (pylint, etc.):
mypy python_module_name.py --ignore-missing-imports
Should any type inconsistencies be detected, then such errors will be reported! Otherwise, mypy will run silently!
The — ignore-missing-imports flag makes mypy ignore all missing imports that do not have type definitions!
Usually, just like other linters, it's convenient to run this tool with a file watcher! That is: once you save your module, mypy is executed to type check your code! Here's a screenshot of my mypy configuration for Pycharm used as a file watcher tool:
And that's it guys! This post has covered many important type hints which can be used to make your Python code more documented and also safer for larger projects! Not every type/usage has been mentioned because this post is already big! However, many useful ones were introduced here!
If there are any doubts, please refer to the official documentation for Python type hints:
That's all folks! See you guys in the next post!!