CodingZen — Static Typing in Python? No Way…

Static Typing? What does that even mean?!

Before I dive into this topic, let me take a step back and explain two foundational topics:

1. Static typing v dynamic typing
2. Strong typing v weak typing

Static Typing v Dynamic Typing

Static typing and dynamic typing describe when types are checked. Languages such as Java, C++ and C# check types before run-time, while languages such as Python and Go check types after run-time during the course of execution.

Weak Typed v Strongly Typed Languages

Another key dimension to consider is weakly typed v strongly typed, which (generally) describes what happens at compile time or runtime when it comes to type conversions and related errors. This is generally true as many languages have attributes of both or parts of each. For example, Python is considered a strongly typed language, but does not type checking at compile time, but does ensure that type errors are prevented at runtime. In contrast, a language like Java requires a declared type for all variables and type conversions generally require an explicit cast. Type checking is done at compile time.

Putting it All Together

In general, these concepts combined contribute to the overall type safety of the language or languages in question. Type safety has important implications that need to be considered. While this article is not intended to explore this topic in depth, let’s take a look at Python specifically in this context:

1. Python is dynamically typed, which has three key implications:

• Python is significantly less verbose than most statically typed languages
• Developers can be much more productive as a result
• However, errors are not caught at compile time, resulting in a hire risk of runtime errors

2. Python is strongly typed and has extra tools to further extend type checking, which has three key implications:

• Type errors will occur at run-time even if they are not caught at compile time
• However, not all type errors will be caught if proper enforcement is not in place
• This can result in bugs that are difficult to track down. We will explore some later.

That’s a Lot to Take In! What Now?

The critical thing to remember is that Python comes with the “tools” to implement the appropriate degree of type checking. This is one of the reasons that Python is great for beginners and for analytics. You don’t have to use the features of Python that enable stronger type checking, but you can if you want to. You as the developer simply needs to make that choice and add the validation routines necessary to ensure type safety in your code.

The Newest Tool in the Toolbox

Python developers have always had tools to improve type safety such as explicitly using the type function or isinstance function. There have also been third party libraries at their disposal such as MyPy that helped to add static type checking into the mix, so errors could be detected before run-time. MyPy in particular has been incredibly successful and led to a critical change in Python 3.5: the addition of type hints in the core language. Another good option is to use PyCharm which uses type hints to perform real-time checks on your code.

What are type hints?

Type hints are essentially syntatic sugar that can be sprinkled on a function to “hint” at the type that the developer intended for either a function parameter or a return type. However, there are a couple of things to keep in mind:

1. Type hints are not enforced. In other words, the Python language itself does nothing specific with the type hints in line with idea that thte developer should actually do the work if appropriate.
2. Type hints can be added to existing code without impacting the ability for that code to run.

So if they are not enforced, why do they matter? For three reasons:

1. By using type hints, you can make it much easier for others to understand your intentions.
2. Type hints can be leveraged by third party tools, such as MyPy or Pyre to perform effective static type checking on your code before run-time.
3. Instead of writing complex unit tests to test types, you can integrate MyPy and Pyre into your testing pipeline and also leverage both tools while you code.

With that in mind, let’s dive into some examples. All code uses Python 3.6+.

A Quick Example of Type Hints to Get You Started

To help you get started, let’s begin with a simple type hints example. The good news is that type hints are relatively intuitive to use. Here is an example:

def simple_car(make, model, year):
    """Print out the metadata for a simple car."""
    return list([make, model, year])
results = simple_car("Ford", "Mustang", "2018")
print(results)

On the surface this looks fine, but lets say that the year has to be an int for a downstream database or application. This could easily result in a hidden bug that may be difficult to pinpoint. To avoid this, we can add type hints to the function and leverage MyPy, Pyre, or other third party tools to validate our code is correct:

from typing import List  # List type check
from typing import Any  # Matches any type for type check
def simple_car_type(make: str, model: str, year: int) -> List[Any]:
    """Print out the metadata for a simple car with type hints."""
    return list([make, model, year])
results_type: List[Any] = simple_car_type("Ford", "Mustang", "2018")
print(results_type)

See the difference? Three things to point out:

1. By adding “year: int”, we are now indicating that year must be an int.
2. By adding “-> List[Any]” after the function parameters, we are now indicating that the results must be returned as a list.
3. As a result, since the year in line 8 is passed as a string instead of an integer this should result in an error using MyPy.

And so it does! The screen shot below shows the error MyPy generates:

A More Detailed Example — The Cars Class

Let’s take this a step further with a more complex example. To test this out, we’ll use a cars class with a metadata method to print out the details of the class. Two things to note:

1. Normally you might use the Python dunder methods such as __str__ and/or __repr__ to do this properly. You can find more details on this here.
2. There are also better ways to set attributes on a class. In Python 3.7 for example, data classes are introduced, which seem to be a superior approach. Details can be found here.

The class below does not include type hints:

class CarsNoType():
    """ The CarsNoType class demonstrates no type checking

        Attributes:
           make (str): The make of the car, such as Ford or chevy
           model (str): The model of the car such as mustang or camaro
           type (str): The type of car such as sports or truck
           year (int): Four digit representation of the year
           owner (str): The name of the owner
           last_update (str): The last update of the record in
               'YYYY-MM-DD' format
    """

    def __init__(self, make, model, type, year, owner, last_update):
        """Initializes the Cars class without type checking"""
        self.make = make
        self.model = model
        self.type = type
        self.year = year
        self.owner = owner
        self.last_update = last_update

    def car_metadata(self):
        """Prints cars metadata out to the user"""
        print("This car with no types has the following attributes: ")
        print(f"Make: {self.make}")
        print(f"Model: {self.model}")
        print(f"Type: {self.type}")
        print(f"Year: {self.year}")
        print(f"Owner: {self.owner}")
        print(f"Last Update: {self.last_update}\n")

Note: Are you wondering what those “f”s are used for in the print statements in the car_metadata function? They represent a new approach to formatting strings in Python 3.6. Find out more details here.

Testing Out the CarsNoType Class

The code below is a simple way to test out the Cars class. Note that all the parameters are strings except for the year. If you run this code, it will run successfully as expected:

print("Demonstrate the cars class with no type hints: \n")
test_car_no_types = CarsNoType("Ford", "Mustang", "Sports Car", 2018,
                               "Daniel", "2018-05-18")
test_car_no_types.car_metadata()

But the trouble is if you enter in the parameters to CarsNoType in the wrong order it will still run successfully, despite the fact the types are incorrect:

print("Demonstrate the cars class with no type hints and an error.")
print("This illustrates that incorrect data types for year and ",
      "owner won't trigger an error: \n")
test_car_types_errors = CarsNoType("Ford", "Mustang", "Sports Car",
                                   "Daniel - wrong spot",  # Name in wrong spot
                                   2018,  # Year in wrong spot
                                   "2018-05-18")
test_car_types_errors.car_metadata()

Implementing Type Hints

Type hints were an extensive addition to the Python language, so, even with this more detailed example, we will only scratch the surface here. With that said, it is easy to get started. Let’s rename the class CarsType to distinguish it and update the init method of our class by adding type hints to the function definition. Save the file as carstype.py:

def __init__(self, make: str, model: str, type: str,
                 year: int, owner: str, last_update: str) -> None:
        """Initializes the Cars class without type checking"""
        self.make = make
        self.model = model
        self.type = type
        self.year = year
        self.owner = owner
        self.last_update = last_update

Let’s deconstruct what is happening:

1. Types have been added to each parameter using a colon followed by the type. As an example, “make” becomes “make: str”, indicating that the make is of type string.
2. A return type has been added after the parameter list. In this case, there is no return type, so the return type is “None” as shown by “-> None”.
3. Return None does not have to be specified in this case.

Testing Out Our Type Hints

Let’s first test out the modified class using the following code:

print("Demonstrate that the carstype class still works as intended: \n")
test_car_types = CarsType("Ford", "Mustang", "Sports Car", 2018,
                          "Daniel", "2018-05-18")
test_car_types.car_metadata()
print("Demonstrate what happens when the carstype",
      "class is used with incorrect data.\n")
print("When using mypy or pyre this will throw an error: ")
test_car_types_errors = CarsType("Ford", "Mustang", "Sports Car",
                                 "Daniel - wrong spot",  # Name in wrong spot
                                 2018,  # Year in wrong spot
                                 "2018-05-18")
test_car_types_errors.car_metadata()

Wait? The code didn’t return any errors. What gives?!

Remember, Python doesn’t actually do anything with the type hints we have added, so we have to take an additional step to make this useful. Enter MyPy. MyPy is a tool that will leverage the type hints to perform static type checking, using the type hints as inputs. Before we can do that, let’s first install MyPy:

python -m pip install mypy

Depending on how Python 3 is installed on your system, you may need to change “python” to something else such as “python3”. MyPy also requires Python 3.4+ for reference.

Now that you have installed mypy, you can invoke mypy as follows:

mypy carstype.py

You should see output similar to this:

This is a result of the following code from our test. Both the owner of the car and year the car was made are in the wrong place and thus have the wrong type:

test_car_types_errors = CarsType("Ford", "Mustang", "Sports Car",
                                 "Daniel - wrong spot",  # Name in wrong spot
                                 2018,  # Year in wrong spot
                                 "2018-05-18")

Let’s take a minute to deconstruct the output from MyPy:

1. carstype.py:57 — Refers to the line number within the carstype file. This may differ in your own file, but will help you track down the error.
2. error: Argument 4 to “CarsType” has incompatible type “str”; expected “int” — This refers to the fact that “Daniel — wrong spot” was provided to the “Year” parameter which is an int according to our type hint.
3. error: Argument 5 to “CarsType” has incompatible type “int”; expected “str” — This refers to the fact that “2018” was provided to the “Owner” parameter which is a str according to our type hint.

As you can see, this provides a much needed check for these scenarios and can help you avoid hidden bugs that can pop up later.

One Step Further — Checking Return Types

In our last example, we focused on the types of our function parameters and set the return type hint to None. However, this can be leveraged to enforce return types using mypy, including more complex types such as lists using the new typing library that was implemented along with type hints in Python 3.5. Here is the new class in it’s entirety from the carstypereturn.py file:

from typing import List
from pprint import pprint

class CarsTypeReturn:
    """ The CarsType class demonstrates type checking

        Attributes:
           make (str): The make of the car, such as Ford or chevy
           model (str): The model of the car such as mustang or camaro
           type (str): The type of car such as sports or truck
           year (int): Four digit representation of the year
           owner (str): The name of the owner
           last_update (str): The last update of the record in
               'YYYY-MM-DD' format
    """
    def __init__(self, make: str, model: str, type: str,
                 year: str, owner: str, last_update: str) -> None:
        """Initializes the Cars class without type checking"""
        self.make = make
        self.model = model
        self.type = type
        self.year = year
        self.owner = owner
        self.last_update = last_update

    def car_metadata(self) -> List[str]:
        """Prints cars metadata out to the user"""

        return list([self.make, self.model, self.type,
                    self.year, self.owner, self.last_update])

    def car_metadata_error(self) -> List[str]:
        """Demonstrates incorrect return type"""

        return tuple([self.make, self.model, self.type,
                     self.year, self.owner, self.last_update])

Notice that we have two functions in this case. The car_metadata function remains, but we have also added the car_metadata_error function to demonstrate how the return type hint can be used and enforced using MyPy. A couple of things to point out:

1. The return type hint used in this case is “List[str]”. [str] represents the type that is expected within the list. In this case, I have updated the class so that all parameters are of str type for the sake of simplicity.
2. In order to use complex types such as list, you must import “List” from the typing library using the line “from typing import List”.
3. In cars_metadata_error, the function returns a tuple instead of a list. This should error out when we run MyPy.

So let’s test this out by adding the following code:

print("Demonstrate what happens when the carstype",
      "class returns the incorrect type.\n")
print("When using mypy or pyre this will throw an error: ")
test_car_types_return_errors = CarsTypeReturn("Ford",
                                              "Mustang",
                                              "Sports Car",
                                              "2018",
                                              "Daniel",
                                              "2018-05-18")
pprint(test_car_types_return_errors.car_metadata_error())

Then run MyPy as follows:

mypy carstypereturn.py

You should see output similar to this:

Notice that this error calls out that there is an “Incompatible return value”. This can be particular useful when you are heavily using properties throughout your Python classes, ensuring that you are returning the correct types in each case.

That’s great! How do I learn more?

As noted earlier, this just scratches the surface of type hints and how they can be used to improve the readability of your code and to implement static type checks. Here are some additional resources you can check out:

1. Type Checking Python Programs with Type Hints and MyPy by Dan Bader — https://www.youtube.com/watch?v=2xWhaALHTvU
2. Type Hints in PyCharm — https://www.jetbrains.com/help/pycharm/type-hinting-in-product.html
3. Deep Dive into Type Hints in PyCharm — https://www.youtube.com/watch?v=JqBCFfiE11g
4. MyPy Integration for Vim — https://github.com/Integralist/vim-mypy
5. Static Types for Python from PyCon 2017 (MyPy) — https://www.youtube.com/watch?v=7ZbwZgrXnwY
6. Pyre Performant Type Checking (Alternative to MyPy) — https://pyre-check.org/
7. MonkeyType (for adding type hints to legacy code) — https://pypi.org/project/MonkeyType/

Conclusions

Type hints are begging to be used in application development projects. Now that they are available, they may be overkill for data scientists and analysts, but are no doubt an essential new tool for software and data engineers. Static type checking has long been a missing piece in the puzzle for large scale Python projects and type hints are a step in the right direction that I am sure will continue to evolve in the coming years. In the meantime, it is prudent for most developers to add type hints to their code to improve readability and future proof code to be used with tools such as MyPy and Pyre. With libraries such as MonkeyType available to quickly update legacy code, it simple and straightforward to update entire python code bases with type hints, unleashing the full benefits of type hints and static type checking with little effort!