Python Dictionary Validation: How to Handle Errors Gracefully and Improve Your Code’s Performance

How you can avoid dreaded ValidationError responses when sending data via API calls to Notion.

I want to share some of the challenges I’ve come across building my automations and how I overcame them. I hope you find this useful.

There’s nothing worse than the excitement of sending data to Notion and receiving the dreaded undefined error response from Notion. We’ve all been there thinking — I checked and double checked. Why is this wrong?

Photo by Uday Mittal on Unsplash

I have utilised multiple ways to handle errors in Python dictionaries before sending them to Notion. I discuss the problem of KeyErrors as well as None values in dictionaries and how Notion does not accept them as valid inputs for its API.

The article provides a solution using dictionary comprehensions to remove None values from dictionaries and a recursive function to handle nested dictionaries.

Over the past months, I have tried to make my daily tracking updates in Notion more robust and resilient to missing data. For example, how will my automation handle missing data when I do not measure my weight or if my watch battery dies overnight and I do not have an up to date sleep metrics?

My answer is outlined below:

The high level process I use to remove None values is provided below. I will then go into more detail on what these functions actually do and provide you with some more insight in how we can remove None values from dictionaries.

I wrote in more detail on the steps I take to capture garmin data and upload to Notionhere:

Building better habits by switching from Huawei to GarminConnect and connecting it all with Notion…

KeyErrors in Dictionaries

Anyway, back to the main topic. When working with dictionaries in Python, there are several ways to handle the KeyError that may occur when accessing a key that does not exist. Here are some main approaches:

1. Using the get() method: Instead of directly accessing the key, you can use the get() method of dictionaries. This method allows you to provide a default value that will be returned if the key is not found. For example: value = my_dict.get(key, default_value)
2. Using the in operator: You can use the in operator to check if a key exists in a dictionary before accessing it. This prevents the KeyError from being raised. For example: if key in my_dict: value = my_dict[key] else: value = default_value
3. Using exception handling: You can use a try-except block to catch the KeyError and handle it gracefully. By wrapping the dictionary access in a try block, you can execute alternative code in the except block if the key is not found. For example:

These are some common ways to handle KeyError with dictionaries in Python. Each approach has its own advantages and can be used depending on the specific requirements of your code.

However, if you are using Python to send data to Notion, you may encounter some errors due to the way Notion accept dictionaries that contain database property values. Dictionaries are one of the most common and useful data structures in Python, but they also have some quirks that you need to be aware of, especially when dealing with None values.

Why None Values Are a Problem for Notion

While the above methods help you avoid the KeyError problem, they don’t completely help with uploaded the data to Notion.

None values cause problems when you try to send them to Notion, because Notion does not accept None values as valid inputs for its API. For example, if you try to create a new page in Notion using a dictionary that contains a key with a None value, you will get an error like this:

Failing to upload to Notion!

As you can see, Notion expects the value of the text content property to be defined, but instead it receives a None value, which is undefined. This causes the API to reject the request and raise a validation error with Status Code: 400

You can find more about Notion Error Codes here:

Start building with the Notion API

To avoid this error, you can need to remove any keys that have None values from your dictionary before sending it to Notion. This will ensure that your dictionary only contains valid and meaningful data that Notion can process and store.

How to Remove None Values from a Dictionary in Python

There are several ways to remove None values from a dictionary in Python, but one of the simplest and most elegant ways is to use a dictionary comprehension. A dictionary comprehension is a concise and expressive way to create a new dictionary from an existing one, by applying some condition or transformation to each key-value pair.

For example, to remove any keys that have None values from a dictionary, you can use a dictionary comprehension like this:

The dictionary comprehension iterates over each key-value pair in the original dictionary, and only keeps the ones that have a value that is not None. The result is a new dictionary that does not contain any None values.

You can then send the clean dictionary to Notion.

How to Remove None Values from a Nested Dictionary in Python

The previous method works well for a flat dictionary, but what if you have a nested dictionary, as Notion requires. The values you send are nested dictionaries! For example, suppose you have a dictionary like this:

As you can see, the dictionary has two None values, one in the Water_ML property and one in the BMI property. If you try to use the same dictionary comprehension as before, you will not remove any of the properties, because the dictionary comprehension does not recurse into the nested dictionaries. The result will be a dictionary like this:

This dictionary still contains a None value in the Water_ML value, which will cause an error when sending it to Notion.

To remove all None values from a nested dictionary, I use two helper functions. One is used to identify if the value for a particular key is a dictionary or not. If it is not, I would check if its value is None. If it is a dictionary, I check if the key contains null, number, rich_text, or plain-text as these are the property types used in notion.

In order to remove the keys which contain a dictionary and a none value in that dictionary, you can utilise two functions to determine if the value in a key value pair is to be removed (ie the nested dictionary contains a None Value) and another which is essentially a dictionary comprehension to then actually remove the key:

I will then use this function within another function to remove the actual keys.

Conclusion

In this article, I showed you how you can clean up your Python dictionaries before sending them to Notion, by removing any keys that have None values. This will ensure that your data is consistent and compatible with Notion’s API, and avoid any potential errors or data loss.

I also showed you how to use a dictionary comprehensions and helper functions to remove None values from both flat and nested dictionaries, using simple and elegant Python code.

I hope you found this article useful and informative. If you have any questions or feedback, please let me know in the comments.

Thank you for reading. 😊