Testing for Beginner Python Developers

Building Confidence in Your Python Code

When you start working in a corporate setting and push your Python code, one of the things you should be careful about is testing. No matter if it is an ad-hoc script, an ETL fetcher-transformer or even processing asynchronous web requests, testing is crucial. And we do it in many ways without even realizing sometimes. In this article, let’s look at the building blocks of testing we need to include in our everyday python development.

Photo by Markus Spiske on Unsplash

Print Statements

Inserting print statements to test your initial code is the go to for all developers (unless you are an expert ninja blessed by thy Lord Pythonic). It helps with quick debugging, looking at the contents of your data structures. input/outputs of your method and the general flow of the program(are the methods executed as expected).

It is completely fine to use print statement to manually test during initial development. However, remember to:

• Not leave any print statements in the production code or your pull requests(PR).
• Start removing them during unit testing phase(discussed more later in this article)
• And once again, absolutely none in production or PR

If you are like me who uses it pretty often, forgets to remove them, and inadvertently gets nagged by someone, consider using conditional printing

DEBUG = True

def your_func(x):
    """Does something with x and generates y"""
    if DEBUG:
        print(f'Input received: {x}')
        print(f'Output generated: {y}') 
    return y

Logging Modules

Logging modules have more advantages over print statements. And you can leave them in your production code. Some useful features:

• Ability to log files, sockets and anything else, and at the same time
• Displays the position from where the logging call was made, including the line number
• Supports different log levels (DEBUG, INFO, WARNING, ERROR, CRITICAL).
• Output the log output to various destinations such as console, files, etc.

Here is the example from earlier with logging module

import logging

# Configure logging level here
logging.basicConfig(level=logging.DEBUG)

def your_func(x):
    """Does something with x and generates y"""
    logging.debug(f"Input: {x}, Output: {y}")
    return y

Unit Testing

Unit testing is where you test your methods in isolation, aka unit by unit. So, do you write test for every method? How granular do you get? You will seem to ask these questions when you are working on your test_*.py files. So try below:

• Test for methods that have public interfaces. Testing should include normal inputs, edge cases, and invalid inputs.
• Test for methods that have a complex logic. The more the complexity, more the chances of failure during production run
• Testing every method can be an overkill. If you have a simple/one liner methods whose functionality is tested in other more complex methods, feel free to move on.
• When you are constrained on the time, testing every unit can also be an overkill so plan well.

Unit testing for the same example with the unittest library

import unittest

class TestYourFunc(unittest.TestCase):
    def test_positive_number(self):
        self.assertEqual(your_func(5), 10)

    def test_zero(self):
        self.assertEqual(your_func(0), 0)

    def test_negative_number(self):
        self.assertEqual(your_func(-3), -6)


if __name__ == "__main__":
    unittest.main()

If you are conflicted which ones to write first, is it test_func.py or func.py then aim for test-driven development. From AWS documentation:

Test-driven development (TDD) is when developers build tests to check the functional requirements of a piece of software before they build the full code itself. By writing the tests first, the code is instantly verifiable against the requirements once the coding is done and the tests are run.

Integration tests

While unit testing was for units in isolation, integration will be checking how these units and components of your system work together. This is where you should think outside the units and consider

• Data flows between methods
• API interaction between components(specially externally)
• Database queries

And as you write your integration tests, try paying attention to

• Error Handling — when something does go wrong, is it handled in an orderly and expected way?
• Performance — are there any performance bottlenecks between the flow and method interaction or even the overall responses?
• Security — does the code leave any loop holes that could be susceptible to future vulnerabilities?

And yes, this can be a lot but try not to do everything at once, especially when your system has a complex architecture. The approach that you would see to write integration tests would either be a top-up, a bottom-up or even a hybrid one. Bottom up is my choice as it makes finding anomalies or issues much easier. From Testlio:

It works best when low-level components are critical to overall functionality, ensuring robust and error-free integration.

Your integration tests can be automated and run during build as part of CI/CD pipeline. This provides you with confidence for production deployment so try not to avoid this. It also prevents any future outages that may arise. Let’s take the same example(your_func) and add a method to it that extract numbers from a file, and a main method that calls these methods. In this case, our integration tests would look like

import unittest

def extract_input(filepath):
    """Read numbers from file and return list"""
    with open(filepath, 'r') as file:
        numbers = [int(line.strip()) for line in file]
    return numbers

def your_func(x):
    """Does something with x and generates y"""
    return y

def main(filepath):
    executable_numbers = extract_input(filepath)
    result = [your_func(num) for num in executable_numbers]
    return result


class TestIntegration(unittest.TestCase):
    def setUp(self):
        self.test_filepath = "test_numbers.txt"

    def test_extract_input_and_your_func(self):
        with open(self.test_filepath, "w") as f:
            f.write("1\n2\n3\n")
        expected_output = [2, 4, 6]
        result = main(self.test_filepath)
        self.assertEqual(result, expected_output)

    def test_empty_file(self):
        with open(self.test_filepath, "w") as f:
            pass # creates empty file
        expected_output = []
        result = main(self.test_filepath)
        self.assertEqual(result, expected_output)

if __name__ == "__main__":
    unittest.main()
    

Conclusion

There are other types of testing for different purpose in the development lifecycle. Such as Functional, Acceptance, Security and more that you should look into. But start small with the aforementioned testing blocks.

Testing is the backbone that will ensure the integrity of your e̶x̶i̶s̶t̶e̶n̶c̶e̶ system. Your goal should be to catch issues and fix them early. This will not only improve your code but give you confidence.