PART FOUR: Logging — Building a Production-Ready Algorithmic Trading Framework in Python

Photo by Waldemar on Unsplash

If you ever build any software (especially the kind that will be left unattended), you will need logging. It is the go-to source of information. Logging in Python can be done in many ways; the most common one you will see is Print(‘my logging text’).

This is great for spinning up ideas and printing info to screen, but if your whole program is built like this, it can become very inefficient to have all this information printed every time something runs.

“But Joe, I need this information to understand what is going on while I debug the program?” Never fear. Every magician has a trick up their sleeve or developer… one and the same sometimes.

The logging module allows you to assign importance to the different kinds of logging your program will encounter. Below are the examples:

DEBUG: As the name suggests, you use this for more verbose information when developing/debugging, and it can be hidden when you take the software into production.

INFO: Used when you need to provide feedback every time the program runs. It can be simple, the program has started… the program has ended.

WARNING: Typically, feedback to a user or developer lets them know something isn’t running as expected.

ERROR: This is used when handling errors; this will be another topic I will explore later.

CRITICAL: Something is wrong, and the whole program needs to stop.

Photo by Joshua J. Cotten on Unsplash

All these importance levels are for you to pick and choose throughout your program.

All this information must go somewhere, and this is where you can save your logs to disk in a ‘.log’ file. The code I have provided will save by date and then the log name you set when you create the log object. I have provided some boiler plate code at the bottom.

pimport configparser
import traceback
import logging
import os
import datetime as dt

# Get root directory of environment
ROOT_DIR = os.path.dirname(os.path.abspath(__file__))


def log_maker(log_name, configFilePath: str = '|'):
    """
    :param configFilePath: default is a file in the root called config.ini
    :param log_name: Name of the Logs file or program that is running the Logs.
    """

    # set empty objects for error checking
    streamLogLevel = None
    fileLogLevel = None
    logger = None

    try:

        file_name = f"{dt.date.today()}-{log_name}"
        # Create base logger
        logger = logging.getLogger(file_name)

        # Create app settings parser to fetch settings
        config_parser = configparser.RawConfigParser()

        if configFilePath != '|':
            config_parser.read(configFilePath)
        else:
            config_parser.read('configs.ini')

        # Fetch the section in the app settings for logging
        log_level_stream = config_parser.get('Logging', 'log_level_stream')
        log_level_file = config_parser.get('Logging', 'log_level_file')

        # Check what the Logs level is and set the appropriate settings

        # This is for the stream that prints into the console
        if log_level_stream == "DEBUG":
            streamLogLevel = logging.DEBUG
        elif log_level_stream == "INFO":
            streamLogLevel = logging.INFO
        elif log_level_stream == "ERROR":
            streamLogLevel = logging.ERROR

        # This is for the file that saves to disk
        if log_level_file == "DEBUG":
            fileLogLevel = logging.DEBUG
        elif log_level_file == "INFO":
            fileLogLevel = logging.INFO
        elif log_level_file == "ERROR":
            fileLogLevel = logging.ERROR

        # Set the level for the console logging
        logger.setLevel(streamLogLevel)

        # Adjust the formatting for how the logs will save
        formatter = logging.Formatter('%(asctime)s : %(name)s  : %(funcName)s : %(levelname)s : %(message)s')

        # Save the Logs to the target output file and set all logging settings for the output file
        file_handler = logging.FileHandler(f"{ROOT_DIR}/Logs/{file_name}.Logs")
        file_handler.setFormatter(formatter)
        file_handler.setLevel(fileLogLevel)

        # Set all Logs streams to the current Logs object
        logger.addHandler(file_handler)

        if config_parser.getboolean('Logging', 'log_to_console'):
            logger.addHandler(logging.StreamHandler())

    except Exception as er:
        print(f"An Error Occurred While Creating Logging Object: {er}\n{traceback.format_exc()}")

    return logger

Finally, to control this, you will need a handy dandy config file, I will be covering the files in another post, a short and sweet description is, they are a file that helps you control settings that would be a pain to go and change in the code. This could be a small parameter buried deep within your program. Copy the below code and save it as “config.ini” in the same folder that you save the code above.

[Logging]
testing = True
developer = True
log_to_console = True
log_level_stream = INFO
log_level_file = INFO

Here is some boiler plate to get you started in your next Juypter-Notebook or Python Project, I saved my logging script as ‘Logger.py’.

# import the logging script into your target python script
from Logger import log_maker

# Create the log object and set the name of the logging file
logger = log_maker('Logging')

# Make logs not war
logger.debug("DEBUG STUFF")
logger.info("INFORMATION STUFF")
logger.warning("WARNING LOGS")
logger.critical("BIG PROBLEMS")
logger.error("LETS RETHINK THE PROBLEM")

That’s all she wrote, folks. I hope you learnt some things and will use my tools; hopefully, it will help you along the way. Peace.

If you want to help buy me a coffee that fuels these articles’ late nights of research and development, please consider donating to my PayPal link below. Thanks so much, and have a great day.

paypal.me/JEFSBLOG