Are you tired of using print statements for debugging?

Beyond Print Statements: Elevating Debugging with Python Logging

Step into the next level of debugging and application monitoring

Photo by Julien L on Unsplash

In the bustling world of software development, robust logging is essential — not just as a means of catching errors, but as a powerful instrument for understanding and optimizing application behavior.

Python’s logging module provides a comprehensive toolkit designed to meet the logging needs of any application, from the simplest scripts to the most complex distributed systems. With its ability to record various levels of information, from critical errors to debug insights, logging serves as the eyes and ears of your application in a live environment.

Why settle for print statements when you can implement a dynamic, scalable logging system?

This article will guide you through Python’s logging module, showing you how to set up different loggers, handlers and formatters to capture, process and output logs exactly as you need.

By utilizing Python’s logging, you can gain deeper insights into your application’s operation, troubleshoot more effectively and ensure that your software can grow and adapt without losing clarity or control over its internal processes.

Why Use Logging?

Logging offers several advantages over using print statements for debugging and monitoring:

1. Granularity: Logging allows you to specify the importance of messages (debug, info, warning, error, critical).
2. Configurability: You can configure different log handlers to direct messages to various destinations (console, files, network, etc.).
3. Consistency: Centralized logging configuration ensures consistent formatting and behavior across your application.
4. Performance: Logging can be configured to write messages asynchronously, reducing performance overhead.

Basic Configuration:

To start using the logging module, you need to import it and set up a basic configuration. Here’s a simple example:

import logging

# Basic configuration
logging.basicConfig(level=logging.DEBUG, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')

# Log some messages
logging.debug('This is a debug message')
logging.info('This is an info message')
logging.warning('This is a warning message')
logging.error('This is an error message')
logging.critical('This is a critical message')

In the above example:

• basicConfig sets up the default configuration for the logging system.
• level specifies the lowest-severity log message a logger will handle (DEBUG, INFO, WARNING, ERROR, CRITICAL).
• level=logging.DEBUG: This sets the threshold for what level of logs should be displayed. DEBUG is the lowest level, meaning all messages at this level and higher will be logged (INFO, WARNING, ERROR, CRITICAL).
• format='...': This defines the format of the log messages. Each message will include the timestamp (%(asctime)s), the logger's name (%(name)s), the level of severity (%(levelname)s) and the log message itself (%(message)s).

Logging Messages: Each logging call logs a message at a specific severity level which will be printed on the console/terminal:

• logging.debug(...): Logs a message with level DEBUG.
• logging.info(...): Logs a message with level INFO.
• logging.warning(...): Logs a message with level WARNING.
• logging.error(...): Logs a message with level ERROR.
• logging.critical(...): Logs a message with level CRITICAL.

By default, the logging.basicConfig() function configures the logging system to output messages to the console (standard output). The configuration set up in your code snippet will cause all messages at levels DEBUG and above to be printed to the console, formatted with the specified pattern (time, logger name, severity, message).

Output:

Output on console log Messages

File Logging:

Beyond logging to the console, logging also allows you to easily direct these messages to a file. This is particularly useful for maintaining persistent records of an application’s operation, which can be reviewed later for debugging or auditing purposes. Here’s how you can modify the basic configuration to also log to a file:

import logging

# Configure logging to file
logging.basicConfig(filename='app.log', filemode='w', level=logging.DEBUG,
                    format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')

# Using the same methods as before, your log messages will now go to app.log:
logging.critical('This is a critical message')
try:
    1 / 0
except ZeroDivisionError as e:
    logging.error("ZeroDivisionError", exc_info=True)
    # alternative way: logging.exception("ZeroDivisionError")

• filename='app.log': Specifies the file to which logs should be written.
• filemode='w': Defines the mode in which the file is opened. Using 'w' means the log file is overwritten each time the application is started. Use 'a' for appending if preserving logs is required.

Output:

Custom Handlers and Loggers

As your logging needs grow more complex, you might want to customize how logs are handled.

Step 1: Create a Custom Logger

Create your own logger to have more control over which messages get logged and where they go:

# Create a custom logger
logger = logging.getLogger('my_logger')
logger.setLevel(logging.DEBUG)  # Capture all messages of debug or higher severity

Step 2: Add Handlers

Add different handlers to send log messages to multiple destinations or in different formats:

# Create a custom logger
logger = logging.getLogger('my_logger')
logger.setLevel(logging.DEBUG)  # Capture all messages of debug or higher severity

### File handler for errors
# Create a file handler that writes log messages to 'error.log'
file_handler = logging.FileHandler('error.log') 
# Set the logging level for this handler to ERROR, which means it will only handle messages of ERROR level or higher
file_handler.setLevel(logging.ERROR)  

### Console handler for info and above
# Create a console handler that writes log messages to the console
console_handler = logging.StreamHandler()  
# Set the logging level for this handler to INFO, which means it will handle messages of INFO level or higher
console_handler.setLevel(logging.INFO)  

### Set formats for handlers
# Define the format of log messages
formatter = logging.Formatter('%(asctime)s - %(levelname)s - %(message)s') 
# Apply the formatter to the file handler 
file_handler.setFormatter(formatter) 
# Apply the formatter to the console handler
console_handler.setFormatter(formatter)  

### Add handlers to logger
# Add the file handler to the logger, so it will write ERROR level messages to 'error.log'
logger.addHandler(file_handler)  
# Add the console handler to the logger, so it will write INFO level messages to the console
logger.addHandler(console_handler)  

# Now when you log messages, they are directed based on their severity:
logger.debug("This will print to console")
logger.info("This will also print to console")
logger.error("This will print to console and also save to error.log")

Output:

Best Practices for Using the logging Module

• Do not use the same logger across different modules; instead, create a new logger for each module using logging.getLogger(__name__).
• Configure loggers at the entry point of your application; this centralizes the logging configuration and makes it easier to adjust it for different deployment environments.
• Include tracebacks in your logs when logging exceptions, which can be done with logging.exception("Exception occurred"), providing full stack traces.

Photo by visuals on Unsplash

Conclusion

Python’s logging module is an essential tool for modern software development. With its powerful features for handling different logging levels and output formats, including file outputs, it offers a more nuanced approach than simple print statements, supporting complex applications and systems. By integrating sophisticated logging into your applications, you can enhance your ability to monitor their health and behavior, leading to more robust and reliable software.

Whether you’re debugging issues, monitoring performance, conducting security audits, or simply wanting to maintain a detailed record of your application’s operations, the logging module provides the flexibility and functionality required. By configuring logging appropriately, you ensure that your software is not only easier to maintain but also more resilient to unexpected issues. Embrace the power of logging to elevate your development practices and deliver high-quality, dependable applications.

Final Words

Thank you for taking the time to read my article.

This article was first published on medium by CyCoderX.

Hey there! I’m CyCoderX, a data engineer who loves crafting end-to-end solutions. I write articles about Python, SQL, AI, Data Engineering, lifestyle and more! Join me as we explore the exciting world of tech, data, and beyond.

Interested in more content?

• For Python content and tips, click here to check out my list on Medium.
• For SQL, Databases, and data engineering content, click here to find out more!

Connect with me on social media:

• Medium: CyCoderX — Explore similar articles and updates.
• LinkedIn: CyCoderX — Connect with me professionally.

If you enjoyed this article, consider following me for future updates.

Please consider supporting me by:

1. Clapping 50 times for this story
2. Leaving a comment telling me your thoughts
3. Highlighting your favorite part of the story

Please consider supporting me by:

1. Clapping 50 times for this story
2. Leaving a comment telling me your thoughts
3. Highlighting your favorite part of the story

Let me know in the comments below … or above, depending on your device 🙃