Logging in Python: A Step-by-Step Tutorial

Python logging is like a detective tool for your code. It helps you catch mistakes, understand how your program is running, and even get real-time updates on what it’s doing. Think of it as your code’s diary, but one that can help you fix problems and track its actions. In this blog, we’ll show you how to use this diary in clever and creative ways to make your Python programs better and more reliable.

What is logging and why it is necessary ?

Logging in programming is like keeping a logbook in real life. It’s a way for a program to record what it’s doing, events, errors, and other important information. Just as a logbook helps a ship’s captain keep track of their journey, programming logs help developers keep track of what their code is doing.

Here’s why logging is necessary and how it relates to a logbook example:

1. Record Keeping: Imagine a ship’s logbook; it records the ship’s course, any unusual events, weather conditions, and more. In programming, logs similarly record what’s happening in the code, such as when a user logs in, when an error occurs, or when data is being processed.

2. Troubleshooting: When something goes wrong on a ship, the logbook helps the crew figure out what happened. In programming, logs are crucial for debugging. They provide a history of events that can help developers pinpoint issues and understand what led to errors or unexpected behavior.

3. Monitoring: A ship’s logbook can help track the ship’s performance and identify trends. In programming, logs can be used for monitoring the health and performance of software, allowing developers to detect anomalies and ensure everything is running smoothly.

Let’s take other example for better understanding logging in Python:

In the context of an e-commerce application with various interconnected modules, consider the process as a sequence of steps, each responsible for a specific function.

Throughout this operation, we meticulously document every detail in a logbook, mirroring the diagram’s structure. This logbook serves as our comprehensive record of the entire process.

Now, should an error arise at any point in this intricate journey, the logbook becomes our indispensable resource. We turn to it to pinpoint the exact stage where the error occurred, identify the nature of the error, and subsequently, initiate the troubleshooting process. Thus, logging emerges as an imperative and indispensable task, enabling us to maintain the seamless operation of our e-commerce system by swiftly resolving issues as they arise.

Logging Levels:

There are several standard logging levels provided by the Python logging module, each serving a specific purpose. There are 5 levels of logging which are as given :

1. DEBUG (the lowest level):

• Use for detailed information during development and debugging.
• Example: Tracking variable values or function calls.

import logging 
logging.debug("This is a debug message")

2. INFO:

• Use to confirm that things are working as expected.
• Example: Recording when a user logs in.

import logging  
logging.info("User 'Ravi' logged in successfully.")

3. WARNING:

• Use for potential issues that don’t stop the program but might need attention.
• Example: A low disk space warning.

import logging  
logging.warning("Low disk space: Only 5% free.")

4. ERROR:

• Use for error messages indicating a significant problem.
• Example: A file not found error.

import logging  
logging.error("File 'report.pdf' not found.")

5. CRITICAL (the highest level):

• Use for critical errors that might lead to the application’s termination.
• Example: A database connection failure.

import logging  
logging.critical("Database connection failed.")

By assigning an appropriate logging level to each message, you can control the amount of information you see in the logs. During development, you might set the level to DEBUG to capture all details, while in production, you'd typically set it to INFO or higher to focus on significant events and issues. This way, you can manage the volume of log data and prioritize what's most important for your specific context.

Integers for Logging Levels :

In Python’s logging module, logging levels are represented by integer values. These integer values are used to assign a level to a log message. Here are the standard integer values for each logging level:

• DEBUG: 10
• INFO: 20
• WARNING: 30
• ERROR: 40
• CRITICAL: 50

You can use these integer values when configuring the logging system or when setting the minimum logging level for a logger or handler. For example, if you want to set the minimum level to WARNING, you would use an integer value of 30.

Here’s an example of how to set the minimum logging level using integer values:

import logging

# Set the minimum logging level to WARNING (integer value 30)
logging.basicConfig(level=30)

# Now, messages with level WARNING, ERROR, and CRITICAL will be displayed.
logging.warning("This is a warning message.")
logging.error("This is an error message.")
logging.critical("This is a critical message.")

If you don’t explicitly set a logging level for your logger, it will default to WARNING. This means that it will capture log messages with a level of WARNING, ERROR, and CRITICAL, while DEBUG and INFO messages will be ignored by default.

If you want to capture DEBUG and INFO messages as well, you can explicitly set the logging level to a lower value, such as DEBUG, when configuring the logger:

import logging

# Explicitly set the level to DEBUG
logging.basicConfig(level=logging.DEBUG)

# Now, all messages will be captured.
logging.debug("This is a debug message.")
logging.info("This is an info message.")
logging.warning("This is a warning message.")
logging.error("This is an error message.")
logging.critical("This is a critical message.")

By explicitly setting the logging level, you have full control over which log messages are captured.

Saving logs in logfile :

import logging

# Configure the basic logging settings
logging.basicConfig(filename='app.log', level=logging.DEBUG)

# Now, you can log messages at different levels, and they will be saved to the 'app.log' file
logging.debug("This is a debug message")
logging.info("This is an info message")
logging.warning("This is a warning message")
logging.error("This is an error message")
logging.critical("This is a critical message")

If you want to store log file at desired place then use the below code :

import logging

# Specify the absolute path for the log file
log_file_path = '/path/to/your/desired/directory/app.log'

# Configure the basic logging settings with the custom file path
logging.basicConfig(filename=log_file_path, level=logging.DEBUG)

# Now, log messages will be saved to the custom location
logging.debug("This is a debug message")
logging.info("This is an info message")

In this example, replace '/path/to/your/desired/directory/app.log' with the absolute path where you want to save your log file. This ensures that the log file is stored in the specified directory rather than the default location.

Formatting logs in Python :

You can specify a custom format for log messages using logging.basicConfig with the format parameter. Here's how to configure the logging to include date, time, level name, module name, line number, and the log message:

import logging

# Define a custom log format with date and time formatting
log_format = "%(asctime)s - %(levelname)s - %(module)s - %(lineno)d - %(message)s"
date_format = "%Y-%m-%d %H:%M:%S"

# Configure logging with the custom format and date/time format
logging.basicConfig(format=log_format, datefmt=date_format, level=logging.DEBUG)

# Log a message
logging.debug("This is a debug message")

Creating a logger object :

To create a logger object in Python’s logging module, you can use the logging.getLogger() method. Here's an example of how to create a logger object:

import logging

# Create a logger object
logger = logging.getLogger("my_logger")

# Configure the logger level to capture messages of INFO level and above
logger.setLevel(logging.INFO)

# Log messages
logger.debug("This is a debug message (won't be displayed)")
logger.info("This is an info message (will be displayed)")

In this simplified example, we create a logger object, set its log level to INFO, and then directly log messages using the logger.

Logging Exception details in Python :

In Python, you can use the logging module to log exception details, including the exception type, message, and traceback. Here's how you can do it:

import logging

# Configure the logging settings
logging.basicConfig(filename='error.log', level=logging.ERROR, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s')

try:
    # Code that may raise an exception
    result = 10 / 0
except Exception as e:
    # Log the exception details
    logging.error("An exception occurred: %s", str(e), exc_info=True)

In this example:

1. We configure the logging settings to capture log messages at the ERROR level or higher. We also specify a log file and a log message format.
2. We have a try block where you place the code that may raise an exception.
3. Inside the except block, when an exception occurs, we log the exception details using logging.error. The exc_info=True parameter tells the logger to include the exception information in the log message.

The log entry will include the exception type, the exception message, and a traceback showing where the exception occurred. This is helpful for debugging and diagnosing issues in your code.

I hope this article has been valuable in addressing your logging-related questions. If you found it helpful, please consider showing your appreciation by liking my work. Thank you for taking the time to read this article.😀