Why it matters how you log in python

Mike Taylor
Apr 20 · 3 min read
Image for post
Image for post

Logging in python is something that won’t matter to you until it matters. If you’ve ever seen a message like one of these, I’m here to explain why it’s a bad idea to ignore them:

Image for post
Image for post
Common logging pylint errors
W1201: Specify string format arguments as logging function parameters (logging-not-lazy)W1202: Use % formatting in logging functions and pass the % parameters as arguments (logging-format-interpolation)

When you’re building a production application, logging is never supposed to be disruptive to the application’s function. If you’re not writing code that others have to support, you may not have encountered this problem before — but when other people need to support and troubleshoot your code (Or even if YOU have to support and troubleshoot your code), logging will make that a million times better. Here is an example of where you might hit a snag that you can avoid:

import logging
logger = logging.getLogger()
def bad_call_with_problem():
try:
logger.error('haha %d' % None)
except:
print("Raised an error.")
raise
def good_call_with_problem():
try:
logger.error('haha %d', None)
except:
print("This will never be raised.")
raise

Non-Lazy Logging

Image for post
Image for post

The difference is subtle but to Python important. If you’ve read the documentation about python’s logging module you may have seen that logging in python is lazy — this is only the case if you’re actually using the logging call correctly.

In the first example, there is one line of code that is actually doing two things, and the first action raises an assertion, which prevents logging from being lazy (i.e. only evaluating things passed to the logging call if it needs to be emitted):

logger.error('haha %d' % None)

This is equivalent to:

message = 'haha %d' % None
logger.error(message)

This is NOT a contrived example — it’s literally an error we were troubleshooting in production today that caused a worker to restart. In the first line, we are trying to define a message, but there is a TypeError while we’re defining that message. The habit of building your logging message within the logging call is common for new loggers. Hell — it’s common for me, and I know the pitfalls! The upside is that logging handles it well *if you do it right*.

Lazy Logging

The second example in our logging functions above is actually using logging’s “lazy” logging to evaluate — but it has an added benefit: Logging that’s called correctly won’t actually raise an assertion if there’s a problem building your logging message. There’s one obscure line in the logging documentation that describes this:

The message is actually a format string, which may contain the standard string substitution syntax of %s, %d, %f, and so on. The rest of their arguments is a list of objects that correspond with the substitution fields in the message.

So putting two and two together, you get this sucker:

logger.error('haha %d', None)

What the logging module will do in this case is log the error that occurred when trying to log the message. That means you get the messages in your logs, and you don’t break your production application via logging.

Trust Your Linter

If you’re seeing those linting errors — don’t ignore them! They’re not outdated and they aren’t useless — they will prevent you from making your logging call accidentally break your application!

The Startup

Medium's largest active publication, followed by +729K people. Follow to join our community.

Medium is an open platform where 170 million readers come to find insightful and dynamic thinking. Here, expert and undiscovered voices alike dive into the heart of any topic and bring new ideas to the surface. Learn more

Follow the writers, publications, and topics that matter to you, and you’ll see them on your homepage and in your inbox. Explore

If you have a story to tell, knowledge to share, or a perspective to offer — welcome home. It’s easy and free to post your thinking on any topic. Write on Medium

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store