Write Better Python Scripts
Use these three built-in modules to format your scripts the Pythonic way.
In this article, I’ll show you three scripting conventions and corresponding built-in modules to help better format your Python scripts. These modules are designed to adhere to the DRY (don’t repeat yourself) principle and are there to improve the quality of your code and scripts!
In short, we’ll go over the following three components:
- Use of an
- How to parse command-line arguments with
- How to debug with the
logging()module instead of
Always Use an ifmain
ifmain refers to the last lines of code in a Python script that you often see:
if __name__ == "__main__":. When Python scripts are parsed by the Python interpreter, a few top-level variables are set, including the
Often, you’re writing a Python script that you’re reading from an interactive prompt like so:
$ python3 file.py. And when you do so, the
__name__ variable is set to
In concordance with the DRY principle, you should want to write functions that you can reuse in other Python scripts. You do this by importing a module (a class or function) from a Python script with
import modular_function from file.py:
Now here’s the kicker: If you import a function, the
__name__ variable will be set to the name of the function (in this case,
modular_function). The code under the
ifmain will thus only be executed if the main program is executed — for instance, from the Python interpreter
This enables us to write
.py files that can both be used by other scripts and also be run as the main program. I often use this when creating a machine learning pipeline with different modular steps, such as pre-processing or training. It is also useful when writing unit tests in a testing framework that imports modules and runs special test functions on test databases. In any case, you’re still able to import the entire file and run the main function:
Parse Command-Line Arguments With argparse
Positional command-line arguments, such as
$ python3 file.py arg1 arg2, can be parsed with
sys.argv. Python also comes with the built-in
argparse module, which automatically generates help and usage messages and issues errors when users give the program invalid arguments.
You can then use these arguments in your
main function by calling the argument by its value (e.g.
def main():with open(args.i, "r") as inp_file:
# do something meaningful
Log Errors Instead of Printing
Printing errors can be useful for debugging, but using Python’s built-in
logging module enables all Python modules to participate in logging, so your application log can include your messages integrated with messages from third-party modules.
logging module provides several
LEVELS ranging from least severe to most severe:
DEBUG— Detailed information, typically of interest only when diagnosing problems.
INFO— Confirmation that things are working as expected.
WARNING— An indication that something unexpected happened or will soon happen (e.g. “disk space low”). The software is still working as expected.
ERROR— Due to a more serious problem, the software has not been able to perform some function.
CRITICAL— A serious error indicating that the program itself may be unable to continue running.
The level of the
basicConfig indicates which logs to include. If, for instance, you use
info(), but the logging level is set to
WARNING, these logs won’t show up. Instead of streaming our logs to the standard output (equal to printing them), you can also use the configurations to output the logs to an
Putting It All Together
With our knowledge of
argparse and the
logging module, we can now create a standard script format in which we can set the logging
LEVEL of the script as one of the command-line arguments.
This way, you can call the script from a command line with
$ python3 format.py -v 2 and set the logging level with the argument
-v. You’re now able to see the errors for debugging if you want or just keep the output clean!
add_pargument takes a
default parameter, so even if you call the script without argument
-v, it will still default to
logging.WARNING (line 16).