Python Logging – Best Practices

The python logging module offers a wide variety of logging options and handlers.  One thing missing from the documentation is when to use each level.

A quick foreword

You really should familiarize yourself with the logging package.  How to create new loggers (I find creating them by module very useful).  There are many ways to configure logging, I tend to like dictConfig from logging.config (but start off with basicConfig form logging).

A Word on Optimal Setups

I prefer to setup my logging with each module having its own logger.  This allows me to configure logging levels at a package and/or module level.  I typically do the following in each module to create a logger.

from logging import getLogger
...
logger = getLogger(__name__)
...

Assuming my package structure consists of the following:
– foo (package)
—– core (module)
—– bar (module)
We can configure varying levels of logging for each element, as seen in the following snippet from a dictConfig.

...
'loggers': {
    '': {
        'handlers': ['default'],
        'level': 'INFO',
        'propagate': True
    },
    'foo': {
        'handlers': ['default'],
        'level': 'WARNING',
        'propagate': True
    },
    'foo.bar': {
        'handlers': ['default'],
        'level': 'DEBUG',
        'propagate': True
    },
}
...

In this example, the root ( ” ) logger (those not configured by any other settings) reports INFO level and up messages.  With the exception of the bar module, the foo package only reports WARNING level and up messages.  The bar module is set to a more verbose DEBUG level, to show information needed for debugging.

Selecting A Log Message Level

Out of the box, there are six default logging levels recognized by the logging module, most are self-explanatory.  I’ll just make some notes about usage.  (From here on out, I’ll refer to my logging instance as logger.)
For general status messages, you should use logger.info (INFO).  For errors, use either logger.critical (CRITICAL) or logger.error (ERROR).  For all exceptions, use logger.exception (ERROR).  logger.exception will automatically include stack trace information about the exception for you in the log. When you want verbose debugging information, use logging.debug (DEBUG)

In Closing

• Use the logging module instead of print statements.
• Always use logger.exception for logging exceptions.
• Favor logger.debug for verbose log statements.
• Favor logger.info for most other log statements (with the exception of errors).
• Don’t forget that each of the logging functions uses C-style formatting.