Python: Using the logging lib

In this post we are going to use the logging lib to write messages, in production we are going to write to a Rotating File and in development we are going to write to the console.

Create a logger.py file

The first step is to create a logger file, in my case i’m going to create it in the src folder:

mkdir src
cd src
touch __init__.py
touch logger.py

Shell

And let’s create a main.py file in the parent folder to test the logger.

cd ..
touch main.py

Shell

Code the logger

Let’s read the environment variables to determinate if we are in development and if the user wants to use a specific folder to store the logs.

If we are working on development we are going to use a logging handler that writes to the console and we want to log everything above DEBUG level.

In production we just want to log the messages above INFO and directly to a file.

In the src/logger.py file add this:

import logging
import os
from logging.handlers import RotatingFileHandler

ENV = os.getenv('ENV', 'DEV')
LOG_FOLDER = os.getenv('LOG_FOLDER', '/tmp')

logger = logging.getLogger()

if ENV == 'DEV':
    # Console logger
    handler = logging.StreamHandler()
    formatter = logging.Formatter('[%(asctime)s][%(levelname)s]: %(message)s', '%H:%M:%S')
    level = logging.DEBUG
else:
    # Rotating file handler
    handler = RotatingFileHandler(f'{LOG_FOLDER}/my_app.log', maxBytes=2000, backupCount=10)
    formatter = logging.Formatter('%(asctime)s,%(msecs)03d;%(levelname)s;%(message)s', '%Y-%m-%d %H:%M:%S')
    level = logging.INFO

logger.setLevel(level)
handler.setFormatter(formatter)
logger.addHandler(handler)

Python

src/logger.py

In the main.py file add this:

import time

from src.logger import logger

if __name__ == '__main__':
    logger.info('Info message')
    time.sleep(0.1)
    logger.debug('Debug message')

Python

main.py

NOTE: you can use the logger anywhere just importing logger from src.logger

Test the logger:

Run the main.py file without any option and the output should be like this:

$ python main.py 
[13:11:51][INFO]: Info message
[13:11:51][DEBUG]: Debug message

Shell

Run the main.py file setting the environment as production, the log will be in a file and without the debug info.

$ ENV=PROD python main.py 
$ cat /tmp/my_app.log 
2021-05-16 13:13:20,353;INFO;Info message

Shell