Error handling and logging in Python: fail clearly and leave a trail

Programs fail: files go missing, networks drop, input is wrong. Good error handling decides what the program does when that happens, and good logging records what happened so you can fix it later. The code and output below were run with Python 3 before publishing.

Rules for exceptions

1. Catch specific errors, not everything. except FileNotFoundError is clear; except Exception hides bugs, and a bare except: also catches Ctrl+C.
2. Catch where you can do something useful: retry, use a default, show a message. Otherwise let it travel up.
3. Wrap low-level errors in your own exception with raise ... from e. Callers get a meaningful error, and the original is kept as the cause.
4. Never silence an error with except: pass.
5. Clean up with with (files, locks, connections) so the resource closes even when something fails.

Log levels

Level	Use it for
DEBUG	detail for developers, off in production
INFO	normal events (service started)
WARNING	something odd, but it continued
ERROR	an operation failed
CRITICAL	the program cannot continue

Set the level once at start-up. Messages below it are dropped.

A tested example

import json, logging, sys

class ConfigError(Exception):
    """Raised when the configuration file is unusable."""

def load_config(path):
    try:
        with open(path) as f:
            return json.load(f)
    except FileNotFoundError as e:
        raise ConfigError(f"config file not found: {path}") from e
    except json.JSONDecodeError as e:
        raise ConfigError(f"config is not valid JSON: {e.msg} at line {e.lineno}") from e

logging.basicConfig(stream=sys.stdout, level=logging.INFO, format="%(levelname)s %(name)s: %(message)s")
log = logging.getLogger("app")

def main(path):
    try:
        cfg = load_config(path)
    except ConfigError as e:
        log.error("cannot start: %s", e)
        log.debug("this debug line is hidden at INFO level")
        return 1
    log.info("started with %d settings", len(cfg))
    return 0

def risky():
    try:
        1 / 0
    except ZeroDivisionError:
        log.exception("calculation failed")     # logs the message and the traceback

INFO app: started with 2 settings
exit 0
ERROR app: cannot start: config is not valid JSON: Illegal trailing comma before end of object at line 1
exit 1
ERROR app: cannot start: config file not found: missing.json
exit 1
ERROR app: calculation failed
Traceback (most recent call last):
  File "app.py", line 30, in risky
    1 / 0
    ~~^~~
ZeroDivisionError: division by zero

What happened:

• The good file started normally and logged at INFO.
• A bad file and a missing file both became one clear ConfigError, handled in one place that logs an ERROR and returns an exit code.
• The DEBUG line never appeared, because the level was INFO.
• log.exception inside an except block recorded the message and the traceback.

Good logging habits

• Log events with context: order id, user id, file name. "Failed" alone is not useful.
• Use lazy formatting, log.error("cannot start: %s", e), not string building.
• Never log secrets: passwords, tokens, card numbers or full personal data.
• Log an error once, at the place that handles it, not at every layer.
• Send logs to standard output in containers, and let the platform collect them.

Common mistakes

• except Exception: pass, or catching so broadly that real bugs vanish.
• Returning None or -1 for errors that the caller will forget to check.
• Logging and re-raising at every layer, producing the same error ten times.
• Putting too much inside one try block, so you cannot tell which line failed.
• Using print for diagnostics in services. You lose levels, timestamps and routing.

Main idea: catch specific exceptions where you can act, wrap them with raise ... from, and log with the right level and enough context, including log.exception for tracebacks.

---

Programming roadmaps series, Part 3 (writing better Python): 4 of 5. Previous: async/await in Python: asyncio explained with timings.