PYCON 2017: Python Logging in Production

It is the only talk I have jot notes

Why we need to log everything?

Log is the final fallback of any unexpected requirements:

  • Provide support for new business decide which does not know when develop the system
  • Audit

Log system structure

  • Loggers
    • Filter
    • Handlers
      • Filter
      • Formatter

Components

  • Logger
    • A channel of a set of log events
    • Classified by the nature of log events
    • Examples
      • Database related logger
      • Feature related logger
      • Web access logger
  • Filter
    • Filter out unwant events
      • log levels (usually 5 levels) are handled here
    • Sanitize data
      • Remove sensitive data (e.g. Password)
      • Mask personal data (Email, phone etc)
  • Handler
    • Dispatch logging event to different destination
    • Can integrate bug tracker inside logger
    • Types
      • Log file
      • External log services
        • Papertrail, Bugsnag, Slack etc
  • Formatter
    • The function which transform log events into different format
      • JSON, XML, TXT etc

logs and reporting

  • Log important data change
  • Log request/response
    • Can use log to check check slow query
  • Define a fixed format or logs
  • Logging code should have no side effect
  • Integrate bug tracker inside logger