Beyond grep: Practical Logging and Metrics

Your Python applications are running but you’re wondering what they are doing? Your only clue about their current state is the server load after ssh-ing into the servers? Let’s change that!

So far, I’ve held it at PiterPy 2015 in Saint Petersburg, Russia, PyCon US 2015 in Montreal (short version), Canada, and EuroPython in Bilbao, Spain.

Video of the EuroPython version:

Slides of the EuroPython version:


Five Easy Steps To Get Started

  1. Get a free Sentry account.
  2. Install Raven into your project.
  3. Configure logging to forward exceptions to Sentry. [Bonus: look at other integrations that are just as easy.]
  4. Install a time series database:
  5. Use uWSGI and tell it to ship its metrics1 to your database.

Now you don’t miss any errors anymore and you have a general idea of what your system is doing. You can start refining.

Errors

Metrics

Metrics are the only way to get your job done. —Jeff Hodges

General

Collecting

Storing & Viewing

Logging

Credits

Footnotes

  1. In order to build uWSGI together with the StatsD plugin, you need to set UWSGI_EMBED_PLUGINS="stats_pusher_statsd" while pip-installing it. Be careful if you pre-wheel your dependencies. For Graphite/Carbon support, no special action is necessary.