Integrating logging with diverse notification services

tl;dr: Nowadays, there is a growing number of ways in which people prefer to receive notifications about happenings of interest. While Python logging cannot provide, out-of-the-box, handlers that allow you to send notifications to all of these different systems, the basic functionality provided in the stdlib makes it fairly easy for developers to support the notification mechanisms that are preferred by their audiences. This post will give some examples of how this can be economically achieved.

Logging is basically about getting information about things that happen in your software (whether application or library) out to an interested audience. The diversity of ways in which that audience wants to receive information is growing all the time: from the earliest days of being limited to email and pagers, people have progressed to a using a plethora of social networks, IM and mobile phone platforms, different desktop notification systems - and the list keeps growing. Even if the number of basic types of notification method grows slowly, the number of individual instances of those types can grow faster. For example, if you just look at a subset of desktop notification methods, you have Growl (OS X, Windows), Snarl (Windows), libnotify and mumbles (Linux). The Wikipedia page for Instant messaging lists (at the time of writing) over 20 different IM systems, some of which are only used in particular parts of the world. Likewise, for social networks one might think first of Facebook, Bebo or Twitter, but there are many others which are very popular, but in different parts of the world (for example, Orkut in South and Central America and parts of Asia).

How can you, as a developer who uses logging in your library or application, take account of your users’ preferences when getting information out to them? Of course, the large majority of logging messages will be sent to console or file. However, there will be certain messages in the ERROR or CRITICAL category which need to be sent to people so that urgent action to be taken. These have generally been sent by email, but there are circumstances when you might want to use alternative mechanisms (or even multiple mechanisms) to get information out so that it can be received as soon as possible and (hopefully) acted on in a timely fashion.

Clearly, it’s not practical for the logging package in the stdlib to provide native support for the myriad specialized ways of sending notifications which exist today. Logging currently provides 14 handlers (not including base classes), just about all of which relate to basic infrastructure communication or storage mechanisms:

NullHandler	Used by libraries to avoid misconfiguration messages	StreamHandler	Used to write to streams
FileHandler	Used to write to disk files	RotatingFileHandler	Used to write to size-based rotating log files
TimedRotatingFileHandler	Used to write to time-based rotating log files	WatchedFileHandler	A FileHandler which supports external log file rotation mechanisms
SocketHandler	Used to write to TCP sockets	DatagramHandler	Used to write to UDP sockets
SysLogHandler	Used to write to a syslog daemon via either UDP or Unix domain sockets	SMTPHandler	Used to send emails
NTEventLogHandler	Used to write to Windows NT event logs	HTTPHandler	Used to send to arbitrary web sites
MemoryHandler	Used to buffer up events and process in batches	QueueHandler (3.2)	Used to send to in-process or multiprocessing queues

As you can see, all of these are very generic and relate to storage mechanisms and protocols which are unlikely to go out of fashion any time soon.

However, you don’t need to worry unduly if you want to send notifications using some of the newer facilities available today. Of course you can use something like HTTPHandler (subclassing it if necessary) to send information to any web site, but you may not see any support for e.g. desktop notification systems. How can logging provide this?

It doesn’t make much sense for me to put in “native” support for multiple notification protocols into the stdlib, for a number of reasons:

• The methods, protocols and sites which are popular today may not be popular tomorrow. Any support added for them in the stdlib would have to live on for a very long time, even if their popularity waned.
• The methods, protocols and sites may keep changing how they work, leading to an increased maintenance burden.
• Python is used globally, and people would (understandably) expect to have support for popular systems in their neck of the woods – which may not be easy if, for example, all the documentation about the protocols used is inadequate or in a foreign language I’m not familiar with.
• Native support of external protocols may require additional dependencies, which is not an appropriate burden for Python to carry, or use third-party modules which give rise to warnings or errors (for example, some Python libraries for Growl use the deprecated md5 module, which can give rise to deprecation warnings – this requires code in the libraries or applications using them to turn those warnings off).
• Even if native support is provided for various systems, there are many options that are available when using such systems.The API for stdlib facilities to support such flexibility in options may be unwieldy, and even if that isn’t the case, it will certainly evolve over time, likely requiring you to subclass and change things, or fork and modify if the original classes weren’t reusable via subclassing. So, any idea of a long-lived “out-of-the-box” solution which requires no work from you is possibly an impossible dream :-(

Of course, developers could write “native” handlers for systems and upload them to PyPI (or post them elsewhere), and indeed people have done this in the past. That may be an approach that works for you; but anyone can upload their offerings to PyPI, which leads to what has been called the selection problem.

Nevertheless, some people might be thinking that some support for these newer notification systems would be nice to have with stdlib logging. What’s a good way of achieving this? We should take inspiration from the Unix philosophy:

Write programs that do one thing and do it well. Write programs to work together.

To apply this philosophy to the problem at hand, we can take advantage of the fact that many of these notification systems have command line interfaces! This is true at least on Unix and Unix-like systems, meaning primarily Linux and OS X. Here are some examples:

Twitter

Before Twitter changed their authentication system to disallow Basic authentication, you could just use curl like this: curl –u username:password -d status="Hello, world!" http://twitter.com/statuses/update.json Now that Twitter has switched to using OAuth, things are not quite so easy, but they can be after a small amount of one-time set-up. One-time setup We’ll use Mike Verdone’s Python Twitter Tools. It’s as easy as doing easy_install twitter and waiting for the installation to complete. Then, just type twitter in the shell. You’ll see something like this: vinay@zeta-lucid:~/tools$ twitter Hi there! We're gonna get you all set up to use the Command-Line Tool. In the web browser window that opens please choose to Allow access. Copy the PIN number that appears on the next page and paste or type it here: Please enter the PIN:  A browser window opens on the Twitter website, and you enter your username and password into the presented form and click “Allow” to submit. A numeric PIN code is displayed in the response page, which you type or paste into the shell at the above prompt. You’re then told: That's it! Your authorization keys have been written to /home/vinay/.twitter_oauth. and that’s the end of the one-time setup. You can type twitter –-help to find the commands options available via the tool; they’ll fit most people’s needs. Regular usage Once you’ve done the above setup, you can just use a command line like twitter set hello to update your status on Twitter:

libnotify

To use libnotify from the command-line, use the notify-send program, which is part of libnotify. (On Ubuntu, you should be able to do sudo apt-get install notify-bin to install it). The man page will tell you all you need to know, but a simple usage would be notify-send title message –I icon_path You can set the urgency, time limit etc. through command-line parameters.

Growl

You can use the growlnotify program which is part of Growl. This has numerous options, but a simple usage would be growlnotify –n appname –m message –I iconpath You can set priority, stickiness etc. using various command-line parameters.

Mumbles

If you use Mumbles, you can use the mumbles-send script which comes with it, for example as follows. mumbles-send title message

Snarl

If you use Snarl, you can use the Snarl_CMD.exe program, for example as follows. Snarl_CMD nShowMessage TIME TITLE BODY [iconPATH]

Other Web-based notification services (for example, ticketing systems)

For Web-based services in general, you may be able to use logging.handlers.HTTPHandler either directly or via subclassing, but if you want to look at a command-line based solution which is similar to the ones above, you can use the excellent curl command-line tool: curl is a command line tool for transferring data with URL syntax, supporting DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, TELNET and TFTP. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, cookies, user+password authentication (Basic, Digest, NTLM, Negotiate, kerberos...), file transfer resume, proxy tunnelling and a busload of other useful tricks.

You might be thinking that it seems a bit clunky in this day and age to be using the command-line in this way, and you may be right to think so. But it represents a straightforward, practical approach to integrating logging with many popular notification services in a way which doesn’t load the stdlib with unnecessary baggage. Plus, it may well involve less work for you: for example, you might be able to use a single command-line handler class, configured with various different command lines for different notification methods. Less code is good! And while in the past there was less than an ideal level of support under Windows for command-line tools, perhaps the development of PowerShell shows that the situation is changing for the better.

Perhaps you’re concerned about the overhead of running external command-line programs in subprocesses and the delays this might cause; in that case, you might want to look at this earlier post which explains how you can use QueueHandler and QueueListener classes to delegate the heavy lifting to separate threads or processes, leaving your Web application threads (and other performance-critical threads) as responsive as possible.

One possible realization of a command-line handler might be:

class CommandLineHandler(object):
    """
    A class which executes parametrized command lines in a separate process.
    """
    def __init__(self, cmdline):
        """
        Initialize an instance with a command-line template.
        
        The template consists of a set of strings, some of which may contain
        variable content merged in from a LogRecord.
        
        The LogRecord merge will be done before executing as a command.
        """
        self.cmdline = cmdline
        
    def handle(self, record):
        """
        Handle a record.
        
        This just merges its data into the command-line template and
        executes the resulting command.
        """
        cmdline = [c % record.__dict__ for c in self.cmdline]
        self.execute(cmdline)
        
    def execute(self, cmdline):
        """
        Execute the specified command
        """
        import subprocess
        p = subprocess.Popen(cmdline)
        p.wait()

Recall from the earlier post that QueueListener can be given (as a handler) any object which has a handle method: so a CommandLineHandler instance could be used as a handler.

Of course, you should very careful with any facility which executes potentially arbitrary commands on your server, making sure that command line configuration is covered by security diligence. An example command line as passed to the constructor might be ['notify-send', '%(appname)s - %(name)s', '%(message)s'] where appname is a context value inserted in the LogRecord (e.g. by a Filter, see this documentation).

I’d be grateful to get any feedback about the ideas expressed in this post. If you got this far, thanks for reading :-)

---