Monday, 17 January 2011

A desktop log viewer

I’ve created a simple desktop application which is a viewer for viewing logging records sent from applications via the SocketHandler and DatagramHandler. It’s early days and the viewer is fairly simple, so I’ve created a screencast so that I can get some feedback on the user interface before spending too much more time on it. If people think it will be useful, I can release it under a 3-clause BSD license.

I realize that web interfaces are all the rage, but it took me less time to produce this desktop app using PyQt than an equivalent web application would have.

I developed this on Ubuntu Maverick using the bundled versions of Qt and PyQt, and it seems pretty stable. With older versions of Qt/PyQt (e.g. the versions bundled with Ubuntu Karmic and earlier) I have had segfaults with the identical code - I'm not sure where the problem is.

The code also works with a recent version of PyQt on Windows 7.

The screencast is around 3 minutes long.

The video content presented here requires JavaScript to be enabled and the latest version of the Macromedia Flash Player. If you are you using a browser with JavaScript disabled please enable it now. Otherwise, please update your version of the free Flash Player by downloading here.

I’d be grateful for any feedback about the user interface. I know it’s not especially innovative but then it’s doing a fairly simple job :-)


  1. That's a really nice interface. I especially like the logger hierarchy and the column selection.

    I don't use python at the moment, but when I do again I'll checkout your viewer!

    I wrote a file log viewer some time ago in wxPython. A parser can be specified to parse different log file formats. I use it regularly at work now. One thing I like to do is filtering by a regular expression. See for a screenshot.

    -- Gerhard

  2. @Petar: The code's not yet ready for release. As I said in the post, I'd like some UI feedback before progressing it further.

    @Gerhard: Thanks for the feedback. The viewer doesn't parse files (it's not intended to - it's intended only for capturing events via socket listeners). I'm thinking about additional filtering via regexp for the message portion, and also for searching via matching a message. BTW, thanks for pycron! I've found it a useful tool in the past.

    I wrote an early version of this viewer using wxPython. I switched for two reasons: wxWidgets is not quite so uniform across platforms (e.g. tooltips on disabled widgets) and Qt's model/view architecture makes writing the application easier. Sadly, wxWidgets doesn't seem to have anything analogous.

  3. This is better than any other "not ready for release" 0.1 version of anything I've ever seen!

    Some UI feedback: it'd be nice if the logger tree pane could be optionally hidden -- think people with small screens (netbooks, etc.).

    It'd be nice if the logger tree was initially expanded, instead of requiring multiple mouse clicks to see what loggers you have.

    It'd be nice if the name column in the bottom-right corner were initially wider, so that all the typical names fit without ellipsization.

    It'd be nice if the creation time in that pane was shown in human-readable time, instead of scientific notation.

    Some systems (*cough*zope*cough*) produce tracebacks that definitely won't fit in a tooltip. It'd be nice to have a little [...] button on the right side of that row to pop up a window with a read-only text area (to allow people to select and copy traceback text).

  4. I didn't want to give too much feedback before getting a chance to play with it, but if you want feedback before then...

    I'd suggest a pause button to pause the update of the log window. Especially if there is a high volume of messages it may make the real-time update hard to follow.

    A regexp filter for filtering log messages would also be a nice enhancement. Of course eventually people will want arbitrary types of filters, but I think filtering on message regexp is the bare minimum apart from the level.

  5. Thank you, Marius and Kamil, for your feedback.

    @Marius: I didn't make it clear in the screencast, but all the main panels are resizable because of the use of splitters. The tree is split from the master/detail on the right, and the master/detail panes can also be sized using a horizontal splitter.

    The points about the "Name" column and traceback pop-up are valid. The creation time is visible in ASCII format using the "asctime" attribute of the record (which I didn't highlight in the screencast).

    @Kamil: In practice the message update doesn't usually get in the way. It's a bit distracting at first but then all the activity happens out of sight, and all you see is slight adjustments in the scroll bar as the number of records collected goes up.

    I'm working on limiting the memory usage - at the moment the collected records are kept indefinitely (until manually cleared) but a ring buffer will probably need to be implemented before release.

    I'm not sure about filtering on regexp, because you generally want to see context around the specific message of interest - you would lose this with filtering. However, searching for a record via message content is definitely useful - I've already added a "contains" matcher, but not yet a regexp matcher.

    In general there are quite a few other things to be added before release: e.g. preferences for listening ports and other settings, limiting memory usage, providing command line options, perhaps writing records to file etc. I'll get on with these as time permits!

  6. 1. You may have a custom value for log level. I'd prefer to have a slider with numeric values and text labels at some points.

    2. Scroll should stick if you're at the bottom.

    3. IMHO details are better viewed in separate big popup message. Always hated to resize those tiny boxes.

    4. Filters: regexp

    5. Filters: hide repeated messages

    6. Filters: higlight

    7. Multiple logs in tabs/multiple views of the same log

    8. Visualize 'hit' of the log message in the right tree

    9. Take a look at Spyder - it may have some reusable components

  7. Very nice. I have been looking around at apps trying to find the visibility I wanted for me and my team members.

    As far as automatically saving to a file, I think that may be out of scope. Now, "saving selected" records to an output format (email, txt, xml) may be nice.

    I think the logger itself should send to permanent storage explicitly.

    I do like the idea of being able to open historical data from a datasource. I do not know what that datasrc would be. I have a syslog server for permanent storage now, but there is so much wiggle room for formatting messages. Do we have anything like GELF in teh python community?

  8. @Slestak: You've seen my later post describing the released viewer, right?

    Writing to log files or syslog can be done at source in addition to sending files to the viewer. To preserve maximal semantic information in history, one approach would be to append pickled records to a binary file. It would appear to be out of scope for a simple viewer, though, as you'd likely want things such as better search features, filtering functionality etc.