Log Digest Generator

A great deal of information for monitoring can be provided by simple means, such as CGI scripts or Zope DTML pages. To facilitate generating more data, however, a log digest generator is planned as the second stage of the project (in true UML iterative lifecycle fashion [Mul97], the stages will be revisited repeatedly as the system evolves).

The first version of the digest generator will be written in Python; later versions will most likely use C++ or C for enhanced performance, possibly augmented with Fortran if the mathematical library requirements warrant.

The goal of the log digest generator is to scan server logs looking for patterns. It will use a regular expression and numerical expression filter just like MVM. Results will be written to a text file, which will then be served out by other means (e.g. an HTTP server) to the MVM clients. An additional pattern will group the digests of the log entries into "zones". This will break down the data in a system into several labeled sets of numbers, which can be adjusted to be detailed but compact enough to support efficient monitoring.