One of the problems that one tends to find when attempting to diagnose a problem using logs is the sheer volume of information that one has to digest. I've been thinking about that a little over the last couple of days, and it seems to me that with a little work you could display logs in tree form, like a call stack, which would give you the ability to do many of the kinds of things you can only do with a profiler right now, such as:
- Track down the hibernate logging that relates to a particular HTTP request so that you can diagnose a tricky persistence problem.
- Analyze the number of SQL calls made by a single HTTP request.
- Get a rough outline of the time involved at each layer of a request.
- Judicious application of the nested diagnostic context in Log4J or Logback to store a sense of the layer. This is probably done easiest with an aspect that could be applied at various tiers (services, daos, etc.)
- Some kind of request-tracking key, whether this is the thread, or some kind of context-free identifier stored in the mapped diagnostic context. Again, this could be applied via an aspect.
- Better log-parsing/viewing tooling. Using the above pieces of information, you could display logs as a tree, allow for querying, etc.