Why is there a new Event Log Format?

Perhaps the most important question is: Why is there a need for a new format? The "old" format was used over a decade from Windows NT 3.5 up to Windows Server 2003. Of course some smaller improvements were made over the time, but the binary file format remained the same.

NT: One major drawback of the old NT event logging system is the need to map the whole log file into memory. So large logs waste precious address space in a region which is also used for inter-process communication and shared memory. Now think what this for an Exchange server or the security log of a busy domain controller.

Vista: Log files now consist of a small header which is followed by a series of chunks. Chunks are self-contained. No event record will extend over the boundary between two chunks. So only the current chunk (64 kiB) and the file header (4 kiB) have to be mapped into memory. This significantly reduces the impact on system resources.

NT: There's no built in way to collect logs on a central monitoring station. Of course you can open an RPC connection over the net and display remote logs in the event viewer. But there's no way to consolidate logs and show the application logs of all monitored systems in one view for example. There is some third party software to fill in this gap. Usually it registers notification callbacks which are then called upon every event. This again has an impact on performance.

Vista: The new logging subsystem natively allows forwarding of event messages.

NT: The event viewer supports filtering by event type and source, category, the event id number, the affected user, the originating computer and a date/time interval. But there's no way to filter records based on the event message text or the binary data which may be associated with an event record. In addition the view hardly can't be tailored to the user's needs. One can move the fields around and flip the sort order, that's it. A (static) detailed view of the event record requires a double click. Log files can be saved in the proprietary EVT format or as tab delimited text.

Vista: The log now is based on XML technology. Queries are formulated as XPath statements. The verbosity and appearance of event records can be customised by XML Stylesheets (XSL) and XML Transformations (XSLT). Consequently the data is stored in XML format. As XML is a very "talkative" format, Microsoft decided to encode the data in a tokenized and slightly compressed format. This format is proprietary and (up to now) undocumented.

So, what does this all mean for day-to-day forensic examinations? The new format is beneficial for IT operations. The new logging system is expected to have a smaller impact on system performance than the old one. So chances are, that logging will be activated on more systems and at a higher degree of verbosity. In addition it's becoming easier to forward event records in real time to a trusted log host. Good.

On the other hand tools have to be adopted to the new and undocumented file format. It reminds me of the times when there were no forensically sound parsers available for the old EVT format. But it's not only a matter of tools. Some problems might persist, as they seem to be immanent to the new format. For example, some essential information like frequently used strings and XML templates is stored at the beginning of each chunk. When you'll find an event record in file slack, it's probably towards the end of that chunk. The record will refer back to the chunk's header - which is likely to be overwritten again by other data at the time of examination. So the whole record might be meaningless.

I'm planning to release some more information regarding the new Windows Event Log file format over the next time. For now I recommend to glance over the information available at the Microsoft Developer Network. Caveat: The new logging service is called "Windows Event Log" while the old one is referred to as "Event Logging".