2003-01-16T16:05:03 Buck Buchanan: > The Institute for Security Technology Studies at Dartmouth College has a > paper out titled "Law Enforcement Tools and Technologies for Investigating > Cyber Attacks" and you can get it via > http://www.ists.dartmouth.edu/TAG/lena.htm. Nice pointer, thanks! > Another part suggests the need for a tool to merge multiple logs > from multiple machines into a timeline. Different folks will have different approaches they favour for this problem. My own favourite approach is to (a) canonicalize the log records to have a timestamp that collates in chronological order, at the beginning of the record; e.g. RFC 3339 / ISO 8601 canonicalized to a single uniform timezone, and with uniform precision (making a uniform-length timestamp field). Then (b) append to the timestamp record numbers (relative to the input files to be merged), to make a stable sort. (c) Concatenate and sort the lot, then (d) strip the record numbers back out. > Hopefully this tool would have the smarts to figure out the > differences between the clocks on the system supplying the logs, > and to be able to determine the clock drift. That'd be nice, but it requires more information than I've seen preserved in syslogs. The easiest fix to this problem is to just eliminate the clock skew and drift, by running good time synchronization code like ntp (which I'm not all that fond of) or clockspeed (which I prefer). Failing that, the next easiest would be to determine the clock skew by external means, then use it to retroactively adjust the logs. Clockspeed is probably the most convenient tool I know of for determining clock skew. The folks who are working on enhancing the payload of syslog are also hoping to collect the info needed for this clock-deskew process, by preserving both original log timestamp and additional stamps applied by each relay daemon. But a really totally completely robust solution to this is hard enough that I doubt it'll ever be accomplished. For folks who really, really care about the best possible timestamps for logfile entries, the best solution I know of is to synchronize all systems with clockspeed, keeping them on TAI. The "UTC" time as implemented by ntp follows the mandate of Posix, and shifts its "epoch" every time there's a leap second, making historical timestamps ambiguous during any leap second. -Bennett
This archive was generated by hypermail 2b30 : Sat Jan 18 2003 - 21:12:50 PST