On Thursday, August 22, 2002, at 02:45 , Ogle Ron (Rennes) wrote: > Unless, you get rid of UDP totally for a transport for log messages > (completely destroying backward syslog compatibility), you are stuck > with a > definite length log message. That means the more things you put in the > message to help us poor humans out, the less space is available for > content/message which is the reason for logging in the first place. First, I have to ask just how common are messages approaching 64K? A backwards compatible XML wrapper is going to add a few hundred bytes of overhead, not tens of KB. Since a backwards compatible syslogd would merely be wrapping the message, it's not going to include any of the extensions - the whole purpose is simply to avoid having to do all of the parsing and reformatting later on the analysis side: Aug 22 10:43:34 firewall_box %PIX-4-106023: Deny tcp src outside:192.168.1.1/3760 dst inside:192.168.2.1/25 by access-group "list_out" becomes <event time="1234567890" srchost="firewall_box" facility="%PIX-4-106023">Deny tcp src outside:192.168.1.1/3760 dst inside:192.168.2.1/25 by access-group "list_out"</event> Increasing from 135 characters to 172 does not seem like an unworkable increase. None of which really matters since you'd implement this in a syslog replacement somewhere and that would either be receiving traditional UDP and translating at that point or forwarding local messages after encapsulating them, which in either case would mean that you can do the sane thing and use TCP which would also give some much needed reliability and performance - UDP's already disappearing from syslogd replacements for the same reasons. > XML is good for supporting a large variation in data types with varying > structure. Logs are very structured with little or no variation in data > types. The log content could be varied, but again it is still well I have to disagree here. Syslog is not very structured at all - you get facility, priority and a blob of text. There is no standard format for that text and you need custom parsers to handle every programs's messages. Consider OpenSSH and the commercial SSH - by nature they're extremely similar but they don't exactly log the same thing. That Cisco PIX in the example above logs in a completely different fashion than ipfw or iptables - if I want to keep track of hosts which are being accessed, I have to write some code to parse and reformat, which is a huge pain. The firewall vendors are working on a standard logging format for exactly this reason - even if you have to deal with vendor extensions in certain cases it's nice to be able to do most of your analysis without having to monkey around with Perl for awhile first converting things. Switching to a structured format would greatly simplify the process of analyzing logs in large multi-vendor environment. Any structured replacement is going to have to be extensible which in practice means something like XML or Marcus' tagged format and I have a feeling that in practice they'll end up being very similar in size and complexity. Chris _______________________________________________ LogAnalysis mailing list LogAnalysisat_private https://lists.shmoo.com/mailman/listinfo/loganalysis
This archive was generated by hypermail 2b30 : Thu Aug 22 2002 - 11:41:02 PDT