Chris, Could you repost in TEXT-mode? Best regards, Corey Corey J. Steele, Security Analyst Good Samaritan Society e-mail: csteele@good-sam.com voice: (605) 362-3899 >>> Chris Calabrese <chris_calabreseat_private> 08/15/01 01:49PM >>> > However, I have gotten some interest in it from time> to time, and I offer it here as a good starting> point...Oh bother, it looks like my attachments didn't make it through some step of the way.Tina, are you stripping attachments?Hmm, here's HTML. If you want this in ASCII, send me an e-mail... (it's easier to do the HTML because the Yahoo mail client reformats the text bits if they're not attachments)--------------------------XML for Secure System Logs.title { color: #990000; font-size: 22px; line-height: 22px; font-weight: bold; text-align: right; font-family: helvetica, arial, sans-serif } .filename { color: #666666; font-size: 18px; line-height: 28px; font-weight: bold; text-align: right; font-family: helvetica, arial, sans-serif } p.copyright { color: #000000; font-size: 10px; font-family: verdana, charcoal, helvetica, arial, sans-serif } p { margin-left: 2em; margin-right: 2em; } ol { margin-left: 2em; margin-right: 2em; } ul.text { margin-left: 2em; margin-right: 2em; } pre { margin-left: 3em; color: #333333 } ul.toc { color: #000000; line-height: 16px; font-family: verdana, charcoal, helvetica, arial, sans-serif } H3 { color: #333333; font-size: 16px; line-height: 16px; font-family: helvetica, arial, sans-serif } H4 { color: #000000; font-size: 14px; font-family: helvetica, arial, sans-serif } TD.header { color: #ffffff; font-size: 10px; font-family: arial, helvetica, san-serif; valign: top } TD.author-text { color: #000000; font-size: 10px; font-family: verdana, charcoal, helvetica, arial, sans-serif } TD.author { color: #000000; font-weight: bold; margin-left: 4em; font-size: 10px; font-family: verdana, charcoal, helvetica, arial, sans-serif } A:link { color: #990000; font-size: 10px; text-transform: uppercase; font-weight: bold; font-family: MS Sans Serif, verdana, charcoal, helvetica, arial, sans-serif } A:visited { color: #333333; font-weight: bold; font-size: 10px; text-transform: uppercase; font-family: MS Sans Serif, verdana, charcoal, helvetica, arial, sans-serif } A:name { color: #333333; font-weight: bold; font-size: 10px; text-transform: uppercase; font-family: MS Sans Serif, verdana, charcoal, helvetica, arial, sans-serif } .link2 { color:#ffffff; font-weight: bold; text-decoration: none; font-family: monaco, charcoal, geneva, MS Sans Serif, helvetica, monotype, verdana, sans-serif; font-size: 9px } .RFC { color:#666666; font-weight: bold; text-decoration: none; font-family: monaco, charcoal, geneva, MS Sans Serif, helvetica, monotype, verdana, sans-serif; font-size: 9px } .hotText { color:#ffffff; font-weight: normal; text-decoration: none; font-family: charcoal, monaco, geneva, MS Sans Serif, helvetica, monotype, verdana, sans-serif; font-size: 9px } TOC Network Working GroupC. J. CalabreseInternet-DraftThe SANS InstituteExpires: July 16, 2001January 15, 2001 XML for Secure System Logsdraft-calabrese-xml-log-00Status of this Memo This document is an Internet-Draft and is in full conformance with all provisions of Section 10 of RFC2026. Internet-Drafts are working documents of the Internet EngineeringTask Force (IETF), its areas, and its working groups.Note that other groups may also distribute working documents asInternet-Drafts. Internet-Drafts are draft documents valid for a maximum of six monthsand may be updated, replaced, or obsoleted by other documents at any time.It is inappropriate to use Internet-Drafts as reference material or to citethem other than as "work in progress." The list of current Internet-Drafts can be accessed athttp://www.ietf.org/ietf/1id-abstracts.txt. The list of Internet-Draft Shadow Directories can be accessed athttp://www.ietf.org/shadow.html. This Internet-Draft will expire on July 16, 2001. Copyright Notice Copyright (C) The Internet Society (2001). All Rights Reserved. Abstract This memo presents an Extensible Markup Language (XML)[16] encoding that applications may use to express logging information in a structured manner. Thus making these logs easier for computer programs to parse the the free-form data traditionally associated with Syslog[3]-based logs. --------------------------------- TOC Table of Contents1. Introduction 2. Schema Overview 3. Examples 3.1 Nested `log:EVNT' Entries 4. Deviation from Pure XML 5. Implementation Considerations 5.1 Event Filtering 5.2 Event Viewing/Reporting 5.3 Logging API's 5.4 Backward Compatibility 5.4.1 Syslog 5.4.2 NT Event Logger 5.5 Network Communications 6. Security Considerations 6.1 Correctness and Completeness of Identifying Information 6.2 Confidentiality, Integrity, and Availability of Log Messages #167; References #167; Author's Address A. XML DTD for the `log:EVNT' Element B. XML DTD for Data Types Used in the `log:EVNT' Element #167; Full Copyright Statement --------------------------------- TOC 1. Introduction One of the most important aspects of system, application, and security administration is event logging. Such logging involves capturing information about important events, possibly transmitting that information to a centralized logging facility, timestamping/filtering/manipulating the information, recording the information to persistent storage, and finally filtering, retrieval and analysis by events by an off-line program or person. This memo presents an Extensible Markup Language (XML)[16] encoding that applications may use to express logging information in a structured manner. Thus making these logs easier for computer programs to parse the the free-form data traditionally associated with Syslog[3]-based logs. This scheme is, for the most part, an "XML-ization" of the Abela and Debeaupuis' Universal Format for Logger Messages (ULM)[11]. A new version of their ULM tool has been written by Nicolas Jombart to deal with an earlier version of this XML-ized format[12]. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119[1]. --------------------------------- TOC 2. Schema Overview The basic idea is very straight forward. Each log entry is represented by a 'log:EVNT' element. Inside this element are the following attributes: log:HOST The host on which the log entry was originally generated. log:DATE The date/time at which the log was generated in 'ISO.8601:1998_Date_Format'. [2] log:SEV Severity of the event using the standard Syslog[3] as enumerated in 'log:SEV_Format' (`LOG_DEBUG', `LOG_INFO', `LOG_NOTICE', `LOG_WARNING', `LOG_ERR', `LOG_CRIT', `LOG_ALERT', `LOG_EMERG',). When operating in Windows NT environments, the native NT Event Types (`Information', `Warning', `Error', `Success Audit', and `Falure Audit') should map into `LOG_INFO', `LOG_WARNING',, `LOG_ERR', `LOG_INFO', and `LOG_INFO' respecitively. Similarly, the standard SNMP Trap Severities (`Normal', `Warning', `Minor', `Major', and `Critical') should map into `LOG_INFO', `LOG_WARNING', `LOG_ERR', `LOG_ERR', and `LOG_CRIT' respecitively. NOTE: Need references for NT and SNMP log:PROC.NAME The name of the program generating the event. Can either be a simple name (`Sendmail'), or a hierarchy identifying the program and it's Facility (`Mail/MTA/Sendmail', `Audit/Tripwire', `Kernel/FS/Buffer_Cache'). NOTE: Standard hiercharies need to be specified. log:PROC.ID An identification of the instance of the program/process that generated the event. On a Unix system, this would by the process identification (`PID') plus the thread identifier for multi-threaded processes (`2207.3'). log:PROC.PRIV A list of any security ID's/groups/labels associated with the process generating the log entry (`USR-chris, UID-748, GRP-adm, GRP-sys, GRP-users, EUID-root, EGID-kmem, DAC_READ, SYSTEM_HIGH') NOTE: Need to specify standard nonclemature for Unix, NT, and IEEE 1003.1e/2c priveleges. Also, do we really want to do it this way, or would it be better done as different attribute types, or even nested elements or something? log:PROC.MAIL An e-mail address associated with the usr/login/account the process is running as. log:PROC.TTY The TTY or other description of the user's physical connection to the host generating the event. log:EVENT.TYPE Gives a further taxonomy of the type of event that occurred. Allowed values given under 'log:Event_Type_Enumeration'. XML:LANG The language used for any textual messages in the standard XML `LangCode' format[16]. log:DUR Duration of the event being logged in 'Seconds.Decimel' format. log:SRC.ADDR (log:DST.ADDR) The network source (destination) address of any network traffic represented by this event in 'log:Address_Format'. log:SRC.PROT (log:DST.PROT) The network protocols/ports used by the source (destination) of network traffic in 'log:Protocol_Format' (`IP/TCP/25/ESMTP', `IP/ICMP/Source_Quench'). log:SRC.FQDN (log:DST.FQDN) The fully qualified domain name for the traffic source (destination). log:SRC.NAME (log:DST.NAME) Some other unique identifier for the traffic source (destination). log:SRC.USR (log:DST.USR) A user/login/account associated with the traffic source (destination). log:SRC.MAIL (log.DST.MAIL) An e-mail address associated with the traffic source (destination). log:REL.IN.ADDR, log:REL.IN.PROT, log:REL.IN.FQDN, log:REL.IN.NAME, log:REL.IN.USR, log:REL.IN.MAIL, log:REL.OUT.ADDR, log:REL.OUT.PROT, log:REL.OUT.FQDN, log:REL.OUT.NAME, log:REL.OUT.USR, log:REL.OUT.MAIL Similar to the `log:SRC.' and `log:DST.' attributes, but representing inbound and outbound traffic through a relay or proxy. log:VOL, VOL.SENT, VOL.RCVD Total data volume in octets, volume sent, and volume received. log:CNT, log:CNT.SENT, log:CNT.RCVD Total data volume in records, records sent, and records received. log:PROG.FILE, log:PROG.LINE The name of the program source file and line number within the source file. Typically used for debugging and assert messages. log:DOC Name of an accessed document/file, such as an FTP file, a newsgroup, or a URL. log:CMD Issued command that generated the event. For example: <log:EVNT ... log:CMD="/bin/rm -rf /"> log:MSG Free form message text. --------------------------------- TOC 3. Examples3.1 Nested `log:EVNT' Entries At first blush, it's not obvious why we need a MSG attribute when we could use something like: <log:EVNT ...>this is the text</log:EVNT> The reason is to allow `log:EVNT' elements to nest. One possible interpretation of nested log entries is to be able to represent complex relationships in the data. Instead, here we use nesting to collapse redundant data. For example, the following are considered equivalent: <log:EVNT log:SEV=80 log:PROC.NAME="Audit/Tripwire" log:PROC.ID=1234 log:MSG="unexpected file" log:DOC="/a/b/c"/><log:EVNT log:SEV=80 log:PROC.NAME="Audit/Tripwire" log:PROC.ID=1234 log:MSG="unexpected file" log:DOC="/a/b/d"/><log:EVNT log:SEV=80 log:PROC.NAME="Audit/Tripwire" log:PROC.ID=1234 log:MSG="unexpected file" log:DOC="/a/b/e"/><log:EVNT log:SEV=80 log:PROC.NAME="Audit/Tripwire"log:PROC.ID=1234 log:MSG="unexpected file"><log:EVNT log:DOC="/a/b/c"/><log:EVNT log:DOC="/a/b/d"/><log:EVNT log:DOC="/a/b/e"/></log:EVNT> As a result, only inner-most `log:EVNT' elements are log entries. Outer `log:EVNT' elements set attributes common to the contained entries. --------------------------------- TOC 4. Deviation from Pure XML Although XML is a good fit for the structure needed to describe system logs, it is not a perfect fit. In particular: In the interest of compactness, logs will be considered complete even if they do not contain the standard XML prolog containing the XML version, element declarations, etc. Also in the interest of compactness, log message streams not beginning with `<' will assume to be bracketed by `<log:' and `>'. This way, the existing syslog behavior of sending a single log entry inside a single UDP packet may be preserved. Some might argue that having the free-form messages as `log:MSG' attribute values is somewhat messy from an XML standpoint and that "proper" XML would be to unravel the nesting and put the message text in as the "payload" of the `log:EVNT' elements. Implementations may contain a tool to generate 100% XML compliant "documents" from log files so that the logs may be processed by standard XML tools. --------------------------------- TOC 5. Implementation Considerations5.1 Event Filtering NOTE: Need info about filtering... 5.2 Event Viewing/Reporting NOTE: Need info viewing/reporting... 5.3 Logging API's NOTE: Need info on API's, especially making sure that the TCB generates whatever information it can. Refer to sec section. 5.4 Backward Compatibility5.4.1 Syslog NOTE: Need info on this..., especially issue of including level/facility info in both XML and native syslog levels. 5.4.2 NT Event Logger NOTE: Need info on this... 5.5 Network Communications NOTE: Need info on this..., including references to Syslog WG work. --------------------------------- TOC 6. Security Considerations6.1 Correctness and Completeness of Identifying Information NOTE: TCB should/must generate log:HOST log:DATE log:SEV log:PROC.NAME log:PROC.ID log:PROC.PRIV 6.2 Confidentiality, Integrity, and Availability of Log Messages The confidentiality, integrity, and availability of log messages or logging mechanisms is beyond the scope of this memo. However, these issues are addressed in the following places. NOTE: need references here... --------------------------------- TOC References[1]Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", RFC 2119, BCP 14, March 1997.[2]International Standards Organization, "Data elements and interchange formats - Information interchange - Representation of dates and times", International Standard ISO 8601:1988, available from http://www.iso.ch/markete/8601.pdf, 1988.[3]University of California Computer Systems Research Group, "Unix Programmer's Manual, 4.3 Berkeley Software Distribution, Virtual VAX-11 Version", April 1986.[4]U.S. Department of Defense, Assistant Secretary of Defense for Command, Control, Communications, and Intelligence, "Department of Defense Trusted Computer System Evaluation Criteria", DoD 5200.28-STD, available from http://www.radium.ncsc.mil/tpep/library/rainbow/5200.28-STD.html, December 1985.[5]International Standards Organization, "Common Criteria for Information Technology Security Evaluation Version 2.1", International Standard ISO/IEC 15408:1999, available from http://www.radium.ncsc.mil/tpep/library/ccitse/ccitse.html, August 1999.[6]U.S. National Security Agency, Information Systems Security Organization, "Labeled Security Protection Profile", available from http://www.radium.ncsc.mil/tpep/library/protection_profiles/index.html, October 1999.[7]U.S. National Security Agency, Information Systems Security Organization, "Controlled Access Protection Profile", available from http://www.radium.ncsc.mil/tpep/library/protection_profiles/index.html, October 1999.[8]Lonvick, C.M., "syslog Protocol", draft-ietf-syslog-syslog-03 (work in progress), January 2001.[9]Schneier, B. and J. Kelsey, "Cryptographic Support for Secure Logs on Untrusted Machines", The Seventh USENIX Security Symposium Proceedings, USENIX Press pp. 53-62, available from http://www.counterpane.com/secure-logs.html, January 1998.[10]Schneier, B. and J. Kelsey, "Minimizing Bandwidth for Remote Access to Cryptographically Protected Audit Logs", Second International Workshop on the Recent Ad vances inIntrusion Detection (RAID '99), available from http://www.counterpane.com/auditlog2.html, September 1999.[11]Abela, J. and T. Debeaupuis, "Universal Format for Logger Messages", May 1999.[12]Jombart, N., "Private correspondence", May 2000.[13]New, D. and M.T. Rose, "Reliable Delivery for Syslog", draft-ietf-syslog-reliable-02 (work in progress), November 2000.[14]Kelsey, J., "Syslog-Auth Protocol", draft-ietf-syslog-auth-00 (work in progress), December 2000.[15]Kelsey, J., "Syslog-Sign Protocol", draft-ietf-syslog-sign-00 (work in progress), December 2000.[16]World Wide Web Consortium, "Extensible Markup Language (XML) 1.0.", available from http://www.w3.org/TR/REC-xml, February 1998. --------------------------------- TOC Author's Address Christopher J. Calabrese The SANS Institute 26 Wellesley Road Upper Montclair, NJ 07043 USPhone: +1 973 233 1464EMail: chris_calabreseat_private --------------------------------- TOC Appendix A. XML DTD for the `log:EVNT' Element <!-- Note that attributes are marked REQUIRED -- here if it is implied by draft-calabrese-requir-loggprot-01. -- This assumes this information is not being provided by -- another source. --><!ELEMENT log:ENTRY (log:ENTRY)*><!ATTLIST log:ENTRY log:HOST CDATA #REQUIRED log:DATE ISO.8601:1998_Date_Format #REQUIRED log:SEV log:Severity_Type #REQUIRED log:PROC.NAME CDATA #REQUIRED log:PROC.ID CDATA #REQUIRED log:PROC.PRIV CDATA #REQUIRED log:PROC.MAIL CDATA #IMPLIED log:PROC.TTY CDATA #IMPLIED log:EVENT.TYPE log:Event_Type_Enumeration #IMPLIED log:XML:LANG LangCode "EN" log:DUR Seconds.Decimal #IMPLIED log:SRC.ADDR log:Address_Format #IMPLIED log:SRC.PROT log:Protocol_Format #IMPLIED log:SRC.FQDN CDATA #IMPLIED log:SRC.NAME CDATA #IMPLIED log:SRC.USR CDATA #IMPLIED log:SRC.MAIL CDATA #IMPLIED log:DST.ADDR log:Address_Format #IMPLIED log:DST.PROT log:Protocol_Format #IMPLIED log:DST.FQDN CDATA #IMPLIED log:DST.NAME CDATA #IMPLIED log:DST.USR CDATA #IMPLIED log:DST.MAIL CDATA #IMPLIED log:REL.IN.ADDR log:Address_Format #IMPLIED log:REL.IN.PROT log:Protocol_Format #IMPLIED log:REL.IN.FQDN CDATA #IMPLIED log:REL.IN.NAME CDATA #IMPLIED log:REL.IN.USR CDATA #IMPLIED log:REL.IN.MAIL CDATA #IMPLIED log:REL.OUT.ADDR log:Address_Format #IMPLIED log:REL.OUT.PROT log:Protocol_Format #IMPLIED log:REL.OUT.FQDN CDATA #IMPLIED log:REL.OUT.NAME CDATA #IMPLIED log:REL.OUT.USR CDATA #IMPLIED log:REL.OUT.MAIL CDATA #IMPLIED log:VOL Integer #IMPLIED log:VOL.SENT Integer #IMPLIED log:VOL.RCVD Integer #IMPLIED log:CNT Integer #IMPLIED log:CNT.SENT Integer #IMPLIED log:CNT.RCVD Integer #IMPLIED log:PROG.FILE CDATA #IMPLIED log:PROG.LINE Integer #IMPLIED log:DOC CDATA #IMPLIED log:CMD CDATA #IMPLIED log:MSG CDATA #IMPLIED > --------------------------------- TOC Appendix B. XML DTD for Data Types Used in the `log:EVNT' Element <!NOTATION log:Severity_Type ('0'|'1'|...|'99')><!NOTATION ISO.8601:1998_Date_Format (YYYY '-' MM '-' DD 'T' hh ':' mm [':' ss ['.' f+]]('+' | '-') zzzz)><!-- NOTE: Need reference and further explanation of date format --><!NOTATION Seconds.Decimal (ss ['.' f+])<!NOTATION log:Event_Type_Enumeration ('AUTH.FAIL'|'AUTH.SUCCESS'|'PROG.FAIL'|'PROG.START'|...)><!-- NOTE: Need more complete enumeration. --><!NOTATION log:Address_Format (IPv4_Address|IPv6_Address|IPX_Address|...)><!-- NOTE: Do we need more a complete enumeration? of address types? --><!NOTATION IPv4_Address CDATA ><!-- NOTE: Need proper definition and reference for IPv4 addresses --><!NOTATION IPv6_Address CDATA><!-- NOTE: Need proper definition and reference for IPv6 addresses --><!NOTATION IPX_Address CDATA><!-- NOTE: Need proper definition and reference for IPX addresses --><!NOTATION log:Protocol_Format ( ('IPv4/' ('UDP'|'TCP') '/' IPv4_Port_Number ['/' CDATA]) | ('IPv4/ICMP/' IPv4_ICMP_Message_Type) | ( ('IPv6/' ('UDP'|'TCP') '/' IPv6_Port_Number ['/' CDATA]) | ('IPv6/ICMP/' IP64_ICMP_Message_Type) | ('IPX' ['/SPX/'] IPX_Port_Number ['/' CDATA]) ) <!-- NOTE: Need further definitions of referenced types... --> --------------------------------- TOC Full Copyright Statement Copyright (C) The Internet Society (2001). All Rights Reserved. This document and translations of it may be copied and furnished toothers, and derivative works that comment on or otherwise explain itor assist in its implementation may be prepared, copied, published anddistributed, in whole or in part, without restriction of any kind,provided that the above copyright notice and this paragraph areincluded on all such copies and derivative works. However, thisdocument itself may not be modified in any way, such as by removingthe copyright notice or references to the Internet Society or otherInternet organizations, except as needed for the purpose ofdeveloping Internet standards in which case the procedures forcopyrights defined in the Internet Standards process must befollowed, or as required to translate it into languages other thanEnglish. The limited permissions granted above are perpetual and will not berevoked by the Internet Society or its successors or assigns. This document and the information contained herein is provided on an"AS IS" basis and THE INTERNET SOCIETY AND THE INTERNET ENGINEERINGTASK FORCE DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDINGBUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATIONHEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OFMERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE. Acknowledgement Funding for the RFC editor function is currently provided by theInternet Society. --------------------------------- Do You Yahoo!? Make international calls for as low as $0.04/minute with Yahoo! Messenger. --------------------------------------------------------------------- To unsubscribe, e-mail: loganalysis-unsubscribeat_private For additional commands, e-mail: loganalysis-helpat_private
This archive was generated by hypermail 2b30 : Thu Aug 16 2001 - 09:23:17 PDT