You wrote: > > Its source is at > > http://jade.cs.uct.ac.za/idsa/tmp/idsalite-0.1.tar.gz > > gee... ;) another program with a 0.x version number. Darren? Ha, it is up to 0.2 by now ;) > > See above, only interested in API for the time being. > > Idsafull can do almost arbitrary formatting. > > can you be more specific? 'arbitrary' is just too arbitrary for me ;) Oh, I just mentioned that in passing. The full idsa library has a little formatting language - it is internal and rather unpolished, but can be thought of as printf with loops. See lib/print.c struct lookup { char *key; char *value; } map[] = { { "csv", "\"%{:1}\"%{1>},\"%{:1}\"%{<}\n"}, { "ulm", "%{*}=\"%{:1}\"%{1>} %{*}=\"%{:1}\"%{<}\n"}, { "tulm", "%{*}:%{+}=\"%{:1}\"%{1>} %{*}:%{+}=\"%{:1}\"%{<}\n"}, { "syslog", "%{time:100} %{host:1} %{service:1}[%{pid}]: %{scheme:1}.%{name:1} %{12>} %{:1}%{<}\n"}, { "native", "%{time:102} %{host:1} %{uid}:%{gid} %{honour} %{arisk}:%{crisk}:% {irisk} %{service:1}:%{pid} %{scheme:1}:%{name:1}%{12>} %{*}=\"%{:1}\"%{<}\n"}, { "xml", "<event>%{>}<%{*} type=\"%{+}\">%{:2}</%{*}>%{<}</event>\n"}, { NULL, NULL} }; But I don't think we should aim to standardise that (yet). > the point is that it is possible to add further transports. I think running > a UNIX-like system without local syslogd is an attainable target. Exactly, and idsad would be an example - hopefully one of several. > > Not visible at the API level, so not considered. idsafull has its own > > PID, UID, GID, TIME, HOST, SERVICE, SCHEME, NAME, ARISK, CRISK, IRISK, HONOUR > > I think the first is to define the set of tags and their naming. Later the > library can pick which tags it wants to generate automatically. This part of > EventLog is meant as an example only. I agree completely - the automatically generated tags are easy to change, and can either be made specific to an implementation or changed. > It is again up to us to specify tag naming. I'd go for XML namespaces > instead of inventing our own naming scheme. So either ASN.1 OIDs (booo) or > XML namespaces. But XML can handle tag nesting, which we would have to simulate with something.something in which case there are subtle issues anyway. But I am open to suggestions on that. > I was under the impression that a single log 'connection' will be used by a > single application, therefore the global state is stored in a single, global > EVTCONFIG structure. > > Thread safety is a must for me, so it will be added later, but I'd focus > more on evt_log() racing with another evt_log() call. (and not evt_open > racing with evt_log) Safety is the major problem. But also high performance (otherwise you block all threads), and then library components or dll's which want to do their own logging. Maybe tep has some comments ? > > Almost the same. Though you are using the old syslog severities. > > I think mjr proposed an 11 valued field, while I use 3 continuous > > values in the range -1 to 1, to represent threat to availability, > > confidentiality and integrity. I can put in a message field. > > Though in my implementation the message is just > > another tag - which I think is cleaner. > > The description should be as mandatory as possible. If it was provided as a > tag it would be possible to log an event without one. > > I think it is difficult to completely forget about syslog severities though. > The major transport for these messages will be syslog. We either need a > function to map some 'new' priority field to standard syslog priority, _or_ > we use syslog priority and require the application to define its own tag if > it wants more. I agree that it will be the major one, but if you make it make it part of that call the temptation is to make it "special", and note how the special status of the priority in "<12>Message..." leads to it being lost in the log file. Anyway - see previous mail for my idea. > The 11 valued field mjr has proposed was quite IDS specific. The ones you > use are IDSA specific, and I don't either of you could work with the other's > priority definition. Actually, I like to think my approach can encode anything ;). Especially since it has a confidence value, which is useful, since most people who don't care about priority under syslog use "INFO", but info and don't know have different meaning. Anyway, a topic for another discussion. > > Ditto. PS: Idsalite doesn't do fancy escapes, idsafull does > > Escaping is a must, otherwise you cannot warrant well-formedness. I'm for > dropping the priority field. It was a bad decision. Has been added as of idsalite-0.2. But how the escaping is done is a transport problem - so not part of the API which I would like to reach consensus on. > > > evt_tag_errno(e, -5, "test:error", EAGAIN); > > > > Idsalite doesn't have this, idsafull does, plus another 11 or so > > types. But for a simple standard, maybe we should stick to strings and > > integers for the time being ? > > I don't think adding more types hurts Well, in a way I agree, as I have lots of types, but for the sake of a simple small standard it makes things difficult as people are likely to argue what types they want, and that windows pids don't map to unix, etc, etc, > What are these: > > int idsa_add_scan(IDSA_EVENT * e, char *n, unsigned int t, char *s); > int idsa_add_set(IDSA_EVENT * e, char *n, unsigned int t, void *p); The full libidsa has a type associated with each tag (like you have priorities ;). Each type has a binary representation, and a string representation. These are the two low level handlers for it. idsa_add_string is implemented as idsa_add_set(e, "label", IDSA_T_STRING, value); idsa_scan allows the conversion: i=42; idsa_add_set(e, "lable", IDSA_T_INT, &i); has the same result as idsa_add_scan(e, "label", IDSA_T_INT, "42"); I think it is neat to have, but probably beyond a standard, like your priorities. > > > es = evt_format(e); > > > printf("%s\n", es); > > > free(es); > > > > I don't have that in idsalite > > This function is needed only if the application wants to perform its own > logging but still wants to use the tagged message format. Nice. But I don't think that should be part of the immediate standard, as that presupposes a pretty printing API which can be rather large, as above ;) > > So you have an explicit free. In idsalite I do the free inside > > the log to avoid leaks. But one also wants to re-use events. > > In idsafull one can control if the event needs to be free'ed > > explicitly. For the proposed standard I can implement a do_nothing > > function, but are you sure the risk of leaks is worth the gain ? > > I don't mind either way. > > I think evt_log() should consume the event for the very reason you > mentioned. (thus: another change in the API) Event records are reference > counted so the application can keep a record if it wants to: > > evt_rec_ref(e); > evt_log(e); Ok - so I'll have to update my wrapper too ;). See idsa_template() on how I chose to make event data available across logging calls. > > > * tag ordering > > > currently implemented as a number assigned to tags > > this should be dropped. Cool. > > > * output modules character conversion > > > should we use a single character encoding on the wire (UTF8?) would it be > > > mandatory or it should be configurable? > > > > Just one. I prefer plain ASCII ;) > > I'm not so sure ;) But make this a topic discussed later. Agreed. > > > The syslog facility and priority values are currently mandatory, they > > > might be converted to simple tags, but how to send messages which do not > > > specify them? > > > > I'd prefer it optional, so that the new API is not permanently tied > > to syslog. Actually the same holds for the second field of evt_open > > but the primary transport of these messages will be syslog() > > should we use a default priority for messages which do not specify syslog > priority on their own? I have this notion of using a very rare priority (see the comment in idsalite) so that a log processor can then assume that anything on that has the new format (whatever format it may be). Another reason for not giving special dispensation to the priority at the API level. regards marc _______________________________________________ LogAnalysis mailing list LogAnalysisat_private http://lists.shmoo.com/mailman/listinfo/loganalysis
This archive was generated by hypermail 2b30 : Wed Jan 08 2003 - 09:22:20 PST