On Wed, 2002-06-05 at 10:18, Sweth Chandramouli wrote:
> (I also started a conversation this morning with Damian
> Conway and Mark-Jason Dominus about a faster way to implement a parser
> in Perl, using iteration rather than recursion; it might be a long time
> before that pans out, but if it does, maybe Perl could remain a valid
> option as well.)
Hmmm... One way to do this in perl is to use an OOP approach where we
create a Logs::Parse module with all the generic fuctionality we need
(interfaces to databases, sorting, filtering and a generic data
structures). We then have a bunch of specific modules
Logs::Parse::Syslog, Logs::Parse::Pix, Logs::Parse::FW1, etc. which
inherit all the generic methods and data structures from the parent
module and define the actual parsing rules in native perl. This is not
such an elegant solution as Parse::Recurse but should run much faster.
Hmmmm... a more general solution would be to have a heirarchy rooted at
Logs::ZLogs (for want of a better name).
Logs:ZLogs.pm (generic data structures)
Logs:ZLogs::Parse.pm (generic parsing module etc.)
Logs:ZLogs::Parse.Syslog.pm (handle input from file, tcp and
udp streams)
Logs:ZLogs::Parse.Apache.pm
Logs:ZLogs::Parse.Pix.pm
...
Logs:ZLogs::Filter.pm
Logs:ZLogs::Output.pm
Logs:ZLogs::Output::Syslog.pm
Logs:ZLogs::Output::MySQL.pm
Logs:ZLogs::Output::Oracle.pm
...
...
I think perl allows dynamic inheritance, i.e. we can define a new class
on the fly which uses specific input and output modules. I've never
actually done this myself but I am pretty sure that I have seen other
modules that do this.
Once we have the generic structures sorted out then individuals can
write simple modules based on supplied templates or other modules and
contribute them back to the project.
Longer teerm if performance continues to be an issue we can migrate the
critical bits into C.
I *may* have some time later in the year to work on something like this,
but I'm not holding my breath!
The other possibility is that if we could come up with a spec then maybe
we could find a student to do this as a third/fourth year project.
I'll ask our Software Engineering people.
--
Russell Fulton, Computer and Network Security Officer
The University of Auckland, New Zealand
---------------------------------------------------------------------
To unsubscribe, e-mail: loganalysis-unsubscribeat_private
For additional commands, e-mail: loganalysis-helpat_private
This archive was generated by hypermail 2b30 : Tue Jun 04 2002 - 19:47:41 PDT