Re: [logs] Seeking suggestions on a secure central syslog setup..

From: Sean McNamara (sean.mcnamaraat_private)
Date: Mon Nov 12 2001 - 13:02:49 PST

  • Next message: Sean McNamara: "[logs] A bit of clairification on my last question.."

    "Desai, Ashish" wrote:
    > One of the things we did is to write a script/program
    > [ in language of your choice, we chose perl]  for every data source we have.
    > All the scripts support at least the following arguments,
    >         start date
    >         end date
    >         keyword regex (OPTIONAL)
    > Example
    >         pdata -s 12/15/2001 -e 12/20/2001 -k "128.*"
    >         # this would spit out to stdout all the data between those dates for
    > the matched pattern
    First off, thanks for a response.   Actually, what you've mentioned right here
    is quite similar to what I had thought out for writing myself.   The basis of it
    all would be a more or less 'framework' cgi that accepted plugin 'modules' such
    as what you described that would do the individual searching and parsing
    necessary; all fitting together to make a mostly dynamic searching/viewing
    interface.   I'd do this all in perl.
    > This satisfied our power/intermediate users as they could continue to use
    > the standard
    > unix tools, grep, pipe etc to play with the results. Note, we output data in
    > tab delimited format so
    > the standard IFS work fine with the output.
        The main problem we have, is that we do not necessarily want to grant user
    accounts on the viewing machine to anyone and everyone who would fit (or claim
    to fit) into the power/intermediate range of users.
    Being that we're a university, there are many people from various departments
    and colleges who could have a legitimate reason to view portions of data, but
    we're sort of cautious of granting direct machine access, thus our desire to
    create a web frontend.
    > Next we wrote small cgi's that call these scripts and save the output to a
    > file. We then
    > send the requester email with the URL to the results (compressed with gzip).
    > This allows the
    > naive users to request data and fetch it via a browers. Some of these users
    > put the data in Excel
    > for analysis while others just use a text editor.
    > The advantage of this is once you implement the scripts you can trivally
    > reuse them for the web interface
    > Also some of our data sets are large, can take upto 48 hrs to search through
    > them. So the web interface
    > just dumps the request into a file and a shell scripts queues up the
    > request. This way the requestor
    > does not have to wait at the web interface.
        I like what you're saying here, I think something along these lines would
    probably work for us as a beginning, at least to get us started with something
    for the higher-end users.   Like you said, a backend 'searching' script would be
    simple to tie into a frontend cgi, and thus allow us to easily generate search
    results via the web.   I guess the real problem is really how to cater to the
    varying levels of expertise versus what data they need available without handing
    the keys to the kingdom to the expert users, and ignoring the novices.
        For example, in my design, it would be possible for a user to select a
    filter (or have it forcefully selected for them) that would parse/translate
    certain sets of data depending on what they were looking for.   For example, a
    helpdesk technician trying to troubleshoot a user's RAS (dial-in) problem, could
    look at a parsed log of only login attempts/failures/successes w/ only necessary
    details without having to understand the 'meat' of what's being logged back from
    any of the involved components -- not to mention, in some cases, some details of
    the logs might be inappropriate for certain folks to view.
    Go figure-- this is probably more of a 'moral' dilemma than not having the
    technology available to do the job.
    Do you (or anyone else) know offhand if anyone maintains a list of some type of
    the various options for syslog consoles and monitors?  .. or maybe can make a
    recommendation based on what they're using?   I guess what I'm really trying to
    avoid is to have to write the system myself if something close enough to it
    already exists; you know -- no sense reinventing the wheel.
    Thanks a lot for your help!
    To unsubscribe, e-mail: loganalysis-unsubscribeat_private
    For additional commands, e-mail: loganalysis-helpat_private

    This archive was generated by hypermail 2b30 : Mon Nov 12 2001 - 13:35:18 PST