RE: [logs] Syslog payload format

fodat_private

Tina Bird wrote:
> So, never mind what actually shows up in your operating system and
> application logs.  What's the information that you log-weenies and
> sys-admin-weenies actually >>use<< to keep things up and running?  Or what
> would you use if it was there?

In my opinion, the basic questions a log should answer are who, what, when.
(Extra points for 'from where?' and 'for pity's sake why?' :)

Unfortunately the current state of affairs is BADLY broken on all three
counts.

When? - well we know that syslog is broken on that one as it doesn't record
the year or timezone, and clocks also need to be in synch, but at least
there are fairly simple tech solutions...

What? - this is 99% application dependent, and most interesting events
aren't logged in the first place, or the capability is there but not turned
on. There needs to be some identifier of the event that took place, together
with enough application data to reconstruct what precisely happened. That
might be an FTP command or it might be a funds transfer, so there is a huge
variation in what's needed & indeed how much you care about it.

The event itself needs to pass the sniff test of being 'business
significant' - with few exceptions doing that right requires getting IT to
sit down with the business, or suppliers with customers, and agree with them
what is significant. This almost never happens.

Who? - enough information to ultimately associate the event with a warm
body, or at least a system. Made difficult/impossible by the parlous state
of authentication in the average organisation. Typically needs a great deal
of correlation between logs in order even to begin to construct an argument
(e.g. all you may have is an IP address, or you may have shared accounts, a
server working "on behalf of", or you may have identifier reuse or
movers/leavers to contend with. More likely, all of those.)

A lot of times answering who/what takes *more than one event*, it is a
literal chain of events you are looking for. For example if the organisation
has shared accounts (and it probably does, though it may be in denial about
it :) you may even need to go as far back as looking at some paper trail to
find out who was on a certain shift.

There is some commonality on the 'what' - for example the basics of logging
in the security domain are pretty clear, and pretty much any OS or
application needs these:

- Anything to do with authentication - logging in (attempted/successful),
logging out, delegating, impersonating.
- Anything to do with privileges - acquiring them, using them
(successful/unsuccessful), delegating them.
- Anything to do with security admin - account management (e.g. add/delete
user), credential management (e.g. password changes), access control changes
(roles, groups, ACLs, privileges), etc.

(Noting that there are a multitude of avenues for doing all of that, and
many interesting apps define their own security admin. Also noting that the
initial authentication may be weak/absent, or Billy (a contractor hired on
capex budget) is one of 3 people using Jo (who left 3 months ago)'s account,
so it's close to meaningless. ahem.)

Some other misc stuff that is partly app independent:
- process (or service) start/stop/suspend/reconfigure
- process/service health (crashes, respawns too quickly)
...but you still want to know who made it do that, and what it was trying to
do at the time.

Sometimes the OS can log some of the above, sometimes it can't.
(Unfortunately even if it can, you will have a battle on your hands getting
a decent level of OS level auditing turned on as it usually canes
"performance". As with the fight to turn encryption on, this battle will
typically be lost even if the box is some kind of Cray used only for word
processing and running at 1% capacity...sigh)

Hmm...all in all, a bit of an uphill struggle :)

Cheers,
Frank

_______________________________________________
LogAnalysis mailing list
LogAnalysisat_private
http://lists.shmoo.com/mailman/listinfo/loganalysis