RE: [logs] collecting remote windows logs

From: Jim Lancaster (jlancaster@private)
Date: Fri Sep 03 2004 - 11:55:53 PDT


Yes, EvLog is good.  I also have used Adiscon EventReporter (and related
products).  Actually, the EventReporter products are terrific, but my
unusual requirements drove me to strike out on my own.  

I run a managed services company. We support a bunch of small customers,
about 150 servers in all.  Most network management products are designed
for a single enterprise to manage their private devices.  There is
usually no way to organize the devices into customers, much less provide
a customer with a secure interface to the system that allows them to see
their own data for themselves.  As I designed my system, I also had
these criteria in mind:

1. Ideally, I don't want to install anything on a customer server. I
prefer to poll remotely whenever possible.

2. The client servers are across the Internet behind firewalls, most of
which I manage, but not all.  I don't want to open holes in the
firewalls to get to the servers, and it is impractical to setup and
manage VPNs to the remote networks.

3. I'm not always sure what I'm looking for.  I'm not interested in all
events--I only know what I'm NOT interested in.  I want to see
everything else.  It has been my experience that a small number of
events are indicative of a large percentage of the actual problems.  I
want the logs filtered and summarized in such a way that I only have to
look through the shortest list possible.

My solution is still very crude, but it is working.  This is what I have
(and do not have) so far:

1. Windows event log agent - Written in perl and compiled using Komodo.
Capable of polling multiple devices, either local or remote.  Uses
Win32::OLE to poll WMI interface for event logs.  Queries limited to
records added since last poll.  Agent installs and runs as a service.
Polls every 15 minutes.  Uses SOAP to communicate with server over HTTP.
XML config file used to specify what the events to ignore.

2. Server - Apache 1.3, mod_perl, MySQL (on Gentoo Linux server).  Hosts
two sites.  One listens for agents, receives data and writes to MySQL
table.  The other provides a web interface to the system.  Currently, it
only provides two reports: One is a summary report that shows the number
of occurrences for each event, summarized by client then server.  It
also provides links to the EventID website.  The other shows the last
time an agent checked in.  Both reports provide the ability to flag any
row to send an e-mail to our ticketing system, along with reviewer
comments.

The most glaring omission is that I haven't built in any kind of
authentication mechanism.  I haven't figured out how, or even where to
start.

The reports, obviously, are quite limited.  One of the other comments in
this thread suggests that data collection is only a stop-off on the way
to data analysis.  I completely agree.  My intention was only to build a
mechanism to gather the data that I could then use with something like
Lire to analyze it.  

I go on too long.

Jim

-----Original Message-----
From: loganalysis-bounces+jlancaster=sagiss.com@private
[mailto:loganalysis-bounces+jlancaster=sagiss.com@private] On
Behalf Of Adrian Grigorof
Sent: Thursday, September 02, 2004 1:37 PM
To: loganalysis@private
Subject: Re: [logs] collecting remote windows logs

Just to add to the list... EvLog: http://www.altairtech.ca/evlog/, a
freeware that can be used to retrive remote event logs and put them in
HTML
format with links to www.eventid.net. It can be scheduled with MS Task
Scheduler (so it can be as realtime as other agents using a polling
interval).

Regards,

Adrian Grigorof
www.altairtech.ca
www.firegen.com


-----Original Message-----
From: loganalysis-bounces+jlancaster=sagiss.com@private
[mailto:loganalysis-bounces+jlancaster=sagiss.com@private] On
Behalf Of Jian Zhen
Sent: Friday, July 30, 2004 1:31 PM
To: loganalysis@private
Subject: Re: [logs] collecting remote windows logs

Hi all..

many thanks to everyone who's responded. I received many tips on how
this can be accomplished...basically i've gathered 4 ways...

1. Snare (or similar agents) sending to syslog...the kewl thing is
   that Intersect Alliance has provided a couple scripts that will
   allow you to install Snare onto remote machines in your domain...

2. DumpEvt...this is actually a pretty good way as you can dump
   logs from local and remote machines and format it the way you
   want it to...only thing is this is not real-time, so if you are
   looking for real-time, this is not the solution; however, in my
   case, I was not looking for real-time...

3. Win32::EventLog...works pretty good, since you can write your own
   script to do stuff, you can be flexible in what you want to keep
   or discard...not real-time...but you can easily write something
   that does what DumpEvt does with the added benefit of dumping the
   events to syslog...and still be agentless

4. Win32::OLE using WMI...pro'ly the most flexible solution, it can
   monitor for new log entries...so it can be a real-time solution..
   plus you have the flexibility of deciding what you want to do with
   the log once you receive it...send to syslog, discard, etc...however,
   i have read that this is pretty resource intensive...have not tested
   this approach to its limit tho...

again, thanks for the response and tips...

Jian

Jian Zhen (jlz@private) [040726 15:34]:
> Hi everyone,
>
> Was wondering if anyone has written anything to remotely collect
windows
> logs.
>
> I wrote a couple perl scripts (ActivePerl), one using Win32::EventLog
and
> the other using Win32::OLE (WMI). Both of them work fine on the local
> machine but I am having a bit of trouble getting it to work for remote

> machines.
>
> According to (search results from) Google, there's a Win32::OLD method
called
> "ConnectServer" in which one can use to connect to remote machines,
but
> I haven't been able to find any examples on using that with collecting
> logs.
>
> If you have any insight into this, I would love to chat with you.
>
> Thanks in advance
>
> Jian
>
> _______________________________________________
> LogAnalysis mailing list
> LogAnalysis@private
> http://lists.shmoo.com/mailman/listinfo/loganalysis
_______________________________________________
LogAnalysis mailing list
LogAnalysis@private
http://lists.shmoo.com/mailman/listinfo/loganalysis
_______________________________________________
LogAnalysis mailing list
LogAnalysis@private
http://lists.shmoo.com/mailman/listinfo/loganalysis

_______________________________________________
LogAnalysis mailing list
LogAnalysis@private
http://lists.shmoo.com/mailman/listinfo/loganalysis
_______________________________________________
LogAnalysis mailing list
LogAnalysis@private
http://lists.shmoo.com/mailman/listinfo/loganalysis



This archive was generated by hypermail 2.1.3 : Fri Sep 03 2004 - 17:17:47 PDT