RE: [logs] Faster unix 'sort' replacement?

From: Mike Blomgren (mike.blomgren@private)
Date: Thu Sep 16 2004 - 10:59:37 PDT


Thanks for the tip -  I'll have to try that one with perl doing the sort
instead of gnu sort. I have been somewhat reluctant to use perl since I find
it has a severe performance impact in some cases - but that may be related
to my regexp's and not the sorting. For a fact though, I do know that using
associative arrays is a good way to consume memory in a hurry. And thus
causing the os to start swapping memory to disk, which is not very
beneficial for speed, to say the least... 

~Mike


> From: Ed Schmollinger [mailto:schmolli@private] 
> Sent: den 16 september 2004 18:15
> To: Mike Blomgren

<snip>

> You can get rid of the multiple sorts/uniq thing by doing it all at
> once:
> 
> --- CUT HERE ---
> #!/usr/bin/perl -wT
> 
> use strict;
> 
> my %msg = ();
> 
> while (<>) { chomp; $msg{$_} = $msg{$_} ? $msg{$_} + 1 : 1; }
> 
> for(sort { $msg{$a} <=> $msg{$b} } keys %msg) { print 
> "$msg{$_}\t$_\n"; }
> --- CUT HERE ---


_______________________________________________
LogAnalysis mailing list
LogAnalysis@private
http://lists.shmoo.com/mailman/listinfo/loganalysis



This archive was generated by hypermail 2.1.3 : Thu Sep 16 2004 - 17:34:38 PDT