Re: [logs] Faster unix 'sort' replacement?

From: Marcus J. Ranum (mjr@private)
Date: Wed Sep 15 2004 - 15:43:33 PDT

Mike Blomgren wrote:
>I'm using sort in the standard 'cat <file> | awk '{"compute..."}' | sort |
>uniq -c | sort -n -r' type analysis.

You might want to just use a Btree or something that auto-sorts.
It depends on the amount of data you're managing.

BUT - take a look at "NBS" - basically it is designed to do
_exactly_ the
sort | uniq -c | sort -n -r
pipeline, with built-in persistence and a bunch of other
goodies. Just use the NBS database and throw away
the output NBS (-o /dev/null) and then use nbsdump to
output the topN values.


LogAnalysis mailing list

This archive was generated by hypermail 2.1.3 : Wed Sep 15 2004 - 16:00:32 PDT