logresolvemerge 1 2

download logresolvemerge 1 2

If you can't read please download the document

description

logresolvemerge 1.2 (build 1.38) (c) Laurent DestailleurAWStats(Advanced Web Statistics) tool http://awstats.sourceforge.net/docs/awstats_tools.html

Transcript of logresolvemerge 1 2

----- logresolvemerge 1.2 (build 1.38) (c) Laurent Destailleur ----logresolvemerge allows you to get one unique output log file, sorted on date, built from particular sources: - It can read several input log files, - It can read .gz/.bz2 log files, - It can also makes a fast reverse DNS lookup to replace all IP addresses into host names in resulting log file. logresolvemerge comes with ABSOLUTELY NO WARRANTY. It's a free software distributed with a GNU General Public License (See COPYING.txt file). logresolvemerge is part of AWStats but can be used alone as a log merger or resolver before using any other log analyzer. Usage: logresolvemerge.pl [options] file logresolvemerge.pl [options] file1 ... filen logresolvemerge.pl [options] *.* perl logresolvemerge.pl [options] *.* > newfile Options: -dnslookup make a reverse DNS lookup on IP adresses -dnslookup=n same with a n parallel threads instead of serial requests -dnscache=file make DNS lookup from cache file first before network lookup -showsteps print on stderr benchmark information every 8192 lines -addfilenum if used with several files, file number can be added in first -addfilename if used with several files, file name can be added in first field of output file. This can be used to add a cluster id when log files come from several load balanced computers. This runs logresolvemerge in command line to open one or several server log files to merge them (sorted on date) and/or to make a reverse DNS lookup (if asked). The result log file is sent on standard output. Note: logresolvemerge is not a 'sort' tool to sort one file. It's a software able to output sorted log records (with a reverse DNS lookup included or not) even if log records are dispatched in several files. Each of thoose files must be already independently sorted itself (but that is the case in all web server log files). So you can use it for load balanced log files or to group several old log files. Don't forget that the main goal of logresolvemerge is to send log records to a log analyzer in a sorted order without merging files on disk (NO NEED OF DISK SPACE AT ALL) and without loading files into memory (NO NEED OF MORE MEMORY). Choose of output records is done on the fly. So logresolvemerge is particularly usefull when you want to output several and/or large log files in a fast process, with no use of disk or more memory, and in a chronological order through a pipe (to be used by a log analyzer). Note: If input records are not 'exactly' sorted but 'nearly' sorted (this occurs with heavy servers), this is not a problem, the output will also be 'nearly' sorted but a few log analyzers (like AWStats) knowns how to deal with such logs. WARNING: If log files are old MAC text files (lines ended with CR char), you can't run this tool on Win or Unix platforms. WARNING: Because of memory holes in ActiveState Perl version, use another Perl interpreter if you need to process large log files.

Now supports/detects: Automatic detection of log format Files can be .gz/.bz2 files if zcat/bzcat tools are available in PATH. Multithreaded reverse DNS lookup (several parallel requests) with Perl 5.8+. New versions and FAQ at http://awstats.sourceforge.net