Log cutting and cleaning tool implemented in Java

Log cleanup is very important for routine server maintenance. If too many logs are left, disk space is seriously wasted and service performance is affected. Manual cleanup may take too much time and is difficult to meet actual requirements. For example, how do I cut a log file larger than 2 GB at every Saturday and keep the latest MB log records?

No log cutting tool can meet the requirements of this seat on the Internet, so I spent some time writing one by myself. Because it needs to be used on multiple platforms, Java is used for convenience. This tool is namedLogcutterAnd has the following features:

1. Supports all common operating systems such as Linux, Mac, and windows.
2. Supports interactive command line running.
3. Supports non-interactive running in the background (daemon process implementation in Linux/Mac, and system service implementation in Windows)
4. Two log cleanup methods are supported (delete log files or cut log files)
5. Support for gb18030, UTF-8, UTF-16LE, UTF-16BE and other common log file types cut (not cut off half a character)
6. Highly configurable (ProgramThe execution cycle, expiration time of the log file to be deleted, threshold value of the log file to be cut, and retention size can be configured.

• Usage:

**************************************** ***************
* *** Logcutter-Bruce Liang, All rights reserved .****
**************************************** ***************

I. Startup Mode
--------------------------------------------------
1) Windows
A) Foreground running:> ./ Run . Bat
B) running in the background:>./logcutter.exe {
-Install-demand (install and manually start the service)
-Install-auto (install auto-Start Service)
- Start (Start the service)
-Stop)
-Status (View service status)
}

* ** Note ***
@ Logcutter.exe runs as a Windows system service. After installation, you can manage it through the Windows Service Manager.
@ Logcutter.exe depends on JRE 1.6. Therefore, % java_home % must be set in the registry or environment variable.

2) Linux/Unix
A) Foreground running: $ ./ Run . Sh
B) running in the background: $ ./ Run . Sh-d

* ** Note ***
@ It can be set to run automatically in the background when the instance is started. The method is as follows:

$ VI/etc/rc. d/rc. Local
(Add: $ {your_logcutter_path }/ Run . Sh-d)
--------------------------------------------------

Ii. Configuration File
--------------------------------------------------
1) program configuration file:./CONF/config. xml
2) log configuration file:./CONF/log4j. properties (default)
--------------------------------------------------

Iii. environment requirements
--------------------------------------------------
1) Java version: JRE 1.6 or above
2) Dependency packages: dom4j, log4j, juniversalchardet
--------------------------------------------------

 

• Run. Sh (MAC platform)

#! /Bin/bash

Java_home ="/Library/Java/home "
Classpath = $ java_home/lib/dt. jar: $ java_home/lib/tools. Jar
Java_bin = $ java_home/bin
Java = $ java_bin/Java

App_path =" /Users/Kingfisher/mywork/Java/logcutter "
App_classpath = $ app_path/classes
App_libpath = $ app_path/lib
App_libs = $ app_libpath/dom4j-1.6.1.jar: $ app_libpath/log4j-1.2.16.jar: $ app_libpath/juniversalchardet-1.0.3.jar
App_confpath = $ app_path/Conf
App_conffile = $ app_confpath/config. xml
App_main_class = com. Bruce. Main. logcutter

Daemon = '-d'
CMD = "$ Java-CP $ classpath: $ app_classpath: $ app_libs $ app_main_class-F $ app_conffile"

 If [$ #-EQ 0]
 Then 
$ CMD 
Elif ["$ #"-EQ 1-A "$1" = "$ daemon"]
 Then 
$ CMD &
 Else 
 Echo "Usage: $0 [$ daemon]"
 Exit 1
Fi

 

• Interactive Operation Demonstration:

[Kingfisher @ Bruce-MBP logcutter] $ PWD 
/Users/Kingfisher/mywork/Java/logcutter
[Kingfisher @ Bruce-MBP logcutter] $ ./Run. Sh  
 Command Line usage
------------------------------------------------------------
Help: Show Help
Jobs: Show jobs status
CFG: Show configuration Summary
 Run : Schedule jobs manually
! Q: shutdown Application
? : About me
------------------------------------------------------------
> Help 
 Command Line usage
------------------------------------------------------------
Help: Show Help
Jobs: Show jobs status
CFG: Show configuration Summary
 Run : Schedule jobs manually
! Q: shutdown Application
? : About me
------------------------------------------------------------
> Jobs 
Jobs Summary (total: 2, active: 0)
------------------------------------------------------------
1. delfilerunner @ 334936591 [idle]
2. cutfilerunner @ 724646150 [idle]
------------------------------------------------------------
> Cfg 
Configuration Summary
------------------------------------------------------------
[Global]
 Start -Check-delay: 1 hours
Check-interval: 24 hours
Log4j-config-file:/users/Kingfisher/mywork/Java/logcutter/classes/../CONF/log4j. Properties
Lock-file:/users/Kingfisher/mywork/Java/logcutter/classes/.../logcutter. Lock
[Delete- Files ] (Expire: 30 days)
1./opt/logcutter/logs/logcutter. log *
2./opt/tomcat6-logs/*. Log
3./opt/tomcat7-logs/*. Log
[CUT- Files ] (Threshold: 30960 KBS, reserve: 1024 KBS)
1./opt/Apache-logs/* _ log
2./opt/nginx-logs/*. Log
3./opt/tomcat6-logs/Catalina. Out
4./opt/tomcat7-logs/Catalina. Out
5./var/lib/MySQL/*. Err
------------------------------------------------------------
> Run 
Manual jobs are scheduled!
> ? 
Logcutter 1.0.2-Bruce Liang, All rights reserved.
------------------------------------------------------------
Description: Schedule to delete and cut text log Files Automatically or manually.
Support: gb18030, UTF-8, UTF-16LE and UTF-16BE text file types.
Usage: Java com. Bruce. Main. logcutter [-F <config-File>]
(Default config file is '/users/Kingfisher/mywork/Java/logcutter/classes/../CONF/config. xml ')
------------------------------------------------------------
> ! Q 
Be about to shutdown, please wait...
Shutdown perfectly!

 

• Background running demo (MAC platform ):

[Kingfisher @ Bruce-MBP logcutter] $./Run. Sh-d
[Kingfisher @ Bruce-MBP logcutter] $
Logcutter is running in background,Use'Kill 4544 'to stop me.

[Kingfisher @ Bruce-MBP logcutter] $Killed 4544
[Kingfisher @ Bruce-MBP logcutter] $
!! Logcutter specified ed terminate signal !!
Be about to shutdown, please wait...
Shutdown perfectly!

[Kingfisher @ Bruce-MBP logcutter] $
 

 

• Program running log

 05:52:20, 705 info [main]: -> -> -> 
05:52:20, 706 info [main]: Starting: Class com. Bruce. Main. logcutter... 
 05:52:20, 707 info [main]: configuration Summary 
 ------------------------------------------------------------ 
 [Global] 
 Start-check-delay: 1 hours 
 Check-interval: 24 hours 
 Log4j-config-file:/users/Kingfisher/mywork/Java/logcutter/classes/../CONF/log4j. Properties 
 Lock-file:/users/Kingfisher/mywork/Java/logcutter/classes/.../logcutter. Lock 
[Delete-files] (expire: 30 days) 
 1./opt/logcutter/logs/logcutter. log * 
2./opt/tomcat6-logs/*. Log
3./opt/tomcat7-logs/*. Log
[CUT-files] (threshold: 30960 KBS, reserve: 1024 KBS)
1./opt/Apache-logs/* _ log
2./opt/nginx-logs/*. Log
3./opt/tomcat6-logs/Catalina. Out
4./opt/tomcat7-logs/Catalina. Out
5./var/lib/MySQL/*. Err
------------------------------------------------------------
05:52:39, 062 info [Pool-1-thread-2]:---> Start cutfilerunner @ 724646150 <--------
05:52:39, 062 info [Pool-1-thread-1]:---> Start delfilerunner @ 334936591 <--------
05:52:39, 063 info [Pool-1-thread-2]:---> end cutfilerunner @ 724646150 <--------
05:52:39, 063 info [Pool-1-thread-1]:---> end delfilerunner @ 334936591 <--------
05:52:54, 222 info [main]: stoping: Class com. Bruce. Main. logcutter...
05:52:54, 222 info [main]: <- <- <-

 

• Program configuration file (./CONF/config. XML)

<?  XML version = "1.0" encoding = "UTF-8"  ?> 
 <  Config  > 
 <  Global  > 
 <! --  Start Log check delay (hours), default: 0 (start now)  --> 
 <  Start-check-Delay  > 1</  Start-check-Delay  > 
 <! --  Log check interval (hours). Default Value: 72  --> 
 <  Check-interval  > 24 </  Check-interval  > 
 <! --  Log4j configuration file. Default: $ {class_root}/../CONF/log4j. Properties --> 
 <  Log4j-config-file  > </  Log4j-config-file  > 
 <! --  Program lock, default: $ {class_root}/../$ {app_name}. Lock  --> 
 <  Lock-File  > </  Lock-File  > 
 </ Global  > 
 <! --  List of log files to be deleted  --> 
 <! --  Delete-files.expire: Log File expiration time (days), default: 90  --> 
 <  Delete-Files  Expire  = "30"  > 
 <! -- Files: File Name (excluding Directories), which must be filled in with wildcards  --> 
 <! --  Files. Path: directory of the file, which must be filled in and cannot contain wildcards  --> 
 <  File  Path  = "/Opt/logcutter/logs"  > Logcutter. log * </  File  > 
 <  File Path  = "/Opt/tomcat6-logs"  > *. Log </  File  > 
 <  File  Path  = "/Opt/tomcat7-logs"  > *. Log </  File  > 
 <! -- 
<File Path = "/var/lib/MySQL"> mysql-bin. * </File>
<File Path = "/var/lib/MySQL"> efly-test-relay-bin. * </File>
  --> 
 </  Delete-Files  > 
 <! --  List of log files to be truncated  --> 
 <! --  Cut-files.threshold: Log File truncation threshold (Kb), default: 10240  --> 
 <! -- Cut-files.reserve: Log File reserved content (Kb), default: 1024  --> 
 <! --  
<! Note!> 'Cut-files. reserv' is the approximate value of the reserved content, and the actual content is rounded and retained by line.
See the following log file content:
..................
Line1: xxxxxxxxxxxxxxxxxxxxx
Line2: xxxxxxxxxxPYyyyyyyyyy
Line3: zzzzzzzzzzzzzzzzzzzzz
Line4: zzzzzzzzzzzzzzzzzzzzz
.......... (EOF)
'P'Is the starting point for positioning, and the program will start searching for the next line break at 'P,
It is retained from the last character of the linefeed, that is, from the third row to the end of the file.
  --> 
 <  Cut-Files Threshold  = "30960"  Reserve  = "1024"  > 
 <  File  Path  = "/Opt/Apache-Logs"  > * _ Log </  File  > 
 <  File  Path = "/Opt/nginx-Logs"  > *. Log </  File  > 
 <  File  Path  = "/Opt/tomcat6-logs"  > Catalina. Out </  File  > 
 <  File  Path = "/Opt/tomcat7-logs"  > Catalina. Out </  File  > 
 <  File  Path  = "/Var/lib/MySQL"  > *. Err </  File  > 
 </  Cut-Files > 
 </  Config  > 

Download trial

Codeproject