Cross-platform Log cleanup tool Log-Cutter v2.0.2 officially released, log-cutterv2.0.2

Cross-platform Log cleanup tool Log-Cutter v2.0.2 officially released, log-cutterv2.0.2

 

Log-Cutter is a simple and practical Log cutting and cleaning tool developed by JessMA open-source organization. Log cleanup is very important for routine server maintenance. If too many logs are left, disk space is seriously wasted and service performance is affected. Manual cleanup may take too much time and is difficult to meet actual requirements. For example, how do I cut a log file larger than 2G at every Saturday to keep the latest 100 M log records? No log cutting tool can meet the requirements of this seat on the Internet, so I spent some time writing one by myself. Because it needs to be used on multiple platforms, Java is used for convenience. This tool is named Log-Cutter and has the following features:

 

1. Supports all common operating systems such as Linux, Mac, and Windows.
2. Supports interactive command line running.
3. Supports non-interactive running in the background (daemon process implementation in Linux/MAC, and system Service implementation in Windows)
4. Three log cleanup methods are supported (delete log files, cut log files, or archive log files)
5. Supports cutting common log file types such as GB18030, UTF-8, UTF-16LE, and UTF-16BE
6. Highly configurable (program execution cycle, log file expiration time to be deleted, log file threshold to be cut, and retained size)

Log-Cutter: http://www.jessma.org

Log-Cutter: https://github.com/ldcsaa/

 

• Usage

**************************************** * ************************ LogCutter-JessMA Open Source, all rights reserved. **************************************** * ************************* I. environment requirements ------------------------------------------------------ 1) java version: JDK/JRE 1.6 or above 2) Dependency package: dom4j, log4j, ant, juniversalchardet Release 2. configuration file ------------------------------------------------ 1) program configuration file: conf/config. xml (default) (example reference: conf/config-template.xml) 2) log configuration file: conf/log4j2. xml (default) (for example, see conf/log4j2. xml) Release 3. installation and deployment (Note: LogCutter needs to configure 'java _ home'/'jre _ home' and 'classpath' system environment variables) -------------------------------------------- 1) configure the system environment variables 'java _ home' (or 'jre _ home') and 'classpath' 2) in the LogCutter configuration file (default: conf/config. xml) Configure Clearing Rules 3) Start LogCutter ---------------------------------------------------- 4. startup method supervisor 1) Windows A) run at the front end:> run. bat [-f config-file] B) run in the background:> LogCutter.exe {-install-demand (install manual startup Service)-install-auto (install automatic startup Service) -uninstall (delete Service)-start (start Service)-stop (stop Service)-status (View service status )} * ** note ** @ LogCutter.exe runs as a Windows service. After installation, you can use the Windows Service Manager to manage it. @ LogCutter.exe is a 32-bit program, and logcutter_x64.exe is a 64-bit program, according to the current system platform using one of the C) single run:> run. bat-1 [-f config-file] 2) Linux/Unix A) run at the front end: $ run. sh [-f config-file] B) run in the background: $ run. sh [-f config-file]-d C) single run: $ run. sh-1 [-f config-file] [-d] *** note *** @ can run. sh start command to add/etc/rc. d/rc. local, so as to automatically run at startup @ run. sh-1 is periodically executed in CronTab without resident memory, such as: #30 2 ** 2, 4, 6 root/usr/local/LogCutter/bin/run. sh-1>/dev/null --------------------------------------------------

 

For detailed instructions and usage, see operation guide.

 

• Configuration File 

<? Xml version = "1.0" encoding = "UTF-8"?> <CONFIG xmlns = "http://www.jessma.org" xmlns: xsi = "http://www.w3.org/2001/XMLSchema-instance" xsi: schemaLocation = "http://www.jessma.org http://www.jessma.org/schema/log-cutter-cfg-2.0.xsd"> <global> <! -- Start the Log check delay (hours). Default Value: 0 (start immediately) 1) when it is specified as an integer, the first start delay time of the executor is the number of hours set for this value (for example: start the actuator for the first time after 12 hours) <start-check-delay> 12 </start-check-delay> 2) when it is specified as a value in 'hh: mm' format, the initial start time of the actuator is the value (for example, the first start time of the actuator at, if the current time exceeds, the first start of the actuator at tomorrow) <start-check-delay> 12: 34 </start-check-delay> --> <! -- Start the executor at for the first time --> <start-check-delay> </start-check-delay> <! -- Log check interval (hours); default value: 72 --> <check-interval> </check-interval> <! -- Log4J configuration file, default: $ {CLASS_ROOT}/../conf/log4j. properties --> <log4j-config-file> </log4j-config-file> <! -- Process lock file, default: $ {CLASS_ROOT}/../$ {APP_NAME}. lock --> <lock-file> </global> <! -- List of log files to be deleted (multiple log files can be configured and executed by DelFileRunner) 1) DelFileRunner recursively deletes the folders that meet the conditions and all the files in them. 2) Note: when determining whether to delete a folder, the last modification time of the folder is detected, instead of the last modification time of the file. 3) delFileRunner is applicable to clearing application logs for "periodically generating new log files" --> <! -- Delete-files.expire: Log File expiration time (days), default: 90 --> <delete-files expire = "30"> <! -- File. path: directory where the file is located. It must be filled in and cannot contain wildcards --> <! -- File: file or folder name, which must be filled in. It can contain wildcards --> <file path = "/opt/LogCutter/logs"> LogCutter. log * </file> <file path = "/opt/hMailServer/Logs"> *. log </file> <file path = "/opt/tomcat-6.0/logs"> *. log </file> </delete-files> <delete-files expire = "365"> <file path = "~ /Backup "> * </file> </delete-files> <! -- List of log files to be truncated (multiple log files can be configured and executed by CutFileRunner) 1) CutFileRunner only scans the files that meet the condition and does not scan the folder 2) cutFileRunner truncates the content of the front part of the file and retains the content of the back part. 3) CutFileRunner is suitable for clearing the application logs of "log files are constantly appended". --> <! -- Cut-files.threshold: Log File truncated threshold (KB), default: 10240 --> <! -- Cut-files.reserve: Log File keep content (KB), default: 1024 --> <! -- <! Note!> 'Cut-files. reserv' is the approximate value of the reserved content. The actual content is rounded up by line to keep the content of the following log files :.................. line1: xxxxxxxxxxxxxxxxxxx line2: xxxxxxxxxxPyyyyyyyyyy line3: zzzzzzzzzzzzzzzzzzz line4: zzzzzzzzzzzzzzzzz .................. (EOF) 'P' is the starting point for positioning. The program will actually start searching for the next line break at 'P', starting from the next character of the line break, that is, keep it from the third line to the end of the file --> <cut-files threshold = "10240" reserve = "512"> <file path = "/opt/apache-2.2/logs "> *. log </file> <file path = "/opt/ Mysql-server-5.1/data "> *. err </file> </cut-files> <! -- List of log files to be archived (multiple log files can be configured and executed by ArcFileRunner) 1) ArcFileRunner compresses and archives the files or folders that meet the conditions to the specified directory, and delete the original file or folder 2) archive file format: {Original file/folder name __{system time ..zip 3) Note: when determining whether to archive the folder, check the last modification time of the folder, rather than the last modification time of the files. 4) ArcFileRunner is applicable to clearing application logs that regularly generate new log files or log directories. --> <! -- Archive-files.expire: Log File expiration time (days), default: 90 --> <! -- Archive-files.archive-path: log file archiving directory --> <archive-files expire = "120" archive-path = "~ /Backup "> <file path ="/opt/mysql-server-5.1/data "> mysql-bin. * </file> </archive-files> </CONFIG>

Log-Cutter 2.0.2 update:

 

1. added the file archiving executor ArcFileRunner.

1) ArcFileRunner compresses and archives qualified files or folders to a specified directory and deletes the original files or folders.

2) The attribute expire specifies the archive expiration time, and the attribute archive-path specifies the archive directory.

3) archive file format: {Original file/folder name pai_{system time pai.zip

 

<! -- For example, the following configuration archives files or folders that have been last modified for more than 90 days to the 'e: \ backup' directory --> <archive-files archive-path = "E: \ backup "expire = 90> <file path =" X: \ MyApp \ logs "> * </file> <file path =" X: \ nginx-1.4.1 \ logs "> *. log </file> </archive-files>

 

2. Multiple File cleanup policies are supported.

1) config. xml can be configured with multiple <delete-files>, <cut-files>, or <archive-files>

2) Different cleaning policy parameters can be used for different configuration items

 

<! -- For example, in the following two sets of archive cleanup policies, specify different archive expiration times and archive directories --> <archive-files archive-path = "E: \ backup "expire = 90> <file path =" X: \ MyApp \ logs "> * </file> <file path =" X: \ nginx-1.4.1 \ logs "> *. log </file> </archive-files> <archive-files archive-path = "E: \ backup2" expire = 300> <file path = "X: \ MyApp2 \ logs "> * </file> <file path =" X: \ Tomcat-6 \ logs "> *. log </file> </archive-files>

3. File Deletion executor DelFileRunner supports recursive Folder deletion

1) DelFileRunner recursively deletes the qualified folder and all files in it.

2) Note: when determining whether to delete a folder, the last modification time of the folder is detected, instead of the last modification time of the file.

<! -- For example, the following configuration deletes all files or folders in the logs directory that have been last modified for more than 90 days --> <delete-files expire = 90> <file path = "X: \ MyApp \ logs "> * </file> </delete-files>

 

4. The executor's first start delay configuration '<start-check-delay>' supports specifying the absolute time

1) when it is specified as an integer, the first start delay time of the actuator is the number of hours set for this value.

2) When it is specified as a value in 'hh: mm' format, the first start time of the executor is the value.

<! -- (For example) --> <! -- Start the actuator for the first time in 12 hours --> <start-check-delay> 12 </start-check-delay> <! -- Start the actuator at for the first time, if the current time exceeds, start the executor for the first time at tomorrow --> <start-check-delay> </start-check-delay>

5. added the running-only-once mode)

1) in single-run mode, LogCutter immediately runs all the executors at startup and exits the program after all the executors have finished running.

2) in a single running mode, you can avoid LogCutter running as a resident process, and use it with task schedulers such as CronTab to achieve Scheduled cleaning.

3) Running LogCutter in single run mode ignores the 'start-check-delay' and 'check-interval' configuration items in the configuration file.

4) use the command line parameter '-1' to indicate that LogCutter is started in a single running mode.

 

(For example)

Windows:>Run. bat-1

Unix/Linux:>Run. sh-1 [-d]

 

6. Optimize the startup script run. sh/run. bat.

1) You only need to configure the system environment variables 'java _ home' (or 'jre _ home') and 'classpath' to run LogCutter through the startup script.

2) the startup script accepts the command line parameters and passes them to the LogCutter program. Run. sh indicates that LogCutter is run in the background using the '-d' parameter.