Zookeeper Delete snapshot and transaction log of source code interpretation

Reprint please specify source address http://www.cnblogs.com/dongxiao-yang/p/4910059.html

Zookeeper has the ability to automatically clear the snapshot log and transaction log, which can be implemented in the configuration file settings Autopurge.purgeinterval, the problem is that the time unit of this property is the hour,

In some cases, an hour of log too large (such as the transaction log into memory), need to manually delete, so you need to study the ZK delete log file source.

Cleanup log main class: Org.apache.zookeeper.server.PurgeTxnLog, contains several static tool methods

static void Printusage () {

System. out. println ("Purgetxnlog datalogdir [Snapdir]-n count");

System. out. println ("\tdatalogdir – Path to the Txn log directory");

System. out. println ("\tsnapdir – Path to the snapshot directory");

System. out. println ("\tcount – the number of old snaps/logs you want to keep");

System.exit (1);

}

A common help method that tells the user the incoming order of parameters, where the Snapdir parameter is optional, and if the two logs are configured under the same path, it is good to pass only one path parameter.

The main method, there is nothing to say, just parse the parameters.

Public static void purge (file datadir, file snapdir, int num) throws IOException { /c5>

if (num < 3) {

throw New IllegalArgumentException ("count should be greater than 3");

}

Filetxnsnaplog txnlog = new Filetxnsnaplog (datadir, snapdir);

list<file> snaps = txnlog.findnrecentsnapshots (num);

Retainnrecentsnapshots (txnlog, snaps);

}

Main method of deleting files, mainly divided into two parts

1:txnlog.findnrecentsnapshots (num);

Find the file you want to keep

The main logic code is

Public list<file> findnrecentsnapshots (int N) throws IOException {

list<file> files = Util.sortdatadir (snapdir.listfiles (), "snapshot", false);

int i = 0;

list<file> list = new arraylist<file> ();

For (File F: files) {

if (i==N)

Break ;

i++;

List.add (f);

}

return list;

}

private static class Datadirfilecomparator

implements Comparator<file>, Serializable

{

private Static final long serialversionuid = -2648639884525140318l;

private String prefix;

Private Boolean ascending;

Public datadirfilecomparator (String prefix, boolean ascending) {

This . prefix = prefix;

This . ascending = ascending;

}

public int compare (file O1, file O2) {

long z1 = Util.getzxidfromname (o1.getname (), prefix);

long z2 = Util.getzxidfromname (o2.getname (), prefix);

int result = Z1 < Z2?-1: (z1 > z2 1:0);

return ascending? Result:-result;

}

}

/**

* Sort the list of files. Recency as determined by the version component

* of the file name.

*

* @param files array of files

* @param prefix files not matching this prefix is assumed to a

* Version = -1)

* @param ascending true sorted in ascending order, false results in

* Descending order

* @return sorted input Files

*/

public static list<file> Sortdatadir (file[] files, String prefix, boolean Ascending)

{

if (files==null)

return new arraylist<file> (0);

list<file> filelist = arrays.aslist (files);

Collections.sort (filelist, new Datadirfilecomparator (prefix, Ascending));

return filelist;

}

2 Deleting files

Visiblefortesting

static void Retainnrecentsnapshots (Filetxnsnaplog txnlog, list<file> snaps) {

Found any valid recent snapshots?

if (snaps.size () = = 0)

return;

File snapShot = snaps.get (snaps.size ()-1);

int ii=snaps.size ()-1;

System. Out.println (II);

final long leastzxidtoberetain = Util.getzxidfromname (

snapshot.getname (), prefix_snapshot);

class Myfilefilter implements filefilter{

Private final String prefix;

Myfilefilter (String prefix) {

This . prefix=prefix;

}

Public Boolean accept (File f) {

if (! F.getname (). StartsWith (prefix + "."))

return false;

long Fzxid = Util.getzxidfromname (f.getname (), prefix);

if (fzxid >= leastzxidtoberetain) {

return false;

}

return true;

}

}

Add all non-excluded log files

list<file> files = new Arraylist<file> (Arrays.aslist (txnlog

. Getdatadir (). Listfiles (new Myfilefilter (prefix_log) ));

Add all non-excluded snapshot files to the deletion list

Files.addall (Arrays.aslist (txnlog.getsnapdir () listfiles (

New Myfilefilter (prefix_snapshot) ));

Remove the old files

For (File F: files)

{

System. out.println ("removing file:" +

Dateformat.getdatetimeinstance (). Format (f.lastmodified ()) +

"\ t" +F.getpath ());

if (! F.delete ()) {

System. err.println ("Failed to remove" +F.getpath ());

}

}

}

Util.getzxidfromname Tool Method Code

public static long Getzxidfromname (string name, string prefix) {

long Zxid =-1;

String nameparts[] = name.split ("\ \");

if (nameparts. Length = = 2 && nameparts[0].equals (prefix)) {

try {

Zxid = Long.parselong (nameparts[1], +);

} catch (NumberFormatException e) {

}

}

return Zxid;

}

Zookeeper Delete snapshot and transaction log of source code interpretation