Lzo local compression and decompression example

/** * @author HJX * @version 1.0,2013-01-16 * @since jdk1.7,ubuntu-12.04-64bit * Run in Hadoop environment * writes a string to the local Lzo file (
Not Hadoop HDFs) * Then read from the Lzo file and proofread with the original string/import Java.io.BufferedReader;
Import Java.io.FileInputStream;
Import java.io.FileNotFoundException;
Import Java.io.FileOutputStream;
Import java.io.IOException;
Import Java.io.InputStream;
Import Java.io.InputStreamReader;
Import Java.io.OutputStream;
Import java.util.ArrayList;

Import java.util.List;

Import org.apache.hadoop.conf.Configuration;

Import Com.hadoop.compression.lzo.LzopCodec; 
        public class Lzocompress {/** * @param args */public static void main (string[] args) {//Generate data
        String DataSource = "abcdefghijklmnopqrstuvwxyz0123456789~!%#^@*#*%$ (\ n";
        DataSource = Datasource.concat (DataSource);
        DataSource = Datasource.concat (DataSource);
DataSource = Datasource.concat (DataSource);
       /* SYSTEM.OUT.PRINTLN ("DataSource =" + DataSource); String Lzofilepath = "/home/hadoop/lzocompresstest.lzo";
        Write to Lzo file, that is, Lzo compression write2lzofile (Lzofilepath, getdefaultconf (), datasource.getbytes ());
        
        StringBuilder sb = new StringBuilder ();
        Read Lzo file, that is, Lzo decompression list<string> lines = Readlzofile (Lzofilepath, getdefaultconf ());
            for (String line:lines) {sb.append (line);            
Linux/unix Add a newline character sb.append ("\ n"); /*//windows Add a newline character sb.append ("\ r \ n"); */} if (Sb.tostring (). Equals (DataSource))
        {System.out.println (sb.tostring ());
        else {System.err.println ("Error line:" + sb.tostring ());
        } private static Configuration getdefaultconf () {Configuration conf = new Configuration ();
        Conf.set ("Mapred.job.tracker", "local");
        Conf.set ("Fs.default.name", "file:///"); Conf.set ("Io.compression.codecs", "com.hadoop.compresSion.lzo.LzoCodec ");
    return conf;  /** * Write data to Lzo file, that is, lzo compression * @param destlzofilepath * @param conf * @param datas * @return void */public static void Write2lzofile (String destlzofilepath,configuration conf,byte[] datas) {LZOPC
        Odec lzo = null;
        
        OutputStream out = null;
            try {/* System.setproperty ("Java.library.path", "/usr/local/hadoop/lib/native/linux-amd64-64/lib");
            Lzo = new Lzopcodec ();
            lzo.setconf (conf);
            out = Lzo.createoutputstream (new FileOutputStream (Destlzofilepath));
        Out.write (datas);
        catch (FileNotFoundException e) {e.printstacktrace ();
        catch (IOException e) {e.printstacktrace ();
                Finally {try {if (out!= null) {out.close ();
            } catch (IOException e) {e.printstacktrace ();}}/** * Reads data from Lzo file, that is, Lzo decompression * @param lzofilepath * @param conf * @return void  */public static list<string> Readlzofile (String lzofilepath,configuration conf) {Lzopcodec Lzo =
        Null
        InputStream is = null;
        InputStreamReader ISR = null;
        BufferedReader reader = null;
        list<string> result = null;
        
        String line = null;
            try {/* System.setproperty ("Java.library.path", "/usr/local/hadoop/lib/native/linux-amd64-64/lib");
            Lzo = new Lzopcodec ();
            lzo.setconf (conf);
            is = Lzo.createinputstream (new FileInputStream (Lzofilepath));
            ISR = new InputStreamReader (IS);
            reader = new BufferedReader (ISR);
            result = new arraylist<string> ();
            while (line = Reader.readline ())!= null) {Result.add (line);
            } catch (FileNotFoundException e) {E.printstacktrace ();
        catch (IOException e) {e.printstacktrace ();
                Finally {try {if (reader!= null) {reader.close ();
                } if (ISR!= null) {isr.close ();
                } if (is!= null) {is.close ();
            } catch (IOException e) {e.printstacktrace ();
    } return result; }
}

The program is not wrong, but at the beginning of the run will always be prompted not to read libgplcompression This library, in fact, I know less of which libraries, respectively, is
Libgplcompression.a
Libgplcompression.la
Libgplcompression.so
libgplcompression.so.0
libgplcompression.so.0.0.0
But the question is where to put the libraries. Tried to put these libraries under the $classpath, but no use. So I looked at the error message, Hint The missing library is referenced in the Com.hadoop.compression.lzo.GPLNativeCodeLoader class, so look at the Hadoop-lzo-0.45.jar source file (when compiling Hadoop-lzo-0.45.jar, leave source file, in kevinweil-hadoop-lzo-6bb1b7f/src/java/com/hadoop/compression/lzo/), Gplnativecodeloader.java's content is like this:

Package Com.hadoop.compression.lzo;

Import Org.apache.commons.logging.Log;
Import org.apache.commons.logging.LogFactory;

public class Gplnativecodeloader {

  private static final log = Logfactory.getlog (Gplnativecodeloader.class);
  private static Boolean nativelibraryloaded = false;

  static {
    try {
      //try to load the Lib
      system.loadlibrary ("gplcompression");
      nativelibraryloaded = true;
      Log.info ("Loaded native GPL library");
    } catch (Throwable t) {
      log.error ("Could not load native GPL library", T);
      nativelibraryloaded = false;
    }
  }

  /**
   * Are the native GPL libraries loaded?
   * @return True if loaded, otherwise false
   *
  /public static Boolean isnativecodeloaded () {return
    nativelib raryloaded
  }

}

Here, the statement related to the Libgplcompression Library of load should be the System.loadlibrary ("Gplcompression") inside the try statement block; So I looked up the action of this LoadLibrary in the end. So I found the answer in this blog: http://blog.csdn.net/forandever/article/details/5983846
System.loadlibrary () load is the library in the path to which the JVM variable Java.library.path. Then I just have to add the folder where the Libgplcompression library is in the Java.library.path. So I looked for a way to set Java.library.path,
Method 1: Command line
java-djava.library.path=/path/to/libgplcompression/***.class
Method 2:java Statement
In the program to add such a sentence, System.setproperty ("Java.library.path", "/path/to/libgplcompression/");
I've found 2 methods, but these 2 methods can only temporarily change the value of Java.library.path.
In addition to these 2 methods, I can not find another way, exhausted, no longer find, simply put libgplcompression these libraries to copy to the folder Java.library.path point.
After the copy is finished, execute again, OK.
To get the value of the Java.library.path, you can use the Java statement
System.out.println (System.getproperty ("Java.library.path")); Mine is.
/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
Resources:
http://guoyunsky.iteye.com/blog/1266226
http://blog.csdn.net/forandever/article/details/5983846
Gpllibcompression Library and Hadoop-lzo-0.4.15.jar download links
Http://pan.baidu.com/s/1mgJQ1tQ