Hadoop Deep Research: Codec

CODEC is actually an acronym for the prefix of the coder and decoder two words. Compressionhttp://www.aliyun.com/zixun/aggregation/29788.html ">codec defines a compression and decompression interface, The codec we're talking about here are the classes that implement some of the compressed formats of the Compressioncodec interface, and here's a list of these classes:

Using Compressioncodes decompression Compressioncodec There are two ways to compress and decompress easily. Compression: Compressionoutputstream Object decompression via Createoutputstream (OutputStream Out) method: Through Createinputstream (InputStream in) Method gets the sample code for Compressioninputstream object compression [Java] view plaincopy package com.sweetop.styhadoop;    import org.apache.hadoop.conf.configuration;  import org.apache.hadoop.io.ioutils;  import org.apache.hadoop.io.compress.compressioncodec;  import  org.apache.hadoop.io.compress.compressionoutputstream;  import org.apache.hadoop.util.reflectionutils ;   /**  * created with intellij idea.  * user:  lastsweetop  * date: 13-6-25  * time:  Afternoon 10:09  * to  change this template use file | settings | file templates.   */  public class streamcompressor {      public static void main (String[] args)  throws Exception {          String  codecClassName = args[0];          class<?> codecclass =  class.forname (codecclassname);          Configuration conf = new  configuration ();          compressioncodec codec =  (COMPRESSIONCODEC)  reflectionutils.newinstance (codecclass, conf);            Compressionoutputstream out = codec.createoutputstream (System.out);          Ioutils.copybytes (system.in, out, 4096, false);          Out.finish () ;     } }  accepts a COMPRESSIONCODEC implementation class parameter from the command line, and then instantiates the class by Reflectionutils, Call the Compressioncodec interface method to encapsulate the standard output stream, encapsulate it into a compressed stream, copy the standard input stream into the compressed stream through the Copybytes method of the Ioutils class, and finally call the Compressioncodec finish method. Complete the compression. Look at the command line again: [PLAIn] view plaincopy echo  "Hello lastsweetop"  | ~/hadoop/bin/hadoop  com.sweetop.styhadoop.streamcompressor  org.apache.hadoop.io.compress.gzipcodec | gunzip -   Use the Gzipcodec class to compress "Hello lastsweetop" and unzip it through the Gunzip tool. Let's take a look at the output: [Plain] view plaincopy [exec] 13/06/26 20:01:53 info util.820.html > nativecodeloader: loaded the native-hadoop library      [EXEC] 13/06/26  20:01:53 info zlib. zlibfactory: successfully loaded & initialized native-zlib library      [exec] hello lastsweetop  use compressioncodecfactory to decompress if you want to read a compressed file, First you have to determine which codec to use by extension, and you can look at Hadoop's in-depth study: (vii)--the corresponding relationship in compression. Of course there is a more convenient way, Compressioncodecfactory has helped you to do this, by passing in a path call it Getcodec method, you can get the corresponding codec. Let's take a look at the code [java] view plaincopy package com.sweetop.styhadoop;    import  Org.apache.hadoop.conf.Configuration;  import org.apache.hadoop.fs.filesystem;  import org.apache.hadoop.fs.path;