Hadoop Deep Research: Codec

Source: Internet
Author: User
Keywords nbsp through in depth to see
CODEC is actually an acronym for the prefix of the coder and decoder two words. Compressionhttp://www.aliyun.com/zixun/aggregation/29788.html ">codec defines a compression and decompression interface, The codec we're talking about here are the classes that implement some of the compressed formats of the Compressioncodec interface, and here's a list of these classes:

Using Compressioncodes decompression Compressioncodec There are two ways to compress and decompress easily. Compression: Compressionoutputstream Object decompression via Createoutputstream (OutputStream Out) method: Through Createinputstream (InputStream in) Method gets the sample code for Compressioninputstream object compression [Java] view plaincopy package com.sweetop.styhadoop;    import org.apache.hadoop.conf.configuration;  import org.apache.hadoop.io.ioutils;  import org.apache.hadoop.io.compress.compressioncodec;  import  org.apache.hadoop.io.compress.compressionoutputstream;  import org.apache.hadoop.util.reflectionutils ;   /**  * created with intellij idea.  * user:  lastsweetop  * date: 13-6-25  * time:  Afternoon 10:09  * to  change this template use file | settings | file templates.   */  public class streamcompressor {      public static void main (String[] args)  throws Exception {          String  codecClassName = args[0];          class<?> codecclass =  class.forname (codecclassname);          Configuration conf = new  configuration ();          compressioncodec codec =  (COMPRESSIONCODEC)  reflectionutils.newinstance (codecclass, conf);            Compressionoutputstream out = codec.createoutputstream (System.out);          Ioutils.copybytes (system.in, out, 4096, false);          Out.finish () ;     } }  accepts a COMPRESSIONCODEC implementation class parameter from the command line, and then instantiates the class by Reflectionutils, Call the Compressioncodec interface method to encapsulate the standard output stream, encapsulate it into a compressed stream, copy the standard input stream into the compressed stream through the Copybytes method of the Ioutils class, and finally call the Compressioncodec finish method. Complete the compression. Look at the command line again: [PLAIn] view plaincopy echo  "Hello lastsweetop"  | ~/hadoop/bin/hadoop  com.sweetop.styhadoop.streamcompressor  org.apache.hadoop.io.compress.gzipcodec | gunzip -   Use the Gzipcodec class to compress "Hello lastsweetop" and unzip it through the Gunzip tool. Let's take a look at the output: [Plain] view plaincopy [exec] 13/06/26 20:01:53 info util.820.html > nativecodeloader: loaded the native-hadoop library      [EXEC] 13/06/26  20:01:53 info zlib. zlibfactory: successfully loaded & initialized native-zlib library      [exec] hello lastsweetop  use compressioncodecfactory to decompress if you want to read a compressed file, First you have to determine which codec to use by extension, and you can look at Hadoop's in-depth study: (vii)--the corresponding relationship in compression. Of course there is a more convenient way, Compressioncodecfactory has helped you to do this, by passing in a path call it Getcodec method, you can get the corresponding codec. Let's take a look at the code [java] view plaincopy package com.sweetop.styhadoop;    import  Org.apache.hadoop.conf.Configuration;  import org.apache.hadoop.fs.filesystem;  import org.apache.hadoop.fs.path; 
Related Article

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.