Installation of compression software snappy
Last Update:2017-05-24
Source: Internet
Author: User
<span id="Label3"></p><p><p>1, <span style="font-family: 宋体">Download the source code, by compiling the source installation</span></p></p><p><p>Tar-zxvf/home/zfll/soft/snappy-1.1.2.tar.gz</p></p><p><p>CD snappy-1.1.2</p></p><p><p>./configure</p></p><p><p>Make</p></p><p><p>sudo make install<span style="font-family: 宋体">(enter</span> <span style="font-family: 宋体">the root password)</span><br>2. <span style="font-family: 宋体">Install</span> hadoop-snappy <span style="font-family: 宋体">Package</span></p></p><p><p><span style="font-family: 宋体">Decompression </span>Unzip/home/zfll/soft/hadoop-snappy-master.zip</p></p><p><p><span style="font-family: 宋体">Compiling </span>CD hadoop-snappy-master/</p></p><p><p>MVN Clean Package-dmaven.test.skip=true</p></p><p><p><span style="font-family: 宋体">after compiling the local library files and</span> jar <span style="font-family: 宋体">packages (may have been compiled in a compressed package, you need to Unzip) to the corresponding directory</span></p></p><p><p>Cd/home/zfll/hadoop-snappy-master/target</p></p><p><p>TAR-ZXVF hadoop-snappy-0.0.1-snapshot.tar.gz</p></p><p><p>Cp-r hadoop-snappy-0.0.1-snapshot/lib/* $HADOOP _home/lib</p></p><p><p>CP Hadoop-snappy-0.0.1-snapshot.jar $HADOOP _home/lib<br>The Snappy 0.0.1-snapshot/ <span style="font-family: 宋体">directory contains both local and</span> <span style="font-family: Calibri">Hadoop-snappy-0.0.1-snapshot.jar</span> <span style="font-family: 宋体">packages that need to be copied to the appropriate directory. </span></p></p><p><p></p></p><p><p>3. <span style="font-family: 宋体">Configure</span> hadoop-env.sh <span style="font-family: 宋体">add:</span></p></p><p><p>Cd/home/zfll/hadoop/etc/hadoop</p></p><p><p>VI hadoop-env.sh</p></p><p><p>Export ld_library_path= $LD _library_path: $HADOOP _home/lib/native/linux-amd64-64/:/usr/local/lib/</p></p><p><p></p></p><p><p>4, <span style="font-family: 宋体">configuration</span> mapred-site.xml <span style="font-family: 宋体">increase</span></p></p><p><p>Cd/home/zfll/hadoop/etc/hadoop</p></p><p><p>VI Mapred-site.xml</p></p><pre><span style="color: #0000ff"><span style="color: #0000ff"><</span></span><span style="color: #800000"><span style="color: #800000"></span> property</span><span style="color: #0000ff"><span style="color: #0000ff">></span></span> <span style="color: #0000ff"><span style="color: #0000ff"><</span></span><span style="color: #800000"><span style="color: #800000">name</span></span><span style="color: #0000ff"><span style="color: #0000ff">></span></span>Mapred.compress.map.output<span style="color: #0000ff"><span style="color: #0000ff"></</span></span><span style="color: #800000"><span style="color: #800000">name</span></span><span style="color: #0000ff"><span style="color: #0000ff">></span></span> <span style="color: #0000ff"><span style="color: #0000ff"><</span></span><span style="color: #800000"><span style="color: #800000">value</span></span><span style="color: #0000ff"><span style="color: #0000ff">></span></span>True<span style="color: #0000ff"><span style="color: #0000ff"></</span></span><span style="color: #800000"><span style="color: #800000">value</span></span><span style="color: #0000ff"><span style="color: #0000ff">></span></span> <span style="color: #0000ff"><span style="color: #0000ff"></</span></span><span style="color: #800000"><span style="color: #800000"></span> property</span><span style="color: #0000ff"><span style="color: #0000ff">></span></span> <span style="color: #0000ff"><span style="color: #0000ff"><</span></span><span style="color: #800000"><span style="color: #800000"></span> property</span><span style="color: #0000ff"><span style="color: #0000ff">></span></span> <span style="color: #0000ff"><span style="color: #0000ff"><</span></span><span style="color: #800000"><span style="color: #800000">name</span></span><span style="color: #0000ff"><span style="color: #0000ff">></span></span>Mapred.map.output.compression.codec<span style="color: #0000ff"><span style="color: #0000ff"></</span></span><span style="color: #800000"><span style="color: #800000">name</span></span><span style="color: #0000ff"><span style="color: #0000ff">></span></span> <span style="color: #0000ff"><span style="color: #0000ff"><</span></span><span style="color: #800000"><span style="color: #800000">value</span></span><span style="color: #0000ff"><span style="color: #0000ff">></span></span>Org.apache.hadoop.io.compress.SnappyCodec<span style="color: #0000ff"><span style="color: #0000ff"></</span></span><span style="color: #800000"><span style="color: #800000">value</span></span><span style="color: #0000ff"><span style="color: #0000ff">></span></span> <span style="color: #0000ff"><span style="color: #0000ff"></</span></span><span style="color: #800000"><span style="color: #800000"></span> property</span><span style="color: #0000ff"><span style="color: #0000ff">></span></span></pre><p><p></p></p><p><p>VI Core-site.xml</p></p><pre><pre><span style="color: #0000ff"><</span> <span style="color: #800000"></span> property <span style="color: #0000ff">></span> <span style="color: #0000ff"><</span> <span style="color: #800000">name</span> <span style="color: #0000ff">></span>io.compression.codecs<span style="color: #0000ff"></</span><span style="color: #800000">name</span><span style="color: #0000ff">></span> <span style="color: #0000ff"><</span><span style="color: #800000">value </span> <span style="color: #0000ff">></span>org.apache.hadoop.io.compress.gzipcodec,org.apache.hadoop.io.compress.defaultcodec, Org.apache.hadoop.io.compress.bzip2codec,org.apache.hadoop.io.compress.snappycodec<span style="color: #0000ff"></</span><span style="color: #800000">value </span> <span style="color: #0000ff">></span><span style="color: #0000ff"></</span><span style="color: #800000"></span>Property<span style="color: #0000ff">></span> </pre></pre><p><p></p></p><p><p>Note that the above files, each node should be modified to increase</p></p><p><p><br>6<span style="font-family: 宋体">.</span><span style="font-family: Calibri">Hbase</span> <span style="font-family: 宋体">configuration</span> <span style="font-family: Calibri">Snappy</span></p></p><p><p>1)<span style="font-family: 宋体">,</span><span style="font-family: Calibri">cd/home/zfll/hbase/lib</span></p></p><p><p>Mkdir-p native/linux-amd64-64</p></p><p><p>Cp-r $HADOOP _home/lib/native/linux-amd64-64/* $HBASE _home/lib/native/linux-amd64-64/</p></p><p><p>2)<span style="font-family: 宋体">, configure</span> <span style="font-family: Calibri">hbase</span> <span style="font-family: 宋体">Environment Variables</span></p></p><p><p>Cd/home/zfll/hbase/conf</p></p><p><p>VI hbase-env.sh</p></p><p><p>Export ld_library_path= $LD _library_path: $HADOOP _home/lib/native/linux-amd64-64/:/usr/local/lib/</p></p><p><p>Export hbase_library_path= $HBASE _library_path: $HBASE _home/lib/native/linux-amd64-64/:/usr/local/lib/</p></p><p><p>VI hbase-site.xml <span style="font-family: 宋体">guaranteed start up in</span> <span style="font-family: Calibri">snappy</span> <span style="font-family: 宋体">mode</span></p></p><pre><pre><span style="color: #0000ff"><</span> <span style="color: #800000"></span> property <span style="color: #0000ff">></span> <span style="color: #0000ff"><</span> <span style="color: #800000">name</span> <span style="color: #0000ff">></span>hbase.regionserver.codecs<span style="color: #0000ff"></</span><span style="color: #800000">name</span><span style="color: #0000ff">></span> <span style="color: #0000ff"><</span><span style="color: #800000">value </span> <span style="color: #0000ff">></span>snappy<span style="color: #0000ff"></</span><span style="color: #800000">value</span><span style="color: #0000ff">></span> <span style="color: #0000ff"></</span><span style="color: #800000">Property </span> <span style="color: #0000ff">></span></pre></pre><p><p></p></p><p><p>3)<span style="font-family: 宋体">,</span> restart <span style="font-family: Calibri">hadoop</span> <span style="font-family: 宋体">and</span> <span style="font-family: Calibri">hbase</span></p></p><p><p><span style="font-family: 宋体">in the</span> <strong>server1-hadoop-namenode-01</strong> <span style="font-family: 宋体">Upper</span> stop-dfs.sh stop-yarn.sh</p></p><p><p>stop-hbase.sh</p></p><p><p>Start-dfs.sh start-yarn.sh</p></p><p><p>hbase-daemon.sh Start Master</p></p><p><p><span style="font-family: 宋体">in the</span> Start standby process on <strong>server2-hadoop-namenode-02</strong> :</p></p><p><p>yarn-daemon.sh Stop ResourceManager</p></p><p><p>yarn-daemon.sh start ResourceManager</p></p><p><p>hbase-daemon.sh Start Master</p></p><p><p><span style="font-family: 宋体">to</span> start the <span style="font-family: Calibri">hbase</span> <span style="font-family: 宋体"></span> <span style="font-family: Calibri">regionserver</span> <span style="font-family: 宋体">separately on</span> the HBase contact installation</p></p><p><p>hbase-daemon.sh Start Regionserver</p></p><p><p>4)<span style="font-family: 宋体">, verify</span> that <span style="font-family: 宋体">the</span> <span style="font-family: Calibri">Hbase</span> <span style="font-family: 宋体">configuration is successful</span> <span style="font-family: Calibri">(</span> <span style="font-family: 宋体">operation</span> on <strong>server1-hadoop-namenode-01</strong> )</p></p><p><p><span style="font-family: 宋体">next Create and manipulate</span> Snappy <span style="font-family: 宋体">Compressed-format Tables</span></p></p><p><p>1<span style="font-family: 宋体">.</span>hbase Shell</p></p><p><p></p></p><p><p>2<span style="font-family: 宋体">.</span><span style="font-family: Calibri">Create ' tsnappy ', {NAME = ' F ', COMPRESSION = ' Snappy '}</span></p></p><p><p>3<span style="font-family: 宋体">,</span><span style="font-family: Calibri">describe ' tsnappy '</span></p></p><p><p>4<span style="font-family: 宋体">,</span><span style="font-family: Calibri">put ' tsnappy ', ' row1 ', ' f:col1 ', ' value '</span></p></p><p><p>5<span style="font-family: 宋体">.</span><span style="font-family: Calibri">Scan ' tsnappy '</span></p></p><p><p>Installation of compression software snappy</p></p></span>
