cdh installation

1. Configure the yum SourceLs-L/dev | grep CDMkdir/mnt/CDROMMount/dev/CDROM/mnt/CDROMCD/etc/yum. Repos. dBack up and delete other Yum sourcesVI media. Repo[Rh6-media] Name = rh6-mediaautorefresh = 0 baseurl = file: // MNT/CDROM/gpgcheck = 0 enabled = 1Yum clean allYum makecacheIi. Install KerberosRefer to another article:Add an RPM package: rpm-IVH krb5-server-ldap-1.10.3-65.el6.x86_64.rpm3. Install LDAPYum install OpenLDAP openldap-servers openldap-clients openldap-devel compat-OpenLDAPInstall

Recently CDH cluster frequent alarm, because some host frequent swapping, greatly affected the performance of the cluster.Later found a setting (/proc/sys/vm/swappiness) needs to be modified, the default value of 60Setting the vm.swappiness Linux Kernel Parametervm.swappinessis a Linux Kernel Parameter This controls how aggressively memory pages are swapped to disk. It can set to a value between 0-100; The higher the value, the more aggressive the ker

Error when executing hive count query:Error:java Heap SpaceThe solution is set io.sort.mb=10;Error when executing Hadoop exeample, also Java heap space problemDiagnostic Messages for this Task:Error:java Heap SpaceFailed:execution Error, return code 2 from Org.apache.hadoop.hive.ql.exec.mr.MapRedTaskMapReduce Jobs Launched:Stage-stage-1: map:1 reduce:1 hdfs read:0 HDFs write:0 FAILTotal MapReduce CPU time spent:0 msecHive execution hql prompt error Error:java heap spaceJava Heap Space

configuring CDH and Managing servicesTuning of HDFs before closing DatanodeRole requirements: Configurator, Cluster Administrator, full Administratorwhen a datanode is closed, Namenode ensures that each block in each Datanode is still available based on the replication factor (the replication factor) across the cluster. This process involves the block duplication of small batches between datanode. In this case, a datanode has thousands of blocks, and

How to do integration, in fact, especially simple, online is actually a tutorial.http://blog.csdn.net/fighting_one_piece/article/details/40667035 look here.I'm using the first integration. When you do, there are a variety of problems. Probably from from 2014.12.17 5 o'clock in the morning to 2014.12.17 night 18 o'clock 30 summed up in fact very simple, but do a long time AH Ah!!! This kind of thing, a fall into your wit. Question 1, need to refer to a variety of packages, these packages to bre

1Create a collection SSH connects remotely to the CDH node that has SOLR installed. Running the solrctl instancedir--generate/solr/test/gx_sh_tl_tgryxx_2015 command generates the default configuration for the Gx_sh_tl_tgryxx_2015 collection. Enter the/solr/test/gx_sh_tl_tgryxx_2015/conf directory, first edit the Schema.xml configuration field information, the specific online search one piece. Solrconfig.xml file in the other The following

/54/41/wKioL1R9Zgrxe0EmAAFxN4JJK7M227.jpg "style=" float: none; "title=" snap4.jpg "alt=" Wkiol1r9zgrxe0emaafxn4jjk7m227.jpg "/>650) this.width=650; "src=" http://s3.51cto.com/wyfs02/M01/54/41/wKioL1R9ZgqTaU5IAAMQFy65Jcc514.jpg "style=" float: none; "title=" snap5.jpg "alt=" Wkiol1r9zgqtau5iaamqfy65jcc514.jpg "/>8. Start the Cloudera Management ServiceUpgrade CDH to 5.2.11. Before you BeginRun sudo-u HDFs hdfs fsck/and sudo-u HDFs hdfs dfsadmin-report

/init.d/cloudera-scm-agent stopThe other salve nodes are also executed separately:/opt/cm-5.0.0/etc/init.d/cloudera-scm-agent stop3. Then start SCM Server in master, and all nodes start the SCM Agent separately/opt/cm-5.0.0/etc/init.d/cloudera-scm-server start/opt/cm-5.0.0/etc/init.d/cloudera-scm-agent start4, the last Login Cloudera Manage Management page, to see if it can re-detect the CDH version of each host, if unable to detect, you need to click

One: the preparatory work1. Steps1) Hadoop-"Download Unzip"-"Modify the configuration file-"hadoop-envJava_home-"Core-siteFs.defaultfsHadoop.tmp.dir-"Hdfs-siteDfs.replicationPermission-"Mapred-siteMapreduce.frame.workHistoryserver-"Yarn-siteMapreduce-"ShuffleResourceManager Address: 0.0.0.0Log Aggregation-"yarn-envJava_home-"SlavesDatanode/nodemanager hostname-"FormattingBin/hdfs NAMENODE-FORMATF-"Start2) Hive-"Download Unzip"-"Create Data Warehouse"/user/hive/warehouse-"Modify the configuration

When I deleted a hive table today, I found that the HDFS space was not released. At first I thought it was a problem to delete the table. The results were not found on HDFS, the final result is that the CDH namenode has a file system spam interval setting. The default setting is one day. That is to say, it takes one day to delete the deleted file.Configuration, hoping to help people with the same questions. 650) This. width = 650; "src =" http://s3.51

CDH cm interface has an unreachable situation, view cloudera-scm-server status # Service Cloudera-scm-server Status Cloudera-scm-server dead but PID file exists Hint Cloudera-scm-server dead# Service Cloudera-scm-server Stop# Service Cloudera-scm-server StatusCloudera-scm-server is stopped Delete Cloudera-scm-server.pid# Rm/var/run/cloudera-scm-server.pid Service Cloudera-scm-server-db StartDoes not start properly cloudera-scm-server-dbWaiting fo

Carbondata is a new type of tabular file format for distributed computing, this time using Spark-thrift mode to operate Carbondata, briefly describes how to start Spark-carbondata-thriftserver. version CDH 5.10.3 spark 2.1.0 carbondata 1.2.0 download spark https://archive.apache.org/dist/spark/spark-2.1.0 /spark-2.1.0-bin-hadoop2.6.tgz Carbondata https://dist.apache.org/repos/dist/release/carbondata/1.2.0/ Apache-carbondata-1.2.0-source-release.zip ca

Hadoop Foundation----Hadoop Combat (vi)-----HADOOP management Tools---Cloudera Manager---CDH introduction We have already learned about CDH in the last article, we will install CDH5.8 for the following study. CDH5.8 is now a relatively new version of Hadoop with more than hadoop2.0, and it already contains a number of components and components in the Hadoop ecosystem that we need to learn next. Environ

Method The installation method is yum. Because the Intranet machine cannot access the Internet, we must build a local yum warehouse, yum warehouse machine 10.100.3.17.Build a local YUM Repository The rpm installation package required for yum installation includes the cloudera manager 5 related installation package: ht

First, IntroductionCDH is a commercial product developed by Cloudera Corporation to rapidly deploy and efficiently manage Hadoop and its various components. It is mainly divided into two parts, Cloudera Manager and CDH package respectively. Where Cloudera Manager is responsible for the deployment and management of the cluster. The CDH package includes installation