Usage of hue3.5.0 (CDH Version)

I have been using phphiveadmin and have been paying attention to Hue. I plan to investigate hue recently. Hue has developed rapidly in the last two years, and the page effects and functions have been greatly improved, more and more services are supported. Besides hive and hbase, sqoop, impala, and pig are also supported.

Shows a general hue architecture.

650) This. width = 650; "src =" http://www.cloudera.com/content/cloudera-content/cloudera-docs/CDH4/4.2.0/CDH4-Installation-Guide/images/image1.jpeg "alt =" images/image1.jpeg "/>

The core service is Hue server. sqllite is used as the default dB locally. You can view hue services and use related services through hue UI (that is, through a browser. Hue is implemented by Python and interacts with various supported services through thrift.

Currently, the cluster version is cdh5.0.0, and hue-related services are easy to install. You can use Yum to install them. The hue installation machine can be a machine that is out of the cluster.

sudo yum install hue

I am mainly studying the interaction between hue and hive this time. beeaswax is the module responsible for interaction with hive in hue. Therefore, we need to make some configuration before starting the hue service. Configurations are mainly divided into two aspects: one is the configuration that hue needs to do, and the other is the configuration that needs to be modified in the hadoop cluster to work with hue. The hue configuration file is/etc/hue/CONF/hue. ini. First, you need to perform some basic hue configurations. The main attributes are as follows:

[Desktop]

Secret_key: a random string of 30 to 60 characters. It is mainly related to cookies.

Http_host

Http_port: port used to access hue

If you use https, You need to configure SSL-related configurations.

Ssl_private_key

Ldap_password user password for LDAP Authentication

Beeaswax has the following attributes:

[Beeswax]

Hive_server_host hiveserver host

Hive_server_port port of hiveserver

Hive_conf_dir hive configuration file directory

Hive_server_bin bin directory of hiveserver

After hue-related configuration, you also need to configure some attributes of the hadoop cluster. The important ones are as follows:

Hive-site.xml

Hive. server2.enable. impersonation = true

Run the hive job as the task submitter. If this parameter is not configured, the job is run by the startup user of hiveserver by default.

Hdfs-site.xml

DFS. webhdfs. Enabled = true

Core-site.xml

Hadoop. proxyuser. Hue. Groups = *

Hadoop. proxyuser. Hue. hosts = *

For configuration related to webhdfs, hue needs to use webhdfs or httpfs to access HDFS. Configure either of them.

After the configuration is complete, you can start the hue Service (hiveserver needs to be started to manage hive using hue. hiveserver2 is recommended currently)

service hue start

Enter hue_host: hue_port in the browser to enter the logon page. The first input is the administrator's username and password. Other users and groups can be created through administrator permissions. After logging on, you can go to the hive editor to compile hql. After submission, you can view the job running status in real time, as shown in

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M00/38/FC/wKioL1O1HhHRl7qIAAFutGNNxv8362.jpg "Title =" qq 40703170529.jpg "alt =" wkiol1o1hhhrl7qiaafutgnnxv8362.jpg "/>

However, during subsequent use and testing, it was found that the use of hue to access hive was unstable, and the MetaStore information on the left side was always unable to be loaded. there were basically two errors when I checked the logs of hiveserver2, one is thrift-related connection exceptions, and the other is outofmemory.

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M01/38/FC/wKiom1O1H_mS-zofAA0P9gu4Xy4579.jpg "style =" float: none; "Title =" out.png "alt =" wKiom1O1H_mS-zofAA0P9gu4Xy4579.jpg "/>

650) This. width = 650; "src =" http://s3.51cto.com/wyfs02/M02/38/FC/wKiom1O1H_rhsBrOAA4sI6-foDQ219.jpg "style =" float: none; "Title =" outmeme.png "alt =" wKiom1O1H_rhsBrOAA4sI6-foDQ219.jpg "/>

I used jstat to check the usage of hiveserver2 heap memory, and did not find any obvious exceptions. Then I went to Google,

Hue's Google Contact Group has a discussion about these two issues (https://groups.google.com/a/cloudera.org/forum! Topic/CDH-user/nkm1griiijo). It can be determined that it is a bug level problem. Someone suggested to upgrade it to cdh5.0.1. Someone on Jira also reflected this problem (hive-6390 ), at present, I have not performed tests after the upgrade, and will continue to explore related issues in the future.

This article is from blog 17, please be sure to keep this source http://xiaowuliao.blog.51cto.com/3681673/1434099