Cdh4.1.2 integration of hadoop and oozie

Source: Internet
Author: User

1. The exception information is as follows:

Caused by: COM. google. protobuf. serviceexception: java.net. connectexception: call from slave4/10.95.3.65 to 0.0.0.0: 10020 failed on connection exception: java.net. connectexception: disconnects; for more details see: http://wiki.apache.org/hadoop/ConnectionRefused
Later, by debugging and tracking the hadoop source code, we found that oozie will indeed connect to jobhistory after submitting the task,
After the debug mode is enabled, in org. Apache. hadoop. mapred. clientcache:
Protected mrclientprotocol instantiatehistoryproxy ()
Throws ioexception {
Final string serviceaddr = Conf. Get (jhadminconfig. mr_history_address );
If (stringutils. isempty (serviceaddr )){
Return NULL;
}
Log. debug ("connecting to historyserver at:" + serviceaddr );
Final yarnrpc RPC = yarnrpc. Create (CONF );
Log. debug ("connected to historyserver at:" + serviceaddr );
Usergroupinformation currentuser = usergroupinformation. getcurrentuser ();
Return currentuser. DOAs (New privilegedaction <mrclientprotocol> (){
@ Override
Public mrclientprotocol run (){
Return (mrclientprotocol) RPC. getproxy (hsclientprotocol. Class,
Netutils. createsocketaddr (serviceaddr), conf );
}
});
}
The above code can tell that the historyserver is connected, so I use the command
Mr-jobhistory-daemon.sh start historyserver to start this service, so on NN will start the jobhistoryserver process, the process of listening port is 10020, I thought this problem can be overcome, I didn't expect the following error to be reported in the oozie log. The only possible reason is that oozie didn't read the jobhistory configuration. Therefore, I set this Configuration:
<Property>
<Name> mapreduce. jobhistory. address </Name>
<Value> master: 10020. </value>
</Property>
Again in oozie/CONF/hadoop-CONF/core-site.xml file added again, the above error disappears, job can also run properly.


2. after oozie is installed, caused by: org is always reported when the task is submitted. apache. openjpa. lib. JDBC. reportingsqlexception: Data truncation: data too long for column 'proto _ action_conf 'at Row 1 {prep1_nt 184665592 insert into wf_jobs (ID, app_name, app_path, Conf, group_name, parent_id, run, user_name, bean_type, auth_token, created_time, end_time, external_id, last_modified_time, log_token, proto_action_conf, sla_xml, start_time, sta Tus, wf_instance) values (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?) [Params = ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?]} [Code = 1406, state = 22001]


After searching, the proto_action_conf field in wf_jobs stores workflow in the workflow. XML content. The field type is text (MySQL) When oozie creates a metadata table by default. The maximum length of this field is

65536 characters of the text column, so sometimes it cannot accommodate the content of workflow. XML, you need to change it to the longtext type, just change it.

 

Contact Us

The content source of this page is from Internet, which doesn't represent Alibaba Cloud's opinion; products and services mentioned on that page don't have any relationship with Alibaba Cloud. If the content of the page makes you feel confusing, please write us an email, we will handle the problem within 5 days after receiving your email.

If you find any instances of plagiarism from the community, please send an email to: info-contact@alibabacloud.com and provide relevant evidence. A staff member will contact you within 5 working days.

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.