對Hadoop2.7.2文檔的學習-Yarn部分(4)Yarn Application

來源:互聯網
上載者:User

標籤:分享   bsp   rms   第一個   actual   date   準備   htm   pos   

Writing YARN Applications

Link:http://hadoop.apache.org/docs/r2.7.2/hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html

Application submission client向Yarn ResourceManager提交一個Application,RM、NM、AM處理流程。 

首先,建立一個YarnClient對象並start它,然後Client可以設定ApplicationContext。為app準備第一個containercontaineApplicationMaster,然後提交Application。

RM在已經指定的Container中啟動ApplicationMaster。AM與YARN叢集通訊,處理Application的執行。在app啟動的過程中(app的啟動個AM與RM的通訊是非同步),AM的主要工作包括:

(1)與RM通訊,協商為之後的Containers分配資源(通過AMRMClientAsync對象,AMRMClientAsync.CallbackHandler指定事件的處理方法);

(2)Container分配之後,與NodeManagers通訊,啟動它們所在節點的app的Containers(啟動一個Runnable對象,當為Containers分配資源之後,啟動containers。作為啟動Container的一部分,AM需要指定帶有啟動資訊的ContainerLaunchContext)。

在Application執行的過程中,ApplicationMaster通過NMClientAsync對象與NodeManagers通訊。所有Containers的事件由NMClientAsync.CallbackHandler處理。

一個callback handler處理Client的start,stop,status update以及error。

 

 

 

原文:

The general concept is that an application submission client submits an application to the YARN ResourceManager (RM). This can be done through setting up a YarnClient object. After YarnClient is started, the client can then set up application context, prepare the very first container of the application that contains the ApplicationMaster (AM), and then submit the application. You need to provide information such as the details about the local files/jars that need to be available for your application to run, the actual command that needs to be executed (with the necessary command line arguments), any OS environment settings (optional), etc. Effectively, you need to describe the Unix process(es) that needs to be launched for your ApplicationMaster.

The YARN ResourceManager will then launch the ApplicationMaster (as specified) on an allocated container. The ApplicationMaster communicates with YARN cluster, and handles application execution. It performs operations in an asynchronous fashion. During application launch time, the main tasks of the ApplicationMaster are: a) communicating with the ResourceManager to negotiate and allocate resources for future containers, and b) after container allocation, communicating YARN *NodeManager*s (NMs) to launch application containers on them. Task a) can be performed asynchronously through an AMRMClientAsync object, with event handling methods specified in a AMRMClientAsync.CallbackHandler type of event handler. The event handler needs to be set to the client explicitly. Task b) can be performed by launching a runnable object that then launches containers when there are containers allocated. As part of launching this container, the AM has to specify the ContainerLaunchContext that has the launch information such as command line specification, environment, etc.

對Hadoop2.7.2文檔的學習-Yarn部分(4)Yarn Application

相關文章

聯繫我們

該頁面正文內容均來源於網絡整理,並不代表阿里雲官方的觀點,該頁面所提到的產品和服務也與阿里云無關,如果該頁面內容對您造成了困擾,歡迎寫郵件給我們,收到郵件我們將在5個工作日內處理。

如果您發現本社區中有涉嫌抄襲的內容,歡迎發送郵件至: info-contact@alibabacloud.com 進行舉報並提供相關證據,工作人員會在 5 個工作天內聯絡您,一經查實,本站將立刻刪除涉嫌侵權內容。

A Free Trial That Lets You Build Big!

Start building with 50+ products and up to 12 months usage for Elastic Compute Service

  • Sales Support

    1 on 1 presale consultation

  • After-Sales Support

    24/7 Technical Support 6 Free Tickets per Quarter Faster Response

  • Alibaba Cloud offers highly flexible support services tailored to meet your exact needs.