YARN Timeline Server Introduction

1. Background introduction:
The monitoring of tasks performed prior to the hadoop2.4 version only developed a job history Server for Mr, which provides users with information about jobs that have already been run, but later, as more and more computing frameworks are integrated on yarn, such as Spark, Tez, it is also necessary to develop the corresponding Job task monitoring tool for the technology based on these computing engines, so Hadoop developers are considering developing a more general job history server, yarn Timeline server.

2.Introduction
Register the current and historical state of the application to YARN Timeline server in a common way for easy storage and retrieval. It has two major responsibilities:
(1) Specific information for persistent applications
Collect and retrieve specific information about the application or framework. For example, information about the relationship between the Shard lines in the Hadoop Mr Framework, such as map tasks, reduce tasks, counters, and so on. Application developers can send this information to timeline Server via timelineclient on the app master or in the containers required by the application.
This information can be queried through the rest APIs in the specific app or UI interface of the execution framework.
(2) Generic information for persistent completed applications
In this regard, in application history server, it is clear that only the job of the Mr Framework is supported. Generic information includes user program-level data like *queue-name,* user information, as well as information set in Applicationsubmissioncontext, The list of application-attempts used to run the application, information about each application-attempt, container list and running in each application-attempt * Information for each container under the

3.Timeline Structure

(1) Timeline Domain
Timeline Domain provides a command space for Timeline server that allows users to collect multiple nodes and isolate them from other users and applications. Timeline server security is defined at this level.
A domain is first used to store the user's information, read and write ACL information, create and modify timestamps. Each domain is identified in the entire yarn cluster with a unique ID.
(2) Timeline Entity
A timeline entity (that is, timeline entities) contains meta-information about a conceptual entity and its related events. An entity can be a application, a application attempt, A custom object for any other application of container.
It also contains multiple primary filters used as an index for multiple entities in the timeline store. Other data can be stored in a non-indexed manner. Each entity is uniquely determined by a entityid and EntityType.
(3) Timeline Events
Timeline events is used to describe an event associated with a Timeline entity of a specific application. Users can also arbitrarily define an event method, such as launching an application, getting the assigned container, failing the operation, or other failure information related to user and cluster operations, and so on.