Druid Indexing Service

The indexing service consists of three main components: one is the peon component, which can run a task, the other is the middle managers component, which manages peons, and the other is the Overlord component which manages tasks and distributes them to middle managers. Overlord and middle managers can be on one server or different servers. Middle managers and peons are generally on the same server.

Indexing Service Flowchart

 

Overlord Service

The Overlord node is responsible for accepting tasks, coordinating task allocation, creating task creation locks, and returning the status to the caller. Overlord can be configured to run in either of the two modes-local or remote (local default ). In local mode, overlord is also responsible for creating peons to execute tasks. When overlord runs in local mode, all middle managers and peon configurations must be provided. Local Mode is usually used for simple workflows. In remote mode, both overlord and middle managers run in separate processes and can run on different servers. This mode is recommended if you plan to use a single Indexing Service as the index for all druid services.

Submit and Query Task status

Submit a task. The task is submitted to Overlord in JSON format. The request method is post. The ID taskid of the submitted task is returned.

 
Http ://<Overlord_ip>:<Port>/Druid/Indexer/V1/task
 

Cancel the task. The request method is post. After the request is sent, the task kill-9.

 
Http ://<Overlord_ip>:<Port>/Druid/Indexer/V1/task/{taskid}/Shutdown
 

 

View the task. The request method is get.

Http ://<Overlord_ip>:<Port>/Druid/Indexer/V1/task/{taskid}/segments
 

 

 

Overlord Console

 

The Overlord console can be used to view waiting tasks, running tasks, recently available tasks, and task creation and termination.

 
Http ://<Overlord_ip>:<Port>/Console.html
 

 

Middle Managers

 

Http://druid.io/docs/0.6.145/Middlemanager.html

 

Peons

See peon.

Tasks

See tasks.