1. Esrejectedexecutionexception Exception Example
java.util.concurrent.executionexception:remotetransportexception[[node-client10][10.93.21.21:9300][indices:data/Write/update]]; nested:remotetransportexception[[node-Client the][10.93.18.35:9300][indices:data/Write/Update[s]]; nested:esrejectedexecutionexception[rejected Execution of Org.elasticsearch.transport.netty.MessageChannelHandler $Reque [email protected] on Esthreadpoolexecutor[index, queue capacity= $, [email protected]7f6431[running, Pool size = A, Activethreads= A, Queued tasks = $, Completed tasks =15656095]]];
2, esrejectedexecutionexception abnormal explanation
The esrejectedexecutionexception exception, in the literal sense, is that ES refuses to execute the request. The triggering scenario for this exception is as follows.
When using Elasticsearch, when the concurrent query volume is large, access traffic exceeds the processing power of a single Elasticsearch instance in the cluster, and the ELASTICSEARCH server triggers a protective mechanism that refuses to perform new access. and throws a Esrejectedexecutionexception exception.
This protection mechanism and exception triggering are determined by the thread pool in the Elasticsearch API implementation and the matching queue.
In the example, Elasticsearch is assigned a thread pool for the index operation, pool size=12, queue capacity=200, when 12 threads are not processed, and the queue has more than 200 buffered tasks, Then the new task is simply discarded, and the esrejectedexecutionexception exception is thrown.
A detailed explanation of the official link is here:
Https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-threadpool.html
Here are the default configurations for several important thread pools, many of which are related to processors (CPU core).
1) processors
Processors refers to the core number of the CPU, which can be automatically known and automatically introduced in the configuration of the online pool, processors do not have to be configured in Elasitcsearch.yml, of course you can rewrite the configuration.
Override method:
Processor:8
If you deploy multiple Elasticsearch node on a single machine, you can distribute the CPU cores evenly to different node, such as 2nodes deployed on 12 core machines, you can configure processors = 6.
2) thread pool and queue
A elasticsearch node will have multiple thread pools, but it is important to have the following four:
Index: Primarily index data and delete data operations (default is cached type)
Search: Mostly get, count and search operations (default is cached type)
Bulk operations (BULK): Primarily bulk operations on indexes (default is cached type)
Update (refresh): Mainly update operation (default is cached type)
genericfor generic operations (e.g., background node discovery). The Thread pool type is scaling. Index For index/delete operations. Thread pool type is the fixed with a size of # of available processors, queue_size of 200. The maximum size for ThisPool is 1 +# of available processors. Search For Count/search/suggest operations. Thread pool type is fixed with a size ofint((# of Available_processors * 3)/2) + 1, queue_size of 1000. Get For get operations. Thread pool type is the fixed with a size of # of available processors, queue_size of1000. Bulk For bulk operations. Thread pool type is the fixed with a size of # of available processors, queue_size ofThe maximum size for ThisPool is 1 +# of available processors. Snapshot For snapshot/restore operations. Thread pool type is scaling with a keep-alive of 5m and a max of Min (5, (# of available processors)/2). Warmer For segment Warm-up operations. Thread pool type is scaling with a keep-alive of 5m and a max of Min (5, (# of available processors)/2). Refresh For refresh operations. Thread pool type is scaling with a keep-alive of 5m and a max of Min (# of available processors)/2). Listener Mainly forJava Client executing of action when listener threaded are set totrue. Thread pool type is scaling with adefaultMax of Min (# of available processors)/2).
3) thread pool type
fixed
The fixed thread pool holds a fixed size of threads to handle the requests with a queue (optionally bounded) for pending R Equests that has no threads to service them. The size parameter controls the number of threads, and defaults to the number of cores times 5.The Queue_size allows to co Ntrol the size of the queue of pending requests that has no threads to execute them. By default, it was set to-1 which means its unbounded. When a request comes in and the queue are full, it'll abort the request.
The fixed thread pool maintains a number of threads to handle the request queue.
The size parameter sets the number of threads, the default setting being 5 times times the number of CPU cores.
The queue_size can control the size of the pending request queue. The default is set to-1, which means no limit. When a request arrives but the queue is full, the Reject_policy parameter can control its behavior. The default is abort, which causes the request to fail. Set to caller causes the request to execute in the IO thread.
ThreadPool: index: type:fixed size:30 queue:1000 Reject_policy:caller
scaling
The scaling thread pool holds a dynamic number of threads. This are proportional to the workload and varies between the value of the core and Max parameters. The keep_alive parameter determines how long a thread should is kept around in the thread pool without it doing any work.
The scaling thread pool maintains a dynamic number of threads. The dynamic change in the number of cores and Max, the time that Keep_alive is configured, maintains that the thread has not been called for a long time.
Thread_pool: Warmer: core:1 max:8 keep_alive:2m
Blocking
There are different types in different versions. The thread pool characteristics of the blocking type are
The blocking thread pool allows you to set a minimum value (min, default of 1) and thread pool size (size, which defaults to 5 times times the number of CPU cores). It also has a wait queue, and the queue size (queue_size) defaults to 1000 when the queue is full. It calls the IO thread based on the fixed wait time (Wait_time, which is 60 seconds by default) and will error if not executed.
ThreadPool: index: type:blocking min:1 size:30 wait_time:30s
Esrejectedexecutionexception Debug and thread pool type