Change of InnoDB Buffer active merge

First, srv0srv.cc:: Srv_master_thread

Description

1, the main thread has 3 kinds of states: active, IDLE, shutdown state

2, Active:srv_master_do_active_tasks ()

3, Idle:srv_master_do_idle_tasks ()

4, Shutdown:srv_master_do_shutdown_tasks

1, Srv_master_thread

...

while (Srv_shutdown_state ==srv_shutdown_none) {

Srv_master_sleep ();

Monitor_inc (Monitor_master_thread_sleep);

Srv_current_thread_priority= srv_master_thread_priority;

if (srv_check_activity (Old_activity_count)) {

Old_activity_count= Srv_get_activity_count ();

Srv_master_do_active_tasks ();//active status

}else {

Srv_master_do_idle_tasks ();//idle status

}

}

while (Srv_master_do_shutdown_tasks (&last_print_time)) {//shutdown status

/*shouldn ' t loop here in case of very fast shutdown */

Ut_ad (srv_fast_shutdown< 2);

}

Pay attention to the loop here: Srv_master_do_shutdown_tasks loop to do, finish all the tasks, nothing to do before exiting.

...

2, Srv_master_do_active_tasks ()

Description

1. The second parameter of the Ibuf_contract_in_background function is False

...

/*do an ibuf merge */

srv_main_thread_op_info= "Doing insert buffer merge";

Counter_time= Ut_time_us (NULL);

ibuf_contract_in_background (0, FALSE);

Monitor_inc_time_in_micro_secs (

Monitor_srv_ibuf_merge_microsecond,counter_time);

...

3, Srv_master_do_idle_tasks ()

......

/* Do an ibuf merge */

Counter_time= Ut_time_us (NULL);

srv_main_thread_op_info= "Doing insert buffer merge";

ibuf_contract_in_background (0, TRUE);

Monitor_inc_time_in_micro_secs (

Monitor_srv_ibuf_merge_microsecond,counter_time);

......

4, Ibuf0ibuf.cc::ibuf_contract_in_background

Function:

Shrink insert buffer trees by reading pages to buffer pool

Input parameters:

TABLE_ID: If the merge operation is on the specified table. When 0 represents all tables

Full:true says caller to do fullcontract based on Pct_io (100)

False indicates the size of the Ibuf tree based on

Description

1, when full is true, N_pages=pct_io (100), that is, srv_io_capacity 100%

2. When full is false, depending on the ibuf tree size, how many pages of the merge are determined

N_pages=pct_io (5)

When Ibuf->size is greater than IBUF->MAX_SIZE/2

Diff=ibuf->size–ibu->max_size/2

N_pages=n_pages+pct_io ()

Ibuf->max_size+1 is to avoid dividing by 0.

......

if (full) {

/*caller have requested a full batch */

n_pages = pct_io (+);

} else {

/*by Default We do a batch of 5% of the io_capacity */

N_pages= Pct_io (5);

if (Ibuf->size > Ibuf->max_size/2) {

Ulintdiff = ibuf->size-ibuf->max_size/2;

N_pages + = Pct_io ((diff * +)/(Ibuf->max_size + 1));

}

}

The true merge operation is completed by the following loop

while (Sum_pages < n_pages) {

Ulint n_bytes;

n_bytes = Ibuf_merge (table_id, &n_pag2, FALSE);

if (n_bytes = = 0) {

return (sum_bytes);

}

sum_bytes+= n_bytes;

sum_pages+= N_pag2;

}

5, Srv_master_do_shutdown_tasks

Description

1, one time srv_master_do_shutdown_tasks with 100% io capacity to merge. Maybe a merge is not complete at one time. It doesn't matter, the merge succeeds then n_bytes_merged is not 0, then exits, the next cycle is true, still in the shutdown tasks loop

......

/* Do an ibuf merge */

srv_main_thread_op_info= "Doing insert buffer merge";

n_bytes_merged = ibuf_contract_in_background (0, TRUE);

......

Return (n_bytes_merged | | n_tables_to_drop);

Ii. Summary: Active merge

Active merge is in the master thread and is divided into 3 main cases:

1, Idle: the instance is in idle state, with 100% IO capacity to do the merge operation. The equivalent page number of a merge is equal to Innodb_io_capacity

2, Active: Instance in active state, this time back to the following algorithm to calculate the number of pages required to merge:

N_pages = Pct_io (5);

if (Ibuf->size >ibuf->max_size/2) {

Ulint diff = ibuf->size-ibuf->max_size/2;

N_pages + = Pct_io ((diff * +)/(Ibuf->max_size + 1));

}

Visible when the system is active back in a more moderate way to do the merge, if and ibuftree size exceeds the maximum value of half, then try to do more merge operations.

3, SHUTDOWN: When the execution of slow SHUTDOWN, will be forced to do a full ibuf merge

Problem:

1. How to interpret if the master thread is idle or in active state?

1) Srv0srv.cc::srv_inc_activity_count function completes the server activity count+1

This function is called on all task threads. For example:

Srv_master_do_active_tasks

Ibuf_contract_in_background (0,false)//xtradb Yes, InnoDB No, InnoDB may be in//ibuf_merge

while (Sum_pages < n_pages) {

...

N_bytes = Ibuf_merge (Table_id,&n_pag2, FALSE);

...

Srv_inc_activity_count ();

}

2) The main thread is compared with srv_check_activity every 1 seconds. If Srv_sys->activity_count is not changed, it is considered idle.

Srv_master_thread

......

while (srv_shutdown_state = = Srv_shutdown_none) {

Srv_master_sleep ();//1 sec

......

if (srv_check_activity (old_activity_count)) {

Old_activity_count= Srv_get_activity_count ();

Srv_master_do_active_tasks ();

}else {

Srv_master_do_idle_tasks ();

}

}

Srv_check_activity (Old_activity_count)

Return (Srv_sys->activity_count! = Old_activity_count);

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

Change of InnoDB Buffer active merge