MySQL source code (1) _ MySQL

MySQL Source Code Tracing (1) Note:

1. all of the following discussions are based on mysql 5.5.37 and official documents and are not necessarily applicable to other versions.

2. the full disk mentioned below refers to the disk partition where the data file (data file, log file, configuration file) is located.

3. due to space limitations, the final part of the code is only the key function and logic judgment part.

My colleague discussed a problem two days ago. when mysql is fully occupied from the database disk, show status and show slave status will be stuck, but other select operations will not be affected. However, if the database is a master database, when the disk is full, only dml will be blocked, and select and show will not be affected. So a group of people discussed for a while and finally decided, SMC. The following is my conclusion.

The official statement is as follows:

Https://dev.mysql.com/doc/refman/5.5/en/full-disk.html

1. check whether the instance has enough space to write data every minute. If you already have space, continue with the operation.

2. write a record to the log file every 10 minutes, and the report disk is full.

However, in my experiment, the above statement has many problems.

The test results of my official documents are as follows:

1. if binlog is enabled on the master database, the database reports a Disk is full writing every 10 minutes after the Disk is full '. /mysql-bin.000001 '(Errcode: 28 ). waiting for someone to free space... (keep Ct up to 60 secs delay for server to continue after freeing disk space), that is to say, the bin log is full and waits for disk space, which is the same as the description in this document.

2. if binlog is disabled on the master database, when the disk is full, any insertion will fail. the ERROR is [ERROR]. /mysqld: The table 'X' is full, which is not mentioned in The official documentation. The table x here is an innodb table.

The above is the response to the database instance when the disk of the master database is full. The following describes our situation: After the disk is full, the show status and show slave status will be stuck, however, other select operations are not affected.

First, the following is the conclusion:

The entire process involves three locks:

1. mi-> data_lock

2. LOCK_active_mi

3. LOCK_status

Note ):

1. after a new operation is received by the slave io thread, if the disk is full at this time, the write operation will be blocked and then wait until the disk has space to continue writing, in this operation, the mi-> data_lock lock will be held. the lock will be released only after the operation is completed or fails. if the disk is full, it is not an error and the operation is blocked, this thread will always hold the mi-> data_lock lock.

2. when a show slave status request is initiated, the LOCK_active_mi lock will be locked first and then the mi-> data_lock lock will be locked. of course, in the current situation, mi-> data_lock won't be obtained, so the LOCK_active_mi lock will be continuously held by this thread.

3. in addition, when a session initiates the show global status, the LOCK_status lock is first locked during execution. because the show status includes the statuses Slave_heartbeat_period, Slave_open_temp_tables, failed, and terminated, the LOCK_active_mi lock is also required, this session will also be blocked.

4. if another request is initiated later, all requests involving show status will be blocked because LOCK_status has been locked.

5. all subsequent show slave status requests will also be blocked at the LOCK_active_mi lock.

After reading the above conclusions, do you think of another operation sequence: disk full-> show status? the result of this operation is: show status will not be blocked.

The following are specific components of mysql source code (5.5.37:

1. io thread blocking related functions and some code

Slave. cc

Pthread_handler_t handle_slave_io (void * arg) if (queue_event (mi, event_buf, event_len) // write the slave log function {mi-> report (ERROR_LEVEL, identifier, ER (identifier ), "cocould not queue event from master"); goto err ;}

Slave. cc

Static int queue_event (Master_info * mi, const char * buf, ulong event_len) mysql_mutex_lock (& mi-> data_lock); mysql_mutex_lock (log_lock); if (likely (! (Rli-> relay_log.appendv (buf, event_len, 0) // write the execution function {mi-> master_log_pos + = inc_pos; DBUG_PRINT ("info", ("master_log_pos: % lu ", (ulong) mi-> master_log_pos); rli-> relay_log.harvest_bytes_written (& rli-> log_space_total);} mysql_mutex_unlock (log_lock); err: mysql_mutex_unlock (& mi-> data_lock );

Log. cc

Bool MYSQL_BIN_LOG: appendv (const char * buf, uint len ,...) {bool error = 0; DBUG_ENTER ("MYSQL_BIN_LOG: appendv"); va_list (args); va_start (args, len); DBUG_ASSERT (log_file.type = SEQ_READ_APPEND ); mysql_mutex_assert_owner (& LOCK_log); do {if (my_ B _append (& log_file, (uchar *) buf, len) {error = 1; goto err;} bytes_written + = len ;} while (buf = va_arg (args, const char *) & (len = va_arg (args, uint); DBUG _ PRINT ("info", ("max_size: % lu", max_size); if (flush_and_sync (0) // fl log data into the disk goto err; if (uint) my_ B _append_tell (& log_file)> max_size) error = new_file_without_locking (); err: if (! Error) signal_update (); DBUG_RETURN (error );}

Log. cc

Boolean MYSQL_BIN_LOG: flush_and_sync (bool * synced) {int err = 0, fd = log_file.file; if (synced) * synced = 0; mysql_mutex_assert_owner (& LOCK_log ); if (flush_io_cache (& log_file) return 1; uint sync_period = get_sync_period (); if (sync_period & ++ sync_counter> = sync_period) {sync_counter = 0; err = mysql_file_sync (fd, MYF (MY_WME); // synchronously write the file if (synced) * synced = 1;} return err ;}

Mf_locache.c

Int my_ B _flush_io_cache (IO_CACHE * info, int need_append_buffer_lock _ attribute _ (unused) if (mysql_file_write (info-> file, info-> write_buffer, length, info-> myflags | MY_NABP) info-> error =-1; elseinfo-> error = 0; mysql_file.hstatic inline alias (# ifdef HAVE_PSI_INTERFACE const char * src_file, uint src_line, # endif File file, const uchar * buffer, size_t count, myf flags) {size_t Result; # ifdef HAVE_PSI_INTERFACE struct PSI_file_locker * locker = NULL; PSI_file_locker_state state; if (likely (PSI_server! = NULL) {locker = PSI_server-> get_thread_file_descriptor_locker (& state, file, if (likely (locker! = NULL) PSI_server-> start_file_wait (locker, count, src_file, src_line) ;}# endif result = my_write (file, buffer, count, flags ); // write the file # ifdef HAVE_PSI_INTERFACE if (likely (locker! = NULL) {size_t bytes_written; if (flags & (MY_NABP | MY_FNABP) bytes_written = (result = 0 )? Count: 0; else bytes_written = (result! = MY_FILE_ERROR )? Result: 0; PSI_server-> end_file_wait (locker, bytes_written) ;}# endif return result ;}

My_write.c

Size_t my_write (File Filedes, const uchar * Buffer, size_t Count, myf MyFlags) {for (;) {# ifdef _ WIN32 writtenbytes = my_win_write (Filedes, Buffer, Count ); # else writtenbytes = write (Filedes, Buffer, Count); // call the system function # endif}/* my_write */

Errors. c

Void wait_for_free_space (const char * filename, int errors) {if (! (Errors % MY_WAIT_GIVE_USER_A_MESSAGE) {my_printf_warning (EE (EE_DISK_FULL), filename, my_errno, timeout); my_printf_warning ("Retry in % d secs. message reprinted in % d secs ", response, MY_WAIT_GIVE_USER_A_MESSAGE * handle);} DBUG_EXECUTE_IF (" simulate_no_free_space_error ", {(void) sleep (1); // exit return directly ;}); (void) sleep (MY_WAIT_FOR_USER_TO_FIX_PANIC); // wait time}

2. show slave status functions and some code

SQL _parse.cc

Bool dispatch_command (enum enum_server_command command, THD * thd, // forward request char * packet, uint packet_length) case COM_QUERY :... mysql_parse (thd, thd-> query (), thd-> query_length (), & parser_state );

SQL _parse.cc

Void mysql_parse (THD * thd, char * rawbuf, uint length, Parser_state * parser_state) error = mysql_execute_command (thd); define mysql_execute_command (THD * thd) case when: // execute {/* Accept one of two privileges */if (check_global_access (thd, SUPER_ACL | REPL_CLIENT_ACL) goto error; unlock (& LOCK_active_mi); // lock if (active_mi! = NULL) {res = show_master_info (thd, active_mi); // Get information} else {push_warning (thd, MYSQL_ERROR: WARN_LEVEL_WARN, WARN_NO_MASTER_INFO, ER (WARN_NO_MASTER_INFO )); my_ OK (thd);} mysql_mutex_unlock (& LOCK_active_mi); // unlock break ;}

Slave. cc

Bool show_master_info (THD * thd, Master_info * mi) if (mi-> host [0]) {DBUG_PRINT ("info", ("host is set: '% s '", mi-> host); String * packet = & thd-> packet; protocol-> prepare_for_resend (); /* slave_running can be accessed without run_lock but not other non-volotile members like mi-> io_thd, which is guarded by the mutex. */mysql_mutex_lock (& mi-> run_lock); protocol-> store (mi-> io_thd? Mi-> io_thd-> proc_info: "", & my_charset_bin); mysql_mutex_unlock (& mi-> run_lock); mysql_mutex_lock (& mi-> data_lock ); // lock mysql_mutex_lock (& mi-> rli. data_lock); mysql_mutex_lock (& mi-> err_lock); mysql_mutex_lock (& mi-> rli. err_lock );... mysql_mutex_unlock (& mi-> rli. err_lock); mysql_mutex_unlock (& mi-> rli. data_lock); mysql_mutex_unlock (& mi-> data_lock); // unlock if (my_net_write (& thd-> net, (uchar *) thd-> packet. ptr (), packet-> length () DBUG_RETURN (TRUE );}

3. show status functions and some code

Mysqld. cc

Static int show_heartbeat_period (THD * thd, SHOW_VAR * var, char * buff) {mysql_mutex_lock (& LOCK_active_mi); // lock if (active_mi) {var-> type = SHOW_CHAR; var-> value = buff; sprintf (buff, "%. 3f ", active_mi-> heartbeat_period);} else var-> type = SHOW_UNDEF; mysql_mutex_unlock (& LOCK_active_mi); // unlock return 0 ;}

SQL _show.cc

Static bool show_status_array (THD * thd, const char * wild, SHOW_VAR * variables, enum 1_value_type, truct system_status_var * status_var, onst char * prefix, TABLE * table, bool ucase_names, COND * cond) {for (var = variables; var-> type = SHOW_FUNC; var = & tmp) (mysql_show_var_func) (var-> value) (thd, & tmp, buff); // call the previous function show_heartbeat_period}

SQL _show.cc

Int fill_status (THD * thd, TABLE_LIST * tables, COND * cond) {if (thd-> fill_status_recursion_level ++ = 0) mysql_mutex_lock (& LOCK_status ); // lock if (option_type = OPT_GLOBAL) then (& tmp); res = show_status_array (thd, wild, (SHOW_VAR *) all_status_vars.buffer, option_type, tmp1 ,"", tables-> table, upper_case_names, cond); if (thd-> fill_status_recursion_level -- = 1) mysql_mutex_unlock (& LOCK_status); // unlock DBUG_RETURN (res );} mysql_mutex_lock (& mi-> data_lock );