QEMU-KVM Threading Event Model "QEMU-KVM Code Analysis III"

QEMU-KVM Threading Event Model
1. Primary (parent) thread.
Main thread execution loop, mainly do three things
1). Perform a select operation, query file descriptor has no read and write operations
2). Execution Timer callback function
3). Perform the lower half (BH) callback function. Why to adopt BH, the data says main avoids reentrant and call stack overflow.

2. The thread that executes the client code
Discuss only the KVM execution client code (regardless of TCG,TCG using dynamic translation technology), if there are multiple vcpu, it means there are multiple threads.

3. Asynchronous IO File Manipulation thread
The commit I/O operation requests to the queue, which takes the request from the queue and processes it.

4. The main thread is synchronized with the executing client code
The main thread cannot run concurrently with the executing client code, mainly through a global mutex.

Code Analysis

1. Primary (parent) thread.
The following function is the primary execution function of the main thread: When the file descriptor, the timer, and the lower half trigger the corresponding event, the corresponding callback function is executed.
void main_loop_wait (int timeout) {
ret = SELECT (Nfds + 1, &rfds, &wfds, &xfds, &TV);
if (Ret > 0) {
Iohandlerrecord *pioh;
Qlist_foreach (Ioh, &io_handlers, next) {
if (!ioh->deleted && ioh->fd_read && fd_isset (IOH->FD, &rfds)) {
Ioh->fd_read (Ioh->opaque);
if (!) ( Ioh->fd_read_poll && Ioh->fd_read_poll (ioh->opaque))
FD_CLR (IOH->FD, &rfds);
}
if (!ioh->deleted && ioh->fd_write && fd_isset (IOH->FD, &wfds)) {
Ioh->fd_write (Ioh->opaque);
}
}
}

Qemu_run_timers (&active_timers[qemu_clock_host],
Qemu_get_clock (Host_clock));

/* Check bottom-halves the earlier events triggered
them. */
Qemu_bh_poll ();
}

For the Select function, the round file descriptor and the operation function for the description are mainly added through the Qemu_set_fd_handler () and QEMU_SET_FD_HANDLER2 functions.
int Qemu_set_fd_handler (int fd,
Iohandler *fd_read,
Iohandler *fd_write,
void *opaque);
int qemu_set_fd_handler2 (int fd,
Iocanrwhandler *fd_read_poll,
Iohandler *fd_read,
Iohandler *fd_write,
void *opaque)

For timer functions that are due to execute, the callback function is added by the Qemu_new_time function, and the trigger time Qemu_mod_timer function modifies the
Emutimer *qemu_new_timer (Qemuclock *clock, QEMUTIMERCB *cb, void *opaque)
void Qemu_mod_timer (Qemutimer *ts, int64_t expire_time)

The lower half is to add a dispatch function that is completed by Qemu_bh_new and Qemu_bh_schedule.
Emubh *qemu_bh_new (Qemubhfunc *cb, void *opaque)
void Qemu_bh_schedule (Qemubh *bh)


2. The thread that executes the client code
When the client hardware is initialized, a thread is created for each CPU, each thread executes the Ap_main_loop function, which runs the Kvm_run function and runs the client code.
/* PC Hardware initialisation * *
static void Pc_init1 (ram_addr_t ram_size,
const Char *boot_device,
const Char *kernel_filename,
const Char *kernel_cmdline,
const Char *initrd_filename,
const Char *cpu_model,
int pci_enabled)
{
for (i = 0; i < Smp_cpus; i++) {
env = PC_NEW_CPU (Cpu_model);
}

}

void Kvm_init_vcpu (Cpustate *env)
{
Pthread_create (&env->kvm_cpu_state.thread, NULL, Ap_main_loop, env);

while (env->created = 0)
Qemu_cond_wait (&qemu_vcpu_cond);
}
Executing the client thread calls the function Ap_main_loop, which ultimately calls the function Kvm_main_loop_cpu, which works as follows:
1. Inject interrupt, execute client code, resolve client exit reason, such as Kvm_exit_mmio, Kvm_exit_io. If the resolution succeeds, continue running. Failure, go to step 2
2. This step, if VCPU exists, is passed but has not yet been processed in signal Sig_ipi,sigbus, which is blocked, which means pausing the processor client code until the corresponding signal is processed.
3. If the above process is complete, continue to allow the client code to execute.
static int kvm_main_loop_cpu (Cpustate *env)
{
while (1) {
int run_cpu =!is_cpu_stopped (env);
if (run_cpu &&!kvm_irqchip_in_kernel ()) {
Process_irqchip_events (env);
RUN_CPU =!env->halted;
}
if (RUN_CPU) {
Kvm_cpu_exec (env);
Kvm_main_loop_wait (env, 0);
} else {
Kvm_main_loop_wait (env, 1000);
}
}
Pthread_mutex_unlock (&qemu_mutex);
return 0;
}
static void Kvm_main_loop_wait (cpustate *env, int timeout)
{
struct TIMESPEC ts;
int R, E;
siginfo_t Siginfo;
sigset_t Waitset;
sigset_t Chkset;

Ts.tv_sec = timeout/1000;
ts.tv_nsec = (timeout% 1000) * 1000000;
Sigemptyset (&waitset);
Sigaddset (&waitset, Sig_ipi);
Sigaddset (&waitset, Sigbus);

do {
Pthread_mutex_unlock (&qemu_mutex);

R = sigtimedwait (&waitset, &siginfo, &ts);
e = errno;

Pthread_mutex_lock (&qemu_mutex);

if (r = = 1 &&! ( E = = Eagain | | E = = eintr)) {
printf ("Sigtimedwait:%s\n", Strerror (e));
Exit (1);
}

Switch (r) {
Case Sigbus:
Kvm_on_sigbus (env, &siginfo);
Break
Default
Break
}
R = sigpending (&chkset);
if (r = = 1) {