Netback CPU affinity problem encountered in Kthread

Recent upgrade Netback, during the testing process, looked for VM full-duplex pressure, Rx pps fluctuated very badly, see Rx Kthread Despite CPU affinity it was 0-7 (Dom0 8vcpu), but often she went to physically destroy the CPU at.

Rx Kthread CPU is bound to other CPUs, Rx pps up, and stable, obviously Rx PPS fluctuation is due to CPU scheduling, is dispatched to different CPU caused.

There is a question, why the CPU scheduling, will be the Rx Kthread Dispatch to si% the highest CPU, load balance mechanism does not play a role?

Take a look at the code first. The implementation of the RX Kthread is a wait_event, and the packet is received from the NIC to call Vif's start_xmit. Trigger wake_up.

This means that the packet comes up from the NIC and triggers a soft interrupt on that CPU, and then wake_up our Rx Kthread to get up and work.

So does the Rx Kthread have something to do with Wake_up's CPU?

Xiantao Daniel, I looked at a KVM on the VM thread scheduling problem that they discovered very early, https://lkml.org/lkml/2010/4/11/108. Should be the same problem in essence.

Simple tracking of the code, TRY_TO_WAKE_UP will call Sched_fair.c's Select_task_rq_fair to pick a CPU, as the woken task of the execution CPU. Suppose Sched_feature.h defines the

Affine_wakeups so want_affine=1, then there is a AFFINE_SD, which indicates a affinity scheduling_domain, and then calls Wake_affine, which will be on the last executed CPU and the current Wake_ Up

The CPU. For some load related comparison, choose whether to select an idle sibling (select_idle_sibling) based on the PREV_CPU or WAKE_UP CPU

In our scene. Want_affine=1, Wake_affine=1, select_idle_sibling (wake_up CPU). And the WAKE_UP CPU is also idle. Although the soft interrupt is very high, there is no other thread scheduling. Soft interrupts are triggered in the idle context all the time.

If the above conditions are met, the new CPU try_to_wake_up get is WAKE_UP CPU. In rare cases, wake_affine=0, the new CPU is still prev_cpu, equals no migration.

Have done an experiment, put sched_features.h inside the affine_wakeups = 0, then Want_affine = 0. Would not go up to the logic above. Finally, basically there is no migration, has been executed on the PREV_CPU.

No previous fluctuations, improved performance.

Copyright notice: This article Bo Master original articles, blogs, without consent may not be reproduced.

Netback CPU affinity problem encountered in Kthread