Xen4.2 added new CPU scheduling Parameters

Xen 4.2 will contain two new scheduling parameters for the credit1 scheduler which significantly increase its confurability and performance for cloud-based workloads:Timeslice_msAndRatelimit_us. This blog post describes what they do, and how to configure them for best performance.

Timeslice

The timeslice for the credit1 has historically been fixed at 30 ms. this is actually a fairly long time-it's great for computationally-intensive workloads, but not so good for latency-sensitive workloads, special ones involving network traffic or audio.

X en 4.2 introducesTslice_msParameter, which sets the timeslice of the scheduler in milliseconds. This can be set either using the xen command-line option,Sched_credit_tslice_ms, Or by using the new scheduling parameter interfaceXL sched-credit:

# XL sched-credit-T [N]


Interesting values you might give try are 10 ms, 5 ms, and 1 ms. one millisecond might be a good choice for particle ularly latency-sensitive workloads; but beware that causes the timeslice also increases the overhead from context switching and CES the specified tiveness of the CPU cache. values of 5 ms or 10 ms give a good balance. the default, 30 ms, is probably too long; but we're re going to do some more experimentation and probably switch the default in 4.3. if you try any values that turn out to be particle Ly good or bad, let us know.

Schedule Rate limiting

Hui LV over at Intel did some fascinating work analyzing the performance overhead of running specvirt inside of xen. what he discovered was that under certain circumstances, some VMS were waking up, doing a few microseconds worth of work, and going back to sleep, only to be woken up microseconds later. the credit1 scheduler correctly identified that these were probably latency-sensitive applications and gave them priority to run whenever they needed. the problem was that they were causing thousands of schedules per second-in some cases up to 15,000 schedules per second. this meant that there was a very significant amount of time actually spent in the scheduler switching back and forth between the two tasks, rather than doing the actual work.

the credit1 sched1_was giving these processes microsecond latency; but there are very few network-based workloads that require Sub-millisecond latency. hui worked with the xen development community to introduce a simple mechanic that wocould be predictable and robust, and improve the performance for this workload without degrading the performance of other workloads. the result was ratelimit_us . the ratelimit is a value in microseconds. it is a minimum amount of time which a VM is allowed to run without being preempted. the default value is 1000 (that is, 1 ms ). so if a VM starts running, and another Vm with higher priority wakes up, if the first VM has run for less than 1 ms, it is allowed to continue to run until its 1 ms is up; only after that will the higher-priority VM be allowed to run.

One millisecond is not generally too long for network-based workloads to wait; and the effect is to have more "batching", so the whole system is used more than tively. this caused significant increase in specvirt performance in Hui's tests.

This feature can be disabled by setting the ratelimit to 0. one cocould imagine, on a computation-heavy workload, setting this to something higher, like 5 ms or 10 ms; or if you have a participates ularly latency-sensitive workload, bringing it down to 500us or even 100us.

This value can be set either on the xen command-line usingSched_ratelimit_us(Note no "credit" in this one-it's meant to be consumed by other schedulers as well) or the XL command-line:

# XL sched-credit-R [N]


Curent values of both can also be viewed using XL:

# XL sched-creditcpupool pool-0: tslice = 30 ms ratelimit = 1000 usname ID weight CapDomain-0 0 256 0
 

Wait, what's this "cpupool" thing, you say? That's another new feature in xen 4.2-but the topic of another blog post. Watch this space!

From: http://blog.xen.org/index.php/2012/04/10/xen-4-2-new-scheduler-parameters-2/