NOHZ_FULL, isolated CPUs and reading CPU MSR

Posted Apr 8, 2020 14:20 UTC (Wed) by nix (subscriber, #2304)
In reply to: NOHZ_FULL, isolated CPUs and reading CPU MSR by vstinner
Parent article: Frequency-invariant utilization tracking for x86

Kernel realtime developers told me that NOHZ_FULL cannot work with intel_pstate by design.

Really? This configuration is the common case for every distro kernel I checked. Sounds like we need better communication somewhere...

to post comments

NOHZ_FULL, isolated CPUs and reading CPU MSR

Posted Apr 8, 2020 22:26 UTC (Wed) by vstinner (subscriber, #42675) [Link] (3 responses)

> Kernel realtime developers told me that NOHZ_FULL cannot work with intel_pstate by design.

Sorry, my sentence is wrong: the issue is not NOHZ_FULL alone, but NOHZ_FULL+isolated CPUs. I understood that intel_pstate is not compatible with isolated CPUs using NOHZ_FULL.

NOHZ_FULL, isolated CPUs and reading CPU MSR

Posted Apr 9, 2020 0:44 UTC (Thu) by nix (subscriber, #2304) [Link] (2 responses)

Oh right, that makes a lot more sense and explains why this hasn't caused more trouble (isolated CPUs are an exceedingly rare use case in the sort of generalist domains where enterprise kernels are used).

NOHZ_FULL, isolated CPUs and reading CPU MSR

Posted Apr 18, 2020 18:20 UTC (Sat) by zlynx (guest, #2285) [Link] (1 responses)

I'm not too sure about "exceedingly rare" because several enthusiast forums I read give advice to use isolated CPUs, NOHZ_FULL and then explicitly assign CPUs to KVM virtual machines in order to get the very best Windows virtual machine performance.

Of course they also set performance to maximum, so this wouldn't affect power and frequency management.

Anyway, I believe this is more common than you may think.

NOHZ_FULL, isolated CPUs and reading CPU MSR

Posted Jun 2, 2020 1:36 UTC (Tue) by nix (subscriber, #2304) [Link]

Yeah: also enthusiast forums and enterprise kernels on stodgy old stability-first enterprise distros seem like things that won't be mixing very often. :)

(What they are presumably actually looking for here is CPU affinity with QEMU to try to keep a roughly 1:1 mapping between QEMU vCPU cores and real CPU cores. There were patches to do it inside QEMU but they never made it upstream and eventually bitrotted: it looks like libvirt does it from outside QEMU by brute force and cgroups.)