[LinuxPPS] 13 Jul 09:37 - cpusets, hotplug, scheduler: fix scheduler domain breakage

[linuxpps: Dmitry Adamushko <>]

cpusets, hotplug, scheduler: fix scheduler domain breakage  
  
Commit f18f982ab ("sched: CPU hotplug events must not destroy scheduler  
domains created by the cpusets") introduced a hotplug-related problem as  
described below:  
  
Upon CPU_DOWN_PREPARE,  
  
update_sched_domains() -> detach_destroy_domains(&cpu_online_map)  
  
does the following:  
  
/*  
* Force a reinitialization of the sched domains hierarchy. The domains  
* and groups cannot be updated in place without racing with the balancing  
* code, so we temporarily attach all running cpus to the NULL domain  
* which will prevent rebalancing while the sched domains are recalculated.  
*/  
  
The sched-domains should be rebuilt when a CPU_DOWN ops. has been  
completed, effectively either upon CPU_DEAD{_FROZEN} (upon success) or  
CPU_DOWN_FAILED{_FROZEN} (upon failure -- restore the things to their  
initial state). That's what update_sched_domains() also does but only  
for !CPUSETS case.  
  
With f18f982ab, sched-domains' reinitialization is delegated to  
CPUSETS code:  
  
cpuset_handle_cpuhp() -> common_cpu_mem_hotplug_unplug() ->  
rebuild_sched_domains()  
  
Being called for CPU_UP_PREPARE and if its callback is called after  
update_sched_domains()), it just negates all the work done by  
update_sched_domains() -- i.e. a soon-to-be-offline cpu is included in  
the sched-domains and that makes it visible for the load-balancer  
while the CPU_DOWN ops. is in progress.  
  
__migrate_live_tasks() moves the tasks off a 'dead' cpu (it's already  
"offline" when this function is called).  
  
try_to_wake_up() is called for one of these tasks from another CPU ->  
the load-balancer (wake_idle()) picks up a "dead" CPU and places the  
task on it. Then e.g. BUG_ON(rq->nr_running) detects this a bit later  
-> oops.  
  
Signed-off-by: Dmitry Adamushko   
Tested-by: Vegard Nossum   
Cc: Paul Menage   
Cc: Max Krasnyansky   
Cc: Paul Jackson   
Cc: Peter Zijlstra   
Cc: miaox at cn.fujitsu.com  
Cc: rostedt at goodmis.org  
Cc: Linus Torvalds   
Signed-off-by: Ingo Molnar   
  
kernel/cpuset.c

URL: http://gitweb.enneenne.com/?p=linuxpps;a=commit;h=3e84050c81ffb4961ef43d20e1fb1d7607167d83