futex: Prevent robust futex exit race some more

Summary

In the Linux kernel, the following vulnerability has been resolved: futex: Prevent robust futex exit race some more A robust futex unlock stores 0 over the whole futex value - wiping FUTEX_WAITERS - and wakes a single waiter. That wakeup is a one-shot notification: the protocol relies on its recipient to either acquire the futex (and eventually unlock while aware of the remaining contention) or re-arm FUTEX_WAITERS before sleeping again. If the woken waiter is killed before it can do either, the kernel must jump in and wake the next task down the line. This is a known complication of the futex protocol with a previous partial fix in commit ca16d5bee598 ("futex: Prevent robust futex exit race"). Unfortunately, that fix is insufficient. If a third task re-acquired the futex through the uncontended fast path in the meantime, the notification is lost: robust exit processing sees that it is owned by another task and does nothing, while the new owner sees no FUTEX_WAITERS when it unlocks and wakes nobody. The remaining waiters sleep forever behind a free futex: A owns the futex, B and C sleep in FUTEX_WAIT uval == A | FUTEX_WAITERS A robust unlock: store 0, FUTEX_WAKE(1) wakes B uval == 0 D fast path acquire: cmpxchg(0 -> D) uval == D, no FUTEX_WAITERS B killed before acting on the wakeup B exit walk, pending op: owner D != B -> no action D unlock: no FUTEX_WAITERS -> no wake C sleeps forever This is clearly a shortcoming in the implementation, which fails to keep the FUTEX_WAITERS bit consistent. Work around this by augmenting the robust list exit processing to also perform the extra wakeup if the futex word is owned by another thread but FUTEX_WAITERS is not set. This does not fix the problem of a non-contended take over/release and free sequence, which has been discussed for years and has been addressed by commit 3ca9595d9fb6 ("futex: Add support for unlocking robust futexes") and subsequent changes, but failed to take the problem described above into account. A more complete solution which is based on the in kernel unlock of contended robust futexes has been discussed in the context of this change and should show up in mainline sooner than later. [ tglx: Amend change log slightly and fixup coding style ]