DragonFly kernel List (threaded) for 2005-06
Re: SPL vs. Critical Section vs. Mutexing for device synchronisation
Matthew Dillon wrote:
This only works because we hold the BGL. Without the BGL critical
sections cannot be used to protect against interrupts. Therefore,
while the interrupt subsystem is able to depend on critical sections
now, IT WON'T BE ABLE TO IN THE FUTURE. A new MP-SAFE API is needed.
As I understand it, Joerg's major point is that until we have such an
MP-safe API, we shouldn't change the running system yet, but instead do
it at once once the new API has been tested.
Frankly I don't see how we can possibly avoid the use of a
locked bus cycle instruction for either case. The only way to truely
avoid locked bus cycles is through cpu localization (aka using IPIs
to execute all related operations on the same cpu).
I am in favour of this. From the performance point of view, servicing
interrupts on another machine is very degrading, and I assume that DFly
isn't supposed to be run on just SMP machines sharing APICs, but also on
clusters of machines with small to broad interconnects. Also, any
related activity, i.e. pushing packets through the network stack, should
be done on the CPU which has the shortest path to the device in question.
The use of cpu localization is one of DragonFly's major goals. I
*WANT* to use a cpu localization mechanism whenever possible, and indeed
we are using such a mechanism in our networking code with incredible
results. But that doesn't mean that cpu localization works in every
Oh wait. I should have read this first :). Can you give me an example of
a case in which CPU localization doesn't work?
In anycase, these are all very complex issues. Even if we come up with
a clean solution, we can't go from step A to step Z in a single step.
Even though -Development is already here, isn't it an option to run a
separate tree just for this major change, so we can compare them once
this is 'done'?
-- Thomas E. Spanjaard