DragonFly kernel List (threaded) for 2003-07
Re: Initial messaging / message-port infrastructure is in
Looks interesting... I need to get my FBSD box running again so I can
play with this
I suppose? [I clobbered it a few days ago :)]
Anyway.. what does LWKT mean? I somehow have missed it... I assume its
On Saturday, July 19, 2003, at 8:56PM, Matthew Dillon wrote:
I've committed the initial messaging/message-port infrastructure.
are the cvsweb links:
None of it is tested yet since nothing uses it yet (though DEV is
start using it in a degenerate form), but I think people will be
interested in seeing all of those comments I was making about
and IPIs realized by this commit.
This also shows off one of the big advantages of asynchronous IPI
messaging. The messaging and port functions do not need to obtain
any mutexes (even if there were no MP lock) even when they wind up
queueing or dequeueing something. But that isn't the only
because the messaging operations are combined with scheduling ops
one IPI will cover both the queueing and the scheduling aspects of
So right now we save at least 8 mutex equivalents with a single IPI
Traditional mutex Design LWKT/IPI design
cpu1: get_queue_mtx cpu1: send IPIQ to target cpu
cpu1: (IPI the target cpu anyway
in case it is idle??)
cpu2: (receive wakeup IPI??)
cpu2: get_sched_mtx cpu2: receive IPIQ
cpu2: locate_scheduled_thread cpu2: queue message
cpu2: rel_sched_mtx cpu2: schedule target thread
cpu2: get_queue_mtx cpu2: dequeue message
cpu2: dequeue_message cpu2: execute message function
cpu2: execute message function
Even if I were generous and ignored the fact that even in a mutexed
system cpu1 might want to wake up cpu2 with an IPI, the fact
that in a mutexed system *8* mutex operations are executed before
the message can be acted upon, and in the LWKT system *NO* mutex
operations are executed before the message can be acted upon.
The question then becomes: is the IPI latency (which costs neither
any actual cycles but can take a 'long' time (e.g. 1uS)) and the
interrupt overhead (on cpu2, which does eat cycles on cpu2) worth
8 *CONTESTED* mutexes we just saved? I would say: probably.
And when you add in the other efficiencies.. for example, the fact
lwkt_getport() and lwkt_waitport() (basically ALL message
involving the thread owning the message port in question) requires
a critical section to manipulate. No mutexes, no IPIs, no nothing.
Now is it worth it? I think so!