DragonFly BSD
DragonFly users List (threaded) for 2006-11
[Date Prev][Date Next]  [Thread Prev][Thread Next]  [Date Index][Thread Index]

Fwd: load balancing

From: "Miguel Figueiredo Mascarenhas Sousa Filipe" <miguel.filipe@xxxxxxxxx>
Date: Thu, 2 Nov 2006 11:29:31 +0000


On 10/31/06, Joerg Sonnenberger <joerg@xxxxxxxxxxxxxxxxx> wrote:
 On Tue, Oct 31, 2006 at 12:24:05AM +0100, Simon 'corecode' Schubert wrote:
> Joerg Sonnenberger wrote:
> >The problem with proxy architectures is of course the second copying,
> >esp. kernel -> userland -> kernel, which can put a bunch of additional
> >load on frontend machines. Using approaches similar to state-full
> >firewall handling e.g. of FTP can be used for this as well, e.g. make it
> >a transparent proxy. This is a lot more work though. I'm not aware of
> >any such Open Source solution though. It shouldn't be hard to do that
> >e.g. in a small kernel module though.
> are you thinking of a way to bond two file descriptors together?  that
 > would be nice.  first you do all header processing and whatnot, and then
> you just say
> bond_fd(server_fd /* read side */, client_fd /* write side */);

That and/or a generalised sendfile. That would be more in the line of
sendfile(src, dst, size), whre src can be a socket as well. Both is
useful for a number of situations to avoid unncessary round trips.

check out the new splice() and tee() syscalls on linux, they allow for
zero-copy data movement...


quoting from Linus Torvalds explanation:

Now, the advantage of splice()/tee() is that you can do zero-copy movement
of data, and unlike sendfile() you can do it on _arbitrary_ data (and, as
shown by "tee()", it's more than just sending the data to somebody else:
you can duplicate the data and choose to forward it to two or more
different users - for things like logging etc).

So while sendfile() can send files (surprise surprise), splice() really is
a general "read/write in user space" and then some, so you can forward
data from one socket to another, without ever copying it into user space.

Or, rather than just a boring socket->socket forwarding, you could, for
example, forward data that comes from a MPEG-4 hardware encoder, and tee()
it to duplicate the stream, and write one of the streams to disk, and the
other one to a socket for a real-time broadcast. Again, all without
actually physically copying it around in memory.

Best Regards,

-- Miguel Sousa Filipe

[Date Prev][Date Next]  [Thread Prev][Thread Next]  [Date Index][Thread Index]