DragonFly kernel List (threaded) for 2006-03
HEAD bumped to 1.5.1, Preview tagged slipped, destabilization warning.
I have bumped HEAD's version to 1.5.1 and synchronized the
DragonFly_Preview tag with all recent commits. The current HEAD
and the recently updated release (1.4.2 now) are considered very stable.
However, we are fast approaching a time where some rather serious
surgery in HEAD is going to occur, so I am issuing a destabilization
warning for upcoming work. The work involves the following, in
approximate order of implementation:
* The buffer cache and I/O subsystem will be converted from 32 bit
block numbers to 64 bit offsets.
* The front-end read() code for vnodes will be adjusted to directly
access the buffer cache and only push into the VFS if the requested
data is not in the cache. It should be possible to completely bypass
vnode locks for cached reads.
I'm doing this bit as a 'carrot' for people to brag about before I
get to the really hard stuff.
* VOP_LEASE() will be completely ripped out, along with NFS-NQ support
(which never worked well in the first place).
Once that is all done the code base will be well positioned to finally
start implementing a universal MESI-style cache coherency management
infrastructure. 'universal' in this sense means that the infrastructure
will be implemented in the vnode and namecache layers, *ABOVE* the
VFS layer. That is, it will be filesystem-agnostic.
Building an efficient cache coherency management layer for the namecache,
vnode (file read/write), and VM object (mmap) is probably the single
most complex piece of code that is planned for DragonFly. Due to this
complexity I intend to have the kernel manage the structures, but
implement the entire protocol in userland. Cache management operations
will be made efficient (almost transparently so in terms of performance)
by granting cache states recursively for the namecache and by granting
a file's entire data space (64 bit range) for vnode/VM object access,
and only breaking the range(s) and topologies down when there is an
actual conflict between machines.
To give you an idea of how important this layer is going to be, consider
the idea of creating a shared memory segment by mmap()ing a NFS-based
file read/write. Our goal is nothing less then to make such a mapping
operate entirely coherently between machines as it does now between two
processes within a single machine. Our goal is to be able to, for
example, issue locked bus cycle instructions on such mmap()'d data
and have it work coherently across physical machines.
Ultimately this cache management abstraction will allow the VM space for
a process to be spread across multiple machines, migrated on the fly,
and a host of other very important operations to occur as if they were
occuring on a single machine. That in turn will form the basis which
will allow us to migrate processes across physical machines, and so on
and so forth. Its the lynch pin of the entire effort, so to speak.
I hope to get the basics of this infrastructure in place by USENIX, but
I don't know if I'm going to make it. Hence the performance carrot.