DragonFly BSD
DragonFly kernel List (threaded) for 2004-12
[Date Prev][Date Next]  [Thread Prev][Thread Next]  [Date Index][Thread Index]

Re: Description of the Journaling topology

From: Gary Thorpe <gathorpe79@xxxxxxxxx>
Date: Tue, 28 Dec 2004 14:41:13 -0500

Matthew Dillon wrote:

When all is said and done the journaling mechanism is going to look like this:

			      -> [ MEMORY FIFO ] -> [ worker thread ] -> STREAM
			     /	   e.g. 16MB	    [ secondary spool]
    VFSOP -> [journal shim]--				e.g. 16GB
            (transaction id) \
			      -> [filesystem VFS op]

------> target (e.g. an off-site machine). STREAM | <----------+ [transid acks going back]

	     STREAM = generic file descriptor.  e.g. regular file, socket,
	     fifo, pipe, whatever.  Half or full duplex.

This ASCII art does not at all look clear on the reader I am using....?

Eventually (not in two weeks) the journaling layer will make these acked
transaction ids available to any journal-aware VFS filesystem allowing
the filesystem to leverage the kernel's journaling layer for its own use
and/or to control the underlying filesystem's own management of commits
to physical storage. I also intend to use the journaling layer, with
suitable additional cache coherency protocols, to handle filesystem
synchronization in a clustered environment. In particular, an ability
to do high-level cache-coherent replication that would be immune to catastrophic corruption rather then block-device-level replication which
tends to propogate corrupting events. As you can see, I have *BIG* plans
for the journaling layer over the next few years.


I think referring to Journaling is a bit misleading as these features will not help the traditional case of recovery after power loss on a file system (all buffered data would be lost as it is in memory). It seems to be more of an integration of transactions into the VFS layer, so that the kernel will use a Transaction topology when reading/writting data. I am guessing that the actual file system/remote file system will do any 'journaling' and indicate when transactions are committed etc. How will non-journaling file systems work? Will this inter operate with anything else that is not DragonFlyBSD (if it doesn't it makes remote backup less useful).

Could do all of this from a user application (a lot of distributed file system's work from user land I think)?

[Date Prev][Date Next]  [Thread Prev][Thread Next]  [Date Index][Thread Index]