DragonFly kernel List (threaded) for 2004-12
Re: Description of the Journaling topology
:All this work on the VFS layer looks very exciting and I agree with most
:of what you've said, specially the "Solaris did it that way" comment. To
:get to the point, I have a couple of questions about this
:Where will the log reside? As a special file in /, at the end of the
:partition, in another section of the disk? I take a transparent
:migration from normal UFS to journaled UFS will be provided, at least I
:hope so :)
The log is just a file descriptor, which means that it could represent
a special journaling device, a pipe to a process, a regular file, and
in particular it could represent a socket piping the journaled data
to an off-site machine.
The plan is to evolve this basic mechanism into a more sophisticated
one as time passes, introducing a stream in the reverse direction to
allow the journaling target to tell the journaling system when a
piece of data has been physically committed to hard storage. This
information could in turn be fed back to a journal-aware filesystem
but I would stress that awareness of the journal by the filesystem
is not a requirement. One can reap huge benefits from the journaling
mechanism whether the filesystem is aware of it or not.
:Are you going to do this as a "black box" or provide an API for
:tuning/configuring? I had this idea of writing a journaling FS for
:FreeBSD some time ago, even wrote some code and one of my ideas was to
The descriptor is roughly equivalent to the "black box". The
kernel's journaling layer will not know or care where the journaled
data is actually going.
:provide a mechanism of hinting the FS with what you wanted to with the
:file (IIRC VxFS has some of this functions). The idea was that you could
:open any file and issue a set of ioctl calls saying e.g. I want RAW
:access to this file, no buffer cache involved (this one I called direct
:i/o), or I want that this file deleted in a secure way, or I want
:that a kernel event is generated whenever an unauthorized user/program
:attempts to access the file. Stuff like that.
This is a more traditional approach to journaling... that is, fully
integrating it into the filesystem, but it also greatly reduces the
flexibility of a journaling system and given a properly managed VFS
API into the filesystem it should not be necessary to actually integrate
the journaling code into the filesystem at a thousand different points.
It would suffice to simply supply feedback to a journal-aware filesystem
with regards to when the journaled data has been committed to hard
:A problem I found was that concurrent access to the logging system
:exposes new problems you might not have taken into account when first
:designing such subsystem, but I'm sure you've thought about that
:already. What's your solution for a mutex-less concurrent access to the
:log? Is it possible to do without mutexes at all?
Yes and no. Concurrent access to the journal 'stream' is easy, you
just encapsulate the data into short-lived logical 'streams' (you'll
see the protocol spec commit on that tomorrow probably). Each
stream represents a transaction which, typically, means a VFS operation
(write, create, rmdir, mkdir, link, truncate, etc).
here is a quick example:
multiple journaling sources, one target
[------------------- IN MEMORY FIFO ----------------------]
[strmid,bytes][stream data]...[strmid,bytes][stream data]...
[id1][data] [id2][data] [id1][data] [id3][data] [id2][data] ...
process 1 process 1
This allows both blocking and non-blocking transactions to be output
to the journal as the data becomes available without tripping over
or stalling each other out.
Mutexless operation is a different issue. In the current scheme I am
implementing there is a single memory FIFO going between the N processes
issuing VFS ops and the (1) worker thread that is responsible for
writing the journal out.
However, with the logical stream abstraction there is no reason why we
couldn't have a per-cpu FIFO (one FIFO per cpu per journal). Since
DragonFly processes do not migrate between cpus while operating in the
kernel, even if they block, the serialization abstraction that the
logical stream mechanism provides would still work, and a per-cpu
memory FIFO would not require any mutexes to operate. The worker thread
would collect the data from all available FIFOs rather then from a single
FIFO. I am not going to do this initially because it is unnecessarily
complex at this early stage, but the API will easily support such an
:Other stuff I wanted to implement, but isn't really related to logging,
:was alternate streams (like NTFS has) and rich metadata, like BeFS had.
:What do you think about that?
Sure, and you have the choice of either integrating the mechanism
into a filesystem directly or providing an 'emulation' layer in the
kernel that provides the same API but is able to run on top of
filesystems that are not alternative-streams aware.
:I remember someone (maybe you) talking about per cylinder group dirty
:flags on UFS some time ago, to reduce fsck time. This could also be a
:nice addition, although this is definitely FS-dependent code.
I believe this is doable for UFS and could greatly reduce fsck
times. It isn't on my hot list (too many other interesting things
in my queue that I want to do).
:Like I've said, this looks very interesting and exciting area to work
:Miguel Mendez <flynn@xxxxxxxxxxxxxxxxxx> | lea gfx_lib(pc),a1