DragonFly kernel List (threaded) for 2004-01
Re: Background fsck
Content-Type: text/plain; charset=us-ascii; format=flowed
X-Trace: 1074716458 crater_reader.dragonflybsd.org 181 126.96.36.199
Xref: crater_reader.dragonflybsd.org dragonfly.kernel:3374
Chris Pressey wrote:
> On Wed, 21 Jan 2004 11:10:52 -0500
> Gary Thorpe <gathorpe79@xxxxxxxxx> wrote:
>>Smarter and more self-contained devices that lie about caches being
>>on/off and cannot even automatically relocate bad sectors properly?
> OK, "cleverer" (in quotes) might have been a better choice of word than
> "smarter". :)
There was a thread on tech-kern@xxxxxxxxxx regarding relocation of bad
sectors and caching and the informal observations were that some IDE
drives LIE about this when the cache is enabled. That's such a wonderful
improvement don't you think? In fact, without hardware being able to
reliably and truthful inform the OS on what is happening, no filesystem
can guarantee anything. But ATA gets high sequential transfers so I
guess thats all that matters.
>>Data consistency is still the responsibility of the OS
> I respectfully disagree - data consistency is the responsibility of the
And a filesystem is a part of the OS, whether its in a monolithic kernel
or exists as a user-space server. Critical functionality like that is by
definition a part of the OS. Device drivers can run at least partially
in user-space, but I would still consider them OS components.
> There are two models you can follow here - roughly, the BSDs treat the
> filesystem as part of the OS while the Linuxes don't. I'd rather DFBSD
> go on a path closer to the Linux model. Instead of inventing a new
> journalled filesystem, why not use existing ones. At least to start.
I agree. Instead of redesigning one, it would be better to use an
existing one: ext3, reiserfs, xfs, jfs, bfs, vxfs - there are a lot of
choices. All of the open sourced ones are GPLed however and some like
XFS have almost as much source code as the rest of the linux kernel put
together. Existing designs may be MORE complex and harder to maintain
than softdeps: do you want that? If it is necessary, design one from
scratch using principles the others explored/built on. Its not impossible.
> This means fixing VFS, but that's got to be done anyway.
> Perhaps the filesystem will never migrate to the device itself, but on
> the other hand, there's no reason not to cut its umbilical cord.
Since the VFS is NOT the major obstacle to supporting jounraling (almost
all discussions I have seen end with "lets get LFS working right instead
" which implies that the people who will actually decide want to keep it
a BSD-based) and the VFS systems in all the BSDs already support
multiple file system, I don't see where you are going? Do you mean to
make them more modular/flexible to allow module loading unloading and
dynamic addition of filesystems? Do you honestly think Linux has a good
design for this or is it a hack (I don't know I am asking)?
>>Right: so design a file system with atomicity.
> Right, or adopt one, or several.
>>The way this is most commonly do is with journaling.
> Sure. But journalling != atomicity, and I don't care nearly as much
> about the former as I do the latter.
Yes journaling is atomicity: either a change makes it into the log or it
doesn't, or at least thats the impression I get on HOW they _should_ be
designed (with journaling commits being atomic). The other alternative
is to try and get EVERY file system operation to be atomic, which will
probably be infeasible or completely destroy performance. Disks can only
guarantee that small blocks are read/written atomically, so could you
please elabourate on how this would work?
>>Suppose the driver has a bug which cause the kernel to use an invalid
>>pointer: since most OS's are still monolithic, you are more unsure
>>about what you may have just corrupted (including FS code).
> Or suppose the kernel just refuses to use the invalid pointer.
Or suppose it IS valid, but it points to the wrong data and you
overwrite something and it is only caught later? What error handling can
you do: the error is asynchronous as it will either go undetected
immediately and be revealed later OR it will cause a trap. Unless you
want to add exception handling to a kernel, there is not much else you
can do if the error occurs in the same module as the core kernel (as in
a monolithic and not in a microkernel, although faults within a
microkernel and not one of the servers would have the same result).
> I guess the usual argument against that is that it's inefficient to
> determine the validity of every single pointer before using it, and to
> a great extent that's true. There's always going to be a tradeoff
> between efficiency and robustness.
Something like that is what hardware is needed for: AS400's implement
this I believe with hardware support (no FS journaling in hardware
though). I suppose it would be interesting to people working on fault
tolerance/corrections in things like space exploration, but I doubt
there is enough will to get it working on even commercial systems.
> So this brings me to something that's got me in a bit of a tizzy.
> Basically, the website is good at describing the technical goals of
> DFly, but it's a bit light on overall philosophy. The following design
> goals are mentioned on the main page:
> - Scaleability
> - Robustness
> - Debuggability
> - Compatibility (forwards and backwards)
> - Upgradeability
> The rest is nuts-and-bolts stuff, which is fine, but from it, you can't
> easily take Idea X for Improvement Y and see how well it fits in The
> Big Picture[tm]. That's why it's hard to evaluate stuff like
> internationalization and other suggestions that cross this list every so
> often - sure, they'd be great, lots of things would be great, but
> there's really nothing in the project that implies that it should (or
> shouldn't) get a high (or low) priority.
> Another (although much more hackneyed) way to put this, to paraphrase
> Ulrich Spoerlein's sig: "FreeBSD is the most powerful OS. NetBSD is the
> most portable OS. OpenBSD is the most secure OS. Linux is the most
> popular OS. So what is DragonFlyBSD?"
> Most scaleable? Most robust? Most debuggable?
Why does this matter? Who needs a cheap slogan anyway?
> Can the design goals of DFly even *be* reduced to a sound bite that fits
> in a sig? In a sense, I hope not, but in another sense, if they can't,
> the project's direction risks being unclear.
> Not sure what can be done about this except to lay out DFly's
> philosophical goals more explicitly.
> Anyway, sorry for ranting, take it with a grain of salt.