HAMMER update 25-June-2008 - HEADS up on pruning / historical access changes

[Matthew Dillon <dillon@xxxxxxxxxxxxxxxxxxxx>]

    Here's another update on HAMMER's progress.

    As people know, HAMMER is basically ready for release minus the mirroring
    code.  I believe I will have the mirroring code done by release time.
    A few other loose ends also need to be dealt with, like removing numerous
    calls to Debugger() and returning EIO for those situations instead of
    panicing.

    I have hit a snag with the mirroring code that will necessitate a change
    to the way the pruning code works.  In a nutshell, the pruning code
    removes 'dead' records inbetween snapshot softlinks.

    What people may not remember is that the pruning code also 'fills the
    gap' left by removing a dead record.  It does this by adjusting the
    creation and deletion transaction ids of adjacent records, in order
    to maintain a continuum in case the user does a random as-of query
    with transaction ids which are not on snapshot boundaries.

    --

    Unfortunately this realignment breaks every conceivable mirroring
    algorithm I can think of, even algorithms which only mirror 'current'
    data, because the creation transaction id of a current data record
    can be realigned backwards in time.

    To solve this problem I have decided to adjust the pruning code to 
    simply remove the dead records and NOT realign adjacent records.

    This means that you will only be able to safely access historical data
    that is:

    (1) On a snapshot (softlink) boundary.

    and

    (2) Any continuum of transaction ids between the most recent snapshot 
	softlink and 'now'.  This fine-grained access will still work
	properly.

    I'll repeat that.  Fine-grained access still works, I haven't broken it.
    What doesn't work any more is trying to access the filesystem between
    two softlink snapshot tranasction ids.  That information of course doesn't
    exist anyway due to the pruning, but before the change it would at least
    represent one of the adjacent snapshot boundaries.  Now, after the change,
    you will get garbage if you try to do that.

    Making this change will make mirroring possible, and I've decided that
    it is probably a good idea to have HAMMER formally track the pruning
    demarcation points anyway (though that part I may not get in by the
    release).  Once HAMMER does formally track the demarcation points it
    won't be possible to specify illegal transaction ids in your historical
    accesses which will neatly hide the fact that such accesses now return
    junk.


						-Matt