Re: Hammer on-the-move

[Michael Neumann <mneumann@xxxxxxxx>]

Matthew Dillon wrote:
> :Hi,
> :
> :I think it shouldn't be too hard to switch a hammer master into a slave
> :and a slave to a master, isn't it?
> :...
> :Many years ago I really hoped that what I descibed above would work out
> :well using the Coda File System. Then came hammer... :)
> :
> :Regards,
> :
> :   Michael
>
>     You don't have to convince me :-)  I want to be able to do that to
>     implement fail-over support.

Hehe.

>     I restricted those options a few days ago (its basically just two lines
>     of code in the HAMMER user utility) and I haven't decided whether
>     to unrestrict them for the release or not.  The main reason is to
>     prevent early adopters of HAMMER from shooting themselves in the foot
>     because we do *NOT* have multi-master support yet and trying to
>     cross-mirror two masters can blow up a filesystem faster then chocolate
>     melts on a hot day.   The mirroring occurs at the B-Tree record level
>     and has no concept of high level filesystem topology.

Trying to understand what could be a worst-case scenario. Would I just
get mixed versions of files, i.e. a highly inconsistent view of files,
while the contents of the files stay the same, or would I get corrupted
file content?

What if every B-tree record would contain an origin field, which
identifies where this record was first created. When mirroring, this
field would not be modified, so it would become easy to "undo" a
mirroring operation just by removing all records of that origin.

>     To do multi-master merging support the mirroring program needs to be
>     made a lot smarter.  The low level structures *CAN* already identify
>     which master made a modification (the low 4 bits of the transaction id
>     will identify which master made the change, as long as every master

Ah, that's basically the "origin" field I'm talking about ;-)

>     is given a different master id).  But the mirroring program does not
>     yet use that information to resolve conflicts between masters when
>     you want to merge their data sets into a single coherent whole.

Do you plan to implement that (resolving conflicts)? I thought that a
multi-master hammer would work using quorums, so that no conflicts could
occur as there is always a well-defined state.

>     We are right on cusp of being able to do this but I am awfully worried
>     that enabling the feature in the release, before all the support
>     is in place, will cause too many user-support headaches and foot-shooting.
>
>     Maybe what I should do is allow slaves to be upgraded to masters but
>     put all sorts of 'are you really really really sure' warnings in if
>     someone tries to downgrade a master into a slave.  Either way I will
>     not allow the mirroring program to mirror between two masters, so
>     you'd have to downgrade a master into a slave first.  With the proviso
>     that merging two master data sets where BOTH may have been independantly
>     modified is strictly off limits, I could allow it for the release.

Yes, better be conservative here.

Regards,

  Michael