Re: off-box mirror-stream and friends

[Bill Hacker <wbh@xxxxxxxxxxxxx>]

Michael Neumann wrote:
> Am Sun, 15 Feb 2009 21:38:54 -0800 (PST)
> schrieb Matthew Dillon <dillon@apollo.backplane.com>:
>
> > :I have what appears to be a 'Catch 22', wherein:
> > :
> > :hammer mirror-stream /master <user>@<remote_IP>:/new_slave
> > :
> > :returns:
> > :
> > :PFS slave /new-slave does not exist.
> > :Do you want to create a new slave PFS? (yes|no) No terminal for
> > response :Aborting operation
> > :validate_mrec_header: short read
> > :
> > :'No terminal for response'  .was ass u me ed to be a byproduct of
> > comign :in off an Xfce4-terminal (Xorg & Xfce4 are quite happy on
> > 2.3.0, BTW) :
> > :Dropped back out to the raw tty0 console and tried it from there.
> > :
> > :No joy.
> >
> >     Definitely a bug in the hammer utility, I'm not sure there is 
> >     anything I can do about it though because the remote ssh
> > connection has no channel to accept a Y or N answer... stdin and
> > stdout are used for the protocol stream and I think stderr is output
> > only.
> >
> >     In anycase, I think what this means is that this feature currently
> >     only works if the slave is local (non-ssh connection).  So you
> >     would be able to do it with <remote_master> <local_slave>.
>
> Hm, I remember that we implemented this feature (auto-creation of
> slaves) so that it can operate over ssh. And IIRC, it once worked 
> for me using ssh (I am not sure if this was a remote machine or not).
> Does this mean, it is broken?

Hi, Michael,

I suspect it is an edge case.  IIRC, most of the DragonFly team work 
with ssh authed by matching pem certs or such.  I am far too 'mobile' 
for that, so still use passwords (long phrases, actually), and from any 
of many possible machines.

When using user@ for both ends, the existing code accepted ONE password, 
but only one, as the establishment of either end of the ssh session 
seems to have changed the environment w/r stdio.

*Despite which* the utility persisted in asking for the second password, 
ID'ing which machine wanted it, detected 'a' response 'coz it asked 
again, but did not accept it (UID, user number, and passwd same on both 
machines, so no confusion there).

At the same time, interspersing the odd 'yes' led to no joy, either.

So, yes, 'broken' in the general sense that parts of it (ssh) can 'see' 
the keyboard, the 'yes' seeker cannot (unless no password is required?).

In any case, scripts cannot easily render the 'yes' either.

Further, my view is that while over-writing a pre-existing slave is 
potentially dangerous, creating one that had not previously existed is 
harmless.

*EXCEPT* if/as/when it could be determined in advance that the 
known-size of the master exceeds the known-available space for a new 
slave on its intended  host.

'Wishlist' item, that sort of check...

> > :Command *appear* to succeed if/as/when I *manually* create
> > 'new_slave' :in advance with a matching shared_uuid. A local
> > mirror-copy to it :suceeds, with new_slave showing the files mirrored.
> > :
> > :However, while the -vvv flag gives 5-sec updates, they all show a
> > newer :starting point that pfs-status has for the target, and the
> > contents of :the slave never change.
> >
> >     You must access the slave via its softlink to get the latest
> > version synced from the master.  If you try to access the slave via a
> > null-mount you will be accessing a snapshot of the slave, not the
> > current state of the slave.  The null mount locks in the transaction
> > id of the slave.
> >
> > :By way of contrast, mirror-stream between on-box master and on-box
> > slave :  - same command otherwise - works fine.  No chdir needed to
> > see the :updates, just a 'View, Reload' in thunar and sputniks.
> >
> >     You are probably accessing it via the softlink, yes?  The gui is
> >     probably using an absolute path.  If you were to CD into a
> > sub-directory (even through the softlink), you would be accessing a
> > snapshot as-of when you did the CD, not the latest synced copy.

Not a concern. We'll adapt as suits.

A bit more good news:

Just before going off to a long meeting, I unplugged the CAT5 from the 
master. sh was not st to not time-out, so quit.

Four hours later, I came back, created a new file on the master, plugged 
in the CAT5, <up-arrow><Enter> to restart the stream. Saw the new file 
appear on the slave.

Painless, that. Clear and obvious that HAMMER did all the work.

Takes a failry pricey hardware RAID controller and hotswap to match the 
painlessness. And those do not remote well.

Nor do the re-sync in five seconds.

> > :Query: Can the loop that seeks a 'yes' be changed to a 5-second 
> > :countdown-timer with a message such as:
> > :
> > :Creating <new_slave> Hit Ctrl-c to abort
> > :
> > :.absent which it JFDI.
> > :
> > :Thanks,
> > :
> > :Bill Hacker
> >
> >     That won't work, the target over an ssh link has no tty channel.

Well - what I've done so far in that direction (Counown and Ctrl-c) very 
much DOES work, so I'll cite John J. Pershing on the US Army Corps of 
Engineers:

'The scientist said that it couldn't be done, but the damn fool Engineer 
didn't know that. So he did it.'

> >     Adding an option to create the slave automatically and passing it
> > to the target hammer utility when it is run via the ssh, so it never
> > has to ask at all, would work.  If someone would like to do that and
> > submit a patch, I don't think it would take more then 20 minutes of
> >     programming.

Done and in use. Patch to follow when I get the more elegant version tested.

> You mean, something like a -f (force) option? Should be damn easy to 
> implement. I can do that, once I sit in front of a real computer
> (with DragonFly) again :)

No. 'force' should be the default *but only in this case*.

The OTHER modes of failure seem to be things one would NOT want to 
force. For the tiem bing, I'm not fussed if the default to FAIL.

Think it through - most of the others require manual fixing.

If one has ssh at all, one can perform that.

ELSE neither can the toolset.

> Regards,
>
>   Michael

The process should probably NOT become so 'intelligent' as to do 
anything more benign than create a new slave (with proper shared_uuid)

'maybe someday' .. but no automated foot-shooting before the hammerfs 
gets a larger following and wider testing overall.

ZFS is a good example of a product fit for a (different) purpose being 
maligned because of too much early-adopter griping.

HAMMER has been spared that, and I suspect we all want to keep it that way.

Best,

Bill