Re: Portable vkernel (emulator)

[Matthew Dillon <dillon@xxxxxxxxxxxxxxxxxxxx>]

:It's not even finished yet, I don't think it's fair to hype it as a
:killer feature. There's a lot of proving and testing left before it is
:competitive with other modern filesystems. Right now it's still
:competing with UFSv1, which is how many decades old? Let's not get
:ahead of ourselves.

    Well, it isn't that bad :-).  All of HAMMER's major features are in
    the tree now and neither UFS1 nor UFS2 can hold a candle to any of them.

    But from a stability and use point of view I would agree that HAMMER
    needs to age a bit to achieve the reputation that UFS has acquired
    over the last 20 years.  This means people have to start using it for
    more then testing (once we release).

    This is why I'm making a big push to get it running as smoothly as
    possible for this release.  A lot of people outside the DragonFly
    project are going to be judging HAMMER based on its early use cases,
    and the clock starts ticking with the first official release.

:Last time this was discussed, it was revealed that the vkernel
:implementation is so efficient specifically because it required
:extensions to the host DragonFly. So yes, it's a pure userland process
:in DragonFly, but it uses system calls that Linux doesn't have (and
:may not even have an equivalent to, since user-mode-linux is
:implemented very differently to vkernel). If I recall correctly these
:were extensions to mailbox signals and page protection and stuff like
:that. The closest you could do is add analogues of those system calls
:and semantics to your host of choice.

    I think I stopped using the mailbox signals (they got replaced by the
    co-thread I/O model), but the vkernel still needs the syscall support
    for managing VM spaces and virtualized page tables.  I'm not sure
    what kind of APIs Linux has to support their UVM stuff.

:Until then, you can use KVM on Linux which has very close to hardware
:performance on modern CPUs. I use KVM to emulate FreeBSD on Linux and
:"I can't believe it's not native". Linux these days makes a fantastic
:virtual server, supporting every cross-platform virtualisation
:technology, most of them very efficiently. You can install Ubuntu
:Hardy in text mode, add on KVM and install DragonFly, all with 100%
:free software provided in the official repositories.
:
:In fact, just as we speak I'm using QEMU to install a big-endian
:Debian for testing C code's endian correctness. We're spoiled these
:days.
:...
:> Even if it might be super inefficient, it would be super cool to be able
:> to run a vkernel, which runs hammer, which runs samba, which serves as
:> fileserver for windoze ;-)
:> (or FreeBSD -> vkernel -> hammer -> nfs -> FreeBSD :)
:
:Yeah, you can do that with KVM. I'm not convinced that'd perform any
:better, or be any more reliable, than using native FreeBSD 7 with ZFS.
:
:-- 
:Dmitri Nikulin

    The KVM stuff is pretty cool but the performance claims, particularly
    by companies such as VMWare, are all hype.  The bare fact of the matter
    is that no matter what you do you still have to cross a protection
    boundary to make a system call or do I/O or take a page fault.  

    Hardware-supported hypervisors don't help much in that regard, they
    just aren't advanced enough yet.  They will get there, for sure, but
    it will be another couple of years at least.  In many respects as cpu's
    continue to evolve into more and more cores (or threads if you take
    Sun's viewpoint), the overheads imposed by hypervisors and KVMs will
    continue to drop.  Strangly enough hyper-threading might be the best
    solution for reducing KVM support overheads.

    The issue, for a KVM, is that crossing this boundary takes about 2uS
    verses the 150nS it takes on real hardware.  Everything else will run
    at full speed.  Also, to run reasonably well on a KVM kernels often have
    to be compiled to use far lower clock interrupt rates (an example of
    this would be the custom linux kernels IBM runs on their main-frames).
    For example, the scheduler interrupt might have to run at 50hz in a KVM
    environment instead of 1000hz.  This is because idle overheads are
    magnified considerably.  Multi-threading and interactive performance 
    of a kernel running under a KVM tends to suffer a lot due to the necessity
    of reducing idle and clock interrupt overheads.

    So the performance for something running under a KVM depends a lot on
    what that something is doing.  Cpu-bound programs which don't make many
    system calls (such as gcc), or I/O-bound programs which would be
    blocked on I/O much of the time anyway (such as sendmail), will
    perform fairly well.   System-call intensive programs, such as a web
    server, will lose a lot in the translation.

    A very large problem set is solved through the use of KVMs, even web
    serving, as long as the load does not exceed available cpu power.  As
    machines have gotten faster the cpu bar has continued to move up and
    more and more programs can be run in such an environment.

    If you are pegging your cpu to the wall and need performance, though,
    you don't want to use a KVM.  If you need more hardware connectivity
    then a simple network and disk interface, KVMs tends to be more trouble
    then they are worth.  A KVM emulating more complex and varied hardware,
    such as audio and video hardware, very quickly runs into trouble from
    a performance standpoint.

						-Matt