Build qemu-kvm with native VDE support

Thanks for submitting this bug. Indeed VDE support would be nice to have. The hangup is that libvde-dev is in universe, which will require a MIR (see https://wiki.ubuntu.com/MainInclusionProcess). Would you be willing to follow up on that?

Setting this to Confirmed and Medium priority, and hoping we can get libvde-dev into main during oneiric cycle.

Thanks. I've opened a MIR #776818.

Note, this bug is related to or even possibly a duplicate of #253230.

MIR: https://bugs.launchpad.net/ubuntu/+source/vde2/+bug/776818
Related: https://bugs.launchpad.net/ubuntu/+source/qemu-kvm/+bug/253230

Quoting Joseph Coffland (<email address hidden>):
> MIR: https://bugs.launchpad.net/ubuntu/+source/vde2/+bug/776818
> Related: https://bugs.launchpad.net/ubuntu/+source/qemu-kvm/+bug/253230

Thanks very much, Joseph.

Ok, re-reading all the related bugs, I see that I was misunderstanding
the status of this. Specifically, I missed Jamie's comment:
https://bugs.launchpad.net/ubuntu/+source/vde2/+bug/776818/comments/13,
"it is unclear if the server team actually wants this".

Users want it. I (as member of server team) am not opposed to it.

Ideally someone in the community would be willing to maintain it. I'll
happily help, so lack of packaging experience should not stop anyone.

I do think it's worth a debate over whether this can be considered
"duplicate functionality".

How is this "duplciate functionality" ? I know there are other ways of providing networking support for qemu-kvm, but none that I know of provide the same functionalilty as vde that I'm aware of. (bridging and nat provide networking support in different ways).

Quoting Dwight Schauer (<email address hidden>):
> How is this "duplciate functionality" ? I know there are other ways of
> providing networking support for qemu-kvm, but none that I know of
> provide the same functionalilty as vde that I'm aware of. (bridging and
> nat provide networking support in different ways).

As far as inclusion in main is concerned, it may be close enough. I
don't know how similar they need to be for it to be a problem. It'd be
worthwhile listing here (or actually in the vde2 MIR bug) the things
that vde can do which the others cannot, to cut off such an argument.

NAT is of limited usefulness if I want to use network sevices provided by my VM guests. (ssh tunneling can be used but it makes for limited uselfulness).

Bridging is invasive because it takes over the primary host networking interface (there may be ways around this, but they are not straight forward).

With vde2 the VMs and host share a seperate network (I use iptables for forwarding). Using vde2 the primary physical host network interface is left alone. With vde2 one can use VM guest services from the host and vice versa without any special forwarding (via ssh or something similar). I use iptables and dnsmasq on the host to provide gateway, DHCP, and DNS for the VMs.

With vde2 I can use the VMs on a laptop and still have full access to them, and the VMs can indirectly use (via iptables) whatever internet route is being used by the host.

Thanks, Dwight.

The MIR team is very overloaded this cycle, but I intend to push for the MIR decision right around UDS time.

I'd like to expand on Dwight's comments a bit. Qemu network cards have two sides, the side the VM sees (virtio is probably fastest, if you can get drivers for it), and the "backend", which is how that's connected to anything else. Here are the backends it has:

-net user. Dirt simple to set up. Pure user-mode networking -- the VM intercepts all requests the guest makes, and just does those again on the host. Doesn't need any special privileges, and for most purposes, it's basically just like having a NAT. It even has a bit of other scattered functionality, like a built in SMB server, so Windows machines can access the host filesystem. Not the fastest, though, and not terribly flexible.

-net tap. This would be plenty for Dwight's use case. The host gets a "tap" interface, which is connected (as if via a crossover cable) to the guest -- with pretty much full networking capabilities. May need to run as root. With iptables, you can certainly set up forwarding, and let guest VMs talk to each other. You can run dnsmasq on the host to provide DHCP and DNS for the VMs. It's just slightly obnoxious, because this is one tap device per client.

-net socket -- two modes here: TCP or UDP multicast. TCP lets you wire one NIC on one VM to another NIC on another VM -- basically, a crossover cable. You *could* build a VDE-like switch out of this, by having one VM connect directly to the host with TAP, and have it bridge to other VMs connected with the socket. But this is a pain -- you're basically building a switch out of a VM, which is hugely inefficient compared to VDE. Multicast probably isn't as pleasant -- I don't know enough about multicast to be sure, but it looks like this could end up broadcasting to the entire real network -- better than bridging, but not terribly secure.

-net vde -- actual VDE support. VDE lets you do much more than what Dwight is asking for. You can create entire virtual networks, with functioning switches and everything, that aren't connected to the host at all -- and you can connect these to other VMs on other machines. Tonight, I wanted to play with UDP hole punching (http://en.wikipedia.org/wiki/UDP_hole_punching) and other NAT traversal stuff, but I don't have a spare router, and my desktop actually has a real Internet IP address (no NAT). VDE would've let me create as many virtual routers as I wanted, between as many virtual networks as I wanted, all virtualized inside my desktop. It's not just that bridging is a bad idea here, it's that it wouldn't even solve the problem -- the *whole point* is that I want to "punch through" to an otherwise inaccessible network.

I should mention that vdekvm actually does not work for what I want to do, at least as far as I can tell. When I tell it to connect to a switch, it actually tries to create that switch and connect it to the host -- but I want a network that _isn't_ connected to the host!

Actually I have been using "-net tap". I don't run qemu-kvm as root per se, although the "tapper" script runs as root to bring configure the tap interface and assign it the user running qemu-kvm.

Sufficient to say, VDE is more convenient to set up than bridging/tapping.

What's the status on this? I've been wanting this for some time now too and the comments left here are not very clear as to what the actual stumbling block is for why this isn't fixed yet.

Connecting n VMs together without connecting them to the host and without being root is possible in only two ways at the moment:
- Using UDP MC sockets which are terribly inefficient (A file transfer between two VMs through ssh managed to load my machine up to 2% CPU. The same transfer via the user network between VM and host loads it up to about 50% as it should (4 cores))
- Using VDE which currently requires me to recompile qemu/kvm.

Can anybody from the server team or whoever knows about these processes comment on what it would take to get this progressed?

A previous commented noted the MIR which would have been needed to move vde2 into main so that qemu could depend on it.

https://bugs.launchpad.net/ubuntu/+source/vde2/+bug/776818

It was rejected by the security team.

I'd like to add to the discussion, that it's annoying that qemu does not complain when you try to invoke it with an option like:

-net vde

that qemu was built without kvm support. So a user, like myself, might spend hours trying to figure-out what's misconfigured and preventing qemu networking from working, when, in fact, the problem is that the option is simply unsupported.

That seems like a very valid objection. Could you please file a separate
bug to throw an error in that case? I'll try to send a patch upstream
to fix that.