Why do I get a permissions error when starting a KVM VM with a TAP interface?

1st Post, thanks for the community. Apologies if it is better suited in another forum

I am running Ubuntu 12.04 LTS Server as my host.

/etc/libvirt# kvm --version

QEMU emulator version 1.0 (qemu-kvm-1.0), Copyright (c) 2003-2008 Fabrice Bellard

I built out a VM running Ubuntu 13.10 just fine. When I went into the XML to configure an ethernet tap0 interface for a static IP I have run into troubles. I have scoured the net and tried several things, to no avail.

Here is the error I get.
Beneath the error is my config and my log.
Beneath that are the list of things I have tried and not had success with...

-# virsh start phabVM error: Failed to start domain phabVM error: internal error Process exited while reading console log output: char device redirected to /dev/pts/1 kvm: -netdev tap,ifname=tap0,id=hostnet0: could not open /dev/net/tun: Operation not permitted kvm: -netdev tap,ifname=tap0,id=hostnet0: Device 'tap' could not be initialized

VM XML

    # domain type='kvm'>
    # name>phabVM
    # uuid>26a54ab3-7c67-507b-f066-09015dd19942
    # memory>4194304
    # currentMemory>4194304
    # vcpu>1
    # os>
    # type arch='x86_64' machine='pc-1.0'>hvm
    # boot dev='hd'/>
    # /os>
    # features>
    # acpi/>
    # apic/>
    # pae/>
    # /features>
    # clock offset='utc'/>
    # on_poweroff>preserve
    # on_reboot>restart
    # on_crash>restart
    # devices>
    # emulator>/usr/bin/kvm
    # disk type='file' device='disk'>
    # driver name='qemu' type='raw'/>
    # source file='/home/inevirt/images/PhabVM.img'/>
    # target dev='sda' bus='sata'/>
    # address type='drive' controller='0' bus='0' unit='0'/>
    # /disk>
    # disk type='block' device='cdrom'>
    # driver name='qemu' type='raw'/>
    # target dev='hdc' bus='ide'/>
    # readonly/>
    # address type='drive' controller='0' bus='1' unit='0'/>
    # /disk>
    # controller type='ide' index='0'>
    # address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x1'/>
    # /controller>
    # controller type='sata' index='0'>
    # address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    # /controller>
    # interface type='ethernet'>
    # mac address='52:54:00:6d:ab:17'/>
    # target dev='tap0'/>
    # model type='virtio'/>
    # address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    # /interface>
    # serial type='pty'>
    # target port='0'/>
    # /serial>
    # console type='pty'>
    # target type='serial' port='0'/>
    # /console>
    # input type='mouse' bus='ps2'/>
    # graphics type='vnc' port='-1' autoport='yes'/>
    # sound model='ich6'>
    # address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    # /sound>
    # video>
    # model type='cirrus' vram='9216' heads='1'/>
    # address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    # /video>
    # memballoon model='virtio'>
    # address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    # /memballoon>
    # /devices>
    #/domain

LOG

2013-11-21 17:38:59.075+0000: starting up
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin        QEMU_AUDIO_DRV=none /usr/bin/kvm -S -M pc-1.0 -enable-kvm -m 4096 -smp 1,sockets=1,cores=1,threads=1 -name PhabVM -uuid 26a54ab3-7c67-507b-f066-09015dd19942 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/PhabVM.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device ahci,id=ahci0,bus=pci.0,addr=0x5 -drive file=/home/inevirt/images/PhabVM.img,if=none,id=drive-sata0-0-0,format=raw -device ide-drive,bus=ahci0.0,drive=drive-sata0-0-0,id=sata0-0-0,bootindex=1 -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,ifname=tap0,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:6d:ab:17,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -vnc 127.0.0.1:0 -vga cirrus -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
char device redirected to /dev/pts/1
kvm: -netdev tap,ifname=tap0,id=hostnet0: could not open /dev/net/tun: Operation not permitted
kvm: -netdev tap,ifname=tap0,id=hostnet0: Device 'tap' could not be initialized
2013-11-21 17:38:59.205+0000: shutting down
2013-11-21 17:40:57.554+0000: starting up
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin QEMU_AUDIO_DRV=none /usr/bin/kvm -S -M pc-1.0 -enable-kvm -m 4096 -smp 1,sockets=1,cores=1,threads=1 -name PhabVM -uuid 26a54ab3-7c67-507b-f066-09015dd19942 -nodefconfig -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/PhabVM.monitor,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc -no-shutdown -device ahci,id=ahci0,bus=pci.0,addr=0x5 -drive file=/home/inevirt/images/PhabVM.img,if=none,id=drive-sata0-0-0,format=raw -device ide-drive,bus=ahci0.0,drive=drive-sata0-0-0,id=sata0-0-0,bootindex=1 -drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw -device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -netdev tap,ifname=tap0,id=hostnet0 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:6d:ab:17,bus=pci.0,addr=0x3 -chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 -usb -vnc 127.0.0.1:0 -vga cirrus -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6
char device redirected to /dev/pts/1
kvm: -netdev tap,ifname=tap0,id=hostnet0: could not open /dev/net/tun: Operation not permitted
kvm: -netdev tap,ifname=tap0,id=hostnet0: Device 'tap' could not be initialized
2013-11-21 17:40:57.686+0000: shutting down

Things I have tried (from internet research) that have not worked.

Added user to group kvm
Config'ed libvirtd.conf and uncommented listen_tls and listen_tcp
Config'ed qemu.conf by setting root as user/group to run VMs & uncommented
clear_emulator_capabilities = 0
Disabled apparmor
Ensured kvm binary was set to kvm group

Libvirt + Android emulator

Is it possible to manage the Android Emulator with libvirt? And if possible, how?

Thanks, Adam

Windows time drifting with Qemu/KVM

I have a Windows VM running with kvm/qemu which is experiencing time drift. What would be the optimal libvirt settings for a Windows guest?

Currently I am using this, but it did not help:

<clock offset='localtime'>
  <timer name='rtc' tickpolicy='catchup' track='guest'>
    <catchup threshold='123' slew='120' limit='10000'/>
  </timer>
</clock>

How to set boot order on KVM (libvirt/virsh)?

How would you set the boot order on KVM when using libvirt/virsh? (Either via config or command)

KVM installation in Ubuntu

I have tried installing KVM according to the guidelines provided on Ubuntu wiki but when I open Virtual Machine Manager it says that,

'Warning: KVM is not available. This may mean the KVM package is not installed, or the KVM kernel modules are not loaded. Your virtual machines may perform poorly.'

I have hardware virtulization enabled in my BIOS settings. Here, is a output of sudo /usr/sbin/kvm-ok:

INFO: /dev/kvm does not exist HINT: sudo modprobe kvm_intel modprobe: FATAL: Module msr not found.

Also, output of sudo modprobe kvm_intel is as follows:

modprobe: FATAL: Module kvm_intel not found.

KVM and libvirt: wrong CPU type in virtual host

We us KVM and libvirt on a 6 core (12 HT cores) machine for virtualization.

Problem: wrong CPU type in virtual host.

used KVM, libvirt, kernel version:

libvirt version: 0.9.8
QEMU emulator version 1.0 (qemu-kvm-1.0), Copyright (c) 2003-2008 Fabrice Bellard
Ubuntu 12.04.1 LTS
kernel: 3.2.0-32-generic x86_64 

/usr/share/libvirt/cpu_map.xml does not support more recent cpu types than Westmare.

Do I need this kind of virtualisation of the cpu at all? because of some reasons we need maximum cpu-performance in the virtual host. Will be glad to have some cores of the server's i7-3930K CPU@3.20GHz available in my virtual machines.

Maybe we do too muczh virtualization...?

my virtual host's xml looks like: where can I set the cpu -host flag?

<domain type='kvm'>
  <name>myVirtualServer</name>
  <uuid>2344481d-f455-455e-9558</uuid>
  <description>Test-Server</description>
  <memory>4194304</memory>
  <currentMemory>4194304</currentMemory>
  <vcpu>2</vcpu>
  <cpu match='exact'>
    <model>Westmere</model>
    <vendor>Intel</vendor>
  </cpu>
  <os>
    <type arch='x86_64' machine='pc-1.0'>hvm</type>
    <boot dev='hd'/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <pae/>
  </features>

$ lscpu of physical Server with 6 (12) cores with HT

Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                12
On-line CPU(s) list:   0-11
Thread(s) per core:    2
Core(s) per socket:    6
Socket(s):             1
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 45
Stepping:              7
CPU MHz:               1200.000
BogoMIPS:              6400.05
Virtualization:        VT-x
L1d cache:             32K
L1i cache:             32K
L2 cache:              256K
L3 cache:              12288K
NUMA node0 CPU(s):     0-11

$ lscpu of virtual Server (wrong CPU type, wrong L2-Cache, wrong MHz)

Architecture:          x86_64
CPU op-mode(s):        32-bit, 64-bit
Byte Order:            Little Endian
CPU(s):                2
On-line CPU(s) list:   0,1
Thread(s) per core:    1
Core(s) per socket:    1
Socket(s):             2
NUMA node(s):          1
Vendor ID:             GenuineIntel
CPU family:            6
Model:                 15
Stepping:              11
CPU MHz:               3200.012
BogoMIPS:              6400.02
Virtualisation:        VT-x
Hypervisor vendor:     KVM
Virtualisation type:   full
L1d cache:             32K
L1i cache:             32K
L2 cache:              4096K
NUMA node0 CPU(s):     0,1

Converting QEMU command line args to libvirt domain xml

I have a hard time converting this QEMU command to a libvirt domain XML:

qemu-system-x86_64 -enable-kvm \
-m 3072 \
-cpu core2duo \
-machine q35 \
-smp 2 \
-usbdevice keyboard \
-usbdevice mouse \
-vga std \
-device isa-applesmc,osk="${osx_osk_key}" \
-kernel "${chameleon_file_path}" \
-device ide-drive,bus=ide.2,drive=MacHDD \
-drive id=MacHDD,if=none,cache=none,file="${vm_hdd_img_file_path}"
-netdev user,id=hub0port0 \
-device virtio-net,netdev=hub0port0,id=eth0

It works perfectly if I call it from a Terminal (Ubuntu 14.10) but the virsh domxml-from-native qemu-argv fails for the MacHDD -device with this error:

error: internal error: missing index/unit/bus parameter in drive 'id=MacHDD,if=none,cache=none,file="${vm_hdd_img_file_path}"'

(Note: of course I tried it with actual paths in place of the variables, same error - and if I remove the last two args it converts it to domain XML even with the variables included)

How could I solve this? Or what would be the equivalent of the last two args in domain xml? (Tried a lot of different combinations but couldn't get it work with libvirt/virsh)

EDIT

For the record this is what I get running virsh's domain-from-native command without the -device -drive

<domain type='kvm' xmlns:qemu='http://libvirt.org/schemas/domain/qemu/1.0'>
  <name>unnamed</name>
  <uuid>38cafecb-4774-4590-83eb-e576a79aab93</uuid>
  <memory unit='KiB'>3145728</memory>
  <currentMemory unit='KiB'>3145728</currentMemory>
  <vcpu placement='static'>2</vcpu>
  <os>
    <type arch='x86_64' machine='q35'>hvm</type>
    <kernel>${chameleon_file_path}</kernel>
  </os>
  <features>
    <acpi/>
  </features>
  <cpu mode='custom' match='exact'>
    <model fallback='allow'>core2duo</model>
  </cpu>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <controller type='sata' index='0'/>
    <controller type='pci' index='0' model='pcie-root'/>
    <controller type='pci' index='1' model='dmi-to-pci-bridge'/>
    <controller type='pci' index='2' model='pci-bridge'/>
    <input type='keyboard' bus='usb'/>
    <input type='mouse' bus='usb'/>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='sdl'/>
    <video>
      <model type='vga' vram='9216' heads='1'/>
    </video>
    <memballoon model='none'/>
  </devices>
  <qemu:commandline>
    <qemu:arg value='-device'/>
    <qemu:arg value='isa-applesmc,osk=&quot;${osx_osk_key}&quot;'/>
    <qemu:arg value='-netdev'/>
    <qemu:arg value='user,id=hub0port0'/>
    <qemu:arg value='-device'/>
    <qemu:arg value='virtio-net,netdev=hub0port0,id=eth0'/>
  </qemu:commandline>
</domain>

But I can't figure out what would be the equivalent of the -device -drive in this situation and I'm unable to create a VM from this domain xml.

Even without the MacHDD -device and -drive I get this error when I try to create the VM with $ virsh create

error: Failed to create domain from libvirt.xml
error: internal error: cannot load AppArmor profile 'libvirt-38cafecb-4774-4590-83eb-e576a79aab93'

Sending packet through linux macvtap interface

Linux supports an interface type, macvtap. I can't find great documentation on it, but this is a good starter: http://virt.kernelnewbies.org/MacVTap

I'm trying to use this to create a new virtual net device that I can programmatically read and write to for sending raw packets. While I can read from the tap interface fine, I can't figure out any way to write to it.

1. I'm opening the tap device with a regular open syscall.
2. Tried using write to send a packet, but I get an Invalid argument (EINVAL) error response.
3. Also tried to use sendmsg but of course get an error that I have a plain file descriptor, not a socket.

Can this be done? Or do macvtap interfaces only support reading from the tap interface and require you use raw sockets or something else for sending packets?

(Libvirt code for macvtap is here: https://gitorious.org/libvirt/libvirt/source/63889e0c775010d8d70b71d25340bab995aa83ce:src/util/virmacaddr.c)

Blockquote

Ansible: virt module XML error

I'm trying to define a VM using the virt module and a XML file. This task:

 - name: Define VM using the XML file
   virt: command=define
         name={{ new_vm_name }}
         xml={{ vm_images_path }}/{{ new_vm_xml }}
   remote_user: root
   delegate_to: "{{ hyperv }}"

fails with this error:

failed: [vm-002 -> nat] => {"failed": true}
msg: (domain_definition):1: Start tag expected, '<' not found
/var/lib/libvirt/images//vm-002-ver1.1.0-00042.xml

while this works:

 - name: Define VM using the XML file
   command: virsh define {{ vm_images_path }}/{{ new_vm_xml }}
   remote_user: root
   delegate_to: "{{ hyperv }}"

XML file is the same in both cases, any idea of why the first one fails? I do always prefer using modules than custom commands so I'd really like to make the first one work.

Create Virtual Machine using libvirt error related to AppArmor

I am trying to create a virtual machine using libvirt using the command:

virsh create file

Contents of "file":

<domain type='qemu' id='3'>
        <name>testvm</name>
        <memory>100</memory>
        <vcpu>1</vcpu>
        <os>
                <type arch='i686'>hvm</type>
        </os>
        <devices>
                <disk type='file' device='disk'>
                        <source file='/libtmp/VM-linux.0.2.img'/>
                        <target dev='hdc'/>
                </disk>
        </devices>
        <on_reboot>restart</on_reboot>
        <on_poweroff>preserve</on_poweroff>
        <on_crash>restart</on_crash>
</domain>

Here is error which occur.

error: Failed to create domain from file
error: internal error cannot load AppArmor profile 'libvirt-9cb01efc-ed3b-ff8e-4de5-7227d311dd15'

I am able to create the vm without loading the image file.

Everytime the profile name keeps on changing. I tried stopping it and creating the vm but I got the same error.

Any pointers will be very helpful.

apparmor causes issues on libvirt with custom qemu

I wanted to run libvirt using a customized version of qemu. However, after I installed my version of qemu and rebooted I get the following message in dmesg

type=1400 audit(1338385059.381:51): apparmor="DENIED" operation="exec" parent=1700 profile="/usr/sbin/libvirtd" name="/usr/local/bin/qemu-system-x86_64" pid=1746 comm="libvirtd" requested_mask="x" denied_mask="x" fsuid=0 ouid=0

I tried editing the /etc/apparmor.d/abstractions/libvirt-qemu and added the following line and rebooted

/usr/local/bin/qemu-system-x86_64 rmix,

However the issue is still present. I am really new to apparmor, in fact hadn't read much on it till I found this error. Any help would be greatly appreciated.

Programatically run a QEMU VM instance from snapshot

I have a QEMU image with a snapshot stored in it. Right now I'm using libvirt to start it.

However, I want to be able to run more than one instance out of the same image/snapshot.

I guess I can do that by cloning the virtual-hd and installing/creating a new domain (virsh) and then running revert from snapshot. But I want to be able to do that pretty much "on-the-fly" with as little as possible latency from the time I decide I need to run another instance of image X to the time that instance is running from the stored snapshot.

Anyone did anything like that? I started thinking maybe libvirt is not low-level enough for this ?

[EDIT: Sorry if this wasn't clear - I'm talking about a RAM+HD snapshot, not just HD snapshot, which I already know how to create...]

Thanks

File download speed from Amazon S3 drops to <2 kb/s = very slow download (on the machine it has been uploaded from)

Basic Problem:

We've been experiencing very strange behaviour in our current infrastructure setup:

• file download speed from Amazon S3 drops to <2 kb/s (after ~10 downloads that have perfectly normal download speed) if the file is downloaded from the same IP/machine it has been uploaded from
• on our other machines we can download the file a couple of thousand times and don't see this behaviour

Additional details:

• the machines are setup identical using puppet
• they are all virtual machines running ubuntu 12.04.4 on KVM with libvirtd on ubuntu 12.04.4 and 13.04 hosts
• each VM has it's own public IP from which the traffic is originating
• after a couple of minutes-hours it's possible to download the file again with >5 mb/s for a couple of times (seems to be 10 times)
• files are uploaded from rails applications using the fog gem

Tests with wget:

Using wget, you see this output on the affected machines for a file we uploaded:

--2014-07-31 16:33:38--  http://s3-eu-west-1.amazonaws.com/not_the_real_file_url
Resolving s3-eu-west-1.amazonaws.com (s3-eu-west-1.amazonaws.com)... 178.236.6.160
Connecting to s3-eu-west-1.amazonaws.com (s3-eu-west-1.amazonaws.com)|178.236.6.160|:80...      connected.
HTTP request sent, awaiting response... 200 OK
Length: 2801149 (2.7M) [text/plain]
Saving to: `/dev/null'

0% [                                ] 10,111      1.05K/s  eta 68m 26s

and it stays like this for 68m! (it does finish the download after that time though)

And this output for a random file hosted on amazon s3 by somebody else:

--2014-07-31 16:39:21--  https://s3.amazonaws.com/Minecraft.Download/versions/14w31a/minecraft_server.14w31a.jar
Resolving s3.amazonaws.com (s3.amazonaws.com)... 72.21.211.199
Connecting to s3.amazonaws.com (s3.amazonaws.com)|72.21.211.199|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 10342238 (9.9M) [application/octet-stream]
Saving to: `/dev/null'

32% [====================================>    ] 3,370,945    747K/s  eta 12s

Our current workaround

Our current solution, is to use our HAProxy as a transparent HTTP proxy.

Meaning we have a frontend "cloud.example.com" defined, and a backend that first replaces the requests HOST with "s3-eu-west-1.amazonaws.com" and then uses s3-eu-west-1.amazonaws.com:80 as a server. For amazon it then looks like the request is coming from our proxy and we can download the files we stored on S3 thousands of times again. :)

[2014-07-31 16:56:57 +0200] RUN[28] AVG: '0.9612743812142854' s, LAST_RUN: '0.711118431' s
--2014-07-31 16:56:57--  https://cloud.example.com/not_the_real_file_url
Resolving cloud.example.com (cloud.example.com)... 1.2.3.4
Connecting to cloud.example.com (cloud.example.com)|1.2.3.4|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 2801149 (2.7M) [text/plain]
Saving to: `/dev/null'

100%[====================>] 2,801,149   2.47M/s   in 1.1s

How to create tap devices using virsh?

This is a snippet from configuration file created using qemu-kvm management tool. How do I create the same config using VMM or virsh?

[net]
  type = "nic"

[net]
  type = "tap"
  script = "/etc/ovs-ifup"
  downscript = "/etc/ovs-ifdown"

Does libvirt support clone qcow2 with copy on write

The format qcow2 support copy-on-write.detail about qcow2 is here. Does libvirt suport clone qcow2 VM Image with copy-on-write. I find none options about:

virt-clone