apparmor profile prevents libvirtd from creating a socket
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
libvirt (Ubuntu) |
Expired
|
High
|
Unassigned |
Bug Description
I'd like to emphasize that I upgraded from Ubuntu Gnome 14.4 to 14.10
I installed systemd.
libvirtd fails to start on the stock upgrade system. Examining the log files it looks like apparmor prevents libvirtd from creating a net socket.
Okt 28 00:31:49 rivendell kernel: audit: type=1400 audit(141445270
Okt 28 00:31:49 rivendell kernel: audit: type=1400 audit(141445270
I tried to run
aa-logprof
in order to have apparmor fix the permissions but that did not work. that is why modified apparmor profiles are attached.
Putting apparmor in audit mode
aa-audit /usr/sbin/libvirtd
enables me to start libvirt
systemctl restart libvirtd
It looks like the apparmor profile permissions have to be adjusted.
ProblemType: Bug
DistroRelease: Ubuntu 14.10
Package: libvirt-bin 1.2.8-0ubuntu11
ProcVersionSign
Uname: Linux 3.16.0-23-generic x86_64
NonfreeKernelMo
ApportVersion: 2.14.7-0ubuntu8
Architecture: amd64
CurrentDesktop: GNOME
Date: Tue Oct 28 01:20:45 2014
InstallationDate: Installed on 2013-01-08 (657 days ago)
InstallationMedia: Ubuntu 12.10 "Quantal Quetzal" - Release amd64 (20121017.5)
KernLog:
SourcePackage: libvirt
UpgradeStatus: Upgraded to utopic on 2014-10-23 (4 days ago)
modified.
modified.
mtime.conffile.
Christian Kirbach (christian-kirbach-e) wrote : | #1 |
- Dependencies.txt Edit (4.9 KiB, text/plain; charset="utf-8")
- ProcEnviron.txt Edit (344 bytes, text/plain; charset="utf-8")
- RelatedPackageVersions.txt Edit (173 bytes, text/plain; charset="utf-8")
Changed in libvirt (Ubuntu): | |
status: | New → Confirmed |
Serge Hallyn (serge-hallyn) wrote : | #2 |
Changed in libvirt (Ubuntu): | |
importance: | Undecided → High |
status: | Confirmed → Incomplete |
Christian Kirbach (christian-kirbach-e) wrote : | #3 |
I did not have libvirt-bin installed before the upgrade, I installed it afterwards.
I renamed /etc/apparmor.
to my surprise the diff between /etc/apparmor.
For some reason I am no longer able to reproduce the issue. apparmor is set to enforcing
root@rivendell:
apparmor module is loaded.
24 profiles are loaded.
24 profiles are in enforce mode.
...
/usr/
thanks for your efforts
Serge Hallyn (serge-hallyn) wrote : Re: [Bug 1386465] Re: apparmor profile prevents libvirtd from creating a socket | #4 |
Thanks you for the information - that's quite frustrating. It seems quite
clear (between this bug and some others) that there is a hard-to-trigger
bug in the libvirt apparmor policy, but I've not yet spotted any obvious
trigger.
I'm marking this bug invalid meaning "cannot be reproduced", but if
anyone sees anything like it again please to mark it confirmed and
add what informatino you can.
status: invalid
Changed in libvirt (Ubuntu): | |
status: | Incomplete → Invalid |
Ben Schweikert (b-schweikert) wrote : | #5 |
Hi,
I have similar problems after upgrading my KVM host from 14.04 to 14.10. I cannot start libvirt. In the libvirt log I see these errors:
2014-11-07 08:46:15.137+0000: 4550: error : virAuditOpen:62 : Unable to initialize audit layer: Permission denied
2014-11-07 08:46:15.138+0000: 4550: error : virNetlinkEvent
Only workaround is the mentioned
aa-audit /usr/sbin/libvirtd
I try to attach a apport
Fredrik Normann (smartypants) wrote : | #6 |
I can reproduce this bug
Changed in libvirt (Ubuntu): | |
status: | Invalid → Confirmed |
Fredrik Normann (smartypants) wrote : | #7 |
[49578.653469] audit: type=1400 audit(142028482
[49578.677733] audit: type=1400 audit(142028482
[49578.833315] audit: type=1400 audit(142028482
[49578.835134] audit: type=1400 audit(142028482
[49578.838352] init: libvirt-bin main process (24713) terminated with status 6
[49578.838367] init: libvirt-bin main process ended, respawning
[49578.881341] audit: type=1400 audit(142028482
[49578.883046] audit: type=1400 audit(142028482
[49578.885421] init: libvirt-bin main process (24743) terminated with status 6
[49578.885431] init: libvirt-bin main process ended, respawning
[49578.983903] audit: type=1400 audit(142028482
[49578.985971] audit: type=1400 audit(142028482
[49578.988992] init: libvirt-bin main process (24773) terminated with status 6
[49578.989004] init: libvirt-bin main process ended, respawning
[49579.032821] audit: type=1400 audit(142028482
[49579.034446] audit: type=1400 audit(142028482
[49579.037827] init: libvirt-bin main process (24792) terminated with status 6
[49579.037843] init: libvirt-bin main process ended, respawning
[49579.084741] init: libvirt-bin main process (24812) terminated with status 6
[49579.084757] init: libvirt-bin main process ended, respawning
[49579.134834] init: libvirt-bin main process (24831) terminated with status 6
[49579.134844] init: libvirt-bin main process ended, respawning
[49579.182895] init: libvirt-bin main process (24850) terminated with status 6
[49579.182930] init: libvirt-bin main process ended, respawning
[49579.229375] init: libvirt-bin main process (24869) terminated with status 6
[49579.229388] init: libvirt-bin main process ended, respawning
[49579.275437] init: libvirt-bin main process (24888) terminated with status 6
[49579.275450] init: libvirt-bin m...
Fredrik Normann (smartypants) wrote : | #8 |
After aa-audit
[49992.022321] audit: type=1400 audit(142028523
[50004.285006] audit: type=1400 audit(142028524
[50004.285015] audit: type=1400 audit(142028524
[50004.285035] audit: type=1400 audit(142028524
[50004.285045] audit: type=1400 audit(142028524
[50004.285074] audit: type=1400 audit(142028524
[50004.285080] audit: type=1400 audit(142028524
[50004.285107] audit: type=1400 audit(142028524
[50004.285113] audit: type=1400 audit(142028524
[50004.285142] audit: type=1400 audit(142028524
[50005.905499] ip_tables: (C) 2000-2006 Netfilter Core Team
[50005.935465] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[50005.973242] IPv6: ADDRCONF(
Fredrik Normann (smartypants) wrote : | #9 |
Now I have my virbr0 interface and libvirt-bin starts
Jeff Burns (admiraljkb) wrote : | #10 |
I had the same issue upgrading from 14.04 to 14.10. Purged/reinstallled libvirt-bin as in Message3, but with no effect. Installed apparmor-utils, and put libvirt into audit mode per first message, and libvirt works again. Thanks Christian for the workaround.
error from /var/log/
2015-01-13 03:43:18.605+0000: 16428: info : libvirt version: 1.2.8, package: 1.2.8-0ubuntu11.2
2015-01-13 03:43:18.605+0000: 16428: error : virAuditOpen:62 : Unable to initialize audit layer: Permission denied
2015-01-13 03:43:18.607+0000: 16428: error : virNetlinkEvent
Serge Hallyn (serge-hallyn) wrote : | #11 |
@Jeff and @Smartypants,
are you both running systemd in 14.10, as the original bug reported was? Or are you running upstart?
datenteiler (datenteiler) wrote : | #12 |
Same problem here: I have upgraded vom 14.04 to 14.10.
Putting apparmor in audit mode
aa-audit /usr/sbin/libvirtd
enables me to start libvirt. I use Upstart:
$ ps -eaf|grep [u]pstart
root 541 1 0 18:57 ? 00:00:00 upstart-udev-bridge --daemon
root 1155 1 0 18:57 ? 00:00:00 upstart-
root 1171 1 0 18:57 ? 00:00:00 upstart-file-bridge --daemon
christi+ 2219 2209 0 18:57 ? 00:00:00 upstart --user
christi+ 2435 2219 0 18:57 ? 00:00:00 upstart-
christi+ 2469 2219 0 18:57 ? 00:00:00 upstart-file-bridge --daemon --user
christi+ 2513 2219 0 18:57 ? 00:00:00 upstart-dbus-bridge --daemon --session --user --bus-name session
christi+ 2514 2219 0 18:57 ? 00:00:00 upstart-dbus-bridge --daemon --system --user --bus-name system
$ ps -eaf|grep [s]ystemd
root 426 1 0 18:57 ? 00:00:00 /sbin/cgmanager --sigstop -m name=systemd
root 549 1 0 18:57 ? 00:00:00 /lib/systemd/
root 1214 1 0 18:57 ? 00:00:00 /lib/systemd/
$ sudo /sbin/init --version
init (upstart 1.13.2)
$ type init
init ist /sbin/init
Serge Hallyn (serge-hallyn) wrote : | #13 |
The bug description shows you have a modified /etc/apparmor.
Could you please attach your /etc/apparmor.
Changed in libvirt (Ubuntu): | |
status: | Confirmed → Incomplete |
Christian Kirbach (christian-kirbach-e) wrote : | #14 |
- usr.sbin.libvirtd Edit (2.0 KiB, text/plain)
This is my libvirtd apparmor profile, however I have no issues any more at this time.
Serge Hallyn (serge-hallyn) wrote : | #15 |
Thanks - as you have no more issues i'll mark the bug 'invalid' meaning cannot currently be reproduced. If it happens again please re-open the bug.
Changed in libvirt (Ubuntu): | |
status: | Incomplete → Invalid |
Norberto Bensa (nbensa) wrote : | #16 |
Hello. I'm having the same problem. I just upgraded to 14.10 (from 14.04).
'service libvirt-bin start' fails to start (althrough it gives a pid).
/var/log/
2015-03-10 03:22:13.546+0000: 10223: info : libvirt version: 1.2.8, package: 1.2.8-0ubuntu11.4
2015-03-10 03:22:13.546+0000: 10223: error : virAuditOpen:62 : Unable to initialize audit layer: Permission denied
2015-03-10 03:22:13.548+0000: 10223: error : virNetlinkEvent
/etc/apparmor.
# Last Modified: Mon Jul 6 17:23:58 2009
#include <tunables/global>
@{LIBVIRT}
/usr/sbin/libvirtd {
#include <abstractions/base>
#include <abstractions/dbus>
# Site-specific additions and overrides. See local/README for details.
#include <local/
capability kill,
capability net_admin,
capability net_raw,
capability setgid,
capability sys_admin,
capability sys_module,
capability sys_ptrace,
capability sys_nice,
capability sys_chroot,
capability setuid,
capability dac_override,
capability dac_read_search,
capability fowner,
capability chown,
capability setpcap,
capability mknod,
capability fsetid,
capability ipc_lock,
capability audit_write,
# Needed for vfio
capability sys_resource,
network inet stream,
network inet dgram,
network inet6 stream,
network inet6 dgram,
network packet dgram,
network netlink,
dbus bus=system,
signal,
ptrace,
unix,
# for now, use a very lenient profile since we want to first focus on
# confining the guests
/ r,
/** rwmkl,
/bin/* PUx,
/sbin/* PUx,
/usr/bin/* PUx,
/usr/sbin/* PUx,
/lib/udev/scsi_id PUx,
/usr/
/usr/
/usr/
# Required by nwfilter_
# write and run an ebtables script.
/var/
# force the use of virt-aa-helper
audit deny /sbin/apparmor_
audit deny /etc/apparmor.
audit deny /sys/kernel/
audit deny /sys/kernel/
audit deny /sys/kernel/
/sys/
/usr/
/etc/
/etc/
# allow changing to our UUID-based named profiles
change_profile -> @{LIBVIRT}
}
After 'aa-audit /usr/sbin/libvirtd' everything seems to work, but with a lot of chat in dmesg.
I can spend some time debugging this but I'll need someone to guide me.
Regards,
Norberto
Norberto Bensa (nbensa) wrote : | #17 |
After 'aa-audir -r /usr/sbin/libvirtd' and a reboot, libvirtd works as expected (and with no chat in dmesg).
Anyway, I still offer my help in debugging this.
Antony Chen (tchen) wrote : | #18 |
Having the same issue. Can't create the libvirt.sock when trying to start up libvirt.
By using "aa-audit /usr/sbin/
[67418.135152] audit: type=1400 audit(142685732
I have "network netlink" in my usr.sbin.libvirtd configuration as well.
Could it be that for netlink, you have to specify TYPE (e.g. raw)? I know for the rest, having no subsequent parameter assumes "all", but perhaps for netlink, it's changed behavior. I'll test and report back.
Serge Hallyn (serge-hallyn) wrote : | #19 |
@nbensa,
I'm sorry, Idon't know what aa-audir is.
@tchen,
have you been able to verify whether behavior changed at all (per comment #18)? If you are still having this issue in uptodate 15.04, please
set log_level=1 in /etc/libvirt/
stop libvirt-bin
rm /var/log/
run "sudo /usr/sbin/libvirtd 2>&1 | tee libvirt.debug" until it (I assume) breaks.
Attach libvirt.debug and any apparmor DENIED messages relating to libvirt from syslog.
David Peall (dkpeall) wrote : | #20 |
- Before aa-audit Edit (4.6 KiB, text/plain)
I upgraded from 14.04 to 14.10 installed libvirt and got the same error:
From syslog I have pre aa-audit and then with aa-audit it seems to aa-audit clears the bug with audit removed it continues to work.
Post audit log:
Apr 14 20:19:50 dnshost11 kernel: [ 1284.666816] audit_printk_skb: 36 callbacks suppressed
Apr 14 20:19:50 dnshost11 kernel: [ 1284.666820] audit: type=1400 audit(142903559
Apr 14 20:19:55 dnshost11 kernel: [ 1289.734099] audit: type=1400 audit(142903559
Apr 14 20:19:55 dnshost11 kernel: [ 1289.734119] audit: type=1400 audit(142903559
Apr 14 20:19:55 dnshost11 kernel: [ 1289.734156] audit: type=1400 audit(142903559
Apr 14 20:19:55 dnshost11 kernel: [ 1289.734177] audit: type=1400 audit(142903559
Apr 14 20:19:55 dnshost11 kernel: [ 1289.734241] audit: type=1400 audit(142903559
Apr 14 20:19:55 dnshost11 kernel: [ 1289.734255] audit: type=1400 audit(142903559
Apr 14 20:19:55 dnshost11 kernel: [ 1289.734308] audit: type=1400 audit(142903559
Apr 14 20:19:55 dnshost11 kernel: [ 1289.734322] audit: type=1400 audit(142903559
Apr 14 20:19:55 dnshost11 kernel: [ 1289.734380] audit: type=1400 audit(142903559
Apr 14 20:19:56 dnshost11 kernel: [ 1290.908063] Bridge firewalling registered
Apr 14 20:19:56 dnshost11 kernel: [ 1290.988004] ip_tables: (C) 2000-2006 Netfilter Core Team
Apr 14 20:19:56 dnshost11 kernel: [ 1291.129991] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
Apr 14 20:19:56 dnshost11 kernel: [ 1291.233695] I...
Changed in libvirt (Ubuntu): | |
status: | Invalid → Confirmed |
David Peall (dkpeall) wrote : | #21 |
This is very reproducible
- install 14.04
- do-release-upgrade
- restart
- apt-get install qemu-kvm libvirt-bin ubuntu-vm-builder bridge-utils
- /etc/init.
Serge Hallyn (serge-hallyn) wrote : | #22 |
I cannot reproduce this following the above recipe.
Please add the information requested in comment #19,
set log_level=1 in /etc/libvirt/
stop libvirt-bin
rm /var/log/
run "sudo /usr/sbin/libvirtd 2>&1 | tee libvirt.debug" until it (I assume) breaks.
Attach libvirt.debug and any apparmor DENIED messages relating to libvirt from syslog.
Changed in libvirt (Ubuntu): | |
status: | Confirmed → Incomplete |
Lee Revell (rlrevell-k) wrote : | #23 |
I can reproduce the bug. Attaching the requested information.
Lee Revell (rlrevell-k) wrote : | #24 |
Lee Revell (rlrevell-k) wrote : | #25 |
Lei Wang (raywang) wrote : | #26 |
this is bug is reproducible.
Install 14.04
dist-upgrade
install libvirtd-bin
I also hit this bug..
Serge Hallyn (serge-hallyn) wrote : | #27 |
Quoting Ray Wang (<email address hidden>):
> this is bug is reproducible.
>
> Install 14.04
> dist-upgrade
to 14.10?
> install libvirtd-bin
>
> I also hit this bug..
I'm still unable to reproduce. To be sure, are you running upstart
and a mostly stock Ubuntu system?
Launchpad Janitor (janitor) wrote : | #28 |
[Expired for libvirt (Ubuntu) because there has been no activity for 60 days.]
Changed in libvirt (Ubuntu): | |
status: | Incomplete → Expired |
mahmoh (mahmoh) wrote : | #29 |
Hi Serge,
I hit this problem on a stock install of 14.04, only installed MAAS and libvirt-bin + dist-upgrade. When I ran your debug commands (@22) libvirt-bin failed to crash but when I start the process it still fails, here's the only log output I see below:
/var/log/
Sep 16 11:14:55 maas kernel: [ 1310.741951] init: libvirt-bin main process (30028) terminated with status 6
Sep 16 11:14:55 maas kernel: [ 1310.741980] init: libvirt-bin main process ended, respawning
Sep 16 11:15:05 maas kernel: [ 1320.786451] init: libvirt-bin post-start process (30033) terminated with status 1
/var/log/
/usr/sbin/libvirtd: error: Unable to initialize network sockets. Check /var/log/messages or run without --daemon for more info.
Giving up waiting for /var/run/
libvirt-bin stop/post-start, (post-start) process 30244
One thing I did notice is that I had set /etc/default/
Recipe:
1) Trusty stock install (and install MAAS from stable PPA?)
2) apt-get install libvirt-bin
3) /etc/default/
4) apt-get dist-upgrade
5) sudo service libvirt-bin restart
Linux maas 3.16.0-30-generic #40~14.04.1-Ubuntu SMP Thu Jan 15 17:43:14 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
ii libvirt-bin 1.2.2-0ubuntu13
ii upstart 1.12.1-0ubuntu4.2 amd64 event-based init daemon
ii apparmor 2.8.95~
Additional:
$ sudo /usr/sbin/libvirtd -l 2>&1 | tee libvirt.debug
2015-09-16 15:30:51.587+0000: 30946: info : libvirt version: 1.2.2
2015-09-16 15:30:51.587+0000: 30946: error : virNetTLSContex
$ dmesg | grep -i armo | grep libv
[ 835.996698] audit: type=1400 audit(144241602
[ 836.134080] audit: type=1400 audit(144241602
The problem may just be the /etc/libvirt/
listen_tls = 0
listen_tcp = 1
Serge Hallyn (serge-hallyn) wrote : | #30 |
Thanks @mahmoh,
that's interesting. Perhaps we should add a comment in the shipped /etc/default/
Why had you added the -l? Is there a published recipe you were following, and should that be updated?
Changed in libvirt (Ubuntu): | |
status: | Expired → Confirmed |
aahernan (aahernan) wrote : | #31 |
Problemas con mi kvm
Serge Hallyn (serge-hallyn) wrote : | #32 |
@mahmoh - ping (question in comment #30).
Changed in libvirt (Ubuntu): | |
status: | Confirmed → Incomplete |
Launchpad Janitor (janitor) wrote : | #33 |
[Expired for libvirt (Ubuntu) because there has been no activity for 60 days.]
Changed in libvirt (Ubuntu): | |
status: | Incomplete → Expired |
Trent Lloyd (lathiat) wrote : | #34 |
I had this issue today after upgrading from trusty->wily (yes I know not technically supported). Notably I was running the lts-wily kernel on trusty, and I had a cached profile.
So I am wondering if this combination results in the cache needing regeneration but not being triggered for regeneration?
I fixed the issue with:
apparmor_parser --purge-cache
apparmor_parser -R /etc/apparmor.
apparmor_parser -r /etc/apparmor.
systemctl restart libvirt-bin
Thomas B. Rücker (thomas-ruecker) wrote : | #35 |
The bug is STILL present and breaks libvirt-bin upon upgrade from 14.04 to 16.04
Performing the steps from comment 34 worked around the breakage.
https:/
Changed in libvirt (Ubuntu): | |
status: | Expired → Confirmed |
tags: | added: xenial |
Serge Hallyn (serge-hallyn) wrote : | #36 |
Thomas,
would you mind filing a new bug using apport? In particular I'd like to see any local changes to your /etc/libvirt/
Andrea Bernabei (faenil) wrote : | #37 |
@Serge why a new bug?
Serge Hallyn (serge-hallyn) wrote : | #38 |
Because this bug report has quite a bit of information and history has taught me that not keeping information from different reporters separate can greatly complicate matters.
Since I'm asking for apport-uploaded information, it would be best that it not be mixed with information from another system.
Serge Hallyn (serge-hallyn) wrote : | #39 |
Also note that there are already several different directions into which this bug has been taken. A crucial question is whether step 3 in @mahmoh's recipe is really needed:
3) /etc/default/
If not then there are different bugs at work.
guessi (guessi) wrote : | #40 |
also hit this issue with a clean setup, but not sure how to reproduce,
since I've setup multiple server with the same "script",
but only one server hit the problem, and problem still after applying the workaround mention in #34
here's how I setup the services,
1. apt-get install qemu-kvm
2. apt-get install libvirt-bin
3. boot up VMs, and make sure it is running ( virsh list --all )
4. reboot host
5. wait for server start-up
6. login, check service libvirt-bin running state => not running, and VMs not start, of course
7. try to apply the workaround mention in #34, trick of apparmor_parser -r / -R
8. reboot again
9. login, check service libvirt-bin running state => running (at the first time)
10. reboot again
11. login, check service libvirt-bin running state => not running (seems like workaround not always works?)
look into the `syslog`, each time libvirt-bin unsuccessful start-up at boot, it will have the following log in syslog,
==> Apr 16 21:02:09 host2 kernel: [ 313.059830] init: libvirt-bin post-start process (2430) terminated with status 1
and here's my system information,
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.4 LTS
Release: 14.04
Codename: trusty
$ uname -a
Linux host2 4.2.0-35-generic #40~14.04.1-Ubuntu SMP Fri Mar 18 16:37:35 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
hope these information could help,
reference:
- https:/
guessi (guessi) wrote : | #41 |
Hi,
I've opened another bug/issue report, including patch, for the issue of "libvirt-bin not start",
please see #1571209 for detail,
https:/
it seems to the problem I've ran into, I'm wondering if it could fix your problem,
please give help to test it, thanks !!!
Serge Hallyn (serge-hallyn) wrote : | #42 |
@lathiat,
when I take a 14.04 system,
grep netlink /etc/apparmor.
returns nothing; then do-release-upgrade -d, agree to the reboot, and
grep netlink /etc/apparmor.
returns
network netlink,
Is it possible that you did not reboot after the release upgrade?
Changed in libvirt (Ubuntu): | |
status: | Confirmed → Incomplete |
Launchpad Janitor (janitor) wrote : | #43 |
[Expired for libvirt (Ubuntu) because there has been no activity for 60 days.]
Changed in libvirt (Ubuntu): | |
status: | Incomplete → Expired |
Thanks for reporting this bug. The default libvirt profile does provide 'network netlink', so I'm not sure why you are having this problem.
Could you please attach the /etc/apparmor. d/usr.sbin. libvirtd from a fresly updated host that is having this issue?