ppp files are created one level too deep

Details:

~$ systemctl status pppd-dns.service
● pppd-dns.service - Restore /etc/resolv.conf if the system crashed before the ppp link was shut down.
   Loaded: loaded (/lib/systemd/system/pppd-dns.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Tue 2015-09-22 18:07:22 UTC; 6min ago
  Process: 644 ExecStart=/etc/ppp/ip-down.d/0000usepeerdns (code=exited, status=203/EXEC)
 Main PID: 644 (code=exited, status=203/EXEC)

I did some digging and found that the ppp files are one level too deep

find /etc/ppp/ -type f
/etc/ppp/ppp/ip-down.d/000resolvconf
/etc/ppp/ppp/ip-down.d/0000usepeerdns
/etc/ppp/ppp/pap-secrets
/etc/ppp/ppp/ipv6-up
/etc/ppp/ppp/ip-up
/etc/ppp/ppp/options
/etc/ppp/ppp/ip-down
/etc/ppp/ppp/peers/provider
/etc/ppp/ppp/ipv6-down
/etc/ppp/ppp/chap-secrets
/etc/ppp/ppp/ip-up.d/000resolvconf
/etc/ppp/ppp/ip-up.d/0000usepeerdns

This looks like a issue with the writable-paths in some way, both /etc/ppp and /etc/modprobe.d are at the wrong level. /writable/system-data/etc/modprobe.d/modprobe.d contains also a wrong level. Both of these files are marked as:
"""
/etc/modprobe.d/ auto persistent transition none
/etc/ppp/ auto persistent tran
"""
Note the trailing "/". I suspect this confuses the writable-path code in the initramfs.

This is fixed for new image with the latest ubuntu-core-config upload. We also need a fixup script for existing users, I will add that next.

The workaround is now also uploaded.

I can confirm that the files are at the right level using 15.04 edge #179 on kvm.
But if I update from #178 to #179, the service still shows Active: failed and the ppp files are one level too deep.

No wait. It says failed, but the files are at the right level.

(amd64)ubuntu@localhost:~$ systemctl status pppd-dns.service
● pppd-dns.service - Restore /etc/resolv.conf if the system crashed before the ppp link was shut down.
   Loaded: loaded (/lib/systemd/system/pppd-dns.service; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Wed 2015-09-23 20:17:44 UTC; 3min 58s ago
  Process: 603 ExecStart=/etc/ppp/ip-down.d/0000usepeerdns (code=exited, status=203/EXEC)
 Main PID: 603 (code=exited, status=203/EXEC)

$ find /etc/ppp/ -type f
/etc/ppp/ip-up
find: `/etc/ppp/peers': Permission denied
/etc/ppp/chap-secrets
/etc/ppp/ip-up.d/000resolvconf
/etc/ppp/ip-up.d/0000usepeerdns
/etc/ppp/ipv6-up
/etc/ppp/pap-secrets
/etc/ppp/options
/etc/ppp/ip-down
/etc/ppp/ip-down.d/000resolvconf
/etc/ppp/ip-down.d/0000usepeerdns
/etc/ppp/ipv6-down

After a restart, the service seems to be working.

So, #178 fails -> #180 fails -> restart service works.

(amd64)ubuntu@localhost:~$ sudo journalctl -u pppd-dns.service
-- Logs begin at Wed 2015-09-23 20:17:43 UTC, end at Wed 2015-09-23 20:23:57 UTC. --
Sep 23 20:17:44 localhost.localdomain systemd[1]: Starting Restore /etc/resolv.conf if the system crashed before the ppp link was shut down....
Sep 23 20:17:44 localhost.localdomain systemd[1]: pppd-dns.service: main process exited, code=exited, status=203/EXEC
Sep 23 20:17:44 localhost.localdomain systemd[1]: Failed to start Restore /etc/resolv.conf if the system crashed before the ppp link was shut down..
Sep 23 20:17:44 localhost.localdomain systemd[1]: Unit pppd-dns.service entered failed state.
Sep 23 20:17:44 localhost.localdomain systemd[1]: pppd-dns.service failed.

(amd64)ubuntu@localhost:~$ sudo systemctl restart pppd-dns.service

(amd64)ubuntu@localhost:~$ sudo journalctl -u pppd-dns.service
-- Logs begin at Wed 2015-09-23 20:17:43 UTC, end at Wed 2015-09-23 20:25:22 UTC. --
Sep 23 20:17:44 localhost.localdomain systemd[1]: Starting Restore /etc/resolv.conf if the system crashed before the ppp link was shut down....
Sep 23 20:17:44 localhost.localdomain systemd[1]: pppd-dns.service: main process exited, code=exited, status=203/EXEC
Sep 23 20:17:44 localhost.localdomain systemd[1]: Failed to start Restore /etc/resolv.conf if the system crashed before the ppp link was shut down..
Sep 23 20:17:44 localhost.localdomain systemd[1]: Unit pppd-dns.service entered failed state.
Sep 23 20:17:44 localhost.localdomain systemd[1]: pppd-dns.service failed.
Sep 23 20:25:19 localhost.localdomain systemd[1]: Starting Restore /etc/resolv.conf if the system crashed before the ppp link was shut down....
Sep 23 20:25:19 localhost.localdomain systemd[1]: Started Restore /etc/resolv.conf if the system crashed before the ppp link was shut down..

(amd64)ubuntu@localhost:~$ systemctl status pppd-dns.service
● pppd-dns.service - Restore /etc/resolv.conf if the system crashed before the ppp link was shut down.
   Loaded: loaded (/lib/systemd/system/pppd-dns.service; enabled; vendor preset: enabled)
   Active: inactive (dead) since Wed 2015-09-23 20:25:19 UTC; 21s ago
  Process: 893 ExecStart=/etc/ppp/ip-down.d/0000usepeerdns (code=exited, status=0/SUCCESS)
 Main PID: 893 (code=exited, status=0/SUCCESS)

This sounds a lot like ppp is started before the snappy workarounds run. We need to run the fixes earlier I guess.

flashed 15.04 edge #178, the ppp service doesn't start. updated to #182, reboot, the service starts. All good now!

Thanks longsleep for the report. Thanks mvo for the fix. Beers for everybody!

ugh, there's something wrong with this in the alpha.

Flashed 15.04 alpha #11, the ppp service doesn't start. updated to alpha #12, reboot and it still doesn't start. Restart the service and it works. What could be missing in the alpha #12 that's present in edge #182?

Thanks Leo! I found the issue why the workaround was not correctly applied in the right order and fixed with with the latest image. The fix should be in 15.04-alpha r185.