After some debugging with scott, it turns out that starting the cloud-config-puppet job on stopped cloud-apt-update-upgrade-conf fixes the issue (rather than started).
After some debugging with scott, it turns out that starting the cloud-config-puppet job on stopped cloud-apt- update- upgrade- conf fixes the issue (rather than started).