add a motd script for news

This bug was fixed in the package base-files - 9.6ubuntu7

---------------
base-files (9.6ubuntu7) zesty; urgency=medium

  * etc/default/motd-news, update-motd.d/50-news: LP: #1637800
    - add an update-motd script, that dynamically retrieves a
      message-of-the-day from a news service, defaults to
      https://motd.ubuntu.com
    - fail quietly, gracefully, and quickly

 -- Dustin Kirkland <email address hidden> Fri, 11 Nov 2016 09:54:28 -0600

"- fail quietly, gracefully, and quickly"

I'm not sure how a 2-second delay on login for people who firewall off outside access can be qualified as "quickly".

I'm also skeptical in general here about this being a thing that should be happening by default.

Please use .timer systemd unit to asynchronously fetch these news on boot, and periodically thereafter (there is support for both stanzas), do so safely from https, have templated news, and cache them, at the end include the cached news into motd.

@xnox:

Totally agreed on https. I just uploaded that fix to zesty, defaulting to https://motd.ubuntu.com (which exists now, it didn't at the time), and enforcing that all URLs should start with https://.

The .timer systemd unit + caching + periodic refresh is interesting, indeed. I'll look into a rewrite of update-motd to do it this way. What minimum version of systemd would I need for it to work this way?

Okay, reworked as a systemd timer. Tested here in a zesty LXD container, working well.

Why is this in base-files? You might seriously want to reconsider adding a phone-home script to a package that can't be removed considering what happened back a couple years with the amazon lens. Especially with a user agent that unique.

```
read up idle < /proc/uptime
uptime="uptime/$up/$idle"
USER_AGENT="curl/$curl_ver $lsb $platform $cpu $uptime"
```

Uh, okay, I can understand the curl version, the platform and the cputype; but the uptime of my nodes is nobody's business but my own.

The platform and CPU type are exposed by normal browser user agent strings anyway, I'm not really concerned if someone knows I'm amd64 or arm64; there will be other ways to discover that.... but my uptime?! really? REALLY?

And I wouldn't have minded as much if there was a comment describing why uptime was even being included ("This data is used to improve the graphs publically available at https://errors.ubuntu.com/") and some kind of documentation somewhere that this was occurring.

jeez, and i nearly missed
```
cpu="$(grep -m1 "^model name" /proc/cpuinfo | sed -e "s/.*: //" -e "s:\s\+:/:g")"
```

So not just uptime, but you get
"Intel(R) Xeon(R) CPU E31230 @ 3.20GHz"
from me too?

As with any user agent string sent by any web browser, the intention is such that the server can customize the response appropriately for user.

e.g. There may be a bug that affects ARM64 users, but not any other architecture. There might be a kernel vulnerability that is only exploitable machines that have been running a long time, or perhaps machines which are definitely not vulnerable to a regression, if they've been running for longer than the regression. In this way, the news server can generate the most relevant message-of-the-day for a given system.

Or, hopefully, everything is hunky dory, and then we can wish Grace Hopper or Alan Turing a happy birthday (a la Google Doodle) in your Ubuntu system's MOTD :-)

Moreover, it's always easy to disable or customize the urls to the local user or administrator's liking, or firewall it off entirely.

The goal is useful, helpful, tailored information in the MOTD. To actually make the "message of the day", a dynamically and informative "message of the day".

Cheers!
Dustin

"the news server can generate the most relevant message-of-the-day"

So, you promise it's not just going to return the output of GNU fortune over and over?

And that the serverside will be hardened against exploitation? PS, do me a favor and make sure it's written in a dynamic scripting language so at least I know y'all have to waste CPU time on every request...

Anyway -- I've got to go make sure my containers run this on every login, and update all of my live USB sticks so all six automatic logins on the VTs in the background ends up hitting your motd server.

Y'all don't mind if I patch it to send random uptimes between 4 seconds and 4700 years, and claiming to be a ["Tensilica LX6", "Zilog EZ-80", "Microchip Technology Inc. PIC16F874‑04/P", "EpsonS1C60A16"], right?

</sarcasm>

But seriously -- some comments in the script (actually, your response prefixed with #s would pretty much be spot on) would be appreciated; even if another couple hundred bytes on disk are wasted.

> So, you promise it's not just going to return the output of GNU fortune over and over?

https://bugs.launchpad.net/ubuntu/+source/base-files/+bug/1701068 is confirmed proof what this ticket is for is now being used inappropriately (unprofessionally).

@Jeremy: I think it would be more appropriate to say that the current motd.ubuntu.com shows an item about an episode of a TV show, which depicts Ubuntu being used in some IT task.

Hello Dustin, or anyone else affected,

Accepted base-files into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/base-files/9.4ubuntu4.9 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Note: during verification, it would be good to also run the test case from LP: #1829968.

All autopkgtests for the newly accepted base-files (9.4ubuntu4.9) for xenial have finished running.
There have been regressions in tests triggered by the package. Please visit the sru report page and investigate the failures.

https://people.canonical.com/~ubuntu-archive/pending-sru.html#xenial

I installed the version of base-files from Ubuntu 18.04's proposed repository and after installation of the package not timer was running for motd-news. I rebooted to see if that would active the timer and it did not.

bdmurray@clean-xenial-amd64:~$ ls -lh /var/cache/motd-news
-rw-r--r-- 1 root root 0 Jul 18 12:32 /var/cache/motd-news
bdmurray@clean-xenial-amd64:~$ systemctl list-timers --all
NEXT LEFT LAST PASSED UNIT ACTIVATES
Thu 2019-07-18 21:33:41 PDT 8h left Thu 2019-07-18 12:20:38 PDT 1h 8min ago apt-daily.timer apt-daily.service
Fri 2019-07-19 06:01:14 PDT 16h left Thu 2019-07-18 12:20:38 PDT 1h 8min ago apt-daily-upgrade.timer apt-daily-upgrade.service
Fri 2019-07-19 12:47:35 PDT 23h left Thu 2019-07-18 12:47:35 PDT 41min ago systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.service
n/a n/a n/a n/a snapd.refresh.timer
n/a n/a n/a n/a snapd.snap-repair.timer snapd.snap-repair.service
n/a n/a n/a n/a ureadahead-stop.timer ureadahead-stop.service

6 timers listed.
bdmurray@clean-xenial-amd64:~$ uptime
 13:29:32 up 57 min, 1 user, load average: 0.00, 0.00, 0.00
bdmurray@clean-xenial-amd64:~$ apt-cache policy base-files
base-files:
  Installed: 9.4ubuntu4.10
  Candidate: 9.4ubuntu4.10
  Version table:
 *** 9.4ubuntu4.10 100
        100 /var/lib/dpkg/status
     9.4ubuntu4.8 500
        500 https://ubuntu.osuosl.org/ubuntu xenial-updates/main amd64 Packages
     9.4ubuntu4 500
        500 https://ubuntu.osuosl.org/ubuntu xenial/main amd64 Packages

I also noticed this when installing the package:

Setting up base-files (9.4ubuntu4.10) ...
motd-news.service is a disabled or a static unit, not starting it.
motd-news.timer is a disabled or a static unit, not starting it.

+ [ -d /run/systemd/system ]
+ systemctl --system daemon-reload
+ deb-systemd-invoke start motd-news.service
motd-news.service is a disabled or a static unit, not starting it.
+ [ -d /run/systemd/system ]
+ systemctl --system daemon-reload
+ deb-systemd-invoke start motd-news.timer
motd-news.timer is a disabled or a static unit, not starting it.

The xenial package is not using debhelper, and the manual entries in postinst do not enable it, just try to start it.

This is the bit that dh_systemd_enable adds to postinst in bionic, for example:
# Automatically added by dh_systemd_enable/11.1.6ubuntu2
if [ "$1" = "configure" ] || [ "$1" = "abort-upgrade" ] || [ "$1" = "abort-deconfigure" ] || [ "$1" = "abort-remove" ] ; then
        # This will only remove masks created by d-s-h on package removal.
        deb-systemd-helper unmask 'motd-news.timer' >/dev/null || true

        # was-enabled defaults to true, so new installations run enable.
        if deb-systemd-helper --quiet was-enabled 'motd-news.timer'; then
                # Enables the unit on first installation, creates new
                # symlinks on upgrades if the unit file has changed.
                deb-systemd-helper enable 'motd-news.timer' >/dev/null || true
        else
                # Update the statefile to add new symlinks (if any), which need to be
                # cleaned up on purge. Also remove old symlinks.
                deb-systemd-helper update-state 'motd-news.timer' >/dev/null || true
        fi
fi

And then come the deb-systemd-invoke start bits that were added manually to the xenial package.

I'm addressing this in a new upload.

Hello Dustin, or anyone else affected,

Accepted base-files into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/base-files/9.4ubuntu4.11 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Xenial verification

No warnings or errors during upgrade:
Get:1 http://azure.archive.ubuntu.com/ubuntu xenial-proposed/main amd64 base-files amd64 9.4ubuntu4.11 [67.5 kB]
Fetched 67.5 kB in 0s (1,082 kB/s)
(Reading database ... 78928 files and directories currently installed.)
Preparing to unpack .../base-files_9.4ubuntu4.11_amd64.deb ...
Unpacking base-files (9.4ubuntu4.11) over (9.4ubuntu4.8) ...
Processing triggers for plymouth-theme-ubuntu-text (0.9.2-3ubuntu13.5) ...
update-initramfs: deferring update (trigger activated)
Processing triggers for man-db (2.7.5-1) ...
Processing triggers for install-info (6.1.0.dfsg.1-5) ...
Processing triggers for initramfs-tools (0.122ubuntu8.15) ...
update-initramfs: Generating /boot/initrd.img-4.15.0-1060-azure
W: mdadm: /etc/mdadm/mdadm.conf defines no arrays.
Setting up base-files (9.4ubuntu4.11) ...

motd message is now displayed:
ubuntu@xenial-motd:~$ sudo sh /etc/update-motd.d/50-motd-news --force

 * Kata Containers are now fully integrated in Charmed Kubernetes 1.16!
   Yes, charms take the Krazy out of K8s Kata Kluster Konstruction.

     https://ubuntu.com/kubernetes/docs/release-notes

Confirmed upon logging in:
andreas@nsnx:~$ ssh ubuntu@13.90.92.78
Welcome to Ubuntu 16.04.6 LTS (GNU/Linux 4.15.0-1057-azure x86_64)

 * Documentation: https://help.ubuntu.com
 * Management: https://landscape.canonical.com
 * Support: https://ubuntu.com/advantage

 * Kata Containers are now fully integrated in Charmed Kubernetes 1.16!
   Yes, charms take the Krazy out of K8s Kata Kluster Konstruction.

     https://ubuntu.com/kubernetes/docs/release-notes

26 packages can be updated.
0 updates are security updates.

New release '18.04.2 LTS' available.
Run 'do-release-upgrade' to upgrade to it.

*** System restart required ***
Last login: Fri Oct 4 18:15:19 2019 from 187.107.24.4

This bug was fixed in the package base-files - 9.4ubuntu4.11

---------------
base-files (9.4ubuntu4.11) xenial; urgency=medium

  * update-motd.d/50-motd-news: add cloud_id to user-agent string
    (LP: #1840946)
  * d/postinst.in, d/postrm: add missing dh_systemd_enable pieces. This
    completes the fix for LP #1637800
  * d/postinst.in: do not start the motd-news.service, as it won't be
    in an enabled state anyway, and the timer will run it when needed. This
    addresses the motd-news.service bit in comment #19 of #1637800.

base-files (9.4ubuntu4.10) xenial; urgency=medium

  * Fix syntax error in new base-files postinst code. LP: #1836236.

base-files (9.4ubuntu4.9) xenial; urgency=medium

  * Backport /etc/update-motd.d/50-motd-news, along with the systemd units
    that are used to keep the local cache file up-to-date. LP: #1637800

 -- Andreas Hasenack <email address hidden> Mon, 30 Sep 2019 10:31:01 -0300

The verification of the Stable Release Update for base-files has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

motd-news vs privacy / security / telemetry

https://bugs.launchpad.net/ubuntu/+source/base-files/+bug/1867424