EC2 credentials are cached on disk

Attaching the suggested fix per Andrew. I fixed flake8 complaints in the test, but that is all.

Andrew,
Please let me know if you want some path for this other than me making this a public bug and fixing.

Hi Scott,
Anthony Liguori will coordinate this with you.

Anthony, Andrew.
I'd fix this now, and we could start getting it back into 16.04 and the like, I'm waiting on you though.

I'm not sure why (or if?) folks on Amazon's side have dropped the ball here, but please go ahead and publish this fix at your convenience, if you haven't yet.

Has a CVE been assigned to this issue? Or does it fall into the category of "security hardening" and thus not qualify for a CVE?

Thanks

No CVE has been assigned, and in fact it seems only Amazon Linux was vulnerable, because Ubuntu and others were using an API version for instance metadata that did not include IAM instance credentials.

This bug was fixed in the package cloud-init - 0.7.9-19-ge987092-0ubuntu1

---------------
cloud-init (0.7.9-19-ge987092-0ubuntu1) zesty; urgency=medium

  * New upstream snapshot.
    - manual_cache_clean: When manually cleaning touch a file in instance dir.
    - Add tools/ds-identify to identify datasources available.
    - Fix small typo and change iso-filename for consistency [Robin Naundorf]
    - Fix eni rendering of multiple IPs per interface
      [Ryan Harper] (LP: #1657940)
    - tools/mock-meta: support python2 or python3 and ipv6 in both.
    - tests: remove executable bit on test_net, so it runs, and fix it.
    - tests: No longer monkey patch httpretty for python 3.4.2
    - Add 3 ecdsa-sha2-nistp* ssh key types now that they are standardized
      [Lars Kellogg-Stedman] (LP: #1658174)
    - reset httppretty for each test [Lars Kellogg-Stedman] (LP: #1658200)
    - build: fix running Make on a branch with tags other than master
    - EC2: Do not cache security credentials on disk
      [Andrew Jorgensen] (LP: #1638312)
    - doc: Fix typos and clarify some aspects of the part-handler
      [Erik M. Bray]
    - doc: add some documentation on OpenStack datasource.
    - OpenStack: Use timeout and retries from config in get_data.
      [Lars Kellogg-Stedman] (LP: #1657130)
    - Fixed Misc issues related to VMware customization. [Sankar Tanguturi]
    - Fix minor docs typo: perserve > preserve [Jeremy Bicha]
    - Use dnf instead of yum when available [Lars Kellogg-Stedman]
      (LP: #1647118)
    - validate-yaml: use python rather than explicitly python3
    - Get early logging logged, including failures of cmdline url.

 -- Scott Moser <email address hidden> Fri, 03 Feb 2017 21:54:39 -0500

Are we able to turn this bug public?

Hello Scott, or anyone else affected,

Accepted cloud-init into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-48-g1c795b9-0ubuntu1~16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Hello Scott, or anyone else affected,

Accepted cloud-init into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.9-48-g1c795b9-0ubuntu1~16.10.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

In order to see the security credentials, you must launch an instance with
iam role attached (--iam-instance-profile). Other wise they do not show up in the metadata service.

Ubuntu cloud-init did not actually show this bug because it read and stored
a version of the metadata service (2009-04-04) that does not have the iam/
credentials.

This can be verified simply by
$ sudo python3 -c 'from cloudinit.stages import _pkl_load as pl; import pprint; pprint.pprint(pl("/var/lib/cloud/instance/obj.pkl").metadata)'
{'ami-id': 'ami-78b7166e',
 'ami-launch-index': '0',
 'ami-manifest-path': '(unknown)',
 'block-device-mapping': {'ami': '/dev/sda1', 'root': '/dev/sda1'},
 'hostname': 'ip-10-0-0-160',
 'instance-action': 'none',
 'instance-id': 'i-00e90430c5dbe321c',
 'instance-type': 't2.micro',
 'local-hostname': 'ip-10-0-0-160',
 'local-ipv4': '10.0.0.160',
 'placement': {'availability-zone': 'us-east-1b'},
 'profile': 'default-hvm',
 'public-hostname': b'',
 'public-ipv4': '54.86.100.172',
 'public-keys': {'brickies': ['ssh-rsa '
                              'AAAAB3NzaC1yc2EAAAABIwAAAQEA3I7VUf2l5gSn5uavROsc5HRDpZdQueUq5ozemNSj8T7enqKHOEaFoU2VoPgGEWC9RyzSQVeyD6s7APMcE82EtmW4skVEgEGSbDc1pvxzxtchBj78hJP6Cf5TCMFSXw+Fz5rF1dR23QDbN1mkHs7adr8GW4kSWqU7Q7NDwfIrJJtO7Hi42GyXtvEONHbiRPOe8stqUly7MvUoN+5kfjBM8Qqpfl2+FNhTYWpMfYdPUnE7u536WqzFmsaqJctz3gBxH9Ex7dFtrxR4qiqEr9Qtlu3xGn7Bw07/+i1D+ey3ONkZLN+LQ714cgj8fRS4Hj29SCmXp5Kt5/82cD/VN3NtHw== '
                              'brickies']},
 'reservation-id': 'r-07f18d71dc1ebd0ab',
 'security-groups': 'wide-open'}

However, the get_instance_metadata() function would show the credentials.

$ python3 -c 'from cloudinit import ec2_utils; print(ec2_utils.get_instance_metadata("latest")["iam"])'
{'info': {'InstanceProfileArn': 'arn:aws:iam::950047163771:instance-profile/smtest-ec2-ro', 'InstanceProfileId': 'AIPAJ7VG3LHE4JOYDGNG6', 'Code': 'Success', 'LastUpdated': '2017-03-08T19:14:59Z'}, 'security-credentials': {'smtest-ec2-ro': {'Token': 'FQoDY...Bxgu=', 'AccessKeyId': 'XXXXXXXXXXXXXXXXXXXX', 'Expiration': '2017-03-09T01:43:31Z', 'Code': 'Success', 'SecretAccessKey': 'xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx', 'Type': 'AWS-HMAC', 'LastUpdated': '2017-03-08T19:14:40Z'}}}

# ami-78b7166e ubuntu/images-testing/hvm-ssd/ubuntu-yakkety-daily-amd64-server-20170307
$ ec2metadata --ami-id
ami-78b7166e

After upgrade..

$ dpkg-query --show cloud-init
cloud-init 0.7.9-48-g1c795b9-0ubuntu1~16.10.1

$ python3 -c 'from cloudinit import ec2_utils; print(ec2_utils.get_instance_metadata("latest")["iam"])'
{'info': {'InstanceProfileId': 'AIPAJ7VG3LHE4JOYDGNG6', 'InstanceProfileArn': 'arn:aws:iam::950047163771:instance-profile/smtest-ec2-ro', 'Code': 'Success', 'LastUpdated': '2017-03-08T19:33:11Z'}}

Notice that the 'security-credentials' dictionary is not present.

# ami-f4cc1de2 ubuntu/images/hvm-ssd/ubuntu-xenial-16.04-amd64-server-20170221
$ ec2metadata --ami-id
ami-f4cc1de2

$ dpkg-query --show cloud-init
cloud-init 0.7.9-48-g1c795b9-0ubuntu1~16.04.1

$ python3 -c 'from cloudinit import ec2_utils; print(ec2_utils.get_instance_metadata("latest")["iam"])'
{'info': {'Code': 'Success', 'InstanceProfileId': 'AIPAJ7VG3LHE4JOYDGNG6', 'InstanceProfileArn': 'arn:aws:iam::950047163771:instance-profile/smtest-ec2-ro', 'LastUpdated': '2017-03-08T19:15:50Z'}}

Notice that the 'security-credentials' entry is missing from the 'info' dict.

This bug was fixed in the package cloud-init - 0.7.9-48-g1c795b9-0ubuntu1~16.04.1

---------------
cloud-init (0.7.9-48-g1c795b9-0ubuntu1~16.04.1) xenial-proposed; urgency=medium

  * debian/rules: install Z99-cloudinit-warnings.sh to /etc/profile.d
  * debian/patches/ds-identify-behavior-xenial.patch: adjust default
    behavior of ds-identify for SRU (LP: #1669675, #1660385).
  * New upstream snapshot.
    - Support warning if the used datasource is not in ds-identify's list
      (LP: #1669675).
    - DatasourceEc2: add warning message when not on AWS. (LP: #1660385)
    - Z99-cloudinit-warnings: Add profile.d script for showing warnings on
    - Z99-cloud-locale-test.sh: convert tabs to spaces, remove unneccesary
      execute bit in permissions.
    - (RedHat) net: correct errors in cloudinit/net/sysconfig.py
      [Lars Kellogg-Stedman]
    - ec2_utils: fix MetadataLeafDecoder that returned bytes on empty
    - Fix eni rendering of multiple IPs per interface [Ryan Harper]
      (LP: #1657940)
    - Add 3 ecdsa-sha2-nistp* ssh key types now that they are standardized
      [Lars Kellogg-Stedman]
    - EC2: Do not cache security credentials on disk [Andrew Jorgensen]
      (LP: #1638312)
    - OpenStack: Use timeout and retries from config in get_data.
      [Lars Kellogg-Stedman] (LP: #1657130)
    - Fixed Misc issues related to VMware customization. [Sankar Tanguturi]
    - (RedHat) Use dnf instead of yum when available [Lars Kellogg-Stedman]
    - Get early logging logged, including failures of cmdline url.
    - test / doc / build environment changes
      - Remove style checking during build and add latest style checks to
        tox [Joshua Powers]
      - code-style: make master pass pycodestyle (2.3.1) cleanly, currently
        [Joshua Powers]
      - Fix small typo and change iso-filename for consistency
      - tools/mock-meta: support python2 or python3 and ipv6 in both.
      - tests: remove executable bit on test_net, so it runs, and fix it.
      - tests: No longer monkey patch httpretty for python 3.4.2
      - reset httppretty for each test [Lars Kellogg-Stedman]
      - build: fix running Make on a branch with tags other than master
      - doc: Fix typos and clarify some aspects of the part-handler
        [Erik M. Bray]
      - doc: add some documentation on OpenStack datasource.
      - Fix minor docs typo: perserve > preserve [Jeremy Bicha]
      - validate-yaml: use python rather than explicitly python3

 -- Scott Moser <email address hidden> Mon, 06 Mar 2017 16:34:10 -0500

The verification of the Stable Release Update for cloud-init has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package cloud-init - 0.7.9-48-g1c795b9-0ubuntu1~16.10.1

---------------
cloud-init (0.7.9-48-g1c795b9-0ubuntu1~16.10.1) yakkety; urgency=medium

  * debian/rules: install Z99-cloudinit-warnings.sh to /etc/profile.d
  * debian/patches/ds-identify-behavior-yakkety.patch: adjust default
    behavior of ds-identify for SRU (LP: #1669675, #1660385).
  * New upstream snapshot.
    - Support warning if the used datasource is not in ds-identify's list
      (LP: #1669675).
    - DatasourceEc2: add warning message when not on AWS. (LP: #1660385)
    - Z99-cloudinit-warnings: Add profile.d script for showing warnings on
    - Z99-cloud-locale-test.sh: convert tabs to spaces, remove unneccesary
      execute bit in permissions.
    - (RedHat) net: correct errors in cloudinit/net/sysconfig.py
      [Lars Kellogg-Stedman]
    - ec2_utils: fix MetadataLeafDecoder that returned bytes on empty
    - Fix eni rendering of multiple IPs per interface [Ryan Harper]
      (LP: #1657940)
    - Add 3 ecdsa-sha2-nistp* ssh key types now that they are standardized
      [Lars Kellogg-Stedman]
    - EC2: Do not cache security credentials on disk [Andrew Jorgensen]
      (LP: #1638312)
    - OpenStack: Use timeout and retries from config in get_data.
      [Lars Kellogg-Stedman] (LP: #1657130)
    - Fixed Misc issues related to VMware customization. [Sankar Tanguturi]
    - (RedHat) Use dnf instead of yum when available [Lars Kellogg-Stedman]
    - Get early logging logged, including failures of cmdline url.
    - test / doc / build environment changes
      - Remove style checking during build and add latest style checks to
        tox [Joshua Powers]
      - code-style: make master pass pycodestyle (2.3.1) cleanly, currently
        [Joshua Powers]
      - Fix small typo and change iso-filename for consistency
      - tools/mock-meta: support python2 or python3 and ipv6 in both.
      - tests: remove executable bit on test_net, so it runs, and fix it.
      - tests: No longer monkey patch httpretty for python 3.4.2
      - reset httppretty for each test [Lars Kellogg-Stedman]
      - build: fix running Make on a branch with tags other than master
      - doc: Fix typos and clarify some aspects of the part-handler
        [Erik M. Bray]
      - doc: add some documentation on OpenStack datasource.
      - Fix minor docs typo: perserve > preserve [Jeremy Bicha]
      - validate-yaml: use python rather than explicitly python3

 -- Scott Moser <email address hidden> Mon, 06 Mar 2017 16:37:28 -0500

This bug is believed to be fixed in cloud-init in 17.1. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/2750