Support Joyent lx-brand environment in smartos datasource

Bug #1540965 reported by Robert C Jennings
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init (Ubuntu)
Fix Released
High
Scott Moser
Declined for Vivid by Ben Howard
Trusty
Fix Released
High
Unassigned
Wily
Fix Released
High
Unassigned
Xenial
Fix Released
High
Scott Moser

Bug Description

[Impact]

 * Cloud-init does not support the datasource presented to LX-Brand
   zones in SmartOS. Without this support cloud-init does not work
   on the Joyent cloud platform's container service.

 * The metadata is provided in the same format as already supported
   for KVM-based instances for the SmartOS datasource (with
   communication over a serial port). For LX-Brand zones the
   metadata is accessed via a socket file.

[Test Case]

 * Install cloud-init on an lx-brand zone image and
   run 'cloud-init init'. No metadata will be found without
   this patch.

 * With the patched cloud-init, run 'cloud-init init' on the
   new lx-brand zones images. The metadata should be discovered
   and the instance should be recognized as being provisioned.
   Perform the same testing on existing KVM-based SmartOS instances
   to ensure there is no regression.

[Regression Potential]

 * [Trusty only] This SRU backports support for the V2 metadata format
   that has been present since the Vivid release. This is well tested
   code and has no user-facing impact. This was done because the
   V2 datasource allows for serial or socket communication. The existing
   Joyent datasource uses the serial port provided in KVM instances,
   however LX-brand zones have a socket instead @
   /native/.zonecontrol/metadata.sock. Additionally the V2 datasource
   provides checksums and response length fields that allow for
   validation of results. Additional testing was performed on Trusty
   SmartOS and KVM instances to validate these changes.

 * To keep code common for KVM (serial) and LX-Brand (socket)
   communication the code had to switch to read byte-by-byte from
   the metadata source file object to avoid reading past the end of
   input on the socket which would block indefinitely. This is a change
   from using readline and will be slower but not significantly so.

 * This builds on the existing SmartOS datasource to enable LX-Brand.
   There is risk that this will break the datasouce on existing KVM
   instances. To mitigate this risk testing has been performed on
   both environments with Trusty, Vivid (out of support today),
   Wily, and Xenial.

[Other Info]

 * This is python 2/3 byte/string safe on Vivid, Wily, and Trusty
   where cloud-init supports python 3.

==== Original Description ====

The Joyent Metadata Protocol Specification (Version 2)[1] allows for serial or socket communication. The existing Joyent datasource uses the serial port provided in KVM instances, however LX-brand zones have a socket instead @ /native/.zonecontrol/metadata.sock.

Detecting we are in a Joyent LX-brand zone is possible by looking at '/bin/uname -v' which will report "BrandZ virtual linux" or checking for the existence of a /native mount where '/native/usr/bin/uname -s' reports
SunOS.

Related branches

Revision history for this message
Robert C Jennings (rcj) wrote :
Revision history for this message
Robert C Jennings (rcj) wrote :
Revision history for this message
Robert C Jennings (rcj) wrote :
Revision history for this message
Robert C Jennings (rcj) wrote :
Revision history for this message
Robert C Jennings (rcj) wrote :

SmartOS: Add support for Joyent LX-Brand Zones

LX-brand zones on Joyent's SmartOS use a different metadata source
(socket file) than the KVM-based SmartOS virtualization (serial port).
This patch adds support for recognizing the different flavors of
virtualization on SmartOS and setting up a metadata source file object.
After the file object is created, the rest of the code for the datasource
can remain common.

This patch reads the metadata byte-by-byte rather than using readline
because we can not perform a readline on the file-like object for the
socket as this block indefintely waiting for an EOF that is will not be
sent by the host platform. This patch also moves to V2 metadata as it
provides checksum validation and makes reading the metadata much more
reliable.

Revision history for this message
Robert C Jennings (rcj) wrote :

Assigning to Scott for patch review. I will update the description with SRU template.

Changed in cloud-init (Ubuntu):
assignee: Robert C Jennings (rcj) → Scott Moser (smoser)
Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

The attachment "trusty.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

tags: added: patch
Revision history for this message
Robert C Jennings (rcj) wrote :

I am going to delete the patches and post them again. I need to update them from the final patch that was merged upstream. They were here prematurely.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.7~bzr1160-0ubuntu1

---------------
cloud-init (0.7.7~bzr1160-0ubuntu1) xenial; urgency=medium

  * New upstream snapshot.
    * SmartOS: Add support for Joyent LX-Brand Zones (LP: #1540965)
    * systemd/power_state: fix power_state when cloud-final exited failure
      (LP: #1449318)

 -- Scott Moser <email address hidden> Thu, 04 Feb 2016 17:22:36 -0500

Changed in cloud-init (Ubuntu):
status: In Progress → Fix Released
Revision history for this message
Robert C Jennings (rcj) wrote :

The code was accepted upstream and published for Xenial in cloud-init 0.7.7~bzr1160-0ubuntu1 .

Revision history for this message
Robert C Jennings (rcj) wrote :

SRU templated added and ready for upload/sponsorship.

Scott, can you ack the upload?
Ben, can you sponsor this?

Thanks.

description: updated
Revision history for this message
Robert C Jennings (rcj) wrote :

New trusty.debdiff based on feedback from Ben. I have cleaned up the changelog (version and path to patch), fixed the diff to have the correct from address, and changed dmi_data() to call util.read_dmi_data() rather than use a local implementation of that util function.

Revision history for this message
Robert C Jennings (rcj) wrote :

New wily.debdiff based on feedback from Ben. I have cleaned up the changelog (version and path to patch).

Revision history for this message
Ben Howard (darkmuggle-deactivatedaccount) wrote :

Reviewed and upload for 14.04 and 15.10. Pending acceptance into proposed.

Mathew Hodson (mhodson)
Changed in cloud-init (Ubuntu Trusty):
importance: Undecided → High
Changed in cloud-init (Ubuntu Wily):
importance: Undecided → High
Revision history for this message
Chris J Arges (arges) wrote : Please test proposed package

Hello Robert, or anyone else affected,

Accepted cloud-init into wily-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.7~bzr1149-0ubuntu6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in cloud-init (Ubuntu Wily):
status: New → Fix Committed
tags: added: verification-needed
Changed in cloud-init (Ubuntu Trusty):
status: New → Fix Committed
Revision history for this message
Chris J Arges (arges) wrote :

Hello Robert, or anyone else affected,

Accepted cloud-init into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/cloud-init/0.7.5-0ubuntu1.16 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Robert C Jennings (rcj)
tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.5-0ubuntu1.16

---------------
cloud-init (0.7.5-0ubuntu1.16) trusty; urgency=medium

  * Joyent Smart DataOS:
    - d/patches/lp-1540965-SmartOS-Add-support-for-Joyent-LX-Brand-Zones.patch:
      SmartOS: Add support for Joyent LX-Brand Zones (LP: #1540965)

 -- Robert C Jennings <email address hidden> Tue, 02 Feb 2016 09:32:02 -0600

Changed in cloud-init (Ubuntu Trusty):
status: Fix Committed → Fix Released
Revision history for this message
Adam Conrad (adconrad) wrote : Update Released

The verification of the Stable Release Update for cloud-init has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package cloud-init - 0.7.7~bzr1149-0ubuntu6

---------------
cloud-init (0.7.7~bzr1149-0ubuntu6) wily; urgency=medium

  * Joyent Smart DataOS:
    - d/patches/lp-1540965-SmartOS-Add-support-for-Joyent-LX-Brand-Zones.patch:
      SmartOS: Add support for Joyent LX-Brand Zones (LP: #1540965)

 -- Robert C Jennings <email address hidden> Tue, 02 Feb 2016 16:41:52 -0600

Changed in cloud-init (Ubuntu Wily):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.