please use archive packages instead of embedded code copies

Bug #1507156 reported by Jamie Strandboge
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
lxd (Ubuntu)
Fix Released
High
Stéphane Graber

Bug Description

In the MIR bug 1481507 it was brought up that lxd uses many embedded code copies and a condition for MIR acceptance was to use archive packages instead. Unfortunately, this was not able to be performed for 15.10, however the lxd team committed to the following plan:

"> - The lxd team will break apart the vendorized Go dependencies. Discuss
> with the Juju team which shared dependencies, if any, should be split
> apart first, and aim to have all external dependencies removed from the
> lxd package before 16.04 beta. See the Golang MIR bug for full details:
> https://bugs.launchpad.net/ubuntu/+source/juju-mongodb/+bug/1267393

This is planned, current status is:

DEPENDENCY PACKAGE or PLAN
dist/src/code.google.com/p/go-charset/ TODO: Implement tiny HTTP proxy?
dist/src/github.com/chai2010/gettext-go/ golang-gettext-dev
dist/src/github.com/dustinkirkland/golang-petname/ golang-petname-dev
dist/src/github.com/elazarl/goproxy/ TODO: Implement tiny HTTP proxy?
dist/src/github.com/godbus/dbus/ golang-dbus-dev
dist/src/github.com/golang/protobuf/ golang-goprotobuf-dev
dist/src/github.com/gorilla/context/ golang-context-dev
dist/src/github.com/gorilla/mux/ golang-mux-dev
dist/src/github.com/gorilla/websocket/ golang-websocket-dev
dist/src/github.com/inconshreveable/go-vhost/ golang-vhost-dev
dist/src/github.com/mattn/go-colorable/ TODO: Change logging system?
dist/src/github.com/mattn/go-sqlite3/ TODO: Is golang-gosqlite-dev good enough?
dist/src/github.com/olekukonko/tablewriter/ TODO: Alternatives?
dist/src/github.com/satori/go.uuid/ golang-uuid-dev
dist/src/github.com/stgraber/lxd-go-systemd/ golang-go-systemd-dev
dist/src/github.com/stretchr/objx/ golang-objx-dev
dist/src/github.com/stretchr/testify/ golang-testify-dev
dist/src/github.com/syndtr/gocapability/ golang-gocapability-dev
dist/src/golang.org/x/crypto/ golang-go.crypto-dev
dist/src/gopkg.in/check.v1/ golang-check.v1-dev
dist/src/gopkg.in/flosch/pongo2.v3/ NEEDS PACKAGING
dist/src/gopkg.in/inconshreveable/log15.v2/ TODO: Change logging system?
dist/src/gopkg.in/lxc/go-lxc.v2/ NEEDS PACKAGING
dist/src/gopkg.in/tomb.v2/ golang-gopkg-tomb.v2-dev
dist/src/gopkg.in/yaml.v2/ golang-yaml.v2-dev

My plan is to open 16.04 with LXD switching to all the existing packaged
ones, note that this means that 17 MIRs will be filed and sent your way.

We'll then be packaging go-lxc and pongo2 and look at the remaining 6 to
see whether we can drop them somehow and if not, package them."

This bug is to track that work. Also not, MIRs will of course be needed for all of these not already covered by the juju MIR.

Changed in lxd (Ubuntu):
status: New → Triaged
importance: Undecided → High
Changed in lxd (Ubuntu):
milestone: none → ubuntu-16.04
assignee: nobody → Stéphane Graber (stgraber)
Revision history for this message
Stéphane Graber (stgraber) wrote :

Current state of things in the archive is (all in universe):

dist/src/github.com/chai2010/gettext-go golang-gettext-dev
dist/src/github.com/davecgh/go-spew golang-github-davecgh-go-spew-dev
dist/src/github.com/dustinkirkland/golang-petname golang-petname-dev
dist/src/github.com/elazarl/goproxy Not packaged, replace by our own proxy
dist/src/github.com/godbus/dbus golang-dbus-dev
dist/src/github.com/golang/protobuf golang-goprotobuf-dev
dist/src/github.com/gorilla/context golang-context-dev
dist/src/github.com/gorilla/mux golang-mux-dev
dist/src/github.com/gorilla/websocket golang-websocket-dev
dist/src/github.com/inconshreveable/go-vhost golang-vhost-dev
dist/src/github.com/mattn/go-colorable Debian ITP (needed by log15)
dist/src/github.com/mattn/go-sqlite3 golang-github-mattn-go-sqlite3-dev
dist/src/github.com/olekukonko/tablewriter golang-github-olekukonko-tablewriter-dev
dist/src/github.com/pmezard/go-difflib Not packaged (needed by newer testify)
dist/src/github.com/rogpeppe/go-charset Not packaged (needed by goproxy)
dist/src/github.com/satori/go.uuid golang-uuid-dev
dist/src/github.com/stgraber/lxd-go-systemd golang-go-systemd-dev
dist/src/github.com/stretchr/objx golang-objx-dev
dist/src/github.com/stretchr/testify golang-testify-dev
dist/src/github.com/syndtr/gocapability golang-gocapability-dev
dist/src/golang.org/x/crypto golang-go.crypto-dev
dist/src/gopkg.in/check.v1 golang-check.v1-dev
dist/src/gopkg.in/flosch/pongo2.v3 Not packaged, needs packaging
dist/src/gopkg.in/inconshreveable/log15.v2 Not packaged, needs packaging or replacement
dist/src/gopkg.in/lxc/go-lxc.v2 Not packaged, needs packaging
dist/src/gopkg.in/tomb.v2 golang-gopkg-tomb.v2-dev
dist/src/gopkg.in/yaml.v2 golang-yaml.v2-dev

Revision history for this message
Stéphane Graber (stgraber) wrote :

So we have 20 packages that we're using and which are in the archive.

The plan is to write a script to file the MIRs or just do a lot of copy/paste, all of those are source-only (for the bits we care anyway), snapshots from git from usptreams which don't do versioning, stable releases or any kind of security announcements.

By normal MIR standards, none of those are suitable for main, but as it's the security team's wish to have them there as separate package rather than bundled in our source package, we'll file those anyway.

Revision history for this message
Stéphane Graber (stgraber) wrote :

Latest upstream releases changes:
  DROP dist/src/github.com/elazarl/goproxy
  DROP dist/src/github.com/inconshreveable/go-vhost
  DROP dist/src/github.com/rogpeppe/go-charset
  ADD dist/src/github.com/mattn/go-runewidth (used by newer tablewriter)

That addition shouldn't actually matter MIR-wise as the packaged version of tablewriter is old enough not to have this.

Revision history for this message
Stéphane Graber (stgraber) wrote :
Revision history for this message
Stéphane Graber (stgraber) wrote :

Just uploaded a new LXD which uses the following bits from the archive:
    - golang-go.crypto-dev
    - golang-context-dev
    - golang-github-gorilla-mux-dev
    - golang-github-mattn-go-sqlite3-dev
    - golang-github-olekukonko-tablewriter-dev
    - golang-gopkg-tomb.v2-dev
    - golang-goprotobuf-dev
    - golang-petname-dev
    - golang-yaml.v2-dev

We have the following currently still in universe (require some minor packaging fixes):
    - golang-gettext-dev
    - golang-github-coreos-go-systemd-dev
    - golang-github-pborman-uuid-dev
    - golang-gocapability-dev
    - golang-websocket-dev

And then the following which need packaging from scratch:
    - golang-github-inconshreveable-log15.v2-dev
    - golang-github-mattn-go-colorable-dev
    - golang-gopkg-flosch-pong2.v3-dev
    - golang-gopkg-lxc-go-lxc.v2-dev

We're currently re-building the temporary dh-golang GOPATH in our packaging, including the bits from the archive and then for each missing dependency, adding links to use the bundled code.

We expect some version of that to remain in the packaging for use with PPA uploads and backports where backporting 20 or so extra packages and keeping them in sync without causing regressions just isn't feasible.

Revision history for this message
Stéphane Graber (stgraber) wrote :

Answere review comments and fixed when appropriate for:
 - golang-github-coreos-go-systemd-dev
 - golang-github-pborman-uuid-dev
 - golang-gocapability-dev

Uploaded renamed source for:
 - golang-gettext-dev

Blocked on the security team for:
 - golang-websocket-dev

Filed a new MIR bug for:
 - golang-dbus-dev (LP: #1524514)

Uploaded new package for:
 - golang-gopkg-lxc-go-lxc.v2-dev

I'll probably prepare and uploaded the other 3 new sources tonight or tomorrow at which point we'll be entirely blocked on other teams (archive admin for new sources and MIR or security for existing sources).

Revision history for this message
Stéphane Graber (stgraber) wrote :

Uploaded golang-github-mattn-go-colorable-dev to xenial.

Revision history for this message
Stéphane Graber (stgraber) wrote :

Uploaded golang-gopkg-flosch-pongo2.v3 to xenial

Revision history for this message
Stéphane Graber (stgraber) wrote :

Uploaded golang-gopkg-inconshreveable-log15.v2 to xenial

Revision history for this message
Stéphane Graber (stgraber) wrote :

Uploaded a new LXD using the following packages from the archive:
    - golang-dbus-dev
    - golang-github-coreos-go-systemd
    - golang-github-pborman-uuid-dev
    - golang-gocapability-dev

It also introduces a new binary package to be used by our downstreams:
 - golang-github-lxc-lxd-dev

Revision history for this message
Stéphane Graber (stgraber) wrote :

The current status is.

Packages in NEW:
 - golang-github-gosexy-gettext-dev
 - golang-gopkg-inconshreveable-log15.v2-dev
 - golang-github-mattn-go-colorable-dev
 - golang-gopkg-flosch-pongo2.v3-dev
 - golang-gopkg-lxc-go-lxc.v2-dev

Packages stuck on security team review:
 - golang-websocket-dev

Those make up all our remaining dependencies, once all in the archive and promoted to main, LXD will be built entirely from non-bundled sources.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package lxd - 2.0.0~rc8-0ubuntu5

---------------
lxd (2.0.0~rc8-0ubuntu5) xenial; urgency=medium

  * Update dependencies:
    - Depend on dnsmasq-base for lxd-bridge
    - Depend on iptables for lxd-bridge
    - Drop dependency on python3 (lxd-images is no more)
    - Drop dependency on gnupg (lxd-images is no more)
  * Recommend "lxd init" at installation time rather than dpkg-reconfigure.

 -- Stéphane Graber <email address hidden> Tue, 05 Apr 2016 17:31:07 -0400

Changed in lxd (Ubuntu):
status: Triaged → Fix Released
Revision history for this message
Jamie Strandboge (jdstrand) wrote :

Thanks for working on this Stephane! :)

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.