please break out embedded code copies into archive packages

Bug #1508120 reported by Jamie Strandboge
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
juju-core (Ubuntu)
New
Undecided
Unassigned

Bug Description

In the MIR bug 1267393 it was brought up that juju-core uses many embedded code copies and a condition for MIR acceptance was to use archive packages instead. Unfortunately, this work was not completed in time for 15.10, however the juju-core team committed to fixing this up for 16.04.

Quite a few things were done for 15.10 already though, specifically, juju-core is now use the following archive packages instead of using the embedded code copies:
* golang-check.v1
* golang-github-bmizerany-assert
* golang-github-bmizerany-pat
* golang-go-dbus
* golang-go.crypto
* golang-golang-x-net-dev
* golang-juju-loggo
* golang-x-text
* juju-mongodb

For the remaining embedded copies, here is a list to start the conversation for *Ubuntu archive builds* and using embedded sources (again, if it helps to leave the sources in the orig.tar.gz for other reasons, fine):

* embedded that seem obviously ok to leave alone and use during the build:
 * github.com/juju/* (do pull out anything that's reused in other go sources though)
 * gopkg.in/juju/* (same here)

* embedded that seems clear should be cleaned up/pulled out/use existing archive -dev package:
 * gopkg.in/mgo.v2 (use golang-gopkg-mgo.v2-dev)
 * gopkg.in/yaml.v1 (use golang-yaml.v2-dev, juju team already has an MP to transition to golang-yaml.v2-dev)
 * launchpad.net/tomb (use golang-gopkg-tomb.v2-dev)

* embedded that may need other Canonical upstream involvement to create golang-*-dev packages:
 * launchpad.net/golxc (LXD embeds this)
 * launchpad.net/gomaasapi (seems like our maas packages should be adjusted to provide golang-gomaasapi-dev?)

* embedded sources where is is unclear if it is juju-specific or something that should be broken out
 * bitbucket.org/kardianos/osext (seems like something useful to others? ie, break out?)
 * bitbucket.org/kardianos/service (seems like something useful to others? ie, break out?)
 * code.google.com/p/winsvc (seems like something useful to others? ie, break out?)
 * github.com/joyent/* (juju-specific?)
 * gopkg.in/natefinch/lumberjack.v2 (??)
 * gopkg.in/natefinch/npipe.v2 (seems like something useful to others? ie, break out?)
 * launchpad.net/gnuflag (seems like something useful to others? ie, break out?)
 * launchpad.net/goamz/* (juju-specific? (amazon))
 * launchpad.net/goose/* (juju-specific? (openstack))
 * launchpad.net/gwacl/* (seems like something useful to others? ie, break out?)

Revision history for this message
Curtis Hovey (sinzui) wrote :

The 1.24.6-0ubuntu3 package in both wily and xenial use:
golang-go-dbus-dev
golang-go.crypto-dev
golang-go.net-dev
golang-juju-loggo-dev

There are/were inompatabilities with some of the other -dev packakages in
wily. The Juju QA team are working on updating packages in Debian to so that
the they will be available for Ubuntu and the juju-core package.

The listing of go packages used by Juju changes frequently. 1.25.x uses

github.com/ajstarks/svgo
github.com/altoros/gosigma
github.com/bmizerany/pat
github.com/coreos/go-systemd
github.com/dustin/go-humanize
github.com/gabriel-samfira/sys
github.com/godbus/dbus
github.com/joyent/gocommon
github.com/joyent/gomanta
github.com/joyent/gosdc
github.com/joyent/gosign
github.com/juju/blobstore
github.com/juju/cmd
github.com/juju/deputy
github.com/juju/errors
github.com/juju/gojsonpointer
github.com/juju/gojsonreference
github.com/juju/gojsonschema
github.com/juju/govmomi
github.com/juju/httpprof
github.com/juju/jujusvg
github.com/juju/loggo
github.com/juju/names
github.com/juju/persistent-cookiejar
github.com/juju/ratelimit
github.com/juju/replicaset
github.com/juju/schema
github.com/juju/syslog
github.com/juju/testing
github.com/juju/txn
github.com/juju/utils
github.com/juju/xml
golang.org/x/crypto
golang.org/x/net
golang.org/x/oauth2
google.golang.org/api
google.golang.org/cloud
gopkg.in/amz.v3
gopkg.in/check.v1
gopkg.in/errgo.v1
gopkg.in/goose.v1
gopkg.in/juju/charm.v5
gopkg.in/juju/charmstore.v4
gopkg.in/juju/environschema.v1
gopkg.in/macaroon-bakery.v0
gopkg.in/macaroon.v1
gopkg.in/mgo.v2
gopkg.in/natefinch/lumberjack.v2
gopkg.in/natefinch/npipe.v2
gopkg.in/yaml.v1
launchpad.net/gnuflag
launchpad.net/golxc
launchpad.net/gomaasapi
launchpad.net/gwacl
launchpad.net/tomb

And we know that launchpad.net/gwacl will be retired in a few months

Revision history for this message
Jamie Strandboge (jdstrand) wrote :

In https://bugs.launchpad.net/ubuntu/+source/juju-core/+bug/1545913/comments/12 I mentioned that a number of the embedded copies are now being used in the juju 2 packaging (thanks!). I've also asked that the remaining 15 embedded copies be enumerated with the plan to address them.

Revision history for this message
Jamie Strandboge (jdstrand) wrote :

MY last comment was confusing. Let me retry:

In https://bugs.launchpad.net/ubuntu/+source/juju-core/+bug/1545913/comments/12 I mentioned that the number of embedded copies is reduced in juju2 on 16.04 and juju2 is using archive -dev packages for many of the copies listed in this bug. Great! :)

Please enumerate the remaining 15 embedded copies and the plan to address them.

Thanks!

Revision history for this message
Nicholas Skaggs (nskaggs) wrote :

Jamie, the additional packages are:

golang-github-altoros-gosigma
golang-github-dustin-go-humanize
golang-github-gabriel-samfira-sys
golang-github-gosuri-uitable
golang-github-joyent-gocommon
golang-github-joyent-gomanta
golang-github-joyent-gosdc
golang-github-joyent-gosign
golang-github-mattn-go-runewidth
golang-gnuflag-dev
golang-gopkg-errgo.v1
golang-gopkg-goose.v1
golang-gopkg-lxc-go-lxc.v2
golang-gopkg-macaroon.v1
golang-gopkg-natefinch-npipe.v2

These are currently packaged in my ppa and should be ready for upload for Y.

There are 2 additional that are not targeted.

google.golang.org/cloud
gopkg.in/macaroon-bakery.v1

have a spiderweb of depends, and I think we would prefer to leave them as embeds, but we can work on it during Y and come to a solution.

Revision history for this message
Nicholas Skaggs (nskaggs) wrote :

Here's the ppa with the packages:

https://launchpad.net/~nskaggs/+archive/ubuntu/juju/+packages

And here's the status for each dependency mentioned.

http://paste.ubuntu.com/15569917/

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.