juju-core is unavailable for arm64

juju-core relies on mongodb, which is also not available for arm64. mongodb has a build-dependency on libv8, which requires porting to arm64.

juju-core also depend on golang of course, which is also not available for arm64.

Here is the status as of 2012-12-17 as I understand it:

1) Go
   a) there is a version of gccgo in trusty that can compile juju, but only with a patched version of the 'go tool'
      i) need kick these patches upstream (they are in review)
      ii) need a plan to provide a gccgo-compiled go tool in trusty
   b) there is a problem with setting up the heap for statically linked gccgo-compiled binaries
      i) need to beat some feedback out of golang-dev about this
      ii) we can provide a patch for this for trusty regardless of what upstream does

2) mongodb
   a) the v8 thing
      i) it is possible a v8 port to arm64 will appear, although it's not clear that this will suffice for mongodb
      ii) mongodb can be built without js support though, and we think juju will work with such a mongodb (need to check this!)
   b) even without v8, some porting is required
      i) intel assembly can be replaced with gcc intrinsics (Robie has patches for this and is planning on sending them upstream)
      ii) there appears to be a toolchain bug in trusty (I've been pointed to a fix upstream and will be testing that out as soon as I finish typing this comment!)

So I tried this patch:
=== modified file 'testing/mgo.go'
--- testing/mgo.go 2013-11-06 13:38:01 +0000
+++ testing/mgo.go 2013-12-18 06:23:25 +0000
@@ -99,6 +99,7 @@
   "--noprealloc",
   "--smallfiles",
   "--nojournal",
+ "--noscripting",
   "--nounixsocket",
  }
  server := exec.Command("mongod", mgoargs...)

And with it, the test suite still passes 100% for me.

I did try connecting to one of the mongodb instances running during the test suite and run:
> db.eval("hello")
Wed Dec 18 11:06:31 uncaught exception: { "errmsg" : "db side execution is disabled", "ok" : 0 }

I even tried:
> var map = function() { emit ( this._id, this.format) }
> var reduce = function(key, values) {return Array.sum(values)}
> db.charms.mapReduce(map, reduce, {out: "format"})
Wed Dec 18 11:13:09 uncaught exception: map reduce failed:{
 "errmsg" : "exception: cannot run map reduce without the js engine",
 "code" : 16149,
 "ok" : 0
}

So I have a strong feeling that --noscripting does what we want it to, and that we don't, indeed, need the ability.

I'm not sure what test Michael had seen that used something that needed scripting, but I'm running against mongodb 2.2.4 and it seems to be working with --noscripting set up.

John A Meinel <email address hidden> writes:

> So I tried this patch:
> === modified file 'testing/mgo.go'
> --- testing/mgo.go 2013-11-06 13:38:01 +0000
> +++ testing/mgo.go 2013-12-18 06:23:25 +0000
> @@ -99,6 +99,7 @@
> "--noprealloc",
> "--smallfiles",
> "--nojournal",
> + "--noscripting",
> "--nounixsocket",
> }
> server := exec.Command("mongod", mgoargs...)
>
>
> And with it, the test suite still passes 100% for me.

\o/

> So I have a strong feeling that --noscripting does what we want it to,
> and that we don't, indeed, need the ability.
>
> I'm not sure what test Michael had seen that used something that needed
> scripting, but I'm running against mongodb 2.2.4 and it seems to be
> working with --noscripting set up.

It was state/machine_test.go and state/unit_test.go (search for
$where). But on looking, the tests might actually be testing that
queries like this are filtered out or something -- avoiding the
equivalent of an SQL injection? Not really sure.

Thanks heaps for testing this.

With selected changes from Michael and Robie I've managed to build a mongodb with a) javascript support disabled and b) compile support for arm64 and powerpc64el:

 https://launchpad.net/ubuntu/+source/juju-mongodb/2.4.8-0ubuntu5

However there are currently unit test failures for arm64 and ppc64el which need to be resolved.

I also see a failure on armhf - however I am running the Cpp unit tests, rather than the javascript dependent full smoke test suite which could have issues on non-x86 archs.

Reworking the test suite to run the normal smoke tests minus the jstests suite results in a pass and successful build on all architectures including arm64 and ppc64el.

James Page <email address hidden> writes:

> Reworking the test suite to run the normal smoke tests minus the jstests
> suite results in a pass and successful build on all architectures
> including arm64 and ppc64el.

That's pretty awesome. Now, do the juju tests pass?

(Trick question I guess, juju doesn't compile natively on arm64 because
of https://bugs.launchpad.net/ubuntu/+source/gcc-4.8/+bug/1263806)

Cheers,
mwh

Juju for 14.04 will use the juju-mongodb package which does build on arm64

marking v8 and mongodb tasks as invalid.

I've just uploaded gccgo-go to Ubuntu Trusty; this should build the go tool for all architectures using gccgo.

Just needs an archive-admin to review and accept.

This bug was fixed in the package juju-core - 1.17.0-0ubuntu3

---------------
juju-core (1.17.0-0ubuntu3) trusty; urgency=medium

  * Enable support for architectures other than x86/armhf (LP: #1261452):
    - d/control: Add BD on gccgo-go for archs that don't have golang-go,
      limit use of golang-go to supported archs only.
 -- James Page <email address hidden> Mon, 27 Jan 2014 21:55:12 +0000

juju-core 1.17.0-0ubuntu3 timed out building, but it looks like a kernel issue. I reproduced this hang on both a 3.8 and a 3.12 kernel. However, I tested w/ a 3.13 (hand rolled w/ xgene support), and juju-core built fine.

juju-core should build on the buildds if we configure them with:

 echo never | sudo tee /sys/kernel/mm/transparent_hugepages/enabled

(tested w/ 3.8 and 3.13 kernels)