[FFe] Updates to enable us to drop xulrunner from main

I would like to release a JS 1.8.5 tar ball ~ concurrently with Firefox 4.

Here is a sample tar-ball and browseable tree based on a snapshot of today's tracemonkey repo: http://www.page.ca/~wes/js-releng/

Some notes where I've differed from the existing release process at http://www.mozilla.org/js/spidermonkey/release-notes/spidermonkey-releases.html:

 - run autoconf213 in js/src so that we ship a working "configure"
 - steps related to Makefile.ref, js.mak, etc elided
 - I have not updated the change log. Is this really necessary? Is there a non-manual way to do this? There were ... several ... changes made between 1.7.0 and 1.8.5
 - Obviously, CVS tagging no longer applies. How about we tag the hg rev we push with bug 586016?
 - Tar now has one more top-level directory -- this is to allow us to ship things like NSPR and jemalloc later if we want to in the same tar ball
 - README moved to higher level dir

Also.... Can I please, please change "libmozjs.so" to a better name in the tar release? How about libmozjs185.so.1.0.0? I think this would help the distro use-case, and make it possible to ship bug fixes on top of JS-1.8.5-the-language -- while properly advertising API/ABI compatibility. And, of course, making room for JS-1.8.6-the-language.

(In reply to comment #0)
> Also.... Can I please, please change "libmozjs.so" to a better name in the tar
> release? How about libmozjs185.so.1.0.0? I think this would help the distro
> use-case, and make it possible to ship bug fixes on top of
> JS-1.8.5-the-language -- while properly advertising API/ABI compatibility. And,
> of course, making room for JS-1.8.6-the-language.

Bug 618631 does this, sort of. It hasn't had any action lately, so you may have to pick up the pieces.

With regard to the actual number, what version was the last beta we put out? Should we not be changing major numbers (is that the 1 or the 8?) since we've broken everything since our last release? I think we previously were somehow tied to Gecko version numbers (?); are you proposing breaking away from this?

(In reply to comment #1)
> Bug 618631 does this, sort of. It hasn't had any action lately, so you may have
> to pick up the pieces.

I meant 618381, my apologies.

I suppose I should chime in here and say that a new official release is really important for those of us who are working with CouchDB on Android. Newer versions of libmozjs are absolutely essential for Android and there is (not unwarranted) hesitation amongst the CouchDB folks to use an unofficial release of Spidermonkey.

In short, a new release is something that I would REALLY like to see.

For what it's worth -- Chris Coulson (copied on this bug already) from Ubuntu is maintaining a CouchDB fork which compiles on tracemonkey tip.

Hopefully a JS 185 source release will be adopted quickly by Ubuntu and spur Couch into taking Chris' patches.

Once Firefox 4 ships, I'll take a stab another release candidate and hopefully Boss@Moz will bless it.

A few points:

- I think this is a great idea and I'm thrilled that Wes is taking the job.

- Let's call it 1.8.5 and libmozjs185. SpiderMonkey version numbers have
  historically tracked the language version, not the Gecko version.

- We will not take any patches (in the tracemonkey repo or mozilla-central)
  before Firefox 4 ships for this purpose.

- We will not take any patches in the stable Firefox 4 release branch
  for this purpose either.

- Wes's plan, as I understand it, is to wait for Firefox 4 to ship, take
  that revision (the one we ship), make minimal changes to it (which will not
  land upstream, at least not instantly), *maybe* throw in NSPR or something,
  and tar it up. Sounds good to me.

> - run autoconf213 in js/src so that we ship a working "configure"
> - steps related to Makefile.ref, js.mak, etc elided
> - I have not updated the change log. Is this really necessary? Is there a
> non-manual way to do this? There were ... several ... changes made between
> 1.7.0 and 1.8.5

Skip the changelog. Point to http://hg.mozilla.org/tracemonkey if you think anybody cares.

> - Obviously, CVS tagging no longer applies. How about we tag the hg rev we
> push with bug 586016?

If you intend to base the source release off Firefox 4 final, that will already have a tag. There's no point tagging it more. The source release's README that explains what revision it's based on and what changes you made.

> - Tar now has one more top-level directory -- this is to allow us to ship
> things like NSPR and jemalloc later if we want to in the same tar ball
> - README moved to higher level dir

Great.

> Also.... Can I please, please change "libmozjs.so" to a better name in the tar
> release? How about libmozjs185.so.1.0.0? I think this would help the distro
> use-case, and make it possible to ship bug fixes on top of
> JS-1.8.5-the-language -- while properly advertising API/ABI compatibility. And,
> of course, making room for JS-1.8.6-the-language.

Wes clarified on IRC that he means a one-line change. The Makefile.in in the source distribution would have a different LIBRARY_NAME= than trunk. I think this is a good idea. It is definitely better than stopping right now to figure out an ideal versioning story.

Delivering something called "libmozjs185" makes it easy for us to adopt a completely different and superior versioning scheme later with no risk of breaking people who adopt this release.

(In reply to comment #5)
> - I think this is a great idea and I'm thrilled that Wes is taking the job.

Agreed, great job Wes.

> > - Tar now has one more top-level directory -- this is to allow us to ship
> > things like NSPR and jemalloc later if we want to in the same tar ball
> > - README moved to higher level dir

This is pretty non-standard for unix source packages, is it possible to put NSPR/jemalloc into the JS directory?

I think we want to stick with the directory structure current present in the hg repo.

Altering it for source releases means further modifications to the build system, and potential future-compatibilty gotchas; I think we really want to keep the source release story as simple as possible, and to make it as easy as possible for source-release users to become hg-clone users.

If you want to get technical about what's normal for a unix source package - NSPR and jemalloc would normally be completely separate packages, and at least NSPR probably will be distributed separately by Ubuntu et al. Jemalloc is less likely to be distributed separately, as it is radically different from upstream and has only one customer. Well, maybe two if XULRunner uses it.

Including NSPR with the shell is primarily to ease building - particularly for win32 - when JS_THREADSAFE goes away.

I'd just like to add that the CouchDB team would be extremely appreciative of a new source release. Once its released we'll move pretty quickly to pull in the downstream changes from Ubuntu and drop support for the 1.7 release.

I'd also like to specifically voice support for shipping the working configure script with the source release as that'd prevent us from requiring two versions of autoconf when building from a vcs checkout.

Jason, what do you think about taking bug 630209 (Igor's changes to JS_Compile*Script et al) for the the JS 1.8.5 source release?

It seems to be embedder-friendly, ready, and already landed in tracemonkey.

I have created an hg repo at https://bitbucket.org/wesgarland/js-1.8.5/ to track my work on this bug.

The repository currently contains SpiderMonkey based on Mozilla-2.0 rev 5f8f494a4c29 (FIREFOX_4_0rc1_BUILD1), minor standalone-oriented tweaks to Makefile.in, corresponding changes to the README, and the patch for bug 630209 discussed above.

I don't have a Windows platform on which to do a test build. Any volunteers?

http://www.page.ca/~wes/js-releng/js-1.8.5-rc1/js-1.8.5.tar.gz

As an embedder, this is pretty much what I'd want to see in a JS 1.8.5 source release. What do you think?

Works for me,
both "make" and "make install". Got a working shell in dist/bin, and the dll.

Thanks, Tom.

This means we have passed "yes, it compiles" tests on Windows, Leopard/x86, Debian Linux/x86 and Solaris 10/sparc.

> This means we have passed "yes, it compiles" tests on Windows, Leopard/x86,
> Debian Linux/x86 and Solaris 10/sparc.

I suggest pushing to TryServer. That would build a shell with lots of options on those platforms.

 - try server does not do shell-only builds AFAIK
 - try server expects repositories related (in the DAG sense) to mozilla-central

(Unless you know something I don't)

Wes

I've begun writing release notes at https://developer.mozilla.org/en/SpiderMonkey/1.8.5

Excellent, thanks. I'm going to push this to Ubuntu ASAP

I've actually packaged this in Ubuntu now, but I hit a couple of unexpected issues:

1) js-config --libs doesn't return the right linker flags (-lmozjs)

2) Even though it builds an ABI versioned so, the soname of the resulting ELF binary is still unversioned (ie, libmozjs185.so rather than libmozjs185.so.1.0). This would mean we couldn't support multiple ABI's, which would make it difficult to do an ABI transition in the future.

I have patches for both of these problems, although the patch for 2) is a bit hacky.

Created attachment 521592
Fix js-config --libs

This makes js-config --libs return the right linker flags

Created attachment 521593
Fix SONAME

This ensures that the resulting binary has the right soname (libmozjs185.so.1.0). tbh, I couldn't figure out a better way of doing this, so I'm not sure if anyone else has any better ideas

If you're doing official releases, you also want to address bug 554088 at some point.

(In reply to comment #26)
> Created attachment 521593 [details]
> Fix SONAME
>
> This ensures that the resulting binary has the right soname
> (libmozjs185.so.1.0). tbh, I couldn't figure out a better way of doing this, so
> I'm not sure if anyone else has any better ideas

Why does it need to be libmozjs185.so.something instead of libmozjs.so.something?

Chris -- argh, thanks for catching the js-config bug. I'll be cutting a new release to address that specific issue RSN.

Thanks also for your SONAME patch: that's something I don't have a lot of experience with.

Mike -- I named the library libmozjs185.so rather than libmozjs.so specifically because it doesn't work as a drop-in for any previously-released libmozjs.so (not even close -- see the release notes).

Versioning the spidermonkey library based on the JavaScript version it implements (1.8.5) doesn't make much sense to me; I prefer to think of this as version 1.0.0 of the library which implements JavaScript 1.8.5. Trying to connect solib versioning with language versioning has, at least in the past, been completely and utterly futile.

Thanks for pointing out the header bug, BTW. I agree that it should be fixed, and in fact tried to fix it in the past. Maybe now that I know my way around better I can try again.

(Apologies if this is bikeshedding or a repeat of previous discussion)

(In reply to comment #29)
> Mike -- I named the library libmozjs185.so rather than libmozjs.so specifically
> because it doesn't work as a drop-in for any previously-released libmozjs.so
> (not even close -- see the release notes).

Since we're deliberately changing the name, maybe libspidermonkey is a better name? I agree that putting 185 in the name isn't good. When we change to 1.9.0, would we have to change the SO name? That can't be right.

> Versioning the spidermonkey library based on the JavaScript version it
> implements (1.8.5) doesn't make much sense to me; I prefer to think of this as
> version 1.0.0 of the library which implements JavaScript 1.8.5. Trying to
> connect solib versioning with language versioning has, at least in the past,
> been completely and utterly futile.

I believe that the 1.8.5 numbering scheme is a pretty bad one, and is only there because of history. Perhaps we should change it; maybe to the Firefox version number or the Gecko one.

Created attachment 521641
add pkg-config file

(In reply to comment #30)
>
> Since we're deliberately changing the name, maybe libspidermonkey is a better
> name? I agree that putting 185 in the name isn't good. When we change to 1.9.0,
> would we have to change the SO name? That can't be right.

I don't think it matters a lot, honestly. I really would rather we ship. libspidermonkey does make more sense to me though if someone cares to rework the tree though, so we're *very* clearly distinguished from --enable-shared-js from mozilla-central.

> I believe that the 1.8.5 numbering scheme is a pretty bad one, and is only
> there because of history. Perhaps we should change it; maybe to the Firefox
> version number or the Gecko one.

So combining, you're proposing libspidermonkey-4.so ?

(In reply to comment #31)
> Created attachment 521641 [details]
> add pkg-config file

We probably don't care, but you can make the pkg-config file relative/relocatable by using the ${pcfiledir} expansion: see https://github.com/Flusspferd/flusspferd/blob/master/libflusspferd/flusspferd.pc.in (@ REL_PC_PATH_TO_ROOT@ is replaced in this case by CMake to be something like '/../..' or similar)

>
> > I believe that the 1.8.5 numbering scheme is a pretty bad one, and is only
> > there because of history. Perhaps we should change it; maybe to the Firefox
> > version number or the Gecko one.
>
> So combining, you're proposing libspidermonkey-4.so ?

As someone else mentioned, there is also the version numbers from the moz central tree - for instance FF4 was 2.0: http://hg.mozilla.org/releases/mozilla-2.0/tags

But i would also rather we ship with what we have now than get bogged down in deciding something else if there isn't a clear consensus.

Personally the 'JS version' has always seemed fairly arbitrary, and also I don't imagine there will change very fast - so using FF or moz versions would be far superior.

> (In reply to comment #30)
> (Apologies if this is bikeshedding or a repeat of previous discussion)

(Well-aimed kick to the shins)

I don't think anything has changed since comment 5. See the last three paragraphs there.

> Since we're deliberately changing the name, maybe libspidermonkey is a better
> name? I agree that putting 185 in the name isn't good. When we change to 1.9.0,
> would we have to change the SO name? That can't be right.

It isn't ideal, but it reflects reality. I would be thrilled to ship a SM with no stupid version number in its name. We should totally do that, as soon as have a sane upgrade story. First things first.

Fixing SM versioning was a non-goal for this release. Help us get it right in the next one.

(In reply to comment #32)
> I don't think it matters a lot, honestly. I really would rather we ship.

This, with a bullet.

GJS changes to support this release:
https://bugzilla.gnome.org/show_bug.cgi?id=646369

Interested in this aswell for OpenBSD, i was planning to do an update to our old spidermonkey 1.7.0 port from firefox 4 source tarball until i saw that bug report.

We officially published last night --

http://ftp.mozilla.org/pub/mozilla.org/js/js185-1.0.0.tar.gz
https://developer.mozilla.org/En/SpiderMonkey/1.8.5

(In reply to comment #23)
> Normal practice is to only ship generated files in the foo.tar.gz, not in
> revision control. In the same way that firefox-4.0.source.tar.bz2 includes
> configure, but mozilla-central doesn't.

We used to have it in the tree when we were in CVS, and it's only the more complex merge mechanics of hg that made us turn it off. We'd like to have it back, because it obviates the need to have autoconf-2.13 installed for the vast majority of developers.

@Wesley the symlinks are created in the wrong way.

lrwxrwxrwx root/root 0 2011-04-09 10:55 usr/lib/libmozjs185.so.1.0 -> /build/pkg//usr/lib/libmozjs185.so.1.0.0
lrwxrwxrwx root/root 0 2011-04-09 10:55 usr/lib/libmozjs185.so -> /build/pkg//usr/lib/libmozjs185.so.1.0
-rwxr-xr-x root/root 5694336 2011-04-09 10:55 usr/lib/libmozjs185-1.0.a
-rwxr-xr-x root/root 3499992 2011-04-09 10:55 usr/lib/libmozjs185.so.1.0.0

this happens mostly for those who wants to package it and DESTDIR is used and the resulted package doesn't have that target.

Ionut;

I think you did
# ./configure --prefix=/build/pkg/usr/
# make && make install

and then wanted to move/copy the links that were made, but they were absolute rather than relative links, so this did not work?

If I understand correctly, I can fix this in the next release, or perhaps a point release.

Wesley:

i did ./configure --prefix=/usr && make && make DESTDIR=/build/pkg install

in Makefile.in you have on line:

891 ln -s $(SHLIB_EXACT_VER) $(SHLIB_ABI_VER)
892 ln -s $(SHLIB_ABI_VER) $(SHLIB_ANY_VER)

where:

870 # Generic Unix / Linux
871 SHLIB_ANY_VER := $(DESTDIR)$(libdir)/$(SHARED_LIBRARY)
872 SHLIB_ABI_VER := $(DESTDIR)$(libdir)/$(SHARED_LIBRARY).$(SRCREL_ABI_VERSION)
873 SHLIB_EXACT_VER := $(DESTDIR)$(libdir)/$(SHARED_LIBRARY).$(SRCREL_VERSION)

Interesting, that is not a configuration I've tested but I will add it to my (very short) list for the next release.

In the meantime, I think you can change 891 and 892 of Makefile.in like this to achieve the desired outcome:

891 ln -s $(notdir $(SHLIB_EXACT_VER)) $(SHLIB_ABI_VER)
892 ln -s $(notdir $(SHLIB_ABI_VER)) $(SHLIB_ANY_VER)

This would give relative links instead of absolute links, which is the correct behaviour, right?

Wes

yes. that works fine

Thanks - I'll be sure to roll that into the next release.

i'm sorry for posting this issue in here but i don't know where to do it.

mongodb needs utf8 support in js and i had to add -DJS_C_STRINGS_ARE_UTF8 to CXXFLAGS. Can you enable this default or add an option in configure, like --enable-utf8 in the next release?

Ionut, you can get this added to SpiderMonkey for the next release by filing a bug for it in the Core/Javascript component (that same component as this). If you have a patch, mark it for review by setting r? to '<email address hidden>' and I can review it for you.

Ionut, -DJS_C_STRINGS_ARE_UTF8 is not the best way to access that feature, it is deprecated (but still functional) after JS 1.7.0.

Instead please invoke JS_SetCStringsAreUTF8() before your first call to JS_InitRuntime().

https://developer.mozilla.org/en/JS_CStringsAreUTF8

Wes

Sorry to revive an old thread, but I wanted to voice my thoughts for posterity. Right now the embedder's story is something like:

1) Look for js-config. If found, use it to find the sdkdir, then do feature detection to figure out what JSAPI version it is.
2) Otherwise, look for libmozjs185
3) ... or for libmozjs
4) ... or for libjs

I'm glad to see that the mozilla-central repository has not taken the versioning used here into the library name upstream. I encourage everyone involved in releases not to worry about getting the JS language version involved in the library naming at all. It may often be of much less importance to an embedder whether the language changed slightly as whether the embedding ABI changed, so please stick to ABI compatibility when deciding version numbering for the library and keep one library name. I suspect backward-incompatible language changes are less frequent than ABI changes and much of the JS world is used to feature-testing for language compatibility anyway due to the variety of language versions and features supported by browsers. Furthermore, embedders who _do_ care about the language version can look at the define in jsversion.h. Changing library names just complicates things needlessly.

Thanks for all your hard work.

(In reply to Randall Leeds from comment #49)
> Sorry to revive an old thread, but I wanted to voice my thoughts for
> posterity.

Thanks for the input. I don't think we're planning changing much right now, but at some point I think we'll want to do a new source release and it will be good to keep this in mind. I think the point about version numbering going with ABI rather than language features makes sense.