Add rubygems bin to PATH

Bug #145267 reported by Pete Deffendol
92
This bug affects 14 people
Affects Status Importance Assigned to Milestone
gems (Ubuntu)
Invalid
Undecided
Unassigned
Nominated for Gutsy by Martin Eisenhardt
Nominated for Hardy by Martin Eisenhardt
Nominated for Intrepid by Neil Wilson
libgems-ruby (Debian)
Fix Released
Unknown
libgems-ruby (Ubuntu)
Invalid
Low
Unassigned
Nominated for Gutsy by Martin Eisenhardt
Nominated for Hardy by Martin Eisenhardt
Nominated for Intrepid by Neil Wilson
ruby1.8 (Ubuntu)
Invalid
Undecided
Unassigned
Nominated for Gutsy by Martin Eisenhardt
Nominated for Hardy by Martin Eisenhardt
Nominated for Intrepid by Neil Wilson

Bug Description

Binary package hint: rubygems

The rubygems package places the binaries for installed gems into /var/lib/gems/1.8/bin, but this directory is not in the default PATH after the package is installed. It would be nice to add this when installing the package.

Revision history for this message
Rocco Stanzione (trappist) wrote :

I second the motion. The usefulness of the package is severely limited by this.

Changed in libgems-ruby:
importance: Undecided → Low
status: New → Confirmed
Changed in libgems-ruby:
status: Unknown → Won't Fix
Revision history for this message
Martin Eisenhardt (martin-eisenhardt) wrote :

I would like to leave a "me too" note here. Ruby is becoming increasingly important for web applications (Rails, merb, ramaze; you name it!) and administrative tasks (say: glue language), and lots of gems add really useful functionality. It is a *real* PITA that /var/lib/gems/1.8/bin is not in the default $PATH.

Please, fix this as soon as possible.

Changed in gems:
status: New → Invalid
Revision history for this message
chastell (chastell) wrote :

Note that Ruby 1.9 ships RubyGems bundled, so that’s another argument for somehow handling this issue; from 1.9 on, `sudo gem install X` that simply works will be almost crucial.

(Yes, I’m aware of the discussion at the Debian BTS and that this issue is not simple to fix in an uniform, shell-agnostic manner.)

Revision history for this message
Kevin DuBois (kdub) wrote :

perhaps it would be better to have the package install symlinks to directories already in $PATH or to modify the build the package so that it is installed to somewhere in $PATH

Revision history for this message
chastell (chastell) wrote : Re: [Bug 145267] Re: Add rubygems bin to PATH

Kevin DuBois:

> perhaps it would be better to have the package install symlinks to
> directories already in $PATH or to modify the build the package so
> that it is installed to somewhere in $PATH

The catch is, Debian/Ubuntu doesn’t control the files that can come in
the installed gems, so can’t assure there aren’t any clashes in their
filenames (I can easily create a gem that will ship a ‘cp’ binary
and upload it to RubyForge) – hence RubyGems can’t symlink files in
/var/lib/gems/1.8/bin from any dpkg-handled location.

(Now that I look at my $PATH, it seems /usr/local/bin is there by
default, so maybe symlinks from there would make it all work. Still,
adding /var/lib/gems/1.8/bin somehow to the $PATH seems the better
solution – although I know it’s not simple to implement.)

-- Shot
--
Like most computer techie people, I'll happily spend 6 hours trying to
figure out how to do a 3 hour job in 10 minutes. -- James Cort, asr

Revision history for this message
Raphael J. Schmid (raphael-j-schmid) wrote :

Another "me too" notice - it would be nice if the rubygems package actually became useful. I think most people installing it are even aware of possible problems/clashes with the distro's package management system.

Neil Wilson (neil-aldur)
Changed in ruby1.8:
status: New → Invalid
Revision history for this message
Neil Wilson (neil-aldur) wrote :

Right let's have a go at getting this closer to a fix.

My current suggestion is to stick a file in /etc/profile.d/rubygems1.8.sh that includes

PATH=$PATH:/var/lib/gems/1.8/bin

The issue with sudo is a "won't fix" scenario I feel. For those sort of gems, we need an 'apt' package building. (Yes and incorporating so that gem knows that they are there - coming soon if I have my way).

Let me know what you think of the solution and I'll roll a package for it.

Revision history for this message
Neil Wilson (neil-aldur) wrote :

I'll clarify that. The issue with sudo is that it sets its path to to a SECURE_PATH setting for security purposes. Nothing outside of those directories will be run by sudo by default. That is as it should be.

Unfortunately that means anything installed by 'gem' cannot be run directly by sudo without specifying the full path.

That is known as 'syntactic vinegar' and hopefully it will irritate somebody sufficiently to package the gem properly :-)

Revision history for this message
Lucas Nussbaum (lucas) wrote :

See #18808 about /etc/profile.d/ ...

Revision history for this message
Neil Wilson (neil-aldur) wrote :

Have you looked at the filesystem of an Ubuntu Hardy machine recently?
You might want to take a look.

2008/5/28 Lucas Nussbaum <email address hidden>:
> See #18808 about /etc/profile.d/ ...

--
Neil Wilson

Revision history for this message
Lucas Nussbaum (lucas) wrote :

On 28/05/08 at 12:42 -0000, Neil Wilson wrote:
> Have you looked at the filesystem of an Ubuntu Hardy machine recently?
> You might want to take a look.

base-files (4.0.1ubuntu2) hardy; urgency=low

  * Implement LSB-3.1, 16.2 (/etc/profile.d). Addresses LP #102105.
    According to Debian policy 9.9 (Environment variables), programs
    installed by packages must not depend on environment variables
    to get reasonable defaults. Do not use this LSB feature to set
    or modify environment variables.

 -- Matthias Klose <email address hidden> Wed, 06 Feb 2008 15:27:17 +0100
--
| Lucas Nussbaum
| <email address hidden> http://www.lucas-nussbaum.net/ |
| jabber: <email address hidden> GPG: 1024D/023B3F4F |

Revision history for this message
Neil Wilson (neil-aldur) wrote :

Now you're getting desperate. All packages depend upon the setting of
PATH to work at all. You can hardly leave it blank can you.

The alternative is to alter the package to allow gems to write the
binaries to /usr/bin where they belong.

Which policy would you prefer to change?

2008/5/28 Lucas Nussbaum <email address hidden>:
> On 28/05/08 at 12:42 -0000, Neil Wilson wrote:
>> Have you looked at the filesystem of an Ubuntu Hardy machine recently?
>> You might want to take a look.
>
> base-files (4.0.1ubuntu2) hardy; urgency=low
>
> * Implement LSB-3.1, 16.2 (/etc/profile.d). Addresses LP #102105.
> According to Debian policy 9.9 (Environment variables), programs
> installed by packages must not depend on environment variables
> to get reasonable defaults. Do not use this LSB feature to set
> or modify environment variables.
>
> -- Matthias Klose <email address hidden> Wed, 06 Feb 2008 15:27:17 +0100
> --
> | Lucas Nussbaum
> | <email address hidden> http://www.lucas-nussbaum.net/ |
> | jabber: <email address hidden> GPG: 1024D/023B3F4F |
>
> --
> Add rubygems bin to PATH
> https://bugs.launchpad.net/bugs/145267
> You received this bug notification because you are a direct subscriber
> of the bug.
>

--
Neil Wilson

Revision history for this message
Lucas Nussbaum (lucas) wrote :

On 28/05/08 at 13:38 -0000, Neil Wilson wrote:
> Now you're getting desperate. All packages depend upon the setting of
> PATH to work at all. You can hardly leave it blank can you.
>
> The alternative is to alter the package to allow gems to write the
> binaries to /usr/bin where they belong.

Installing to /usr/local/bin could sound acceptable to me. Not sure if
that will be acceptable for the other ruby maintainers in Debian.
--
| Lucas Nussbaum
| <email address hidden> http://www.lucas-nussbaum.net/ |
| jabber: <email address hidden> GPG: 1024D/023B3F4F |

Revision history for this message
Lucas Nussbaum (lucas) wrote :

On 28/05/08 at 13:38 -0000, Neil Wilson wrote:
> Now you're getting desperate.

No, but I'm getting annoyed by your tone in this whole discussion. I
already spent a lot of time on this rubygems stuff, despite not being
interested at all in rubygems (I ship my ruby libs as tarballs AND
gems, and I don't use rails).
--
| Lucas Nussbaum
| <email address hidden> http://www.lucas-nussbaum.net/ |
| jabber: <email address hidden> GPG: 1024D/023B3F4F |

Revision history for this message
Neil Wilson (neil-aldur) wrote :

2008/5/28 Lucas Nussbaum <email address hidden>:

> Installing to /usr/local/bin could sound acceptable to me. Not sure if
> that will be acceptable for the other ruby maintainers in Debian.

Could this be solved with the alternatives system I wonder. The gem
installations are after all alternatives to apt packages.

That way it would work with gem1.9 as well. Apt packages could
override with a higher priority.

--
Neil Wilson

Revision history for this message
Neil Wilson (neil-aldur) wrote :

2008/5/28 Lucas Nussbaum <email address hidden>:

> No, but I'm getting annoyed by your tone in this whole discussion. I

Well I'm sorry that you feel that way. I'm not here to upset anybody.
I'm here to get a problem that has been annoying me and probably about
9000 others for the last two to three years fixed once and for all.

However from where I'm sat, and the others who have diligently
reported this fault time and time again, there appears to be an
impasse that makes no sense to us. And that is what is making me
tetchy because I can see no need for it. Let's make gem and apt play
nicely together and everybody wins. Particularly you Lucas because
I've read the public background and I know you've put your heart into
it. I'm sure you want it fixed as much as I do.

So if you want to say 'no' to something, great, but please offer an
alternative so we can make progress towards a proposal that works for
everybody.

Now what do you think about using the alternatives system?

--
Neil Wilson

Revision history for this message
Rocco Stanzione (trappist) wrote :

Can we nix the personal attacks please. I don't know what policy, if any, is responsible for this, but as far as I can tell no other package installs files in /usr/local/bin, and I don't think they should. See the FHS:
http://www.pathname.com/fhs/pub/fhs-2.3.html#USRLOCALLOCALHIERARCHY
I figure there are 3 reasonable options, which could be checked against the packaging policies:
* Install executables in /usr/bin
* Add the gem bin path to $PATH
* Install symlinks in /usr/bin

I don't think the reasonable-results argument against modifying $PATH is a very strong one, but that's still not my favorite solution. The fewer places my shell has to look when I run a command, the happier I am. Rubygems are sort of a special case, where maybe the policies didn't anticipate certain needs. Maybe a policy exception needs to be made, or maybe a new policy.

Revision history for this message
Rocco Stanzione (trappist) wrote :

Now that's not a bad idea. I hadn't thought of using the alternatives system.

Revision history for this message
Lucas Nussbaum (lucas) wrote :

On 28/05/08 at 14:25 -0000, Neil Wilson wrote:
> 2008/5/28 Lucas Nussbaum <email address hidden>:
>
> > Installing to /usr/local/bin could sound acceptable to me. Not sure if
> > that will be acceptable for the other ruby maintainers in Debian.
>
> Could this be solved with the alternatives system I wonder. The gem
> installations are after all alternatives to apt packages.

That would require hacking rubygems quite deeply. If rubygems provided
some hooks that we could use to implement distro-specific stuff, why
not. But it's not the case, AFAIK.

It could bite us back, by breaking things in subtle ways.

What is the problem with installing to /usr/local/bin? It's in the
default system path, and it's before /usr/bin and /bin.

> That way it would work with gem1.9 as well. Apt packages could
> override with a higher priority.

If we install to /usr/local/bin, and you gem1.9 update --system, you
will get a new gem executable in /usr/local/bin, that will "override"
(by precedence in the path) Debian's.
--
| Lucas Nussbaum
| <email address hidden> http://www.lucas-nussbaum.net/ |
| jabber: <email address hidden> GPG: 1024D/023B3F4F |

Revision history for this message
Neil Wilson (neil-aldur) wrote :

OK.

So patch to gem1.8 to call update-alternatives with a lowish priority
whenever it fiddles with '/var/lib/gems/1.8/bin'.

Similarly to gem1.9.

Now what about the corresponding apt-package. Let's say we have
'mongrel' installed via gems with the alternatives system pointing
/usr/bin/mongrel_rails to the gem version. Then we install the apt
package and there is a name clash. If something has been packaged that
is also a ruby gem how do we do the naming for the binaries?

2008/5/28 Rocco Stanzione <email address hidden>:
> Now that's not a bad idea. I hadn't thought of using the alternatives
> system.
>
> --
> Add rubygems bin to PATH
> https://bugs.launchpad.net/bugs/145267
> You received this bug notification because you are a direct subscriber
> of the bug.
>

--
Neil Wilson

Revision history for this message
Lucas Nussbaum (lucas) wrote :

On 28/05/08 at 15:33 -0000, Neil Wilson wrote:
> OK.
>
> So patch to gem1.8 to call update-alternatives with a lowish priority
> whenever it fiddles with '/var/lib/gems/1.8/bin'.
>
> Similarly to gem1.9.
>
> Now what about the corresponding apt-package. Let's say we have
> 'mongrel' installed via gems with the alternatives system pointing
> /usr/bin/mongrel_rails to the gem version. Then we install the apt
> package and there is a name clash. If something has been packaged that
> is also a ruby gem how do we do the naming for the binaries?

For each Debian package where a gem also exists, you would have to
modify the Debian package to use the alternatives system. That clearly
doesn't work.
--
| Lucas Nussbaum
| <email address hidden> http://www.lucas-nussbaum.net/ |
| jabber: <email address hidden> GPG: 1024D/023B3F4F |

Revision history for this message
Lucas Nussbaum (lucas) wrote :

On 28/05/08 at 14:59 -0000, Rocco Stanzione wrote:
> Can we nix the personal attacks please. I don't know what policy, if any, is responsible for this, but as far as I can tell no other package installs files in /usr/local/bin, and I don't think they should. See the FHS:
> http://www.pathname.com/fhs/pub/fhs-2.3.html#USRLOCALLOCALHIERARCHY

Actually, you see it the wrong way. Gem (the package) should be
installed in /usr/bin. But files from gems, instead with the gem1.{8,9}
command, are not in any Debian package. And it's not reasonable at all
to install them in /usr/bin. (because, then, they could overwrite files
from Debian packages without warning).

> I figure there are 3 reasonable options, which could be checked against the packaging policies:
> * Install executables in /usr/bin

no, risks overwriting files from Debian packages.

> * Add the gem bin path to $PATH

forbidden by policy, not do-able in Debian

> * Install symlinks in /usr/bin

no, risks overwriting files from Debian packages.
--
| Lucas Nussbaum
| <email address hidden> http://www.lucas-nussbaum.net/ |
| jabber: <email address hidden> GPG: 1024D/023B3F4F |

Revision history for this message
Neil Wilson (neil-aldur) wrote :

On 28/05/2008, Lucas Nussbaum <email address hidden> wrote:
> That would require hacking rubygems quite deeply. If rubygems provided
> some hooks that we could use to implement distro-specific stuff, why
> not. But it's not the case, AFAIK.

Not really. It's a relatively straightforward program to follow and
Eric is quite helpful really. I don't perceive a big issue in getting
gem to issue the appropriate 'update-alternatives' command. To be
honest it could probably do with a pre/post hook system.

> What is the problem with installing to /usr/local/bin? It's in the
> default system path, and it's before /usr/bin and /bin.

That's one problem. gem installed systems would then override apt
installed systems and I'm not sure that is where we want to go.

> > That way it would work with gem1.9 as well. Apt packages could
> > override with a higher priority.
>
> If we install to /usr/local/bin, and you gem1.9 update --system, you
> will get a new gem executable in /usr/local/bin, that will "override"
> (by precedence in the path) Debian's.

You mean 'gem1.9 update' (gem1.9 update --system updates rubygems and
should be disabled)

Do we want that? If you install an apt package shouldn't that be the
one the system uses (from a stability point of view).

The problem I see is when gem1.8 and gem1.9 are on the same system. If
you install a gem in 1.8, then install it in 1.9 a remove in 1.8 will
remove the 1.9 binary in /usr/local/bin

--
Neil Wilson

Revision history for this message
Neil Wilson (neil-aldur) wrote :

On 28/05/2008, Lucas Nussbaum <email address hidden> wrote:
> For each Debian package where a gem also exists, you would have to
> modify the Debian package to use the alternatives system. That clearly
> doesn't work.

Why not?

Surely that should be part of the packaging wrapper for a package that
is also a rubygem and which generates executables. (which would
include putting the appropriate data files in the gem data area to
tell gem that the facility is installed so that gem dependencies work
correctly).

It's all about making the debian packages co-operate with gem. They
can co-operate through the alternatives system and avoid standing on
each others toes.

I would think that apt packages generated would be those where gem
normally compiles (like mongrel). In which case there is a dependency
on the ruby version and the binary should really be called
'mongrel_rails1.8' anyway.

I would suggest that gem installs its binaries in
/var/lib/gems/1.x/bin. Packaging creates packages for each version of
ruby (eg. mongrel1.8, mongrel1.9), installs the binaries as
/usr/bin/<whatever>1.x and alternatives is used to create
/usr/bin/<whatever>

--
Neil Wilson

Revision history for this message
Lucas Nussbaum (lucas) wrote :

On 28/05/08 at 19:01 -0000, Neil Wilson wrote:
> On 28/05/2008, Lucas Nussbaum <email address hidden> wrote:
> > That would require hacking rubygems quite deeply. If rubygems provided
> > some hooks that we could use to implement distro-specific stuff, why
> > not. But it's not the case, AFAIK.
>
> Not really. It's a relatively straightforward program to follow and
> Eric is quite helpful really. I don't perceive a big issue in getting
> gem to issue the appropriate 'update-alternatives' command. To be
> honest it could probably do with a pre/post hook system.

See my other mail on the topic: update-alternatives won't work, because
you would have to modify all Debian packages that might provide the same
binary as a gem.

> > What is the problem with installing to /usr/local/bin? It's in the
> > default system path, and it's before /usr/bin and /bin.
>
> That's one problem. gem installed systems would then override apt
> installed systems and I'm not sure that is where we want to go.

I think it is where we want to go: if an admin installs stuff manually,
that's probably because the things provided by the distro don't suit his
needs. So the distro stuff should be overriden by default.

> > > That way it would work with gem1.9 as well. Apt packages could
> > > override with a higher priority.
> >
> > If we install to /usr/local/bin, and you gem1.9 update --system, you
> > will get a new gem executable in /usr/local/bin, that will "override"
> > (by precedence in the path) Debian's.
>
> You mean 'gem1.9 update' (gem1.9 update --system updates rubygems and
> should be disabled)

no, I meant --system. By "That way it would work with gem1.9 as well.",
I thought you meant "gem1.9 could be overriden as well".

> Do we want that? If you install an apt package shouldn't that be the
> one the system uses (from a stability point of view).
>
> The problem I see is when gem1.8 and gem1.9 are on the same system. If
> you install a gem in 1.8, then install it in 1.9 a remove in 1.8 will
> remove the 1.9 binary in /usr/local/bin

By default, gem installs binaries to /usr/bin. The /var/lib/gems/.../bin
hack is Debian-specific, see the second part of
http://svn.debian.org/wsvn/pkg-ruby-extras/packages/libgems-ruby/trunk/debian/patches/01_default_gem_path.dpatch?op=file&rev=0&sc=0

So this problem (co-installation of gems with binaries for 1.8 and 1.9)
is a rubygems problem, not a Debian one.
--
| Lucas Nussbaum
| <email address hidden> http://www.lucas-nussbaum.net/ |
| jabber: <email address hidden> GPG: 1024D/023B3F4F |

Revision history for this message
Lucas Nussbaum (lucas) wrote :

On 28/05/08 at 19:21 -0000, Neil Wilson wrote:
> On 28/05/2008, Lucas Nussbaum <email address hidden> wrote:
> > For each Debian package where a gem also exists, you would have to
> > modify the Debian package to use the alternatives system. That clearly
> > doesn't work.
>
> Why not?

Take a given Debian package. How will you determine if this package
might be coinstalled with a gem providing the same binary at some point?

Will you just ask all packages providing ruby apps with executables to
switch to the alternatives system?

> Surely that should be part of the packaging wrapper for a package that
> is also a rubygem and which generates executables. (which would
> include putting the appropriate data files in the gem data area to
> tell gem that the facility is installed so that gem dependencies work
> correctly).

How do you deal with non-ruby packages having the same name as a binary
from a gem? Name clashes will happen for sure.

> It's all about making the debian packages co-operate with gem. They
> can co-operate through the alternatives system and avoid standing on
> each others toes.

That really doesn't seem like a possible solution to me. It's also too
complicated for users: most users don't know anything about the
alternatives system.
--
| Lucas Nussbaum
| <email address hidden> http://www.lucas-nussbaum.net/ |
| jabber: <email address hidden> GPG: 1024D/023B3F4F |

Revision history for this message
Victor Costan (costan) wrote :

Please do not disable "gem update --system". I use this to bypass the problem discussed above, and get gems on my server's path.

It's unlikely that people will issue this command "by accident". Most rails users will install gems using "rake gems:install" now that Rails 2.1.0 is out, and ruby people that use this command either know or should know what they're doing.

Thanks!

Revision history for this message
technomancy (technomancy) wrote :

>> I figure there are 3 reasonable options, which could be checked against the packaging policies:
>> * Install executables in /usr/bin
>
> no, risks overwriting files from Debian packages.

For what it's worth, the behaviour of the version of rubygems that comes with apt-get is so annoying that *everyone* I know ends up installing from source anyway. So this risk is there whether the packaged version of rubygems puts executables on users' $PATHs or not.If it doesn't, people will just use one that does.

The only difference is that currently if a gem overwrites a file the Debian/Ubuntu maintainers have the luxury of pointing fingers and saying, "hey, I told you not to do that."

Revision history for this message
mathew (meta23) wrote :

Given the continuing intransigence of the debian maintainers, I've been wondering if it isn't time for a bunch of Ruby folk who use debian/Ubuntu to put together an alternative Ruby release as a single .deb that actually works properly.

Revision history for this message
Lucas Nussbaum (lucas) wrote :

mathew, nobody is stopping you from starting to work in a PPA.

Revision history for this message
Neil Wilson (neil-aldur) wrote :

On 12/07/2008, mathew <email address hidden> wrote:
> Given the continuing intransigence of the debian maintainers, I've been
> wondering if it isn't time for a bunch of Ruby folk who use
> debian/Ubuntu to put together an alternative Ruby release as a single
> .deb that actually works properly.

That work has already started mathew. I work at Brightbox, which
you'll know as the UK Ruby on Rails host. We really need apt and gem
to play nicely together for our next generation of Rails services.

So I've kicked off the ~ubuntu-ruby and ~ubuntu-ruby-backports groups
on Launchpad to try and drive work into improving the Ruby and Rails
packaging systems (along with lots of other stuff to do with our
virtualisation offering). There are packages in the PPAs there -
mostly straight backports at present.

I've already talked to Eric Hodel who looks after Rubygems and he has
put installation hooks into edge Rubygems to allow us to call out pre-
and post- install if we want.

If there is community interest in helping us improve Rubygems (ie
you're going to do some coding/testing/documenting, not just cheer on
from the sidelines) then I'll upload the current work to github and we
can take it from there.

--
Neil Wilson

Revision history for this message
Neil Wilson (neil-aldur) wrote :

OK We now have code for a proposed fix for rubygems.

I've implemented an 'update-alternatives' system that creates a link to gem binaries in /usr/local/bin. It arbitrates correctly between ruby1.8 and ruby1.9 gem installations.

At the moment the first gem installed wins, ie if you install capistrano using gem1.8 and then with gem1.9, 'cap' will still call the ruby1.8 version. Please let me know if this behaviour is acceptable.

I've used the new 'operating_system.rb' override facilities in edge Rubygems to create the mechanism. I've also taken the opportunity to fix a few things I find annoying. Again I'd appreciate it if you could let me know if this works for you.

* rdoc and ri documentation is *not* installed by default. You have to ask for them.
* Executables are *always* removed on uninstall. I have no idea at all why you'd want them to remain to create an error. If you know tell me.
* 'update -system' now calls 'apt-get install rubygems{version}' automatically.

Patches incorporated upstream have been removed, and new patches for new problems added :-)

You can get the packages from the ubuntu-ruby group archive (https://edge.launchpad.net/~ubuntu-ruby/+archive). There are intrepid packages and Hardy backports. (Be warned that the hardy backports will pull in ruby1.8.7 and ruby1.9.0-2).

Code is on Github. http://github.com/NeilW/deb-rubygems/tree/master

Have a play. Let me know what you think. I'm off for a week's break now, but I'll pick up comments when I get back.

Revision history for this message
Mathias Gug (mathiaz) wrote :

I had a look at your implementation and it looks interesting. However I wonder if the usage of the update-alternative system is necessary as it brings additional complexity (as already pointed by Lucas above).

Another option would be to simply force symlink binaries from /var/lib/ruby1.*/bin/ to /usr/local/bin/, that way the last installed gem always wins.

The main point being that the gem command should installed everything in /usr/local/ - it can be considered as a program installed by the local administrator. It should not touch anything in /usr (which is reserved for dpkg).

So rather than trying to figure out ways to make dpkg and gem know about each other, it seems easier to make sure that they won't interfere (dpkg works in /usr, gem in /usr/local).

If you think that update-alternatives is still worth using, I'd suggest you use the --altdir and --admindir options, so that you don't step on the update-alternatives system configuration (using /usr/local/etc/gems/alternatives and /usr/local/lib/gems/alternatives respectively). Going down that road, it may also be useful to extend the gem command to provide a way to switch from one alternative to the other so the end user won't have to learn about the update-alternative system.

Revision history for this message
Neil Wilson (neil-aldur) wrote :

Linking to /usr/local/bin sounds appealling until you do the following rather reasonable sequence of events.

Install 1.8 Gem
Install 1.9 Gem
Uninstall 1.8 Gem.

Result is no link in /usr/local/bin and the user installing gem from source again 'cos those packages don't work properly'.

The choice to install alternatives in /usr/local/bin is precisely so dpkg doesn't have to get involved in the alternatives process and the gem installed ones override the dpkg installed ones - as the user would expect.

However that doesn't get away from the problem of dpkg packages failing to inform gem that they exist so that gem pulls (and compiles!) gems it doesn't really need. That discussion is probably a separate bug.

By default gem always reinstalls even if the gem is up to date, so a simple 'last gem installed wins' solution using alternatives to deal with the /usr/local/bin clash issue is probably the simplest solution that will work.

I will cut some code that implements this solution (including operating the separate gem alternatives system area) and we'll take it from there.

Revision history for this message
Mathias Gug (mathiaz) wrote :

On Fri, Aug 01, 2008 at 08:23:02AM -0000, Neil Wilson wrote:
> Linking to /usr/local/bin sounds appealling until you do the following
> rather reasonable sequence of events.
>
> Install 1.8 Gem
> Install 1.9 Gem
> Uninstall 1.8 Gem.
>
> Result is no link in /usr/local/bin and the user installing gem from
> source again 'cos those packages don't work properly'.

I'm not sure I understand what you're referring to with 'packages' -
the rubygem package ?

What does the "gem from source" do ? Install in /usr/bin/ ? In
/var/lib/gems/ruby1.{8,9}/bin ? How does "gem from source" handle the
scenario you've outlined above ?

> However that doesn't get away from the problem of dpkg packages failing
> to inform gem that they exist so that gem pulls (and compiles!) gems it
> doesn't really need. That discussion is probably a separate bug.

Agreed. Feel free to open a new bug.

--
Mathias Gug
Ubuntu Developer http://www.ubuntu.com

Revision history for this message
Neil Wilson (neil-aldur) wrote :

2008/8/1 Mathias Gug <email address hidden>:
> I'm not sure I understand what you're referring to with 'packages' -
> the rubygem package ?

The Debian/Ubuntu dpkg package.

> What does the "gem from source" do ? Install in /usr/bin/ ? In
> /var/lib/gems/ruby1.{8,9}/bin ? How does "gem from source" handle the
> scenario you've outlined above ?

Source gem installs executable wrappers (essentially programmatic
symbolic links that select the correct gem executable) directly into
/usr/bin and stores gem libraries in the main ruby library area.
(/usr/lib/ruby/gems/1.8)

The way it deals with the version clash is that by default 'uninstall'
doesn't remove the executable wrapper.

That leads to the alternative bug of the program appearing to exist in
/usr/bin but but being unable to find the appropriate gem library
after you have done:

gem1.8 install
gem1.8 uninstall

So after a while your system ends up with a load of garbage wrappers
on the disk.

--
Neil Wilson

Neil Wilson (neil-aldur)
Changed in libgems-ruby:
assignee: nobody → neil-aldur
Revision history for this message
Neil Wilson (neil-aldur) wrote :

New version in https://edge.launchpad.net/~ubuntu-ruby/+archive

This version uses alternatives in the background to handle the links in /usr/local/bin using the separate admin dirs suggested earlier (although I use /var/lib/gems/alternatives rather than /usr/local/lib/gems/alternatives to keep things in the right place).

The user should never know that alternatives is operating the gem system. It should just work.

- Last gem wins. The last gem installed will always be the one available.
- Links are always cleaned up whatever order you install/uninstall things.
- It's ready for the other ruby interpreters that are around the corner - Jruby and Rubinus.
- If you use the gem '-V' verbose option then you can see what alternatives is doing.

I've also implemented a rubygems package that follows the mechanisms used for the ruby package, ie it installs rubygems1.8. That means that other packages (eg passenger) can just depend upon the default version of gems as they can ruby.

I think this is now the release candidate. Feedback please.

Revision history for this message
Lucas Nussbaum (lucas) wrote :

Some notes in random order:

I agree that installing the binary wrappers, or symlinks, to /usr/local/bin, so that they are in $PATH, is a good thing.

The install / install / uninstall problem you mention is a gem problem. I think that it should be solved at the rubygem side, not specifically for Debian. That's over-engineered. Have you talked to the gems developers about that? Maybe you could implement a solution directly in rubygems.

> I've also implemented a rubygems package that follows the mechanisms used for the ruby package, ie it installs
> rubygems1.8. That means that other packages (eg passenger) can just depend upon the default version of gems
> as they can ruby.

Please check what has been done in Debian recently with the rubygems and ruby1.9 packages. How rubygems is managed changed a bit. See source packages: libgems-ruby >= 1.2.0-1 and ruby1.9 >= 1.9.0.1-5. (ie, the versions in intrepid, not hardy).

Neil Wilson (neil-aldur)
Changed in libgems-ruby:
status: Confirmed → In Progress
Revision history for this message
Neil Wilson (neil-aldur) wrote :

2008/8/4 Lucas Nussbaum <email address hidden>:
> Some notes in random order:
> The install / install / uninstall problem you mention is a gem problem.
> I think that it should be solved at the rubygem side, not specifically
> for Debian. That's over-engineered. Have you talked to the gems
> developers about that? Maybe you could implement a solution directly in
> rubygems.

To do that would essentially require duplicating much of the
alternatives system within rubygems. Feel free to code that up if you
have a few weeks or months available.

So I can have a 91 line fix in the packaging and hit Intrepid or
several hundred within rubygems which nobody seems that keen on
writing and get nowhere.

The gem installation problem is fixed within Debian Policy, the system
is kept clean of dangling links and broken Gems and from the user's
perspective they just see a system that works.

When rubygems finally get around to implementing something that stops
gem1.8 and gem1.9 running into each other then we can delete the two
small procedures that implement the system.

It's just a patch, Lucas, to make Gems work better in a packaging
environment. It will allow people to switch ruby interpreters with
greater ease. With good packaging people can do that, and that will
give them a reason to use the packages.

> Please check what has been done in Debian recently with the rubygems and
> ruby1.9 packages. How rubygems is managed changed a bit. See source
> packages: libgems-ruby >= 1.2.0-1 and ruby1.9 >= 1.9.0.1-5. (ie, the
> versions in intrepid, not hardy).

If you looked at the package you'd notice that the code is based upon
the latest Debian package in Intrepid (which is missing a default
rubygems package BTW). Using the new 'operating_system.rb' override
facility simplifies the package immensely by getting all the policy
defaults in one place as well as allowing the user home area gems
facility to work that is currently crippled by the Debian packaging.

The user home area gem system is used by the 'gems:install' task
within Rails 2.1

--
Neil Wilson

Revision history for this message
Lucas Nussbaum (lucas) wrote :

On 04/08/08 at 21:02 -0000, Neil Wilson wrote:
> 2008/8/4 Lucas Nussbaum <email address hidden>:
> > Some notes in random order:
> > The install / install / uninstall problem you mention is a gem problem.
> > I think that it should be solved at the rubygem side, not specifically
> > for Debian. That's over-engineered. Have you talked to the gems
> > developers about that? Maybe you could implement a solution directly in
> > rubygems.
>
> To do that would essentially require duplicating much of the
> alternatives system within rubygems. Feel free to code that up if you
> have a few weeks or months available.
>
> So I can have a 91 line fix in the packaging and hit Intrepid or
> several hundred within rubygems which nobody seems that keen on
> writing and get nowhere.
>
> The gem installation problem is fixed within Debian Policy, the system
> is kept clean of dangling links and broken Gems and from the user's
> perspective they just see a system that works.
>
> When rubygems finally get around to implementing something that stops
> gem1.8 and gem1.9 running into each other then we can delete the two
> small procedures that implement the system.
>
> It's just a patch, Lucas, to make Gems work better in a packaging
> environment. It will allow people to switch ruby interpreters with
> greater ease. With good packaging people can do that, and that will
> give them a reason to use the packages.

No, it's a patch that makes rubygems work better on systems with
update-alternatives, while you should aim at a global solution instead.

I won't be the one making the final decision on this, but for this patch
to be added to the package, I would either want:
(a) that the patch is very, very small
(b) or that the patch is going to be integrated upstream in the near future

> > Please check what has been done in Debian recently with the rubygems and
> > ruby1.9 packages. How rubygems is managed changed a bit. See source
> > packages: libgems-ruby >= 1.2.0-1 and ruby1.9 >= 1.9.0.1-5. (ie, the
> > versions in intrepid, not hardy).
>
> If you looked at the package you'd notice that the code is based upon
> the latest Debian package in Intrepid (which is missing a default
> rubygems package BTW).

What do you mean with "default rubygems package"?

> Using the new 'operating_system.rb' override
> facility simplifies the package immensely by getting all the policy
> defaults in one place as well as allowing the user home area gems
> facility to work that is currently crippled by the Debian packaging.

If you could avoid words like "crippled" in this discussion, it would
help *a lot* and wouldn't make me consider unsubscribing from this bug.
--
| Lucas Nussbaum
| <email address hidden> http://www.lucas-nussbaum.net/ |
| jabber: <email address hidden> GPG: 1024D/023B3F4F |

Revision history for this message
Neil Wilson (neil-aldur) wrote :

2008/8/4 Lucas Nussbaum <email address hidden>:

> No, it's a patch that makes rubygems work better on systems with
> update-alternatives, while you should aim at a global solution instead.

No I aim for the simplest solution that will solve the most pain in
the shortest possible time. Others can then generalise that if they
want. My time is paid for don't forget.

From an Ubuntu point of view a superior package is good, because it
gives people a reason why they should use Ubuntu and switch to the
package rather than continue to mess around with the source package as
they do now.

Certainly I'm not going to get my 200 odd customers to move away from
source installation without a nice fat carrot to offer them.

> I won't be the one making the final decision on this, but for this patch
> to be added to the package, I would either want:
> (a) that the patch is very, very small
> (b) or that the patch is going to be integrated upstream in the near future

Technically this isn't a patch. It is using the upstream designed
interface to allow an operating system to add aspects to operations.
That makes it much more stable over time.

I'm asking upstream for their view on the binary clash problem. It's
only going to get worse as more interpreters come on line. But it
ain't going to happen quickly (certainly not by Augst 28) and in the
meantime you have unhappy users of Debian/Ubuntu who would be made
happy by what we have here - today.

> What do you mean with "default rubygems package"?

One like the package 'ruby' that installs the current 'system default' version.

All I have is a rubygems package that depends upon rubygems1.8 and
ruby so that I get the commands 'gem' and 'ruby' with the default 1.8
versions of each.

The user gem mechanism is broken by the Debian packaging and that
stops Rails 2.1 using it as it expects.

--
Neil Wilson

Revision history for this message
Lucas Nussbaum (lucas) wrote :

On 05/08/08 at 07:40 -0000, Neil Wilson wrote:
> The user gem mechanism is broken by the Debian packaging and that
> stops Rails 2.1 using it as it expects.

Bug # ?
--
| Lucas Nussbaum
| <email address hidden> http://www.lucas-nussbaum.net/ |
| jabber: <email address hidden> GPG: 1024D/023B3F4F |

Revision history for this message
Mathias Gug (mathiaz) wrote :

I've reviewed the libgem-ruby package from the ubuntu-ruby ppa
(libgems-ruby_1.2.0+2008072001-0ubuntu1~bbox4). Here are my comments:

On Mon, Aug 04, 2008 at 11:43:26AM -0000, Neil Wilson wrote:
> New version in https://edge.launchpad.net/~ubuntu-ruby/+archive
>
> - It's ready for the other ruby interpreters that are around the corner - Jruby and Rubinus.

How so ?

> I think this is now the release candidate. Feedback please.
>

===
You've switched to cdbs simplepatch system and it seems you've removed a
  couple of patches in the process:

 01_default_gem_path.dpatch
 03_disable_update_system.dpatch
 08_tighter_search_regex.dpatch
 21_avoid_ioseek.dpatch

Are these no longer required ?

===
You should merge the latest version of debian (-2) as your package is
currently uninstallable on intrepid.

===
While testing the package I came across the usage of gem libraries:
according to the rubygems documentation, you need to do some
post-install work in order to setup ruby gems correctly. I was wondering
if the ruby libraries could be symlinked to
/usr/local/lib/site_ruby/RUBY_VERSION/ when a gem is installed ? That
way you wouldn't have to modify your environment or call ruby with the
-rubygems option. Since site_ruby is already versioned
update-alternatives is not needed in that case.

===
I've also come across the following situation:

 $ sudo gem1.8 install rails
 $ sudo gem1.9 install rails
 $ sudo gem1.8 install rails

However /usr/local/bin/rake is still using ruby1.9. So gems dependencies
are not switched to the ruby version used by the installed gem. Is this
a valid use case ? How should it be handled ? Could the slave option of
update-alternatives be used to handle binaries from dependencies ?

--
Mathias Gug
Ubuntu Developer http://www.ubuntu.com

Revision history for this message
Neil Wilson (neil-aldur) wrote :

2008/8/12 Mathias Gug <email address hidden>:
> How so ?

In that the alternatives system will support and switch between
another two sets of binary wrappers once we write the packages.

> Are these no longer required ?

08 exists. All the other patches have been superceded. I've put the
details in the changelog.

>
> ===
> You should merge the latest version of debian (-2) as your package is
> currently uninstallable on intrepid.

Done.

> While testing the package I came across the usage of gem libraries:
> according to the rubygems documentation, you need to do some
> post-install work in order to setup ruby gems correctly. I was wondering
> if the ruby libraries could be symlinked to
> /usr/local/lib/site_ruby/RUBY_VERSION/ when a gem is installed ? That
> way you wouldn't have to modify your environment or call ruby with the
> -rubygems option. Since site_ruby is already versioned
> update-alternatives is not needed in that case.

The rubygems documentation is somewhat out of date. The way you use
libraries in a rubygem is simply to do

require 'library'

or if you want a particular version

gem 'gemname', >=2.1.0
require 'library'

Rubygems monkey-patches the ruby require system so that you get the
correct gem library automagically. It even works most of the time.

Have you an example that doesn't work like that?

> I've also come across the following situation:
>
> $ sudo gem1.8 install rails
> $ sudo gem1.9 install rails
> $ sudo gem1.8 install rails
>
> However /usr/local/bin/rake is still using ruby1.9. So gems dependencies
> are not switched to the ruby version used by the installed gem. Is this
> a valid use case ? How should it be handled ? Could the slave option of
> update-alternatives be used to handle binaries from dependencies ?

That's either a bug or a feature depending upon your viewpoint and is
merely a function of the way gem does reinstalls and handles
versioning (ie it reinstalls the requested gem, but not its
dependencies). If you do that with a source installed gem system, you
would get precisely the same result.

For me its an edge case with no easy win (the list of binaries of all
the dependencies is not readily available in the program).

Gem's dependency mechanism is primitive and it does get itself in a
mess if you do anything wildly unorthodox - like uninstall or
reinstall things :-). For this cycle I'd be happy if gem puts its
binaries on the system path and doesn't leave junk lying around or
break things when uninstalling stuff.

Do you consider this a show stopper?

I'll get a new version of the package into the PPA just as soon as
Intrepid lets me build one!

--
Neil Wilson

Revision history for this message
Neil Wilson (neil-aldur) wrote :

New version with latest upstream fixes in https://edge.launchpad.net/~ubuntu-ruby/+archive

Revision history for this message
Mathias Gug (mathiaz) wrote :
Download full text (3.2 KiB)

On Wed, Aug 13, 2008 at 09:20:54AM -0000, Neil Wilson wrote:
> > While testing the package I came across the usage of gem libraries:
> > according to the rubygems documentation, you need to do some
> > post-install work in order to setup ruby gems correctly. I was wondering
> > if the ruby libraries could be symlinked to
> > /usr/local/lib/site_ruby/RUBY_VERSION/ when a gem is installed ? That
> > way you wouldn't have to modify your environment or call ruby with the
> > -rubygems option. Since site_ruby is already versioned
> > update-alternatives is not needed in that case.
>
> The rubygems documentation is somewhat out of date. The way you use
> libraries in a rubygem is simply to do
>
> require 'library'
>
> or if you want a particular version
>
> gem 'gemname', >=2.1.0
> require 'library'
>
> Rubygems monkey-patches the ruby require system so that you get the
> correct gem library automagically. It even works most of the time.
>
> Have you an example that doesn't work like that?

I was using the progressbar example from the rubygems documentation[1]:

The command 'ruby test.rb' fails with a "no such file to load -
progressbar" error message.

The command 'ruby -rubygems test.rb' works as expected.

If I symlink /var/lib/gems/1.8/gems/progressbar-0.0.3/lib/progress.rb in
/usr/local/lib/site_ruby/1.8/, the command 'ruby test.rb' works as
expected.

[1]: http://rubygems.org/read/chapter/4#page16

>
> > I've also come across the following situation:
> >
> > $ sudo gem1.8 install rails
> > $ sudo gem1.9 install rails
> > $ sudo gem1.8 install rails
> >
> > However /usr/local/bin/rake is still using ruby1.9. So gems dependencies
> > are not switched to the ruby version used by the installed gem. Is this
> > a valid use case ? How should it be handled ? Could the slave option of
> > update-alternatives be used to handle binaries from dependencies ?
>
> That's either a bug or a feature depending upon your viewpoint and is
> merely a function of the way gem does reinstalls and handles
> versioning (ie it reinstalls the requested gem, but not its
> dependencies). If you do that with a source installed gem system, you
> would get precisely the same result.
>
> For me its an edge case with no easy win (the list of binaries of all
> the dependencies is not readily available in the program).
>
> Gem's dependency mechanism is primitive and it does get itself in a
> mess if you do anything wildly unorthodox - like uninstall or
> reinstall things :-). For this cycle I'd be happy if gem puts its
> binaries on the system path and doesn't leave junk lying around or
> break things when uninstalling stuff.
>
> Do you consider this a show stopper?
>

So it seems that upstream gem would behave the same way as your
proposal. I don't think it's a show stopper then.

Other comments on the diff between 1.2.0-2 and
1.2.0+2008081301-0ubuntu1~bbox1:

* debian/control:

 You've modified the build-dependencies - you're depending on rdoc
 rather than an explicit version of it. I think keeping the
 dependencies as close as possible to the ones in debian would help.

  ruby-pkg-tools has been dropped - why ?

  rubygems depends on rubygems1.8 *and* ruby. ...

Read more...

Revision history for this message
Lucas Nussbaum (lucas) wrote :

Hi,

Some comments:
- debian/operating_system.rb is not properly licensed, and not mentioned in debian/copyright.
- I'm still not convinced by your update-alternatives hack. This should *really* go upstream, so it's fixed for every distro, not just Ubuntu.
- why the switch to simple-patchsys?
- you base your version on a git snapshot, with a >5kloc diff compared to the current version in debian unstable. Is that really reasonable, since we are far in the Ubuntu release cycle AFAIK?
- have you talked to Daigo Moriwaki about those deep changes to his Debian package? If not, when do you plan to?
- You never answered by question about a bug# in https://bugs.launchpad.net/ubuntu/+source/libgems-ruby/+bug/145267/comments/42 .
- What's the Ubuntu policy about hosting VCS branches for packages? Shouldn't you use bzr on launchpad instead?

Most importantly, your motivations sound unclear to me:
> > No, it's a patch that makes rubygems work better on systems with
> > update-alternatives, while you should aim at a global solution instead.
>
> No I aim for the simplest solution that will solve the most pain in
> the shortest possible time. Others can then generalise that if they
> want. My time is paid for don't forget.
>
> From an Ubuntu point of view a superior package is good, because it
> gives people a reason why they should use Ubuntu and switch to the
> package rather than continue to mess around with the source package as
> they do now.
>
> Certainly I'm not going to get my 200 odd customers to move away from
> source installation without a nice fat carrot to offer them.

If I understand it correctly, you want to give Ubuntu a competitive advantage by not working with upstream to address this problem globally. That doesn't sound right.

In conclusion, with my Debian ruby maintainer and pkg-ruby-extras member hats, I still don't think this is the way to go. But you are free to ignore my comments of course, as you did in the past.

Revision history for this message
Mathias Gug (mathiaz) wrote :
Download full text (3.5 KiB)

Hi,

On Thu, Aug 14, 2008 at 05:48:34AM -0000, Lucas Nussbaum wrote:
> Some comments:
> - debian/operating_system.rb is not properly licensed, and not mentioned in debian/copyright.

debian/operating_system.rb has the following statement:

# Licensed unded the GPL. See /usr/share/common-licenses/GPL

Isn't that enough ? Adding a mention to the debian/copyright file would
be advisable though.

> - I'm still not convinced by your update-alternatives hack. This should *really* go upstream, so it's fixed for every distro, not just Ubuntu.

Agreed. The gem system has currently shortcomings. The
update-alternatives proposal is one way to address the issue. Upstream
seems cooperative on that point as it provided the necessary hooks to
implement such a system. So upstream is aware of the problem. They may
work on solving it and it can take some time.

On the other hand this proposal is one step toward fixing the issue, and
it works now. Once upstream comes up with a good solution, we can
revisit the usage of update-alternatives to manage gems binaries in
/usr/local/bin.

> - you base your version on a git snapshot, with a >5kloc diff compared to the current version in debian unstable. Is that really reasonable, since we are far in the Ubuntu release cycle AFAIK?

IMO this is perfectly acceptable as we're not past FeatureFreeze. That
means new upstream version can still be uploaded to the archive.
Moreover this is code that is coming from upstream, so it will be
available in the next upstream release.

> - have you talked to Daigo Moriwaki about those deep changes to his Debian package? If not, when do you plan to?

Talking to Daigo Moriwaki would be helpful. However considering the
current freeze for Lenny I doubt that this patch will be accepted in
Debian before Ubuntu enters FeatureFreeze.

> - What's the Ubuntu policy about hosting VCS branches for packages? Shouldn't you use bzr on launchpad instead?

There isn't a real policy. People have the choice of VCS and where to
host their code repository. It's true that most of the Ubuntu Developers
will use bzr, but some packages are maintained in git for example
(postfix for example).

>
> Most importantly, your motivations sound unclear to me:
> > > No, it's a patch that makes rubygems work better on systems with
> > > update-alternatives, while you should aim at a global solution instead.
> >
> > No I aim for the simplest solution that will solve the most pain in
> > the shortest possible time. Others can then generalise that if they
> > want. My time is paid for don't forget.
> >
> > From an Ubuntu point of view a superior package is good, because it
> > gives people a reason why they should use Ubuntu and switch to the
> > package rather than continue to mess around with the source package as
> > they do now.
> >
> > Certainly I'm not going to get my 200 odd customers to move away from
> > source installation without a nice fat carrot to offer them.
>
> If I understand it correctly, you want to give Ubuntu a competitive
> advantage by not working with upstream to address this problem globally.
> That doesn't sound right.
>

We're trying to improve the usage of gems so that the end user ha...

Read more...

Revision history for this message
Neil Wilson (neil-aldur) wrote :

2008/8/14 Lucas Nussbaum <email address hidden>:
> Hi,
>
> Some comments:
> - debian/operating_system.rb is not properly licensed, and not mentioned in debian/copyright.

Agreed. That needs some tidying up. I can't see any copyright message
in there about the debian packaging either. Does Debian need that as
well?

> - I'm still not convinced by your update-alternatives hack. This should *really* go upstream, so it's fixed for every distro, not just Ubuntu.

It will do Lucas, but that won't happen in the week I have before
feature freeze. Please consider this a pragmatic prototype.

> - why the switch to simple-patchsys?

That's easy - it's simpler :-). cdbs-edit-patch is the bees knees. No
longer do I hate patching.

> - you base your version on a git snapshot, with a >5kloc diff compared to the current version in debian unstable. Is that really reasonable, since we are far in the Ubuntu release cycle AFAIK?

I have it on good authority that the version in Debian may very well
be out of date by the end of the month. ;-)

> - have you talked to Daigo Moriwaki about those deep changes to his Debian package? If not, when do you plan to?

At the moment Debian bug #403407 is still marked "won't fix" so there
is clearly going to be a divergence whatever happens. You really need
to alter the bug status if Debian is serious about fixing the path
problem.

I'd be more than happy to sync up with the excellent work Daigo has
done on his package, but they're never going to be the same until
Debian accepts that the problem is fixable.

Can you do something about that?

> - You never answered by question about a bug# in https://bugs.launchpad.net/ubuntu/+source/libgems-ruby/+bug/145267/comments/42 .

Could you help and file one? Otherwise Debian will have to wait. I'm
kinda busy with a FeatureFreeze deadline and my real job.

> If I understand it correctly, you want to give Ubuntu a competitive
> advantage by not working with upstream to address this problem globally.
> That doesn't sound right.

Yes I suppose you probably see it that way. I can't help you with that
and I don't understand why you would want to adopt such a viewpoint.

I'll let the rubygems commit logs, bug tracker and mailing list speak
for whether I work with upstream or not.

--
Neil Wilson

Revision history for this message
Neil Wilson (neil-aldur) wrote :

2008/8/14 Mathias Gug <email address hidden>:
> I was using the progressbar example from the rubygems documentation[1]:
>
> The command 'ruby test.rb' fails with a "no such file to load -
> progressbar" error message.

I'm betraying my Rails heritage here and giving you duff instructions.

On Ruby1.8 with a pure ruby application you always have to start a program with

require 'rubygems'

to get the 'require' monkey patching in place so you can use gem
libraries. In ruby1.9 that is done automatically.

(f you work in the Rails framework then Rails does that for you).

So you don't need the symlinks and rubygem1.8 users wouldn't expect
them to be there.

> Other comments on the diff between 1.2.0-2 and
> 1.2.0+2008081301-0ubuntu1~bbox1:
>
> * debian/control:
>
> You've modified the build-dependencies - you're depending on rdoc
> rather than an explicit version of it. I think keeping the
> dependencies as close as possible to the ones in debian would help.

Isn't it a packaging bug? There is no need for a specific version so
it should depend upon the default package. I understood that to be
Ruby policy.

> ruby-pkg-tools has been dropped - why ?

Not used.

> rubygems depends on rubygems1.8 *and* ruby. The dependency on ruby
> seems redundant.

It does until you have a clean machine and do 'apt-get install
rubygems' at which point you would have access to 'gem', but not
'ruby'. '/usr/bin/ruby' is controlled by the ruby package.

If I've installed rubygems without a suffix I sort of expect to be
able to use ruby without a suffix.

Similarly if I've install rubygems1.8 I'd sort of expect ruby to be
called ruby1.8 - hence why the dependency is here and not in the
rubygems1.8 package.

--
Neil Wilson

Revision history for this message
Lucas Nussbaum (lucas) wrote :
Download full text (3.2 KiB)

On 14/08/08 at 07:05 -0000, Mathias Gug wrote:
> Hi,
>
> On Thu, Aug 14, 2008 at 05:48:34AM -0000, Lucas Nussbaum wrote:
> > Some comments:
> > - debian/operating_system.rb is not properly licensed, and not mentioned in debian/copyright.
>
> debian/operating_system.rb has the following statement:
>
> # Licensed unded the GPL. See /usr/share/common-licenses/GPL
>
> Isn't that enough ? Adding a mention to the debian/copyright file would
> be advisable though.

Of course not. Which version of the GPL?

> > - I'm still not convinced by your update-alternatives hack. This
> should *really* go upstream, so it's fixed for every distro, not just
> Ubuntu.
>
> Agreed. The gem system has currently shortcomings. The
> update-alternatives proposal is one way to address the issue. Upstream
> seems cooperative on that point as it provided the necessary hooks to
> implement such a system. So upstream is aware of the problem. They may
> work on solving it and it can take some time.
>
> On the other hand this proposal is one step toward fixing the issue, and
> it works now. Once upstream comes up with a good solution, we can
> revisit the usage of update-alternatives to manage gems binaries in
> /usr/local/bin.

Instead of using a debian-specific feature (update-alternatives), I
think that this should be implemented directly inside rubygems, so it
becomes possible for upstream to integrate this feature. Solving it in a
debian-specific clearly sends the wrong message to upstream, especially
after upstream has been helpful by adding hooks.

> > - you base your version on a git snapshot, with a >5kloc diff compared to the current version in debian unstable. Is that really reasonable, since we are far in the Ubuntu release cycle AFAIK?
>
> IMO this is perfectly acceptable as we're not past FeatureFreeze. That
> means new upstream version can still be uploaded to the archive.

Note that it's not a new upstream version. It's a git snapshot.

> > - have you talked to Daigo Moriwaki about those deep changes to his
> Debian package? If not, when do you plan to?
>
> Talking to Daigo Moriwaki would be helpful. However considering the
> current freeze for Lenny I doubt that this patch will be accepted in
> Debian before Ubuntu enters FeatureFreeze.

The point is not to get it into Debian *now*, but to make sure that
Daigo agrees with the solution, so Ubuntu doesn't maintain divergence on
this.

> > If I understand it correctly, you want to give Ubuntu a competitive
> > advantage by not working with upstream to address this problem globally.
> > That doesn't sound right.
> >
>
> We're trying to improve the usage of gems so that the end user has a
> good experience using it on Ubuntu. This work involves upstream (they
> provided the necessary hooks) and also some integration work by the
> Ubuntu team so that cooperation with dpkg works well. IMO upstream won't
> be able to resolve all of the issue as there will always be some distro
> specific details (location of paths, support for multiple versions).

The path issue is completely orthogonal to the issue of binaries being
overwritten.
--
| Lucas Nussbaum
| <email address hidden> http://www.lucas-nussbaum...

Read more...

Revision history for this message
Lucas Nussbaum (lucas) wrote :

On 14/08/08 at 07:53 -0000, Neil Wilson wrote:
> 2008/8/14 Lucas Nussbaum <email address hidden>:
> > Hi,
> >
> > Some comments:
> > - debian/operating_system.rb is not properly licensed, and not mentioned in debian/copyright.
>
> Agreed. That needs some tidying up. I can't see any copyright message
> in there about the debian packaging either. Does Debian need that as
> well?

Yes, please file a bug.

> > - why the switch to simple-patchsys?
>
> That's easy - it's simpler :-). cdbs-edit-patch is the bees knees. No
> longer do I hate patching.

You are introducing an unnecessary divergence between Debian and Ubuntu,
which will make it harder to merge the changes later.

> > - you base your version on a git snapshot, with a >5kloc diff compared
> to the current version in debian unstable. Is that really reasonable,
> since we are far in the Ubuntu release cycle AFAIK?
>
> I have it on good authority that the version in Debian may very well
> be out of date by the end of the month. ;-)

And? I think Ubuntu (like Debian) cares about stability, not only about
having the very latest software available.

> > - have you talked to Daigo Moriwaki about those deep changes to his
> Debian package? If not, when do you plan to?
>
> At the moment Debian bug #403407 is still marked "won't fix" so there
> is clearly going to be a divergence whatever happens. You really need
> to alter the bug status if Debian is serious about fixing the path
> problem.
>
> I'd be more than happy to sync up with the excellent work Daigo has
> done on his package, but they're never going to be the same until
> Debian accepts that the problem is fixable.
>
> Can you do something about that?

*You* can do something about it: communicate with the Debian developer,
so he knows that you are working on a solution. AFAIK, it's not the case
currently.

> > - You never answered by question about a bug# in
> https://bugs.launchpad.net/ubuntu/+source/libgems-
> ruby/+bug/145267/comments/42 .
>
> Could you help and file one? Otherwise Debian will have to wait. I'm
> kinda busy with a FeatureFreeze deadline and my real job.

That's a different bug. Even in Ubuntu, it should be a different bug.

> > If I understand it correctly, you want to give Ubuntu a competitive
> > advantage by not working with upstream to address this problem globally.
> > That doesn't sound right.
>
> Yes I suppose you probably see it that way. I can't help you with that
> and I don't understand why you would want to adopt such a viewpoint.
>
> I'll let the rubygems commit logs, bug tracker and mailing list speak
> for whether I work with upstream or not.

You talked to upstream to get the hooks, but then you implemented an
Ubuntu-specific solution, that will never be able to be merged upstream,
for an upstream problem. That says it all.
--
| Lucas Nussbaum
| <email address hidden> http://www.lucas-nussbaum.net/ |
| jabber: <email address hidden> GPG: 1024D/023B3F4F |

Revision history for this message
Neil Wilson (neil-aldur) wrote :

New revision with latest upstream fixes and updated copyright detauls in https://launchpad.net/~ubuntu-ruby/+archive

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package libgems-ruby - 1.3.0~RC1-0ubuntu1

---------------
libgems-ruby (1.3.0~RC1-0ubuntu1) intrepid; urgency=low

  [ Neil Wilson ]
  * Make alternatives respond to verbose option.
  * Use a gem specific alternatives database.
  * Add rubygems default package that points at rubygems1.8
  * debian/patches: 40_make_update_system_run_apt_get - make
    'update --system' call apt-get. Replaces 03_disable_update_system.
  * Import from SVN to obtain hooks.
  * Use defaults/operating_system.rb to enforce Debian standards.
  * Use alternatives system to make gems available on PATH.
    Closes LP: #145267.
  * Move to CDBS simple patch system from dpatch.
  * Remove tight ruby1.9 build dependency to allow backports.
  * debian/patches: remove 21_avoid_ioseek - incorporated upstream.
  * debian/patches: remove 01_default_gem_path - replaced by
    debian/operating_system.rb.
  * debian/patches: remove 08_tighter_regex_search - upstream now uses
    a different search system that does not have the substring problem.

  [ Mathias Gug ]
  * Sponsor to intrepid.
  * Remove .cvsignore files.

 -- Mathias Gug <email address hidden> Wed, 27 Aug 2008 12:40:08 -0400

Changed in libgems-ruby:
status: In Progress → Fix Released
Revision history for this message
Scott Kitterman (kitterman) wrote :

Glad to see all the feedback on ubuntu-devel was taken into consideration.

Revision history for this message
Scott Kitterman (kitterman) wrote :

Reopening the bug since the upload was reverted.

Changed in libgems-ruby:
status: Fix Released → Confirmed
Revision history for this message
Neil Wilson (neil-aldur) wrote :

Changing to invalid. Don't bother wasting your time on this bug unless you like religious wars with non-constructive individuals.

The workaround is echo 'PATH=/var/lib/gems/1.8/bin:$PATH' > /etc/profile.d/rubygems1.8.sh
or the equivalent for 1.9.

Changed in libgems-ruby:
assignee: neil-aldur → nobody
status: Confirmed → Invalid
Revision history for this message
Scott Kitterman (kitterman) wrote :

Right. We could probably have a long flamefest about exactly who that
statement best applies to, but let's not.

There are two perspectives here and they each have the own validity. They
each look crazy from the other, but that doesn't mean a compromise the
represent constructive progress is not possible. I don't think it's been
tried (to develop a compromise) yet.

Revision history for this message
Joseph Method (tristil) wrote :

I'd like to file a bug against the communication in this report. Also, there is a caesura at the end leading to a breakup that is hard to follow.

Actually, here's an explanation to save others time:
1. Neil Wilson uploads a package
2. Lucas Nussbaum and Scott Kitterman and others disagree with details about the packaging
3. They discuss it in ubuntu-devel here: https://lists.ubuntu.com/archives/ubuntu-devel/2008-August/026212.html
4. A bug is filed against the uploaded package by Scott Kitterman: https://bugs.launchpad.net/ubuntu/+source/libgems-ruby/+bug/262063
5. The uploaded package is reverted by the powers that be
6. The problem will not be fixed this cycle :(

As Scott Kitterman says above, there could be a long useless discussion about who is being more arrogant or dense or passive-aggressive. In general I see a lot of confusion about priorities in the Debian packaging system, as well as some pervasive confusion about the relationship of Debian policy to the *two different packaging systems* (I think Lucas Nussbaum and Neil Wilson both understood the precise issues, but some others got confused). The best thing to do is start planning for the next cycle on this, especially to make certain process decisions and get certain clarifications and commitments now.

I would suggest:

0. Clarify that "don't use gems" isn't a solution. Gems are used for different reasons than Deb packages.
1. Clarify that Debian policy affects the installation of the rubygems files, but not "the installation of gem files by rubygems".
2. Clarify that Debian policy frowns on installing _any_ rubygems files (that is, files carried by the Apt package) into /usr/local/ AND that there is a resistance to simply adding /var/lib/gems/1.{8,9}/bin to the path. The first is clear but not done by any proposed Rubygems Deb package (right?). The argument for the second is that gem binaries could then supercede system binaries. We need to clarify whether this is really beyond the pale, since the decision to install a gem is an administrator decision, not a user decision. In other words, installing a gem is equivalent to installing a source package except that gems allow for quick uninstall.
3. Commit that *if* Debian rejects movement toward a solution, Ubuntu _can_ maintain its own solution.
4. Clarify what if any changes are required/suggested for upstream
5. Commit that *if* upstream is unresponsive or uninterested, Ubuntu _can_ maintain its own patches.

I'd like to help make this happen for the next cycle and to get a package candidate released into a PPA that Intrepid users can be directed to.

Revision history for this message
Lucas Nussbaum (lucas) wrote :

On 23/09/08 at 05:02 -0000, Joseph Method wrote:
> I'd like to file a bug against the communication in this report. Also,
> there is a caesura at the end leading to a breakup that is hard to
> follow.
>
> Actually, here's an explanation to save others time:
> 1. Neil Wilson uploads a package
> 2. Lucas Nussbaum and Scott Kitterman and others disagree with details about the packaging

No, I also disagree with the solution that was chosen to fix the bug
(the use of alternatives).

> 0. Clarify that "don't use gems" isn't a solution. Gems are used for different reasons than Deb packages.

No, what should be clarified is how much support we want to provide to
users of rubygems on Ubuntu.

> 3. Commit that *if* Debian rejects movement toward a solution, Ubuntu _can_ maintain its own solution.

AFAIK, the Debian rubygems maintainers has never been consulted on this
topic.
--
| Lucas Nussbaum
| <email address hidden> http://www.lucas-nussbaum.net/ |
| jabber: <email address hidden> GPG: 1024D/023B3F4F |

Revision history for this message
Scott Kitterman (kitterman) wrote :

Joseph Method <email address hidden> wrote:
> The argument for the second is that gem binaries could then supercede
system binaries. We need to clarify whether this is really beyond the pale,
since the decision to install a gem is an administrator decision, not a
user decision. In other words, installing a gem is equivalent to installing
a source package except that gems allow for quick uninstall.

Almost. It's the Gem and whatever else the Gem needs. Personally, I see
no problem with installing a Gem that an adminstrator explicitly asks to
have installed in the system path. My primary concern is with the
dependencies Gems pull in with them. If this is largely beyond the scope
of Debian policy, then such 'packages' have an obligation to not cause
problems for packages using the system packages.

Revision history for this message
Joseph Method (tristil) wrote :

FYI, I started a discussion on the Debian bug tracker: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=403407

I think Neil has an interesting idea about gradually integrating Rubygems with Debian. Debian Rubygems users would actually appreciate this. For example, IIRC Rails requires a sqlite3-ruby gem which needs the headers to compile. Much easier to just use the deb package for libsqlite3-ruby. As a farther-out prospect, an Ubuntu-specific Rubygems could intelligently install the sqlite3 package, detecting that the deb version matches what's required by the gem.

This raises the question of whether we should even have Rubygems. The reason we should say yes is that Rubygems is a cross-platform packaging system. Gems have to install on MacOS and Windows, and Ruby development is targeted to the gems. On those other systems it's necessary to install all the headers before installing the related gems (many of them are provided on the MacOS Developer CD). So Debian Rubygems could be a *better* Rubygems.

As Neil mentioned this couldn't happen immediately. I still wonder, though, whether the relevant paradigm isn't build tools, or even rm. Debian can't stop you from hosing your system as long as you can install from source. If files are installed into /usr/local at least the user can delete everything under /usr/local/lib and /usr/local/share to get back to a stable system.

Revision history for this message
technomancy (technomancy) wrote :

> Instead of using a debian-specific feature (update-alternatives), I
> think that this should be implemented directly inside rubygems, so it
> becomes possible for upstream to integrate this feature.

For what it's worth, I've talked to the upstream maintainer of Rubygems (Eric Hodel) about this, and he says it's extremely unlikely that such functionality would be re-implemented inside rubygems.

> Solving it in a debian-specific clearly sends the wrong message to upstream, especially
> after upstream has been helpful by adding hooks.

The hooks were added to support integration with the alternatives system. This has always been the plan from the outset and was well-understood by upstream. Eric has told me he fully supports Neil's work.

Changed in libgems-ruby (Debian):
status: Won't Fix → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.