Conversion to --2a broken when upgrading into a stacked branch

Bug #402778 reported by Andrew Mitchell
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Bazaar
Fix Released
Critical
John A Meinel

Bug Description

This error occurred when trying to fetch launchpad source with rocketfuel-get

Wed 2009-07-22 10:39:36 +1200
0.077 bzr arguments: [u'pull', u'--remember', u'lp:~launchpad/pygettextpo/trunk/', u'--directory', u'/home/ajmitch/launchpad/lp-sourcedeps/sourcecode/pygettextpo']
0.112 looking for plugins in /home/ajmitch/.bazaar/plugins
0.238 looking for plugins in /usr/lib/python2.6/dist-packages/bzrlib/plugins
0.299 encoding stdout as osutils.get_user_encoding() 'UTF-8'
0.374 opening working tree '/home/ajmitch/launchpad/lp-sourcedeps/sourcecode/pygettextpo'
2.628 ssh implementation is OpenSSH
13.126 bzr-svn: using Subversion 1.6.1 ()
13.152 opening SVN RA connection to 'bzr+ssh://bazaar.launchpad.net/~launchpad/pygettextpo/trunk'
72.946 checking remap as 20 deletions
72.967 checking remap as 20 deletions
73.191 checking remap as 362 deletions
73.192 remap generated a new LeafNode
73.290 checking remap as 362 deletions
73.291 remap generated a new LeafNode
73.754 Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 835, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 1030, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 647, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.6/dist-packages/bzrlib/builtins.py", line 1013, in run
    possible_transports=possible_transports, local=local)
  File "/usr/lib/python2.6/dist-packages/bzrlib/decorators.py", line 192, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/workingtree.py", line 1588, in pull
    local=local)
  File "/usr/lib/python2.6/dist-packages/bzrlib/decorators.py", line 192, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/branch.py", line 891, in pull
    possible_transports=possible_transports, *args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/branch.py", line 3154, in pull
    _override_hook_target=_override_hook_target)
  File "/usr/lib/python2.6/dist-packages/bzrlib/branch.py", line 3031, in pull
    overwrite=overwrite, graph=graph)
  File "/usr/lib/python2.6/dist-packages/bzrlib/decorators.py", line 192, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/branch.py", line 839, in update_revisions
    overwrite, graph)
  File "/usr/lib/python2.6/dist-packages/bzrlib/branch.py", line 2974, in update_revisions
    self.target.fetch(self.source, stop_revision)
  File "/usr/lib/python2.6/dist-packages/bzrlib/decorators.py", line 192, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/branch.py", line 565, in fetch
    pb=pb)
  File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line 1544, in fetch
    find_ghosts=find_ghosts, fetch_spec=fetch_spec)
  File "/usr/lib/python2.6/dist-packages/bzrlib/decorators.py", line 192, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line 3735, in fetch
    self._fetch_all_revisions(revision_ids, pb)
  File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line 3689, in _fetch_all_revisions
    basis_id = self._fetch_batch(batch, basis_id, cache)
  File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line 3651, in _fetch_batch
    for parent_tree in self.source.revision_trees(parent_ids):
  File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line 2373, in revision_trees
    for inv in inventories:
  File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line 2198, in _iter_inventories
    for text, revision_id in self._iter_inventory_xmls(revision_ids):
  File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line 2209, in _iter_inventory_xmls
    raise errors.NoSuchRevision(self, record.key)
NoSuchRevision: KnitPackRepository('bzr+ssh://bazaar.launchpad.net/~launchpad/pygettextpo/trunk/.bzr/repository/') has no revision ('Arch-1:<email address hidden>%pygettextpo--devel--0--patch-5',)

73.760 Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/bzrlib/plugin.py", line 407, in _get__version__
    version_string = _format_version_tuple(version_info)
  File "/usr/lib/python2.6/dist-packages/bzrlib/__init__.py", line 90, in _format_version_tuple
    sub = version_info[4]
IndexError: tuple index out of range

73.761 return code 4

Related branches

Revision history for this message
Robert Collins (lifeless) wrote : Re: [Bug 402778] [NEW] NoSuchRevision error when branching with rocketfuel-get

I believe that this is triggered when a 2a repo is used and pygettextpo
is pulled into it - it indicates a bug in the conversion logic.

-Rob

Revision history for this message
Robert Collins (lifeless) wrote : Re: NoSuchRevision error when branching with rocketfuel-get
Download full text (4.7 KiB)

A second example..

ataraxia% bzr branch --no-tree --stacked lp:launchpad /tmp
Created new stacked branch referring to http://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-devel/.
ataraxia% cd launchpad /tmp
ataraxia(bzr)% bzr pull --overwrite lp:~launchpad-pqm/launchpad-cscvs/devel/
bzr: ERROR: bzrlib.errors.NoSuchRevision: KnitPackRepository('bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad-cscvs/devel/.bzr/repository/') has no revision ('Arch-1:<email address hidden>%cscvs--devel--1.0--patch-16',)

Traceback (most recent call last):
  File "/home/dsilvers/dev-bzr/bzr.dev/bzrlib/commands.py", line 835, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/home/dsilvers/dev-bzr/bzr.dev/bzrlib/commands.py", line 1030, in run_bzr
    ret = run(*run_argv)
  File "/home/dsilvers/dev-bzr/bzr.dev/bzrlib/commands.py", line 647, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/home/dsilvers/dev-bzr/bzr.dev/bzrlib/builtins.py", line 1016, in run
    branch_from, overwrite, revision_id, local=local)
  File "/home/dsilvers/dev-bzr/bzr.dev/bzrlib/decorators.py", line 192, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/home/dsilvers/dev-bzr/bzr.dev/bzrlib/branch.py", line 891, in pull
    possible_transports=possible_transports, *args, **kwargs)
  File "/home/dsilvers/dev-bzr/bzr.dev/bzrlib/branch.py", line 3154, in pull
    _override_hook_target=_override_hook_target)
  File "/home/dsilvers/dev-bzr/bzr.dev/bzrlib/branch.py", line 3031, in pull
    overwrite=overwrite, graph=graph)
  File "/home/dsilvers/dev-bzr/bzr.dev/bzrlib/decorators.py", line 192, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/home/dsilvers/dev-bzr/bzr.dev/bzrlib/branch.py", line 839, in update_revisions
    overwrite, graph)
  File "/home/dsilvers/dev-bzr/bzr.dev/bzrlib/branch.py", line 2974, in update_revisions
    self.target.fetch(self.source, stop_revision)
  File "/home/dsilvers/dev-bzr/bzr.dev/bzrlib/decorators.py", line 192, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/home/dsilvers/dev-bzr/bzr.dev/bzrlib/branch.py", line 565, in fetch
    pb=pb)
  File "/home/dsilvers/dev-bzr/bzr.dev/bzrlib/repository.py", line 1544, in fetch
    find_ghosts=find_ghosts, fetch_spec=fetch_spec)
  File "/home/dsilvers/dev-bzr/bzr.dev/bzrlib/decorators.py", line 192, in write_locked
    result = unbound(self, *args, **kwargs)
  File "/home/dsilvers/dev-bzr/bzr.dev/bzrlib/repository.py", line 3735, in fetch
    self._fetch_all_revisions(revision_ids, pb)
  File "/home/dsilvers/dev-bzr/bzr.dev/bzrlib/repository.py", line 3689, in _fetch_all_revisions
    basis_id = self._fetch_batch(batch, basis_id, cache)
  File "/home/dsilvers/dev-bzr/bzr.dev/bzrlib/repository.py", line 3651, in _fetch_batch
    for parent_tree in self.source.revision_trees(parent_ids):
  File "/home/dsilvers/dev-bzr/bzr.dev/bzrlib/repository.py", line 2373, in revision_trees
    for inv in inventories:
  File "/home/dsilvers/dev-bzr/bzr.dev/bzrlib/repository....

Read more...

Changed in bzr:
importance: Undecided → Critical
milestone: none → 2.0
status: New → Triaged
Revision history for this message
John A Meinel (jameinel) wrote :

So

#1) this may be papered over by Andrew's inventory-delta fixes for streaming over the network, so I think it is prudent that any work being done in this direction takes his patch into account. (wait for it to land or test it on his branch)
#2) It only happens when the target branch is stacked and has ghosts. If you look at the code it is specifically:
        if self.target._fallback_repositories:
...
            parent_ids = set()
            revision_ids = set()
            for revision in pending_revisions:
                revision_ids.add(revision.revision_id)
                parent_ids.update(revision.parent_ids)
            parent_ids.difference_update(revision_ids)
            parent_ids.discard(_mod_revision.NULL_REVISION)
            parent_map = self.source.get_parent_map(parent_ids)
            for parent_tree in self.source.revision_trees(parent_ids):
...
                self.target.add_inventory_by_delta(
                    basis_id, delta, current_revision_id, parents_parents)

This wasn't part of my original "IDS" code, but was added by Andrew in:
   4257.3.9 Andrew Bennetts 2009-04-09
            Add fix to InterDifferingSerializer too, although it's pretty ugly.

Which is why I didn't notice it right away.

Looking closely, it is actually both broken and incorrect. Specifically:
1) We can just use the results of "parent_map" so that we don't try to fetch anything that is considered a ghost.
2) It does something really bad, namely it does:
            for parent_tree in self.source.revision_trees(parent_map):
                basis_id, delta = self._get_delta_for_revision(tree, parent_ids, basis_id, cache)
...
                self.target.add_inventory_by_delta(
                    basis_id, delta, current_revision_id, parents_parents)

^- Note that it loops over "parent_tree" but generates the delta for *tree*, which it then adds as "current_revision_id" which is actually the parent_tree's revision..... yikes.

So definitely critical, and I should investigate whatever other fixes happened, in case there are Copy & Paste bugs present elsewhere.

summary: - NoSuchRevision error when branching with rocketfuel-get
+ Conversion to --2a broken when upgrading into a stacked branch
Revision history for this message
John A Meinel (jameinel) wrote :

Fix in associated branch. The IDS code that handles sending parent inventories for stacked branches now also
1) sends correct deltas
2) does not try to fill in parents who are ghosts in the source

Changed in bzr:
assignee: nobody → John A Meinel (jameinel)
status: Triaged → Fix Committed
Revision history for this message
Andrew Bennetts (spiv) wrote :

The fix has been merged into bzr.dev, I believe. (Launchpad certainly thinks the associated branch is merged)

Changed in bzr:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.