Launchpad itself

bzrlib.errors.RevisionNotPresent: Revision {('hg:.hgignore', 'hg-v1:48dfd5c9dc5d1591c6c64b69777d596dda227baf')} not present in "CHKInventoryRepository('file:///srv/importd.launchpad.net/data/worker-for-branch-310101/bzr_branch/.bzr/repository/')".

Bug #519709 reported by Ali Sabil on 2010-02-10

This bug affects 8 people

Affects		Status	Importance	Assigned to	Milestone
	Bazaar Hg Plugin	Confirmed	High	Unassigned
	Launchpad itself	Won't Fix	High	Unassigned

Bug Description

The following mercurial import fails: https://code.launchpad.net/~asabil/prosody-im/trunk

2011-05-02 06:03:04 INFO Starting job.
2011-05-02 06:03:04 INFO Getting exising bzr branch from central store.
2011-05-02 06:03:04 INFO [chan bzr SocketAsChannelAdapter] Opened sftp connection (server version 3)
2011-05-02 06:03:04 INFO 35 bytes transferred
2011-05-02 06:03:05 INFO [chan bzr SocketAsChannelAdapter] Opened sftp connection (server version 3)
2011-05-02 06:03:07 INFO Importing foreign branch.
2011-05-02 06:03:12 INFO fetching changesets 0
2011-05-02 06:03:12 INFO
2011-05-02 06:03:13 INFO updating cache 0/871
2011-05-02 06:03:17 INFO
Traceback (most recent call last):
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/scripts/code-import-worker.py", line 82, in <module>
    sys.exit(script.main())
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/scripts/code-import-worker.py", line 77, in main
    return import_worker.run()
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/lib/lp/codehosting/codeimport/worker.py", line 452, in run
    return self._doImport()
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/lib/lp/codehosting/codeimport/worker.py", line 590, in _doImport
    overwrite=True, **self.getExtraPullArgs())
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/bzrplugins/optional/hg/branch.py", line 279, in pull
    inter.fetch(revision_id=stop_revision)
  File "<string>", line 4, in fetch_write_locked
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/bzrplugins/optional/hg/fetch.py", line 651, in fetch
    self.addchangegroup(cg, mapping)
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/bzrplugins/optional/hg/fetch.py", line 517, in addchangegroup
    self._target_overlay.remember_manifest_texts(self._unpack_manifests(manifestchunks, mapping, filetext_map, todo, pb))
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/bzrplugins/optional/hg/overlay.py", line 125, in remember_manifest_texts
    ((FulltextContentFactory((revid,), [(p,) for p in parent_revids] , None, text) for (revid, parent_revids, text) in entries)))
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/knit.py", line 1600, in insert_record_stream
    for record in stream:
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/bzrplugins/optional/hg/overlay.py", line 125, in <genexpr>
    ((FulltextContentFactory((revid,), [(p,) for p in parent_revids] , None, text) for (revid, parent_revids, text) in entries)))
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/bzrplugins/optional/hg/fetch.py", line 455, in _unpack_manifests
    unpack_chunk_iter(chunkiter, self._target_overlay.get_manifest_text)):
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/bzrplugins/optional/hg/parsers.py", line 153, in unpack_chunk_iter
    textbase = lookup_base(base)
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/bzrplugins/optional/hg/overlay.py", line 200, in get_manifest_text
    return self.get_manifest_text_by_revid(revid)
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/bzrplugins/optional/hg/overlay.py", line 215, in get_manifest_text_by_revid
    (manifest, flags) = self.get_manifest_and_flags_by_revid(revid)
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/bzrplugins/optional/hg/overlay.py", line 246, in get_manifest_and_flags_by_revid
.....
    return self._reconstruct_manifest_and_flags_by_revid(revid)
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/bzrplugins/optional/hg/overlay.py", line 251, in _reconstruct_manifest_and_flags_by_revid
    rev = self.repo.get_revision(revid)
  File "<string>", line 4, in get_revision_read_locked
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/repofmt/knitrepo.py", line 209, in get_revision
    return self.get_revision_reconcile(revision_id)
  File "<string>", line 4, in get_revision_reconcile_read_locked
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/repository.py", line 1892, in get_revision_reconcile
    return self._get_revisions([revision_id])[0]
  File "<string>", line 4, in _get_revisions_read_locked
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/repository.py", line 1907, in _get_revisions
    for revid, rev in self._iter_revisions(revision_ids):
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/repository.py", line 1931, in _iter_revisions
    for record in stream:
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/groupcompress.py", line 1413, in get_record_stream
    orig_keys, ordering, include_delta_closure):
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/groupcompress.py", line 1518, in _get_remaining_record_stream
    locations = self._index.get_build_details(keys)
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/groupcompress.py", line 2006, in get_build_details
    for entry in entries:
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/groupcompress.py", line 1941, in _get_entries
    for node in self._graph_index.iter_entries(keys):
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/index.py", line 1358, in iter_entries
    for node in index.iter_entries(keys):
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/btree_index.py", line 1167, in iter_entries
    nodes, nodes_and_keys = self._walk_through_internal_nodes(needed_keys)
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/btree_index.py", line 1121, in _walk_through_internal_nodes
    nodes = self._get_leaf_nodes(node_indexes)
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/btree_index.py", line 962, in _get_leaf_nodes
    found = self._get_nodes(self._leaf_node_cache, node_indexes)
  File "/srv/importd.launchpad.net/production/launchpad-rev-12915/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/btree_index.py", line 933, in _get_nodes
    found[idx] = cache[idx]
RuntimeError: maximum recursion depth exceeded
Import failed:
Traceback (most recent call last):
Failure: twisted.internet.error.ProcessTerminated: A process has ended with a probable error condition: process ended with exit code 1.

See original description

Tags:

Related branches

lp:~matsubara/launchpad/update-bzr-hg (Merged)

Revision history for this message

Tim Penhey (thumper) wrote on 2010-02-10: Re: [Bug 519709] [NEW] Mercurial import for prosody fails

  importance high
  status triaged
  affects bzr-hg
  tag code-import
  tag hg

Changed in launchpad-code:
importance:	Undecided → High
status:	New → Triaged

Max Bowsher (maxb) on 2010-06-16

summary:

- Mercurial import for prosody fails
+ Import fails with infinite recursion through
+ _reconstruct_manifest_and_flags_by_revid

Jelmer Vernooij (jelmer) on 2010-06-16

Changed in bzr-hg:
status:	New → Triaged
importance:	Undecided → Medium

Revision history for this message

Benjamin Drung (bdrung) wrote on 2010-06-16: Re: Import fails with infinite recursion through _reconstruct_manifest_and_flags_by_revid

audacious fails too: https://code.launchpad.net/~vcs-imports/audacious/trunk

Revision history for this message

Niel Drummond (niel-drummond) wrote on 2010-06-28:

also here https://code.launchpad.net/~niel-cyanescent/jeash/master

Revision history for this message

Timmie (timmie) wrote on 2010-09-11:

Why does this bug occur?

Revision history for this message

Martin Pool (mbp) wrote on 2011-01-31:

These are still failing, but not with infinite recursion:

Traceback (most recent call last):
  File "/srv/importd.launchpad.net/production/launchpad-rev-12200/scripts/code-import-worker.py", line 82, in <module>
    sys.exit(script.main())
  File "/srv/importd.launchpad.net/production/launchpad-rev-12200/scripts/code-import-worker.py", line 77, in main
    return import_worker.run()
  File "/srv/importd.launchpad.net/production/launchpad-rev-12200/lib/lp/codehosting/codeimport/worker.py", line 452, in run
    return self._doImport()
  File "/srv/importd.launchpad.net/production/launchpad-rev-12200/lib/lp/codehosting/codeimport/worker.py", line 590, in _doImport
    overwrite=True, **self.getExtraPullArgs())
  File "/srv/importd.launchpad.net/production/launchpad-rev-12200/optionalbzrplugins/hg/branch.py", line 279, in pull
    inter.fetch(revision_id=stop_revision)
  File "<string>", line 4, in fetch_write_locked
  File "/srv/importd.launchpad.net/production/launchpad-rev-12200/optionalbzrplugins/hg/fetch.py", line 651, in fetch
    self.addchangegroup(cg, mapping)
  File "/srv/importd.launchpad.net/production/launchpad-rev-12200/optionalbzrplugins/hg/fetch.py", line 530, in addchangegroup
    self._add_inventories(todo, mapping, pb)
  File "/srv/importd.launchpad.net/production/launchpad-rev-12200/optionalbzrplugins/hg/fetch.py", line 385, in _add_inventories
    manifest, flags, files, rev, mapping)
  File "/srv/importd.launchpad.net/production/launchpad-rev-12200/optionalbzrplugins/hg/fetch.py", line 337, in _import_manifest_delta
    self._symlink_targets.__getitem__))
  File "/srv/importd.launchpad.net/production/launchpad-rev-12200/optionalbzrplugins/hg/fetch.py", line 207, in manifest_to_inventory_delta
    (fileid, ie.revision))
KeyError: ('hg:jeash_sswf_sEditText.hx', 'hg-v1:dad204a930128dc913c27020691e7be7894bc81f')

Perhaps this is a dupe now?

Revision history for this message

Martin Pool (mbp) wrote on 2011-01-31:

This looks a lot like bug 692901; please un-dupe if it's not.

Revision history for this message

Martin Pool (mbp) wrote on 2011-05-09:

i think some of these branches failed due to bug 692901, but this is actually a distinct bug. it's occurring now in for instance <https://code.launchpad.net/~pythonxy/spyderlib/spyder_devel_mirror> <http://launchpadlibrarian.net/70896727/pythonxy-spyderlib-spyder_devel_mirror.log>.

description:	updated
Changed in bzr-hg:
importance:	Medium → High

Revision history for this message

Jelmer Vernooij (jelmer) wrote on 2011-05-09: Re: [Bug 519709] Re: Import fails with infinite recursion through _reconstruct_manifest_and_flags_by_revid

On Mon, 2011-05-09 at 15:29 +0000, Martin Pool wrote:
> i think some of these branches failed due to bug 692901, but this is
> actually a distinct bug. it's occurring now in for instance
> <https://code.launchpad.net/~pythonxy/spyderlib/spyder_devel_mirror>
> <http://launchpadlibrarian.net/70896727/pythonxy-spyderlib-
> spyder_devel_mirror.log>.
I think this is actually fixed in bzr-hg trunk, trying it manually at
the moment to confirm.

Cheers,

Jelmer

Revision history for this message

Timmie (timmie) wrote on 2011-05-09: Re: Import fails with infinite recursion through _reconstruct_manifest_and_flags_by_revid

Importing manually works in Ubuntu 11.04.
But not in Launchpad.

Revision history for this message

Jelmer Vernooij (jelmer) wrote on 2011-05-09: Re: [Bug 519709] Re: Import fails with infinite recursion through _reconstruct_manifest_and_flags_by_revid

#10

On Mon, 2011-05-09 at 17:52 +0200, Jelmer Vernooij wrote:
> On Mon, 2011-05-09 at 15:29 +0000, Martin Pool wrote:
> > i think some of these branches failed due to bug 692901, but this is
> > actually a distinct bug. it's occurring now in for instance
> > <https://code.launchpad.net/~pythonxy/spyderlib/spyder_devel_mirror>
> > <http://launchpadlibrarian.net/70896727/pythonxy-spyderlib-
> > spyder_devel_mirror.log>.
> I think this is actually fixed in bzr-hg trunk, trying it manually at
> the moment to confirm.
Confirmed that this works with bzr-hg trunk - we should get a new bzr-hg
deployed on Launchpad.

Cheers,

Jelmer

Revision history for this message

Martin Pool (mbp) wrote on 2011-05-27: Re: Import fails with infinite recursion through _reconstruct_manifest_and_flags_by_revid

#11

@Jelmer should the bzr-hg task of this bug be closed?

I believe we were blocked on the update because of the change in handling of incremental fetches and that is now fixed? So all that's needed to close the lp side of this is to just actually push a bzr-hg update into lp's dependencies, and probably also a new bzr?

Revision history for this message

Martin Pool (mbp) wrote on 2011-05-27:

#12

Going off jelmer's comment i'm going to mark this fixed; please reopen if it's not.

Changed in bzr-hg:
assignee:	nobody → Jelmer Vernooij (jelmer)
status:	Triaged → Fix Released
Changed in launchpad:
assignee:	nobody → Diogo Matsubara (matsubara)
status:	Triaged → In Progress

Revision history for this message

Michael Hudson-Doyle (mwhudson) wrote on 2011-05-29: Re: [Bug 519709] Re: Import fails with infinite recursion through _reconstruct_manifest_and_flags_by_revid

#13

On Fri, 27 May 2011 19:47:01 -0000, Martin Pool <email address hidden> wrote:
> @Jelmer should the bzr-hg task of this bug be closed?
>
> I believe we were blocked on the update because of the change in
> handling of incremental fetches and that is now fixed? So all that's
> needed to close the lp side of this is to just actually push a bzr-hg
> update into lp's dependencies, and probably also a new bzr?

I don't know the details, but I'm fairly sure the incremental fetch
stuff has never worked with bzr-hg, so updating just bzr-hg cannot
possibly make this aspect worse.

Cheers,
mwh

Revision history for this message

Jelmer Vernooij (jelmer) wrote on 2011-05-30:

#14

On Sun, 2011-05-29 at 23:39 +0000, Michael Hudson-Doyle wrote:
> On Fri, 27 May 2011 19:47:01 -0000, Martin Pool <email address hidden> wrote:
> > @Jelmer should the bzr-hg task of this bug be closed?
> >
> > I believe we were blocked on the update because of the change in
> > handling of incremental fetches and that is now fixed? So all that's
> > needed to close the lp side of this is to just actually push a bzr-hg
> > update into lp's dependencies, and probably also a new bzr?
>
> I don't know the details, but I'm fairly sure the incremental fetch
> stuff has never worked with bzr-hg, so updating just bzr-hg cannot
> possibly make this aspect worse.
The problem is that updating bzr-hg will require a newer bzr on
Launchpad too (current version is 2.2-something), and updating bzr will
require updating bzr-git/bzr-svn too - and they are affected by that
issue.

All of this is in progress, I've landed newer versions of bzr-svn,
subvertpy, dulwich, bzr and bzr-git on the Launchpad branches and I
think Diogo has landed a newer bzr-hg. There is some fallout in the
Launchpad tree that I hope to work on this week.

Cheers,

Jelmer

Revision history for this message

Diogo Matsubara (matsubara) wrote on 2011-05-31: Re: Import fails with infinite recursion through _reconstruct_manifest_and_flags_by_revid

#15

update-bzr-hg-r13135.subunit.gz Edit (372.8 KiB, application/octet-stream)

I ec2 tested the branch with bzr-hg updated to 284 and bzr updated to 2.3.3. I got the test failures in update-bzr-hg-r13135.subunit.gz. I then ec2 tested the branch only with the bzr-hg update and got less test failure (update-bzr-hg-r13137.subunit.gz), but it failed nonetheless.

Revision history for this message

Diogo Matsubara (matsubara) wrote on 2011-05-31:

#16

update-bzr-hg-r13137.subunit.gz Edit (371.2 KiB, application/octet-stream)

Revision history for this message

Robert Collins (lifeless) wrote on 2012-01-04:

#17

We're now running r292 of bzr-hg

Changed in launchpad:
status:	In Progress → Fix Committed
status:	Fix Committed → Fix Released
status:	Fix Released → Incomplete

Revision history for this message

Robert Collins (lifeless) wrote on 2012-01-04:

#18

I've kicked the import off to see if its really fixed

Revision history for this message

Robert Collins (lifeless) wrote on 2012-01-04:

#19

Download full text (3.2 KiB)

So the referenced branch fails with
Traceback (most recent call last):
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/scripts/code-import-worker.py", line 99, in <module>
    sys.exit(script.main())
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/scripts/code-import-worker.py", line 94, in main
    return import_worker.run()
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/lib/lp/codehosting/codeimport/worker.py", line 591, in run
    return self._doImport()
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/lib/lp/codehosting/codeimport/worker.py", line 745, in _doImport
    inter_branch.fetch(limit=revision_limit)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/bzrplugins/hg/branch.py", line 483, in fetch
    inter.fetch(revision_id=stop_revision, limit=limit)
  File "<string>", line 4, in fetch_write_locked
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/bzrplugins/hg/fetch.py", line 766, in fetch
    self.addchangegroup(cg, mapping, limit=limit)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/bzrplugins/hg/fetch.py", line 631, in addchangegroup
    self.target.texts.insert_record_stream(texts)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/eggs/bzr-2.5.0dev2_r6152-py2.6-linux-x86_64.egg/bzrlib/groupcompress.py", line 1662, in insert_record_stream
    for _ in self._insert_record_stream(stream, random_id=False):
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/eggs/bzr-2.5.0dev2_r6152-py2.6-linux-x86_64.egg/bzrlib/groupcompress.py", line 1752, in _insert_record_stream
    for record in stream:
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/bzrplugins/hg/fetch.py", line 417, in _unpack_texts
    itertextchunks, get_text):
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/bzrplugins/hg/parsers.py", line 166, in unpack_chunk_iter
    textbase = lookup_base(base)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/bzrplugins/hg/fetch.py", line 415, in get_text
    return self._get_target_fulltext(key)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/bzrplugins/hg/fetch.py", line 375, in _get_target_fulltext
    return self._target_overlay.get_file_fulltext(key)
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/bzrplugins/hg/overlay.py", line 229, in get_file_fulltext
    ret = "".join(self.repo.iter_files_bytes([key + (None,)]).next()[1])
  File "/srv/importd.launchpad.net/production/launchpad-rev-14616/eggs/bzr-2.5.0dev2_r6152-py2.6-linux-x86_64.egg/bzrlib/vf_repository.py", line 1516, in iter_files_bytes
    raise errors.RevisionNotPresent(record.key, self)
bzrlib.errors.RevisionNotPresent: Revision {('hg:.hgignore', 'hg-v1:48dfd5c9dc5d1591c6c64b69777d596dda227baf')} not present in "CHKInventoryRepository('file:///srv/importd.launchpad.net/data/worker-for-branch-310101/bzr_branch/.bzr/repository/')".
Import failed:
Traceback (most recent call last):
Failure: twisted.internet.error.ProcessTerminated: A process has ended with a probable error condition: process ended with exit code 1.

which is at least a ...

Duplicates of this bug

You are

Subscribing...

Edit bug mail

Other bug subscribers

Bug attachments

Add attachment

Remote bug watches

Bug watches keep track of this bug in other bug trackers.

Changed in bzr-hg:
status:	Fix Released → Confirmed
Changed in launchpad:
status:	Incomplete → Triaged
summary:	- Import fails with infinite recursion through - _reconstruct_manifest_and_flags_by_revid + https://code.launchpad.net/~asabil/prosody-im/trunk fails to import

Changed in bzr-hg:
assignee:	Jelmer Vernooij (jelmer) → nobody

Changed in launchpad:
assignee:	Diogo Matsubara (matsubara) → nobody