get_record_stream/ContentFactory should expose compression parents

Bug #300177 reported by Martin Pool
2
Affects Status Importance Assigned to Milestone
Bazaar
Fix Released
High
John A Meinel

Bug Description

In the course of the recent thread about bug 288751 and "avoid creating text deltas spanning repository stack", it was mentioned that get_record_stream and ContentFactory don't expose the compression parents clearly, but this is probably necessary to correctly apply the deltas.

At any rate it should be clearly defined in the interface whether the parents are the same as the compression parents or not, and what their ordering is.

Martin Pool (mbp)
Changed in bzr:
assignee: nobody → jameinel
importance: Undecided → High
status: New → Confirmed
Revision history for this message
Martin Pool (mbp) wrote :

For instance there is this code in knit.py, which probably should instead be using an explicit compression parent.

            # Add any records whose basis parent is now available.
            added_keys = [record.key]
            while added_keys:
                key = added_keys.pop(0)
                if key in buffered_index_entries:
                    index_entries = buffered_index_entries[key]
                    self._index.add_records(index_entries)
                    added_keys.extend(
                        [index_entry[0] for index_entry in index_entries])
                    del buffered_index_entries[key]

Revision history for this message
John A Meinel (jameinel) wrote :

I'm not sure that Martin copy & pasted the right code snippet. I would have thought more of:

                index_entry = (record.key, options, access_memo, parents)
                buffered = False
                if 'fulltext' not in options:
                    basis_parent = parents[0]
...
                    if basis_parent not in self.get_parent_map([basis_parent]):
                        pending = buffered_index_entries.setdefault(
                            basis_parent, [])
                        pending.append(index_entry)
                        buffered = True
                if not buffered:
                    self._index.add_records([index_entry])

Actually the real problem is down in _KnitGraphindex.add_records, in that code you have:

                if self._deltas:
                    if 'line-delta' in options:
                        node_refs = (parents, (parents[0],))
                    else:
                        node_refs = (parents, ())

So when this delta is added to the index, it implicitly assumes that parents[0] is the compression parent for this delta.

I'll also note that for knit repositories, they cannot represent a delta which is not against their left-hand ancestor. Only Pack indexes have a field for compression parent. So some care is needed, in that pulling a delta into a repository that cannot represent it will need to be extracted up to a full text and then delta compressed again.

Revision history for this message
John A Meinel (jameinel) wrote :

So it was worked out that Knit*Factory objects just conform to the api that compression parents are always the left-hand parent. And other factory objects (so far) don't have compression parents, so it didn't matter.

Changed in bzr:
status: Confirmed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.