Traceback calling Vte.Terminal.feed_child()

Calling Vte.Terminal.feed_child() with "n\n", -1 works fine with version 0.52.1-1ubuntu1 on Ubuntu 18.04 but using version in bionic-updates and cosmic I receive the following Traceback:

In [10]: term.feed_child("n\n", -1)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-10-aea6cb64c22b> in <module>()
----> 1 term.feed_child("n\n", -1)

TypeError: Vte.Terminal.feed_child() takes exactly 2 arguments (3 given)

In [11]:
Do you really want to exit ([y]/n)? y
[ 8:10AM 10167 ] [ bdmurray@impulse:~ ]
 $ apt-cache policy gir1.2-vte-2.91
gir1.2-vte-2.91:
  Installed: 0.52.2-1ubuntu1~18.04.1
  Candidate: 0.52.2-1ubuntu1~18.04.1
  Version table:
 *** 0.52.2-1ubuntu1~18.04.1 500
        500 http://192.168.10.7/ubuntu bionic-updates/main amd64 Packages

Dropping the patch that appears here:

https://launchpadlibrarian.net/366367148/vte2.91_0.52.1-1ubuntu1_0.52.1-1ubuntu2.diff

Makes Vte.Terminal.feed_child() work again, the way it has been for several releases of Ubuntu.

Calling feed_child() with the version of vte2.91 in bionic-updates this way works:

term.feed_child("n\n".encode("utf-8"))

However, it seems that the length argument disappeared even though it continues to be documented.

And here is how guake deals with this:

121 def feed_child(self, resolved_cmdline):
122 if (Vte.MAJOR_VERSION, Vte.MINOR_VERSION) >= (0, 42):
123 encoded = resolved_cmdline.encode("utf-8")
124 try:
125 super().feed_child_binary(encoded)
126 except TypeError:
127 # The doc doest not say clearly at which version the feed_child* function has lost
128 # the "len" parameter :(
129 super().feed_child(resolved_cmdline, len(resolved_cmdline))
130 else:
131 super().feed_child(resolved_cmdline, len(resolved_cmdline))

This may end of being an issue for terminator as it calls feed_child a couple of times.

It'd be much nicer if you could ask before assigning bugs to me. Thanks.

I'll ask Rico if he can take a look.

For reference https://lazka.github.io/pgi-docs/#Vte-2.91/classes/Terminal.html#Vte.Terminal.feed_child

OK, my feeling is that we shouldn't fix vte2.91 for this. We had accidental downstream API differences here, which projects had to deal with and it's a good thing they don't any more.

Definitely not for cosmic, although possibly more debatable in bionic-updates. Given that, u-r-u is going to have to be fixed for this *anyway* so you might as well SRU.

Feel free to ask the SRU team and if they think we should revert that patch addition in bionic-updates I'll do it. But the discovered fix sounds like the right one for >= cosmic.

s/addition/change/

I agree that having downstream API differences is bad and that it shouldn't be fixed for cosmic. However, projects / packages other than ubuntu-release-upgrader have had to do extra work because of the SRU of this in Ubuntu 18.04. See https://bugs.launchpad.net/cubic/+bug/1779015 and the changes identified in guake.

We don't know how many other packages / projects have code which expects feed_child() to behave a specific way and changing that underneath them is wrong. There was some discussion in the foundations team meeting regarding the issue.

https://irclogs.ubuntu.com/2018/07/12/%23ubuntu-meeting.html#t15:30

Please revert this change in Ubuntu 18.04.

OK.

For the record, it's not an assumption. Catching TypeError is a way to deal with the fact that there were two versions of this function around. The version of Vte in bionic, because of the MIR team's refusal to let us have PCRE 2 in main, accidentally had an API break relative to the same version released upstream. If you wanted your code to work with the same upstream version of Vte on Ubuntu and non-Ubuntu you had to deal with this.

It's uploaded.

Please do test this; I gave it a bit of a go but I probably didn't exercise everything.

Oh and it would be helpful if you could write the SRU description since you have the real test case and I don't. Or waive it if you want.

The regression potential is that I messed up the upload and everything using feed* breaks.

Hello Brian, or anyone else affected,

Accepted vte2.91 into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/vte2.91/0.52.2-1ubuntu1~18.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

I've verified the test case for an upgrade from Bionic to Cosmic.

bdmurray@clean-bionic-amd64:~$ grep vte /var/log/dist-upgrade/apt-term.log
Preparing to unpack .../143-libvte-2.91-common_0.52.2-1ubuntu1_all.deb ...
Unpacking libvte-2.91-common (0.52.2-1ubuntu1) over (0.52.2-1ubuntu1~18.04.2) ...
Preparing to unpack .../144-libvte-2.91-0_0.52.2-1ubuntu1_amd64.deb ...
Unpacking libvte-2.91-0:amd64 (0.52.2-1ubuntu1) over (0.52.2-1ubuntu1~18.04.2) ...
Preparing to unpack .../145-gir1.2-vte-2.91_0.52.2-1ubuntu1_amd64.deb ...
Unpacking gir1.2-vte-2.91:amd64 (0.52.2-1ubuntu1) over (0.52.2-1ubuntu1~18.04.2) ...
Setting up libvte-2.91-common (0.52.2-1ubuntu1) ...
Setting up libvte-2.91-0:amd64 (0.52.2-1ubuntu1) ...
Setting up gir1.2-vte-2.91:amd64 (0.52.2-1ubuntu1) ...

bdmurray@clean-bionic-amd64:~$ grep -C5 "/etc/issue" /var/log/dist-upgrade/apt-term.log
Warning: Stopping motd-news.service, but it can still be activated by:
  motd-news.timer
Unpacking base-files (10.1ubuntu4) over (10.1ubuntu2) ...
Setting up base-files (10.1ubuntu4) ...

Configuration file '/etc/issue'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ? Your options are:
    Y or I : install the package maintainer's version
    N or O : keep your currently-installed version
      D : show the differences between the versions
      Z : start a shell to examine the situation
 The default action is to keep your current version.
*** issue (Y/I/N/O/D/Z) [default=N] ? y
Installing new version of config file /etc/issue ...

However, given the potential for a regression some additional testing should be done.

This bug was fixed in the package ubuntu-release-upgrader - 1:18.10.6

---------------
ubuntu-release-upgrader (1:18.10.6) cosmic; urgency=medium

  * DistUpgradeQuirks.py:
    - make sure that snapd is installed before trying to use it.
      (LP: #1783589)
    - update the view with information regarding the progress of snaps being
      installed. (LP: #1783593)
    - when checking for connectivity to the snap store use C.UTF-8 for the
      language so error message matching works. (LP: #1783738)
  * DistUpgradeController.py:
    - Remove debs from apt's "Dir::Cache::archives" folder after the upgrade
      has completed successfully.
    - Add a telemetry marker to report the time to process migration from deb
      to snaps.
  * DistUpgrade/DistUpgradeViewGtk3.py: call vte's terminal.feed_child() with
    the correct parameters thereby fixing a crash. (LP: #1780501)

 -- Brian Murray <email address hidden> Wed, 01 Aug 2018 15:06:53 -0700

Here is a crash in terminator which occurs with the version of vte2.91 in bionic-updates:

https://errors.ubuntu.com/problem/13b5b72882f409d035b674102f45b0eae419ab2d

Here are some in guake:

https://errors.ubuntu.com/problem/a15ca647934a403f748dad5b3400f805a59a0de7 (count of 52)
https://errors.ubuntu.com/problem/8f1005750706310a47d63b1eaae0b6e823a9194f (count of 93)
https://errors.ubuntu.com/problem/6607888ac9d7c05d5785334483ea2cfe22f9e158 (count of 64)

Looking at the guake crashes I was able to recreate the crash using the following command:

/usr/bin/guake -n -r htop -e htop

I then upgraded to the versions of gir1.2-vte-2.91, libvte-2.91-0, and libvte-2.91-common in -proposed and those also fixed this crash.

Hey guys,

Hope I'm not speaking up too late.

It's indeed a bit nasty situation we're in, and I am also a little bit responsible for this. When updating the PCRE fixes for 0.52 I focused on the actual work, and ignored (just ported blindly in "autopilot" mode) the annotation changes. I should've stopped and asked "what the heck is this?". Sorry for that!

In the mean time vte itself is also somewhat responsible for the mess by silently fixing its python bindings in backwards incompatible ways. (Mainstream 0.52 fixed feed_child() not to take a third parameter, it was broken before and required an explicit length. The same happened to feed_child_binary() in 0.46 which may have been the reason for someone to (accidentally or intentionally) sneak in the revert of this to the PCRE patch.)

So a "faulty" vte update has already been released for bionic, breaking 4 packages we're aware of (ubuntu-release-upgrader, terminator, guake, cubic). Maybe there's one or two more at most, but I don't think so. On the other hand, this change "fixes" vte to be like mainstream, which is a huge advantage for anyone wishing to manually install vte-based software.

I assume you don't intend to carry these changes forever. Brian Murray from comment 10:

> I agree that having downstream API differences is bad and that it shouldn't be fixed for cosmic

I can only parse this sentence assuming a typo: "*should* be fixed for cosmic", am I right, or what am I missing?

ubuntu-release-upgrader has already been fixed to cope with either signatures if I understand correctly, and the other three will also need to be fixed eventually.

At this point, if it was dozens of packages that broke, I'd agree with reverting the vte change. If vte received a broken change, I'd agree with reverting it.

However, vte actually received a fix, which happened to break 4 apps that were strictly speaking broken (to adjust to broken vte), and they'll need to get fixed anyway.

In this situation I find it much clearer to escape forward rather than retreat, and just fix those 4 broken packages. For all of them it's a trivial change, you don't even need to handle two different APIs with a try-expect, you can just go for the final correct version (two parameters) straight away.

I understand that the overhead right now is somewhat bigger for releasing 4 updates rather than 1. However, it results in a much clearer situation that gets rid of the differences from mainstream, more easily supportable for the forthcoming almost 5 years and is much better for any user who hack around on their system (i.e. install vte-based apps from various sources). I do believe that – cleanliness and better overall quality of bionic with this approach over the other put aside – it's even cheaper to fix those 4 packages now, than to maintain and support this crazy oddness for the rest of bionic's lifetime.

Could you guys please seriously consider fixing this situation in the direction where all packages look like they should, rather than none of them? Thanks a lot!

New here, found this while trying to resolve a problem running execute-command in guake. I enabled the proposed repository and upgraded to the version of vte2.91 found there. Problem was indeed fixed. Thanks for all your hard work guys!

On Fri, Aug 03, 2018 at 11:06:38AM -0000, Egmont Koblinger wrote:
> Hey guys,
>
> Hope I'm not speaking up too late.
>
> It's indeed a bit nasty situation we're in, and I am also a little bit
> responsible for this. When updating the PCRE fixes for 0.52 I focused on
> the actual work, and ignored (just ported blindly in "autopilot" mode)
> the annotation changes. I should've stopped and asked "what the heck is
> this?". Sorry for that!
>
> In the mean time vte itself is also somewhat responsible for the mess by
> silently fixing its python bindings in backwards incompatible ways.
> (Mainstream 0.52 fixed feed_child() not to take a third parameter, it
> was broken before and required an explicit length. The same happened to
> feed_child_binary() in 0.46 which may have been the reason for someone
> to (accidentally or intentionally) sneak in the revert of this to the
> PCRE patch.)
>
> So a "faulty" vte update has already been released for bionic, breaking
> 4 packages we're aware of (ubuntu-release-upgrader, terminator, guake,
> cubic). Maybe there's one or two more at most, but I don't think so.

This "faulty" vte update is a regression though in that something which
used to work, calling feed_child() with three parameters, no longer
does and causes other software to break. This is not in line with the
Ubuntu Stable Release Updates policy. Additionally, we don't know what
other software, packaged or not, would be broken by this change and that
is why its best to keep what we had when Ubuntu 18.04 was released.

> On the other hand, this change "fixes" vte to be like mainstream,
> which is a huge advantage for anyone wishing to manually install
> vte-based software.

> I assume you don't intend to carry these changes forever. Brian Murray
> from comment 10:
>
> > I agree that having downstream API differences is bad and that it
> shouldn't be fixed for cosmic
>
> I can only parse this sentence assuming a typo: "*should* be fixed for
> cosmic", am I right, or what am I missing?

I mean that we should leave vte2.91 alone for cosmic.

> ubuntu-release-upgrader has already been fixed to cope with either
> signatures if I understand correctly, and the other three will also need
> to be fixed eventually.

They will need to fixed but that should happen to the packages in cosmic
and not require SRUs for an unknown quantity of packages Ubuntu 18.04.

> At this point, if it was dozens of packages that broke, I'd agree with
> reverting the vte change. If vte received a broken change, I'd agree
> with reverting it.
>
> However, vte actually received a fix, which happened to break 4 apps
> that were strictly speaking broken (to adjust to broken vte), and
> they'll need to get fixed anyway.
>
> In this situation I find it much clearer to escape forward rather than
> retreat, and just fix those 4 broken packages. For all of them it's a
> trivial change, you don't even need to handle two different APIs with a
> try-expect, you can just go for the final correct version (two
> parameters) straight away.

> I understand that the overhead right now is somewhat bigger for
> releasing 4 updates rather than 1. However, it results in a much clearer
> situati...

The verification of the Stable Release Update for vte2.91 has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package vte2.91 - 0.52.2-1ubuntu1~18.04.2

---------------
vte2.91 (0.52.2-1ubuntu1~18.04.2) bionic; urgency=medium

  * Revert the changes to revert-pcre2.patch in the previous SRU since they
    introduced API incompatibilies which aren't OK in an SRU (LP: #1780501).

 -- Iain Lane <email address hidden> Mon, 16 Jul 2018 18:37:09 +0100