node-tap in bionic fails autopkgtests due to timeouts

Patch for Bionic.

Failure examples from autopkgtest log [1].

The first test killed by timeout exhibits this message:

 1) test/run.js dump config stuff unknown short opt timeout!:
     Error: timeout!
      at emit (/usr/lib/nodejs/signal-exit/index.js:77:11)
      at process.listener (/usr/lib/nodejs/signal-exit/index.js:91:7)

Which stops the other tests planned in the *.js file from running,
so there are message like the following:

  2) test/run.js dump config stuff undefined trailing value opts test count !== plan:

      test count !== plan
      + expected - actual

      -0
      +5

This test usually fails if it's timeout value is too short (manually set it to confirm):

  5) test/spawn.js timeout KILL should be equal:

      Error: should be equal
      + expected - actual

      -1..0 # no tests found
      +SIGTERM
      +
      +not ok 1 - timeout!
      + ---
      + expired: killa
      + ...
      +1..1
      +# failed 1 test

Then the first pattern is repeated:

 6) test/tap.js stdout epipe timeout!:
     Error: timeout!
      at emit (/usr/lib/nodejs/signal-exit/index.js:77:11)
      at process.listener (/usr/lib/nodejs/signal-exit/index.js:91:7)

  7) test/tap.js stdout epipe test count !== plan:

      test count !== plan
      + expected - actual

      -1
      +3

[1] https://objectstorage.prodstack4-5.canonical.com/v1/AUTH_77e2ada1e7a84929a74ba3b87153c0ac/autopkgtest-bionic/bionic/amd64/n/node-tap/20180918_192543_bdbd7@/log.gz

For the record,

Funny enough, when building on PPA an asynchronous/parallel
test-case failed once, and passed with a rebuild later [1, 2]

I guess it's just another intermittent testcase, since
these patches only increase the timeout value at build
time, and NOT anything related to non-packaging code,
nor any change that would affect that code at runtime.

If that happens again, we can just trigger a rebuild.
And hopefully not see any error on autopkgtests again.

Build logs:

[1] https://launchpadlibrarian.net/389472632/buildlog_ubuntu-bionic-amd64.node-tap_11.0.0+ds1-2ubuntu1_BUILDING.txt.gz

  1) test/run.js parallel should be equal:

      Error: should be equal
      + expected - actual

       ry2
       f1
       f2
       start
      -end
       start
       end
      +end

      at run (test/run.js:555:7)

[2] https://launchpadlibrarian.net/389473930/buildlog_ubuntu-bionic-amd64.node-tap_11.0.0+ds1-2ubuntu1_BUILDING.txt.gz

The attachment "node-tap-autopkgtest-timeout.debdiff" seems to be a debdiff. The ubuntu-sponsors team has been subscribed to the bug report so that they can review and hopefully sponsor the debdiff. If the attachment isn't a patch, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are member of the ~ubuntu-sponsors, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issue please contact him.]

mark fixed for cosmic.

Uploaded to the Bionic queue.

Hello Mauricio, or anyone else affected,

Accepted node-tap into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/node-tap/11.0.0+ds1-2ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Steve,

The obvious verification in this case is confirm autopkgtests pass for bionic-proposed.

However, the 'excuses' page mentions the package is blocked by freeze? [1]
so it didn't run autopkgtests, I guess.

Could you clarify what freeze is that, and whether it should end before 7 days?

Thanks!

"""
node-tap (11.0.0+ds1-2 to 11.0.0+ds1-2ubuntu0.1)

    Maintainer: Ubuntu Developers
    Section: universe/misc
    2 days old
    Not touching package due to block request by freeze (contact #ubuntu-release if update is needed)
"""

[1] https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html

It turns out the package failed to build.
There's this 'test/run.js parallel' test which fails intermittently.
The package is rebuilding now; let's see whether it gets to autopkgtests.

Good news.

The autopkgtests now pass on all architectures [1]
with version 11.0.0+ds1-2ubuntu0.1 (xenial-proposed).

Changing verification tags to done.

[1] https://autopkgtest.ubuntu.com/packages/node-tap

I suppose you mentioning xenial-proposed was a typo (you meant bionic-proposed I guess?). Even though the node-tap autopkgtest are all passing, this upload triggered some failing tests in other packages. Before we can proceed with releasing of this version into -updates, someone needs to make sure those failures are not regressions caused by this upload.

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#node-tap

Hi Lukasz,

Yes, that was a typo :) (doing other SRU verifications at that time.)

This patch only changes build/test-time timeout for the tests, so I guess the failures in other tests are due to other stuff.

In this recent adventure with nodejs uploads and its rdeps autopkgtest, I have seen a number of flaky tests (retest made them pass),
and some other packages that failed due to being run with more recent libs than when they were first built/tested with.

May I ask you to re-run/trigger the autopkgtests for those 4 failures?
From what I can see, there are mentions to timeout strings in the failures, so a re-run may help.

Thank you!

> May I ask you to re-run/trigger the autopkgtests for those 4 failures?

Done. We'll have to wait to see the results.

The verification of the Stable Release Update for node-tap has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package node-tap - 11.0.0+ds1-2ubuntu0.1

---------------
node-tap (11.0.0+ds1-2ubuntu0.1) bionic; urgency=medium

  * Fix autopkgtest errors due to timeouts (LP: #1793612)
    - Debian commit cb4643d ("Increase timeout")
      - debian/rules: set TIMEOUT=3000 and append to TAP
      - debian/tests/runtestsuite-tap: specify '-t 30000'
    - Debian commit 1d76ed9 ("Increase timeout")
      - debian/rules: set CI=1 for TAP test/*.js
      - debian/tests/runtestsuite-tap: specify CI=1
    - debian/control: update-maintainer

 -- Mauricio Faria de Oliveira <email address hidden> Thu, 20 Sep 2018 19:45:42 -0300

Robie, Brian -- thanks!