launchpad-buildd

Slow IO on intel arches in builders, causes FTBFS

Bug #1794059 reported by Dimitri John Ledkov
24
This bug affects 2 people
Affects Status Importance Assigned to Milestone
launchpad-buildd
New
Undecided
Unassigned
ruby2.5 (Ubuntu)
Confirmed
Undecided
Unassigned
strongswan (Ubuntu)
Invalid
Undecided
Unassigned

Bug Description

1) Failure:
TestIO#test_copy_stream_no_busy_wait [/<<PKGBUILDDIR>>/test/ruby/test_io.rb:549]:
r58534 [ruby-core:80969] [Backport #13533].
Expected 0.011999999999998678 to be <= 0.011000000000000001.

Finished tests in 546.343209s, 31.7127 tests/s, 4140.6006 assertions/s.
17326 tests, 2262189 assertions, 1 failures, 0 errors, 58 skips

ruby -v: ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux-gnu]

on amd64/i386, have to retry each ruby2.5 upload a couple of times to "win".

Tags: ftbfs
Dimitri John Ledkov (xnox)
tags: added: ftbfs
summary: - Slow IO on intel arches in builders, cause FTBFS
+ Slow IO on intel arches in builders, causes FTBFS
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

[FAILEDTOBUILD] Failed to build on lgw01-amd64-033 Retry this build

Started 37 minutes ago
Finished 1 minute ago (took 35 minutes, 20.7 seconds)
buildlog (217.0 KiB)

https://launchpad.net/ubuntu/+source/ruby2.5/2.5.1-5ubuntu3/+build/15481916

Revision history for this message
Dimitri John Ledkov (xnox) wrote :
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

I reported the same a few hours before, duping mine to your bug ...

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in ruby2.5 (Ubuntu):
status: New → Confirmed
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

I didn't want to retrigger your upload since I don't own it.

But I spun up three PPAs to check if this build time test is flaky.
The following three just started building:
- https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3431/+packages
- https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3432/+packages
- https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3433/+packages

Result:
- 1 build
  - no errors at all
- 1 build
  - amd64 error (1) of above
  - i386 error (1) of above
- 1 build
  - i386 error (1) of above

So as you already found it really is a flaky test at build time.

I see for this time you already pushed "retry to win"

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Strongswan hit a rather similar issue with a build time test timing out that was always fine in the past and now seems flaky (worked on retry).

Local tests were all successful, if there really is any slowdown on the builders it might be the same issue - so I dupped it here to re-check once there is an idea what to do about it on LP-builders.

Changed in strongswan (Ubuntu):
status: New → Confirmed
Revision history for this message
Dimitri John Ledkov (xnox) wrote :

retried ruby2.5 build 4 times in the archive now.

Colin Watson (cjwatson)
affects: launchpad → launchpad-buildd
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Whatever it was is resolved now, the recent strongswan builds worked fine

Changed in strongswan (Ubuntu):
status: Confirmed → Opinion
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Working fine now for ruby as well https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/3552/+build/15737990

Either the builder setup was fixed or it was - back then - just a real spike of builds driving it into too much over-commitment.

Changed in ruby2.5 (Ubuntu):
status: Confirmed → Invalid
Changed in strongswan (Ubuntu):
status: Opinion → Invalid
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Ha I spoke too soon, i386 it is
Still hit by this issue ...

Changed in ruby2.5 (Ubuntu):
status: Invalid → Confirmed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.