Cosmic: Mixxx 2.1.3 is not stable with Qt5

...one more reason to not revert the migration from Qt4 to Qt5 in 2.2.0.

This package should be immediately updated to the latest stable release Mixxx 2.2.0
This is the first version that is stable with Qt5.
The package currently shipped with Cosmic is a untested alpha Version.

I have just emailed Matteo.
Is he also responsible for this bug?

Ok, the Debian upstream packages are updated:
https://packages.debian.org/buster/mixxx
https://packages.debian.org/sid/mixxx

Who is responsible to fetch these changes for Ubuntu? It is really a bad situation without this.

Ping! This bug is still pending. Who can update the Mixxx package?
https://packages.ubuntu.com/cosmic/mixxx

Hello, I was asked to take a look at this bug.

In Ubuntu, we have a Stable Release Update process which is in place to ensure a smooth update process for our users: https://wiki.ubuntu.com/StableReleaseUpdates

Part of this process involves ensuring that only bugfixes and patches suitable for stable releases are accepted. The diff between 2.1.3 and 2.2.0 is quite large, and I am unsure whether including a .0 release is suitable for an SRU. That being said, I am personally willing to drive this through to completion if:
 1. I get confirmation that 2.2.0 and not 2.1.7 is what should be backported.
 2. The problems are large enough that it cannot be solved by patching 2.1.3.
 3. Any other changes involved are low-impact and will not cause behavioral regressions compared to 2.1.3 for the end user.

Additionally, this bug description needs to be modified to follow the template in the wiki page linked above, which gives proper justification for such an update. Please do read through the wiki page to get an idea of what they're looking for.

Note: while I am an Ubuntu Core Developer, I am not a member of the SRU Team, and I have no final say on whether this is accepted. I have subscribed the SRU Team in case someone wants to give their feedback.

Thank you for reporting the bug, and I apologize for the delay. I hope we can drive this through to completion.

Thank you for taking care here.

> Part of this process involves ensuring that only bugfixes and patches suitable for stable releases are accepted.

This rule does not apply here, the released binaries are unusable because it is a Qt4 Version build with Qt5 and suffers MANY bugs. Here it is the question how to revert the error.
Solutions would be to go to a 2.1.3 Qt4 build a 2.1.7 Qt4 build or a 2.2.0 Qt5 build.

I recommend to update to 2.2.0 QT5 and follow Debian upstream to avoid that the update drew all the qt4 libraries, which might be annoying compared to some feature updates.

We can certainly consider this _if_ the current version in Cosmic really is a problem for users that cannot practically be fixed any other way. But please could you actually demonstrate that? For the only bug actually linked, surely a cherry-pick would be lower regression risk for existing users?

> We as Mixxx team do not support 2.1.x qt5 builds. So the user is pretty alone with issues with throws a bad light on Mixxx.

We certainly appreciate involvement from upstreams, but I don't think this qualifies as an SRU justification in itself. "We...do not support" is certainly not a reason. You're entitled to choose not to support what you wish (or more specifically you get to choose what you want to spend your time on), but such a position is certainly not a justification to ignore the stability* promises we make to Ubuntu users.

The focus on stable release updates must be on users, and their expectation is of stability* in that, in general, no major version updates are expected to pull the rug out from under their feet. What can we do to fix these problems for this set of users? How difficult would it be to cherry-pick bug fixes for them? If cherry-picking is practical and would avoid problems for users successfully using the Cosmic package today, why aren't we doing that?

Note that there are only four months left of support for Cosmic, and that Disco will be out soon. It seems to be that the benefit of a major update now would be limited, compared to the consequences of ignoring our stability* promise to users. A user who tries to start using the Cosmic package in three weeks will have the option of using Disco. A user who is successfully using the Cosmic package today might never use Ubuntu again if we break our stability promise and this destroys their performance.

These are the promises we make to users for packages that ship as part of the distribution. I can understand that you may find this frustrating, but please remember that users of stable* distributions _want_ this policy. If you would prefer to have control of your own release management directly to your users, you might consider shipping a snap (http://snapcraft.io/).

* Stability means different things to different people. In this context I don't mean "no bugs"; obviously there are bugs and achieving no bugs is generally not practical. In this context I mean "doesn't suddenly change behaviour".

To be clear, I'm not ruling out an update to 2.2.x in Cosmic, but I think there needs to be a real justification that includes an explanation of the actual bugs actually impacting users in Cosmic and why the fixes cannot practically be cherry-picked.

> Solutions would be to go to a 2.1.3 Qt4 build a 2.1.7 Qt4 build or a 2.2.0 Qt5 build.

Sorry, I missed your 2.1.x proposals earlier. My previous comments is referring only to the 2.2.x proposal. For 2.1.x, please see the document Simon linked for details of the policy (specifically https://wiki.ubuntu.com/StableReleaseUpdates#New_upstream_microreleases).

Thank you Robert for you explainations. I appreciate you efforts to keep Ubuntu as stable as possible.

However this case is special. You can't not treat the current repository version as stable. It is basically a untested 2.2 alpha with a wrong version number, that slipped into the Debian repros by an accident and pulled here.

I am just asking to revert this accident and replace it with either 2.1 build with Qt4 or 2.2 with QT5. Both versions where been sorrowly tested and can be treated as stable.

It is fatuos to to explain here every thing left issue we have fixed when porting to Qt5. I have already linked the tracked bugs above.
It would be much more risky to backport all these and more fixes into 2.1, which would be still an untested alpha.

The best user experience would be to switch to 2.2 because this does not involve installing whole Qt4. 2.2 is our Qt5 release the mentioned bugfixes plus additional changes linked above.

During the release you have followed Debian, they have fixed the issue by upgrading to 2.2
 So it is reasonable to follow them again. Downgrading Qt with 2.1 would be also work for me.

I there anything left to do for me here? Who is the responsible package maintainer?

> Thank you Robert for you explainations. I appreciate you efforts to keep Ubuntu as stable as possible.

Thanks, and likewise I appreciate you caring for the user experience of this package in Ubuntu.

> I there anything left to do for me here?

I think there are a bunch of outstanding questions you have yet to answer. See Simon's question 3 above, for example.

> During the release you have followed Debian, they have fixed the issue by upgrading to 2.2
 So it is reasonable to follow them again.

It is reasonable for Disco, and this has already been done. Your logic does not follow for Cosmic, which is now a stable release.

> Who is the responsible package maintainer?

Packages in Ubuntu are team-maintained. There is nobody specific you can pin as "responsible". Somebody will need to volunteer the update. It sounds like Simon may be willing to do this for you. For stable releases, the SRU team will make the final decision on any updates as Simon mentioned.

I'm still open to approving any of your proposed options, but I don't think I yet understand adequately enough what the impact would be to existing users for any of them. I understand your position and your opinion, and this is valuable information since you're the expert. However, I still need the underlying facts, and these are still not clear to me.

If you don't want to go there, and don't feel that you need to justify your position, then I suggest again that you look at snaps for packaging instead - they give you, as upstream, full control over your release management.

I have update the initial post with all the requested info so far. Is anything missing?

The mayor argument is that the currently released version has bypassed all quality checks of the Mixxx community.

The outstanding issue among others is the leak if GUI responds and possible freezes.
https://bugs.launchpad.net/bugs/1805559
https://bugs.launchpad.net/bugs/1789059

There is also a critical issue with HID scaling:
https://bugs.launchpad.net/bugs/1744861
https://bugs.launchpad.net/bugs/1530697

Your bug description is a really good start, but I have a few more questions before we can proceed with this.

The goal from here is to identify problems with the update that may arise. I will admit to not being intimately familiar with Mixxx, but the idea is to not add any bugs in the process. If something was buggy in 2.1.3 and is still buggy in in 2.2.0, that is acceptable for an SRU, but our goal is to identify bugs that exist in 2.2.0 but don't exist in 2.1.3. Usually, a review of the diff between 2.1.3 and 2.2.0 is sufficient to see which parts of the package have been affected, and those parts are then tested (which makes cherry-picking patches preferable), but as I noted before, the diff is very large. My point here is, the potential ramifications of this update should be verbosely stated in the bug report, and a somewhat brief analysis of the features and changes between the two versions should be completed.

That being said, in my personal opinion, your justification warrants an update; please state some of the issues in the bug description. How badly is the package broken currently? Instead of linking the bug page, it helps reviewers take a look at the potential benefits of including this update. As a packager, I can then add the bugs to the changelog, which involves them in the process, and we can make sure that each of the bugfixes individually have been addressed.

Thank you for your patience on this, we're getting closer to a solution. :)

Mixxx doesn't follow a stable/unstable release process. All mixxx releases are considered stable and we recommend users always upgrade to the latest version. Any release may introduce new bugs, but many more are also fixed. We do offer point releases for a limited amount of time to address critical bugs, but most bugfixes are only committed to master.

The biggest change in 2.2 is linking against QT5 by default instead of QT4. Mixxx has been working on the migration from QT4 to QT5 for a while, and as of version 2.1.x, the QT5 build was not ready, so all of our binaries shipped linked against QT4. Any public release of 2.1 linked against QT5 is a mistake and will result in a bad user experience.

We have a quite good experience with 2.2.0 on Linux since the release, no QT unrelated regressions.

We have some regressions tracked here https://launchpad.net/mixxx/+milestone/2.2.1 due to QT5 though.
The current Cosmic version already suffers these issues.

I have tried to improve the initial post. Is it OK like that?

I'll upload this on Monday.

Thank you very much.

Uploaded to the queue, it's up to the SRU team now.

The cosmic package is still 2.1.3~dfsg-1
What can I do to make progress here?
@Robie Basak: can you have a look?

Thanks again for actively working on this! I'd certainly like Robie to take a look at this upload as he was the one originally involved in handling the SRU. But a few of my second-opinion cents here.

With my SRU hat on, it's very difficult handling such huge new upstream release updates to our stable series. I do have some questions before I can actually formulate my own opinion here:
The issues mentioned to be caused by the 'buggy' version do seem to be high-impact, but from what I understand the application is still usable - is that correct? Because if it's not and the current issues are simply making the package unusable, that is certainly a different story. Many of our strict SRU rules are simply to guard from regressions. But in cases where the experience is anyway 'regressed', we can be a bit less strict for the good of everyone.

Looking at the source, it looks to me that there is quite a lot of unit-tests defined - are those being run during build time? How does the test coverage story for mixxx look like? Do all bugfixed/features come with unit-testing always?

Finally, if we decide to include this SRU in our stable series, the manual testing story will have to be improved. This is a lot of changes and just the tests mentioned in the [Test Case] currently are not enough in my opinion to verify if the upload is good to be released. First thing I'd recommend is: the upload links to 4 other bugs (as seen in the bug description here), so what I'd like seeing is getting all 4 of those updated for SRU purposed (i.e. adding all the needed information as per the SRU template [1]). Each of those would have to be verified separately anyway. Also, I'd propose adding some general-usage test cases to the Test Case here. Something to make sure that the application is still working.

[1] https://wiki.ubuntu.com/StableReleaseUpdates#SRU_Bug_Template

> The issues mentioned to be caused by the 'buggy' version do seem to be high-impact, but from what I understand the application is still usable - is that correct?

It depends on the definition of usable. You can start the application play tracks, but it may stop a party, which is the most critical bug we can think of. We actually do not know the effects very well, because the cosmic build linked with Qt5 has bypassed our quality management. The Mixxx 2.1.3 was tested and released with Qt4.8 only. It is strongly recommended to not use this build because of that.

> Looking at the source, it looks to me that there is quite a lot of unit-tests defined - are those being run during build time?

Most unit tests are unit testing against regressions inside the audio engine. The tests are linked in a own binary, not the production binary build on Launchpad, so we cannot detect link and packaging issues like we face here. We execute them automatically in each PR and before release on our own CI.

But IMHO the situation is way better now, thanks to the delay here: Mixxx 2.2.0 was released on 11th January, almost 6 month ago and it has landed in various distros without mayor complains:
https://repology.org/project/mixxx/versions

> Finally, if we decide to include this SRU in our stable series, the manual testing story will have to be improved.

Yes, some "smoke tests" with the production binary should be done help such situation is not repeated.

I will update the test cases and the linked bugs.

Done.

Any updates?

This is obsolete, closing this.

Mixxx now uses GitHub for bug tracking. This bug has been migrated to:
https://github.com/mixxxdj/mixxx/issues/9526