tox lowest-supported-dev tests fail

Thanks, pd9370, for the detailed description.

lowest-supported-dev is supposed to run on the latest supported Python version, which may vary depending on the cloud-init version you are hacking with. One can find the minimum python version changes in the following documentation:

https://cloudinit.readthedocs.io/en/latest/topics/contributing.html#python-support

Hi pd9370, thanks for reporting this.

I know little about Devuan (besides the tagline), but I assume that tox is not actually used during the package build process, right? On Ubuntu/Debian, I think sbuild just invokes a python3 -m pytest -v tests/unittests with system dependencies. The failing test exists in tox to verify the oldest supported distro dependency versions (using our CI infrastructure or locally with pyenv/deadsnakes/etc).

Agreed, the issues you described may be worked around on newer versions, but the intended use case currently is running on 3.6 so I'm not sure working around this issue on other versions really makes sense - what does it buy us? For what it's worth, that test env fails when executed with system-provided python on the latest Ubuntu release (Jammy).

It might make sense to drop that environment from the tox envlist to reduce noise.

Thoughts?

Alberto, Brett and James - thanks for looking into this.

One clarification: as I'm hoping eventually to submit changes to add devuan support to cloud-init, I'm working on a clone of the github repo and keeping the clone up-to-date - so hopefully at the moment I'm doing my investigations at the tip of the 'main' branch. When I know what's required for devuan support, I'll create a fork in my github account and eventually create a pull request.

Re. the jinja2 problem - I've just tried editing tox.ini to remove the markupsafe==2.0.1 line and changing the jinja2 line to a later version. It appears that jinja2 version 3.0.0a1 was the first one in which the soft_unicode problem was fixed (with the deps line in tox.ini set to "jinja2==3.0.0a1", the lowest-supported-dev tests run clean; with it set to "jinja2==2.11.3" (the last version before v3) they fail).

Note that I'm using the following command to find all the available releases of jinja2 (apparently it's the usual way of doing this in pip3):

pip3 install --use-deprecated=legacy-resolver jinja2==

I haven't as yet tried testing this on a distro/release which uses python<3.9 (I'll do this soon). I've also been adding devuan support to walinuxagent, and unfortunately that requires python 3.9 (using lower versions of python 3, walinuxagent can't distinguish between devuan and debian).

When you're ready to upstream requirements for Devuan, feel free to open a PR on Github. In the meantime, we've removed[1] that particular test from the envlist.

With respect to changing that particular set of test requirements: Again, I don't think we want to. It currently serves its purpose and works as is for its intended platform. However, if you feel the need to add a new environment to tox.ini for testing the oldest supported Devuan dep equivalents from pypi, I don't see a reason why we couldn't do that. Either way, we can have that discussion when a PR is opened to upstream support for Devuan.

I don't think there is anything else for upstream to do, so I've marked this a "fix committed".

[1] https://github.com/canonical/cloud-init/commit/5cff3af1011ab562155b7ce42c599ead681f0fb2

Hi Brett - thanks for that. I can confirm that with the latest changes, under devuan chimaera, the tox test suite now runs clean. I'm happy to concur that this is now fixed.

This bug is believed to be fixed in cloud-init in version 22.3. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/3996