[MIR] python-autocommand, python-inflect, pydantic

Review for Package: python-autocommand

[Summary]
This is the typical small python lib, that is well packaged and tested given
its size and scope. There is a bit of uncertainty as it is rather new, but
the openstack team has experience in maintaining those. If - in the future - it
isn't maintained in Debian I'm convinced they will just do fine themself.

MIR team ACK

This does IMHO not need a security review

List of specific binary packages to be promoted to main: python3-autocommand
Specific binary packages built, but NOT to be promoted to main: none

[Duplication]
The only similar function I'Ve found is in python3-argh and that is also
in universe. Therefore there is no other package in main providing the same
functionality.

[Dependencies]
OK:
- no other Dependencies to MIR due to this
- no -dev/-debug/-doc packages that need exclusion
- No dependencies in main that are only superficially tested requiring
  more tests now.

Problems: None

[Embedded sources and static linking]
OK:
- no embedded source present
- no static linking
- does not have unexpected Built-Using entries
- not a go package, no extra constraints to consider in that regard
- not a rust package, no extra constraints to consider in that regard

Problems: None

[Security]
RULE: - Determine if the package may have security implications or history.
RULE: Err on the side of caution.
RULE: - If the package is security sensitive, you should review as much as you
RULE: can and then assign to the ubuntu-security team. The bug will then be
RULE: added to the prioritized list of MIR security reviews.

OK:
- history of CVEs does not look concerning
- does not run a daemon as root
- does not use webkit1,2
- does not use lib*v8 directly
- does not parse data formats (files [images, video, audio,
  xml, json, asn.1], network packets, structures, ...) from
  an untrusted source.
  Essentially it only parses the code it is imported in, if you have access
  to the code then there is no need to exploit this library anymore.
- does not open a port/socket
- does not process arbitrary web content
- does not use centralized online accounts
- does not integrate arbitrary javascript into the desktop
- does not deal with system authentication (eg, pam), etc)
- does not deal with security attestation (secure boot, tpm, signatures)
- does not deal with cryptography (en-/decryption, certificates, signing, ...)

Problems: None

[Common blockers]
OK:
- does not FTBFS currently
- does have a test suite that runs at build time
  - test suite fails will fail the build upon error.
    it runs pytest against all enabled python versions
- does have a non-trivial test suite that runs as autopkgtest
  runs the upstream test in autopkgtest context
- This does not need special HW for build or test
- no new python2 dependency
- Python package, but using dh_python

Problems: None

[Packaging red flags]
OK:
- Ubuntu does not carry a delta
- symbols tracking not applicable for this kind of code.
- d/watch is present and looks ok
- Upstream update history is good
- the current release is packaged
- promoting this does not seem to cause issues for MOTUs that so far
  maintained the package (as I said it is new...

Review for package: src:python-inflect

[Summary]
This is a string processing module based upon the Perl module
Lingua::EN::Inflect. Upstream activity is looking good, but it's very
outdated in Debian and Ubuntu. Furthermore, we're missing automated
integration tests. Please update and improve the testing situation,
before we can re-consider it for promotion.

MIR team ACK under the constraint to resolve the below listed
required TODOs and as much as possible having a look at the
recommended TODOs.

This does not need a security review

List of specific binary packages to be promoted to main: python3-inflect
Specific binary packages built, but NOT to be promoted to main: <None>

Notes:
It processes strings which are explicitly passed into the inflection engine,
returning the modified string. This consists of parsing the input, but I feel
like it does not need security review, as the input is known/trusted.

Required TODOs:
#1 Update to the current version 6.0.2 (as of 2023-01-17)
#2 Agree to keep it updated/maintained in Debian/Ubuntu
#3 Add automated integration tests (autopkgtests), at least running the unit-tests
   at runtime, to check the installed version

Recommended TODOs:
#4 The package should get a team bug subscriber before being promoted
#5 Fix warnings during build:
   SyntaxWarning: 'str' object is not callable; perhaps you missed a comma?
   SetuptoolsDeprecationWarning: setup.py install is deprecated.
#6 Drop python3-nose dependency: https://bugs.debian.org/1018513

[Duplication]
There is no other package in main providing the same functionality.
There's also src:inflection in universe, but python-inflect seems to be the better (upstream) alternative.

[Dependencies]
OK:
- no other Dependencies to MIR due to this
  - SRCPKG checked with `check-mir`
  - all dependencies can be found in `seeded-in-ubuntu` (already in main)
  - none of the (potentially auto-generated) dependencies (Depends
    and Recommends) that are present after build are not in main
 - no -dev/-debug/-doc packages that need exclusion
- No dependencies in main that are only superficially tested requiring
  more tests now.

Problems: None

[Embedded sources and static linking]
OK:
- no embedded source present
 - no static linking
- does not have unexpected Built-Using entries
- not a go package, no extra constraints to consider in that regard
- No vendoring used, all Built-Using are in main
- not a rust package, no extra constraints to consider in that regard
- Does not include vendored code

Problems: None

[Security]
OK:
- history of CVEs does not look concerning
- does not run a daemon as root
- does not use webkit1,2
- does not use lib*v8 directly
- does not parse data formats (files [images, video, audio,
  xml, json, asn.1], network packets, structures, ...) from
  an untrusted source.
- does not open a port/socket
- does not process arbitrary web content
- does not use centralized online accounts
- does not integrate arbitrary javascript into the desktop
- does not deal with system authentication (eg, pam), etc)
- does not deal with security attestation (secure boot, tpm, signatures)
- does not deal with cryptography (en-/decryption, certificates, signi...

python-autocommand still needs ubuntu-openstack subscribed to it.

[Adding bug task for jaraco.text with 'update-excuse' tag for migration tracking. As mentioned in bug description this MIR will solve jaraco.text's migration issue.]

ubuntu-openstack subbed to python-autocommand

python-inflect has been updated to 6.0.4 in Debian; this has expanded the scope of the MIR to include pydantic

>> pydantic <<

[Availability]
The package pydantic is already in Ubuntu universe.
The package pydantic builds for the architectures (arch:all) it is designed to work on.

[Rationale]
New runtime dependency for jaraco.text which is already in Ubuntu main.

[Security]
No security history

- no `suid` or `sgid` binaries
- no binaries generally (python module)
- no services
- no ports opened
- no extensions to security sensitive software

[Quality assurance - function/usage]
- The package works well right after install

[Quality assurance - maintenance]
No open bugs of importance in Ubuntu or Debian
Healthy release activity upstream

[Quality assurance - testing]
Package includes unit tests which are executed as part of the package build and fail the package build as needed
No autopkgtests executed for this package.

[Quality assurance - packaging]
- d/watch present and works
- d/control defines a correct maintainer field
- no lintian overrides
- no debconf questions
- packaging is simple and easy to build (pybuild)

[UI standards]
N/A - not an UI application.

[Dependencies]
All in main

[Standards compliance]
No policy violations

[Maintenance/Owner]
Maintainer in Debian
ubuntu-openstack to maintain in Ubuntu.
Currently 1.10.x - new 2.0 release available upstream

Re-Review for source package: python-inflect

Even though this is a big version bump (2.1.0-4 -> 6.0.4-1) the initial MIR review in comment #2 still holds true. The diff between 2.1 and 6.0 looks sane and manageable. Nothing unexpected (like a full rewrite or anything).

Most of the required TODOs have been addressed:
#6 resolved
#5 resolved
#4 resolved
#3 resolved
#2 downgrade to Recommended
#1 downgrade to Recommended

=> MIR team ACK. No need for security review.
=> This is after its dependencies are resolved:
-- pydantic MIR (LP: #2001699)
-- -- python-typing-extensions MIR (LP: #2002821)

I'd still recommend to look into the following issues:

#1 Update to the current version 7.0.0 (as of 2023-07-11)
-- we're only 3 months behind now, that seems OKish.
-- Therefore, this requirement is downgraded to a Recommended TODO

#2 Agree to keep it updated/maintained in Debian/Ubuntu
-- Maintenance in Debian seems sporadic, the OpenStack team might want to help to keep the package up-to-date
-- Though, good work is being done (adding autotests, fixing deprecation warnings), so this requirement is downgraded to a Recommended TODO

#7 Lintian warnings, somebody might look into:
-- I: python-inflect source: older-debian-watch-file-standard 3 [debian/watch]

Review for Source Package: pydantic

[Summary]
MIR team ACK under the constraint to resolve the below listed
required TODOs and as much as possible having a look at the
recommended TODOs.
This does need a security review. I'll assign ubuntu-security after required TODO's are addressed.
List of specific binary packages to be promoted to main: python3-pydantic

Notes:
Required TODOs:
1. There not seem be any autopkg tests running. The package does have a test suite, so this
   could be used in autopkgtest
2. The version in debian/ubuntu is 1.10.4 but the upstream latest version is 2.0.2.
   Please bump to latest version.
3. The package should get a team bug subscriber before being promoted

[Duplication]
There is no other package in main providing the same functionality.
The package is required in main as a dependency or jarco.txt which is in main.

[Dependencies]
OK:
- no other Dependencies to MIR due to this
  - pydantic checked with `check-mir`
  - all dependencies can be found in `seeded-in-ubuntu` (already in main)
  - none of the (potentially auto-generated) dependencies (Depends
    and Recommends) that are present after build are not in main
- no -dev/-debug/-doc packages that need exclusion
- No dependencies in main that are only superficially tested requiring
  more tests now.

Problems: None

[Embedded sources and static linking]
OK:
- no embedded source present
- no static linking
- does not have unexpected Built-Using entries
- not a go package, no extra constraints to consider in that regard
- not a rust package, no extra constraints to consider in that regard
- Does not include vendored code

Problems: None

[Security]
OK:
- history of CVEs does not look concerning
- does not run a daemon as root
- does not use webkit1,2
- does not use lib*v8 directly
- does not open a port/socket
- does not process arbitrary web content
- does not use centralized online accounts
- does not integrate arbitrary javascript into the desktop
- does not deal with system authentication (eg, pam), etc)
- does not deal with security attestation (secure boot, tpm, signatures)
- does not deal with cryptography (en-/decryption, certificates, signing, ...)

Problems:
- does parse data formats (files [images, video, audio,
  xml, json, asn.1], network packets, structures, ...) from
  an untrusted source.

[Common blockers]
OK:
- does not FTBFS currently
- does have a test suite that runs at build time
  - test suite fails will fail the build upon error.
- This does not need special HW for build or test
- no new python2 dependency
- Python package, but using dh_python

Problems: None
- does not have a non-trivial test suite that runs as autopkgtest

[Packaging red flags]
OK:
- Ubuntu does not carry a delta
- symbols tracking not applicable for this kind of code.
- debian/watch is present and looks ok (if needed, e.g. non-native)
- Upstream update history is good
- Debian/Ubuntu update history is slow
- promoting this does not seem to cause issues for MOTUs that so far
  maintained the package
- no massive Lintian warnings
- debian/rules is rather clean
- It is not on the lto-disabled list

Problems: None
- the current release is not packaged

[Upstream red fla...

pydantic TODO's:

1. There not seem be any autopkg tests running. The package does have a test suite, so this
   could be used in autopkgtest

I've enabled autopkgtests for this package in the most recent upload to noble.

2. The version in debian/ubuntu is 1.10.4 but the upstream latest version is 2.0.2.
   Please bump to latest version.

The update to 2.x series has much wider impacts over and above this scope of this MIR - there are quite alot of breaking changes so I think this is better dealt with in Debian first rather than pushing to 2.x in Ubuntu.

That said I have bumped the version to 1.10.13 which is the most recent release on the 1.x series.

3. The package should get a team bug subscriber before being promoted

ubuntu-openstack will be added once approved.

Please make a separate Launchpad bug for each MIR request (in the future). This makes tracking individual MIRs easier.

https://github.com/canonical/ubuntu-mir/pull/15

MIR team ack on
- not going to 2.x for now, thanks for explaining
- thanks for adding tests
- thanks for updating to the most recent 1.x

Thereby the MIR request needs for "pydantic" are handled.

It still needs the security review, but otherwise pydantic is ready.

I reviewed pydantic 1.10.13-0ubuntu1 as checked into noble. This shouldn't be
considered a full audit but rather a quick gauge of maintainability.

pydantic is a fast and extensible data validation and settings management
library in python. It can be combined with popular IDEs and linters, and its
powered by type hints, making its use easy.

- CVE History
 - CVE-2021-29510 (7.5 CVSS, high)
   - fixed with backports in multiple versions (1.8.2, 1.7.4, 1.6.2)
   - fixed and immediately reviewed by upstream
   - tests provided as PoC for vulnerability
 - CVE-2020-10735 (7.5 CVSS, high)
   - python vulnerability that was affecting the package
   - patched immediately by upstream when made public by python
   - tests provided as PoC for vulnerability
- Build-Depends
 - default and widely-used python libraries, all in main
- pre/post inst/rm scripts
 - prerm script: backported from dh_python, secure
 - postinst script: backported from dh_python, secure
- init scripts
 - none
- systemd units
 - none
- dbus services
 - none
- setuid binaries
 - none
   - no binaries in general: python package
- binaries in PATH
 - none
- sudo fragments
 - none
- polkit files
 - none
- udev rules
 - none
- unit tests / autopkgtests
 - no autopkgtests available. Unittesting with standard python, good and clear
testing process during build. Good coverage of code and functions. New tests
are added for new functionality by upstream, and were added as PoC for
CVE-2021-29510 during its fix, suggesting the same will happen with future CVEs.
- cron jobs
 - none
- Build logs
 - (warning) SetuptoolsDeprecationWarning: setup.py install is deprecated.
 - no other errors/warnings, build runs successfully
- Processes spawned
 - none
   - only in docs, does not involve user input, thus not vulnerable to shell
injection
- Memory management
 - safe
   - python package, not using low level memory management
- File IO
 - mostly in tests/docs
 - when in default functionality of a program, done mostly through python and
do not involve user input
- Logging
 - mostly in tests and docs.
 - when in main functionality, done with caution, while using python
(high-level memory management).
- Environment variable usage
 - only in tests, and setup, not exploitable during runtime, or exploiting
them would require already high privileges by attacker
- Use of privileged functions
 - none
- Use of cryptography / random number sources etc
 - does not involve network communications or encryption
 - sensitive data is handled by masking, ensuring that it is not exposed in
logs / error messages. There is no encryption provided for those passwords, but
they are not stored permanently
 - no SSL/TLS operations
- Use of temp files
 - none
- Use of networking
 - mostly used to assist in testing
 - the ones in main functionality take input from trusted sources, no unsafe
input found
- Use of WebKit
 - only in docs, safe
- Use of PolicyKit
 - none

- Any significant cppcheck results
 - none (not supported)
- Any significant Coverity results
 - none
   - possible DOM-based XSS found in docs, communicated with upstream.
   (waiting on response by them). Issue was fixed in later version,
   as code was deleted
- ...

$ ./subscribe-to-package.py --user ubuntu-openstack --package pydantic,python-typing-extensions,python-autocommand,python-inflect,jaraco.text
ubuntu-openstack is now subscribed to all bugs about pydantic.
ubuntu-openstack is now subscribed to all bugs about python-typing-extensions.
ubuntu-openstack is now subscribed to all bugs about python-autocommand.
ubuntu-openstack is now subscribed to all bugs about python-inflect.
ubuntu-openstack is now subscribed to all bugs about jaraco.text.

Bug subscriptions created.

pydantic upstream developers are concerned that the initial upload to main will be pydantic V1 instead of the V2 (which is a re-write with breaking changes).

Are there plans to eventually use V2? If not, this will affect maintainability of all of these packages.

That is worth to ask, but it has already been discussed - see comment #10 and #12

IIUC pydantic (#12 & #13) and python-inflect (#8) are ready.

Override component to main
pydantic 1.10.13-0ubuntu1 in noble: universe/misc -> main
python3-pydantic 1.10.13-0ubuntu1 in noble amd64: universe/python/optional/100% -> main
python3-pydantic 1.10.13-0ubuntu1 in noble arm64: universe/python/optional/100% -> main
python3-pydantic 1.10.13-0ubuntu1 in noble armhf: universe/python/optional/100% -> main
python3-pydantic 1.10.13-0ubuntu1 in noble i386: universe/python/optional/100% -> main
python3-pydantic 1.10.13-0ubuntu1 in noble ppc64el: universe/python/optional/100% -> main
python3-pydantic 1.10.13-0ubuntu1 in noble riscv64: universe/python/optional/100% -> main
python3-pydantic 1.10.13-0ubuntu1 in noble s390x: universe/python/optional/100% -> main
8 publications overridden.

Override component to main
python-autocommand 2.2.2-2 in noble: universe/misc -> main
python3-autocommand 2.2.2-2 in noble amd64: universe/python/optional/100% -> main
python3-autocommand 2.2.2-2 in noble arm64: universe/python/optional/100% -> main
python3-autocommand 2.2.2-2 in noble armhf: universe/python/optional/100% -> main
python3-autocommand 2.2.2-2 in noble i386: universe/python/optional/100% -> main
python3-autocommand 2.2.2-2 in noble ppc64el: universe/python/optional/100% -> main
python3-autocommand 2.2.2-2 in noble riscv64: universe/python/optional/100% -> main
python3-autocommand 2.2.2-2 in noble s390x: universe/python/optional/100% -> main
8 publications overridden.

Override component to main
python-inflect 7.0.0-1 in noble: universe/misc -> main
python3-inflect 7.0.0-1 in noble amd64: universe/python/optional/100% -> main
python3-inflect 7.0.0-1 in noble arm64: universe/python/optional/100% -> main
python3-inflect 7.0.0-1 in noble armhf: universe/python/optional/100% -> main
python3-inflect 7.0.0-1 in noble i386: universe/python/optional/100% -> main
python3-inflect 7.0.0-1 in noble ppc64el: universe/python/optional/100% -> main
python3-inflect 7.0.0-1 in noble riscv64: universe/python/optional/100% -> main
python3-inflect 7.0.0-1 in noble s390x: universe/python/optional/100% -> main
8 publications overridden.