[sru] sos upstream 4.5.1

[IMPACT]

The sos team is pleased to announce the release of sos-4.5.1. This release follows our recent changes to adopt a faster release cadence, and as such is smaller in scope than previous releases for the project. The next release is slated for March 29.

The sos team is pleased to announce the release of sos-4.5.0. This is the first release of a new, faster release cadence that will see new versions cut approximately every 4 weeks going forward. This release sees design changes to plugins, most notably by now supporting the manual collection and compilation of diagnostic data not strictly limited to existing command output or file collection. Highlighted changes are noted below.

[TEST PLAN]

Documentation for Special Cases:
https://wiki.ubuntu.com/SosreportUpdates

[WHERE PROBLEMS COULD OCCUR]

The changes in sosreport are described below. There are a number of changes related to report, plugins and policy. The changes in sosreport content may break existing tooling, so that will need extensive testing.

Global changes

[4.5.1]
Build failures for snaps will now be available within the CI run in which a particular build failed.
Report Changes
A plethora of new tags and changed tags have been implemented across many collections to assist with Insights inspection.
Fixed a bug where a potential duplicate command when run in a container could result in an incorrectly handled exception within the archive. If duplicate commands are called within the same container from the same plugin, there will no longer be an error.
The ocp preset will no longer use the --verify option
Plugin Changes
New plugins: ceph_iscsi, microshift, microshift_ovn
The azure plugin has been updated to use a newer endpoint for metadata retrieval
The rhui plugin will now properly obfuscate certain sensitive keys from collections
The composer plugin will now capture /etc/osbuild-composer
Running an ostree fsck is now gated behind the new ostree.fsck plugin option, and not tied to --verify

[4.5.0]
Snaps are now created automatically whenever a change is pushed to main. These snaps are available via snapcraft under latest/edge
python3-magic is now a soft dependency, and if not present sos will use a less sophisticated method for determining if a file is binary or not
distutils usage has been fully replaced by setuptools
Policies
Added support for Anolis OS
Added support for Circle Linux
Added support for OpenCloudOS
When loaded for an sos collect execution, a policy's remote_exec will now directly use the loaded transport's run_command functionality, rather than re-building command strings
The Debian policy has been updated to correctly identify many more and newer Debian versions
Fixed an issue with the RHEL policy that would prevent non-anonymous upload to the failover SFTP server if a case ID was not provided
Report
A "tag_summary" section has been added to the report manifest.json. This is a dictionary with keys being tags that were created during collection, with values being all files sharing that tag
sos_get_command_output() timeout handling will now properly handle the situation where a command's child process deadlocks but the timeout wrapper was able to kill the parent process, but left the child behind.
Estimate mode for report will now report real disk usage, rather than apparent size
Plugins
New plugins: containerd, fapolicyd
The kernel plugin will now collect modprobe.d/*conf files
The hpssm plugin will now collect show detail output per array and slot
The crio plugin now supports CoS systems
The dnf plugin will now properly obfuscate password variable values
The flow of plugin code execution has been changed
setup() is now strictly for determining what collections to perform, outside of calls to collect_cmd_output() in order to build further sets of commands
_collect_plugin() is now used to actually perform the collections specfied by setup(). This now includes tailed file collections which were previously part of setup()
collect() can now be used to perform ad-hoc/manual collections that are not strictly part of command output collection. If a plugin needs to manipulate data from commands or system information and then write it out manually (such as with the rpm plugin generating package output), it should now be done inside the collect() method
The composer plugin has been overhauled and updated for the new versions of composer
Enablement triggers have been expanded for the xfs, nvme, firewall_tables, and krb5 plugins
The virsh plugin will now collect more information about the host/hypervisor system
The various ceph_* plugins have been updated to collect the appropriate data for both older and more recent versions of ceph, including traditional installations and those deployed with cephadm
Collect
Cluster profiles may now directly specify sos options to enforce on per-node report collections
Added a new cluster profile for Red Hat Ceph Storage 5
This new profile may work for other Ceph environments deployed with cephadm, but that is not tested
Added a new saltstack transport
Cleaner|Mask
The --domains option is now validated for items that look like a domain
Fixed an issue where a file with encoding issues would be aborted by clean, but left in the archive. Files will now either show replaced content or be removed from the archive, rather than being left unobfuscated in any manner
sos will no longer attempt to obfuscate the temp directory the archive is in before moving the archive at the end of cleaning
Added a new parser to support IPv6 obfuscation

Please report any problems to the sos-devel mailing list, or the GitHub issue tracker:

https://github.com/sosreport/sos/issues/

[OTHER INFORMATION]

Regression could occur at core functionality, which may prevent sos (or its subcommand to work. I consider this regression type as 'low'. That is generally well tested, and we would find a problem at an early stage during the verification phase if it is the case.

On the other end, regression could happen and are some kind of expected at plugins levels. As of today, sos has more than 300 plugins. It is nearly impossible to test them all.

If a regression is found in a plugin, it is rarely affecting sos core functionalities nor other plugins. So mainly the impact would be limited to that plugin. The impact being that the plugin can't or partially can collect the information that it is instructed to gather.

A 3rd party vendor would then ask user/customer to collect the information manually for that particular plugins.

Plugins are segmented by services and/or applications (e.g. openstack_keystone, bcache, system, logs, ...) in order to collect things accordingly to the plugin detected or intentionally requested for.

Sosreport plugins philosophy is to (as much as possible) maintain backward compatibility when updating a plugin. The risk that an ancient version of a software has been dropped, is unlikely, unless it was intended to be that way for particular reasons. Certain plugin also support the DEB installation way and the snap one (MAAS, LXD, ...) so all Ubuntu standard installation types are covered.

Release notes:
https://github.com/sosreport/sos/releases/tag/4.5.0
https://github.com/sosreport/sos/releases/tag/4.5.1