cloud-init devel net-convert crash when --debug is enabled

Bug #1975907 reported by Benjamin Hesmans
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
cloud-init
Fix Released
High
Chad Smith

Bug Description

Since 22.2 enabling "--debug" for "cloud-init devel net-convert" will make cloud-init crash.

Probably linked to 3e5938c6ae22b9f158f1404f41e3e43738cadff0 and the use of safe dumper.

Stack trace shows:
Traceback (most recent call last):
  File "/xyz/git/cloud-init/bin/cloud-init", line 8, in <module>
    sys.exit(main())
  File "/xyz/git/cloud-init/lib/python3.8/site-packages/cloudinit/cmd/main.py", line 1059, in main
    retval = util.log_time(
  File "/xyz/git/cloud-init/lib/python3.8/site-packages/cloudinit/util.py", line 2637, in log_time
    ret = func(*args, **kwargs)
  File "/xyz/git/cloud-init/lib/python3.8/site-packages/cloudinit/cmd/devel/net_convert.py", line 136, in handle_args
    "\n".join(["", "Internal State", safeyaml.dumps(ns, noalias=True), ""])
  File "/xyz/git/cloud-init/lib/python3.8/site-packages/cloudinit/safeyaml.py", line 161, in dumps
    return yaml.dump(
  File "/xyz/git/cloud-init/lib/python3.8/site-packages/yaml/__init__.py", line 253, in dump
    return dump_all([data], stream, Dumper=Dumper, **kwds)
  File "/xyz/git/cloud-init/lib/python3.8/site-packages/yaml/__init__.py", line 241, in dump_all
    dumper.represent(data)
  File "/xyz/git/cloud-init/lib/python3.8/site-packages/yaml/representer.py", line 27, in represent
    node = self.represent_data(data)
  File "/xyz/git/cloud-init/lib/python3.8/site-packages/yaml/representer.py", line 58, in represent_data
    node = self.yaml_representers[None](self, data)
  File "/xyz/git/cloud-init/lib/python3.8/site-packages/yaml/representer.py", line 231, in represent_undefined
    raise RepresenterError("cannot represent an object", data)
yaml.representer.RepresenterError: ('cannot represent an object', <cloudinit.net.network_state.NetworkState object at 0x7fa7979ec340>)

i tried to replace to dumper with the unsafe version and it was working again

Revision history for this message
Chad Smith (chad.smith) wrote :

Thank you for this bug and making cloud-init better and for the additional commit reference as a guess for when regression was introduced.
You are correct that I reintroduced SafeDumper default and that's what broke the cloud-init devel net-convert command.

Because the original commit cf30836645473c62599e838ab48b2d31677fa584 had inadvertently switched to using yaml.Dumper when dropping the direct yaml.safe_dump() call we deemed it reasonable to switch that dumper back to the original SafeDumper as all runtime callsites provide only simple types and not python objects.

Turns out, the command `cloud-init devel net-convert --debug` does attempt to stuff in a python object as you noted NetworkState.

I have reviewed all runtime call-sites to cloudinit.safeyaml.dumps and all of the objects that cloud-init currently provides are dictionaries of simple dicts with the exception of this devel --debug command.

Here is an upstream PR to fix this one callsite and add a unit test so we recognize this type of failure in the future. https://github.com/canonical/cloud-init/pull/1484

Changed in cloud-init:
importance: Undecided → High
status: New → In Progress
assignee: nobody → Chad Smith (chad.smith)
Revision history for this message
Benjamin Hesmans (ben-hesmans) wrote :

Thank you for the quick answer! I tried with your patch, and it looks good. Thx!

(quick test for info)
$ git logo -1
610d1138 (HEAD, chad/net-convert) net-convert: use yaml.dump for debugging python NetworkState obj
[May 30 09:34AM] 0m0s
~/git/cloud-init ((HEAD detached at chad/net-convert)) :)
$ cloud-init devel net-convert --debug -p/tmp/network-config -k yaml -D centos && echo ok
(...)
ok

Revision history for this message
Chad Smith (chad.smith) wrote (last edit ):

Thanks for the test here Benjamin upstream commit landed with this fix. It will be included in cloud-init upstream version 22.3.

https://github.com/canonical/cloud-init/commit/153a7ea25e0d15f707a37655c06cf7155114ea11

Changed in cloud-init:
status: In Progress → Fix Committed
Revision history for this message
Brett Holman (holmanb) wrote : Fixed in cloud-init version 22.3.

This bug is believed to be fixed in cloud-init in version 22.3. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Changed in cloud-init:
status: Fix Committed → Fix Released
Revision history for this message
James Falcon (falcojr) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.