data unavaibility for non-ascii metadata, after python3 upgrade

After digging, we believe that the issue comes from https://opendev.org/openstack/swift/src/commit/9a1bfb8975fb1315fb417161a2865b6bb4760415/swift/obj/diskfile.py#L181-L185 where the string encoded in latin1 with python2 is decoded in utf8 with surrogateescape, generating a "�".

Then the object-server does not answer because "six.b" (https://six.readthedocs.io/#six.b) fails in eventlet/wsgi https://github.com/eventlet/eventlet/blob/6893078d04fa5d46b53269691202e0f709be905a/eventlet/wsgi.py#L498

and eventlet enforces latin1/iso-8859-1 when writing answers to wsgi
https://github.com/eventlet/eventlet/blob/6893078d04fa5d46b53269691202e0f709be905a/eventlet/green/http/server.py#L333
https://github.com/eventlet/eventlet/blob/6893078d04fa5d46b53269691202e0f709be905a/eventlet/green/http/server.py#L564

---

For now, as a workaround, we are thinking to patch diskfile: https://opendev.org/openstack/swift/src/commit/9a1bfb8975fb1315fb417161a2865b6bb4760415/swift/obj/diskfile.py#L181-L185
and ssync_sender: https://opendev.org/openstack/swift/src/commit/9a1bfb8975fb1315fb417161a2865b6bb4760415/swift/obj/ssync_sender.py#L471-L473

What do you think?

Can repro, though I *do* get a traceback in logs:

Mar 22 18:27:41 saio object-6040: 127.0.0.1 - - [22/Mar/2023:18:27:41 +0000] "HEAD /sdb4/6/AUTH_test/c/tox.ini" 200 2935 "HEAD https://saio/v1/AUTH_test/c/tox.ini" "tx444f20aa644e43f48d5bb-00641b489d" "proxy-server 28880" 0.0010 "-" 28867 0
Mar 22 18:27:41 saio object-6040: STDERR: Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/eventlet/wsgi.py", line 607, in handle_one_response
    write(b'')
  File "/usr/local/lib/python3.10/dist-packages/eventlet/wsgi.py", line 493, in write
    towrite.append(six.b('%s: %s\r\n' % header))
  File "/usr/local/lib/python3.10/dist-packages/six.py", line 644, in b
    return s.encode("latin-1")
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 33-35: ordinal not in range(256) (txn: tx444f20aa644e43f48d5bb-00641b489d)

maybe we could start writing down explicitly what version of python was running when the metadata was written down, and then try to guess when decodeing if the key isn't there (configurable timestamp operator can use to indicate py2=>py3 transistion, with -1 default for "always py3")

Reviewed: https://review.opendev.org/c/openstack/swift/+/878558
Committed: https://opendev.org/openstack/swift/commit/780754096267851545f2aa97afb250064a2292e3
Submitter: "Zuul (22348)"
Branch: master

commit 780754096267851545f2aa97afb250064a2292e3
Author: Tim Burke <email address hidden>
Date: Fri Mar 24 13:27:24 2023 -0700

    Properly read py2 object metadata on py3

    Replicated, unencrypted metadata is written down differently on py2
    vs py3, and has been since we started supporting py3. Fortunately,
    we can inspect the raw xattr bytes to determine whether the pickle
    was written using py2 or py3, so we can properly read legacy py2 meta
    under py3 rather than hitting a unicode error.

    Closes-Bug: #2012531
    Change-Id: I5876e3b88f0bb1224299b57541788f590f64ddd4

This issue was fixed in the openstack/swift 2.32.0 release.