py34 and py35 unit tests are broken

Bug #1612149 reported by Valeriy Ponomaryov
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Shared File Systems Service (Manila)
Fix Released
Critical
Tom Barron

Bug Description

Recently manila unit tests using py34 and py35 started failing on several tests, see logs below.

logs: http://logs.openstack.org/86/331586/8/check/gate-manila-python34-db/689727b/testr_results.html.gz

Revision history for this message
Valeriy Ponomaryov (vponomaryov) wrote :

It is blocker for CI. So, setting it as critical.

Changed in manila:
milestone: none → newton-3
importance: Undecided → Critical
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (master)

Fix proposed to branch: master
Review: https://review.openstack.org/354023

Changed in manila:
assignee: nobody → Yingzhe Zeng (zengyingzhe)
status: New → In Progress
Changed in manila:
assignee: Yingzhe Zeng (zengyingzhe) → Tom Barron (tpb)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to manila (master)

Reviewed: https://review.openstack.org/354023
Committed: https://git.openstack.org/cgit/openstack/manila/commit/?id=5ea4911794df4a744afe825e6083b82549a5f384
Submitter: Jenkins
Branch: master

commit 5ea4911794df4a744afe825e6083b82549a5f384
Author: zengyingzhe <email address hidden>
Date: Thu Aug 11 20:24:22 2016 +0800

    Fix the broken UT of huawei driver for py34/35

    The cause of failing unit tests is a bug in huawei driver.
    When huawei driver logins to backend, it reads username/password from
    huawei configuration file. If the username/password are configured as
    plain text, huawei driver'll encode them and overwrite the encoded text
    to the configuration file.

    The encoding logic code is like this:
      six.text_type(base64.b64encode("***"))

    For py27, this code works fine. However, for py34, this text_type
    converting adds some extra words to result unicode string, like
    "b'***'"('***' is the actual encoded result string).
    Once driver reads the username/password again, it'll get the incorrect
    text and then fail while jsonutils.dumps.

    All the failed unit tests tried to call login twice, and triggered the
    error condition said above.

    This patch is just a workaround, because those failed tests actually
    no need to login twice, the redundant login calls are removed.

    Another patch will be commited to fix the problem thoroughly in huawei
    driver. Bug https://launchpad.net/bugs/1612149 has been raised for
    that work, and will also analyze what changed to trigger this issue
    only recently.

    Change-Id: Ia6bf7bbb9ffb9644085bbdf4f5576f09215a877f
    Closes-Bug: #1612149

Changed in manila:
status: In Progress → Fix Released
Revision history for this message
Yingzhe Zeng (zengyingzhe) wrote :

Further analysis about this bug.

The direct reason:
The to_primitive method in oslo_serialization/jsonutils.py was updated recently.
The following code snippet was added:
    if isinstance(value, six.binary_type):
        if six.PY3:
            value = value.decode(encoding=encoding)
        return value

As I said before, the encoded string in huawei driver configuration is a byte string, like b'm\x85\x91\xb5\xa5\xb8'.
This string can't be decode in 'utf-8' because '\x85\x91\xb5\xa5\xb8' are invalid utf8 bytes.
So an exception is raised here, UT fails.

Why all huawei UT can pass before?
Because before the update to to_primitive was made, this method worked for the byte string above,
it would convert the string to an int list, like [109, 133, 145, 181, 165, 184].
This result obviously wasn't right, but cause huawei driver UT mocked the request sending method,
no request was actually sent to backend, so no matter what the input parameter was,
the REST request was gonna return success, then UT succeeded also.

The root reason:
As I said, the encoding logic in huawei driver for username/password has bug for py34.
Also because this bug can only be triggered in a specific circumstance, the bug didn't reveal until recent.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/manila 3.0.0.0b3

This issue was fixed in the openstack/manila 3.0.0.0b3 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/375673

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on manila (stable/mitaka)

Change abandoned by Goutham Pacha Ravi (<email address hidden>) on branch: stable/mitaka
Review: https://review.openstack.org/375673
Reason: Thanks Tom!

This fix isn't needed if we applied the upper constraints, the CI system was out of whack. https://review.openstack.org/#/c/375681/ demonstrates that our upstream pulls down the right version of oslo_serialization as specified in the upper-constraints.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.