LDAP, non ascii characters in CN field couse error while switching projects

Which Release of OpenStack are you seeing this with (e.g. Havana, Icehouse, Juno/Master)?
Which Identity and Assignment driver(s) are you using (e.g. are you using LDAP identity, SQL Assignment)?

We've seen a few of these issues recently and I want to make sure that if we've solved it we can backport the fix or if it's a new issue we can get it dealt with quickly.

Icehouse 2014.1.2

I have narrow it down to function 'def is_dn_equal(dn1, dn2):' in 'site-packages/keystone/common/ldap/core.py'.

Keystone does utf8_decode on everything it reads from LDAP, but it doesn't utf8_encode LDAP calls.

Fails when it encounters non ascii:

    if not isinstance(dn1, list):
        dn1 = ldap.dn.str2dn(dn1)
    if not isinstance(dn2, list):
        dn2 = ldap.dn.str2dn(dn2)

Works even with non ascii characters (proposed fix):

    if not isinstance(dn1, list):
        dn1 = ldap.dn.str2dn(utf8_encode(dn1))
    if not isinstance(dn2, list):
        dn2 = ldap.dn.str2dn(utf8_encode(dn2))

Third question: What entity (user, project, domain, etc) has the non-ascii CN (it looks like it might be project from the traceback, but I want to be 100% sure)?

Please also provide the non-ascii string in question.

Thanks! I'll take a close look at where we sit with the fixes we have added for Juno (and what has been backported to Icehouse), and get an update to this bug once we get into the swing of things later today.

Entity is user, and string is "Aleš Černivec"

This issue only affects the assignment LDAP driver, as it's the only non-test code that currently uses is_dn_equal(). The fix proposal looks good, but a test needs to be added for this. I should have a patch shortly.

While testing this possible fix I've encountered new problem. If I add Cinder to my setup, I can't login with user that have non ascii characters in CN.

When I try to login, it accepts my username and password, but after few minutes I got session timeout. Login page is stuck during this time (web page is waiting for reply). There is no error in logs during this time, even with debug turned on.

from http error/debug log
[Tue Sep 30 18:10:08 2014] [error] REQ: curl -i http://192.168.122.11:8776/v1/faabdcb060924e15ab8c193b3f82864e/limits -X GET -H "X-Auth-Project-Id: faabdcb060924e15ab8c193b3f82864e" -H "User-Agent: python-cinderclient" -H "Accept: application/json" -H "X-Auth-Token: 7d061e89df785976e2547b48b7ef05e1"

Any advice on how to debug this?

@robert-plestenjak
It's hard to say where the failure is with the information you've given in comment#7. It sounds like you are authenticating against Keystone via Horizon just fine, but something falls over when the Horizon session times out. I would start by enabling debug logging in Horizon.

Also, this issue sounds like a different bug, so please file an additional bug to keep these issues separate.

Reviewed: https://review.openstack.org/125097
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=09d38008aa0e7f8c6802088c807dd0ad6ba80e10
Submitter: Jenkins
Branch: master

commit 09d38008aa0e7f8c6802088c807dd0ad6ba80e10
Author: Nathan Kinder <email address hidden>
Date: Tue Sep 30 08:26:35 2014 -0700

    Convert unicode to UTF8 when calling ldap.str2dn()

    The string passed to ldap.str2dn() needs to be UTF8 encoded, otherwise
    we encounter UnicodeEncodeError exceptions.

    This patch adds UTF8 encoding where it was missing for ldap.str2dn()
    calls as well as some unit tests to cover the functions that were
    lacking this encoding step.

    Change-Id: I96e718f1d43f4aad272ca4990d3905a5adf1f07d
    Closes-bug: #1375139

Fix proposed to branch: proposed/juno
Review: https://review.openstack.org/125467

Reviewed: https://review.openstack.org/125467
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=508e904c7410de841b54e4744e43c7470e1f13d0
Submitter: Jenkins
Branch: proposed/juno

commit 508e904c7410de841b54e4744e43c7470e1f13d0
Author: Nathan Kinder <email address hidden>
Date: Tue Sep 30 08:26:35 2014 -0700

    Convert unicode to UTF8 when calling ldap.str2dn()

    The string passed to ldap.str2dn() needs to be UTF8 encoded, otherwise
    we encounter UnicodeEncodeError exceptions.

    This patch adds UTF8 encoding where it was missing for ldap.str2dn()
    calls as well as some unit tests to cover the functions that were
    lacking this encoding step.

    Change-Id: I96e718f1d43f4aad272ca4990d3905a5adf1f07d
    Closes-bug: #1375139
    (cherry picked from commit 09d38008aa0e7f8c6802088c807dd0ad6ba80e10)

Fix proposed to branch: master
Review: https://review.openstack.org/128930

Reviewed: https://review.openstack.org/128930
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=b788121927abe04a1bbbd1d47eb04cd8d9311904
Submitter: Jenkins
Branch: master

commit ef8d9aa4e1ca3b5465a5bba0cdb6dcb9be1fe9ca
Author: Dolph Mathews <email address hidden>
Date: Tue Oct 7 14:35:28 2014 +0000

    updated translations

    this boils down to:

    $ python setup.py extract_messages
    $ python setup.py update_catalog --no-fuzzy-matching \
      --ignore-obsolete=true
    $ source \
      ../../openstack-infra/project-config/jenkins/scripts/common_translation_update.sh
    $ setup_loglevel_vars
    $ cleanup_po_files keystone

    Change-Id: I2a03f3d7eebe0be0250d4834256dfa3c634dbb48

commit 079c6ad6c911226251fa2a601a27296cfe15e0b3
Author: Henry Nash <email address hidden>
Date: Sun Sep 28 11:16:26 2014 +0100

    Ensure sql upgrade tests can run with non-sqlite databases.

    This patch fixes the issues that were preventing the running of
    live sql upgrade tests (either by running test_sql_upgrade directly
    or via test_sql_livetest), namely:

    - Dropping the tables that were in existence before the current
      scope of migration in an order that is FK friendly
    - Fixing an issue where the tables were being dropped in the
      wrong order in the downgrade of federation
    - Ensuring we don't hold sessions open over upgrade/downgrade
      steps in our test methods

    Limitations:

    - This patch has not been tested with DB2

    Closes-Bug: 1363047
    Closes-Bug: 1375937
    Change-Id: Ied4741a9646b57bc6f2ddcdc8a380ea55b2a9634

commit 4ae1879a79e338e7323935fd17896ba8a4e84fb9
Author: David Stanek <email address hidden>
Date: Fri Oct 3 18:52:54 2014 +0000

    Validates controller methods exist when specified

    It was possible to specify an invalid controller method in a router.
    This will not cause an error until runtime. This change catches the
    error much earlier in the application lifecycle. In fact with this
    change errors should not be able to pass unit tests even if there is
    no specific test for the behavior.

    Related-bug: #1377304
    Change-Id: Icc5646c143a234127a8b4ac8a74342ef3dca7e80

commit 5caf29ad5d90a65d3b10dc55bb101c96b543e4f8
Author: David Stanek <email address hidden>
Date: Fri Oct 3 20:00:30 2014 +0000

    Fixes an error deleting an endpoint group project

    Deleting a endpoint group project fails because the router specifies
    a controller method that doesn't exist. This returns a 500 error to
    the user for what should be a successful operation.

    Change-Id: I3b91d8023d31555893fb944da73633a69d8e286f
    Closes-bug: #1377304

commit c64eae8678327067ef22099e846d927bccb4a804
Author: Brant Knudson <email address hidden>
Date: Wed Oct 1 11:11:21 2014 -0500

    Fix tests comparing tokens

    There were tests that verified that the PKI token body could be
    encrypted with CMS and compared to the token ID in the response.
    This test isn't safe because the token body may be different than
    the token encrypted with CMS since the order of items in the dict
    can change.
    ...