validation of app cred tokens is dependent on CONF.token.cache_on_issue

Fix proposed to branch: master
Review: https://review.openstack.org/545945

Based upon research and discussions in IRC, turns out we do not store the application_credential_id in the token payload. This means that if the token is not pre-populated in the cache, the test will fail.

This also means that if the token cache expires, subsequent uses of the token with the application cred will also fail / have inconsistent or inappropriate behavior.

This requires a fix to add a formatter that includes application_credentials (likely more than one). The issue is identified by looking at https://github.com/openstack/keystone/blob/c80df22669ae457f8a64ddef7d31f685f9ad1e01/keystone/token/token_formatters.py and seeing that application credential is not stored anywhere but the auth methods are properly populated.

Here is Lance's output log of keystone during that test. http://paste.openstack.org/show/677699/

Both cmurphy and I were able to recreate this in devstack setup. Just disable CONF.token.cache_on_issue by setting it to False, and reload the keystone service (sudo systemctl reload <email address hidden>; sudo systemctl restart <email address hidden>)

Steps are in the paste [0].

[0] http://paste.openstack.org/show/677772/

Fix proposed to branch: master
Review: https://review.openstack.org/545971

Change abandoned by Lance Bragstad (<email address hidden>) on branch: master
Review: https://review.openstack.org/545945
Reason: This was rolled into https://review.openstack.org/#/c/545971/5

Reviewed: https://review.openstack.org/545971
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=796198f19670e3eb899ca3b1db5d2a21a4127a30
Submitter: Zuul
Branch: master

commit 796198f19670e3eb899ca3b1db5d2a21a4127a30
Author: Lance Bragstad <email address hidden>
Date: Mon Feb 19 18:23:25 2018 +0000

    Populate application credential data in token

    Without this patch, the token formatter does not have enough data to
    construct a token created with an application credential. This means
    that if the token cache is disabled or expired, when keystone goes to
    create the token it will not find any application credential information
    and will not recreate the application_credential_restricted parameter in
    the token data. This patch creates a new Payload class for application
    credentials so that the application credential ID is properly persisted
    in the msgpack'd payload. It also adds more data to the token data
    object so that the application credential ID and name as well as its
    restricted status is available when the token is queried.

    Co-authored-by: Lance Bragstad <email address hidden>

    Change-Id: I322a40404d8287748fe8c3a8d6dc1256d935d84a
    Closes-bug: #1750415

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/546065

Reviewed: https://review.openstack.org/546065
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=afca5cc43bc2442bd95524b7ad030d2e7965902c
Submitter: Zuul
Branch: stable/queens

commit afca5cc43bc2442bd95524b7ad030d2e7965902c
Author: Lance Bragstad <email address hidden>
Date: Mon Feb 19 18:23:25 2018 +0000

    Populate application credential data in token

    Without this patch, the token formatter does not have enough data to
    construct a token created with an application credential. This means
    that if the token cache is disabled or expired, when keystone goes to
    create the token it will not find any application credential information
    and will not recreate the application_credential_restricted parameter in
    the token data. This patch creates a new Payload class for application
    credentials so that the application credential ID is properly persisted
    in the msgpack'd payload. It also adds more data to the token data
    object so that the application credential ID and name as well as its
    restricted status is available when the token is queried.

    Co-authored-by: Lance Bragstad <email address hidden>

    Change-Id: I322a40404d8287748fe8c3a8d6dc1256d935d84a
    Closes-bug: #1750415
    (cherry picked from commit 796198f19670e3eb899ca3b1db5d2a21a4127a30)

This issue was fixed in the openstack/keystone 13.0.0.0rc2 release candidate.

This issue was fixed in the openstack/keystone 14.0.0.0b1 development milestone.