Validation parameter_type.url regex doesn't pass validation for IPv6 addresses

Endpoint url validation is controlled by a regular expression [1]. We had the option to use the url validation FormatChecker object that was built into jsonschema but it relies on rfc3987 [2], which is GPL licensed. The rfc3987 library wasn't added to global requirements since it is GPL licensed [3]. Chances are the regex can be tweaked to allow for ipv6 addresses with ports.

[1] https://github.com/openstack/keystone/blob/15a01f2918b4822b3df660ca2567ef398ed7d0a3/keystone/common/validation/parameter_types.py#L53-L59
[2] https://pypi.python.org/pypi/rfc3987
[3] https://review.openstack.org/#/c/98012/

Fix proposed to branch: master
Review: https://review.openstack.org/131326

Reviewed: https://review.openstack.org/131326
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=7fa6e92c441644c0d990e79a78692b786d88acec
Submitter: Jenkins
Branch: master

commit 7fa6e92c441644c0d990e79a78692b786d88acec
Author: Matthew Edmonds <email address hidden>
Date: Mon Oct 27 16:34:35 2014 -0400

    Adds IPv6 url validation support

    Fixes an issue where url validation fails when attempting to
    create an endpoint with an IPv6 address in the URL.

    Change-Id: I179b0cb7e5546a3a051e5a1e6390385a833b9f96
    Closes-Bug: 1386376

Fix proposed to branch: stable/juno
Review: https://review.openstack.org/131902

Reviewed: https://review.openstack.org/131902
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=bd92689e4915f6dfe4ca72c802146ecd5d206bcf
Submitter: Jenkins
Branch: stable/juno

commit bd92689e4915f6dfe4ca72c802146ecd5d206bcf
Author: Matthew Edmonds <email address hidden>
Date: Mon Oct 27 16:34:35 2014 -0400

    Adds IPv6 url validation support

    Fixes an issue where url validation fails when attempting to
    create an endpoint with an IPv6 address in the URL.

    Change-Id: I179b0cb7e5546a3a051e5a1e6390385a833b9f96
    Closes-Bug: 1386376
    (cherry picked from commit 7fa6e92c441644c0d990e79a78692b786d88acec)

This is a copy-paste of a comment I put on a related bug incorrectly, https://bugs.launchpad.net/keystone/+bug/1266416/comments/3

I've run into another validation problem as well using Juno keystone v3. This test code demonstrates the problem with the regular expression:

import re

foo = {
    'pat': '^https?://'
           '(?:(?:[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?\.)'
           '+[a-zA-Z]{2,6}\.?|'
           'localhost|'
           '\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})'
           '(?::\d+)?'
           '(?:/?|[/?]\S+)$'
}

pat = foo['pat']

print bool(re.match(pat, 'https://foo')) # Bad
print bool(re.match(pat, 'https://foo.l')) # Bad
print bool(re.match(pat, 'https://foo.lo')) # Good
print bool(re.match(pat, 'https://foo.loc')) # Good
print bool(re.match(pat, 'https://foo.loca')) # Good
print bool(re.match(pat, 'https://foo.local')) # Good
print bool(re.match(pat, 'https://foo.locald')) # Good
print bool(re.match(pat, 'https://foo.localdo')) # Bad
print bool(re.match(pat, 'https://foo.localdoma')) # Bad

It appears the above domain name has to be between 2 and 6 characters. The [a-zA-Z]{2,6} part is responsible for that. Looking at the diff for the IPv6 fix, it should solve this one too.

Fix proposed to branch: feature/hierarchical-multitenancy
Review: https://review.openstack.org/138182

Change abandoned by Morgan Fainberg (<email address hidden>) on branch: feature/hierarchical-multitenancy
Review: https://review.openstack.org/138182