nova-api returns 500 error(DBError) if nova list API is called with filters with empty string

Bug #1718877 reported by Dinesh Bhor
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
Dinesh Bhor
Ocata
Fix Committed
Medium
Matt Riedemann
Pike
Fix Committed
Medium
Matt Riedemann

Bug Description

If nova list API is called with filter as empty regex pattern like below:
    http://<host-ip>/compute/v2.1/servers?name=""

it fails at db layer with below error and returns 500 InternalServerError
to user:

  InternalError: (1139, u"Got error 'empty (sub)expression' from regexp")

Looks like the issue is there for every query parameter which is using below
regex parameter types defined in parameter_types.py:

    common_query_regex_param

Reproduction steps:

Current nova master:
commit 2761b7fc40366644f611cb994342c5387f7fc6a1
Merge: bd37c64 8c44be5
Author: Jenkins <email address hidden>
Date: Tue Sep 19 02:58:27 2017 +0000

    Merge "Updated from global requirements"

devstack:
commit 6c87f363a1255270a3711b3fe7a065cc693164b9
Merge: f60babc 1d141da
Author: Jenkins <email address hidden>
Date: Tue Sep 19 02:07:16 2017 +0000

    Merge "Feature flag on tempest conf to notify enabled LDAP"

1] nova list API:

curl -g -i -X GET http://10.232.48.201/compute/v2.1/servers?name=""
-H "OpenStack-API-Version: compute 2.53" -H "User-Agent: python-novaclient"
-H "Accept: application/json" -H "X-OpenStack-Nova-API-Version: 2.53"
-H "X-Auth-Token: 2e403dfaae4a4d7fbcf0a9fc4f97d8e4"

HTTP/1.1 500 Internal Server Error
Date: Fri, 15 Sep 2017 10:34:39 GMT
Server: Apache/2.4.18 (Ubuntu)
OpenStack-API-Version: compute 2.53
X-OpenStack-Nova-API-Version: 2.53
Vary: OpenStack-API-Version,X-OpenStack-Nova-API-Version
Content-Type: application/json; charset=UTF-8
Content-Length: 199
x-openstack-request-id: req-85ae3592-3726-4e95-8bb9-9a9204dd7e68
x-compute-request-id: req-85ae3592-3726-4e95-8bb9-9a9204dd7e68
Connection: close

{"computeFault": {"message": "Unexpected API Error. Please report this at http://bugs.launchpad.net/nova/ and attach the Nova API log if possible.\n<class 'oslo_db.exception.DBError'>", "code": 500}}

Nova-API logs:

Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: ERROR oslo_db.sqlalchemy.exc_filters [None req-85ae3592-3726-4e95-8bb9-9a9204dd7e68 admin admin]
DBAPIError exception wrapped from (pymysql.err.InternalError) (1139, u"Got error 'empty (sub)expression' from regexp") [SQL: u'SELECT anon_1.instances_created_at AS anon_1_instances_created_at,
anon_1.instances_updated_at AS anon_1_instances_updated_at, anon_1.instances_deleted_at AS anon_1_instances_deleted_at, anon_1.instances_deleted AS anon_1_instances_deleted, anon_1.
instances_id AS anon_1_instances_id, anon_1.instances_user_id AS anon_1_instances_user_id, anon_1.instances_project_id AS anon_1_instances_project_id, anon_1.instances_image_ref AS anon_
1_instances_image_ref, anon_1.instances_kernel_id AS anon_1_instances_kernel_id, anon_1.instances_ramdisk_id AS anon_1_instances_ramdisk_id, anon_1.instances_hostname AS anon_1_instanc
es_hostname, anon_1.instances_launch_index AS anon_1_instances_launch_index, anon_1.instances_key_name AS anon_1_instances_key_name, anon_1.instances_key_data AS anon_1_instances_k
ey_data, anon_1.instances_power_state AS anon_1_instances_power_state, ano
n_1.instances_vm_state AS anon_1_instances_vm_state, anon_1.instances_task_state AS anon_1_instances_task_state, anon_1.instances_memory_mb AS anon_1_instances_memory_mb, anon_1.i
nstances_vcpus AS anon_1_instances_vcpus, anon_1.instances_root_gb AS anon_1_instances_root_gb, anon_1.instances_ephemeral_gb AS anon_1_instances_ephemeral_gb, anon_1.instances_ephe
meral_key_uuid AS anon_1_instances_ephemeral_key_uuid, anon_1.instances_host AS anon_1_instances_host, anon_1.instances_node AS anon_1_instances_node, anon_1.instances_instance_type
_id AS anon_1_instances_instance_type_id, anon_1.instances_user_data AS anon_1_instances_user_data, anon_1.instances_reservation_id AS anon_1_instances_reservation_id, anon_1.instances_lau
nched_at AS anon_1_instances_launched_at, anon_1.instances_terminated_at AS anon_1_instances_terminated_at, anon_1.instances_availability_zone AS anon_1_instances_availability
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: _zone, anon_1.instances_display_name AS anon_1_instances_display_name, anon_1.instances_display_description AS anon_1_in
stances_display_description, anon_1.instances_launched_on AS anon_1_instances_launched_on, anon_1.instances_locked AS anon_1_instances_locked, anon_1.instances_locked_by AS anon_1_insta
nces_locked_by, anon_1.instances_os_type AS anon_1_instances_os_type, anon_1.instances_architecture AS anon_1_instances_architecture, anon_1.instances_vm_mode AS anon_1_instances_vm_
mode, anon_1.instances_uuid AS anon_1_instances_uuid, anon_1.instances_root_device_name AS anon_1_instances_root_device_name, anon_1.instances_default_ephemeral_device AS anon_1_i
nstances_default_ephemeral_device, anon_1.instances_default_swap_device AS anon_1_instances_default_swap_device, anon_1.instances_config_drive AS anon_1_instances_config_drive, anon_1.i
nstances_access_ip_v4 AS anon_1_instances_access_ip_v4, anon_1.instances_access_ip_v6 AS anon_1_instances_access_ip_v6, anon_1.instances_auto_disk_config AS anon_1_instances_auto_disk_co
nfig, anon_1.instances_progress AS anon_1_instances_progress, anon_1.instances_shutdown_terminate AS anon_1_instances_shutdown_terminate, anon_1.instances_disable_terminate AS anon_1_in
stances_disable_terminate, anon_1.instances_cell_name AS anon_1_instances_cell_name, anon_1.instances_cleaned AS anon_1_instances_cleaned, security_groups_1.created_at AS security_groups_1
_created_at, security_groups_1.updated_at AS security_groups_1_updated_at, security_groups_1.deleted_at AS security_groups_1_deleted_at, security_groups_1.deleted AS security_groups_1_delete
d, security_groups_1.id AS security_groups_1_id, security_groups_1.name AS security_groups_1_name, security_groups_1.description AS security_groups_1_description, security_groups_1.user_id AS sec
urity_groups_1_user_id, security_groups_1.project_id AS security_groups_1_project_id, instance_info_caches_1.created_at AS instance_info_caches_1_created_at, instance_info_caches_1.updated_at A
S instance_info_caches_1_updated_at, instance_info_caches_1.
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: deleted_at AS instance_info_caches_1_deleted_at, instance_info_caches_1.deleted AS instance_info_caches_1_deleted, instance_in
fo_caches_1.id AS instance_info_caches_1_id, instance_info_caches_1.network_info AS instance_info_caches_1_network_info, instance_info_caches_1.instance_uuid AS instance_info_caches_1_instance_
uuid \nFROM (SELECT instances.created_at AS instances_created_at, instances.updated_at AS instances_updated_at, instances.deleted_at AS instances_deleted_at, instances.deleted AS instances_deleted, in
stances.id AS instances_id, instances.user_id AS instances_user_id, instances.project_id AS instances_project_id, instances.image_ref AS instances_image_ref, instances.kernel_id AS instances_kernel_id, insta
nces.ramdisk_id AS instances_ramdisk_id, instances.hostname AS instances_hostname, instances.launch_index AS instances_launch_index, instances.key_name AS instances_key_name, instances.key_data
AS instances_key_data, instances.power_state AS instances_power_state, instances.vm_state AS instances_vm_state, instances.task_state AS instances_task_state, instances.memory_mb AS instances_me
mory_mb, instances.vcpus AS instances_vcpus, instances.root_gb AS insta
nces_root_gb, instances.ephemeral_gb AS instances_ephemeral_gb, instances.ephemeral_key_uuid AS instances_ephemeral_key_uuid, instances.host AS instances_host, instances.node AS instances_node,
instances.instance_type_id AS instances_instance_type_id, instances.user_data AS instances_user_data, instances.reservation_id AS instances_reservation_id, instances.launched_at AS instances_launched_
at, instances.terminated_at AS instances_terminated_at, instances.availability_zone AS instances_availability_zone, instances.display_name AS instances_display_name, instances.display_description AS insta
nces_display_description, instances.launched_on AS instances_launched_on, instances.locked AS instances_locked, instances.locked_by AS instances_locked_by, instances.os_type AS instances_os_type, inst
ances.architecture AS instances_architecture, instances.vm_mode A
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: S instances_vm_mode, instances.uuid AS instances_uuid, instances.root_device_name AS instances_root_device_name, instances.default
_ephemeral_device AS instances_default_ephemeral_device, instances.default_swap_device AS instances_default_swap_device, instances.config_drive AS instances_config_drive, instances.access_ip_v4 AS i
nstances_access_ip_v4, instances.access_ip_v6 AS instances_access_ip_v6, instances.auto_disk_config AS instances_auto_disk_config, instances.progress AS instances_progress, instances.shutdown_terminat
e AS instances_shutdown_terminate, instances.disable_terminate AS instances_disable_terminate, instances.cell_name AS instances_cell_name, instances.cleaned AS instances_cleaned \nFROM instances \n
WHERE instances.deleted = %(deleted_1)s AND (instances.vm_state != %(vm_state_1)s OR instances.vm_state IS NULL) AND instances.project_id = %(project_id_1)s AND (instances.display_name
REGEXP %(display_name_1)s) ORDER BY instances.created_at DESC, instances.id DESC \n LIMIT %(param_1)s) AS anon_1 LEFT OUTER JOIN (security_group_instance_association AS security_group_instance_association_1
INNER JOIN security_groups AS security_groups_1 ON security_groups_1.id = security_group_instance_association_1.security_group_id AND security_group_instance_association_1.deleted = %(deleted_2)s AND
security_groups_1.deleted = %(deleted_3)s) ON security_group_instance_association_1.instance_uuid = anon_1.instances_uuid AND anon_1.instances_deleted = %(deleted_4)s LEFT OUTER JOIN instance_info_caches
AS instance_info_caches_1 ON instance_info_caches_1.instance_uuid = anon_1.instances_uuid ORDER BY anon_1.instances_created_at DESC, anon_1.instances_id DESC'] [parameters: {u'deleted_3': 0, u'
display_name_1': u'', u'deleted_4': 0, u'param_1': 1000, u'deleted_2': 0, u'deleted_1': 0, u'vm_state_1': 'soft-delete', u'project_id_1': u'038a2813bce843e1a49a512701f17ddf'}]: InternalError:
(1139, u"Got error 'empty (sub)expression' from regexp")
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: ERROR oslo_db.sqlalchemy.exc_filters Traceback (most recent call last):
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: ERROR oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1182, in _execute_context
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: ERROR oslo_db.sqlalchemy.exc_filters context)
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: ERROR oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 470, in do_execute
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: ERROR oslo_db.sqlalchemy.exc_filters cursor.execute(statement, parameters)
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: ERROR oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 166, in execute
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: ERROR oslo_db.sqlalchemy.exc_filters result = self._query(query)
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: ERROR oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 322, in _query
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: ERROR oslo_db.sqlalchemy.exc_filters conn.query(q)
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: ERROR oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 856, in query
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: ERROR oslo_db.sqlalchemy.exc_filters self._affected_rows = self._read_query_result(unbuffered=unbuffered)
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: ERROR oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1057, in _read_query_result
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: ERROR oslo_db.sqlalchemy.exc_filters result.read()
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: ERROR oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1340, in read
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: ERROR oslo_db.sqlalchemy.exc_filters first_packet = self.connection._read_packet()
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: ERROR oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 1014, in _read_packet
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: ERROR oslo_db.sqlalchemy.exc_filters packet.check_error()
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: ERROR oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 393, in check_error
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: ERROR oslo_db.sqlalchemy.exc_filters err.raise_mysql_exception(self._data)
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: ERROR oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python2.7/dist-packages/pymysql/err.py", line 107, in raise_mysql_exception
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: ERROR oslo_db.sqlalchemy.exc_filters raise errorclass(errno, errval)
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: ERROR oslo_db.sqlalchemy.exc_filters InternalError: (1139, u"Got error 'empty (sub)expression' from regexp")
Sep 15 16:04:40 dbopenstack-VirtualBox <email address hidden>[989]: ERROR oslo_db.sqlalchemy.exc_filters

Tags: api
Changed in nova:
assignee: nobody → Dinesh Bhor (dinesh-bhor)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/506585

Changed in nova:
status: New → In Progress
Changed in nova:
assignee: Dinesh Bhor (dinesh-bhor) → Matt Riedemann (mriedem)
Matt Riedemann (mriedem)
Changed in nova:
importance: Undecided → Medium
tags: added: api
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/506585
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=0cc94bfd5934e793cb8d739e132b52fa40bcddb0
Submitter: Jenkins
Branch: master

commit 0cc94bfd5934e793cb8d739e132b52fa40bcddb0
Author: Dinesh Bhor <email address hidden>
Date: Tue Sep 19 14:56:01 2017 +0530

    Fix 500 if list servers called with empty regex pattern

    If nova list API is called with empty regex pattern like below:
        http://<host-ip>/compute/v2.1/servers?name=""

    it fails at db layer and returns 500 InternalServerError.

    Empty string('') is a valid regex for re.compile [1] so nova-api fails
    to catch it at schema layer and it is passed to databse for searching.
    Database fails to search it with "REGEXP %(display_name_1)s)" and
    'display_name_1' as u'' in sql query which leads to below error:

    InternalError: (1139, u"Got error 'empty (sub)expression' from regexp")

    This issue is there for every query parameter which is using below
    regex parameter types defined in parameter_types.py:
        common_query_regex_param

    This patch fixes this issue by rejecting the request with 400 if the
    provided filter regex is empty string. If user is intending to filter
    something, user must pass something to the filter, it cannot be a empty
    string.

    [1] https://github.com/openstack/nova/blob/16.0.0/nova/api/validation/validators.py#L40

    Closes-Bug: #1718877
    Change-Id: I3f6fa04dc7267279964e8e5dd2a790b997a40e4e

Changed in nova:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/506754

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/ocata)

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/506760

Matt Riedemann (mriedem)
Changed in nova:
assignee: Matt Riedemann (mriedem) → Dinesh Bhor (dinesh-bhor)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/pike)

Reviewed: https://review.openstack.org/506754
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=6538175c08f127c20e72e1d347af3eb3d54d6838
Submitter: Jenkins
Branch: stable/pike

commit 6538175c08f127c20e72e1d347af3eb3d54d6838
Author: Dinesh Bhor <email address hidden>
Date: Tue Sep 19 14:56:01 2017 +0530

    Fix 500 if list servers called with empty regex pattern

    If nova list API is called with empty regex pattern like below:
        http://<host-ip>/compute/v2.1/servers?name=""

    it fails at db layer and returns 500 InternalServerError.

    Empty string('') is a valid regex for re.compile [1] so nova-api fails
    to catch it at schema layer and it is passed to databse for searching.
    Database fails to search it with "REGEXP %(display_name_1)s)" and
    'display_name_1' as u'' in sql query which leads to below error:

    InternalError: (1139, u"Got error 'empty (sub)expression' from regexp")

    This issue is there for every query parameter which is using below
    regex parameter types defined in parameter_types.py:
        common_query_regex_param

    This patch fixes this issue by rejecting the request with 400 if the
    provided filter regex is empty string. If user is intending to filter
    something, user must pass something to the filter, it cannot be a empty
    string.

    [1] https://github.com/openstack/nova/blob/16.0.0/nova/api/validation/validators.py#L40

    Closes-Bug: #1718877
    Change-Id: I3f6fa04dc7267279964e8e5dd2a790b997a40e4e
    (cherry picked from commit 0cc94bfd5934e793cb8d739e132b52fa40bcddb0)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/ocata)

Reviewed: https://review.openstack.org/506760
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=befcda5b2149492cf6b5c7856def238fbb9079d8
Submitter: Jenkins
Branch: stable/ocata

commit befcda5b2149492cf6b5c7856def238fbb9079d8
Author: Dinesh Bhor <email address hidden>
Date: Tue Sep 19 14:56:01 2017 +0530

    Fix 500 if list servers called with empty regex pattern

    If nova list API is called with empty regex pattern like below:
        http://<host-ip>/compute/v2.1/servers?name=""

    it fails at db layer and returns 500 InternalServerError.

    Empty string('') is a valid regex for re.compile [1] so nova-api fails
    to catch it at schema layer and it is passed to databse for searching.
    Database fails to search it with "REGEXP %(display_name_1)s)" and
    'display_name_1' as u'' in sql query which leads to below error:

    InternalError: (1139, u"Got error 'empty (sub)expression' from regexp")

    This issue is there for every query parameter which is using below
    regex parameter types defined in parameter_types.py:
        common_query_regex_param

    This patch fixes this issue by rejecting the request with 400 if the
    provided filter regex is empty string. If user is intending to filter
    something, user must pass something to the filter, it cannot be a empty
    string.

    [1] https://github.com/openstack/nova/blob/16.0.0/nova/api/validation/validators.py#L40

    Closes-Bug: #1718877
    Change-Id: I3f6fa04dc7267279964e8e5dd2a790b997a40e4e
    (cherry picked from commit 0cc94bfd5934e793cb8d739e132b52fa40bcddb0)
    (cherry picked from commit 6538175c08f127c20e72e1d347af3eb3d54d6838)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 17.0.0.0b1

This issue was fixed in the openstack/nova 17.0.0.0b1 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 16.0.2

This issue was fixed in the openstack/nova 16.0.2 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 15.0.8

This issue was fixed in the openstack/nova 15.0.8 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.