query option doesn't support special characters

Bug #1314544 reported by ZhiQiang Fan
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
python-ceilometerclient
Fix Released
Undecided
Chris Dent

Bug Description

glance and nova both support special characters in their metadata, even names, but ceilometerclient commands cannot use --query to filter those characters, the problem is caused by the regular expression we use in https://github.com/openstack/python-ceilometerclient/blob/master/ceilometerclient/v2/options.py#L80

here is the test detials:

# ceilometer --version
1.0.9.33
# glance image-create --property key~123=value~123
+--------------------+--------------------------------------+
| Property | Value |
+--------------------+--------------------------------------+
| Property 'key~123' | value~123 |
| checksum | None |
| container_format | None |
| created_at | 2014-04-30T10:09:57.488351 |
| deleted | False |
| deleted_at | None |
| disk_format | None |
| id | f7f822dc-3d81-4794-b593-7698a6bd4721 |
| is_public | False |
| min_disk | 0 |
| min_ram | 0 |
| name | None |
| owner | 23e0bce4a8db4d5e8f0d84d7a1f1bf61 |
| protected | False |
| size | 0 |
| status | queued |
| updated_at | 2014-04-30T10:09:57.488359 |
+--------------------+--------------------------------------+
# ceilometer resource-show -r f7f822dc-3d81-4794-b593-7698a6bd4721
+-------------+--------------------------------------------------------------------------+
| Property | Value |
+-------------+--------------------------------------------------------------------------+
| metadata | {u'status': u'queued', u'name': u'None', u'deleted': u'False', |
| | u'checksum': u'None', u'created_at': u'2014-04-30T10:09:57.488351', |
| | u'disk_format': u'None', u'updated_at': u'2014-04-30T10:09:57.488359', |
| | u'properties.key~123': u'value~123', u'protected': u'False', |
| | u'container_format': u'None', u'min_disk': u'0', u'is_public': u'False', |
| | u'deleted_at': u'None', u'min_ram': u'0', u'size': u'0'} |
| project_id | 23e0bce4a8db4d5e8f0d84d7a1f1bf61 |
| resource_id | f7f822dc-3d81-4794-b593-7698a6bd4721 |
| source | |
| user_id | None |
+-------------+--------------------------------------------------------------------------+
# ceilometer sample-list -m image -q metadata.properties.key~123=value~123
WARNING (http:173) Request returned failure status.
HTTPBadRequest (HTTP 400) ERROR Unknown argument: "123": unrecognized field in query: [<Query u'123' eq u'value~123' None>], valid keys: set(['end', 'start', 'metaquery', 'meter', 'project', 'source', 'user', 'start_timestamp_op', 'resource', 'end_timestamp_op'])
# nova boot --image 389750e2-7a2c-44ca-8bd2-9700745a424d --flavor 1 --meta key~1=value~1 --nic net-id=e6673495-1d96-4de0-9b06-227e8f94eb7d 'special!key'
+--------------------------------------+-----------------------------------------------+
| Property | Value |
+--------------------------------------+-----------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-SRV-ATTR:host | - |
| OS-EXT-SRV-ATTR:hypervisor_hostname | - |
| OS-EXT-SRV-ATTR:instance_name | instance-00000009 |
| OS-EXT-STS:power_state | 0 |
| OS-EXT-STS:task_state | scheduling |
| OS-EXT-STS:vm_state | building |
| OS-SRV-USG:launched_at | - |
| OS-SRV-USG:terminated_at | - |
| accessIPv4 | |
| accessIPv6 | |
| adminPass | 7r3fmMCCouvr |
| config_drive | |
| created | 2014-04-30T10:40:19Z |
| flavor | m1.tiny (1) |
| hostId | |
| id | cb286d91-fefa-4179-9e5a-4e8add0d1825 |
| image | cirros (389750e2-7a2c-44ca-8bd2-9700745a424d) |
| key_name | - |
| metadata | {"key~1": "value~1"} |
| name | special!key |
| os-extended-volumes:volumes_attached | [] |
| progress | 0 |
| security_groups | default |
| status | BUILD |
| tenant_id | 23e0bce4a8db4d5e8f0d84d7a1f1bf61 |
| updated | 2014-04-30T10:40:19Z |
| user_id | 1b419837582145269b26c67041b2b8d3 |
+--------------------------------------+-----------------------------------------------+
# ceilometer resource-show -r cb286d91-fefa-4179-9e5a-4e8add0d1825
+-------------+--------------------------------------------------------------------------+
| Property | Value |
+-------------+--------------------------------------------------------------------------+
| metadata | {u'state_description': u'', u'image_meta.base_image_ref': u'389750e2 |
| | -7a2c-44ca-8bd2-9700745a424d', u'event_type': |
| | u'compute.instance.create.end', u'availability_zone': u'nova', |
| | u'terminated_at': u'', u'ephemeral_gb': u'0', u'instance_type_id': u'2', |
| | u'message': u'Success', u'deleted_at': u'', u'reservation_id': |
| | u'r-g325kqrq', u'instance_id': u'cb286d91-fefa-4179-9e5a-4e8add0d1825', |
| | u'display_name': u'special!key', u'hostname': u'specialkey', u'state': |
| | u'active', u'image_ref_url': u'http://172.25.150.9:9292/images/389750e2 |
| | -7a2c-44ca-8bd2-9700745a424d', u'launched_at': |
| | u'2014-04-30T10:40:25.982597', u'node': u'n707be8ecc11a', u'ramdisk_id': |
| | u'', u'access_ip_v6': u'None', u'disk_gb': u'1', u'access_ip_v4': |
| | u'None', u'kernel_id': u'', u'image_name': u'cirros', |
| | u'image_meta.disk_format': u'qcow2', u'image_meta.container_format': |
| | u'bare', u'user_id': u'1b419837582145269b26c67041b2b8d3', |
| | u'image_meta.min_disk': u'1', u'root_gb': u'1', u'tenant_id': |
| | u'23e0bce4a8db4d5e8f0d84d7a1f1bf61', u'created_at': |
| | u'2014-04-30T10:40:19.328585', u'image_meta.min_ram': u'0', u'host': |
| | u'compute.n707be8ecc11a', u'memory_mb': u'512', u'instance_type': |
| | u'm1.tiny', u'vcpus': u'1', u'architecture': u'None', u'os_type': |
| | u'None', u'instance_flavor_id': u'1'} |
| project_id | 23e0bce4a8db4d5e8f0d84d7a1f1bf61 |
| resource_id | cb286d91-fefa-4179-9e5a-4e8add0d1825 |
| source | |
| user_id | 1b419837582145269b26c67041b2b8d3 |
+-------------+--------------------------------------------------------------------------+
# ceilometer sample-list -m instance -q 'metadata.display_name=special!key'
+-------------+------+------+--------+------+-----------+
| Resource ID | Name | Type | Volume | Unit | Timestamp |
+-------------+------+------+--------+------+-----------+
+-------------+------+------+--------+------+-----------+

ZhiQiang Fan (aji-zqfan)
Changed in python-ceilometerclient:
assignee: nobody → ZhiQiang Fan (aji-zqfan)
Revision history for this message
Openstack Gerrit (openstack-gerrit) wrote : Fix proposed to python-ceilometerclient (master)

Fix proposed to branch: master
Review: https://review.openstack.org/92074

Changed in python-ceilometerclient:
status: New → In Progress
Changed in python-ceilometerclient:
assignee: ZhiQiang Fan (aji-zqfan) → Chris Dent (chdent)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on python-ceilometerclient (master)

Change abandoned by ZhiQiang Fan (<email address hidden>) on branch: master
Review: https://review.openstack.org/92074
Reason: @Chris Dent

I'm sad that this patch is reviewed so late, and it is not a so good fix to the original problem (although it does fix), so I'm going to abandon this one, you can merge (or drop all of) the code to you patch

thanks

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to python-ceilometerclient (master)

Reviewed: https://review.openstack.org/96514
Committed: https://git.openstack.org/cgit/openstack/python-ceilometerclient/commit/?id=2a2e2dd2732240a08d7f0e1288229a39ae380d1c
Submitter: Jenkins
Branch: master

commit 2a2e2dd2732240a08d7f0e1288229a39ae380d1c
Author: Chris Dent <email address hidden>
Date: Fri May 30 11:40:11 2014 +0100

    Refactor split_by_op and split_by_datatype

    split_by_op was returning data against which further error checking was
    then required. This change moves that error handling inside the method
    and simplifies the regular expression so that it splits (greedily) on
    the first operator it finds. If the split is not possible, it is
    a ValueError. If the field or value are empty, that is a ValueError.

    Both split_by_op and split_by_datatype were doing a findall() where
    a match() and split() do the right job and more efficiently.

    Regular expression compilation has been moved to the module level
    to insure they need only be compiled once. Operator keys must be
    sorted by length to ensure the point at which the split happens is
    most greedy. Using a split keeps the regex short and removes any
    statements about the left and right hand sides of the operator.

    Tests added to cover the method more completely, including testing
    for corner cases such as single character field or values or
    operators showing up in unexpected locations. 'string' variable
    renamed to 'query' and 'query_value' to avoid confusion. Named parameters
    on string substitution for clarity.

    Note that the tests which do self.assertRaises could more explicitly
    check the exception with self.assertRaisesRegexp but that would
    break compatibility with Python 2.6.

    Change-Id: Icd815ff65aba9eae3f76afee3bb33e85d85bea72
    Closes-Bug: #1314544

Changed in python-ceilometerclient:
status: In Progress → Fix Committed
Changed in python-ceilometerclient:
milestone: none → 1.3.0
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.