Ironic's postgres CI test job has started to fail with an error rooted in Neutron's database API layer. Specifically how that API is interacting with SQLAlchemy to interact with postgres.
Error:
DBAPIError exception wrapped.: psycopg2.errors.GroupingError: column "subnet_service_types.subnet_id" must appear in the GROUP BY clause or be used in an aggregate function
This is likely just a command formatting issue in the database interaction, and should be easily fixed.
Job Logs: https://96a560a38139b70cb224-e9f29c7afce5197c5c20e02f6b6da59e.ssl.cf5.rackcdn.com/888500/7/check/ironic-tempest-pxe_ipmitool-postgres/7eeffae/controller/logs/screen-q-svc.txt
Full error:
Jul 17 15:02:54.203622 np0034696541 neutron-server[69958]: DEBUG neutron.pecan_wsgi.hooks.quota_enforcement [req-36ab3b86-999f-48a0-87f8-e2613909b6c4 req-8aa9b5ab-4403-42dc-b82e-c28f1a37c843 tempest-BaremetalBasicOps-471932799 tempest-BaremetalBasicOps-471932799-project-member] Made reservation on behalf of 9d6bf2710477411887e0dcc4386b458a for: {'port': 1} {{(pid=69958) before /opt/stack/neutron/neutron/pecan_wsgi/hooks/quota_enforcement.py:53}}
Jul 17 15:02:54.206063 np0034696541 neutron-server[69958]: DEBUG neutron_lib.callbacks.manager [req-36ab3b86-999f-48a0-87f8-e2613909b6c4 req-8aa9b5ab-4403-42dc-b82e-c28f1a37c843 tempest-BaremetalBasicOps-471932799 tempest-BaremetalBasicOps-471932799-project-member] Publish callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler-595366'] for port (None), before_create {{(pid=69958) _notify_loop /usr/local/lib/python3.10/dist-packages/neutron_lib/callbacks/manager.py:176}}
Jul 17 15:02:54.215796 np0034696541 neutron-server[69958]: WARNING oslo_db.sqlalchemy.exc_filters [req-36ab3b86-999f-48a0-87f8-e2613909b6c4 req-8aa9b5ab-4403-42dc-b82e-c28f1a37c843 tempest-BaremetalBasicOps-471932799 tempest-BaremetalBasicOps-471932799-project-member] DBAPIError exception wrapped.: psycopg2.errors.GroupingError: column "subnet_service_types.subnet_id" must appear in the GROUP BY clause or be used in an aggregate function
Jul 17 15:02:54.215796 np0034696541 neutron-server[69958]: LINE 2: ...de, subnets.standard_attr_id AS standard_attr_id, subnet_ser...
Jul 17 15:02:54.215796 np0034696541 neutron-server[69958]: ^
Jul 17 15:02:54.215796 np0034696541 neutron-server[69958]: ERROR oslo_db.sqlalchemy.exc_filters Traceback (most recent call last):
Jul 17 15:02:54.215796 np0034696541 neutron-server[69958]: ERROR oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python3.10/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
Jul 17 15:02:54.215796 np0034696541 neutron-server[69958]: ERROR oslo_db.sqlalchemy.exc_filters self.dialect.do_execute(
Jul 17 15:02:54.215796 np0034696541 neutron-server[69958]: ERROR oslo_db.sqlalchemy.exc_filters File "/usr/local/lib/python3.10/dist-packages/sqlalchemy/engine/default.py", line 736, in do_execute
Jul 17 15:02:54.215796 np0034696541 neutron-server[69958]: ERROR oslo_db.sqlalchemy.exc_filters cursor.execute(statement, parameters)
Jul 17 15:02:54.215796 np0034696541 neutron-server[69958]: ERROR oslo_db.sqlalchemy.exc_filters psycopg2.errors.GroupingError: column "subnet_service_types.subnet_id" must appear in the GROUP BY clause or be used in an aggregate function
Jul 17 15:02:54.215796 np0034696541 neutron-server[69958]: ERROR oslo_db.sqlalchemy.exc_filters LINE 2: ...de, subnets.standard_attr_id AS standard_attr_id, subnet_ser...
Jul 17 15:02:54.215796 np0034696541 neutron-server[69958]: ERROR oslo_db.sqlalchemy.exc_filters ^
Jul 17 15:02:54.215796 np0034696541 neutron-server[69958]: ERROR oslo_db.sqlalchemy.exc_filters
Jul 17 15:02:54.215796 np0034696541 neutron-server[69958]: ERROR oslo_db.sqlalchemy.exc_filters
Jul 17 15:02:54.218977 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation [req-36ab3b86-999f-48a0-87f8-e2613909b6c4 req-8aa9b5ab-4403-42dc-b82e-c28f1a37c843 tempest-BaremetalBasicOps-471932799 tempest-BaremetalBasicOps-471932799-project-member] POST failed.: oslo_db.exception.DBError: (psycopg2.errors.GroupingError) column "subnet_service_types.subnet_id" must appear in the GROUP BY clause or be used in an aggregate function
Jul 17 15:02:54.218977 np0034696541 neutron-server[69958]: LINE 2: ...de, subnets.standard_attr_id AS standard_attr_id, subnet_ser...
Jul 17 15:02:54.218977 np0034696541 neutron-server[69958]: ^
Jul 17 15:02:54.218977 np0034696541 neutron-server[69958]: [SQL: SELECT anon_1.project_id AS anon_1_project_id, anon_1.id AS anon_1_id, anon_1.name AS anon_1_name, anon_1.network_id AS anon_1_network_id, anon_1.segment_id AS anon_1_segment_id, anon_1.subnetpool_id AS anon_1_subnetpool_id, anon_1.ip_version AS anon_1_ip_version, anon_1.cidr AS anon_1_cidr, anon_1.gateway_ip AS anon_1_gateway_ip, anon_1.enable_dhcp AS anon_1_enable_dhcp, anon_1.ipv6_ra_mode AS anon_1_ipv6_ra_mode, anon_1.ipv6_address_mode AS anon_1_ipv6_address_mode, anon_1.standard_attr_id AS anon_1_standard_attr_id, subnetpools_1.shared AS subnetpools_1_shared, subnetpoolrbacs_1.project_id AS subnetpoolrbacs_1_project_id, subnetpoolrbacs_1.id AS subnetpoolrbacs_1_id, subnetpoolrbacs_1.target_project AS subnetpoolrbacs_1_target_project, subnetpoolrbacs_1.action AS subnetpoolrbacs_1_action, subnetpoolrbacs_1.object_id AS subnetpoolrbacs_1_object_id, standardattributes_1.id AS standardattributes_1_id, standardattributes_1.resource_type AS standardattributes_1_resource_type, standardattributes_1.description AS standardattributes_1_description, standardattributes_1.revision_number AS standardattributes_1_revision_number, standardattributes_1.created_at AS standardattributes_1_created_at, standardattributes_1.updated_at AS standardattributes_1_updated_at, subnetpools_1.project_id AS subnetpools_1_project_id, subnetpools_1.id AS subnetpools_1_id, subnetpools_1.name AS subnetpools_1_name, subnetpools_1.ip_version AS subnetpools_1_ip_version, subnetpools_1.default_prefixlen AS subnetpools_1_default_prefixlen, subnetpools_1.min_prefixlen AS subnetpools_1_min_prefixlen, subnetpools_1.max_prefixlen AS subnetpools_1_max_prefixlen, subnetpools_1.is_default AS subnetpools_1_is_default, subnetpools_1.default_quota AS subnetpools_1_default_quota, subnetpools_1.hash AS subnetpools_1_hash, subnetpools_1.address_scope_id AS subnetpools_1_address_scope_id, subnetpools_1.standard_attr_id AS subnetpools_1_standard_attr_id, networkrbacs_1.project_id AS networkrbacs_1_project_id, networkrbacs_1.id AS networkrbacs_1_id, networkrbacs_1.target_project AS networkrbacs_1_target_project, networkrbacs_1.action AS networkrbacs_1_action, networkrbacs_1.object_id AS networkrbacs_1_object_id, standardattributes_2.id AS standardattributes_2_id, standardattributes_2.resource_type AS standardattributes_2_resource_type, standardattributes_2.description AS standardattributes_2_description, standardattributes_2.revision_number AS standardattributes_2_revision_number, standardattributes_2.created_at AS standardattributes_2_created_at, standardattributes_2.updated_at AS standardattributes_2_updated_at, subnet_dns_publish_fixed_ips_1.subnet_id AS subnet_dns_publish_fixed_ips_1_subnet_id, subnet_dns_publish_fixed_ips_1.dns_publish_fixed_ip AS subnet_dns_publish_fixed_ips_1_dns_publish_fixed_ip
Jul 17 15:02:54.218977 np0034696541 neutron-server[69958]: FROM (SELECT subnets.project_id AS project_id, subnets.id AS id, subnets.name AS name, subnets.network_id AS network_id, subnets.segment_id AS segment_id, subnets.subnetpool_id AS subnetpool_id, subnets.ip_version AS ip_version, subnets.cidr AS cidr, subnets.gateway_ip AS gateway_ip, subnets.enable_dhcp AS enable_dhcp, subnets.ipv6_ra_mode AS ipv6_ra_mode, subnets.ipv6_address_mode AS ipv6_address_mode, subnets.standard_attr_id AS standard_attr_id, subnet_service_types.subnet_id AS subnet_id, subnet_service_types.service_type AS service_type
Jul 17 15:02:54.218977 np0034696541 neutron-server[69958]: FROM subnets LEFT OUTER JOIN networkrbacs ON subnets.network_id = networkrbacs.object_id LEFT OUTER JOIN subnet_service_types ON subnets.id = subnet_service_types.subnet_id
Jul 17 15:02:54.218977 np0034696541 neutron-server[69958]: WHERE (subnets.project_id = %(project_id_1)s OR networkrbacs.action IN (%(action_1_1)s, %(action_1_2)s) AND (networkrbacs.target_project = %(target_project_1)s OR networkrbacs.target_project = %(target_project_2)s)) AND subnets.network_id = %(network_id_1)s AND (subnet_service_types.service_type IS NULL OR subnet_service_types.service_type = %(service_type_1)s OR false) GROUP BY subnets.id) AS anon_1 LEFT OUTER JOIN subnetpools AS subnetpools_1 ON anon_1.subnetpool_id = subnetpools_1.id LEFT OUTER JOIN subnetpoolrbacs AS subnetpoolrbacs_1 ON subnetpools_1.id = subnetpoolrbacs_1.object_id LEFT OUTER JOIN standardattributes AS standardattributes_1 ON standardattributes_1.id = subnetpools_1.standard_attr_id LEFT OUTER JOIN networkrbacs AS networkrbacs_1 ON anon_1.network_id = networkrbacs_1.object_id LEFT OUTER JOIN standardattributes AS standardattributes_2 ON standardattributes_2.id = anon_1.standard_attr_id LEFT OUTER JOIN subnet_dns_publish_fixed_ips AS subnet_dns_publish_fixed_ips_1 ON anon_1.id = subnet_dns_publish_fixed_ips_1.subnet_id
Jul 17 15:02:54.218977 np0034696541 neutron-server[69958]: WHERE anon_1.segment_id IS NULL]
Jul 17 15:02:54.218977 np0034696541 neutron-server[69958]: [parameters: {'project_id_1': '9d6bf2710477411887e0dcc4386b458a', 'target_project_1': '9d6bf2710477411887e0dcc4386b458a', 'target_project_2': '*', 'network_id_1': '1d7db2b4-d1be-4559-b226-56f7d7a877fe', 'service_type_1': '', 'action_1_1': 'access_as_shared', 'action_1_2': 'access_as_readonly'}]
Jul 17 15:02:54.218977 np0034696541 neutron-server[69958]: (Background on this error at: https://sqlalche.me/e/14/f405)
Jul 17 15:02:54.218977 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation Traceback (most recent call last):
Jul 17 15:02:54.218977 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
Jul 17 15:02:54.218977 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation self.dialect.do_execute(
Jul 17 15:02:54.218977 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/sqlalchemy/engine/default.py", line 736, in do_execute
Jul 17 15:02:54.218977 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation cursor.execute(statement, parameters)
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation psycopg2.errors.GroupingError: column "subnet_service_types.subnet_id" must appear in the GROUP BY clause or be used in an aggregate function
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation LINE 2: ...de, subnets.standard_attr_id AS standard_attr_id, subnet_ser...
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation ^
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation The above exception was the direct cause of the following exception:
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation Traceback (most recent call last):
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/pecan/core.py", line 693, in __call__
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation self.invoke_controller(controller, args, kwargs, state)
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/pecan/core.py", line 584, in invoke_controller
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation result = controller(*args, **kwargs)
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/neutron_lib/db/api.py", line 137, in wrapped
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation with excutils.save_and_reraise_exception():
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation self.force_reraise()
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation raise self.value
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/neutron_lib/db/api.py", line 135, in wrapped
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation return f(*args, **kwargs)
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/oslo_db/api.py", line 144, in wrapper
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation with excutils.save_and_reraise_exception() as ectxt:
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation self.force_reraise()
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation raise self.value
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/oslo_db/api.py", line 142, in wrapper
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation return f(*args, **kwargs)
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/neutron_lib/db/api.py", line 183, in wrapped
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation with excutils.save_and_reraise_exception():
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation self.force_reraise()
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation raise self.value
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/neutron_lib/db/api.py", line 181, in wrapped
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation return f(*dup_args, **dup_kwargs)
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/opt/stack/neutron/neutron/pecan_wsgi/controllers/utils.py", line 65, in wrapped
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation return f(*args, **kwargs)
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/opt/stack/neutron/neutron/pecan_wsgi/controllers/resource.py", line 163, in post
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation return self.create(resources)
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/opt/stack/neutron/neutron/pecan_wsgi/controllers/resource.py", line 181, in create
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation return {key: creator(*creator_args, **creator_kwargs)}
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/opt/stack/neutron/neutron/common/utils.py", line 731, in inner
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation return f(*args, **kwargs)
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/neutron_lib/db/api.py", line 223, in wrapped
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation return f_with_retry(*args, **kwargs,
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/neutron_lib/db/api.py", line 137, in wrapped
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation with excutils.save_and_reraise_exception():
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation self.force_reraise()
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise
Jul 17 15:02:54.220270 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation raise self.value
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/neutron_lib/db/api.py", line 135, in wrapped
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation return f(*args, **kwargs)
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/oslo_db/api.py", line 144, in wrapper
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation with excutils.save_and_reraise_exception() as ectxt:
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation self.force_reraise()
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation raise self.value
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/oslo_db/api.py", line 142, in wrapper
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation return f(*args, **kwargs)
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/neutron_lib/db/api.py", line 183, in wrapped
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation with excutils.save_and_reraise_exception():
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 227, in __exit__
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation self.force_reraise()
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation raise self.value
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/neutron_lib/db/api.py", line 181, in wrapped
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation return f(*dup_args, **dup_kwargs)
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/opt/stack/neutron/neutron/plugins/ml2/plugin.py", line 1586, in create_port
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation result, mech_context = self._create_port_db(context, port)
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/opt/stack/neutron/neutron/plugins/ml2/plugin.py", line 1551, in _create_port_db
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation port_db = self.create_port_db(context, port)
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/opt/stack/neutron/neutron/db/db_base_plugin_v2.py", line 1538, in create_port_db
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation self.ipam.allocate_ips_for_port_and_store(
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/opt/stack/neutron/neutron/db/ipam_pluggable_backend.py", line 219, in allocate_ips_for_port_and_store
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation ips = self.allocate_ips_for_port(context, port_copy)
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/neutron_lib/db/api.py", line 221, in wrapped
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation return f(*args, **kwargs)
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1044, in wrapper
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation return fn(*args, **kwargs)
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/opt/stack/neutron/neutron/db/ipam_pluggable_backend.py", line 226, in allocate_ips_for_port
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation return self._allocate_ips_for_port(context, port)
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/opt/stack/neutron/neutron/db/ipam_pluggable_backend.py", line 258, in _allocate_ips_for_port
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation subnets = self._ipam_get_subnets(
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/opt/stack/neutron/neutron/db/ipam_backend_mixin.py", line 679, in _ipam_get_subnets
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation subnets = subnet_obj.Subnet.find_candidate_subnets(
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/opt/stack/neutron/neutron/objects/subnet.py", line 351, in find_candidate_subnets
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation ).all()
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/sqlalchemy/orm/query.py", line 2772, in all
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation return self._iter().all()
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/sqlalchemy/orm/query.py", line 2907, in _iter
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation result = self.session.execute(
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/sqlalchemy/orm/session.py", line 1712, in execute
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation result = conn._execute_20(statement, params or {}, execution_options)
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation return meth(self, args_10style, kwargs_10style, execution_options)
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation return connection._execute_clauseelement(
Jul 17 15:02:54.223946 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement
Jul 17 15:02:54.227676 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation ret = self._execute_context(
Jul 17 15:02:54.227676 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context
Jul 17 15:02:54.227676 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation self._handle_dbapi_exception(
Jul 17 15:02:54.227676 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception
Jul 17 15:02:54.227676 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation util.raise_(newraise, with_traceback=exc_info[2], from_=e)
Jul 17 15:02:54.227676 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_
Jul 17 15:02:54.227676 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation raise exception
Jul 17 15:02:54.227676 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
Jul 17 15:02:54.227676 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation self.dialect.do_execute(
Jul 17 15:02:54.227676 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation File "/usr/local/lib/python3.10/dist-packages/sqlalchemy/engine/default.py", line 736, in do_execute
Jul 17 15:02:54.227676 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation cursor.execute(statement, parameters)
Jul 17 15:02:54.227676 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation oslo_db.exception.DBError: (psycopg2.errors.GroupingError) column "subnet_service_types.subnet_id" must appear in the GROUP BY clause or be used in an aggregate function
Jul 17 15:02:54.227676 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation LINE 2: ...de, subnets.standard_attr_id AS standard_attr_id, subnet_ser...
Jul 17 15:02:54.227676 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation ^
Jul 17 15:02:54.227676 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation
Jul 17 15:02:54.227676 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation [SQL: SELECT anon_1.project_id AS anon_1_project_id, anon_1.id AS anon_1_id, anon_1.name AS anon_1_name, anon_1.network_id AS anon_1_network_id, anon_1.segment_id AS anon_1_segment_id, anon_1.subnetpool_id AS anon_1_subnetpool_id, anon_1.ip_version AS anon_1_ip_version, anon_1.cidr AS anon_1_cidr, anon_1.gateway_ip AS anon_1_gateway_ip, anon_1.enable_dhcp AS anon_1_enable_dhcp, anon_1.ipv6_ra_mode AS anon_1_ipv6_ra_mode, anon_1.ipv6_address_mode AS anon_1_ipv6_address_mode, anon_1.standard_attr_id AS anon_1_standard_attr_id, subnetpools_1.shared AS subnetpools_1_shared, subnetpoolrbacs_1.project_id AS subnetpoolrbacs_1_project_id, subnetpoolrbacs_1.id AS subnetpoolrbacs_1_id, subnetpoolrbacs_1.target_project AS subnetpoolrbacs_1_target_project, subnetpoolrbacs_1.action AS subnetpoolrbacs_1_action, subnetpoolrbacs_1.object_id AS subnetpoolrbacs_1_object_id, standardattributes_1.id AS standardattributes_1_id, standardattributes_1.resource_type AS standardattributes_1_resource_type, standardattributes_1.description AS standardattributes_1_description, standardattributes_1.revision_number AS standardattributes_1_revision_number, standardattributes_1.created_at AS standardattributes_1_created_at, standardattributes_1.updated_at AS standardattributes_1_updated_at, subnetpools_1.project_id AS subnetpools_1_project_id, subnetpools_1.id AS subnetpools_1_id, subnetpools_1.name AS subnetpools_1_name, subnetpools_1.ip_version AS subnetpools_1_ip_version, subnetpools_1.default_prefixlen AS subnetpools_1_default_prefixlen, subnetpools_1.min_prefixlen AS subnetpools_1_min_prefixlen, subnetpools_1.max_prefixlen AS subnetpools_1_max_prefixlen, subnetpools_1.is_default AS subnetpools_1_is_default, subnetpools_1.default_quota AS subnetpools_1_default_quota, subnetpools_1.hash AS subnetpools_1_hash, subnetpools_1.address_scope_id AS subnetpools_1_address_scope_id, subnetpools_1.standard_attr_id AS subnetpools_1_standard_attr_id, networkrbacs_1.project_id AS networkrbacs_1_project_id, networkrbacs_1.id AS networkrbacs_1_id, networkrbacs_1.target_project AS networkrbacs_1_target_project, networkrbacs_1.action AS networkrbacs_1_action, networkrbacs_1.object_id AS networkrbacs_1_object_id, standardattributes_2.id AS standardattributes_2_id, standardattributes_2.resource_type AS standardattributes_2_resource_type, standardattributes_2.description AS standardattributes_2_description, standardattributes_2.revision_number AS standardattributes_2_revision_number, standardattributes_2.created_at AS standardattributes_2_created_at, standardattributes_2.updated_at AS standardattributes_2_updated_at, subnet_dns_publish_fixed_ips_1.subnet_id AS subnet_dns_publish_fixed_ips_1_subnet_id, subnet_dns_publish_fixed_ips_1.dns_publish_fixed_ip AS subnet_dns_publish_fixed_ips_1_dns_publish_fixed_ip
Jul 17 15:02:54.227676 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation FROM (SELECT subnets.project_id AS project_id, subnets.id AS id, subnets.name AS name, subnets.network_id AS network_id, subnets.segment_id AS segment_id, subnets.subnetpool_id AS subnetpool_id, subnets.ip_version AS ip_version, subnets.cidr AS cidr, subnets.gateway_ip AS gateway_ip, subnets.enable_dhcp AS enable_dhcp, subnets.ipv6_ra_mode AS ipv6_ra_mode, subnets.ipv6_address_mode AS ipv6_address_mode, subnets.standard_attr_id AS standard_attr_id, subnet_service_types.subnet_id AS subnet_id, subnet_service_types.service_type AS service_type
Jul 17 15:02:54.227676 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation FROM subnets LEFT OUTER JOIN networkrbacs ON subnets.network_id = networkrbacs.object_id LEFT OUTER JOIN subnet_service_types ON subnets.id = subnet_service_types.subnet_id
Jul 17 15:02:54.229096 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation WHERE (subnets.project_id = %(project_id_1)s OR networkrbacs.action IN (%(action_1_1)s, %(action_1_2)s) AND (networkrbacs.target_project = %(target_project_1)s OR networkrbacs.target_project = %(target_project_2)s)) AND subnets.network_id = %(network_id_1)s AND (subnet_service_types.service_type IS NULL OR subnet_service_types.service_type = %(service_type_1)s OR false) GROUP BY subnets.id) AS anon_1 LEFT OUTER JOIN subnetpools AS subnetpools_1 ON anon_1.subnetpool_id = subnetpools_1.id LEFT OUTER JOIN subnetpoolrbacs AS subnetpoolrbacs_1 ON subnetpools_1.id = subnetpoolrbacs_1.object_id LEFT OUTER JOIN standardattributes AS standardattributes_1 ON standardattributes_1.id = subnetpools_1.standard_attr_id LEFT OUTER JOIN networkrbacs AS networkrbacs_1 ON anon_1.network_id = networkrbacs_1.object_id LEFT OUTER JOIN standardattributes AS standardattributes_2 ON standardattributes_2.id = anon_1.standard_attr_id LEFT OUTER JOIN subnet_dns_publish_fixed_ips AS subnet_dns_publish_fixed_ips_1 ON anon_1.id = subnet_dns_publish_fixed_ips_1.subnet_id
Jul 17 15:02:54.229096 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation WHERE anon_1.segment_id IS NULL]
Jul 17 15:02:54.229096 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation [parameters: {'project_id_1': '9d6bf2710477411887e0dcc4386b458a', 'target_project_1': '9d6bf2710477411887e0dcc4386b458a', 'target_project_2': '*', 'network_id_1': '1d7db2b4-d1be-4559-b226-56f7d7a877fe', 'service_type_1': '', 'action_1_1': 'access_as_shared', 'action_1_2': 'access_as_readonly'}]
Jul 17 15:02:54.229096 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation (Background on this error at: https://sqlalche.me/e/14/f405)
Jul 17 15:02:54.229096 np0034696541 neutron-server[69958]: ERROR neutron.pecan_wsgi.hooks.translation
[1] was marked as duplicated of this bug. But it worths mentioning (from this bug) that the Neutron periodic CI job "neutron- ovn-tempest- postgres- full" was also failing. That will help debugging this issue only with the Neutron "experimental" queue.
[1]https:/ /bugs.launchpad .net/neutron/ +bug/2028037