instance_group_members also exist after delete vm

Bug #1751186 reported by tangxing
20
This bug affects 3 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Low
Matt Riedemann
Queens
Confirmed
Low
Unassigned
Rocky
Fix Committed
Low
Matt Riedemann

Bug Description

Description
===========
instance_group_members record also exist in database after delete vm

Steps to reproduce
==================
1:create affinity instance_group
2:create vm
3:delete vm

Actual result
=============

Database changed
MariaDB [nova_api]> select * from instance_group_member where group_id='3';
+---------------------+------------+-----+--------------------------------------+----------+
| created_at | updated_at | id | instance_uuid | group_id |
+---------------------+------------+-----+--------------------------------------+----------+
| 2018-02-12 09:14:42 | NULL | 106 | 3b9aa471-612f-4967-a405-574241e84aa0 | 3 |
| 2018-02-12 09:15:14 | NULL | 107 | e04244d9-2832-482a-9793-b5cc5b018de2 | 3 |
| 2018-02-22 03:05:40 | NULL | 108 | ff7a2194-9f38-4a7e-a67f-f0c53e409e9b | 3 |
| 2018-02-22 04:07:43 | NULL | 109 | 50000a55-8d0f-4592-ae0f-e9223c3f6a32 | 3 |
| 2018-02-22 08:36:44 | NULL | 110 | 4ef31c5c-d96f-4e24-ba86-75c5576ae4ed | 3 |
| 2018-02-22 09:36:39 | NULL | 111 | 0378c1bb-2d79-4b33-8115-77a75996af38 | 3 |
| 2018-02-23 00:40:05 | NULL | 112 | 08dac72d-8f04-4f7d-8b9a-60b0cf000951 | 3 |
+---------------------+------------+-----+--------------------------------------+----------+
7 rows in set (0.00 sec)

MariaDB [nova_api]> use nova;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Database changed
MariaDB [nova]> select * from instances where uuid='3b9aa471-612f-4967-a405-574241e84aa0';
+---------------------+---------------------+---------------------+-----+-------------+----------------------------------+----------------------------------+--------------------------------------+-----------+------------+--------------+----------+----------+-------------+----------+-----------+-------+------------+-------------+-----------+----------------+---------------------+---------------------+--------------+---------------------+-------------------+--------+---------+-------------+------------------+---------+--------------------------------------+--------------+------------------+--------------+--------------+--------------+------------+--------------------------+---------------------+----------+------------------+--------------------+-------------------+---------+--------------+-----------+-------------+---------+-----------+---------+--------------------+
| created_at | updated_at | deleted_at | id | internal_id | user_id | project_id | image_ref | kernel_id | ramdisk_id | launch_index | key_name | key_data | power_state | vm_state | memory_mb | vcpus | hostname | host | user_data | reservation_id | launched_at | terminated_at | display_name | display_description | availability_zone | locked | os_type | launched_on | instance_type_id | vm_mode | uuid | architecture | root_device_name | access_ip_v4 | access_ip_v6 | config_drive | task_state | default_ephemeral_device | default_swap_device | progress | auto_disk_config | shutdown_terminate | disable_terminate | root_gb | ephemeral_gb | cell_name | node | deleted | locked_by | cleaned | ephemeral_key_uuid |
+---------------------+---------------------+---------------------+-----+-------------+----------------------------------+----------------------------------+--------------------------------------+-----------+------------+--------------+----------+----------+-------------+----------+-----------+-------+------------+-------------+-----------+----------------+---------------------+---------------------+--------------+---------------------+-------------------+--------+---------+-------------+------------------+---------+--------------------------------------+--------------+------------------+--------------+--------------+--------------+------------+--------------------------+---------------------+----------+------------------+--------------------+-------------------+---------+--------------+-----------+-------------+---------+-----------+---------+--------------------+
| 2018-02-12 09:14:42 | 2018-02-13 00:58:48 | 2018-02-13 00:58:48 | 255 | NULL | 86019a90ad0b44609cca8b0eb9eee79b | 01f2572fd19743558be05561e09c3850 | 22164f51-c353-4f8a-a073-cbb2930bf25f | | | 0 | NULL | NULL | 0 | deleted | 2048 | 1 | test-tx001 | nail-5300-2 | NULL | r-j42bg7io | 2018-02-12 09:14:56 | 2018-02-13 00:58:48 | test_tx001 | NULL | test1 | 0 | NULL | nail-5300-2 | 14 | NULL | 3b9aa471-612f-4967-a405-574241e84aa0 | NULL | /dev/vda | NULL | NULL | | NULL | NULL | NULL | 0 | 0 | 0 | 0 | 10 | 0 | NULL | nail-5300-2 | 255 | NULL | 1 | NULL |
+---------------------+---------------------+---------------------+-----+-------------+----------------------------------+----------------------------------+--------------------------------------+-----------+------------+--------------+----------+----------+-------------+----------+-----------+-------+------------+-------------+-----------+----------------+---------------------+---------------------+--------------+---------------------+-------------------+--------+---------+-------------+------------------+---------+--------------------------------------+--------------+------------------+--------------+--------------+--------------+------------+--------------------------+---------------------+----------+------------------+--------------------+-------------------+---------+--------------+-----------+-------------+---------+-----------+---------+--------------------+
1 row in set (0.00 sec)

Tags: api db
Revision history for this message
Sylvain Bauza (sylvain-bauza) wrote :

That looks fine to me, given we lazy-load the members field when calling it. So, I'm pretty sure the InstanceGroup.members shouldn't be saying that 3b9aa471-612f-4967-a405-574241e84aa0 is in there.

Could you please verify that ?
Also, which Nova version are you using ?

tags: added: api db
Changed in nova:
status: New → Incomplete
Revision history for this message
tangxing (tang-xing) wrote :

I‘m sure.Under the same conditions,many times, the results are the same.
nova version: Pike

tangxing (tang-xing)
Changed in nova:
status: Incomplete → New
Revision history for this message
Fan Zhang (fanzhang) wrote :

Hi, there. Does anyone work on it?

I noticed this bug today in my Pike env. IMHO, it's kind inappropriate if we don't delete the records in table instance_group_member after deleting the vms.

This action will eventually cause bigger table, worse query performance.

Revision history for this message
tangxing (tang-xing) wrote :

Let me try work on it

Changed in nova:
assignee: nobody → tangxing (tang-xing)
status: New → Confirmed
importance: Undecided → Low
Changed in nova:
assignee: tangxing (tang-xing) → Chen (chenn2)
status: Confirmed → In Progress
Revision history for this message
Zhenyu Zheng (zhengzhenyu) wrote :

I did this in current master(Rocky), the db records did exists, but yes, InstanceGroup.Members won't have the deleted member so it might not affect the scheduling process, but still, growing db records could be a problem.

Revision history for this message
Matt Riedemann (mriedem) wrote :

See related (duplicate) bug 1778804.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on nova (master)

Change abandoned by Chen (<email address hidden>) on branch: master
Review: https://review.openstack.org/581403
Reason: In favor of approach https://review.openstack.org/#/c/588943

Changed in nova:
assignee: Chen (chenn2) → Matt Riedemann (mriedem)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/588943
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=5162a9a1de3cf4d652707d8a1ff73b7b136b8053
Submitter: Zuul
Branch: master

commit 5162a9a1de3cf4d652707d8a1ff73b7b136b8053
Author: Matt Riedemann <email address hidden>
Date: Sat Aug 4 18:57:35 2018 -0400

    Delete instance_group_member records from API DB during archive

    Like we do for instance mappings and request specs in the API DB
    when archiving deleted instances, this adds code to delete
    instance group member records from the API DB when archiving deleted
    instances. This should improve performance in the server groups
    API because it will have a smaller set of group members to determine
    if they are actually related to deleted instances, see change
    Idd2e35bc95ed98ebc0340ff62e109e23c8adcb21 for context.

    Change-Id: I960f8fd44d98427a72cb2bb0b238fdf2f734390f
    Closes-Bug: #1751186

Changed in nova:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/rocky)

Fix proposed to branch: stable/rocky
Review: https://review.openstack.org/598953

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/rocky)

Reviewed: https://review.openstack.org/598953
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=4c27443e6fe91095318204e7ec86ef97ad5b9efa
Submitter: Zuul
Branch: stable/rocky

commit 4c27443e6fe91095318204e7ec86ef97ad5b9efa
Author: Matt Riedemann <email address hidden>
Date: Sat Aug 4 18:57:35 2018 -0400

    Delete instance_group_member records from API DB during archive

    Like we do for instance mappings and request specs in the API DB
    when archiving deleted instances, this adds code to delete
    instance group member records from the API DB when archiving deleted
    instances. This should improve performance in the server groups
    API because it will have a smaller set of group members to determine
    if they are actually related to deleted instances, see change
    Idd2e35bc95ed98ebc0340ff62e109e23c8adcb21 for context.

    Change-Id: I960f8fd44d98427a72cb2bb0b238fdf2f734390f
    Closes-Bug: #1751186
    (cherry picked from commit 5162a9a1de3cf4d652707d8a1ff73b7b136b8053)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 18.0.1

This issue was fixed in the openstack/nova 18.0.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 19.0.0.0rc1

This issue was fixed in the openstack/nova 19.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.