cinder backup not releasing the memory after backup create

Need to add more detail on this case, release version, backup driver, command or API, and the most important if you can provide, the backup-log.

release: stable/pike

Test uses the backup create API
backup logs attached.

create backup for volume 1.7 TB

backup_file_size = 268435456
backup_sha_block_size_bytes = 32768
backup_driver = cinder.backup.drivers.posix
backup_compression_algorithm = none

Let me know if we need any more details.

Can't reproduce this failure with stable/pikre release on my own devstack environment :)

The above usecase was with 1.7 TB ( ~1700GB), To simplify I tried to recreate using the 50 GB just to show the locked memory usage by cinder-backup.

[root@pvc180 cinder]# cinder --service-type volume backup-create --name chec-backup b0f5eec3-1e25-4b2e-8b91-f947b9361dfc
+-----------+--------------------------------------+
| Property | Value |
+-----------+--------------------------------------+
| id | 4d2a21e4-63e7-4018-a5ba-5c79bdcbfee1 |
| name | chec-backup |
| volume_id | b0f5eec3-1e25-4b2e-8b91-f947b9361dfc |
+-----------+--------------------------------------+
[root@pvc180 cinder]# cinder --service-type volume backup-list
+--------------------------------------+--------------------------------------+----------+-------------+------+--------------+-------------------------- ------------------+
| ID | Volume ID | Status | Name | Size | Object Count | Container |
+--------------------------------------+--------------------------------------+----------+-------------+------+--------------+-------------------------- ------------------+
| 4d2a21e4-63e7-4018-a5ba-5c79bdcbfee1 | b0f5eec3-1e25-4b2e-8b91-f947b9361dfc | creating | chec-backup | 50 | 0 | 4d/2a/4d2a21e4-63e7-4018- a5ba-5c79bdcbfee1 |
+--------------------------------------+--------------------------------------+----------+-------------+------+--------------+-------------------------- ------------------+

Memory consumption by cinder-backup
top - 04:49:27 up 1 day, 3:57, 6 users, load average: 1.96, 1.69, 1.24
Tasks: 1 total, 1 running, 0 sleeping, 0 stopped, 0 zombie
%Cpu(s): 3.8 us, 0.4 sy, 0.0 ni, 95.2 id, 0.6 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 15684416 total, 500672 free, 10916160 used, 4267584 buff/cache
KiB Swap: 524224 total, 524224 free, 0 used. 3496064 avail Mem

  PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
23693 cinder 20 0 859008 630592 35584 R 73.7 4.0 13:19.05 cinder-backup

Once the backup completes only half of the memory is released.
top - 04:50:18 up 1 day, 3:57, 6 users, load average: 2.04, 1.76, 1.29
Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie
%Cpu(s): 1.4 us, 0.2 sy, 0.0 ni, 98.4 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem : 15684416 total, 2085312 free, 10653504 used, 2945600 buff/cache
KiB Swap: 524224 total, 524224 free, 0 used. 3759040 avail Mem

  PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
23693 cinder 20 0 600000 372800 35584 S 2.0 2.4 13:49.59 cinder-backup

With huge volumes this impact ends up in 40% of memory being locked.