Openstack image upload failed with evicted pods
Bug #1943674 reported by
OpenInfra
This bug affects 1 person
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
StarlingX |
New
|
Undecided
|
Unassigned |
Bug Description
In a freshly installed stx release 5 (standard dedicated storage) system, I tried to upload 41GB of qcow2 image but failed [1]. Noticed that some of the pods are evicted [2].
Ceph cluster has enough space and health is ok [3].
I can upload smaller files without any issue.
[1] https:/
[2] https:/
[3] https:/
tags: | added: stx.5.0 |
To post a comment you must log in.
I have increased both docker (150GB) and kublet (25GB) size in both controllers while working on this issue. And also increase the ceph mon size. [0]. /paste. opendev. org/show/ 809328/
https:/
Disk usage of kubelet controller-0: [1] and controller-1: [2]
Disk usage of controller-0 [3] and controller-1 [4].
There were couple of pods failed with the following error and few failed with DiskPressure [7][8][9].
The node was low on resource: ephemeral-storage. Container horizon was using 19361853, which exceeds its request of 0. [5][6]
I was able to upload the image after increasing kubelet size to 50GB.
[0] https:/ /paste. opendev. org/show/ 809328/ /paste. opendev. org/show/ 809325/ /paste. opendev. org/show/ 809324/ /paste. opendev. org/show/ 809326/ /paste. opendev. org/show/ 809323/
[1] https:/
[2] https:/
[3] https:/
[4] https:/
[5] https:/ /paste. opendev. org/show/ 809320/ /paste. opendev. org/show/ 809319/ /paste. opendev. org/show/ 809318/ /paste. opendev. org/show/ 809321/ /paste. opendev. org/show/ 809322/
[6] https:/
[7] https:/
[8] https:/
[9] https:/