CPU usage of object-replicator spikes to more than 100% when
ojbect-replicator calls replicate() to ensure consistency of objects.
Object-replicator generates many REPLICATE requests at the same time
in replicate(), so this generations causes this spike.
This generations are related on the number of partitions. Object-replicater
generates as many requests as the number of partitions on a machine.
In our case, CPU usage reaches 100% when more than 1000 partitions exist
on a physical machine.
If we configure settings as each disk has approximately 100 partitions,
we can use at most 10 disks per machine to avoid this spike.
We think the request generation should be limited to average this spike and
use more disks per machine.
In the simplest case, swift limits the number of request generations
by a config-paramater like the object-auditor.
In the strict case, swift should change the limitation depending on the machine
resources.
(e.g. CPU, I/O, network resources)
(swift version is 1.5.0)
I think this problem is caused by the current mechanism of object-replicator , and besides that we should not set small partitions .
Maybe we need to change the object-replicator mechanism.