We have an environment that was set with the default partition power (8) which caused there to be 256 partitions across 3 zones with 3 hosts each, but the usage on this environment is over 500,000 objects. The partition power on this environment needs to be raised from 8 to 18, but it appears that this ability is not available in the product at the moment.
This bug is to track a request to progress and complete implementation of the following spec:
https://specs.openstack.org/openstack/swift-specs/specs/in_progress/increasing_partition_power.html
The current partition power causes over 12 hours of replication lag which then puts us into contention with another issue noted in lp:1691570 with hanging and zombie rsync processes due to "lockup detected killing coros" not actually cleaning up coros fully.
Using Mitaka on Trusty with Swift 2.7.1 and charm version 17.02.
You can likely replicate the issue in lp:1691570 with a swift environment setup on xfs backed storage with over 2.5 million objects per partition.
Hi Drew, it should be possible to modify partition power with the charms today. There are some conditions that need to be met in order for it to work though. To help me figure out what's going on can you please provide the following:
* steps (charm config changes etc) you tried to change partition power
* your swift-proxy charm config
* /var/log/ juju/unit- swift-proxy- <id>.log from your leader unit (dump in a pastebin if possible)
* output of 'swift-ring-builder /etc/swift/ object. ring.gz'