I think container mirroring is an interesting use-case
Currently one option that jumps out is global clusters - if you want all of the objects of a container in region A to also be accessible with all the same metadata and behaviors in region B - you create that container with a storage policy that will house multiple replicas in each region. There might be some deficiencies with how container requests are currently propagated, maybe some inefficiencies with how data uploaded in region A is synced to region B, but I think I'd rather enumerate those and fix them than try to do "container mirroring" via an extension to container-sync that also requires the user to enumerate all the things they want mirrored?
Idk, i suppose I'm currently skeptical mainly because container-sync's ability to scale is tied to the database replicas and container servers. I have a notion that the queue based approach used by the object expirer and reconciler - despite the i/o overhead of duplicating the rows into another container for staging/sequencing - actually has a better chance to scale because the heavy lifting of moving the data can be spread across any N nodes that can reach the queue. But that's somewhat orthogonal to container-metadata syncing.
Do you have another use case for this feature in mind besides mirroring a container between two sites? Because that use-case sounds like "I really want a multi-site storage policy" - but I'm guessing you have some operator requirement that makes the existing implementation there less suitable somehow?
I think container mirroring is an interesting use-case
Currently one option that jumps out is global clusters - if you want all of the objects of a container in region A to also be accessible with all the same metadata and behaviors in region B - you create that container with a storage policy that will house multiple replicas in each region. There might be some deficiencies with how container requests are currently propagated, maybe some inefficiencies with how data uploaded in region A is synced to region B, but I think I'd rather enumerate those and fix them than try to do "container mirroring" via an extension to container-sync that also requires the user to enumerate all the things they want mirrored?
Idk, i suppose I'm currently skeptical mainly because container-sync's ability to scale is tied to the database replicas and container servers. I have a notion that the queue based approach used by the object expirer and reconciler - despite the i/o overhead of duplicating the rows into another container for staging/sequencing - actually has a better chance to scale because the heavy lifting of moving the data can be spread across any N nodes that can reach the queue. But that's somewhat orthogonal to container-metadata syncing.
Do you have another use case for this feature in mind besides mirroring a container between two sites? Because that use-case sounds like "I really want a multi-site storage policy" - but I'm guessing you have some operator requirement that makes the existing implementation there less suitable somehow?