Objects stored on PUT even when etags don't match on object-servers
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Object Storage (swift) |
Triaged
|
Medium
|
Unassigned |
Bug Description
We run a multi region cluster (version 2.7.1 although have also replicated in test with version 2.15.1)
One of our regions has a firewall/
Even though the object PUT returned a 500 the object is stored and is listed in the container etc.
When querying all object servers for the object however we get a different etag for the dodgy region:
root@swift:~# curl -g -I -XHEAD "http://
HTTP/1.1 200 OK
Content-Length: 10
X-Backend-
Content-Type: application/
Last-Modified: Wed, 24 Jan 2018 00:41:09 GMT
Etag: "b2cfa4183267af
X-Timestamp: 1516754468.33842
X-Backend-
X-Backend-
X-Object-
Date: Wed, 24 Jan 2018 22:47:37 GMT
root@swift:~# curl -g -I -XHEAD "http://
HTTP/1.1 200 OK
Content-Length: 10
X-Backend-
Content-Type: application/
Last-Modified: Wed, 24 Jan 2018 00:41:09 GMT
Etag: "9d240a40422483
X-Timestamp: 1516754468.33842
X-Backend-
X-Backend-
X-Object-
Date: Wed, 24 Jan 2018 22:47:37 GMT
root@swift:~# curl -g -I -XHEAD "http://
HTTP/1.1 200 OK
Content-Length: 10
X-Backend-
Content-Type: application/
Last-Modified: Wed, 24 Jan 2018 00:41:09 GMT
Etag: "b2cfa4183267af
X-Timestamp: 1516754468.33842
X-Backend-
X-Backend-
X-Object-
Date: Wed, 24 Jan 2018 22:47:38 GMT
Also upon inspecting the container dbs I see that one of the dbs contains the bad etag, the rest show the good etag so there are inconsistencies there too.
Should some swift process be fixing these inconstancies up somehow? or possibly not allowing the data to be stored at all?
summary: |
- Objects still on PUT even when etags don't match on object-servers + Objects stored on PUT even when etags don't match on object-servers |
Changed in swift: | |
importance: | Undecided → High |
Changed in swift: | |
importance: | High → Medium |
status: | New → Confirmed |
For replicated objects this isn't currently easy to solve. For EC and encryption the MIME/chunked packing protocol supports the proxy sending metadata after the object body which the object-server should consider before the finalize.
Moving forward with https:/ /review. openstack. org/#/c/ 427911/ may offer a framework by which we can have a common pattern between unencrypted replicated and other storage formats that allows us to always send a proxy computed etag behind a object PUT.