Will corruption auditor work if ETag is ommitted?

Asked by Steve

From the notes:
"You can ensure end-to-end data integrity by including an MD5 checksum of your object's data in the ETag header. You are not required to include the ETag header, but it is recommended to ensure that the storage system successfully stored your object's content.

The HTTP response will include the MD5 checksum of the data written to the storage system. If you do not send the ETag in the request, you should compare the value returned with your content's MD5 locally to perform the end-to-end data validation on the client side."

So what happens if the client does not send the ETag? Will the system still be able to find bit-rot, or other forms of corruption? It says it is recommended to check that it was stored properly, so that to me is just a time-of-write check, but what about data-at-rest periodic checks?

And if there is no ETag, how does the system know which version of an object is valid if it is in 3 different zones and they are all different versions of the file (let's say because of corruption).

Thanks!

Question information

Language:
English Edit question
Status:
Solved
For:
OpenStack Object Storage (swift) Edit question
Assignee:
No assignee Edit question
Solved by:
gholt
Solved:
Last query:
Last reply:
Revision history for this message
Best gholt (gholt) said :
#1

Swift will still compute and store the md5 of the objects it stores and use those for auditing purposes, even if the client never gave an etag.

Revision history for this message
Mike Barton (redbo) said :
#2

Yes, swift still calculates the MD5 of the file and stores it as the ETag.

Sending the MD5 as an ETag header just lets swift verify that it received what you intended to upload.

Revision history for this message
Steve (lardcanoe) said :
#3

Thanks gholt, that solved my question.