OpenStack Security Notes

Bug #1990157
Comment #27

Comment 27 for bug 1990157

Revision history for this message

Brian Rosmaita (brian-rosmaita) wrote on 2022-09-27: Re: Malicious image data modification can happen when using COW

#27

OK, I think I am confusing two issues here:

1. The image-location-substitution-via-Image-API exploit. That can be prevented solely on the glance side by Erno's suggestion -- not allowing a location to be visible on an image until the data it points to has been hashed, and the result either equals the os_hash_value (if present), or sets the os_hash_value (if it's empty).

2. Backend-data-substitution exploit. Even with #1 closed, we still have the situation where:
  time t: end user requests nova createImage action
  time t+1: glance posts os_hash_value
  time t+2: end user downloads image and computes hash, OK
  time t+3: end user requests nova to boot an instance from the image
If nova doesn't check the hash before booting the image, which it doesn't in the COW configuration, then how does the end user know that the image data hasn't been modified between t+2 and t+3? This exploit is facilitated by exposing the direct_url or locations on images to end users, so having an internal-facing-only glance is still good to have, at least until a single Glance can handle only exposing the direct_url and locations to other services.
  But it doesn't solve the larger issue of backend-data-substitution. However, if Glance is doing #1, then it would now at least be *possible* for consuming services to verify image data, and it would be up to nova and cinder to decide how to handle this. And since this requires the malicious actor to independently compromise the storage backend, it's not as critical as #1.

So, if we can get #1 done quickly in a way that doesn't kill the performance of hyperconverged infrastructure, then I am OK with keeping this private. I think that #2 is a real problem, however, that could use some discussion at the PTG. My question is whether we think the backend-data-substitution exploit can be discussed without revealing #1. Of course, if we can get #1 completed and new glance releases out before 17 October, then that's not an issue.

OK, I think I am confusing two issues here:

1. The image-location-substitution-via-Image-API exploit.  That can be prevented solely on the glance side by Erno's suggestion -- not allowing a location to be visible on an image until the data it points to has been hashed, and the result either equals the os_hash_value (if present), or sets the os_hash_value (if it's empty).

2. Backend-data-substitution exploit.  Even with #1 closed, we still have the situation where:
  time t: end user requests nova createImage action
  time t+1: glance posts os_hash_value
  time t+2: end user downloads image and computes hash, OK
  time t+3: end user requests nova to boot an instance from the image
If nova doesn't check the hash before booting the image, which it doesn't in the COW configuration, then how does the end user know that the image data hasn't been modified between t+2 and t+3?  This exploit is facilitated by exposing the direct_url or locations on images to end users, so having an internal-facing-only glance is still good to have, at least until a single Glance can handle only exposing the direct_url and locations to other services.
  But it doesn't solve the larger issue of backend-data-substitution.  However, if Glance is doing #1, then it would now at least be *possible* for consuming services to verify image data, and it would be up to nova and cinder to decide how to handle this.  And since this requires the malicious actor to independently compromise the storage backend, it's not as critical as #1.

So, if we can get #1 done quickly in a way that doesn't kill the performance of hyperconverged infrastructure, then I am OK with keeping this private.  I think that #2 is a real problem, however, that could use some discussion at the PTG.  My question is whether we think the backend-data-substitution exploit can be discussed without revealing #1.  Of course, if we can get #1 completed and new glance releases out before 17 October, then that's not an issue.