ScaleIO volumes contain previous data

The volume_clear option is not appropriate for scaleio, it's only for drivers where the data path is managed by cinder volume (LVM, block device driver, etc.)

If scaleio is provisioning volumes that have pre-existing data on them, that is a serious bug in the scaleio backend or driver.

It should not expose data of previously deleted volumes, and whatever is needed to fix that behavior should not be optional.

If true, this is a security issue, marking as such.

Is this something that can be fixed/mitigated by Cinder?

Not sure, we need input from the ScaleIO driver owners.

Assigned to Eric Young. Eric Y., please take a look. Thanks.

This can occur with thick volumes when zero padding is disabled in the storage pool, it can be worked around within the scaleio volume driver and that enhancement isplanned for the Queens release.

If thin volumes are used, this does not occur.

Is the patch to correct this behavior one which can be backported safely to prior releases/stable branches? If not, and the only mitigation is to adjust configuration, then this information is probably better suited as a security note than an advisory.

Fix proposed to branch: master
Review: https://review.openstack.org/494702

Given that the proposed fix seems like it's taking a non-backportable route, I'm going to mark the security advisory task as won't fix citing report class B1 or maybe C2 per the OpenStack VMT's report taxonomy: https://security.openstack.org/vmt-process.html#incident-report-taxonomy (which class it is depends on whether you view the cinder patch as a vulnerability fix or merely a workaround for a vulnerability in the vendor's device).

I've also added a new OSSN task in case the security notes editors are interested in documenting the situation/solution once it reaches some conclusion.

To be clear, this is not a vulnerability in the cinder volume driver but is a side effect of having "zero padding" disabled in the storage pool within the ScaleIO system.

I am currently evaluating the possibilities of providing a workaround within the driver, including the viability of backporting.

For customers concerned about this issue, it is advised to either utilize thin volumes or enable zero padding within the ScaleIO Storage Pools.

This issue cannot be successfully worked around in the Cinder driver.

Users who are worried about this occurring should ensure that zero padding is enabled within the ScaleIO Storage Pool(s). Full information can be found within the Dell EMC ScaleIO documentation.

The setting can be queried by running:

scli --query_all

The command to enable zero padding is:

scli --modify_zero_padding_policy
              (((--protection_domain_id <ID> |
              --protection_domain_name <NAME>)
              --storage_pool_name <NAME>) | --storage_pool_id <ID>)
              (--enable_zero_padding | --disable_zero_padding)

Can't this be fixed in the driver by looking at the volume_clear flag and performing some of this automatically?

I don't believe the driver should be modifying the configuration of the ScaleIO system. These steps need to be performed by an administrator on one of the ScaleIO systems as there is no remote capability to enable this.

I do plan to check the status of zero padding and warn the users that it should be enabled. A review is already submitted that contains this warning, for the Queens release. Older releases are soon to follow.

Related fix proposed to branch: stable/pike
Review: https://review.openstack.org/504288

Change abandoned by Eric Young (<email address hidden>) on branch: master
Review: https://review.openstack.org/494702
Reason: This cannot be worked around in the driver. The solution is to make sure that zero padding is enabled within the ScaleIO storage pool

Change abandoned by Eric Young (<email address hidden>) on branch: stable/pike
Review: https://review.openstack.org/504288

Reviewed: https://review.openstack.org/502473
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=aa8b87a83cc5a8cfcaa3080c9a6080d8289716a4
Submitter: Jenkins
Branch: master

commit aa8b87a83cc5a8cfcaa3080c9a6080d8289716a4
Author: Eric Young <email address hidden>
Date: Mon Sep 11 09:46:11 2017 -0400

    ScaleIO Driver - adding cache and refactoring tests

    Changing static lists to a simple cache.
    Refactoring some of the unit tests to simplify maintenance.

    Related-Bug: #1699573

    Change-Id: Idff127801da9e286a6b634594e5577eeb9782571

So a fix/workaround for this problem has been merged right? OK if I draft a OSSN to explain it?

A fix for this has not been landed in Cinder.

We should add a fix to the Cinder driver that refuses volume creation if this configuration is detected, to prevent leaking data between tenants. A warning message is not adequate.

There is not going to be a fix made to the cinder driver for this. The workaround is documented starting at comment #11 (utilize thin volumes or ensure zero-padding is enabled int he storage pool).

Fix proposed to branch: master
Review: https://review.openstack.org/555421

Fix proposed to branch: master
Review: https://review.openstack.org/555546

Change abandoned by Eric Harney (<email address hidden>) on branch: master
Review: https://review.openstack.org/555421
Reason: https://review.openstack.org/#/c/555546/ is looking good, let's go that route. Thanks Eric

Reviewed: https://review.openstack.org/555546
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=7feb62197d371ab7253dc86a34af6ff8b484b4df
Submitter: Zuul
Branch: master

commit 7feb62197d371ab7253dc86a34af6ff8b484b4df
Author: Eric Young <email address hidden>
Date: Thu Mar 22 20:24:01 2018 -0400

    ScaleIO: Prevent usage of unsafe volumes

    It is possible for thick volumes, created from storage pools
    which have zero-padding disabled, to contain previous data. This
    change prevents these volumes from being created by default. A
    user can override this behavior by acknowleding the possibility
    with a configuration option.

    Change-Id: I62f8f48b1624fc9abb7427bd4ca51f7873d35b96
    Closes-bug: #1699573

This issue was fixed in the openstack/cinder 13.0.0.0b1 development milestone.

This has been assigned CVE-2017-15139.

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/596879

Reviewed: https://review.openstack.org/596879
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=f0cef07bef5ea8ed29179ee3774df5f4a634ba86
Submitter: Zuul
Branch: stable/queens

commit f0cef07bef5ea8ed29179ee3774df5f4a634ba86
Author: Eric Young <email address hidden>
Date: Thu Mar 22 20:24:01 2018 -0400

    ScaleIO: Prevent usage of unsafe volumes

    It is possible for volumes, created from storage pools
    which have zero-padding disabled, to contain previous data. This
    change prevents these volumes from being created by default. A
    user can override this behavior by acknowleding the possibility
    with a configuration option.

    This is a squash of the four commits that led to the final state in
    rocky to not allow the creation of any type of non-zero-padded volumes
    to be created. This adds a config option that defaults to the safe
    behavior. It is backporting a new config option, and a change in default
    behavior, but it should be acceptable in this case so that the security
    vulnerability can be addressed.

    Closes-Bug: #1784871

    Change-Id: I62f8f48b1624fc9abb7427bd4ca51f7873d35b96
    Closes-bug: #1699573
    (cherry picked from commit 7feb62197d371ab7253dc86a34af6ff8b484b4df)
    (cherry picked from commit 949cc46e162e00092aa85a7be921649c8dbf2bf8)
    (cherry picked from commit 8d0dea694a366cb3797748d389ca76b7864af16f)
    (cherry picked from commit 13a6689ccb7751c9f9b5c37ce0a3f75eb7665a95)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/601681

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/604105

Reviewed: https://review.openstack.org/601681
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=6309c097e653c5f8b40e0602950d0ef54a9efb37
Submitter: Zuul
Branch: stable/pike

commit 6309c097e653c5f8b40e0602950d0ef54a9efb37
Author: Eric Young <email address hidden>
Date: Thu Mar 22 20:24:01 2018 -0400

    ScaleIO: Prevent usage of unsafe volumes

    It is possible for volumes, created from storage pools
    which have zero-padding disabled, to contain previous data. This
    change prevents these volumes from being created by default. A
    user can override this behavior by acknowleding the possibility
    with a configuration option.

    This is a squash of the four commits that led to the final state in
    rocky to not allow the creation of any type of non-zero-padded volumes
    to be created. This adds a config option that defaults to the safe
    behavior. It is backporting a new config option, and a change in default
    behavior, but it should be acceptable in this case so that the security
    vulnerability can be addressed.

    Closes-Bug: #1784871

    Change-Id: I62f8f48b1624fc9abb7427bd4ca51f7873d35b96
    Closes-bug: #1699573
    (cherry picked from commit f0cef07bef5ea8ed29179ee3774df5f4a634ba86)

Reviewed: https://review.openstack.org/604105
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=2dc52153215bb6a37532a959c5c98239be21bb56
Submitter: Zuul
Branch: stable/ocata

commit 2dc52153215bb6a37532a959c5c98239be21bb56
Author: Eric Young <email address hidden>
Date: Thu Mar 22 20:24:01 2018 -0400

    ScaleIO: Prevent usage of unsafe volumes

    It is possible for volumes, created from storage pools
    which have zero-padding disabled, to contain previous data. This
    change prevents these volumes from being created by default. A
    user can override this behavior by acknowleding the possibility
    with a configuration option.

    This is a squash of the four commits that led to the final state in
    rocky to not allow the creation of any type of non-zero-padded volumes
    to be created. This adds a config option that defaults to the safe
    behavior. It is backporting a new config option, and a change in default
    behavior, but it should be acceptable in this case so that the security
    vulnerability can be addressed.

    Closes-Bug: #1784871

    Change-Id: I62f8f48b1624fc9abb7427bd4ca51f7873d35b96
    Closes-bug: #1699573
    (cherry picked from commit f0cef07bef5ea8ed29179ee3774df5f4a634ba86)
    (cherry picked from commit 6309c097e653c5f8b40e0602950d0ef54a9efb37)

Fix proposed to branch: driverfixes/newton
Review: https://review.openstack.org/606130

This issue was fixed in the openstack/cinder 10.0.8 release.

Reviewed: https://review.openstack.org/606130
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=234aab19f1677a337abd4cc37ede5dc5e455f258
Submitter: Zuul
Branch: driverfixes/newton

commit 234aab19f1677a337abd4cc37ede5dc5e455f258
Author: Eric Young <email address hidden>
Date: Thu Mar 22 20:24:01 2018 -0400

    ScaleIO: Prevent usage of unsafe volumes

    It is possible for volumes, created from storage pools
    which have zero-padding disabled, to contain previous data. This
    change prevents these volumes from being created by default. A
    user can override this behavior by acknowleding the possibility
    with a configuration option.

    This is a squash of the four commits that led to the final state in
    rocky to not allow the creation of any type of non-zero-padded volumes
    to be created. This adds a config option that defaults to the safe
    behavior. It is backporting a new config option, and a change in default
    behavior, but it should be acceptable in this case so that the security
    vulnerability can be addressed.

    Closes-Bug: #1784871

    Change-Id: I62f8f48b1624fc9abb7427bd4ca51f7873d35b96
    Closes-bug: #1699573
    (cherry picked from commit f0cef07bef5ea8ed29179ee3774df5f4a634ba86)
    (cherry picked from commit 6309c097e653c5f8b40e0602950d0ef54a9efb37)
    (cherry picked from commit 2dc52153215bb6a37532a959c5c98239be21bb56)

This issue was fixed in the openstack/cinder 12.0.4 release.

Related fix proposed to branch: stable/pike
Review: https://review.openstack.org/625041

Related fix proposed to branch: stable/ocata
Review: https://review.openstack.org/633570

Related fix proposed to branch: driverfixes/newton
Review: https://review.openstack.org/633598

Reviewed: https://review.openstack.org/625041
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=0e9b173381fce28fded543095360dca5198c4810
Submitter: Zuul
Branch: stable/pike

commit 0e9b173381fce28fded543095360dca5198c4810
Author: Eric Young <email address hidden>
Date: Mon Sep 11 09:46:11 2017 -0400

    ScaleIO Driver - adding cache and refactoring tests

    Changing static lists to a simple cache.
    Refactoring some of the unit tests to simplify maintenance.

    Related-Bug: #1699573

    Change-Id: Idff127801da9e286a6b634594e5577eeb9782571
    (cherry picked from commit aa8b87a83cc5a8cfcaa3080c9a6080d8289716a4)

This issue was fixed in the openstack/cinder 11.2.0 release.

Reviewed: https://review.openstack.org/633570
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=034feb6d749ace24fe709c9bf1aa8e9eb87a0cff
Submitter: Zuul
Branch: stable/ocata

commit 034feb6d749ace24fe709c9bf1aa8e9eb87a0cff
Author: Eric Young <email address hidden>
Date: Mon Sep 11 09:46:11 2017 -0400

    ScaleIO Driver - adding cache and refactoring tests

    Changing static lists to a simple cache.
    Refactoring some of the unit tests to simplify maintenance.

    Related-Bug: #1699573

    Change-Id: Idff127801da9e286a6b634594e5577eeb9782571
    (cherry picked from commit aa8b87a83cc5a8cfcaa3080c9a6080d8289716a4)
    (cherry picked from commit 0e9b173381fce28fded543095360dca5198c4810)

Reviewed: https://review.openstack.org/633598
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=442832947f4e4968562679ab53378aae8aca560a
Submitter: Zuul
Branch: driverfixes/newton

commit 442832947f4e4968562679ab53378aae8aca560a
Author: Eric Young <email address hidden>
Date: Mon Sep 11 09:46:11 2017 -0400

    ScaleIO Driver - adding cache and refactoring tests

    Changing static lists to a simple cache.
    Refactoring some of the unit tests to simplify maintenance.

    Related-Bug: #1699573

    Change-Id: Idff127801da9e286a6b634594e5577eeb9782571
    (cherry picked from commit aa8b87a83cc5a8cfcaa3080c9a6080d8289716a4)
    (cherry picked from commit 0e9b173381fce28fded543095360dca5198c4810)
    (cherry picked from commit 034feb6d749ace24fe709c9bf1aa8e9eb87a0cff)