Nodes are in unuseble state. Possible typo "'NoneType' object has no attribute 'startswith'"

Collected logs are attached.

Sanity was green using: http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/flock/20210214T023311Z/outputs/iso/

and is red using:
http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/flock/20210216T003258Z/outputs/iso/

Based on the changelogs, the additional commits between the two loads are:
./stx-tools 07b8d07a36942e88b45bc5fa7c95cfa76af08463 2021-02-15 15:06:36 +0000 Gerrit Code Review <email address hidden> Merge "nspr/nss/nss-softokn/nss-util: CVE-2018-12404 and CVE-2019-11745"
./stx-tools 6ed078685c413b7199cd18bdde6cb8ad33fdb711 2021-01-28 21:36:35 -0500 Zhixiong Chi <email address hidden> nspr/nss/nss-softokn/nss-util: CVE-2018-12404 and CVE-2019-11745

See:
http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/flock/20210216T003258Z/outputs/CHANGELOG.txt

In fact, the merged commit 07b8d07a36942e88b45bc5fa7c95cfa76af08463 doesn't modify anything, since the same previous commit https://review.opendev.org/plugins/gitiles/starlingx/tools/+/ebc9f32d7de526b44234a56423dd6785243cc386 had been merged firstly.

Next step I will continue investigate the root cause.

According to the following error information:
 sysinv.conductor.manager [-] Failed check_nodes_stable. ('NoneType' object has no attribute 'startswith')
The call stack is as below(the file sysinv/sysinv/sysinv/sysinv/conductor/manager.py):
   function check_nodes_stable
     hosts = self.dbapi.ihost_get_list()
       ...
       host.vim_progress_status.startswith(...)

then use "except Exception as e" to catch the exception NoneType's startswith. That means the hosts is none. we couldn't got the nodes information by the dbapi.ihost_get_list function.
Even though I couldn't find any relationship between this issue and nss packages upgraded so far,
I find that, as the last comment shown, the following commits

./stx-tools 07b8d07a36942e88b45bc5fa7c95cfa76af08463 2021-02-15 15:06:36 +0000 Gerrit Code Review <email address hidden> Merge "nspr/nss/nss-softokn/nss-util: CVE-2018-12404 and CVE-2019-11745"
./stx-tools 6ed078685c413b7199cd18bdde6cb8ad33fdb711 2021-01-28 21:36:35 -0500 Zhixiong Chi <email address hidden> nspr/nss/nss-softokn/nss-util: CVE-2018-12404 and CVE-2019-11745

are the NULL operation, because we merge the same code in the commit https://review.opendev.org/plugins/gitiles/starlingx/tools/+/ebc9f32d7de526b44234a56423dd6785243cc386, which was provided by Joe Slater.
If this issue is introduced by this upgraded nss package, it will be possible Joe can provide some more useful information and diagnosis for it.

Hi Alexandru Dimofte,
Could you provide the detailed command and steps to reproduce this issue? Thanks.

Hi Zhixiong,
On my case the issue is visible only on Standard configuration and only for baremetal, during the provisioning. So I am unable to install stx in the end.
At some point I see:
system host-show compute-0|grep -w install_state|awk '{print$4}'
so.. it is checking the install state of compute-0 which should be "completed", but it is "None":

Fails if objects are unequal after converting them to strings.
Start / End / Elapsed: 20210218 11:25:27.775 / 20210218 11:25:27.776 / 00:00:00.001
11:25:27.776 FAIL None != completed

Just to clarify, I simply compared the changelogs between the green build and the red build and asked Zhixiong to investigate if it could be related to these commits. I'm not sure if it is or not.
The other question would be if there has been a recent change in the sanity setup/automation at Intel.

During the weekend I tried to reinstall some older green images of stx on standard baremetal and I was shocked to see that the error is still visible. I suspected our validation S1_STANDARD setup, this is why today I triggered the latest available image on our second setup S2_STANDARD and I can say that it passed.
So, now it is clear that we have some issues with our S1_STANDARD setup, the image is fine and this bug should be considered INVALID. Thank you and sorry for consuming your time with this.

Marking this LP as Invalid. As per above, it was confirmed that this is an issue with the sanity setup at Intel not a StarlingX software issue.