Gate jobs failed to start on Workflow+1

Bug #1358788 reported by Mark T. Voelker
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Core Infrastructure
Invalid
Low
Jeremy Stanley

Bug Description

https://review.openstack.org/#/c/100632/

This patch got a second +2 and a Workflow+1 on Aug 18 4:44 AM. However, the gate jobs never started and over 24 hours later it doesn't appear in the zuul pipeline.

Jeremy Stanley (fungi)
Changed in openstack-ci:
status: New → In Progress
importance: Undecided → Low
assignee: nobody → Jeremy Stanley (fungi)
milestone: none → juno
Revision history for this message
Jeremy Stanley (fungi) wrote :

At 22:10 UTC on August 12 the VMware NSX CI account left a mass recheck comment on hundreds of open reviews. The comment matched the pattern accepted by the upstream OpenStack CI system and so it removed its existing "Jenkins" account verify vote and reenqueued that change into the check pipeline for testing. Because of the load imposed on our system by the bulk of recheck comments from that account, we disabled it and restarted Zuul, intentionally discarding contents of its check pipeline. This left many changes, including the one in question in this bug, with no existing "Jenkins" account verify vote.

Later, when Ihar Hrachyshka approved the current patchset of this change, it was not considered as suitable for enqueuing into the gate because it lacked an appropriate check verify. It was not until testing was re-triggered through a reverify comment that it got retested, and is now in the check pipeline as intended (hopefully on its way into the gate pipeline once it succeeds there).

The short answer is that this is not a bug, but merely fallout from an unfortunate incident, and to keep in mind that if you're going to approve a change which lacks any verify vote at all from the "Jenkins" and is not already in the check pipeline, you should recheck it to kick off testing. The longer-term answer is that we're reevaluating our current "clean check" implementation and possibly replacing it with other similarly effective but less resource-intensive solutions, so this may cease to be an issue soon.

Changed in openstack-ci:
status: In Progress → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.