Mechanism for blocking nodowntime rollouts on a given bug

If we have to cowboy a change to a specific server that's part of the "nodowntime" deployment alias for some reason, it'd be nice to have a tag that could be associated with the bug that fixes the need for that cowboy such that we could know that we can't rollout to "nodowntime" until it's fixed. If this tied into https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html this would mean that it was obvious to everyone that we couldn't have another deployment to nodowntime until that bug was fixed.