Capomastro lacks the ability to build images when their code tree changes

Bug #1456160 reported by Caio Begotti
This bug affects 1 person
Affects Status Importance Assigned to Milestone

Bug Description

When using Capomastro for an OEM project, it would be good to be able to automatically start builds when the code changes. It would be desirable to simply build all incoming merge proposals on the gerrit source tree used for OEM phone projects. These could then be used to automatically +1 merge proposals.

Tags: stakeholder
Revision history for this message
Daniel Manrique (roadmr) wrote :

Some thought would need to be given to locking in this scenario. If build time is longer than "poll for new merges" time, then we risk overlapping builds which can have adverse effects, mainly on performance.

Daniel Manrique (roadmr)
tags: added: stakeholder
Daniel Manrique (roadmr)
Changed in capomastro:
status: New → Confirmed
importance: Undecided → High
Daniel Manrique (roadmr)
Changed in capomastro:
milestone: none → future
Revision history for this message
Daniel Manrique (roadmr) wrote :

As noted in

On the topic of change-based builds (and which interestingly also affects "only build if source has changed" periodic builds), is that capomastro has no concept or knowledge of revision control utilities, and thus has no idea that a build is for a specific revision in a tree. So this functionality would have to be implemented in the project-specific build scripts. Consecutive revnos like bzr make it easy to figure out if the tree has changed, with git it may be harder because there's no consecutive revision ID, but I'm sure we can figure something out. But again, that probably belongs in and its per-project scripts.

To update the above, Capomastro now has the "scheduled builds" functionality and that means what we have a periodic task scheduler (celery beat). So we could design a solution that periodically runs $SOMETHING that is included in each job type's branch (under and depending on whether that $SOMETHING returns true or false, launch a build for that dependency.

A brain dump of things needed for this:

Some sort of dependency locking is needed. If a dependency is already building, subsequent builds should either be discarded (silently? or loudly with exceptions and errors?) or queued so it runs after any previously-scheduled ones (can celery do this?).

We also need to define (and if needed, fix) the "build a single dependency" behavior. I've argued that rebuilding a dependency should still result in the project showing a build with all dependencies linked, though perhaps only the rebuilt one will be up-to-date, while the rest will be links to old artifacts. This is needed to avoid surprises if builds are triggered by some magic.

Also "runs $SOMETHING" needs to be defined better. Capomastro doesn't run anything itself, it delegates everything to Jenkins, which again, tries to run everything inside an LXC. I wonder if spawning a full LXC environment to run what essentially amounts to "bzr revno" is overkill, but it would be architecturally consistent.

Related to the above, we should figure out where/how to store revision data and how to decide if a rebuild is needed. For instance, if we store bzr revnos, they're easy to compare because they're sequential, unlike git commit identifiers. On the other hand, we could simply say "if it changed, then launch a build", which works for both bzr and git, and as long as e.g. someone doesn't overwrite a branch such that revnos or commit ids change, should be consistent. It still needs storing "last revno" somewhere, and bygmester jobs are designed to be mostly stateless, so we'd also need to decide where/how to store this.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.