Jobs should not be restarted indefinitely

Bug #1260910 reported by Jeremy Stanley
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Zuul
New
Undecided
Unassigned

Bug Description

Zuul is willing to restart a job if it ends without Jenkins providing a status. In some cases Jenkins jobs can fail consistently in such a way that Zuul willingly loops them over and over forever. An example of a job where this behavior was seen--a test hangs and when Jenkins tries to kill it for the configured timeout, an exception is thrown communicating with the slave agent...

    test_iter_nodes_with_custom_node_iter OK  0.02
    test_leak_1 Build timed out (after 60 minutes). Marking the build as failed.
++ rm -rf /tmp/tmp.ojOMmvvlVW
ERROR 3438.30
    test_mismatched_etags FATAL: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
hudson.remoting.RequestAbortedException: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
 at hudson.remoting.RequestAbortedException.wrapForRethrow(RequestAbortedException.java:41)
 at hudson.remoting.RequestAbortedException.wrapForRethrow(RequestAbortedException.java:34)
 at hudson.remoting.Request.call(Request.java:174)
 at hudson.remoting.Channel.call(Channel.java:713)
 at hudson.Launcher$RemoteLauncher.kill(Launcher.java:887)
 at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:609)
 at hudson.model.Run.execute(Run.java:1593)
 at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
 at hudson.model.ResourceController.execute(ResourceController.java:88)
 at hudson.model.Executor.run(Executor.java:247)
Caused by: hudson.remoting.RequestAbortedException: java.io.IOException: Unexpected termination of the channel
 at hudson.remoting.Request.abort(Request.java:299)
 at hudson.remoting.Channel.terminate(Channel.java:773)
 at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:69)
Caused by: java.io.IOException: Unexpected termination of the channel
 at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:50)
Caused by: java.io.EOFException
 at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2596)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1316)
 at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
 at hudson.remoting.Command.readFrom(Command.java:92)
 at hudson.remoting.ClassicCommandTransport.read(ClassicCommandTransport.java:72)
 at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:48)

At first I thought the test loop was due to this patchset getting switched to a draft after testing began, but were that the case switching it back to non-draft should have allowed it to complete. Investigating logs on zuul.openstack.org it looks like Jenkins does not report a completion status for this job.

Jeremy Stanley (fungi)
summary: - Switching patchset to draft causes indefinite job loop
+ Jobs should not be resubmitted indefinitely
summary: - Jobs should not be resubmitted indefinitely
+ Jobs should not be restarted indefinitely
Jeremy Stanley (fungi)
description: updated
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.