"hub txn watcher sync error" and one controller is lost

Bug #2037740 reported by Jeffrey Chang
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Canonical Juju
Triaged
Low
Unassigned

Bug Description

Solution QA found this error from https://solutions.qa.canonical.com/testruns/66c91e32-35bf-4ba8-9a20-9c00ae73767e

We can see
Unit Workload Agent Machine Public address Ports Message
controller/0* active idle 0 10.246.64.201
controller/1 active idle 1 10.246.64.203
controller/2 unknown lost 2 10.246.64.202 agent lost, see 'juju show-status-log controller/2'

In the juju debug log
2023-09-29 14:32:20 ERROR juju.api.watcher watcher.go:95 error trying to stop watcher: hub txn watcher sync error: starting change stream: Closed explicitly
2023-09-29 14:32:20 ERROR juju.api.watcher watcher.go:95 error trying to stop watcher: hub txn watcher sync error: starting change stream: Closed explicitly
2023-09-29 14:32:20 ERROR juju.api.watcher watcher.go:95 error trying to stop watcher: websocket: close sent
2023-09-29 14:32:20 DEBUG juju.api monitor.go:35 RPC connection died
2023-09-29 14:32:20 DEBUG juju.rpc server.go:328 error closing codec: tls: failed to send closeNotify alert (but connection was closed anyway): write tcp 10.246.64.209:41288->10.246.64.203:17070: write: connection reset by peer

We found a very old similar bug, LP#1802067. But decided to open a new one.
More logs can be found from https://oil-jenkins.canonical.com/artifacts/66c91e32-35bf-4ba8-9a20-9c00ae73767e/index.html.

Revision history for this message
Joseph Phillips (manadart) wrote :

This appears to be a simple disconnection of the controller from the MongoDB primary.

Was it an intermittent failure, as in the controller re-established its presence?

I'll triage as low, as it appears to be an intermittent failure due to the networking environment, and we're working to replace MongoDB. If it is appearing frequently, we can update the status.

Changed in juju:
status: New → Triaged
importance: Undecided → Low
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.