1. The lease manager makes claims against what appears to be an empty FSM *before* it has completed restoring from its snapshot - you can see the "no leasing found, claiming.." entries.
2. The initial clock advancement, the worker having been instantiated with an empty FSM, is made with an old time of 0001-01-01 00:00:00 +0000 UTC, which is incorrect following the FSM restoration (mentioned above).
3. Raft leader election occurs, relevant workers bounce, there are lease claim time-outs following.
The logs show convergence and success following this jitter, but we don't really want large models spammed with time-out errors every time there is a Raft leader election.
This is pertinent log capture following a restart. /pastebin. canonical. com/p/DjnWdt5rs W/
https:/
I can see some issues here.
1. The lease manager makes claims against what appears to be an empty FSM *before* it has completed restoring from its snapshot - you can see the "no leasing found, claiming.." entries.
2. The initial clock advancement, the worker having been instantiated with an empty FSM, is made with an old time of 0001-01-01 00:00:00 +0000 UTC, which is incorrect following the FSM restoration (mentioned above).
3. Raft leader election occurs, relevant workers bounce, there are lease claim time-outs following.
The logs show convergence and success following this jitter, but we don't really want large models spammed with time-out errors every time there is a Raft leader election.