Comment 4 for bug 1841634

I think I need to talk a bit of theory first. What I see here is a queue of queues, or a chain of queues. In such a case, it is optimal to handle overflows at the head of the first queue. Anything else results in work being done by earlier queue(s), only to be discarded when a later queue overflows. This is wasted work is what I meant by "churn".

The dbCa queue is 5 deep, while the RPRO "queue" is only 1. Potentially queuing up 5 scans only to discard 4 of them seems unnecessarily wasteful. Is is "a lot of".

imo. The proper way to handle these queue of queue situations is to couple the queues so that flow control can be applied. A full queue, other than the first, can then apply back-pressure to the previous queue.

This idea played into my adding scanOnceCallback() and using it to provide feedback to dbCa. What I see here is that that this feedback comes too early.

I raise the possibility of using dbNotify because it seems like to only mechanism I know of in Base at present which might provide the additional information which scanOnceCallback() doesn't, when an inprogress async record has finished.

I say "at present" because I don't particularly like dbNotify. It's too narrow in what it can do, and yet has enough options to be confusing. I would rather have a more general mechanism which doesn't try to handle the "put" or "process" part of the sequence, which can be accomplished with other APIs, but rather focuses on providing notification when current (async) processing completes. There is no other way to do this part.