In https://review.opendev.org/c/openstack/manila/+/789702, we switched from using oslo.service's FixedIntervalLoopingCalls to run the periodic "report_state" route [1] (a health-check on the service's state).
During this switch we started observing that occasionally this sort of a warning was published in the logs:
Aug 31 21:37:30.867153 ubuntu-focal-ovh-gra1-0026219541 manila-scheduler[119378]: WARNING oslo.service.loopingcall [None req-f4e96796-0735-453a-bc0a-628b6201f280 None None] Function 'manila.service.Service.report_state' run outlasted interval by 5.31 sec
Aug 31 21:38:04.385570 ubuntu-focal-ovh-gra1-0026219541 manila-scheduler[119378]: WARNING oslo.service.loopingcall [None req-f4e96796-0735-453a-bc0a-628b6201f280 None None] Function 'manila.service.Service.report_state' run outlasted interval by 3.52 sec
Aug 31 21:39:31.867318 ubuntu-focal-ovh-gra1-0026219541 manila-scheduler[119378]: WARNING oslo.service.loopingcall [None req-f4e96796-0735-453a-bc0a-628b6201f280 None None] Function 'manila.service.Service.report_state' run outlasted interval by 7.45 sec
Aug 31 21:40:58.464518 ubuntu-focal-ovh-gra1-0026219541 manila-scheduler[119378]: WARNING oslo.service.loopingcall [None req-f4e96796-0735-453a-bc0a-628b6201f280 None None] Function 'manila.service.Service.report_state' run outlasted interval by 5.42 sec
It's uncertain when this happens, but it appears that the "report_state" routine is taking longer than the default periodic timer for the routine: 10s.
[1] https://github.com/openstack/manila/blob/0339802a5d0801cda19c464a87853b60982ae011/manila/service.py#L278-L316
Additional comments: https:/