Adjust controller polling frequency with the audio buffer

wow my laptop is getting hot over 100% cpu usage while doing nothing

Here is the patch to get the polling frequency back to the old setting.

I'd like Mixxx to auto-detect whether a system can support a 1ms timer and automatically use it if so. Does anyone know of a way to do this? Qt doesn't seem to have any mechanism.

Or, ideally, we patch PortMIDI to use a callback-based system, truely solving the problem.

Moving away from polling would be the ideal solution, but I don't know if the PortMIDI API allow it. Otherwise, regarding linux, getting the HZ value of the currently running kernel doesn't seems to be straightforward. Dunno how it's like on other systems, but making the poll frequency configurable is probably the simplest.

Neither PortMidi nor RtMidi support any of the event based I/O APIs on any of the platforms or a blocking poll call. This might actually be a portability problem.

As quoted from the PortMidi Doxygen docs at http://portmedia.sourceforge.net/portmidi/doxygen/group__grp__io.html#ge88e6c682ee3550defdd0c74bcbdd508:

  "with a lock-free fifo, which is pretty much what we're stuck with to enable portability to the Mac, it's tricky for the producer and consumer to synchronously reset the buffer and resume normal operation."

Does a lock-free FIFO prevent blocking (for a set amount of time) until a message is received like HIDAPI does?

I suffer the same problem, and I am running on a 1000HZ custom kernel.

I went looking at portaudio's alsa implementation and rtmidi's alsa implementation. Both use polling without burning up users' systems. They both do this by sleeping for a very short (1ms) amount of time when no data is available after a poll() call.

I've implemented something similar to this at lp:~mixxxdevelopers/mixxx/fixes_midi_burnout. Results, in CPU usage:

                                 | Under load | Idle |
Without sleep (current behavior) | 150% | 110% |
With sleep (as revised) | 70% | 35% |

I'll be doing a merge request shortly.

Bleh so much for my lovely table. Basically, without my changes, I see 150%/110% CPU usage under load/idle. With the patch, it's 70%/35% (the under load figure doesn't include any analyzers).

If someone whose system is noticeably heated by the unrestricted polling can test this it'd be much appreciated, I'm in the process of moving country so I just had enough time to write a fix and watch htop for a few minutes.

Can you give the load/idle numbers for no MIDI device enabled? (that way we
see a baseline for no polling going on at all)

On Mon, May 21, 2012 at 1:54 PM, Bill Good <email address hidden>wrote:

> If someone whose system is noticeably heated by the unrestricted polling
> can test this it'd be much appreciated, I'm in the process of moving
> country so I just had enough time to write a fix and watch htop for a
> few minutes.
>
> --
> You received this bug notification because you are a member of Mixxx
> Development Team, which is subscribed to Mixxx.
> https://bugs.launchpad.net/bugs/990992
>
> Title:
> The new controller polling frequency is too high for desktop systems
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/mixxx/+bug/990992/+subscriptions
>

I can confirm, mixxx from lp:~mixxxdevelopers/mixxx/fixes_midi_burnout not uses more then 100% CPU, when controller is pluged in!

Also, I don't really see why you use QThread::msleep(1) instead of just
increasing kPollIntervalMillis to 2 in controllermanager.cpp. We are
polling on a timer in our own poll thread. msleeping a callback in that
thread should be roughly identical to setting the polling interval higher.
Based on that reasoning it seems to make sense that by doing msleep(1) you
cut your CPU usage roughly in half since the poll frequency will roughly
halve

On Mon, May 21, 2012 at 2:27 PM, RJ Ryan <email address hidden> wrote:

> Can you give the load/idle numbers for no MIDI device enabled? (that way
> we see a baseline for no polling going on at all)
>
>
> On Mon, May 21, 2012 at 1:54 PM, Bill Good <email address hidden>wrote:
>
>> If someone whose system is noticeably heated by the unrestricted polling
>> can test this it'd be much appreciated, I'm in the process of moving
>> country so I just had enough time to write a fix and watch htop for a
>> few minutes.
>>
>> --
>> You received this bug notification because you are a member of Mixxx
>> Development Team, which is subscribed to Mixxx.
>> https://bugs.launchpad.net/bugs/990992
>>
>> Title:
>> The new controller polling frequency is too high for desktop systems
>>
>> To manage notifications about this bug go to:
>> https://bugs.launchpad.net/mixxx/+bug/990992/+subscriptions
>>
>
>

come on guys just revert it to 250hz or lower. it should be enough

> Can you give the load/idle numbers for no MIDI device enabled? (that way we
> see a baseline for no polling going on at all)

12% idle/55% load.

I have no issue with simply increasing the polling interval.

> Also, I don't really see why you use QThread::msleep(1) instead of just
> increasing kPollIntervalMillis to 2 in controllermanager.cpp.

It only sleeps when the poll didn't see any data. So if mixxx starts taking in a stream of data (scratching), it'll hopefully not sleep. It does increase latency from no data to data though, which will be common scenario (no midi input to hitting a button, for instance). Honestly I thought there were limitations of midi over usb that prevented data from coming in at 1ms intervals so this could all be for naught.

On Mon, 21 May 2012 18:50:23 -0000
julius von kohout <email address hidden> wrote:

> come on guys just revert it to 250hz or lower. it should be enough
>

I fully agree. I'm using the current trunk with 250Hz and it work
great. Keep it simple!

Alban

i like bill's method - it does seem to be exactly how rtmidi handles the same thing, to emulate blocking nreads and callbacks.

There are other parts of Mixxx that we use a 1ms timer, for example in the script-engine scratch timers.

@juliusvonkohout we are searching for the right fix. "just reverting it" doesn't give us any useful information about how to avoid issues like this in the future. High-frequency timers are a very necessary part of getting rid of latency in Mixxx and making it more responsive.

@Bill -- we must be missing something... if RtMidi sleeps for 1ms when no input is available then we are sleeping strictly /more/ often than RtMidi (we sleep 1ms regardless of whether input is available) so where is the difference?

On Thu, 24 May 2012 15:11:44 -0000
RJ Ryan <email address hidden> wrote:

> @juliusvonkohout we are searching for the right fix. "just reverting it"
> doesn't give us any useful information about how to avoid issues like
> this in the future. High-frequency timers are a very necessary part of
> getting rid of latency in Mixxx and making it more responsive.

Reliable timer with 1ms accuracy are just not possible without a
realtime OS. In my experience anything under 10ms is not reliable
enough to be really useful on a desktop OS.

If the design really require such high frequency timer it is a big flaw
IMHO. One *must* use OS events (select() and the like) to efficiently
implement anything that would otherwise need polling with more than
100Hz. Desktop OS are just no realtime OS with guaranteed timing
constraints!

Alban

Hi Alban -- while it's true that without a realtime kernel you lose out on
a lot of guarantees about scheduling, scheduling is /unrelated/ to timing
accuracy in this case. Scheduling describes when your handler for a timer
is scheduled, not when the timer itself is triggered as complete by the OS.

Using basic timing primitives like select() is bad because you tie yourself
to the system timer and your timer can't complete between jiffies. It's not
true that those are the only option. Linux has a high-resolution timer
framework that doesn't require having a realtime kernel. In fact, Ubuntu
comes with the HRT system enabled by default. This high-res timer framework
allows your timer to trigger between jiffies. I think we should use this
instead of QTimer simply because it's more accurate.

To address your point about scheduling.. we strongly suggest people enable
realtime scheduling by tweaking their /etc/security/limits.conf file on
Linux (note we don't suggest people use a realtime kernel since there are
many stability issues with realtime kernels). This allows Mixxx to set its
mixing thread to have higher priorities over its other threads. It's not
true realtime but it's much better than the default.

High-resolution, accurate timing is a requirement of any DSP-heavy or
latency-sensitive application. Even just processing jog-wheels accurately
and with no drift requires accurate timing. There's no way around this. :)

On Thu, May 24, 2012 at 12:22 PM, Alban <email address hidden> wrote:

> On Thu, 24 May 2012 15:11:44 -0000
> RJ Ryan <email address hidden> wrote:
>
> > @juliusvonkohout we are searching for the right fix. "just reverting it"
> > doesn't give us any useful information about how to avoid issues like
> > this in the future. High-frequency timers are a very necessary part of
> > getting rid of latency in Mixxx and making it more responsive.
>
> Reliable timer with 1ms accuracy are just not possible without a
> realtime OS. In my experience anything under 10ms is not reliable
> enough to be really useful on a desktop OS.
>
> If the design really require such high frequency timer it is a big flaw
> IMHO. One *must* use OS events (select() and the like) to efficiently
> implement anything that would otherwise need polling with more than
> 100Hz. Desktop OS are just no realtime OS with guaranteed timing
> constraints!
>
> Alban
>
> --
> You received this bug notification because you are a member of Mixxx
> Development Team, which is subscribed to Mixxx.
> https://bugs.launchpad.net/bugs/990992
>
> Title:
> The new controller polling frequency is too high for desktop systems
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/mixxx/+bug/990992/+subscriptions
>

In general it doesn't make sense that scheduling a timer for 1ms when the
system tick is 250Hz would inherently cause any additional CPU burden than
if the system tick were 1000Hz. The only thing that should happen in that
case is that the timer will be triggered later than 1ms -- potentially 4ms
later.

Since we're asking for a 1ms repeated timer from QTimer, is it possible
that when the system tick expires, QTimer realizes that 4 timer events
should have happened and calls them all at once? This would cause us to
issue 4 polls every 4ms instead of 1 poll every 1ms.

On Thu, May 24, 2012 at 12:43 PM, RJ Ryan <email address hidden> wrote:

> Hi Alban -- while it's true that without a realtime kernel you lose out on
> a lot of guarantees about scheduling, scheduling is /unrelated/ to timing
> accuracy in this case. Scheduling describes when your handler for a timer
> is scheduled, not when the timer itself is triggered as complete by the OS.
>
> Using basic timing primitives like select() is bad because you tie
> yourself to the system timer and your timer can't complete between jiffies.
> It's not true that those are the only option. Linux has a high-resolution
> timer framework that doesn't require having a realtime kernel. In fact,
> Ubuntu comes with the HRT system enabled by default. This high-res timer
> framework allows your timer to trigger between jiffies. I think we should
> use this instead of QTimer simply because it's more accurate.
>
> To address your point about scheduling.. we strongly suggest people enable
> realtime scheduling by tweaking their /etc/security/limits.conf file on
> Linux (note we don't suggest people use a realtime kernel since there are
> many stability issues with realtime kernels). This allows Mixxx to set its
> mixing thread to have higher priorities over its other threads. It's not
> true realtime but it's much better than the default.
>
> High-resolution, accurate timing is a requirement of any DSP-heavy or
> latency-sensitive application. Even just processing jog-wheels accurately
> and with no drift requires accurate timing. There's no way around this. :)
>
>
> On Thu, May 24, 2012 at 12:22 PM, Alban <email address hidden> wrote:
>
>> On Thu, 24 May 2012 15:11:44 -0000
>> RJ Ryan <email address hidden> wrote:
>>
>> > @juliusvonkohout we are searching for the right fix. "just reverting it"
>> > doesn't give us any useful information about how to avoid issues like
>> > this in the future. High-frequency timers are a very necessary part of
>> > getting rid of latency in Mixxx and making it more responsive.
>>
>> Reliable timer with 1ms accuracy are just not possible without a
>> realtime OS. In my experience anything under 10ms is not reliable
>> enough to be really useful on a desktop OS.
>>
>> If the design really require such high frequency timer it is a big flaw
>> IMHO. One *must* use OS events (select() and the like) to efficiently
>> implement anything that would otherwise need polling with more than
>> 100Hz. Desktop OS are just no realtime OS with guaranteed timing
>> constraints!
>>
>> Alban
>>
>> --
>> You received this bug notification because you are a member of Mixxx
>> Developme...

In addition, see #8: someone with a 1000Hz kernel has the same problem. There's something else mysterious going on here. I just wish one of our devs had the problem. raskolnikov (or anyone affected by this): are you gather some profiling data for us? We would need you to build Mixxx from source with profiling=1, run it, try to use the MIDI controller for about a minute, then close Mixxx. Then run gprof ./mixxx and attach the output to this bug.

I dug into the QEventDispatcher code a little bit and it looks like my
hypothesis that multiple timer invocations come after a tick is wrong. On
UNIX, Qt checks whether multiple timer periods have elapsed and only sends
one timer event. Timer waits are accomplished by setting the timeout of the
event loop select() to the time to the next timer event, so this is
inherently limited to boundaries of jiffies.

Other interesting tidbits from reading the Qt source is that for timers
below 20ms on Windows, Qt uses the WinMM timeSetEvent method which provides
millisecond accuracy but from various Googling appears to be able to
reliably deliver 1ms callbacks with low jitter. Good to know.

On Thu, May 24, 2012 at 1:34 PM, Sean M. Pappalardo <
<email address hidden>> wrote:

> In addition, see #8: someone with a 1000Hz kernel has the same problem.
> There's something else mysterious going on here. I just wish one of our
> devs had the problem. raskolnikov (or anyone affected by this): are you
> gather some profiling data for us? We would need you to build Mixxx from
> source with profiling=1, run it, try to use the MIDI controller for
> about a minute, then close Mixxx. Then run gprof ./mixxx and attach the
> output to this bug.
>
> --
> You received this bug notification because you are a member of Mixxx
> Development Team, which is subscribed to Mixxx.
> https://bugs.launchpad.net/bugs/990992
>
> Title:
> The new controller polling frequency is too high for desktop systems
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/mixxx/+bug/990992/+subscriptions
>

On Thu, 24 May 2012 16:43:07 -0000
RJ Ryan <email address hidden> wrote:

> Hi Alban -- while it's true that without a realtime kernel you lose
> out on a lot of guarantees about scheduling, scheduling
> is /unrelated/ to timing accuracy in this case. Scheduling describes
> when your handler for a timer is scheduled, not when the timer itself
> is triggered as complete by the OS.

Assuming the handler thread is sleeping, waiting for some event or
sleeping, it still need to be rescheduled by the kernel, and there it
will most likely have to wait for the next jiffy.

> Using basic timing primitives like select() is bad because you tie
> yourself to the system timer and your timer can't complete between
> jiffies. It's not true that those are the only option. Linux has a
> high-resolution timer framework that doesn't require having a
> realtime kernel. In fact, Ubuntu comes with the HRT system enabled by
> default. This high-res timer framework allows your timer to trigger
> between jiffies. I think we should use this instead of QTimer simply
> because it's more accurate.

High *resolution* doesn't imply that you can use high *frequency*. You
can schedule with better accuracy, but it doesn't change much on how
often you can schedule a given action. The CPU just can run that much
code in a given time, at some point too much task switching is just
wasting time.

> To address your point about scheduling.. we strongly suggest people
> enable realtime scheduling by tweaking
> their /etc/security/limits.conf file on Linux (note we don't suggest
> people use a realtime kernel since there are many stability issues
> with realtime kernels). This allows Mixxx to set its mixing thread to
> have higher priorities over its other threads. It's not true realtime
> but it's much better than the default.

This only allow fine tuning the priority between the threads, a good
negative nice get one a bit further to avoiding being unscheduled, but
it will still have to happen sometimes.

> High-resolution, accurate timing is a requirement of any DSP-heavy or
> latency-sensitive application. Even just processing jog-wheels
> accurately and with no drift requires accurate timing. There's no way
> around this. :)

Sure high *resolution* help a lot, but we are talking about wasting
time checking for events that the kernel probably knew weren't there in
the first place.

Alban

Bill -- Hm, sleeping the controller manager thread isn't going to work since that also blocks other processing done in that thread for 1ms (e.g. midi script timers, control object changes, etc.). For 1.11.0 beta1 I'm just going to ifdef the polling frequency to 5ms for Linux.

I do like your branch's changes though.. except that I think if we processed events while polling we should immediately poll again (this matches RtMidi's behavior and our 1.10.0 and below behavior). So I'm merging it now. Thanks!

This bug is fixed for 1.11.0 isn't it?
I have opened a new bug Bug #1157579 with a different approach to sync the polling with the audio callback

This is not fixed in 1.11, as there's just an IFDEF that sets the polling frequency to 5ms on Linux only. Syncing the polling with the audio callback is an interesting idea, so we should explore that too.

Mixxx now uses GitHub for bug tracking. This bug has been migrated to:
https://github.com/mixxxdj/mixxx/issues/6383