[alsa-devel] Immediate underrun with PulseAudio ALSA plugin when PA and ALSA buffer sizes differ
I'm investigating an issue in Firefox's audio code when the PulseAudio ALSA plugin is in use. I posted about this on pulseaudio-discuss last week (http://lists.freedesktop.org/archives/pulseaudio-discuss/2012-July/014091.ht...), but I hoped I might have more success here.
Firefox requests a particular latency (100ms, 4410 frames at 44.1kHz) via snd_pcm_set_params. Inside the plugin (pcm_pulse.c:pulse_hw_params), that value is used to set up buffer_attr. When the PA stream is connected in pcm_pulse.c:pulse_prepare, PA may configure the stream with larger buffer_attr values (e.g. because the minimum sink latency has increased over time due to underruns on the server, or because the sink hardware doesn't support lower latency), but this isn't reflected in pcm->buffer_attr or higher layers in ALSA (i.e. pcm->buffer_size is not updated).
The problem I'm faced with is that there doesn't appear to be a way to detect and handle this issue at the ALSA API level, and requesting a too low latency results in broken audio playback rather than a PCM setup failure or a larger buffer than requested being used.
In the case of the PA server's minimum latency increasing over time, this also means that a stream that was configured and running correctly may break while running if PA increases the minimum latency above what the PCM was originally configured with.
I've attached a simple testcase that uses snd_pcm_wait, snd_pcm_avail_update, and snd_pcm_writei. Run it with a latency argument specified in milliseconds on the command line. For my local machine, 55ms works and 54ms fails immediately like so:
snd_pcm_wait wakes snd_pcm_avail_update returns 4410 snd_pcm_writei writes 4410 snd_pcm_wait wakes immediately snd_pcm_avail_update returns -EPIPE
(Note that when I reported this on pulseaudio-discuss, my server's minimum latency was 45ms, and now pacmd list-sinks | grep configured\ latency reports a minimum latency of 56ms)
I'd expect to see one of the following behaviours instead: 1. PCM setup fails due to requesting a too small buffer. 2. Buffer is silently raised during setup and snd_pcm_avail_update requests the correct number of frames.
Presumably this could be achieved by having the PA plugin report valid values from pcm_pulse.c:pulse_hw_constraint, but I'm not sure how to query the necessary values from the server. This also wouldn't address the problem where the buffer_attr changes over time, and I'm not sure what to do about that case.
Thanks, -mjg
[x-posting back to pulseaudio-discuss since it's relevant to both lists]
On Tue, 2012-07-17 at 09:38 +1200, Matthew Gregan wrote:
I'm investigating an issue in Firefox's audio code when the PulseAudio ALSA plugin is in use. I posted about this on pulseaudio-discuss last week (http://lists.freedesktop.org/archives/pulseaudio-discuss/2012-July/014091.ht...), but I hoped I might have more success here.
Firefox requests a particular latency (100ms, 4410 frames at 44.1kHz) via snd_pcm_set_params. Inside the plugin (pcm_pulse.c:pulse_hw_params), that value is used to set up buffer_attr. When the PA stream is connected in pcm_pulse.c:pulse_prepare, PA may configure the stream with larger buffer_attr values (e.g. because the minimum sink latency has increased over time due to underruns on the server, or because the sink hardware doesn't support lower latency), but this isn't reflected in pcm->buffer_attr or higher layers in ALSA (i.e. pcm->buffer_size is not updated).
The problem I'm faced with is that there doesn't appear to be a way to detect and handle this issue at the ALSA API level, and requesting a too low latency results in broken audio playback rather than a PCM setup failure or a larger buffer than requested being used.
In the case of the PA server's minimum latency increasing over time, this also means that a stream that was configured and running correctly may break while running if PA increases the minimum latency above what the PCM was originally configured with.
I've attached a simple testcase that uses snd_pcm_wait, snd_pcm_avail_update, and snd_pcm_writei. Run it with a latency argument specified in milliseconds on the command line. For my local machine, 55ms works and 54ms fails immediately like so:
snd_pcm_wait wakes snd_pcm_avail_update returns 4410 snd_pcm_writei writes 4410 snd_pcm_wait wakes immediately snd_pcm_avail_update returns -EPIPE
(Note that when I reported this on pulseaudio-discuss, my server's minimum latency was 45ms, and now pacmd list-sinks | grep configured\ latency reports a minimum latency of 56ms)
I'd expect to see one of the following behaviours instead:
- PCM setup fails due to requesting a too small buffer.
- Buffer is silently raised during setup and snd_pcm_avail_update requests the correct number of frames.
Presumably this could be achieved by having the PA plugin report valid values from pcm_pulse.c:pulse_hw_constraint, but I'm not sure how to query the necessary values from the server. This also wouldn't address the problem where the buffer_attr changes over time, and I'm not sure what to do about that case.
The necessary values are available via pa_stream_get_buffer_attr(). Potentially we could use this in the pulse_pointer() function to update the corresponding snd_pcm_t's period/buffer sizes, but I don't know if this is kosher with regards to what alsa-lib expects plugins to be doing.
If this is not sufficient for the initial update, from what I can see, snd_pcm_set_params() first sets period/buffer sizes, queries them for later calculations, and then commits them with snd_pcm_hw_params(). If we could move the querying to after the params are committed (and in this case, the stream is connected and buffer attributes are negotiated), that would solve your problem. Again, I'm not sure what side-effects this might have, but I've attached a draft untested patch for it.
Cheers, Arun
On 07/17/2012 07:29 AM, Arun Raghavan wrote:
[x-posting back to pulseaudio-discuss since it's relevant to both lists]
On Tue, 2012-07-17 at 09:38 +1200, Matthew Gregan wrote:
I'm investigating an issue in Firefox's audio code when the PulseAudio ALSA plugin is in use. I posted about this on pulseaudio-discuss last week (http://lists.freedesktop.org/archives/pulseaudio-discuss/2012-July/014091.ht...), but I hoped I might have more success here.
Firefox requests a particular latency (100ms, 4410 frames at 44.1kHz) via snd_pcm_set_params. Inside the plugin (pcm_pulse.c:pulse_hw_params), that value is used to set up buffer_attr. When the PA stream is connected in pcm_pulse.c:pulse_prepare, PA may configure the stream with larger buffer_attr values (e.g. because the minimum sink latency has increased over time due to underruns on the server, or because the sink hardware doesn't support lower latency), but this isn't reflected in pcm->buffer_attr or higher layers in ALSA (i.e. pcm->buffer_size is not updated).
100 ms of latency is a lot, even for PulseAudio - is this some special hardware?
The problem I'm faced with is that there doesn't appear to be a way to detect and handle this issue at the ALSA API level, and requesting a too low latency results in broken audio playback rather than a PCM setup failure or a larger buffer than requested being used.
In the case of the PA server's minimum latency increasing over time, this also means that a stream that was configured and running correctly may break while running if PA increases the minimum latency above what the PCM was originally configured with.
I've attached a simple testcase that uses snd_pcm_wait, snd_pcm_avail_update, and snd_pcm_writei. Run it with a latency argument specified in milliseconds on the command line. For my local machine, 55ms works and 54ms fails immediately like so:
snd_pcm_wait wakes snd_pcm_avail_update returns 4410 snd_pcm_writei writes 4410 snd_pcm_wait wakes immediately snd_pcm_avail_update returns -EPIPE
Could you clarify what versions of PulseAudio and alsa-plugins you're using? The latest improvement to this handling was done less than a year ago and might require the latest versions of these components.
(Note that when I reported this on pulseaudio-discuss, my server's minimum latency was 45ms, and now pacmd list-sinks | grep configured\ latency reports a minimum latency of 56ms)
I'd expect to see one of the following behaviours instead:
- PCM setup fails due to requesting a too small buffer.
- Buffer is silently raised during setup and snd_pcm_avail_update requests the correct number of frames.
I think the better solution would be nr 2 in this case. Nr 1 won't solve the case where the sink's latency is increased dynamically - because the stream is moved, for example.
Presumably this could be achieved by having the PA plugin report valid values from pcm_pulse.c:pulse_hw_constraint, but I'm not sure how to query the necessary values from the server. This also wouldn't address the problem where the buffer_attr changes over time, and I'm not sure what to do about that case.
The necessary values are available via pa_stream_get_buffer_attr(). Potentially we could use this in the pulse_pointer() function to update the corresponding snd_pcm_t's period/buffer sizes, but I don't know if this is kosher with regards to what alsa-lib expects plugins to be doing.
If this is not sufficient for the initial update, from what I can see, snd_pcm_set_params() first sets period/buffer sizes, queries them for later calculations, and then commits them with snd_pcm_hw_params(). If we could move the querying to after the params are committed (and in this case, the stream is connected and buffer attributes are negotiated), that would solve your problem. Again, I'm not sure what side-effects this might have, but I've attached a draft untested patch for it.
I don't know either - and it also does not seem to solve the case where the sink's latency is suddenly increased (e g, when the sink input is moved to another sink).
2012-7-17 上午5:39 於 "Matthew Gregan" kinetik@flim.org 寫道：
I'm investigating an issue in Firefox's audio code when the PulseAudio
ALSA
plugin is in use. I posted about this on pulseaudio-discuss last week (
http://lists.freedesktop.org/archives/pulseaudio-discuss/2012-July/014091.ht... ),
but I hoped I might have more success here.
Firefox requests a particular latency (100ms, 4410 frames at 44.1kHz) via snd_pcm_set_params.
this function does not work for all sound card drivers.
many pci sound cards have hardware constraints which you cannot select 10ms or 100ms period time / buffer time.
Inside the plugin (pcm_pulse.c:pulse_hw_params), that value is used to set up buffer_attr. When the PA stream is connected in pcm_pulse.c:pulse_prepare, PA may configure the stream with larger buffer_attr values (e.g. because the minimum sink latency has increased
over
time due to underruns on the server, or because the sink hardware doesn't support lower latency), but this isn't reflected in pcm->buffer_attr or higher layers in ALSA (i.e. pcm->buffer_size is not updated).
The problem I'm faced with is that there doesn't appear to be a way to detect and handle this issue at the ALSA API level, and requesting a too
low
latency results in broken audio playback rather than a PCM setup failure
or
a larger buffer than requested being used.
do you mean the pulse plugin advertise a low latency but the server in fact force the application to use a large buffer with high latency?
In the case of the PA server's minimum latency increasing over time, this also means that a stream that was configured and running correctly may
break
while running if PA increases the minimum latency above what the PCM was originally configured with.
I've attached a simple testcase that uses snd_pcm_wait, snd_pcm_avail_update, and snd_pcm_writei. Run it with a latency argument specified in milliseconds on the command line. For my local machine, 55ms works and 54ms fails immediately like so:
snd_pcm_wait wakes snd_pcm_avail_update returns 4410 snd_pcm_writei writes 4410 snd_pcm_wait wakes immediately snd_pcm_avail_update returns -EPIPE
(Note that when I reported this on pulseaudio-discuss, my server's minimum latency was 45ms, and now pacmd list-sinks | grep configured\ latency reports a minimum latency of 56ms)
I'd expect to see one of the following behaviours instead:
- PCM setup fails due to requesting a too small buffer.
- Buffer is silently raised during setup and snd_pcm_avail_update
requests
the correct number of frames.
there is no negotiation between pa client and pa server about the capability of different sound cards and you cannot change the peiod time/buffer time when there are another pa client(s) already playing/capturing
Presumably this could be achieved by having the PA plugin report valid values from pcm_pulse.c:pulse_hw_constraint, but I'm not sure how to query the necessary values from the server. This also wouldn't address the problem where the buffer_attr changes over time, and I'm not sure what to
do
about that case.
Thanks, -mjg
At 2012-07-17T10:59:23+0530, Arun Raghavan wrote:
The necessary values are available via pa_stream_get_buffer_attr(). Potentially we could use this in the pulse_pointer() function to update the corresponding snd_pcm_t's period/buffer sizes, but I don't know if this is kosher with regards to what alsa-lib expects plugins to be doing.
Right, I assume alsa-lib expects the buffer size to be static after configuration is complete, and from a quick search through the source of alsa-lib and alsa-plugins I couldn't find any code that updated the buffer size after configuration.
If this is not sufficient for the initial update, from what I can see, snd_pcm_set_params() first sets period/buffer sizes, queries them for later calculations, and then commits them with snd_pcm_hw_params(). If we could move the querying to after the params are committed (and in this case, the stream is connected and buffer attributes are negotiated), that would solve your problem. Again, I'm not sure what side-effects this might have, but I've attached a draft untested patch for it.
Unless I'm missing something, the queried buffer_size in that function is only used to set the start_threshold later on, and doesn't update pcm->buffer_size.
At 2012-07-17T08:22:20+0200, David Henningsson wrote:
100 ms of latency is a lot, even for PulseAudio - is this some special hardware?
No, it's just a random value for media playback. In an older version of the audio backend we're using in Firefox (which was push rather than pull based), we used 500ms and hadn't run into this problem in a way that was obvious to users (i.e. causing broken playback). I chose a lower latency for media playback in the new backend in an attempt to flush out bugs before we introduce features that demand low latency (such as WebRTC).
Could you clarify what versions of PulseAudio and alsa-plugins you're using? The latest improvement to this handling was done less than a year ago and might require the latest versions of these components.
I'm using Fedora 17, which has alsa-plugins-pulseaudio-1.0.25-3.fc17 and pulseaudio-1.1-9.fc17. This was originally discovered by users running ALSA 1.0.25 on various distros (Ubuntu 12.04 LTS and Arch). Two of them happened to have a PA server where the latency had crept up over time, and a third was running the server with tsched=0 on an Audigy SE (CA0106) with a minimum latency of 200ms.
I've also tested against the git HEAD of alsa-plugins.
I think the better solution would be nr 2 in this case. Nr 1 won't solve the case where the sink's latency is increased dynamically - because the stream is moved, for example.
I don't know either - and it also does not seem to solve the case where the sink's latency is suddenly increased (e g, when the sink input is moved to another sink).
At 2012-07-17T15:36:52+0800, Raymond Yau wrote:
this function does not work for all sound card drivers.
many pci sound cards have hardware constraints which you cannot select 10ms or 100ms period time / buffer time.
That's fine, as long as it fails by either silently using the appropriate buffer/period sizes and reporting them via the ALSA API, or causes snd_pcm_set_params to return an error so another configuration can be attempted. The problem in this situation is that neither of those things happen under the conditions I described.
do you mean the pulse plugin advertise a low latency but the server in fact force the application to use a large buffer with high latency?
Yes. The code in pcm_pulse.c:pulse_hw_constraint reports fixed min/max sizes for buffers and periods without verifying that the PA server can honour those values. When it can't, PA will increase PA stream buffer size (tlength) as appropriate, but this isn't reflected in the plugin or ioplug/pcm layers of ALSA, so the calling application can't detect this and adjust accordingly.
Cheers, -mjg
On 07/19/2012 12:14 AM, Matthew Gregan wrote:
At 2012-07-17T08:22:20+0200, David Henningsson wrote:
100 ms of latency is a lot, even for PulseAudio - is this some special hardware?
No, it's just a random value for media playback.
What I meant was that I can successfully run your latency test with 10 ms here - when I go down to 5 I start to get xruns. In that context, getting problems around 50 - 100 ms of latency is quite a lot.
Anyway, I've dived a bit deeper here. I've run your latency test with 5 ms (i e 221 frames), and here's what I believe happens:
1) At the first write, 221 frames of data is written.
2) Now the stream is started. This is done by PulseAudio because the prebuf is reached. (And should we short-cut this, there is also something in snd_pcm_write_areas that would start it due to start_threshold being reached.) We then wait for PulseAudio, because we call pa_stream_writable_size right after write.
3) PulseAudio, on its side, starts the stream, quickly consumes the 221 frames of data from the client buffer, and sends out an underrun.
4) In fact, this underrun reaches alsa-plugins even while waiting for pulse_start to finish.
5) At the next call to snd_pcm_avail_update / pulse_pointer, the XRUN is returned to the client application.
6) snd_pcm_recover / pulse_prepare resets the stream, and then the same thing happens over and over again.
So how do we solve this? Well, I believe the best fix would be to fix PulseAudio to give back underruns later, i e, not until we know for sure that the 221 frames have been played back. Right now we send it out when the client buffer is emptied, which is too early. Deferring the underrun on the PulseAudio side would give the client side a fair chance to fill up PulseAudio's big buffer and thus avoid the underrun. I remember VLC having some trouble with this behaviour as well. This would, however, be some work in PulseAudio to get right. :-/
Meanwhile, you could make a workaround like this in ~/.asoundrc:
pcm.pulse_no_underrun { type pulse handle_underrun 0 }
and then open the device "pulse_no_underrun". With that workaround I can run with 1 ms without problem in your latency test. (We can't ignore underruns for everyone though, as that would break applications depending on these being delivered as well. Been there, done that.)
At 2012-07-19T10:56:09+0200, David Henningsson wrote:
What I meant was that I can successfully run your latency test with 10 ms here - when I go down to 5 I start to get xruns. In that context, getting problems around 50 - 100 ms of latency is quite a lot.
Oh, right. Of the three bug reports I have, two reports are from users with HDA, the third was using snd_ca01016 and tsched=0 with a fixed 200ms latency. On my system (Lenovo W510, 1.73GHz Core i7, HDA/CX20585), PA will creep up to 46-56ms over multiple days of use (across multiple laptop suspend/resume cycles), but will run with 16ms fine after restarting PA.
Meanwhile, you could make a workaround like this in ~/.asoundrc:
pcm.pulse_no_underrun { type pulse handle_underrun 0 }
and then open the device "pulse_no_underrun". With that workaround I can run with 1 ms without problem in your latency test. (We can't ignore underruns for everyone though, as that would break applications depending on these being delivered as well. Been there, done that.)
Thanks, this seems to work around the problem the way I'd hoped. It's not ideal, because you end up spinning on an instantly ready fd until the PCM's buffer has filled, but it's better than anything I had managed to cook up in the meantime. I've added code to set this at runtime[0] via the local config interface, inspired by WINE's winealsa.drv, which explicitly enables rather than disables handle_underrun.
[0] https://github.com/kinetiknz/cubeb/commit/1aa0058d0729eb85505df104cd1ac07243...
Thanks, -mjg
2012-7-19 上午6:14 於 "Matthew Gregan" kinetik@flim.org 寫道：
No, it's just a random value for media playback. In an older version of
the
audio backend we're using in Firefox (which was push rather than pull based), we used 500ms and hadn't run into this problem in a way that was obvious to users (i.e. causing broken playback). I chose a lower latency for media playback in the new backend in an attempt to flush out bugs
before
we introduce features that demand low latency (such as WebRTC).
Could you clarify what versions of PulseAudio and alsa-plugins you're using? The latest improvement to this handling was done less than a year ago and might require the latest versions of these components.
I'm using Fedora 17, which has alsa-plugins-pulseaudio-1.0.25-3.fc17 and pulseaudio-1.1-9.fc17. This was originally discovered by users running ALSA 1.0.25 on various distros (Ubuntu 12.04 LTS and Arch). Two of them happened to have a PA server where the latency had crept up over time,
and a
third was running the server with tsched=0 on an Audigy SE (CA0106) with a minimum latency of 200ms.
without using adjust latency mode, pa server use maximum buffer size by default and latency is fixed at about 340ms for ca0106 which does not support 44100hz.
so this is incorrect for pulse plugin to accept 200ms, in previous version of pulseaudio you can specify the fragment time and the number of fragments to achieve low latency.
participants (4)
-
Arun Raghavan -
David Henningsson -
Matthew Gregan -
Raymond Yau