[alsa-devel] Latency of mixer reconfiguration
Heya!
So, as some of you might know in PA we extend the abilities of the hardware volume control in software. Meaning that if a hw mixer control only allows a limited range or granularity of a volume scale or does not distuingish between channels then PA will configure the hardware to the next higher available setting of the real volume it wants to achieve, and then attenuates the PCM stream to compensate for the rest.
This actually works quite well but there is a certain probelm with latency: i.e. the mixer setting will be applied at a slightly different time than the PCM attenuation.
Now the question I have is, how should I best deal with this? I currently assume that mixer volume changes are basically instantaneous when I call the respective function of ALSA. But are they really? How big is the latency at max? Do we need an API to query it?
Lennart
On 17 February 2010 18:15, Lennart Poettering mznyfn@0pointer.de wrote:
Heya!
So, as some of you might know in PA we extend the abilities of the hardware volume control in software. Meaning that if a hw mixer control only allows a limited range or granularity of a volume scale or does not distuingish between channels then PA will configure the hardware to the next higher available setting of the real volume it wants to achieve, and then attenuates the PCM stream to compensate for the rest.
This actually works quite well but there is a certain probelm with latency: i.e. the mixer setting will be applied at a slightly different time than the PCM attenuation.
Now the question I have is, how should I best deal with this? I currently assume that mixer volume changes are basically instantaneous when I call the respective function of ALSA. But are they really? How big is the latency at max? Do we need an API to query it?
Lennart
The latency is non-deterministic. latency of the PCM stream is more predictable. I am not sure how you might get around this problem.
On Wed, Feb 17, 2010 at 07:15:30PM +0100, Lennart Poettering wrote:
Now the question I have is, how should I best deal with this? I currently assume that mixer volume changes are basically instantaneous when I call the respective function of ALSA. But are they really? How big is the latency at max? Do we need an API to query it?
For the embedded case the hardware update should happen synchronously with the ALSA API call to adjust the setting and begin affecting the played audio instantaneously. The time taken to do the hardware update will be dominated by I/O costs, which will in turn depend on the bus used to access the codec - it'd be good if the buses could provide some information to ASoC to allow it to do an estimate, but at the minute we've got nothing really to go on.
What's more interesting from this point of view is that the update will affect the audio going through the chip at the current moment so there will be latency from the DMA chain and possibly also from the DSP chain within the CODEC (which may vary depending on which volume you're updating).
I suspect that trying to offer additional resolution in this way is more trouble than it's worth if you're concerned about the artifacts that are introduced during updates. Providing per-channel differentiation if the hardware has only mono control has much fewer problems, though.
On Thu, 18.02.10 10:01, Mark Brown (broonie@opensource.wolfsonmicro.com) wrote:
Now the question I have is, how should I best deal with this? I currently assume that mixer volume changes are basically instantaneous when I call the respective function of ALSA. But are they really? How big is the latency at max? Do we need an API to query it?
For the embedded case the hardware update should happen synchronously with the ALSA API call to adjust the setting and begin affecting the played audio instantaneously. The time taken to do the hardware update will be dominated by I/O costs, which will in turn depend on the bus used to access the codec - it'd be good if the buses could provide some information to ASoC to allow it to do an estimate, but at the minute we've got nothing really to go on.
But what would you guess? In which range will this most likely be? < 1ms? 1ms? 10ms? 100ms? 1s? 1h? 10h? 10d? 10y?
tbh I feared less the actually IO latency but more that some PCM data fifos might need flushing before the volume is actually updated. And the latency of those fifos I feared might be more than a handful of samples?
What's more interesting from this point of view is that the update will affect the audio going through the chip at the current moment so there will be latency from the DMA chain and possibly also from the DSP chain within the CODEC (which may vary depending on which volume you're updating).
I suspect that trying to offer additional resolution in this way is more trouble than it's worth if you're concerned about the artifacts that are introduced during updates. Providing per-channel differentiation if the hardware has only mono control has much fewer problems, though.
The current logic is to not do any software adjustment if the hardware adjustment is "close enough" to the total adjustment we want to do, tested against a threshold. Which I think is quite a reasonable approach because it enables/disables this feature not globally, but looks at each case and enables this logic only if it really has a benefit.
Lennart
On Thu, 18 Feb 2010, Lennart Poettering wrote:
On Thu, 18.02.10 10:01, Mark Brown (broonie@opensource.wolfsonmicro.com) wrote:
Now the question I have is, how should I best deal with this? I currently assume that mixer volume changes are basically instantaneous when I call the respective function of ALSA. But are they really? How big is the latency at max? Do we need an API to query it?
For the embedded case the hardware update should happen synchronously with the ALSA API call to adjust the setting and begin affecting the played audio instantaneously. The time taken to do the hardware update will be dominated by I/O costs, which will in turn depend on the bus used to access the codec - it'd be good if the buses could provide some information to ASoC to allow it to do an estimate, but at the minute we've got nothing really to go on.
But what would you guess? In which range will this most likely be? < 1ms? 1ms? 10ms? 100ms? 1s? 1h? 10h? 10d? 10y?
Some drivers uses a queue (like HDA) for serial bus to update the audio mixer registers. But anyway, the real write should take several ms for maximum (maybe more on USB bus).
tbh I feared less the actually IO latency but more that some PCM data fifos might need flushing before the volume is actually updated. And the latency of those fifos I feared might be more than a handful of samples?
It's quite complex question and probably only hardware vendors knows these internals for all devices. I guess also, that some devices also uses volume ramping to avoid clicks.
Jaroslav
----- Jaroslav Kysela perex@perex.cz Linux Kernel Sound Maintainer ALSA Project, Red Hat, Inc.
On Fri, Feb 19, 2010 at 10:01:26AM +0100, Jaroslav Kysela wrote:
It's quite complex question and probably only hardware vendors knows these internals for all devices. I guess also, that some devices also uses volume ramping to avoid clicks.
The most common technique there is actually to do volume updates on zero cross - ie, only update the gain when the signal level is zero.
On Thu, Feb 18, 2010 at 07:04:22PM +0100, Lennart Poettering wrote:
On Thu, 18.02.10 10:01, Mark Brown (broonie@opensource.wolfsonmicro.com) wrote:
will be dominated by I/O costs, which will in turn depend on the bus used to access the codec - it'd be good if the buses could provide some information to ASoC to allow it to do an estimate, but at the minute we've got nothing really to go on.
But what would you guess? In which range will this most likely be? < 1ms? 1ms? 10ms? 100ms? 1s? 1h? 10h? 10d? 10y?
1ms or less normally - worst case will be a couple of I2C writes, though potentially over a congested bus.
tbh I feared less the actually IO latency but more that some PCM data fifos might need flushing before the volume is actually updated. And the latency of those fifos I feared might be more than a handful of samples?
Yes, mostly the buffers in the CPU. These vary from very small to very large - some systems allow relatively large audio buffers (hundreds of kilobytes for example) in order to allow the CPU and RAM to be powered down for extended periods of time during playback. It's the same problem as working out the latency for video sync.
I suspect that trying to offer additional resolution in this way is more trouble than it's worth if you're concerned about the artifacts that are introduced during updates. Providing per-channel differentiation if the hardware has only mono control has much fewer problems, though.
The current logic is to not do any software adjustment if the hardware adjustment is "close enough" to the total adjustment we want to do, tested against a threshold. Which I think is quite a reasonable approach because it enables/disables this feature not globally, but looks at each case and enables this logic only if it really has a benefit.
That sounds reasonable, though it's kind of surprising to me that there is hardware out there which benefits from it - I'd have expected either adequate resolution or nothing at all there.
On Fri, 19.02.10 09:54, Mark Brown (broonie@opensource.wolfsonmicro.com) wrote:
I suspect that trying to offer additional resolution in this way is more trouble than it's worth if you're concerned about the artifacts that are introduced during updates. Providing per-channel differentiation if the hardware has only mono control has much fewer problems, though.
The current logic is to not do any software adjustment if the hardware adjustment is "close enough" to the total adjustment we want to do, tested against a threshold. Which I think is quite a reasonable approach because it enables/disables this feature not globally, but looks at each case and enables this logic only if it really has a benefit.
That sounds reasonable, though it's kind of surprising to me that there is hardware out there which benefits from it - I'd have expected either adequate resolution or nothing at all there.
On my own hardware this mechanism proved useful in two cases: one card had a single volume slider for both channels and with PA you can now set their volume independantly. And one set of external USB speakers has a volume slider that starts at a very high level, so that the minimum volume setting is everything but silent. PA extends the scale downwards so that this limitation of the hw does not become apparent.
Lennart
2010/2/19 Lennart Poettering mznyfn@0pointer.de
On Thu, 18.02.10 10:01, Mark Brown (broonie@opensource.wolfsonmicro.com) wrote:
The current logic is to not do any software adjustment if the hardware adjustment is "close enough" to the total adjustment we want to do, tested against a threshold. Which I think is quite a reasonable approach because it enables/disables this feature not globally, but looks at each case and enables this logic only if it really has a benefit.
Lennart
PA 's watermark is already 20 ms if you want to do any software adjustment by PA server
On Sat, 20.02.10 11:59, Raymond Yau (superquad.vortex2@gmail.com) wrote:
2010/2/19 Lennart Poettering mznyfn@0pointer.de
On Thu, 18.02.10 10:01, Mark Brown (broonie@opensource.wolfsonmicro.com) wrote:
The current logic is to not do any software adjustment if the hardware adjustment is "close enough" to the total adjustment we want to do, tested against a threshold. Which I think is quite a reasonable approach because it enables/disables this feature not globally, but looks at each case and enables this logic only if it really has a benefit.
PA 's watermark is already 20 ms if you want to do any software adjustment by PA server
It's not necessarily 20ms. That's just the default (which we unfortunately had to pick very large, since ALSA right now does not inform us about transfer block granularity).
But anyway, whatever the watermark is: this is a latency we know about, which means that we can delay the hw mixer updating accordingly. But that works correctly only if the hw mixer updating is immediate. WHich it most likely is not.
So we now the software volume adjustment latency. We don't know the hardware volume adjustment latency, and assume it is 0. If we knew both we could sync up both changes perfectly, but this way our model has a weakness.
Lennart
2010/2/22 Lennart Poettering mznyfn@0pointer.de
On Sat, 20.02.10 11:59, Raymond Yau (superquad.vortex2@gmail.com) wrote:
2010/2/19 Lennart Poettering mznyfn@0pointer.de
On Thu, 18.02.10 10:01, Mark Brown (
broonie@opensource.wolfsonmicro.com)
wrote:
The current logic is to not do any software adjustment if the hardware adjustment is "close enough" to the total adjustment we want to do, tested against a threshold. Which I think is quite a reasonable approach because it enables/disables this feature not globally, but looks at each case and enables this logic only if it really has a benefit.
PA 's watermark is already 20 ms if you want to do any software
adjustment
by PA server
It's not necessarily 20ms. That's just the default (which we unfortunately had to pick very large, since ALSA right now does not inform us about transfer block granularity).
I think ALSA did inform the appliication the the granularit is period size, it is the responsibilty of the application which use timer scheduling to find out the optimum granularity which may varies with different sound card, different CPU speed
But anyway, whatever the watermark is: this is a latency we know about, which means that we can delay the hw mixer updating accordingly. But that works correctly only if the hw mixer updating is immediate. WHich it most likely is not.
do you mean PA server will perform snd_pcm_rewind when the user change the mixer volume ?
So we now the software volume adjustment latency. We don't know the hardware volume adjustment latency, and assume it is 0. If we knew both we could sync up both changes perfectly, but this way our model has a weakness.
I have doubt there is any benefit if you even know those hardware volume adjust ment latency which varies even when using the same chipset by different OEM vendor sound card
Lennart lsa-devel http://mailman.alsa-project.org/mailman/listinfo/alsa-devel
participants (5)
-
James Courtier-Dutton -
Jaroslav Kysela -
Lennart Poettering -
Mark Brown -
Raymond Yau