[alsa-devel] [RFC] How to perform audio offloading in none tunneled mode

[Pierre-Louis Bossart]

On 9/8/15 11:04 AM, Arnaud Pouliquen wrote:
> Hello,
>
> I'm looking at possibility to offload atomic actions like mixing,
> decoding, encoding on a co-processor, and collect the result on host side.
>
> Rational:
> - Allow none-tunneled mode:
>      . decoding and mixing on a DSP but A/V synchro on host (android,
> Gstreamer)
>      . re-encode after mixing (HDMI)
> - Allow to get post processing PCM stream to play it on USB or bluetooth
> devices.
> - Allow to add some processing in pipe on host side.
> - MIPS partitioning
> -...
>
> Some constraints:
> - avoid copy (buffer descriptors)
> - dynamic connection for mixing and splitting.
>
> For this, i can list two standard drivers: V4L2 and ALSA.
> If V4L2 should be OK for encode and decode, seems designed more for
> video than audio. For mixing and processing...
>
> ALSA could answer to this kind of use cases, using compress API and ASoC
> dynamic PCM mechanism...
>
> As example a mixer would be a sound card with several PCM playbacks and
> a PCM capture.
>
> For time being i have never seen this kind of implementation...
>
> What is your feeling on possibility to perform it with ALSA (with
> objective to be upstreamable)?
>
> Otherwise, any standard way to do it?
>

The compressed API was more designed for offloading and rendering, if 
you can to pass the mixed result back to the host you will have to setup 
a capture stream using the regular PCM API.
What you are describing is feasible but has issues related to:
- delay control
- DSP scheduling (no real means to process data faster than real-time as 
you would want in a data-driven co-processor)
There are also divergent views on the benefits of offloading 
intermediate operations to a resource-constrained co-processor, you 
might be better off doing everything on the host in terms of power 
consumption.