[RFC] DPCM for Tegra
Hi Mark,
Earlier I had sent Tegra ASoC series [0] for review and writing back to follow up on the same.
Background ========== There were following concerns on [0]:
- Usage of mixer control overrides in each driver for PCM parameters.
- Exposure of routing controls to the user.
The comments are mostly captured in [1] and [2].
There was a suggestion to use DPCM for the Tegra audio requirements.
Note: As of now, below does not cover specific solution for propogation of audio configurations or PCM parameters (like sample rate, sample size and channels).
DPCM Testing ============ Since then I was looking into internals of DPCM and was trying to get it working with Tegra. I was able to get following things working.
- Audio playback/capture over I2S/DMIC/DSPK. - Audio resampling use case. - Mixing of two audio streams with resampler in the path.
Objective was to understand and get audio working with DPCM. Please note that I used simple-card DPCM driver for a quick testing on top of the above Tegra Audio series. I had to tweak the simple-card driver a little, but currently keeping it out of the scope of current discussion.
At a high level Tegra Audio HW is depicted as below.
| Front End PCMs | SoC DSP | Back End DAIs |
************* ADMAIF<0> <------------> * * <----DAI<0>-----> I2S * * ADMAIF<1> <------------> * * <----DAI<1>-----> DMIC * XBAR * ADMAIF<2> <------------> * * <----DAI<2>-----> DSPK * * ADMAIF<N> <------------> * * <----DAI<3>-----> SFC (Resampler) * * * * <----DAI<4>-----> MIXER * * * * <----DAI<N>-----> ... *************
Note: -----
* XBAR is just a cross bar interconnecting one component to another. Specific switch needs to be programmed for audio data to flow from one component to another.
* SFC or Mixer are separate HW blocks and separate ASoC drivers are written for these. These drivers were not sent earlier as part of initial upstream series [0].
Follow up queries ================= Based on the above experience I do have few follow up queries and request for your inputs on this.
a) Can I use a DAPM Mux control to activate a BE path? This in turn can program required switch in XBAR.
This is needed for following reasons:
- For an open platform like Jetson, we want to give maximum flexibility for a user to customize their audio paths. Number of connected components and order of these can vary depending on a use case.
- Allow re-use of audio components across multiple use cases. For example, number of SFC instances are lesser than PCM playback or capture devices.
b) I have modelled SFC and MIXER as backends. Is this allowed?
This was done to include SFC or MIXER HW components as part of the sound card and use like below in one of the audio use cases.
ADMAIF1(FE) --> SFC(BE1) --> I2S(BE2) ... OR ADMAIF2(FE) --> SFC(BE1) --> I2S(BE2) ...
I used following workaround to connect multiple BE components. With this I can see PCM callbacks happen for all BE DAIs along the DAPM path. The obective was to connect multiple components together and (a) was used to connect one component to another. Each "-->" here connects two components and it is a switch in XBAR.
--- sound/soc/soc-pcm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/sound/soc/soc-pcm.c b/sound/soc/soc-pcm.c index e256d43..ee7af55 100644 --- a/sound/soc/soc-pcm.c +++ b/sound/soc/soc-pcm.c @@ -1494,7 +1494,7 @@ int dpcm_path_get(struct snd_soc_pcm_runtime *fe,
/* get number of valid DAI paths and their widgets */ paths = snd_soc_dapm_dai_get_connected_widgets(cpu_dai, stream, list, - dpcm_end_walk_at_be); + NULL);
dev_dbg(fe->dev, "ASoC: found %d audio %s paths\n", paths, stream ? "capture" : "playback"); --
c) Hostless mode did NOT work: - Following audio path was intended to be tested: I2S1 --> SFC --> I2S2
- [3] offers two options: * CODEC<->CODEC: If I were to use a separate DAI link for each BE to BE connection, then it will result in a similar design what we have currently.
* Hostless: I did not come across references for this. (Any references in this regard will be helpful)
May be the current Tegra ASoC design is more suitable for component model as you had previously mentioned. I wanted to understand if above, especially (a) and (b), are acceptable in this regard or if there are better options to interconnect multiple ASoC components.
Looking forward for your feedback.
Thanks, Sameer.
References ========== [0] http://patchwork.ozlabs.org/project/linux-tegra/list/?series=159664&arch... [1] http://patchwork.ozlabs.org/project/linux-tegra/patch/1582180492-25297-6-git... [2] http://patchwork.ozlabs.org/project/linux-tegra/patch/1582180492-25297-4-git... [3] https://www.kernel.org/doc/html/v5.6/sound/soc/dpcm.html
On Thu, Apr 30, 2020 at 06:11:23PM +0530, Sameer Pujar wrote:
a) Can I use a DAPM Mux control to activate a BE path? This in turn can program required switch in XBAR.
If it works then sure, that seems sensible.
b) I have modelled SFC and MIXER as backends. Is this allowed?
This was done to include SFC or MIXER HW components as part of the sound card and use like below in one of the audio use cases. ADMAIF1(FE) --> SFC(BE1) --> I2S(BE2) ... OR ADMAIF2(FE) --> SFC(BE1) --> I2S(BE2) ...
This is the sort of setup that'd be a lot happier using a component model.
I used following workaround to connect multiple BE components. With this I can see PCM callbacks happen for all BE DAIs along the DAPM path. The obective was to connect multiple components together and (a) was used to connect one component to another. Each "-->" here connects two components and it is a switch in XBAR.
This doesn't strike me as something that's likely to be robust but given that that applies to DPCM in general so long as it doesn't break anyone else's existing stuff I guess it should be viable, it's not like there are actually good options that you could use currently. It's really hard to get enthusiastic about it though.
c) Hostless mode did NOT work: - Following audio path was intended to be tested: I2S1 --> SFC --> I2S2
- [3] offers two options: * CODEC<->CODEC: If I were to use a separate DAI link for each BE to BE connection, then it will result in a similar design what we have currently.
This is more in line with components so will probably be easier going forwards.
* Hostless: I did not come across references for this. (Any references in this regard will be helpful)
Not sure anyone has ever done this with DPCM, could be wrong though.
May be the current Tegra ASoC design is more suitable for component model as you had previously mentioned. I wanted to understand if above, especially (a) and (b), are acceptable in this regard or if there are better options to interconnect multiple ASoC components.
In general most systems would be happier with components but yeah, I think that's particularly the case for something as powerful and flexible as your hardware seems to be.
On Thu 30 Apr 2020 at 14:41, Sameer Pujar spujar@nvidia.com wrote:
At a high level Tegra Audio HW is depicted as below.
| Front End PCMs | SoC DSP | Back End DAIs |
*************ADMAIF<0> <------------> * * <----DAI<0>-----> I2S * * ADMAIF<1> <------------> * * <----DAI<1>-----> DMIC * XBAR * ADMAIF<2> <------------> * * <----DAI<2>-----> DSPK * * ADMAIF<N> <------------> * * <----DAI<3>-----> SFC (Resampler) * * * * <----DAI<4>-----> MIXER * * * * <----DAI<N>-----> ... *************
Follow up queries
Based on the above experience I do have few follow up queries and request for your inputs on this.
a) Can I use a DAPM Mux control to activate a BE path? This in turn can program required switch in XBAR.
My 2 cents: DPCM should activate any BE which has: * a valid DAPM path from the current FE * a valid BE stream (ex: can handle the stream direction)
AFAIK, you can use any combination of DAPM elements to model your components, including the XBAR. Then, it is the job of the card driver to link the DAPM widgets of the different components together and bring the system to life.
If your XBAR is widgets are not provided by a component which also provides a dai_link in the sound card, you'll need to add the component to the auxiliary device list of the card for the widget to be available in the card.
This is needed for following reasons: - For an open platform like Jetson, we want to give maximum flexibility for a user to customize their audio paths. Number of connected components and order of these can vary depending on a use case. - Allow re-use of audio components across multiple use cases. For example, number of SFC instances are lesser than PCM playback or capture devices.b) I have modelled SFC and MIXER as backends. Is this allowed?
This was done to include SFC or MIXER HW components as part of the sound card and use like below in one of the audio use cases. ADMAIF1(FE) --> SFC(BE1) --> I2S(BE2) ... OR ADMAIF2(FE) --> SFC(BE1) --> I2S(BE2) ...
What your are trying to achieve here is DAI chaining. From the DAPM perspective, it is alright. Unfortunately this kind of chaining are not supported (not yet, at least) from the dai_link perspective, even with DPCM.
From the DPCM perspective, I think your patch does the following:
ADMAIF1(FE) ---> SFC(BE1) |--> I2S(BE2) ADMAIF2(FE) ---> SFC(BE1) |--> I2S(BE2)
Your DAIs are not chained, but put in parallel. Maybe all your DAI callbacks are called in a way that is convinient enough for your use cases, but it is not what you intended.
For the example, the rate passed to 'I2S(BE2)' in hw_params() will be the one provided by the 'ADMAIF', not the output rate of the 'SFC'.
I used following workaround to connect multiple BE components. With this I can see PCM callbacks happen for all BE DAIs along the DAPM path. The obective was to connect multiple components together and (a) was used to connect one component to another. Each "-->" here connects two components and it is a switch in XBAR. --- sound/soc/soc-pcm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sound/soc/soc-pcm.c b/sound/soc/soc-pcm.c index e256d43..ee7af55 100644 --- a/sound/soc/soc-pcm.c +++ b/sound/soc/soc-pcm.c @@ -1494,7 +1494,7 @@ int dpcm_path_get(struct snd_soc_pcm_runtime *fe, /* get number of valid DAI paths and their widgets */ paths = snd_soc_dapm_dai_get_connected_widgets(cpu_dai, stream, list, - dpcm_end_walk_at_be); + NULL);dev_dbg(fe->dev, "ASoC: found %d audio %s paths\n", paths, stream ? "capture" : "playback"); --
c) Hostless mode did NOT work: - Following audio path was intended to be tested: I2S1 --> SFC --> I2S2
- [3] offers two options: * CODEC<->CODEC: If I were to use a separate DAI link for each BE to BE connection, then it will result in a similar design what we have currently. * Hostless: I did not come across references for this. (Any references in this regard will be helpful)May be the current Tegra ASoC design is more suitable for component model as you had previously mentioned. I wanted to understand if above, especially (a) and (b), are acceptable in this regard or if there are better options to interconnect multiple ASoC components.
Looking forward for your feedback.
Thanks, Sameer.
References
[0] http://patchwork.ozlabs.org/project/linux-tegra/list/?series=159664&arch... [1] http://patchwork.ozlabs.org/project/linux-tegra/patch/1582180492-25297-6-git... [2] http://patchwork.ozlabs.org/project/linux-tegra/patch/1582180492-25297-4-git... [3] https://www.kernel.org/doc/html/v5.6/sound/soc/dpcm.html
On 5/6/2020 5:21 PM, Jerome Brunet wrote:
External email: Use caution opening links or attachments
On Thu 30 Apr 2020 at 14:41, Sameer Pujar spujar@nvidia.com wrote:
At a high level Tegra Audio HW is depicted as below.
| Front End PCMs | SoC DSP | Back End DAIs |
*************ADMAIF<0> <------------> * * <----DAI<0>-----> I2S * * ADMAIF<1> <------------> * * <----DAI<1>-----> DMIC * XBAR * ADMAIF<2> <------------> * * <----DAI<2>-----> DSPK * * ADMAIF<N> <------------> * * <----DAI<3>-----> SFC (Resampler) * * * * <----DAI<4>-----> MIXER * * * * <----DAI<N>-----> ... *************
Follow up queries
Based on the above experience I do have few follow up queries and request for your inputs on this.
a) Can I use a DAPM Mux control to activate a BE path? This in turn can program required switch in XBAR.
My 2 cents:
DPCM should activate any BE which has:
- a valid DAPM path from the current FE
- a valid BE stream (ex: can handle the stream direction)
Yes, this is taken care.
AFAIK, you can use any combination of DAPM elements to model your components, including the XBAR. Then, it is the job of the card driver to link the DAPM widgets of the different components together and bring the system to life.
XBAR currently exports all routing widgets which can be used to interconnect multiple components and thus implements Mux widgets. Fixing the routing paths in driver would limit anyone to try a different combination as per the need, unless driver is modified. Device tree (DT) can be one of the solutions here, but currently static paths can only be added AFAIK. Even if this is extended to include routes with Mux widgets, still it won't give the real flexibility. I cannot re-use a component for a different routing path, unless DT is modified again.
Hence a Mux widget with user space control could offer required flexibility.
If your XBAR is widgets are not provided by a component which also provides a dai_link in the sound card, you'll need to add the component to the auxiliary device list of the card for the widget to be available in the card.
I registered XBAR device as a component having a dummy DAI just to allow routing paths to be available for the given sound card. Are you suggesting I can register XBAR as an auxiliary device so that all the routing paths are available?
This is needed for following reasons: - For an open platform like Jetson, we want to give maximum flexibility for a user to customize their audio paths. Number of connected components and order of these can vary depending on a use case. - Allow re-use of audio components across multiple use cases. For example, number of SFC instances are lesser than PCM playback or capture devices.b) I have modelled SFC and MIXER as backends. Is this allowed?
This was done to include SFC or MIXER HW components as part of the sound card and use like below in one of the audio use cases. ADMAIF1(FE) --> SFC(BE1) --> I2S(BE2) ... OR ADMAIF2(FE) --> SFC(BE1) --> I2S(BE2) ...
What your are trying to achieve here is DAI chaining. From the DAPM perspective, it is alright. Unfortunately this kind of chaining are not supported (not yet, at least) from the dai_link perspective, even with DPCM.
In the current design we achieve this by using codec-to-codec links and attempt here is if we can use DPCM to simplify few things.
From the DPCM perspective, I think your patch does the following:
ADMAIF1(FE) ---> SFC(BE1) |--> I2S(BE2) ADMAIF2(FE) ---> SFC(BE1) |--> I2S(BE2)
Your DAIs are not chained, but put in parallel. Maybe all your DAI callbacks are called in a way that is convinient enough for your use cases, but it is not what you intended.
Yes this seems to be the case. All of my BE DAIs are added to the list and callbacks happen for all. DPCM is viewing these as parallel paths.
For the example, the rate passed to 'I2S(BE2)' in hw_params() will be the one provided by the 'ADMAIF', not the output rate of the 'SFC'.
I was earlier thinking if dpcm_be_dai_hw_params() can be enhanced to propagate PCM parameters (sample rate, channels, sample size) from BE to BE. This is a rough idea, I have not tried this. But because of above it will be a problem unless real parallel paths are not differentiated. For example: ADMAIF1(FE) ---> SFC1 ---> I2S1 (operated at 48 kHz) |--> SFC2 ---> I2S2 (operated at 16 kHz)
If I understand correctly, this problem will be there if I want to drive multiple paths with the "same FE". I am not sure how often we have such requirements. For example below would work fine (though it is functionally not same as above example): ADMAIF1(FE) ---> SFC1 ---> I2S1 (operated at 48 kHz) ADMAIF2(FE) ---> SFC2 ---> I2S2 (operated at 16 kHz)
I used following workaround to connect multiple BE components. With this I can see PCM callbacks happen for all BE DAIs along the DAPM path. The obective was to connect multiple components together and (a) was used to connect one component to another. Each "-->" here connects two components and it is a switch in XBAR. --- sound/soc/soc-pcm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sound/soc/soc-pcm.c b/sound/soc/soc-pcm.c index e256d43..ee7af55 100644 --- a/sound/soc/soc-pcm.c +++ b/sound/soc/soc-pcm.c @@ -1494,7 +1494,7 @@ int dpcm_path_get(struct snd_soc_pcm_runtime *fe, /* get number of valid DAI paths and their widgets */ paths = snd_soc_dapm_dai_get_connected_widgets(cpu_dai, stream, list, - dpcm_end_walk_at_be); + NULL); dev_dbg(fe->dev, "ASoC: found %d audio %s paths\n", paths, stream ? "capture" : "playback"); --
. . .
References
[0] http://patchwork.ozlabs.org/project/linux-tegra/list/?series=159664&arch... [1] http://patchwork.ozlabs.org/project/linux-tegra/patch/1582180492-25297-6-git... [2] http://patchwork.ozlabs.org/project/linux-tegra/patch/1582180492-25297-4-git... [3] https://www.kernel.org/doc/html/v5.6/sound/soc/dpcm.html
On Wed 06 May 2020 at 16:12, Sameer Pujar spujar@nvidia.com wrote:
| Front End PCMs | SoC DSP | Back End DAIs |
*************ADMAIF<0> <------------> * * <----DAI<0>-----> I2S * * ADMAIF<1> <------------> * * <----DAI<1>-----> DMIC * XBAR * ADMAIF<2> <------------> * * <----DAI<2>-----> DSPK * * ADMAIF<N> <------------> * * <----DAI<3>-----> SFC (Resampler) * * * * <----DAI<4>-----> MIXER * * * * <----DAI<N>-----> ... *************
Follow up queries
Based on the above experience I do have few follow up queries and request for your inputs on this.
a) Can I use a DAPM Mux control to activate a BE path? This in turn can program required switch in XBAR.
My 2 cents:
DPCM should activate any BE which has:
- a valid DAPM path from the current FE
- a valid BE stream (ex: can handle the stream direction)
Yes, this is taken care.
AFAIK, you can use any combination of DAPM elements to model your components, including the XBAR. Then, it is the job of the card driver to link the DAPM widgets of the different components together and bring the system to life.
XBAR currently exports all routing widgets which can be used to interconnect multiple components and thus implements Mux widgets. Fixing the routing paths in driver would limit anyone to try a different combination as per the need, unless driver is modified.
I did not mean that you should restrict the routing ability of your SoC, quite the opposite actually.
You should just expose it correctly
From the diagram above, each comp (ASMAIFs, BE even the XBAR) should
export AIF_IN/AIF_OUT widgets. Think of those as the interfaces of your components.
The connections between those interfaces are fixed, the represent what the HW is (IOW, the wires). They could possibly be described in DT (audio-routing prop)
The XBAR should implement the N to M MUX and export all the necessary kcontrols to act it. The routing magic happens in this component. There is no limit to the flexibility this provides, unless you set one ;)
Device tree (DT) can be one of the solutions here, but currently static paths can only be added AFAIK. Even if this is extended to include routes with Mux widgets, still it won't give the real flexibility. I cannot re-use a component for a different routing path, unless DT is modified again.
If you implement the XBAR capability in the comp, not a use case, there is no reason to change it.
DT describes the HW, changing it to describe a use case means something is wrong, usually.
Hence a Mux widget with user space control could offer required flexibility.
Exactly, your XBAR comp should provide the widget(s) and related kcontrol(s).
If your XBAR is widgets are not provided by a component which also provides a dai_link in the sound card, you'll need to add the component to the auxiliary device list of the card for the widget to be available in the card.
I registered XBAR device as a component having a dummy DAI just to allow routing paths to be available
From you description, you XBAR should not have DAIs, just DAPM widgets
and kcontrols
for the given sound card. Are you suggesting I can register XBAR as an auxiliary device so that all the routing paths are available?
Precisely. I have been using the same method on Amlogic cards.
On Wed, May 06, 2020 at 04:47:09PM +0200, Jerome Brunet wrote:
On Wed 06 May 2020 at 16:12, Sameer Pujar spujar@nvidia.com wrote:
XBAR currently exports all routing widgets which can be used to interconnect multiple components and thus implements Mux widgets. Fixing the routing paths in driver would limit anyone to try a different combination as per the need, unless driver is modified.
I did not mean that you should restrict the routing ability of your SoC, quite the opposite actually.
You should just expose it correctly
Yes, it's going to be less effort in the long run if nothing else.
On 5/6/2020 9:23 PM, Mark Brown wrote:
On Wed, May 06, 2020 at 04:47:09PM +0200, Jerome Brunet wrote:
On Wed 06 May 2020 at 16:12, Sameer Pujar spujar@nvidia.com wrote:
XBAR currently exports all routing widgets which can be used to interconnect multiple components and thus implements Mux widgets. Fixing the routing paths in driver would limit anyone to try a different combination as per the need, unless driver is modified.
I did not mean that you should restrict the routing ability of your SoC, quite the opposite actually.
You should just expose it correctly
Yes, it's going to be less effort in the long run if nothing else.
This is what below reference tried to achieve in the original series, where all Mux widgets and corresponding kcontrols were exposed. [1] http://patchwork.ozlabs.org/project/linux-tegra/patch/1582180492-25297-6-git...
However it has DAI declarations too, that was mainly because codec-to-codec links were used to connect multiple components. DT would expose all these links (please note that machine driver was not sent as part of the original series). May be with DPCM these additional DAIs can be avoided, but it comes with few challenges highlighted in this RFC patch.
participants (3)
-
Jerome Brunet -
Mark Brown -
Sameer Pujar