How to handle complex Codec2Codec
Hi ALSA ML
ALSA SoC is supporting Codec2Codec connection, and Audio Graph Card2 is supporting it. I think its image is...
+------+ +--- |Codec1| <-- | +------+ | +------+ +--> |Codec2| --> +------+
This is OK, but how to handle below ?
+----------------------+ |CPU | +------+ | +--[rate convert]<--| <-- |Codec1| <-- | | | +------+ | | | +------+ | +------------------>| --> |Codec2| --> +----------------------+ +------+
I think there are 2 issues. (1) How to handle [CPU] part on Codec2Codec ? (2) How to handle [rate convert] on Codec2Codec ?
I guess (1) can be handled if [CPU] was worked as platform, but do you have any other idea/solution for it ?
I think (2) [rate convert] needs DPCM connection, but how to use it with Codec2Codec ?
Or, can we handle Codec2Codec via DPCM somehow ??
FE BE +------+ +------+ --> |Codec1| -> |Codec2| -> +------+ +------+
If we can do this, both (1) and (2) can be solved, like below ? In such case, [Codec1] need to work as [capture] when playback. But I'm not sure how to implement it...
FE BE +------+ +-----------------+ +-----+ +------+ --> |Codec1|-|CPU[rate convert]| -> |dummy|-|Codec2| -> +------+ +-----------------+ +-----+ +------+
Thank you for your help !!
Best regards --- Kuninori Morimoto
On Thu, Aug 29, 2024 at 04:51:49AM +0000, Kuninori Morimoto wrote:
This is OK, but how to handle below ?
+----------------------+ |CPU | +------+ | +--[rate convert]<--| <-- |Codec1| <-- | | | +------+ | | | +------+ | +------------------>| --> |Codec2| --> +----------------------+ +------+
I think there are 2 issues. (1) How to handle [CPU] part on Codec2Codec ? (2) How to handle [rate convert] on Codec2Codec ?
At the minute this would be a DPCM thing. I've never actively worked on a system with DPCM so I'm not as up to speed as I could be but with my current understanding I'd expect the rate conversion to be done as a backend fixup on the DAI link for the first CODEC and just a straight pass through for the second.
I think (2) [rate convert] needs DPCM connection, but how to use it with Codec2Codec ?
The whole thing is a bit fragile so I don't know how well those two play together. Though your picture doesn't look like it's actually doing a codec2codec link? It probably should ideally (with the SRC being a CODEC in the graph) but not right now.
Hi Mark
Thank you for your feedback
+----------------------+ |CPU | +------+ | +--[rate convert]<--| <-- |Codec1| <-- | | | +------+ | | | +------+ | +------------------>| --> |Codec2| --> +----------------------+ +------+
I think there are 2 issues. (1) How to handle [CPU] part on Codec2Codec ? (2) How to handle [rate convert] on Codec2Codec ?
At the minute this would be a DPCM thing. I've never actively worked on a system with DPCM so I'm not as up to speed as I could be but with my current understanding I'd expect the rate conversion to be done as a backend fixup on the DAI link for the first CODEC and just a straight pass through for the second.
I think (2) [rate convert] needs DPCM connection, but how to use it with Codec2Codec ?
The whole thing is a bit fragile so I don't know how well those two play together. Though your picture doesn't look like it's actually doing a codec2codec link? It probably should ideally (with the SRC being a CODEC in the graph) but not right now.
I think we can learn how Codec2Codec is working, and apply it to DPCM somehow. Then, if we can handle above via DPCM, I guess normal Codec2Codec also can be handled via DPCM ?
Thank you for your help !!
Best regards --- Kuninori Morimoto
On Mon, Sep 02, 2024 at 03:48:27AM +0000, Kuninori Morimoto wrote:
I think we can learn how Codec2Codec is working, and apply it to DPCM somehow. Then, if we can handle above via DPCM, I guess normal Codec2Codec also can be handled via DPCM ?
I think that would cause trouble, with DPCM all the CPU drivers know about each other and it's all horribly fragile. With two CODECs there's no CPU involved so the CPU drivers wouldn't know what's going on, I'd expect that at best the code would end up looking similar and we'd not gain much. It's possible I'm not seeing something though.
Hi Mark
I think we can learn how Codec2Codec is working, and apply it to DPCM somehow. Then, if we can handle above via DPCM, I guess normal Codec2Codec also can be handled via DPCM ?
I think that would cause trouble, with DPCM all the CPU drivers know about each other and it's all horribly fragile. With two CODECs there's no CPU involved so the CPU drivers wouldn't know what's going on, I'd expect that at best the code would end up looking similar and we'd not gain much. It's possible I'm not seeing something though.
Hmm... Because Sound data path can be more complex from HW point of view, and customer want to use it. And this trend will be increase/continue. But it seems current ALSA SoC can't handle it in good way.
Unfortunately, indeed DPCM is fragile. We want to keep compatible, and thus updating it to expand feature is not good idea, IMO.
We want to have new style connection which is more flexible, and is possible to handle all connection in same way ? ... that may be an exaggeration.
Thank you for your help !!
Best regards --- Kuninori Morimoto
On Thu, Sep 05, 2024 at 02:21:37AM +0000, Kuninori Morimoto wrote:
We want to have new style connection which is more flexible, and is possible to handle all connection in same way ? ... that may be an exaggeration.
Yeah, at a super high level we want to be able to handle digital audio with DAPM like analog audio, the main thing is we'd need to propagate digital parameters along as well as just on/off status, and support conversions.
Hi Mark
We want to have new style connection which is more flexible, and is possible to handle all connection in same way ? ... that may be an exaggeration.
Yeah, at a super high level we want to be able to handle digital audio with DAPM like analog audio, the main thing is we'd need to propagate digital parameters along as well as just on/off status, and support conversions.
Very rough speaking, we don't want to break existing connections (normal, DPCM, Codec2Codec, etc), and enable to use new style right ? Let's name current style as PCMv1. I think we want to grouping related things into one place, let's say soc-pcm.c, in roughly.
And make new style (PCMv2) at soc-pcm2.c, for example. Then, we can swtich v1 or v2 somehow, like below ?
-------- soc-pcm.c ----- struct pcm_connection pcm1_connection = { .ver = 1, .new_pcm = soc_new_pcm, ... }; -------- soc-pcm2.c ----- struct pcm_connection pcm2_connection = { .ver = 2, .new_pcm = new_pcm, ... }; -------- soc-core.c ----- if (ver == v1) connection = pcm1_connection else (ver == v2) connection = pcm2_connection
- ret = soc_new_pcm(rtd, num); + ret = connection->new_pcm(rtd, num);
We can create PCMv3, v4, v5... in the future if existing connection style was not good enough. ... well... this is almost "ideal" ;P
Thank you for your help !!
Best regards --- Kuninori Morimoto
On Thu, Sep 05, 2024 at 11:27:41PM +0000, Kuninori Morimoto wrote:
Yeah, at a super high level we want to be able to handle digital audio with DAPM like analog audio, the main thing is we'd need to propagate digital parameters along as well as just on/off status, and support conversions.
Very rough speaking, we don't want to break existing connections (normal, DPCM, Codec2Codec, etc), and enable to use new style right ? Let's name current style as PCMv1. I think we want to grouping related things into one place, let's say soc-pcm.c, in roughly.
Well, long term we'd want to remove DPCM and CODEC 2 CODEC but kind of I think.
And make new style (PCMv2) at soc-pcm2.c, for example. Then, we can swtich v1 or v2 somehow, like below ?
-------- soc-pcm.c ----- struct pcm_connection pcm1_connection = { .ver = 1, .new_pcm = soc_new_pcm, ... }; -------- soc-pcm2.c ----- struct pcm_connection pcm2_connection = { .ver = 2, .new_pcm = new_pcm, ... }; -------- soc-core.c ----- if (ver == v1) connection = pcm1_connection else (ver == v2) connection = pcm2_connection
- ret = soc_new_pcm(rtd, num);
- ret = connection->new_pcm(rtd, num);
It's not clear to me if we'd need to specify things as an explicitly PCM link, or if we'd be able to just use a DPCM route to link things and then have be able to automatically configure the digital bits based on capabilities of the things being linked. We would need to provide a lot more information on the things being connected in order to do that, and some of them would need digital operations. Ideally we'd be able to do things in such a way that we can transparently transition the implementation used for simpler existing boards without requring them to be rewriten if they're not using one of the more complex things like DPCM that we're trying to replace.
We can create PCMv3, v4, v5... in the future if existing connection style was not good enough. ... well... this is almost "ideal" ;P
Doing things as described above means that there's much less information in the individual drivers, just descriptions of what's connected to what as much as possible. To a certain extent the fact that that's not the case is kind of the problem we're trying to solve here.
Hi Mark
Very rough speaking, we don't want to break existing connections (normal, DPCM, Codec2Codec, etc), and enable to use new style right ? Let's name current style as PCMv1. I think we want to grouping related things into one place, let's say soc-pcm.c, in roughly.
Well, long term we'd want to remove DPCM and CODEC 2 CODEC but kind of I think.
This is my opinion, but even though if we can have new PCMv2 connections, I think we would like to keep existing DPCM / Codec2Codec as PCMv1 with no touch as much as possible not to break existing working system. Converting PCMv1 -> PCMv2 is maybe easy to break, especially non-normal connection. See below.
It's not clear to me if we'd need to specify things as an explicitly PCM link, or if we'd be able to just use a DPCM route to link things and then have be able to automatically configure the digital bits based on capabilities of the things being linked. We would need to provide a lot more information on the things being connected in order to do that, and some of them would need digital operations. Ideally we'd be able to do things in such a way that we can transparently transition the implementation used for simpler existing boards without requring them to be rewriten if they're not using one of the more complex things like DPCM that we're trying to replace.
Current DPCM vs Normal connections are using different ops, and Codec2Codec is using own style. Because of that, the code is very complex, and difficult to understand IMO.
It is better to have an flexible and integrated system that assumes various connection methods from begining.
In my understanding, for Normal connection, each driver is no need to update, or maybe small update only for PCMv2. But need to use new Card driver to use PCMv2 connection. For complex connection, like DPCM/Codec2Codec on PCMv1, needs to use new style on PCMv2, there is no compatible.
I think if we really start to create PCMv2 connection, I think we can use test-component and new audio-graph-card3, and its sample.dtsi for test/try ? We can try various connections without physical board on it.
This is also my opinion, but I don't think people will be happy if we adds new PCMv2 by using "change everything by big 1 patch". I think it's better to exchange ideas and opinions on ML, and make progress step-by-step.
We can create PCMv3, v4, v5... in the future if existing connection style was not good enough. ... well... this is almost "ideal" ;P
Doing things as described above means that there's much less information in the individual drivers, just descriptions of what's connected to what as much as possible. To a certain extent the fact that that's not the case is kind of the problem we're trying to solve here.
Existing connection style has been created without considering to able to switch connection style of course. But I think we want to create such concept (= style version), and convert existing style into it (= PCMv1). In my quick check, compress / sof are deeply based on DPCM style, so I think using them need to use PCMv1 style. In other words, using/converting it on PCMv2 is maybe very difficult. So we want to have the concept which enable to use both PCMv1 and PCMv2 in the same time. Maybe same things happen on PCMv3 or later in the future (?).
Above all are just my opinion, I'm not sure what is the good way.
Thank you for your help !!
Best regards --- Kuninori Morimoto
participants (2)
-
Kuninori Morimoto -
Mark Brown