[alsa-devel] [RFC 0/4] ASoC DSP topology
It's been about 2 years since posting V1 of the ASoC dynamic firmware patches and a lot has changed since then ...
This RFC series is being posted instead of a V2 patch series atm since we are still completing some testing on the codebase and developing a new userspace tool. Expect V2 once more testing is completed shortly.
ASoC topology (formerly called dynamic firmware) allows audio DSP topology to be defined as part of the firmware package and not hard coded into the driver. This allows DSP vendors to ship multiple different firmwares or firmwares with programmable topologies for different devices (i.e. phones, tablets, TVs) using a single DSP driver. The DSP topology can be contained as part of the firmware file or exist as separate files. The topology core and file format is generic and not tied to any single vendor or device.
The topology core consists of kernel and userspace components :-
Kernel. The kernel core reads in the topology data and builds the driver capabilities and audio topology based on this data. This includes building the DAPM graph, setting PCM and DAI capabilities, registering kcontrols and registering DAPM widgets.
The userspace tool allows the creation of binary DSP FW topology files based on a topology text files.
Changes since V1:-
o Name changed to ASoC Topology o Structures from soc.h and soc-dapm.h no longer exposed as UAPIs o Moved to component model from codec/platform model. o Added reserved fields in each UAPI structure. o UAPI structures packed and endianess defined for each structure. o Added support for PCM, DAI and codec <-> codec link capabilities. o Added support for client drivers manifests. o Added support for private data for each dynamic object type. o Single list of dynamic objects rather than list of each object type.
Liam
On Thu, Apr 16, 2015 at 09:48:05PM +0100, Liam Girdwood wrote:
It's been about 2 years since posting V1 of the ASoC dynamic firmware patches and a lot has changed since then ...
:)
Kernel. The kernel core reads in the topology data and builds the driver capabilities and audio topology based on this data. This includes building the DAPM graph, setting PCM and DAI capabilities, registering kcontrols and registering DAPM widgets.
I think the topology code itself looks good, I'll take another pass over it though since there's a lot to get through. The main thing is to make sure the ABI makes sense though - we can fix up the kernel later.
Liam,
On 04/16/2015 11:48 PM, Liam Girdwood wrote:
It's been about 2 years since posting V1 of the ASoC dynamic firmware patches and a lot has changed since then ...
This RFC series is being posted instead of a V2 patch series atm since we are still completing some testing on the codebase and developing a new userspace tool. Expect V2 once more testing is completed shortly.
ASoC topology (formerly called dynamic firmware) allows audio DSP topology to be defined as part of the firmware package and not hard coded into the driver. This allows DSP vendors to ship multiple different firmwares or firmwares with programmable topologies for different devices (i.e. phones, tablets, TVs) using a single DSP driver. The DSP topology can be contained as part of the firmware file or exist as separate files. The topology core and file format is generic and not tied to any single vendor or device.
The topology core consists of kernel and userspace components :-
Kernel. The kernel core reads in the topology data and builds the driver capabilities and audio topology based on this data. This includes building the DAPM graph, setting PCM and DAI capabilities, registering kcontrols and registering DAPM widgets.
It is not entirely clear for me based on the first look, but who is responsible to initiate the topology load? Is it the component or machine driver? We have had issues with deferred probing when the component driver was in charge of loading the firmware. I got around this by initiating the FW load from the machine driver and via callbacks I notified the component driver to take over from that point. This fixed the probe order and I can also handle cases when the filesystem does not have the firmware so I can fall back to 'legacy' mode.
The userspace tool allows the creation of binary DSP FW topology files based on a topology text files.
Is it possible to get access to the user space part so if I have time I can try to convert the OMAP4/5 AESS/ABE code to use tplg?
Thanks, Péter
Changes since V1:-
o Name changed to ASoC Topology o Structures from soc.h and soc-dapm.h no longer exposed as UAPIs o Moved to component model from codec/platform model. o Added reserved fields in each UAPI structure. o UAPI structures packed and endianess defined for each structure. o Added support for PCM, DAI and codec <-> codec link capabilities. o Added support for client drivers manifests. o Added support for private data for each dynamic object type. o Single list of dynamic objects rather than list of each object type.
Liam
Alsa-devel mailing list Alsa-devel@alsa-project.org http://mailman.alsa-project.org/mailman/listinfo/alsa-devel
On Tue, Apr 21, 2015 at 09:58:27AM +0300, Peter Ujfalusi wrote:
It is not entirely clear for me based on the first look, but who is responsible to initiate the topology load? Is it the component or machine driver? We have had issues with deferred probing when the component driver was in charge of loading the firmware. I got around this by initiating the FW load from the machine driver and via callbacks I notified the component driver to take over from that point. This fixed the probe order and I can also handle cases when the filesystem does not have the firmware so I can fall back to 'legacy' mode.
Could you expand on those issues please? I'd *really* not expect the machine driver to be involved in loading firmware for a component driver (think how this is going to affect generic drivers) and it's not obvious to me what impact this might have on deferred probe either.
On Tue, 2015-04-21 at 10:28 +0100, Mark Brown wrote:
On Tue, Apr 21, 2015 at 09:58:27AM +0300, Peter Ujfalusi wrote:
It is not entirely clear for me based on the first look, but who is responsible to initiate the topology load? Is it the component or machine driver? We have had issues with deferred probing when the component driver was in charge of loading the firmware. I got around this by initiating the FW load from the machine driver and via callbacks I notified the component driver to take over from that point. This fixed the probe order and I can also handle cases when the filesystem does not have the firmware so I can fall back to 'legacy' mode.
Could you expand on those issues please? I'd *really* not expect the machine driver to be involved in loading firmware for a component driver (think how this is going to affect generic drivers) and it's not obvious to me what impact this might have on deferred probe either.
The OMAP4 driver should not have any issues using the new topology code, it's very generic :)
The topology code has moved to the component model so the ABE DSP driver can load it's FW topology. If the FS is not available to load FW or topology then you can wait until FS is ready (the upstream SST drivers wait for FW to be ready before continuing with probing).
Liam
On 04/21/2015 12:28 PM, Mark Brown wrote:
On Tue, Apr 21, 2015 at 09:58:27AM +0300, Peter Ujfalusi wrote:
It is not entirely clear for me based on the first look, but who is responsible to initiate the topology load? Is it the component or machine driver? We have had issues with deferred probing when the component driver was in charge of loading the firmware. I got around this by initiating the FW load from the machine driver and via callbacks I notified the component driver to take over from that point. This fixed the probe order and I can also handle cases when the filesystem does not have the firmware so I can fall back to 'legacy' mode.
Could you expand on those issues please? I'd *really* not expect the machine driver to be involved in loading firmware for a component driver (think how this is going to affect generic drivers) and it's not obvious to me what impact this might have on deferred probe either.
The two issue were interconnected. We had cases when all components have been loaded, but the machine driver was still in the deferred list, if we loaded any non related module the card will suddenly got created. It is due to the firmware loading. The firmware load will not kick the deferred module list (we had a hack in place to kick the queue from the ABE driver): Components are loaded and registered in ASoC with the exception of the parts needing the FW, since the dynamic FW contained widgets, routes, etc. So the ABE driver loaded and requested the FW, probe exited with success. Machine driver is still not able to form the card since the ABE widgets, PCMs are missing, it goes to deferred probe. The ABE driver loads the dynamic FW, parses it, DAPM widgets, PCMs, routes are created and registered. But the machine driver is in deferred list and there is nothing to kick that queue, which means that we have everything, but the card is not created.
But if we initiate the FW load from the machine driver and we pass the FW down to the component driver(s) interested in handling the FW we are going to be fine since we only register the card when the component drivers already parsed and processed the FW image.
In this way we can have fall back for cases when the FW is not available on the filesystem (some distro might want to go w/o binary blob).
We have discussed this with Liam in the past: in my view the DSP topology (or Dynamic FW) should be represented in the machine level and it would be the best if the same image could carry card level widgets routes and links. If you have big enough change in the FW and it's provided widgets/PCMs you would need separate machine driver or at least a way to have different set of machine level routes, widgets, links, etc for different DSP topology file.
One of the issue with this: When we need to call the component driver's firmware load function (to process the FW and create the widgets and then register the component) the component is not registered (we need the firmware to set up things) so generic snd_soc_* callback is not possible to use
On Tue, 2015-04-21 at 15:30 +0300, Peter Ujfalusi wrote:
On 04/21/2015 12:28 PM, Mark Brown wrote:
On Tue, Apr 21, 2015 at 09:58:27AM +0300, Peter Ujfalusi wrote:
It is not entirely clear for me based on the first look, but who is responsible to initiate the topology load? Is it the component or machine driver? We have had issues with deferred probing when the component driver was in charge of loading the firmware. I got around this by initiating the FW load from the machine driver and via callbacks I notified the component driver to take over from that point. This fixed the probe order and I can also handle cases when the filesystem does not have the firmware so I can fall back to 'legacy' mode.
Could you expand on those issues please? I'd *really* not expect the machine driver to be involved in loading firmware for a component driver (think how this is going to affect generic drivers) and it's not obvious to me what impact this might have on deferred probe either.
The two issue were interconnected. We had cases when all components have been loaded, but the machine driver was still in the deferred list, if we loaded any non related module the card will suddenly got created. It is due to the firmware loading. The firmware load will not kick the deferred module list (we had a hack in place to kick the queue from the ABE driver):
Yes, I remember that hack :)
Can you take a look at the work Jarkko did for FW and component loading in sst-acpi.c. This should work for ABE too.
Components are loaded and registered in ASoC with the exception of the parts needing the FW, since the dynamic FW contained widgets, routes, etc. So the ABE driver loaded and requested the FW, probe exited with success. Machine driver is still not able to form the card since the ABE widgets, PCMs are missing, it goes to deferred probe. The ABE driver loads the dynamic FW, parses it, DAPM widgets, PCMs, routes are created and registered. But the machine driver is in deferred list and there is nothing to kick that queue, which means that we have everything, but the card is not created.
But if we initiate the FW load from the machine driver and we pass the FW down to the component driver(s) interested in handling the FW we are going to be fine since we only register the card when the component drivers already parsed and processed the FW image.
In this way we can have fall back for cases when the FW is not available on the filesystem (some distro might want to go w/o binary blob).
We have discussed this with Liam in the past: in my view the DSP topology (or Dynamic FW) should be represented in the machine level and it would be the best if the same image could carry card level widgets routes and links. If you have big enough change in the FW and it's provided widgets/PCMs you would need separate machine driver or at least a way to have different set of machine level routes, widgets, links, etc for different DSP topology file.
The component level allows us to target the physical component devices that may have runtime definable topologies. This would include codecs too, since some vendors are making codecs with built in FW (maybe TI too ?). The machine level more represents the board HW topology and this should be derived from ACPI or DT.
Fwiw, I've not been following ABE development so I'm not sure of the current driver state in newer kernels. I dont see any problems if the ABE driver used the same sst-acpi.c like probing method. Btw, there is now also a non atomic trigger() that would simplify the ABE code too :)
Liam
On Tue, Apr 21, 2015 at 04:23:51PM +0100, Liam Girdwood wrote:
On Tue, 2015-04-21 at 15:30 +0300, Peter Ujfalusi wrote:
We have discussed this with Liam in the past: in my view the DSP topology (or Dynamic FW) should be represented in the machine level and it would be the best if the same image could carry card level widgets routes and links. If you have big enough change in the FW and it's provided widgets/PCMs you would need separate machine driver or at least a way to have different set of machine level routes, widgets, links, etc for different DSP topology file.
The component level allows us to target the physical component devices that may have runtime definable topologies. This would include codecs too, since some vendors are making codecs with built in FW (maybe TI too ?). The machine level more represents the board HW topology and this should be derived from ACPI or DT.
I tend to agree. We should let each vendor provide their own topology information - if they need to do something with this (which does seem unlikely) system integrators should be on an equal footing with silicon vendors here, and we shouldn't be encouraging systems where we need per-board firmware definitions for the silicon components just because the board has differences.
On Tue, Apr 21, 2015 at 05:39:42PM +0100, Mark Brown wrote:
On Tue, Apr 21, 2015 at 04:23:51PM +0100, Liam Girdwood wrote:
On Tue, 2015-04-21 at 15:30 +0300, Peter Ujfalusi wrote:
We have discussed this with Liam in the past: in my view the DSP topology (or Dynamic FW) should be represented in the machine level and it would be the best if the same image could carry card level widgets routes and links. If you have big enough change in the FW and it's provided widgets/PCMs you would need separate machine driver or at least a way to have different set of machine level routes, widgets, links, etc for different DSP topology file.
The component level allows us to target the physical component devices that may have runtime definable topologies. This would include codecs too, since some vendors are making codecs with built in FW (maybe TI too ?). The machine level more represents the board HW topology and this should be derived from ACPI or DT.
I tend to agree. We should let each vendor provide their own topology information - if they need to do something with this (which does seem unlikely) system integrators should be on an equal footing with silicon vendors here, and we shouldn't be encouraging systems where we need per-board firmware definitions for the silicon components just because the board has differences.
One case which comes to my mind worth discussing here is PCMs.
We are trying to move FE dialink PCMs to toplogy as that is SW defined and IMO not linked to HW. So machine driver will not have dailinks to register at probe so we can't get sound card created (other info like widgets map can get get registered after probe though)
I was thinking that in core, a component should register with a toplogy bin as well. Core can load (or with a callback) the toplogy binary and then parse it for PCMs register them, and create sound card...
thoughts...
participants (4)
-
Liam Girdwood -
Mark Brown -
Peter Ujfalusi -
Vinod Koul