[Sound-open-firmware] Distribution of sof firmware and tplg files
Hi Liam, Pierre, Jaroslav,
I know there were a lot of discussions about distribution of SOF firmware and tplg files and I would like to have a conclusion here and also validate that I understood it correctly.
Simplified view:
* sof firmware will go to https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git * tplg files will go to https://github.com/alsa-project/alsa-topology-conf/pull/1
Is that correct, right?
I'm asking this in the context of integrating the binaries with Yocto where we have two options:
* build everything from sources (1) * pull the binaries from repos mentioned above (2)
Option (1) is a little bit trickier because of Xtensa toolchain license.
So, the easiest option for us wuold be (2).
Is there a versioning scheme for linux-firmware and alsa-topology?
Daniel.
On Tue, 2020-01-07 at 09:14 +0000, Daniel Baluta wrote:
Hi Liam, Pierre, Jaroslav,
I know there were a lot of discussions about distribution of SOF firmware and tplg files and I would like to have a conclusion here and also validate that I understood it correctly.
Simplified view:
* sof firmware will go to
https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
Yes, but I will remove the ldc files from FW and move to topology repo. Jaroslav, is I'm assuming it's OK to move the ldc files into the same repos as topology ?
* tplg files will go tohttps://github.com/alsa-project/alsa-topology-conf/pull/1
Is that correct, right?
Yes, being tracked here
https://github.com/thesofproject/sof/issues/2200
I'm asking this in the context of integrating the binaries with Yocto where we have two options:
* build everything from sources (1) * pull the binaries from repos mentioned above (2)Option (1) is a little bit trickier because of Xtensa toolchain license.
Yep, lets discuss this at next monthly call, Cadence may provide free versions of the tooling for certain ISA configs (they did this in the past for BYT)
So, the easiest option for us wuold be (2).
3) Default to option 2, but have setting to enable 1 for users with XCC.
Is there a versioning scheme for linux-firmware and alsa-topology?
I'm assuming alsa-topology may follow alsa versioning ? But maybe best for Jaroslav to answer here.
Linux FW repo just has some tags for dates, but seems to be a monthly tag.
Liam
Daniel.
Sound-open-firmware mailing list Sound-open-firmware@alsa-project.org https://mailman.alsa-project.org/mailman/listinfo/sound-open-firmware
--------------------------------------------------------------------- Intel Corporation (UK) Limited Registered No. 1134945 (England) Registered Office: Pipers Way, Swindon SN3 1RJ VAT No: 860 2173 47
This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
+ Tanu
On Tue, 2020-01-07 at 10:46 +0000, Liam Girdwood wrote:
Is there a versioning scheme for linux-firmware and alsa-topology?
I'm assuming alsa-topology may follow alsa versioning ? But maybe best for Jaroslav to answer here.
Linux FW repo just has some tags for dates, but seems to be a monthly tag.
Jaroslav, IIUC you also proposed having the FW in the same repo as UCM, topology. What's the advantage of this over linux-firmware for RPM/deb packagers ?
Btw, I'm open to whatever is easiest for everyone but I've never been a packager so I'm missing some background.
Thanks
Liam
Dne 07. 01. 20 v 16:30 Liam Girdwood napsal(a):
- Tanu
On Tue, 2020-01-07 at 10:46 +0000, Liam Girdwood wrote:
Is there a versioning scheme for linux-firmware and alsa-topology?
I'm assuming alsa-topology may follow alsa versioning ? But maybe best for Jaroslav to answer here.
Linux FW repo just has some tags for dates, but seems to be a monthly tag.
Jaroslav, IIUC you also proposed having the FW in the same repo as UCM, topology. What's the advantage of this over linux-firmware for RPM/deb packagers ?
I proposed to distribute the topology files with the firmware files. Thus put these non-user space things to linux-firmware and let UCM describe the rest.
The topology and firmware files are required to boot the hardware.
Pierre insists to distribute the topology files separately to allow customizations (not sure who are target users - OEMs?). My opinion is that if the auto-selection for any of firmware and topologies is missing on the kernel side, the topology separation is not required for the Linux mainstream. It would be sufficient to link to the sources in a documentation file in the linux-firmware tree. So the advanced users can work with it.
So my last advice was to push the topologies in the normalized .conf format to alsa-topology-conf repository and let the distributions to choose, if they'll put this to their linux-firmware packages or create a standalone package with topologies (which must be installed in the same way as linux-firmware anyway).
https://github.com/alsa-project/alsa-topology-conf/pull/1
For Fedora, I pack the firmware and topology files in the alsa-firmware package temporary, until we resolve this in upstream.
Btw, I'm open to whatever is easiest for everyone but I've never been a packager so I'm missing some background.
For the packager, the easiest method is to put everything except UCM to linux-firmware.
Jaroslav
Jaroslav, IIUC you also proposed having the FW in the same repo as UCM, topology. What's the advantage of this over linux-firmware for RPM/deb packagers ?
I proposed to distribute the topology files with the firmware files. Thus put these non-user space things to linux-firmware and let UCM describe the rest.
The topology and firmware files are required to boot the hardware.
The topology is a convenience tool, not a requirement. we've had multiple cases where the topology was embedded in the firmware itself to reduce boot time.
Pierre insists to distribute the topology files separately to allow customizations (not sure who are target users - OEMs?). My opinion is that if the auto-selection for any of firmware and topologies is missing on the kernel side, the topology separation is not required for the Linux mainstream. It would be sufficient to link to the sources in a documentation file in the linux-firmware tree. So the advanced users can work with it.
There is a rationale for not bundling firmware and topology together:
1. the firmware cannot be generated by the end-user in general (access to the xcc compiler and signature issues).
2. the topology files can be generated at will
3. the linux-firmware tree is supposed to be for binaries only. When we talked with linux-firmware maintainers several years ago, the recommendation was that everything that can be compiled/scripted is not welcome in that tree. IIRC we had the same feedback in one of the ALSA miniconf.
4. in the linux-firmware tree the expectation is that there are multiple versions of the firmware. This is useful for Intel, e.g. in some cases we had ABI issues with the CSME tools so it's very very interesting to ask users to try and boot a previous version. This is conversely not needed for topologies, there is no real link with the hardware or other low-level firmware.
5. the topology can contain default for volume levels and filters, and different users of the same hardware might have different preferences. I am not aware of the moral equivalent of /usr/local for firmware, so the user/distro would have to override the default files, or provide a kernel parameter to fetch the files from another location (already supported).
So my last advice was to push the topologies in the normalized .conf format to alsa-topology-conf repository and let the distributions to choose, if they'll put this to their linux-firmware packages or create a standalone package with topologies (which must be installed in the same way as linux-firmware anyway).
I think it's not sustainable to import .conf files, it'd be a better idea to move all M4 files into alsa-topology-conf and provide a script to generate topologies - provided that the results are the *same* as what gets tested by the SOF CI and Intel QA. We should not have multiple options for the topology generation, either we all normalize the intermediate .conf or we don't. Doing otherwise is a sure way of having "works for me" bug reports and confused users.
I will also assert that topology and UCM files should be released together, the former define stream numbers and control names that are used by the latter. If we don't release them together we'll be in trouble (e.g. if for the DMIC name changes).
Regards
-Pierre
Dne 07. 01. 20 v 17:46 Pierre-Louis Bossart napsal(a):
So my last advice was to push the topologies in the normalized .conf format to alsa-topology-conf repository and let the distributions to choose, if they'll put this to their linux-firmware packages or create a standalone package with topologies (which must be installed in the same way as linux-firmware anyway).
I think it's not sustainable to import .conf files, it'd be a better idea to move all M4 files into alsa-topology-conf and provide a script to generate topologies - provided that the results are the *same* as what gets tested by the SOF CI and Intel QA. We should not have multiple options for the topology generation, either we all normalize the intermediate .conf or we don't. Doing otherwise is a sure way of having "works for me" bug reports and confused users.
The .conf files can be very easily compared with the original source.
Basically, I really want to let things to move forward. Everything is better than to have the topology configuration in the big repo with all other things, so I can accept this, too.
I had to probably use my time to work on something different than libatopology, but at least I found several problems and did a lot of cleanups.
I will also assert that topology and UCM files should be released together, the former define stream numbers and control names that are used by the latter.
The packagers should ensure the version sync.
If we don't release them together we'll be in trouble (e.g. if forthe DMIC name changes).
We are in trouble anyway :-) The new name is not much better (index inside the control name not respecting the rules in control-names.rst - kernel documentation). But I know, someone missed to implement the control indexes in the whole ASoC tree ;-) SOC_SINGLE() etc... I'm talking about snd_kcontrol_new->index.
Jaroslav
+ Che, Rex and Hui from Canonical
On Tue, 2020-01-07 at 18:33 +0100, Jaroslav Kysela wrote:
Dne 07. 01. 20 v 17:46 Pierre-Louis Bossart napsal(a):
So my last advice was to push the topologies in the normalized .conf format to alsa-topology-conf repository and let the distributions to choose, if they'll put this to their linux-firmware packages or create a standalone package with topologies (which must be installed in the same way as linux-firmware anyway).
I think it's not sustainable to import .conf files, it'd be a better idea to move all M4 files into alsa-topology-conf and provide a script to generate topologies - provided that the results are the *same* as what gets tested by the SOF CI and Intel QA. We should not have multiple options for the topology generation, either we all normalize the intermediate .conf or we don't. Doing otherwise is a sure way of having "works for me" bug reports and confused users.
The .conf files can be very easily compared with the original source.
Basically, I really want to let things to move forward. Everything is better than to have the topology configuration in the big repo with all other things, so I can accept this, too.
I had to probably use my time to work on something different than libatopology, but at least I found several problems and did a lot of cleanups.
I will also assert that topology and UCM files should be released together, the former define stream numbers and control names that are used by the latter.
The packagers should ensure the version sync.
ok, I this should also include the ldc files. I will remove these from linux-firmware.
This would mean that:-
1) linux-firmware would just contain versioned firmware binaries. Soft link would link to latest version unless driver quirk or user (via module param) request earlier version.
2.1) alsa-ucm-conf would contain versioned UCMs
2.2) alsa-ucm-conf would contain versioned topology M4s (local make rule could build and install tplg binaries). Script could copy tagged M4 releases from SOF for convenience.
2.3) alsa-ucm-conf would contain LDC files. Copy script ? Probably better coming from CI as PR as part of a tagged release ?
Question for packagers, how will we version 2.1, 2.2 and 2.3 ? I would assume via git tag ?
If so how do we align alsa-ucm-conf release to kernel and SOF releases ? Do the distros keep manifest of "compatible" ingredient versions that would track this ? I'm guessing the distro maintainer manage this based on the version of kernel in the distro.
If this is acceptable I will push PRs for linux-firmware and alsa-ucm- conf
If we don't release them together we'll be in trouble (e.g. if forthe DMIC name changes).
We are in trouble anyway :-) The new name is not much better (index inside the control name not respecting the rules in control-names.rst - kernel documentation). But I know, someone missed to implement the control indexes in the whole ASoC tree ;-) SOC_SINGLE() etc... I'm talking about snd_kcontrol_new->index.
Oh, that's probably my bad from 15 years ago !
Liam
Jaroslav
ok, I this should also include the ldc files. I will remove these from linux-firmware.
This would mean that:-
- linux-firmware would just contain versioned firmware binaries. Soft
link would link to latest version unless driver quirk or user (via module param) request earlier version.
How would we map an .ldc file to the .ri file in linux-firmware?
the .ldc is generated at the same time as the .ri firmware, they go together. Distributing them through separate channels is a recipe for invalid traces. Murphy's Second Law is "if you provide people with means to break your solution they will"
Dne 10. 01. 20 v 18:09 Pierre-Louis Bossart napsal(a):
ok, I this should also include the ldc files. I will remove these from linux-firmware.
This would mean that:-
- linux-firmware would just contain versioned firmware binaries. Soft
link would link to latest version unless driver quirk or user (via module param) request earlier version.
How would we map an .ldc file to the .ri file in linux-firmware?
the .ldc is generated at the same time as the .ri firmware, they go together. Distributing them through separate channels is a recipe for invalid traces. Murphy's Second Law is "if you provide people with means to break your solution they will"
I'm not really convinced to handle .ldc in the other packages than the debugging ones. In my opinion, you should maintain the complete SOF binary firmware distribution in a separate package / tagged repository for all firmware / topology / .ldc files as the upstream reference and for the debugging. The ldc files do not belong to linux-firmware nor alsa-ucm-conf nor alsa-topology-conf.
https://github.com/thesofproject/sof/issues/2098
Also, if you like to release multiple versions of SOF firmware in the linux-firmware package, it should be properly documented (the purpose, what exact hardware is affected etc.). It is really difficult to have an orientation there (and we as the maintainers for various distributions should know what to do).
BTW: The SST firmware files are also affected with the lack of the documentation. I believe that only few people can tell us what firmware file belongs to the specific hardware / driver.
Jaroslav
I'm not really convinced to handle .ldc in the other packages than the debugging ones. In my opinion, you should maintain the complete SOF binary firmware distribution in a separate package / tagged repository for all firmware / topology / .ldc files as the upstream reference and for the debugging. The ldc files do not belong to linux-firmware nor alsa-ucm-conf nor alsa-topology-conf.
https://github.com/thesofproject/sof/issues/2098
Also, if you like to release multiple versions of SOF firmware in the linux-firmware package, it should be properly documented (the purpose, what exact hardware is affected etc.). It is really difficult to have an orientation there (and we as the maintainers for various distributions should know what to do).
BTW: The SST firmware files are also affected with the lack of the documentation. I believe that only few people can tell us what firmware file belongs to the specific hardware / driver.
Thanks for challenging us and picking our brains.
I must admit I blindly followed the tradition, without asking questions. linux-firmware was required in the past and we also wanted to use alsa-conf-ucm for the rest. So conceptually I was set on a 2-part release.
If it's acceptable to distributions and users, it's true that having a TBD one-stop shop for all required binary/configuration/debug files, along with release collateral, would be a lot simpler. No risk of different parts flowing independently in different packages, pull requests being delayed, etc. Why not indeed?
That's be a big decision though so we'd need to have consensus from others (Debian, Chrome, Yocto, etc).
Also I am not clear on what you'd want to do with UCM files, keep them in alsa-conf but with firmware/topology somewhere else?
-Pierre
Dne 11. 01. 20 v 0:15 Pierre-Louis Bossart napsal(a):
I'm not really convinced to handle .ldc in the other packages than the debugging ones. In my opinion, you should maintain the complete SOF binary firmware distribution in a separate package / tagged repository for all firmware / topology / .ldc files as the upstream reference and for the debugging. The ldc files do not belong to linux-firmware nor alsa-ucm-conf nor alsa-topology-conf.
https://github.com/thesofproject/sof/issues/2098
Also, if you like to release multiple versions of SOF firmware in the linux-firmware package, it should be properly documented (the purpose, what exact hardware is affected etc.). It is really difficult to have an orientation there (and we as the maintainers for various distributions should know what to do).
BTW: The SST firmware files are also affected with the lack of the documentation. I believe that only few people can tell us what firmware file belongs to the specific hardware / driver.
Thanks for challenging us and picking our brains.
I must admit I blindly followed the tradition, without asking questions. linux-firmware was required in the past and we also wanted to use alsa-conf-ucm for the rest. So conceptually I was set on a 2-part release.
If it's acceptable to distributions and users, it's true that having a TBD one-stop shop for all required binary/configuration/debug files, along with release collateral, would be a lot simpler. No risk of different parts flowing independently in different packages, pull requests being delayed, etc. Why not indeed?
That's be a big decision though so we'd need to have consensus from others (Debian, Chrome, Yocto, etc).
I mean to create "reference" repository with all tagged binaries / debuging files, so if we have a problem, we can look back to it and try to find the differences or debug the issue. I don't want to avoid the standard upstream sync.
linux-firmware:
I believe that the stable firmware files should be copied the linux-firmware repository without special firmware versions for the specific hardware or usage. Only the files which supports the "mainstream" hardware. If maintainers / users for the specific hardware cannot use the firmware files in the standard path used in the driver (auto-detection) they must modify the system configuration anyway (thus I don't see an issue to let them download the special firmware from the "reference" repository).
alsa-topology-conf:
I agreed to include SOF m4 files there. The files should be in sync with the linux-firmware updates.
alsa-ucm-conf:
Sync with the alsa-topology-conf updates. The ALSA release version is the link.
Also I am not clear on what you'd want to do with UCM files, keep them in alsa-conf but with firmware/topology somewhere else?
UCM is a bit abstraction and we can have more abstractions for the one kernel driver / interface. UCM should handle also multiple kernel / topology variants in my opinion. I would keep this as a separate interface.
The main problem seems that we have to deal with the multiple layers and you are afraid that we can have sync issues. I already proposed (I think in May last year) to have a way to determine the exact version used components (firmware, topology). Those values may be printed to the kernel log or exposed via sysfs / proc for the further analysis.
The first reported issues with the SOF driver/firmware show that the first question is what firmware / topology version we are using. Please, make this information visible. It would probably mean to put the SOF release version number to the topology binary files, too.
So basically, my proposal is:
1) create one source (tar ball file, repository tag) per SOF release - all things should be there (firmware, debug files, topologies) 2) push stable / tested mainstream firmware files to linux-firmware - licencing / copyright 3) push topology changes to alsa-topology-conf repo 4) propose UCM fixes to alsa-ucm-conf repo if required for the updated topology files
Jaroslav
-Pierre
So basically, my proposal is:
- create one source (tar ball file, repository tag) per SOF release
- all things should be there (firmware, debug files, topologies) 2) push stable / tested mainstream firmware files to linux-firmware - licencing / copyright 3) push topology changes to alsa-topology-conf repo 4) propose UCM fixes to alsa-ucm-conf repo if required for the updated topology files
I am sorry, you've lost me here.
If we push firmware through the linux-firmware tree, topology those alsa-topology-conf and UCM through alsa-ucm-conf, then who would be the users of the package 1)?
Conversely if people use item #1, then why should we care about 2) 3) and 4)?
I'd like to avoid duplication of work and potential misalignment between integration flows.
Dne 11. 01. 20 v 13:17 Pierre-Louis Bossart napsal(a):
So basically, my proposal is:
- create one source (tar ball file, repository tag) per SOF release
- all things should be there (firmware, debug files, topologies) 2) push stable / tested mainstream firmware files to linux-firmware - licencing / copyright 3) push topology changes to alsa-topology-conf repo 4) propose UCM fixes to alsa-ucm-conf repo if required for the updated topology files
I am sorry, you've lost me here.
If we push firmware through the linux-firmware tree, topology those alsa-topology-conf and UCM through alsa-ucm-conf, then who would be the users of the package 1)?
Conversely if people use item #1, then why should we care about 2) 3) and 4)?
I'd like to avoid duplication of work and potential misalignment between integration flows.
The all-in-one package is useful for:
1) debugging .ldc files - they don't belong to any of the upstream repos 2) SOF release - you have full control to put all things together and we can use this as a reference; eventually, we can update the community integration repos from this 3) extra firmware files (not used by default kernel configuration / code)
It does not mean that the stable code should not be pushed to the "integration" repositories (where the code is merged from the multiple sources). But in case of issues (debugging) or if you have special users (firmware for special hardware variants requiring the extra kernel parameters / setup), the users might look back to the all-in-one repo (SOF project release).
I think that this will work for all of us.
Jaroslav
The all-in-one package is useful for:
- debugging .ldc files - they don't belong to any of the upstream repos
I just want to clarify something on those files: Rather than adding strings to all our traces, when generating the firmware we strip them out to reduce the firmware footprint, and the actual strings are stored in the ldc. When you want to extract traces, the sof-logger tool substitutes the codes for human-readable strings. The ldc files only existing in relation with the initial firmware they were extracted from.
I don't see at all why these ldc files can't be distributed along with the firmware. In fact, we could change the firmware format and add them in a single file, which would be stripped during firmware download and you'd not even know about it...
- SOF release - you have full control to put all things together and
we can use this as a reference; eventually, we can update the community integration repos from this
So we are talking about multiple integration channels, that's not so good for us
- extra firmware files (not used by default kernel configuration / code)
We don't have such cases at the moment.
It does not mean that the stable code should not be pushed to the "integration" repositories (where the code is merged from the multiple sources). But in case of issues (debugging) or if you have special users (firmware for special hardware variants requiring the extra kernel parameters / setup), the users might look back to the all-in-one repo (SOF project release).
I would really like to have all .ldc distributed by default so that we can ask for traces without requiring any install shenanigans. People are able to use alsa-info.sh, and I consider the .ldc files as a extension helping provide DSP traces.
Dne 13. 01. 20 v 16:47 Pierre-Louis Bossart napsal(a):
The all-in-one package is useful for:
- debugging .ldc files - they don't belong to any of the upstream repos
I just want to clarify something on those files: Rather than adding strings to all our traces, when generating the firmware we strip them out to reduce the firmware footprint, and the actual strings are stored in the ldc. When you want to extract traces, the sof-logger tool substitutes the codes for human-readable strings. The ldc files only existing in relation with the initial firmware they were extracted from.
I don't see at all why these ldc files can't be distributed along with the firmware. In fact, we could change the firmware format and add them in a single file, which would be stripped during firmware download and you'd not even know about it...
I see, but why to carry files which are not required to boot linux in the linux-firmware? It's only an extra bloat which only few users will use. You can integrated it to one file, of course. But ... you removed this info for the exactly similar reason, don't you? The extra debugging information is not required for the standard runtime.
- SOF release - you have full control to put all things together and
we can use this as a reference; eventually, we can update the community integration repos from this
So we are talking about multiple integration channels, that's not so good for us
We always hit this. Think what's good for all users. It's easy to assign the .ldc file to the firmware file using some checksums or so. And, who are users of this? Few users who want to debug problem or maintainers who can point to the .ldc file. Also, if you have .ldc, you need to have the trace utility (binary). Perhaps, you can create an extra debugging package with all .ldc files for all released firmwares and find a way at runtime to determine the loaded firmware and select the proper .ldc automatically. It's just another hint.
- extra firmware files (not used by default kernel configuration / code)
We don't have such cases at the moment.
You do have. In the Liam's linux-firmware repository, there are several old or special 1.3 firmware files, which are not directly referred from the driver. The user must change the kernel configuration using the kernel / module parameters:
https://github.com/thesofproject/linux-firmware/commit/beff732e6642bd77dac5d...
Example: File: intel/sof/cml/intel/sof-cml-v1.3-0f73628.ri ????
It does not mean that the stable code should not be pushed to the "integration" repositories (where the code is merged from the multiple sources). But in case of issues (debugging) or if you have special users (firmware for special hardware variants requiring the extra kernel parameters / setup), the users might look back to the all-in-one repo (SOF project release).
I would really like to have all .ldc distributed by default so that we can ask for traces without requiring any install shenanigans. People are able to use alsa-info.sh, and I consider the .ldc files as a extension helping provide DSP traces.
Yes, but it's debugging and there is no standard for this. The extra debugging package with .ldc and the trace utility for SOF might be really a good thing.
Jaroslav
On Mon, 2020-01-13 at 17:07 +0100, Jaroslav Kysela wrote:
Dne 13. 01. 20 v 16:47 Pierre-Louis Bossart napsal(a):
The all-in-one package is useful for:
- debugging .ldc files - they don't belong to any of the
upstream repos
I just want to clarify something on those files: Rather than adding strings to all our traces, when generating the firmware we strip them out to reduce the firmware footprint, and the actual strings are stored in the ldc. When you want to extract traces, the sof-logger tool substitutes the codes for human-readable strings. The ldc files only existing in relation with the initial firmware they were extracted from.
I don't see at all why these ldc files can't be distributed along with the firmware. In fact, we could change the firmware format and add them in a single file, which would be stripped during firmware download and you'd not even know about it...
I see, but why to carry files which are not required to boot linux in the linux-firmware? It's only an extra bloat which only few users will use. You can integrated it to one file, of course. But ... you removed this info for the exactly similar reason, don't you? The extra debugging information is not required for the standard runtime.
- SOF release - you have full control to put all things together
and we can use this as a reference; eventually, we can update the community integration repos from this
So we are talking about multiple integration channels, that's not so good for us
We always hit this. Think what's good for all users. It's easy to assign the .ldc file to the firmware file using some checksums or so. And, who are users of this? Few users who want to debug problem or maintainers who can point to the .ldc file. Also, if you have .ldc, you need to have the trace utility (binary). Perhaps, you can create an extra debugging package with all .ldc files for all released firmwares and find a way at runtime to determine the loaded firmware and select the proper .ldc automatically. It's just another hint.
Ok, so if we can place the ldc files in alsa-ucm-conf (installed by default ?) alongside some debug script (to gather FW oops data using the ldc data) then we can retain the debug functionality. The intention is to ask users to run the debug script upon DSP crash and then paste the results in github (script still has to be written though).
- extra firmware files (not used by default kernel configuration
/ code)
We don't have such cases at the moment.
You do have. In the Liam's linux-firmware repository, there are several old or special 1.3 firmware files, which are not directly referred from the driver. The user must change the kernel configuration using the kernel / module parameters:
We need these for certain devices. Canonical guys on the CC need these atm, until they can update to newer version like the upcoming v1.4.2.
https://github.com/thesofproject/linux-firmware/commit/beff732e6642bd77dac5d...
Example: File: intel/sof/cml/intel/sof-cml-v1.3-0f73628.ri ????
It does not mean that the stable code should not be pushed to the "integration" repositories (where the code is merged from the multiple sources). But in case of issues (debugging) or if you have special users (firmware for special hardware variants requiring the extra kernel parameters / setup), the users might look back to the all-in-one repo (SOF project release).
I would really like to have all .ldc distributed by default so that we can ask for traces without requiring any install shenanigans. People are able to use alsa-info.sh, and I consider the .ldc files as a extension helping provide DSP traces.
Yes, but it's debugging and there is no standard for this. The extra debugging package with .ldc and the trace utility for SOF might be really a good thing.
It sounds like we are all aligned on this objective, albeit using different repos. Are there any objections to ldc files in alsa-ucm-conf providing a debug script can find the correct ldc file at runtime ?
Thanks
Liam
Jaroslav
On Wed, Jan 15, 2020 at 1:01 PM Liam Girdwood liam.r.girdwood@linux.intel.com wrote:
On Mon, 2020-01-13 at 17:07 +0100, Jaroslav Kysela wrote:
Dne 13. 01. 20 v 16:47 Pierre-Louis Bossart napsal(a):
The all-in-one package is useful for:
- debugging .ldc files - they don't belong to any of the
upstream repos
I just want to clarify something on those files: Rather than adding strings to all our traces, when generating the firmware we strip them out to reduce the firmware footprint, and the actual strings are stored in the ldc. When you want to extract traces, the sof-logger tool substitutes the codes for human-readable strings. The ldc files only existing in relation with the initial firmware they were extracted from.
I don't see at all why these ldc files can't be distributed along with the firmware. In fact, we could change the firmware format and add them in a single file, which would be stripped during firmware download and you'd not even know about it...
I see, but why to carry files which are not required to boot linux in the linux-firmware? It's only an extra bloat which only few users will use. You can integrated it to one file, of course. But ... you removed this info for the exactly similar reason, don't you? The extra debugging information is not required for the standard runtime.
- SOF release - you have full control to put all things together
and we can use this as a reference; eventually, we can update the community integration repos from this
So we are talking about multiple integration channels, that's not so good for us
We always hit this. Think what's good for all users. It's easy to assign the .ldc file to the firmware file using some checksums or so. And, who are users of this? Few users who want to debug problem or maintainers who can point to the .ldc file. Also, if you have .ldc, you need to have the trace utility (binary). Perhaps, you can create an extra debugging package with all .ldc files for all released firmwares and find a way at runtime to determine the loaded firmware and select the proper .ldc automatically. It's just another hint.
Ok, so if we can place the ldc files in alsa-ucm-conf (installed by default ?) alongside some debug script (to gather FW oops data using the ldc data) then we can retain the debug functionality. The intention is to ask users to run the debug script upon DSP crash and then paste the results in github (script still has to be written though).
- extra firmware files (not used by default kernel configuration
/ code)
We don't have such cases at the moment.
You do have. In the Liam's linux-firmware repository, there are several old or special 1.3 firmware files, which are not directly referred from the driver. The user must change the kernel configuration using the kernel / module parameters:
We need these for certain devices. Canonical guys on the CC need these atm, until they can update to newer version like the upcoming v1.4.2.
https://github.com/thesofproject/linux-firmware/commit/beff732e6642bd77dac5d...
Example: File: intel/sof/cml/intel/sof-cml-v1.3-0f73628.ri ????
It does not mean that the stable code should not be pushed to the "integration" repositories (where the code is merged from the multiple sources). But in case of issues (debugging) or if you have special users (firmware for special hardware variants requiring the extra kernel parameters / setup), the users might look back to the all-in-one repo (SOF project release).
I would really like to have all .ldc distributed by default so that we can ask for traces without requiring any install shenanigans. People are able to use alsa-info.sh, and I consider the .ldc files as a extension helping provide DSP traces.
Yes, but it's debugging and there is no standard for this. The extra debugging package with .ldc and the trace utility for SOF might be really a good thing.
It sounds like we are all aligned on this objective, albeit using different repos. Are there any objections to ldc files in alsa-ucm-conf providing a debug script can find the correct ldc file at runtime ?
In my opionion the .ldc should always stay near the FW file, but we will go with whatever you decide.
On our side we are using Yocto and we plan to build everything from sources, when the Yocto image is build.
Dne 15. 01. 20 v 12:13 Daniel Baluta napsal(a):
On Wed, Jan 15, 2020 at 1:01 PM Liam Girdwood liam.r.girdwood@linux.intel.com wrote:
On Mon, 2020-01-13 at 17:07 +0100, Jaroslav Kysela wrote:
Dne 13. 01. 20 v 16:47 Pierre-Louis Bossart napsal(a):
The all-in-one package is useful for:
- debugging .ldc files - they don't belong to any of the
upstream repos
I just want to clarify something on those files: Rather than adding strings to all our traces, when generating the firmware we strip them out to reduce the firmware footprint, and the actual strings are stored in the ldc. When you want to extract traces, the sof-logger tool substitutes the codes for human-readable strings. The ldc files only existing in relation with the initial firmware they were extracted from.
I don't see at all why these ldc files can't be distributed along with the firmware. In fact, we could change the firmware format and add them in a single file, which would be stripped during firmware download and you'd not even know about it...
I see, but why to carry files which are not required to boot linux in the linux-firmware? It's only an extra bloat which only few users will use. You can integrated it to one file, of course. But ... you removed this info for the exactly similar reason, don't you? The extra debugging information is not required for the standard runtime.
- SOF release - you have full control to put all things together
and we can use this as a reference; eventually, we can update the community integration repos from this
So we are talking about multiple integration channels, that's not so good for us
We always hit this. Think what's good for all users. It's easy to assign the .ldc file to the firmware file using some checksums or so. And, who are users of this? Few users who want to debug problem or maintainers who can point to the .ldc file. Also, if you have .ldc, you need to have the trace utility (binary). Perhaps, you can create an extra debugging package with all .ldc files for all released firmwares and find a way at runtime to determine the loaded firmware and select the proper .ldc automatically. It's just another hint.
Ok, so if we can place the ldc files in alsa-ucm-conf (installed by default ?) alongside some debug script (to gather FW oops data using the ldc data) then we can retain the debug functionality. The intention is to ask users to run the debug script upon DSP crash and then paste the results in github (script still has to be written though).
- extra firmware files (not used by default kernel configuration
/ code)
We don't have such cases at the moment.
You do have. In the Liam's linux-firmware repository, there are several old or special 1.3 firmware files, which are not directly referred from the driver. The user must change the kernel configuration using the kernel / module parameters:
We need these for certain devices. Canonical guys on the CC need these atm, until they can update to newer version like the upcoming v1.4.2.
https://github.com/thesofproject/linux-firmware/commit/beff732e6642bd77dac5d...
Example: File: intel/sof/cml/intel/sof-cml-v1.3-0f73628.ri ????
It does not mean that the stable code should not be pushed to the "integration" repositories (where the code is merged from the multiple sources). But in case of issues (debugging) or if you have special users (firmware for special hardware variants requiring the extra kernel parameters / setup), the users might look back to the all-in-one repo (SOF project release).
I would really like to have all .ldc distributed by default so that we can ask for traces without requiring any install shenanigans. People are able to use alsa-info.sh, and I consider the .ldc files as a extension helping provide DSP traces.
Yes, but it's debugging and there is no standard for this. The extra debugging package with .ldc and the trace utility for SOF might be really a good thing.
It sounds like we are all aligned on this objective, albeit using different repos. Are there any objections to ldc files in alsa-ucm-conf providing a debug script can find the correct ldc file at runtime ?
In my opionion the .ldc should always stay near the FW file, but we will go with whatever you decide.
On our side we are using Yocto and we plan to build everything from sources, when the Yocto image is build.
If the world is perfect, we can compile everything from the sources. But we need to deal with the signed firmware files compiled using the special compilers. We cannot touch them. There is no way to create the firmware (and .ldc files) for the notebooks with the Intel hardware.
Jaroslav
Dne 15. 01. 20 v 12:00 Liam Girdwood napsal(a):
On Mon, 2020-01-13 at 17:07 +0100, Jaroslav Kysela wrote:
Dne 13. 01. 20 v 16:47 Pierre-Louis Bossart napsal(a):
The all-in-one package is useful for:
- debugging .ldc files - they don't belong to any of the
upstream repos
I just want to clarify something on those files: Rather than adding strings to all our traces, when generating the firmware we strip them out to reduce the firmware footprint, and the actual strings are stored in the ldc. When you want to extract traces, the sof-logger tool substitutes the codes for human-readable strings. The ldc files only existing in relation with the initial firmware they were extracted from.
I don't see at all why these ldc files can't be distributed along with the firmware. In fact, we could change the firmware format and add them in a single file, which would be stripped during firmware download and you'd not even know about it...
I see, but why to carry files which are not required to boot linux in the linux-firmware? It's only an extra bloat which only few users will use. You can integrated it to one file, of course. But ... you removed this info for the exactly similar reason, don't you? The extra debugging information is not required for the standard runtime.
- SOF release - you have full control to put all things together
and we can use this as a reference; eventually, we can update the community integration repos from this
So we are talking about multiple integration channels, that's not so good for us
We always hit this. Think what's good for all users. It's easy to assign the .ldc file to the firmware file using some checksums or so. And, who are users of this? Few users who want to debug problem or maintainers who can point to the .ldc file. Also, if you have .ldc, you need to have the trace utility (binary). Perhaps, you can create an extra debugging package with all .ldc files for all released firmwares and find a way at runtime to determine the loaded firmware and select the proper .ldc automatically. It's just another hint.
Ok, so if we can place the ldc files in alsa-ucm-conf (installed by default ?) alongside some debug script (to gather FW oops data using the ldc data) then we can retain the debug functionality. The intention is to ask users to run the debug script upon DSP crash and then paste the results in github (script still has to be written though).
- extra firmware files (not used by default kernel configuration
/ code)
We don't have such cases at the moment.
You do have. In the Liam's linux-firmware repository, there are several old or special 1.3 firmware files, which are not directly referred from the driver. The user must change the kernel configuration using the kernel / module parameters:
We need these for certain devices. Canonical guys on the CC need these atm, until they can update to newer version like the upcoming v1.4.2.
https://github.com/thesofproject/linux-firmware/commit/beff732e6642bd77dac5d...
Example: File: intel/sof/cml/intel/sof-cml-v1.3-0f73628.ri ????
It does not mean that the stable code should not be pushed to the "integration" repositories (where the code is merged from the multiple sources). But in case of issues (debugging) or if you have special users (firmware for special hardware variants requiring the extra kernel parameters / setup), the users might look back to the all-in-one repo (SOF project release).
I would really like to have all .ldc distributed by default so that we can ask for traces without requiring any install shenanigans. People are able to use alsa-info.sh, and I consider the .ldc files as a extension helping provide DSP traces.
Yes, but it's debugging and there is no standard for this. The extra debugging package with .ldc and the trace utility for SOF might be really a good thing.
It sounds like we are all aligned on this objective, albeit using different repos. Are there any objections to ldc files in alsa-ucm-conf providing a debug script can find the correct ldc file at runtime ?
As a packager, I would like to have the debug info and tools separate. We can ask user to install the debug package and the extra tools on demand.
This is very specific to SOF project.
So just split the tools and put all .ldc files for all released firmware files to one package maintained under the SOF project. The you can trace the kernel version / sysfs module parameters to obtain the firmware path and use sha256sum on the loaded firmware file (or another hash) to find this appropriate .ldc file. Or the kernel driver may expose the hash / used firmware file path via sysfs directly. It may be handy. The github will do packaging for you for free.
My proposal is:
1) one repository with all things linked through the annotate tags (release) - firmware binary - .ldc file for the firmware - topology binary 2) sof-debug package with all .ldc files and tools to extract the traces, other debug tools 3) linux-firmware updates (without any special cases) 4) alsa-topology-conf updates (.m4 files)
Think about this in this way:
- release everything properly linked inside the SOF project (user ready binaries, no compilations) - update the upstream repositories with only appropriate pieces
The upstream packages merge the code from multiple sources. The special cases are not wanted too much.
Jaroslav
It sounds like we are all aligned on this objective, albeit using different repos. Are there any objections to ldc files in alsa-ucm-conf providing a debug script can find the correct ldc file at runtime ?
As a packager, I would like to have the debug info and tools separate. We can ask user to install the debug package and the extra tools on demand.
That's not possible in all cases to have an on-demand install. Yocto and others in the embedded world create one image that needs to contain everything at flash time. Granted there may be engineering and user releases, but no everyone has the flexibility to do an rpm/apt install from a command line.
Dne 15. 01. 20 v 17:15 Pierre-Louis Bossart napsal(a):
It sounds like we are all aligned on this objective, albeit using different repos. Are there any objections to ldc files in alsa-ucm-conf providing a debug script can find the correct ldc file at runtime ?
As a packager, I would like to have the debug info and tools separate. We can ask user to install the debug package and the extra tools on demand.
That's not possible in all cases to have an on-demand install. Yocto and others in the embedded world create one image that needs to contain everything at flash time. Granted there may be engineering and user releases, but no everyone has the flexibility to do an rpm/apt install from a command line.
But it's distro decision, isn't? If you bundle .ldc files forcefully, it's not a good way in my eyes. We should pick it depending on the use. Also, flash images are build for one specific device, so there's no issue to bundle one .ldc with tools, if the packager decides. We manage universal distros and the debug symbols should be optional and on demand.
Jaroslav
It sounds like we are all aligned on this objective, albeit using different repos. Are there any objections to ldc files in alsa-ucm-conf providing a debug script can find the correct ldc file at runtime ?
As a packager, I would like to have the debug info and tools separate. We can ask user to install the debug package and the extra tools on demand.
That's not possible in all cases to have an on-demand install. Yocto and others in the embedded world create one image that needs to contain everything at flash time. Granted there may be engineering and user releases, but no everyone has the flexibility to do an rpm/apt install from a command line.
But it's distro decision, isn't? If you bundle .ldc files forcefully, it's not a good way in my eyes. We should pick it depending on the use. Also, flash images are build for one specific device, so there's no issue to bundle one .ldc with tools, if the packager decides. We manage universal distros and the debug symbols should be optional and on demand.
Absolutely it's a distro decision, but your wording suggested otherwise.
I am sorry but I still see obscure points in your proposal:
1) one repository with all things linked through the annotate tags (release) - firmware binary - .ldc file for the firmware - topology binary 2) sof-debug package with all .ldc files and tools to extract the traces, other debug tools
<< why is this needed, if we already have 1)? As mentioned above, are we talking about a repo or a package, which are different things.
3) linux-firmware updates (without any special cases)
<< would you use this, or pick firmware from 1)? And how would you know if you need to take 'special cases' or not?
4) alsa-topology-conf updates (.m4 files)
Dne 15. 01. 20 v 18:08 Pierre-Louis Bossart napsal(a):
It sounds like we are all aligned on this objective, albeit using different repos. Are there any objections to ldc files in alsa-ucm-conf providing a debug script can find the correct ldc file at runtime ?
As a packager, I would like to have the debug info and tools separate. We can ask user to install the debug package and the extra tools on demand.
That's not possible in all cases to have an on-demand install. Yocto and others in the embedded world create one image that needs to contain everything at flash time. Granted there may be engineering and user releases, but no everyone has the flexibility to do an rpm/apt install from a command line.
But it's distro decision, isn't? If you bundle .ldc files forcefully, it's not a good way in my eyes. We should pick it depending on the use. Also, flash images are build for one specific device, so there's no issue to bundle one .ldc with tools, if the packager decides. We manage universal distros and the debug symbols should be optional and on demand.
Absolutely it's a distro decision, but your wording suggested otherwise.
I am sorry but I still see obscure points in your proposal:
- one repository with all things linked through the annotate tags (release)
- firmware binary
- .ldc file for the firmware
- topology binary
- sof-debug package with all .ldc files and tools to extract the traces, other debug tools
<< why is this needed, if we already have 1)? As mentioned above, are we talking about a repo or a package, which are different things.
Because we need something for the end users. Also, it helps packagers. If you don't need an easy debug feedback or you are ready to point users to the .ldc files from the reference repository, we are all fine. It was just an idea.
- linux-firmware updates (without any special cases)
<< would you use this, or pick firmware from 1)? And how would you know if you need to take 'special cases' or not?
My opinion is that any special firmware file which requires additional kernel configuration using the kernel config or kernel parameters is not required to be distributed in linux-firmware. If someone wants it, it can picked from the reference repository. I assume that only the last released SOF firmware will be available via linux-firmware.
Jaroslav
-----Original Message----- From: Sound-open-firmware <sound-open-firmware-bounces@alsa- project.org> On Behalf Of Pierre-Louis Bossart Sent: Monday, January 13, 2020 10:47 AM
The all-in-one package is useful for:
- debugging .ldc files - they don't belong to any of the upstream repos
I just want to clarify something on those files: Rather than adding strings to all our traces, when generating the firmware we strip them out to reduce the firmware footprint, and the actual strings are stored in the ldc. When you want to extract traces, the sof-logger tool substitutes the codes for human-readable strings. The ldc files only existing in relation with the initial firmware they were extracted from.
I don't see at all why these ldc files can't be distributed along with the firmware. In fact, we could change the firmware format and add them in a single file, which would be stripped during firmware download and you'd not even know about it...
- SOF release - you have full control to put all things together and
we can use this as a reference; eventually, we can update the community integration repos from this
So we are talking about multiple integration channels, that's not so good for us
- extra firmware files (not used by default kernel configuration / code)
We don't have such cases at the moment.
It does not mean that the stable code should not be pushed to the "integration" repositories (where the code is merged from the multiple sources). But in case of issues (debugging) or if you have special users (firmware for special hardware variants requiring the extra kernel parameters / setup), the users might look back to the all-in-one repo (SOF project release).
I would really like to have all .ldc distributed by default so that we can ask for traces without requiring any install shenanigans. People are able to use alsa-info.sh, and I consider the .ldc files as a extension helping provide DSP traces.
Hi all,
I've been following this thread with interest and thank you all for your efforts on working on this. Did we reach consensus on the solution? If we did I think I missed it. Please let me know.
We have a lot of users waiting for this and I know I'll be working with the distro's to get this rolled out when it's available. If there's anything I can do to help let me know - I don't have enough expertise to comment on the solution so I've been keeping quiet 😊.
Thanks Mark
On Thu, 2020-01-23 at 16:07 +0000, Mark Pearson wrote:
-----Original Message----- From: Sound-open-firmware <sound-open-firmware-bounces@alsa- project.org> On Behalf Of Pierre-Louis Bossart Sent: Monday, January 13, 2020 10:47 AM
The all-in-one package is useful for:
- debugging .ldc files - they don't belong to any of the
upstream repos
I just want to clarify something on those files: Rather than adding strings to all our traces, when generating the firmware we strip them out to reduce the firmware footprint, and the actual strings are stored in the ldc. When you want to extract traces, the sof-logger tool substitutes the codes for human-readable strings. The ldc files only existing in relation with the initial firmware they were extracted from.
I don't see at all why these ldc files can't be distributed along with the firmware. In fact, we could change the firmware format and add them in a single file, which would be stripped during firmware download and you'd not even know about it...
- SOF release - you have full control to put all things together
and we can use this as a reference; eventually, we can update the community integration repos from this
So we are talking about multiple integration channels, that's not so good for us
- extra firmware files (not used by default kernel configuration
/ code)
We don't have such cases at the moment.
It does not mean that the stable code should not be pushed to the "integration" repositories (where the code is merged from the multiple sources). But in case of issues (debugging) or if you have special users (firmware for special hardware variants requiring the extra kernel parameters / setup), the users might look back to the all-in-one repo (SOF project release).
I would really like to have all .ldc distributed by default so that we can ask for traces without requiring any install shenanigans. People are able to use alsa-info.sh, and I consider the .ldc files as a extension helping provide DSP traces.
Hi all,
I've been following this thread with interest and thank you all for your efforts on working on this. Did we reach consensus on the solution? If we did I think I missed it. Please let me know.
I hope so. The main contention point is the home for .ldc files and probably the topology binaries.
We have a lot of users waiting for this and I know I'll be working with the distro's to get this rolled out when it's available. If there's anything I can do to help let me know - I don't have enough expertise to comment on the solution so I've been keeping quiet 😊.
There was a SOF TSC meeting last week and the outcome was to create a separate "sof-bin" repo to store the binary outputs of the SOF build processes. This would include the FW, .ldc files and binary topologies.
The part that is not clear to me (since I'm not a packager) is how to add the necessary packaging infrastructure in this repos to create rpm and deb files.
I would imagine we would want to create 2 packaging targets
1) sof-bin (fw binaries and topology binaries)
2) sof-bin-dbg (ldc files).
I hope this proposal would also be acceptable to Jaroslav ?
I can create a repo with the correct binaries but would need help from Jaroslav, Tanu or yourself getting the packing added.
Thanks
Liam
Thanks Mark
-----Original Message----- From: Liam Girdwood liam.r.girdwood@linux.intel.com Sent: Monday, January 27, 2020 10:29 AM
There was a SOF TSC meeting last week and the outcome was to create a separate "sof-bin" repo to store the binary outputs of the SOF build processes. This would include the FW, .ldc files and binary topologies.
The part that is not clear to me (since I'm not a packager) is how to add the necessary packaging infrastructure in this repos to create rpm and deb files.
I would imagine we would want to create 2 packaging targets
sof-bin (fw binaries and topology binaries)
sof-bin-dbg (ldc files).
I hope this proposal would also be acceptable to Jaroslav ?
I can create a repo with the correct binaries but would need help from Jaroslav, Tanu or yourself getting the packing added.
Thanks
Liam
Great - thanks for the update. Sounds like a good solution to me.
I'm not a packager either (it's on my todo list to learn about and see if I can get involved with) but I can definitely try and help out working with the distro's to get this rolled out once it is all available. Keep me posted of how this goes.
Mark
On Mon, 2020-01-27 at 15:28 +0000, Liam Girdwood wrote:
On Thu, 2020-01-23 at 16:07 +0000, Mark Pearson wrote:
Hi all,
I've been following this thread with interest and thank you all for your efforts on working on this. Did we reach consensus on the solution? If we did I think I missed it. Please let me know.
I hope so. The main contention point is the home for .ldc files and probably the topology binaries.
We have a lot of users waiting for this and I know I'll be working with the distro's to get this rolled out when it's available. If there's anything I can do to help let me know - I don't have enough expertise to comment on the solution so I've been keeping quiet 😊.
There was a SOF TSC meeting last week and the outcome was to create a separate "sof-bin" repo to store the binary outputs of the SOF build processes. This would include the FW, .ldc files and binary topologies.
The part that is not clear to me (since I'm not a packager) is how to add the necessary packaging infrastructure in this repos to create rpm and deb files.
I would imagine we would want to create 2 packaging targets
sof-bin (fw binaries and topology binaries)
sof-bin-dbg (ldc files).
I hope this proposal would also be acceptable to Jaroslav ?
I can create a repo with the correct binaries but would need help from Jaroslav, Tanu or yourself getting the packing added.
Sorry for the slow reply. It sounds like you want to add packaging to the upstream repo, but why? Distributions can and will maintain the packaging bits themselves.
I maintain the ALSA recipes in the OpenEmbedded core layer (which is used by Yocto), including alsa-topology-conf and alsa-ucm-conf, and I guess I could present my thoughts here for the curious.
Currently SOF isn't packaged in OpenEmbedded. My understanding is that at least currently SOF is only used on Intel platforms, so if someone is going to package SOF for OpenEmbedded, it will probably be via the "meta-intel" layer, which I'm not involved with. I saw a message from Daniel Baluta in this thread, and he said that he's working on Yocto, but I'm not familiar with his work, is it for the meta-intel layer or something else?
I got the impression that the SOF firmware will get distributed also via the linux-firmware repo, and I'm a bit worried about a dependency between linux-firmware and alsa-topology-conf (and alsa-ucm-conf?). The linux-firmware recipe isn't updated in sync with the ALSA recipes, is that going to cause problems? Can an update in linux-firmware cause breakage if alsa-topology-conf isn't updated at the same time, or vice versa? If so, maybe I should push the alsa-topology-conf (and alsa-ucm- conf?) maintenance to the linux-firmware maintainer or otherwise make sure that updates are done in sync. I hope the kernel itself isn't another component in the chain that can cause breakages if not kept in sync with all the other components.
On Thu, Feb 6, 2020 at 11:38 AM Tanu Kaskinen tanuk@iki.fi wrote:
On Mon, 2020-01-27 at 15:28 +0000, Liam Girdwood wrote:
On Thu, 2020-01-23 at 16:07 +0000, Mark Pearson wrote:
Hi all,
I've been following this thread with interest and thank you all for your efforts on working on this. Did we reach consensus on the solution? If we did I think I missed it. Please let me know.
I hope so. The main contention point is the home for .ldc files and probably the topology binaries.
We have a lot of users waiting for this and I know I'll be working with the distro's to get this rolled out when it's available. If there's anything I can do to help let me know - I don't have enough expertise to comment on the solution so I've been keeping quiet .
There was a SOF TSC meeting last week and the outcome was to create a separate "sof-bin" repo to store the binary outputs of the SOF build processes. This would include the FW, .ldc files and binary topologies.
The part that is not clear to me (since I'm not a packager) is how to add the necessary packaging infrastructure in this repos to create rpm and deb files.
I would imagine we would want to create 2 packaging targets
sof-bin (fw binaries and topology binaries)
sof-bin-dbg (ldc files).
I hope this proposal would also be acceptable to Jaroslav ?
I can create a repo with the correct binaries but would need help from Jaroslav, Tanu or yourself getting the packing added.
Sorry for the slow reply. It sounds like you want to add packaging to the upstream repo, but why? Distributions can and will maintain the packaging bits themselves.
I maintain the ALSA recipes in the OpenEmbedded core layer (which is used by Yocto), including alsa-topology-conf and alsa-ucm-conf, and I guess I could present my thoughts here for the curious.
Thanks for jumping in. I work at NXP and we enable SOF on i.MX8 (arm based boards) and we use Yocto for Linux distro.
Currently SOF isn't packaged in OpenEmbedded. My understanding is that at least currently SOF is only used on Intel platforms, so if someone is going to package SOF for OpenEmbedded, it will probably be via the "meta-intel" layer, which I'm not involved with. I saw a message from Daniel Baluta in this thread, and he said that he's working on Yocto, but I'm not familiar with his work, is it for the meta-intel layer or something else?
Partially true. SOF is *currently* supported only on Intel platforms, but this will change very soon.
Supporting SOF in Yocto via "meta-intel" layer doesn't seems what we want for NXP. I think the better solution would be to support it via meta-freescale layer.
Cc-ing NXP Yocto people, Mihai and Lauren.
I got the impression that the SOF firmware will get distributed also via the linux-firmware repo, and I'm a bit worried about a dependency between linux-firmware and alsa-topology-conf (and alsa-ucm-conf?). The linux-firmware recipe isn't updated in sync with the ALSA recipes, is that going to cause problems? Can an update in linux-firmware cause breakage if alsa-topology-conf isn't updated at the same time, or vice versa? If so, maybe I should push the alsa-topology-conf (and alsa-ucm- conf?) maintenance to the linux-firmware maintainer or otherwise make sure that updates are done in sync. I hope the kernel itself isn't another component in the chain that can cause breakages if not kept in sync with all the other components.
Dne 27. 01. 20 v 16:28 Liam Girdwood napsal(a):
On Thu, 2020-01-23 at 16:07 +0000, Mark Pearson wrote:
-----Original Message----- From: Sound-open-firmware <sound-open-firmware-bounces@alsa- project.org> On Behalf Of Pierre-Louis Bossart Sent: Monday, January 13, 2020 10:47 AM
The all-in-one package is useful for:
- debugging .ldc files - they don't belong to any of the
upstream repos
I just want to clarify something on those files: Rather than adding strings to all our traces, when generating the firmware we strip them out to reduce the firmware footprint, and the actual strings are stored in the ldc. When you want to extract traces, the sof-logger tool substitutes the codes for human-readable strings. The ldc files only existing in relation with the initial firmware they were extracted from.
I don't see at all why these ldc files can't be distributed along with the firmware. In fact, we could change the firmware format and add them in a single file, which would be stripped during firmware download and you'd not even know about it...
- SOF release - you have full control to put all things together
and we can use this as a reference; eventually, we can update the community integration repos from this
So we are talking about multiple integration channels, that's not so good for us
- extra firmware files (not used by default kernel configuration
/ code)
We don't have such cases at the moment.
It does not mean that the stable code should not be pushed to the "integration" repositories (where the code is merged from the multiple sources). But in case of issues (debugging) or if you have special users (firmware for special hardware variants requiring the extra kernel parameters / setup), the users might look back to the all-in-one repo (SOF project release).
I would really like to have all .ldc distributed by default so that we can ask for traces without requiring any install shenanigans. People are able to use alsa-info.sh, and I consider the .ldc files as a extension helping provide DSP traces.
Hi all,
I've been following this thread with interest and thank you all for your efforts on working on this. Did we reach consensus on the solution? If we did I think I missed it. Please let me know.
I hope so. The main contention point is the home for .ldc files and probably the topology binaries.
We have a lot of users waiting for this and I know I'll be working with the distro's to get this rolled out when it's available. If there's anything I can do to help let me know - I don't have enough expertise to comment on the solution so I've been keeping quiet 😊.
There was a SOF TSC meeting last week and the outcome was to create a separate "sof-bin" repo to store the binary outputs of the SOF build processes. This would include the FW, .ldc files and binary topologies.
The part that is not clear to me (since I'm not a packager) is how to add the necessary packaging infrastructure in this repos to create rpm and deb files.
I would imagine we would want to create 2 packaging targets
sof-bin (fw binaries and topology binaries)
sof-bin-dbg (ldc files).
I hope this proposal would also be acceptable to Jaroslav ?
I can create a repo with the correct binaries but would need help from Jaroslav, Tanu or yourself getting the packing added.
I think that you can just pack everything together in your repo. We can pick / split things in the packaging process.
My reasoning is only against the .ldc files in the linux-firmware repo.
Thanks, Jaroslav
participants (8)
-
Daniel Baluta -
Daniel Baluta -
Jaroslav Kysela -
Liam Girdwood -
Liam Girdwood -
Mark Pearson -
Pierre-Louis Bossart -
Tanu Kaskinen