[Sound-open-firmware] Distribution of sof firmware and tplg files

[Jaroslav Kysela]

Dne 15. 01. 20 v 12:00 Liam Girdwood napsal(a):
> On Mon, 2020-01-13 at 17:07 +0100, Jaroslav Kysela wrote:
>> Dne 13. 01. 20 v 16:47 Pierre-Louis Bossart napsal(a):
>>>
>>>
>>>> The all-in-one package is useful for:
>>>>
>>>> 1) debugging .ldc files - they don't belong to any of the
>>>> upstream repos
>>>
>>> I just want to clarify something on those files: Rather than adding
>>> strings to all our traces, when generating the firmware we strip
>>> them
>>> out to reduce the firmware footprint, and the actual strings are
>>> stored
>>> in the ldc. When you want to extract traces, the sof-logger tool
>>> substitutes the codes for human-readable strings. The ldc files
>>> only
>>> existing in relation with the initial firmware they were extracted
>>> from.
>>>
>>> I don't see at all why these ldc files can't be distributed along
>>> with
>>> the firmware. In fact, we could change the firmware format and add
>>> them
>>> in a single file, which would be stripped during firmware download
>>> and
>>> you'd not even know about it...
>>
>> I see, but why to carry files which are not required to boot linux in
>> the
>> linux-firmware? It's only an extra bloat which only few users will
>> use. You
>> can integrated it to one file, of course. But ... you removed this
>> info for
>> the exactly similar reason, don't you? The extra debugging
>> information is not
>> required for the standard runtime.
>>
> 
> 
>>>> 2) SOF release - you have full control to put all things together
>>>> and
>>>>       we can use this as a reference; eventually, we can update
>>>> the community
>>>>       integration repos from this
>>>
>>> So we are talking about multiple integration channels, that's not
>>> so
>>> good for us
>>
>> We always hit this. Think what's good for all users. It's easy to
>> assign the
>> .ldc file to the firmware file using some checksums or so. And, who
>> are users
>> of this? Few users who want to debug problem or maintainers who can
>> point to
>> the .ldc file. Also, if you have .ldc, you need to have the trace
>> utility
>> (binary). Perhaps, you can create an extra debugging package with all
>> .ldc
>> files for all released firmwares and find a way at runtime to
>> determine the
>> loaded firmware and select the proper .ldc automatically. It's just
>> another hint.
>>
> 
> Ok, so if we can place the ldc files in alsa-ucm-conf (installed by
> default ?) alongside some debug script (to gather FW oops data using
> the ldc data) then we can retain the debug functionality. The intention
> is to ask users to run the debug script upon DSP crash and then paste
> the results in github (script still has to be written though).
> 
>>>> 3) extra firmware files (not used by default kernel configuration
>>>> / code)
>>>
>>> We don't have such cases at the moment.
>>
>> You do have. In the Liam's linux-firmware repository, there are
>> several old or
>> special 1.3 firmware files, which are not directly referred from the
>> driver.
>> The user must change the kernel configuration using the kernel /
>> module
>> parameters:
>>
>>
> 
> We need these for certain devices. Canonical guys on the CC need these
> atm, until they can update to newer version like the upcoming v1.4.2.
> 
>>
> https://github.com/thesofproject/linux-firmware/commit/beff732e6642bd77dac5dd08514264f055fc4d10#r366567340
>>
>> Example: File: intel/sof/cml/intel/sof-cml-v1.3-0f73628.ri ????
>>
>>>> It does not mean that the stable code should not be pushed to the
>>>> "integration" repositories (where the code is merged from the
>>>> multiple
>>>> sources). But in case of issues (debugging) or if you have
>>>> special users
>>>> (firmware for special hardware variants requiring the extra
>>>> kernel
>>>> parameters / setup), the users might look back to the all-in-one
>>>> repo
>>>> (SOF project release).
>>>
>>> I would really like to have all .ldc distributed by default so that
>>> we
>>> can ask for traces without requiring any install shenanigans.
>>> People are
>>> able to use alsa-info.sh, and I consider the .ldc files as a
>>> extension
>>> helping provide DSP traces.
>>
>> Yes, but it's debugging and there is no standard for this. The extra
>> debugging
>> package with .ldc and the trace utility for SOF might be really a
>> good thing.
> 
> It sounds like we are all aligned on this objective, albeit using
> different repos. Are there any objections to ldc files in alsa-ucm-conf
> providing a debug script can find the correct ldc file at runtime ?

As a packager, I would like to have the debug info and tools separate. We can 
ask user to install the debug package and the extra tools on demand.

This is very specific to SOF project.

So just split the tools and put all .ldc files for all released firmware files 
to one package maintained under the SOF project. The you can trace the kernel 
version / sysfs module parameters to obtain the firmware path and use 
sha256sum on the loaded firmware file (or another hash) to find this 
appropriate .ldc file. Or the kernel driver may expose the hash / used 
firmware file path via sysfs directly. It may be handy. The github will do 
packaging for you for free.

My proposal is:

1) one repository with all things linked through the annotate tags (release)
   - firmware binary
   - .ldc file for the firmware
   - topology binary
2) sof-debug package with all .ldc files and tools to extract the traces,
    other debug tools
3) linux-firmware updates (without any special cases)
4) alsa-topology-conf updates (.m4 files)

Think about this in this way:

- release everything properly linked inside the SOF project
   (user ready binaries, no compilations)
- update the upstream repositories with only appropriate pieces

The upstream packages merge the code from multiple sources. The special cases 
are not wanted too much.

					Jaroslav

-- 
Jaroslav Kysela <perex at perex.cz>
Linux Sound Maintainer; ALSA Project; Red Hat, Inc.