- Sound-open-firmware - mailman.alsa-project.org

[Sound-open-firmware] [PATCH 00/14] ASoC: SOF DSP virtualisation
by Guennadi Liakhovetski 13 Mar '20

13 Mar '20

This patch series extends the SOF driver to add support for DSP virtualisation to ASoC. It is built on top of VirtIO, contains a guest driver and a vhost in-kernel guest driver. This version supports a single playback and a single capture interface on the guest. The specific guest audio topology is supplied by the host from a file, specified on the QEMU command line. This solution is designed to use advanced DSP functionality, available on the host. In case no DSP is available on the host, a fall-back should be provided in the future to the pure audio virtualisation standard, currently developed by OASIS. Extensive documentation is available at [1]. Signed-off-by: Guennadi Liakhovetski <guennadi.liakhovetski(a)linux.intel.com> [1] https://thesofproject.github.io/latest/developer_guides/virtualization/virt… Guennadi Liakhovetski (14): ASoC: add function parameters to enable forced path pruning ASoC: SOF: extract firmware-related operation into a function ASoC: SOF: VirtIO: make a function global vhost: convert VHOST_VSOCK_SET_RUNNING to a generic ioctl ASoC: SOF: support IPC with immediate response ASoC: SOF: add a power status IPC ASoC: SOF: add two helper lookup functions ASoC: SOF: fix uninitialised "work" with VirtIO ASoC: SOF: add a VirtIO DSP driver ASoC: SOF: add a vhost driver: sound part vhost: add an SOF DSP driver ASoC: SOF: VirtIO: check guest component IDs ASoC: SOF: VirtIO: free guest pipelines upon termination ASoC: SOF: VirtIO: enable simultaneous playback and capture drivers/vhost/Kconfig | 7 + drivers/vhost/Makefile | 5 + drivers/vhost/dsp.c | 731 ++++++++++++++++++++++ include/sound/soc-dpcm.h | 28 +- include/sound/soc-topology.h | 3 + include/sound/sof.h | 4 + include/sound/sof/header.h | 3 + include/sound/sof/topology.h | 1 + include/sound/sof/virtio.h | 206 +++++++ include/uapi/linux/vhost.h | 9 +- include/uapi/linux/vhost_types.h | 7 + include/uapi/linux/virtio_ids.h | 1 + sound/soc/soc-compress.c | 2 +- sound/soc/soc-dapm.c | 8 +- sound/soc/soc-pcm.c | 98 ++- sound/soc/sof/Kconfig | 8 + sound/soc/sof/Makefile | 8 + sound/soc/sof/core.c | 114 ++-- sound/soc/sof/ipc.c | 39 +- sound/soc/sof/loader.c | 4 + sound/soc/sof/ops.h | 10 +- sound/soc/sof/pcm.c | 17 +- sound/soc/sof/pm.c | 4 + sound/soc/sof/sof-audio.c | 33 + sound/soc/sof/sof-audio.h | 21 + sound/soc/sof/sof-priv.h | 54 ++ sound/soc/sof/topology.c | 76 ++- sound/soc/sof/vhost-be.c | 1248 ++++++++++++++++++++++++++++++++++++++ sound/soc/sof/virtio-fe.c | 922 ++++++++++++++++++++++++++++ 29 files changed, 3560 insertions(+), 111 deletions(-) create mode 100644 drivers/vhost/dsp.c create mode 100644 include/sound/sof/virtio.h create mode 100644 sound/soc/sof/vhost-be.c create mode 100644 sound/soc/sof/virtio-fe.c -- 1.9.3

2 15

[Sound-open-firmware] [PATCH v2 0/2] ASoC: SOF: preparatory patches
by Guennadi Liakhovetski 12 Mar '20

12 Mar '20

2 preparatory patches for the SOF virtualisation work. No functional changes, only some code refactoring and exporting of helper functions. v2: rebase, no changes otherwise. Thanks Guennadi Guennadi Liakhovetski (2): ASoC: (cosmetic) simplify dpcm_prune_paths() ASoC: export DPCM runtime update functions include/sound/soc-dpcm.h | 4 +-- sound/soc/soc-dapm.c | 8 +++--- sound/soc/soc-pcm.c | 68 +++++++++++++++++++++++++----------------------- 3 files changed, 42 insertions(+), 38 deletions(-) -- 1.9.3

3 5

Re: [Sound-open-firmware] [PATCH 0/3] [RESEND] ASoC: SOF: preparatory patches
by Guennadi Liakhovetski 12 Mar '20

12 Mar '20

On Thu, Mar 12, 2020 at 12:15:49PM +0000, Mark Brown wrote: > On Thu, Mar 12, 2020 at 12:45:37PM +0100, Guennadi Liakhovetski wrote: > > On Tue, Mar 10, 2020 at 12:50:56PM +0000, Mark Brown wrote: > > > > How does this relate to the virtio audio spec that's currently under > > > review? > > > The spec under discussion is only for simple audio virtualisation with fixed > > roles and topologies. With our approach guests get access to advanced DSP > > capabilities. The virtualisation approach under discussion can be a fallback > > for cases when no DSP has been detected on the host. > > So they're orthogonal :/ Have you proposed your spec yet? > > > > It looks to be doing something much lower level than that. > > > I am concerned that this looks to be exposing DPCM as a virtio ABI, > > > we're trying to replace it as an internal API never mind ABI. > > > You mean that our approach works at the widget level, which is a part of the > > DPCM API? Well there is a translation layer between our ABI and DPCM. And by > > virtue of the same argument don't we already have DPCM as an ABI on the > > opposite side of SOF - in its IPC ABI? Largely this virtualisation approach > > doesn't add new interfaces, it re-uses the SOF IPC ABI, which is also one of > > its advantages. > > Please bear in mind that the page you linked to is very high level and > I've not seen the actual spec or anything. The page and your mails both > talk about DPCM so it sounds like that's a part of the interface. Let me rebase my current stack and post it here, stay tuned :-) Thanks Guennadi

1 0

Re: [Sound-open-firmware] [PATCH 0/3] [RESEND] ASoC: SOF: preparatory patches
by Guennadi Liakhovetski 12 Mar '20

12 Mar '20

Hi, On Tue, Mar 10, 2020 at 12:50:56PM +0000, Mark Brown wrote: > On Mon, Mar 09, 2020 at 05:05:06PM -0500, Pierre-Louis Bossart wrote: > > > To get a better picture of the directions, reviewers are invited to take a > > look at the in-depth documentation written by Guennadi since the initial > > patches were shared. This documentation was reviewed by Liam and me and is > > really required to understand the concepts: > > > https://thesofproject.github.io/latest/developer_guides/virtualization/virt… > > How does this relate to the virtio audio spec that's currently under > review? The spec under discussion is only for simple audio virtualisation with fixed roles and topologies. With our approach guests get access to advanced DSP capabilities. The virtualisation approach under discussion can be a fallback for cases when no DSP has been detected on the host. > It looks to be doing something much lower level than that. > I am concerned that this looks to be exposing DPCM as a virtio ABI, > we're trying to replace it as an internal API never mind ABI. You mean that our approach works at the widget level, which is a part of the DPCM API? Well there is a translation layer between our ABI and DPCM. And by virtue of the same argument don't we already have DPCM as an ABI on the opposite side of SOF - in its IPC ABI? Largely this virtualisation approach doesn't add new interfaces, it re-uses the SOF IPC ABI, which is also one of its advantages. Thanks Guennadi

1 0

Re: [Sound-open-firmware] [PATCH 2/3] ASoC: add function parameters to enable forced path pruning
by Guennadi Liakhovetski 11 Mar '20

11 Mar '20

On Wed, Mar 11, 2020 at 12:41:02PM +0000, Mark Brown wrote: > On Wed, Mar 11, 2020 at 01:36:17PM +0100, Guennadi Liakhovetski wrote: > > On Wed, Mar 11, 2020 at 12:25:31PM +0000, Mark Brown wrote: > > > > It's still not clear to me what the issue is here. If something is > > > making a modification to the graph which needs a recheck or update I'd > > > expect that these things happen along with that modification. I don't > > > understand what you're saying about not being able to reproduce > > > scenarios or adding things "on top". > > > VirtIO support. That function currently exists in the kernel, but I don't > > have a test-case to verify its work, to see it called and actually perform > > the pruning. So I don't know how it is supposed to work. And because of > > that I cannot fix my VirtIO use case to use that function properly, without > > forcing it with an additional parameter. > > I think you need to include this with your VirtIO series. That was also my original intention, personally I also would find it difficult to review this kind of work without context. So, would you prefer me to drop this series completely and re-send it with the whole series or should I just drop #2 and resend #1 and #3? Thanks Guennadi

1 0

[Sound-open-firmware] SOF Versioning and stable releases
by Daniel Baluta 11 Mar '20

11 Mar '20

Hi Liam, What is the current stable release? And what is the release versioning scheme. I think it is 1.4-stable right? We plan to create a stable branch for imx. Maybe 1.4-stable-imx and put it into our internal Yocto release. Also, for sof-bin release repo how we will push the binaries? I mean just compile it by yourself and push it there. Or we let a travis build to do the job so that we can also have the compile log.

1 0

Re: [Sound-open-firmware] [PATCH 2/3] ASoC: add function parameters to enable forced path pruning
by Guennadi Liakhovetski 11 Mar '20

11 Mar '20

Hi Mark, On Wed, Mar 11, 2020 at 12:25:31PM +0000, Mark Brown wrote: > On Wed, Mar 11, 2020 at 08:41:27AM +0100, Guennadi Liakhovetski wrote: > > On Tue, Mar 10, 2020 at 12:45:44PM +0000, Mark Brown wrote: > > > > So why doesn't DPCM recognize that the path is inactive and why is it > > > better to do this than fix whatever the issue is there? > > > Of course that would be better abd I'd much prefer that. Unfortunately I > > haven't been able to find a single scenario in which those paths would be > > exercised. As far as I understand path pruning should take place e.g. > > when a mixer modifies audio routing and as a result disables a certain > > pipeline, which is then pruned. If I could reproduce such a scenario I > > would be able to first check whether it's working, then see exactly how > > it is working and then see how best to add my use case to it. Since I > > wasn't able to find such a scenario, my only option was to preserve > > the current state and add my own path "on top." I'd be happy to try the > > other path too, I just need a use case, that I can reproduce. > > It's still not clear to me what the issue is here. If something is > making a modification to the graph which needs a recheck or update I'd > expect that these things happen along with that modification. I don't > understand what you're saying about not being able to reproduce > scenarios or adding things "on top". I mean, that I don't have a test-case to test dpcm_prune_paths() without my VirtIO support. That function currently exists in the kernel, but I don't have a test-case to verify its work, to see it called and actually perform the pruning. So I don't know how it is supposed to work. And because of that I cannot fix my VirtIO use case to use that function properly, without forcing it with an additional parameter. Thanks Guennadi

1 0

Re: [Sound-open-firmware] [PATCH 2/3] ASoC: add function parameters to enable forced path pruning
by Guennadi Liakhovetski 11 Mar '20

11 Mar '20

Hi Mark, On Tue, Mar 10, 2020 at 12:45:44PM +0000, Mark Brown wrote: > On Mon, Mar 09, 2020 at 06:07:48PM +0100, Guennadi Liakhovetski wrote: > > > soc_dpcm_runtime_update() to activate an audio path and update audio > > routing. When streaming is stopped, the vhost driver calls > > soc_dpcm_runtime_update() and snd_soc_runtime_deactivate(). The latter > > doesn't work at the moment, because the DPCM doesn't recognise the > > path as inactive. We address this by adding a "mode" parameter to > > soc_dpcm_runtime_update(). If virtualisation isn't used, the current > > behaviour isn't affected. > > So why doesn't DPCM recognize that the path is inactive and why is it > better to do this than fix whatever the issue is there? Of course that would be better abd I'd much prefer that. Unfortunately I haven't been able to find a single scenario in which those paths would be exercised. As far as I understand path pruning should take place e.g. when a mixer modifies audio routing and as a result disables a certain pipeline, which is then pruned. If I could reproduce such a scenario I would be able to first check whether it's working, then see exactly how it is working and then see how best to add my use case to it. Since I wasn't able to find such a scenario, my only option was to preserve the current state and add my own path "on top." I'd be happy to try the other path too, I just need a use case, that I can reproduce. Thanks Guennadi

1 0

Re: [Sound-open-firmware] [PATCH 0/3] [RESEND] ASoC: SOF: preparatory patches
by Pierre-Louis Bossart 10 Mar '20

10 Mar '20

On 3/10/20 7:50 AM, Mark Brown wrote: > On Mon, Mar 09, 2020 at 05:05:06PM -0500, Pierre-Louis Bossart wrote: > >> To get a better picture of the directions, reviewers are invited to take a >> look at the in-depth documentation written by Guennadi since the initial >> patches were shared. This documentation was reviewed by Liam and me and is >> really required to understand the concepts: > >> https://thesofproject.github.io/latest/developer_guides/virtualization/virt… > > How does this relate to the virtio audio spec that's currently under > review? It looks to be doing something much lower level than that. Mark, to avoid any ambiguity, what 'virtio audio spec that's currently under review' are you referring to? > I am concerned that this looks to be exposing DPCM as a virtio ABI, > we're trying to replace it as an internal API never mind ABI. Valid point indeed.

1 0

[Sound-open-firmware] [PATCH 0/3] [RESEND] ASoC: SOF: preparatory patches
by Guennadi Liakhovetski 10 Mar '20

10 Mar '20

This is the first set of patches for the SOF virtualisation work. We send these patches first because they touch the ASoC core. 2 of them are mostly cosmetic with no functional changes, but patch 2/3 might cause some discussions. Please review and comment. Thanks Guennadi Guennadi Liakhovetski (3): ASoC: (cosmetic) simplify dpcm_prune_paths() ASoC: add function parameters to enable forced path pruning ASoC: export DPCM runtime update functions include/sound/soc-dpcm.h | 30 ++++++++--- sound/soc/soc-compress.c | 2 +- sound/soc/soc-dapm.c | 8 +-- sound/soc/soc-pcm.c | 130 +++++++++++++++++++++++++++++------------------ 4 files changed, 109 insertions(+), 61 deletions(-) -- 1.9.3

2 7