[PATCH 1/2] ucm: docs: Add EchoReferenceDev
From: Curtis Malainey cujomalainey@chromium.org
Sometimes userspace may want to use a reference channel to cancel echos when using video chat, this value identifies the device which carries that channel.
Signed-off-by: Curtis Malainey cujomalainey@chromium.org --- include/use-case.h | 9 +++++++++ 1 file changed, 9 insertions(+)
diff --git a/include/use-case.h b/include/use-case.h index e5d4fd68..3c7c0a26 100644 --- a/include/use-case.h +++ b/include/use-case.h @@ -376,6 +376,15 @@ int snd_use_case_get_list(snd_use_case_mgr_t *uc_mgr, * - CaptureMicInfoFile * - json file with the microphone array placement and type description * (e.g. output from nhlt-dmic-info) + * - EchoReferenceDev + * - The name of a section device that provides some sort of reference + * to the current section device. This can be a echo taken as a + * reference or as IV data from a smart amp. The device should be + * opened immediately after this device and closed right before this + * device. This is meant to support hardware features such as smart + * amps or hardware offloaded processing where the host still needs + * to open the channel. The audio server should open the reference + * device with the same parameters it opens the source device with. * - EDIDFile * - Path to EDID file for HDMI devices * - JackCTL
From: Curtis Malainey cujomalainey@chromium.org
Provide a UCM value to mark devices which are needed to be opened only for HW purposes but should not be consumed for userspace.
Signed-off-by: Curtis Malainey cujomalainey@chromium.org --- include/use-case.h | 6 ++++++ 1 file changed, 6 insertions(+)
diff --git a/include/use-case.h b/include/use-case.h index 3c7c0a26..3d629667 100644 --- a/include/use-case.h +++ b/include/use-case.h @@ -387,6 +387,12 @@ int snd_use_case_get_list(snd_use_case_mgr_t *uc_mgr, * device with the same parameters it opens the source device with. * - EDIDFile * - Path to EDID file for HDMI devices + * - IVDataEcho + * - Used only on devices pointed to by EchoReferenceDev. Used to mark + * references that might be lower quality (e.g. reduced bit depth) as + * they are intended for a HW feature and not userspace consumption. + * These PCMs still need to be opened by userspace but are not + * recommended for consumption. * - JackCTL * - jack control device name * - JackControl
On 24. 08. 23 23:33, cujomalainey@chromium.org wrote:
From: Curtis Malainey cujomalainey@chromium.org
Provide a UCM value to mark devices which are needed to be opened only for HW purposes but should not be consumed for userspace.
It would be probably better to describe this in the standard ALSA PCM API than UCM. If the device is special, applications without UCM should be also allowed to identify them.
Jaroslav
Curtis Malainey | Chrome OS Audio Senior Software Engineer | cujomalainey@google.com | Sound Open Firmware Lead
On Sat, Aug 26, 2023 at 4:31 AM Jaroslav Kysela perex@perex.cz wrote:
On 24. 08. 23 23:33, cujomalainey@chromium.org wrote:
From: Curtis Malainey cujomalainey@chromium.org
Provide a UCM value to mark devices which are needed to be opened only for HW purposes but should not be consumed for userspace.
It would be probably better to describe this in the standard ALSA PCM API than UCM. If the device is special, applications without UCM should be also allowed to identify them.
Jaroslav
Would something like an additional device class and a format subclass be acceptable?
Curtis
On 31. 08. 23 19:06, Curtis Malainey wrote:
Curtis Malainey | Chrome OS Audio Senior Software Engineer | cujomalainey@google.com | Sound Open Firmware Lead
On Sat, Aug 26, 2023 at 4:31 AM Jaroslav Kysela perex@perex.cz wrote:
On 24. 08. 23 23:33, cujomalainey@chromium.org wrote:
From: Curtis Malainey cujomalainey@chromium.org
Provide a UCM value to mark devices which are needed to be opened only for HW purposes but should not be consumed for userspace.
It would be probably better to describe this in the standard ALSA PCM API than UCM. If the device is special, applications without UCM should be also allowed to identify them.
JaroslavWould something like an additional device class and a format subclass be acceptable?
The device class extension sounds good.
I don't know details for the format extension - does app require to set hw_params ? If yes, is the goal to prevent standard applications to set the hw_params ? If no, the format bitmask may be set to zero, thus hw_params will always fail.
Jaroslav
On 24. 08. 23 23:33, cujomalainey@chromium.org wrote:
From: Curtis Malainey cujomalainey@chromium.org
Sometimes userspace may want to use a reference channel to cancel echos when using video chat, this value identifies the device which carries that channel.
The UCM modifier should be used for this - see "Echo Reference" comments in use-case.h.
Note that this allows additional setup (in Sequences) for this stream.
Jaroslav
On Sat, Aug 26, 2023 at 4:28 AM Jaroslav Kysela perex@perex.cz wrote:
On 24. 08. 23 23:33, cujomalainey@chromium.org wrote:
From: Curtis Malainey cujomalainey@chromium.org
Sometimes userspace may want to use a reference channel to cancel echos when using video chat, this value identifies the device which carries that channel.
The UCM modifier should be used for this - see "Echo Reference" comments in use-case.h.
Note that this allows additional setup (in Sequences) for this stream.
Jaroslav
I was under the impression modifiers were state manipulators that acted upon existing devices/pcms and did not designate their own PCM. That is at least how we use them in CRAS.
Are there any examples of how to designate a PCM? I don't see any modifiers at all in ucm-conf repo.
Curtis
-- Jaroslav Kysela perex@perex.cz Linux Sound Maintainer; ALSA Project; Red Hat, Inc.
On 8/28/23 12:59, Curtis Malainey wrote:
On Sat, Aug 26, 2023 at 4:28 AM Jaroslav Kysela perex@perex.cz wrote:
On 24. 08. 23 23:33, cujomalainey@chromium.org wrote:
From: Curtis Malainey cujomalainey@chromium.org
Sometimes userspace may want to use a reference channel to cancel echos when using video chat, this value identifies the device which carries that channel.
The UCM modifier should be used for this - see "Echo Reference" comments in use-case.h.
Note that this allows additional setup (in Sequences) for this stream.
JaroslavI was under the impression modifiers were state manipulators that acted upon existing devices/pcms and did not designate their own PCM. That is at least how we use them in CRAS.
Are there any examples of how to designate a PCM? I don't see any modifiers at all in ucm-conf repo.
I will second Curtis' request for clarifications.
I naively thought that modifiers would be used to e.g. select a 'Deep Buffer' output for low-power playback, or different capture streams based on the needs of the applications. It's not uncommon for capture applications to request different PCM streams for raw, AEC processed, AEC+NS processed data.
Echo reference is not really something that varies based on any qualifiers. And specifically in the Chrome case we want userspace to open the PCM echo reference device whenever the playback is used. So it's not really a use-case dependent thing but more a way to express a dependency between PCM devices.
I would triple this for the steamOS case, which has an inversion of this problem (a playback device that has a coupled capture device for speaker management). Coupling PCM devices statically in the UCM file avoids a class of side effects that could occur if one tried to implement the same behavior with exec calls in Enable/DisableSequence.
On Mon, Aug 28, 2023 at 11:35 AM Pierre-Louis Bossart < pierre-louis.bossart@linux.intel.com> wrote:
On 8/28/23 12:59, Curtis Malainey wrote:
On Sat, Aug 26, 2023 at 4:28 AM Jaroslav Kysela perex@perex.cz wrote:
On 24. 08. 23 23:33, cujomalainey@chromium.org wrote:
From: Curtis Malainey cujomalainey@chromium.org
Sometimes userspace may want to use a reference channel to cancel echos when using video chat, this value identifies the device which carries that channel.
The UCM modifier should be used for this - see "Echo Reference"
comments in
use-case.h.
Note that this allows additional setup (in Sequences) for this stream.
JaroslavI was under the impression modifiers were state manipulators that acted upon existing devices/pcms and did not designate their own PCM. That is at least how we use them in CRAS.
Are there any examples of how to designate a PCM? I don't see any modifiers at all in ucm-conf repo.
I will second Curtis' request for clarifications.
I naively thought that modifiers would be used to e.g. select a 'Deep Buffer' output for low-power playback, or different capture streams based on the needs of the applications. It's not uncommon for capture applications to request different PCM streams for raw, AEC processed, AEC+NS processed data.
Echo reference is not really something that varies based on any qualifiers. And specifically in the Chrome case we want userspace to open the PCM echo reference device whenever the playback is used. So it's not really a use-case dependent thing but more a way to express a dependency between PCM devices.
On 28. 08. 23 20:35, Pierre-Louis Bossart wrote:
On 8/28/23 12:59, Curtis Malainey wrote:
On Sat, Aug 26, 2023 at 4:28 AM Jaroslav Kysela perex@perex.cz wrote:
On 24. 08. 23 23:33, cujomalainey@chromium.org wrote:
From: Curtis Malainey cujomalainey@chromium.org
Sometimes userspace may want to use a reference channel to cancel echos when using video chat, this value identifies the device which carries that channel.
The UCM modifier should be used for this - see "Echo Reference" comments in use-case.h.
Note that this allows additional setup (in Sequences) for this stream.
JaroslavI was under the impression modifiers were state manipulators that acted upon existing devices/pcms and did not designate their own PCM. That is at least how we use them in CRAS.
Are there any examples of how to designate a PCM? I don't see any modifiers at all in ucm-conf repo.
I will second Curtis' request for clarifications.
I naively thought that modifiers would be used to e.g. select a 'Deep Buffer' output for low-power playback, or different capture streams based on the needs of the applications. It's not uncommon for capture applications to request different PCM streams for raw, AEC processed, AEC+NS processed data.
Echo reference is not really something that varies based on any qualifiers. And specifically in the Chrome case we want userspace to open the PCM echo reference device whenever the playback is used. So it's not really a use-case dependent thing but more a way to express a dependency between PCM devices.
It seems that there are some assumptions. I will try to address some things:
You can enable/use multiple modifiers per device.
The modifiers may define extra PCM streams in the standard Value section - you can use CapturePCM value for the modifier like "Echo Reference". Modifiers may alter the characteristics of the original UCM device stream (using command sequences), too.
Modifiers are an extra layer on top of devices. I don't think that we have any default relation between the modifier PCM device and the original PCM device (from the UCM device definition). A new value to describe this (like "ReplacePlaybackPCM 1") may be introduced. Another issue is when multiple modifiers with this description are active - they conflict, so it should be described somewhere, too. Perhaps, "ConflictingModifier" array may be added to API. But those extensions are not required for the "Echo Reference" modifier.
Jaroslav
It seems that there are some assumptions. I will try to address some things:
You can enable/use multiple modifiers per device.
The modifiers may define extra PCM streams in the standard Value section
- you can use CapturePCM value for the modifier like "Echo Reference".
Modifiers may alter the characteristics of the original UCM device stream (using command sequences), too.
Sorry, not following.
The 'modifier' has to be selected by userspace, isn't it? "someone" has to tell UCM that the 'echo reference' is on.
And that's precisely the conceptual issue I have. The echo reference in our cases is ALWAYS available when the speaker output is selected.
In other words, we are asking userspace to select something that is always present and available. Or put differently, a modifier that's always applicable is not really a modifier, is it?
And last, the whole story of the echo reference is that it needs to be opened when the speaker output is opened. How would we model this with the modifier concept?
Modifiers are an extra layer on top of devices. I don't think that we have any default relation between the modifier PCM device and the original PCM device (from the UCM device definition). A new value to describe this (like "ReplacePlaybackPCM 1") may be introduced. Another issue is when multiple modifiers with this description are active - they conflict, so it should be described somewhere, too. Perhaps, "ConflictingModifier" array may be added to API. But those extensions are not required for the "Echo Reference" modifier.
Right, the main issue here is that the PlaybackPCM and Echo reference PCM are joined and need to be handled at the same time. It's not a conflict, they are a bundle or a set of devices that cannot be used independently.
On 29. 08. 23 18:30, Pierre-Louis Bossart wrote:
It seems that there are some assumptions. I will try to address some things:
You can enable/use multiple modifiers per device.
The modifiers may define extra PCM streams in the standard Value section
- you can use CapturePCM value for the modifier like "Echo Reference".
Modifiers may alter the characteristics of the original UCM device stream (using command sequences), too.
Sorry, not following.
The 'modifier' has to be selected by userspace, isn't it? "someone" has to tell UCM that the 'echo reference' is on.
And that's precisely the conceptual issue I have. The echo reference in our cases is ALWAYS available when the speaker output is selected.
The function of modifier is selected by it's name, so an app should know, how to handle the "Echo Reference". And this use is optional.
In other words, we are asking userspace to select something that is always present and available. Or put differently, a modifier that's always applicable is not really a modifier, is it?
Yes, in this special case, only joined PCM will be provided. But do not forget, that we have command sequences for modifiers, if we need to initialize something else like related controls for this stream in future. It's just more universal than to hardcode this to key/value in the UCM device definition.
And last, the whole story of the echo reference is that it needs to be opened when the speaker output is opened. How would we model this with the modifier concept?
The modifiers are allowed to be activated only for the active devices. We can refine the use of the "Echo Reference" for applications and define that the playback PCM should be opened at first. If we need to alter this default behaviour in future, we may put a hint to configuration values.
Jaroslav
On Tue, Aug 29, 2023 at 8:37 AM Jaroslav Kysela perex@perex.cz wrote:
The modifiers may define extra PCM streams in the standard Value section - you can use CapturePCM value for the modifier like "Echo Reference". Modifiers may alter the characteristics of the original UCM device stream (using command sequences), too.
Thanks for the clarification, that indeed is where the assumption I think for most of us is not as clear. I would have never thought a modifier would carry a PCM.
I will trigger some work internally to convert our value to the modifier syntax.
Curtis
Modifiers are an extra layer on top of devices. I don't think that we have any default relation between the modifier PCM device and the original PCM device (from the UCM device definition). A new value to describe this (like "ReplacePlaybackPCM 1") may be introduced. Another issue is when multiple modifiers with this description are active - they conflict, so it should be described somewhere, too. Perhaps, "ConflictingModifier" array may be added to API. But those extensions are not required for the "Echo Reference" modifier.
Jaroslav-- Jaroslav Kysela perex@perex.cz Linux Sound Maintainer; ALSA Project; Red Hat, Inc.
On Tue, Aug 29, 2023 at 12:17 PM Curtis Malainey cujomalainey@google.com wrote:
On Tue, Aug 29, 2023 at 8:37 AM Jaroslav Kysela perex@perex.cz wrote:
Modifiers are an extra layer on top of devices. I don't think that we have any default relation between the modifier PCM device and the original PCM device (from the UCM device definition). A new value to describe this (like "ReplacePlaybackPCM 1") may be introduced. Another issue is when multiple modifiers with this description are active - they conflict, so it should be described somewhere, too. Perhaps, "ConflictingModifier" array may be added to API. But those extensions are not required for the "Echo Reference" modifier.
What is the expectation for naming and relationship to the devices? Is the Modifier a suffix to the device name to associate it?
Curtis
On 30. 08. 23 0:57, Curtis Malainey wrote:
On Tue, Aug 29, 2023 at 12:17 PM Curtis Malainey cujomalainey@google.com wrote:
On Tue, Aug 29, 2023 at 8:37 AM Jaroslav Kysela perex@perex.cz wrote:
Modifiers are an extra layer on top of devices. I don't think that we have any default relation between the modifier PCM device and the original PCM device (from the UCM device definition). A new value to describe this (like "ReplacePlaybackPCM 1") may be introduced. Another issue is when multiple modifiers with this description are active - they conflict, so it should be described somewhere, too. Perhaps, "ConflictingModifier" array may be added to API. But those extensions are not required for the "Echo Reference" modifier.
What is the expectation for naming and relationship to the devices? Is the Modifier a suffix to the device name to associate it?
Verbs, devices and modifiers should be named according use-case.h:
https://www.alsa-project.org/alsa-doc/alsa-lib/group__ucm.html
The relation is defined using SupportedDevices or ConflictingDevices array. Those arrays are available in modifiers, too.
Jaroslav
participants (5)
-
cujomalainey＠chromium.org -
Curtis Malainey -
Ethan Geller -
Jaroslav Kysela -
Pierre-Louis Bossart