Handling complex matrix mixers in ALSA
Hi,
I'm reverse engineering and implementing support for the RME Digiface USB, which is an ADAT interface with a non-class-compliant interface (probably similar to other RME interfaces like the MADIface, but I don't have any others to test). The basic audio streaming works fine with an entry in quirks-table.h and a format quirk to set the system sample rate in quirks.c. Now I need to figure out how to implement the mixer controls.
Currently I have the snd-usb-audio driver claiming only interface #0 (streaming) and I use a Python script to control the mixer/settings directly with libusb (control transfers and interface #1). This works fine and there's some prior art for this in the firewire world (for example, snd-dice doesn't do any mixer control stuff and you have to use ffado-mixer to control everything from userspace) but I assume it's not really the best way to go?
The problem is that the device has a 66x34 matrix mixer, with up to 2048 cross points enabled at once. Exposing each cross point as an ALSA mixer control (similar to how mixer_scarlett2.c does it) would mean 2244 controls just for the mixer... which seems like a bit too much.
On top of that there is also VU meter feedback for all the inputs/outputs, as well as general fader controls for each output and global output configs and status. I'm not sure about the VU meters, but everything else sounds like it would map fine to normal mixer controls.
Is there some recommended way to expose this kind of matrix mixer interface to userspace? I think for something like this you pretty much have to rely on device-specific tools to make the UX manageable, so maybe hwdep... but at least exposing everything as an ALSA control would have the advantage of supporting save/restore with something like alsactl. So I don't really know what's the way to go here.
System settings/general status/output faders go via control transfers, while interface #1 has an interrupt IN endpoint (streaming state feedback, not very useful) and two bulk endpoints (matrix mixer control out, VU meter data in). There's another pair of bulk endpoints in interface #2 which I'm guessing are for firmware updates (I haven't looked at that part). So in principle it's not crazy to expose all the system controls/output faders as mixer controls in ALSA and leave interface #1 entirely unclaimed so a userspace program can directly configure the matrix mixer and access VU meter levels. There is a global mixer enable bit (controlled via ctl transfer), so if that is exposed as an ALSA control and disabled by default the interface will operate as a 1:1 in/out interface without needing any custom userspace to configure the mixer.
There's one other quirky thing: it also needs a way to set the sample rate as a mixer control, because you need to be able to configure the rate even when the PCM device is not open (since that affects stand-alone mixer operation). I imagine the right logic here would be to have a selector control for the system sample rate, and automatically change it and lock it when the PCM is opened with a given rate?
Any thoughts welcome ^^
~~ Lina
Hi,
I'm a maintainer of both ALSA firewire stack and Linux firewire subsystem. As long as I know, there is neither consensus nor specific userspace API/structure to the issue , therefore each developer selects each way within current implementation of ALSA control core and userspace interface.
On Mon, Jul 1, 2024, at 01:04, Asahi Lina wrote:
The problem is that the device has a 66x34 matrix mixer, with up to 2048 cross points enabled at once. Exposing each cross point as an ALSA mixer control (similar to how mixer_scarlett2.c does it) would mean 2244 controls just for the mixer... which seems like a bit too much.
On top of that there is also VU meter feedback for all the inputs/outputs, as well as general fader controls for each output and global output configs and status. I'm not sure about the VU meters, but everything else sounds like it would map fine to normal mixer controls.
Is there some recommended way to expose this kind of matrix mixer interface to userspace? I think for something like this you pretty much have to rely on device-specific tools to make the UX manageable, so maybe hwdep... but at least exposing everything as an ALSA control would have the advantage of supporting save/restore with something like alsactl. So I don't really know what's the way to go here.
In my opinion, expose of such many control elements would not be necessarily convenient to users, especially to who eager to use GUI for such matrix mixer. Additionally, initialization for all of configurable elements in device would sometimes require conditional operation somehow (e.g. dependency on sampling rate or any mode of digital interfaces).
If you have no need to share one of the USB endpoints for any configurable elemenets between kernel/userspace, it is a simple way to implement such confuguration application as userspace application, as you did with Python 3 and libusb.
Anyway, if you eager to change ALSA control core and its interface to userspace for the purpose, we go for more discussion about it. We share the same interest.
P.S. for devices supported by drivers in ALSA firewire stack, I write some service programs in userspace to utilize ALSA control "user-defined control element set". You can find the implementation of protocol to configure RME Fireface 400/800/UCX/808 in it, as well as any DICE-based models:
* https://github.com/alsa-project/snd-firewire-ctl-services/
Regards
Takashi Sakamoto
Hi Lina,
On Mon, Jul 01, 2024 at 01:04:41AM +0900, Asahi Lina wrote:
Hi,
I'm reverse engineering and implementing support for the RME Digiface USB, which is an ADAT interface with a non-class-compliant interface (probably similar to other RME interfaces like the MADIface, but I don't have any others to test). The basic audio streaming works fine with an entry in quirks-table.h and a format quirk to set the system sample rate in quirks.c. Now I need to figure out how to implement the mixer controls.
Currently I have the snd-usb-audio driver claiming only interface #0 (streaming) and I use a Python script to control the mixer/settings directly with libusb (control transfers and interface #1). This works fine and there's some prior art for this in the firewire world (for example, snd-dice doesn't do any mixer control stuff and you have to use ffado-mixer to control everything from userspace) but I assume it's not really the best way to go?
I'm the developer of the Scarlett2 driver. Doing as much as possible through ALSA mixer controls has worked well in my experience; as you say, being able to do save/restore with alsactl is useful.
The problem is that the device has a 66x34 matrix mixer, with up to 2048 cross points enabled at once. Exposing each cross point as an ALSA mixer control (similar to how mixer_scarlett2.c does it) would mean 2244 controls just for the mixer... which seems like a bit too much.
Note that mixer controls may have more than one value. So I think you could have 66 controls with 34 values or 34 controls with 66 values or 1 control with 2244 values.
On top of that there is also VU meter feedback for all the inputs/outputs, as well as general fader controls for each output and global output configs and status. I'm not sure about the VU meters, but everything else sounds like it would map fine to normal mixer controls.
I handle the VU meter feedback in the Scarlett2 driver with a read-only volatile control that contains multiple values (see scarlett2_meter_ctl).
Regards, Geoffrey.
Is there some recommended way to expose this kind of matrix mixer interface to userspace? I think for something like this you pretty much have to rely on device-specific tools to make the UX manageable, so maybe hwdep... but at least exposing everything as an ALSA control would have the advantage of supporting save/restore with something like alsactl. So I don't really know what's the way to go here.
System settings/general status/output faders go via control transfers, while interface #1 has an interrupt IN endpoint (streaming state feedback, not very useful) and two bulk endpoints (matrix mixer control out, VU meter data in). There's another pair of bulk endpoints in interface #2 which I'm guessing are for firmware updates (I haven't looked at that part). So in principle it's not crazy to expose all the system controls/output faders as mixer controls in ALSA and leave interface #1 entirely unclaimed so a userspace program can directly configure the matrix mixer and access VU meter levels. There is a global mixer enable bit (controlled via ctl transfer), so if that is exposed as an ALSA control and disabled by default the interface will operate as a 1:1 in/out interface without needing any custom userspace to configure the mixer.
There's one other quirky thing: it also needs a way to set the sample rate as a mixer control, because you need to be able to configure the rate even when the PCM device is not open (since that affects stand-alone mixer operation). I imagine the right logic here would be to have a selector control for the system sample rate, and automatically change it and lock it when the PCM is opened with a given rate?
Any thoughts welcome ^^
~~ Lina
On Sun, 30 Jun 2024 18:04:41 +0200, Asahi Lina wrote:
Hi,
I'm reverse engineering and implementing support for the RME Digiface USB, which is an ADAT interface with a non-class-compliant interface (probably similar to other RME interfaces like the MADIface, but I don't have any others to test). The basic audio streaming works fine with an entry in quirks-table.h and a format quirk to set the system sample rate in quirks.c. Now I need to figure out how to implement the mixer controls.
Currently I have the snd-usb-audio driver claiming only interface #0 (streaming) and I use a Python script to control the mixer/settings directly with libusb (control transfers and interface #1). This works fine and there's some prior art for this in the firewire world (for example, snd-dice doesn't do any mixer control stuff and you have to use ffado-mixer to control everything from userspace) but I assume it's not really the best way to go?
The problem is that the device has a 66x34 matrix mixer, with up to 2048 cross points enabled at once. Exposing each cross point as an ALSA mixer control (similar to how mixer_scarlett2.c does it) would mean 2244 controls just for the mixer... which seems like a bit too much.
On top of that there is also VU meter feedback for all the inputs/outputs, as well as general fader controls for each output and global output configs and status. I'm not sure about the VU meters, but everything else sounds like it would map fine to normal mixer controls.
Is there some recommended way to expose this kind of matrix mixer interface to userspace? I think for something like this you pretty much have to rely on device-specific tools to make the UX manageable, so maybe hwdep... but at least exposing everything as an ALSA control would have the advantage of supporting save/restore with something like alsactl. So I don't really know what's the way to go here.
System settings/general status/output faders go via control transfers, while interface #1 has an interrupt IN endpoint (streaming state feedback, not very useful) and two bulk endpoints (matrix mixer control out, VU meter data in). There's another pair of bulk endpoints in interface #2 which I'm guessing are for firmware updates (I haven't looked at that part). So in principle it's not crazy to expose all the system controls/output faders as mixer controls in ALSA and leave interface #1 entirely unclaimed so a userspace program can directly configure the matrix mixer and access VU meter levels. There is a global mixer enable bit (controlled via ctl transfer), so if that is exposed as an ALSA control and disabled by default the interface will operate as a 1:1 in/out interface without needing any custom userspace to configure the mixer.
There's one other quirky thing: it also needs a way to set the sample rate as a mixer control, because you need to be able to configure the rate even when the PCM device is not open (since that affects stand-alone mixer operation). I imagine the right logic here would be to have a selector control for the system sample rate, and automatically change it and lock it when the PCM is opened with a given rate?
Any thoughts welcome ^^
As Geoffrey already suggested, the matrix size can be reduced since each kcontrol element can be an array, so the number of controls can be either column or row size of the matrix, which is well manageable.
The VU meter can be provided as volatile read-only controls, too.
So from the API-wise POV, it'll be a most straightforward implementation.
OTOH, if you need more efficiency (e.g. the control access is way too much overhead), it can be implemented freely via a hwdep device and your own ioctls or mmaps, too. But this is literally h/w dependent, and the API becomes too specific, no other way than using own tool, as a drawback.
Takashi
On Mon, Jul 01, 2024 at 04:17:11PM +0200, Takashi Iwai wrote:
As Geoffrey already suggested, the matrix size can be reduced since each kcontrol element can be an array, so the number of controls can be either column or row size of the matrix, which is well manageable.
Additionally, a snd_kcontrol structure can provide multiple control elements by its 'count' member. I call it 'control element set'. It can reduce allocation in kernel space. If the hardware in this case provides software interface to access to all source coefficients to one destination at once, it is suitable in the case.
For example, assuming the matrix mixer has 34 destination and 66 sources, they can be expressed by 34 control elements with 66 array elements. A single snd_kcontrol structure can provide them, as long as they have the same nature. The control elements are identified by index value.
Once I talked with Philippe Bekaert about the issue, then we found another issue about the way to distinguish both each control elements and the array members. The usage of ALSA control interface heavily relies on the name of control elements, while a single snd_kcontrol structure provides one name and all of the controls provided by it have the same name. We've investigated to use TLV (Type-Length-Array) function of ALSA control core to provide channel information about the sources and destinations, but no further work yet[1].
I think it better to have another care that in this case we have restriction for the size of array; e.g. 128 array elements for integer (long) type of value. The restriction is not the matter in your case.
The VU meter can be provided as volatile read-only controls, too.
So from the API-wise POV, it'll be a most straightforward implementation.
As a side note, the design of software interface for recent hardware requires floating point values for this kind of data, while it is not supported in ALSA control core and its userspace interface.
OTOH, if you need more efficiency (e.g. the control access is way too much overhead), it can be implemented freely via a hwdep device and your own ioctls or mmaps, too. But this is literally h/w dependent, and the API becomes too specific, no other way than using own tool, as a drawback.
[1] https://github.com/PhilippeBekaert/snd-hdspe/issues/13
Regards
Takashi Sakamoto
On 7/2/24 9:46 AM, Takashi Sakamoto wrote:
On Mon, Jul 01, 2024 at 04:17:11PM +0200, Takashi Iwai wrote:
As Geoffrey already suggested, the matrix size can be reduced since each kcontrol element can be an array, so the number of controls can be either column or row size of the matrix, which is well manageable.
Additionally, a snd_kcontrol structure can provide multiple control elements by its 'count' member. I call it 'control element set'. It can reduce allocation in kernel space. If the hardware in this case provides software interface to access to all source coefficients to one destination at once, it is suitable in the case.
The hardware interface is basically a command stream of "set" commands for each individual mixer node, so there is no particular dimension/grouping that is more optimal. You can send up to 64~128 arbitrary updates per 512-byte USB bulk packet (depending on whether you have to allocate nodes or just update existing nodes).
Due to the max 2048 active node limitation, the driver is going to have to process/diff bulk updates anyway regardless of the way we group them (I will have to scan the new values, first update any zero values to free up nodes, then update nonzero values, and bail with an error before sending the update to the hardware if we run out of nodes). Hopefully this will never happen in real-life scenarios, but it does mean that some matrix mixer configurations are "illegal" and will be refused by the driver, due to hardware limitations.
For example, assuming the matrix mixer has 34 destination and 66 sources, they can be expressed by 34 control elements with 66 array elements. A single snd_kcontrol structure can provide them, as long as they have the same nature. The control elements are identified by index value.
It took me a while to understand what you meant here, but I think I get it: Using a single snd_kcontrol for the entire mixer, with 34 indexed elements each taking 66 array values, right?
How do these kinds of controls show up in alsamixer and other userspace mixer tools? Are they usable at all, or just with low-level access via amixer/alsactl?
Once I talked with Philippe Bekaert about the issue, then we found another issue about the way to distinguish both each control elements and the array members. The usage of ALSA control interface heavily relies on the name of control elements, while a single snd_kcontrol structure provides one name and all of the controls provided by it have the same name. We've investigated to use TLV (Type-Length-Array) function of ALSA control core to provide channel information about the sources and destinations, but no further work yet[1].
Yeah, that's another issue... these interfaces have fairly homogeneous channels for the mixer, so it's not a big deal, but if we want to settle on any particular standard for matrix mixers we're going to need some way to inform userspace of what the numbered sources/destinations mean...
I think it better to have another care that in this case we have restriction for the size of array; e.g. 128 array elements for integer (long) type of value. The restriction is not the matter in your case.
A related device (the MADIface USB, that I don't own but I can probably support with some traces from someone who does) has, if I'm understanding the manual correctly, up to a 192x128 mixer mode (128 hardware inputs + 64 playback inputs per output mix), so that would exceed the maximum number of array elements. I could split each output submix control into multiple array controls per input group to keep the count under 128 for each, but that starts getting a bit weird and arbitrary I think...
The VU meter can be provided as volatile read-only controls, too.
So from the API-wise POV, it'll be a most straightforward implementation.
As a side note, the design of software interface for recent hardware requires floating point values for this kind of data, while it is not supported in ALSA control core and its userspace interface.
I don't know how the VU meter data works yet, but there's another issue here with the mixer controls. This device uses a 1-bit scale selector and a 14-bit value, basically a trivial floating point format with a 1-bit exponent. There is also a sign bit (can invert the phase of any mixer node).
Effectively I have a linear gain between -0x10000 and 0x10000 where 0x8000 is 0dB and -0x8000 is 0dB(inverted), but for values outside of the -0x3fff..0x3fff range, it loses 3 bits of LSB precision.
The inversion isn't really representable in a single control, right? So I'd have to have a whole separate boolean matrix control set for the sign bits, I think?
And then I need to figure out how to scale the values... if I use the full range of 0..0x10000 then I can use DECLARE_TLV_DB_LINEAR to declare how it maps to dB, and then I guess I would just have to truncate the values sent to the hardware (when in the coarse range) but always keep track of the control value in the driver, so userspace doesn't get confused with "impossible" values? Of course that would be "lying" about the hardware precision, since 8 possible values would map to the same hardware volume at higher scales...
From reading the alsa-lib code, I think I need to use TLV_DB_GAIN_MUTE as the min gain and then that turns it into a linear control where 0 = mute?
I'm also a bit worried about being able to accurately represent +0dB/unity (for bit-perfect passthrough), since ALSA uses 1/100th of a dB as scale for the TLVs (I think?). x2 gain is +6.02(...) dB, but that isn't enough significant digits to precisely map 0x8000 to 0dB:
hex(int(0x10000 * (10**(-6.02 / 20))))
'0x8002'
hex(int(0x10000 * (10**(-6.03 / 20))))
'0x7fdc'
So userspace that tries to use the TLV scale data to set "0dB" using the simple API is unlikely to actually get precisely 0dB.
(This is also a problem for the output fader controls unrelated to the matrix mixer, since they use the exact same scale encoding with inversion)
Maybe the driver should just use an arbitrary log dB TLV scale for the controls and internally map to the hardware values? That means doing the conversions internally and lying about precision at the lower end but at least it would avoid rounding error when setting a gain to 0dB...
OTOH, if you need more efficiency (e.g. the control access is way too much overhead), it can be implemented freely via a hwdep device and your own ioctls or mmaps, too. But this is literally h/w dependent, and the API becomes too specific, no other way than using own tool, as a drawback.
There's one more thing I'd like to ask. Would it be useful for me to submit just the streaming part of the device support upstream first (which would work with the userspace app for config) and then worry about designing the mixer control interface later? Or should it all be done in one submission?
~~ Lina
On Fri, Jul 12, 2024 at 06:48:09PM +0900, Asahi Lina wrote:
For example, assuming the matrix mixer has 34 destination and 66 sources, they can be expressed by 34 control elements with 66 array elements. A single snd_kcontrol structure can provide them, as long as they have the same nature. The control elements are identified by index value.
It took me a while to understand what you meant here, but I think I get it: Using a single snd_kcontrol for the entire mixer, with 34 indexed elements each taking 66 array values, right?
This is rough example for the above idea.
``` static int info_cb(struct snd_kcontrol *kctl, struct snd_ctl_elem_info *einfo) { einfo->type = SNDRV_CTL_ELEM_TYPE_INTEGER; einfo->count = 66; // up to 128 for integer value array. einfo->value.integer.min = 0; // note long type, varies between // ILP32/LP64 data models. einfo->value.integer.max = 256; // ditto. einfo->value.integer.step = 1; // ditto.
return 0; }
static int get_cb(struct snd_kcontrol *kctl, struct snd_ctl_elem_value *uval) { unsigned int index = snd_ctl_get_ioff(kctl, &info->id);
// Any operation according to the index of control elements in the set.
return 0; }
static int put_cb(struct snd_kcontrol *kctl, struct snd_ctl_elem_value *uval) { unsigned int offset = snd_ctl_get_ioff(kctl, &info->id);
// Any operation according to the index of control elements in the set.
return 0; }
int add_control_elements(struct my_data *my_data) { // Emulate AK4396. // 20 * log10(x/255) (dB) // Here, x is written value. // // Some examples in: // https://github.com/alsa-project/alsa-lib/blob/master/test/user-ctl-element-s.... static const SNDRV_CTL_TLVD_DECLARE_DB_LINEAR(range_tlv, -4813, 0);
static const struct snd_kcontrol_new template = { . iface = SNDRV_CTL_ELEM_IFACE_MIXER, .device = 10, .subdevice = 20, .name = "multiplexer", .index = 0, // I guess any operation on ARC USB does not control the hardware directly unlike RC and // ARC, thus SNDRV_CTL_ELEM_ACCESS_VOLATILE is not required. .access = SNDRV_CTL_ELEM_ACCESS_READWRITE | SNDRV_CTL_ELEM_ACCESS_TLV_READ, .count = 34, // The total number of control elements in the set. .info = info_cb, .get = get_cb, .put = pub_cb, .tlv.p = &range_tlv, };
struct snd_kcontrol *kctl = snd_ctl_new1(&template, my_data); err = snd_ctl_add(my_data->card, kctl); ... } ```
How do these kinds of controls show up in alsamixer and other userspace mixer tools? Are they usable at all, or just with low-level access via amixer/alsactl?
These control elements can be enumerated and operated as usual, just have the sequential value in index (and numid) field in element identifier.
``` $ amixer -c 0 controls ... numid=16,iface=MIXER,name='multiplexer',device=10,subdevice=20 numid=17,iface=MIXER,name='multiplexer',index=1,device=10,subdevice=20 numid=18,iface=MIXER,name='multiplexer',index=2,device=10,subdevice=20 numid=19,iface=MIXER,name='multiplexer',index=3,device=10,subdevice=20 numid=20,iface=MIXER,name='multiplexer',index=4,device=10,subdevice=20 numid=21,iface=MIXER,name='multiplexer',index=5,device=10,subdevice=20 numid=22,iface=MIXER,name='multiplexer',index=6,device=10,subdevice=20 numid=23,iface=MIXER,name='multiplexer',index=7,device=10,subdevice=20 numid=24,iface=MIXER,name='multiplexer',index=8,device=10,subdevice=20 numid=25,iface=MIXER,name='multiplexer',index=9,device=10,subdevice=20 ... ```
Of cource, these is no integrations in any type of alsa-lib mixer API abstractions. I use Quasmixer in quastools project to operate them.
* https://gitlab.com/sebholt/qastools
Regards
Takashi Sakamoto
On Mon, 1 Jul 2024, Asahi Lina wrote:
[...]
The problem is that the device has a 66x34 matrix mixer, with up to 2048 cross points enabled at once. Exposing each cross point as an ALSA mixer control (similar to how mixer_scarlett2.c does it) would mean 2244 controls just for the mixer... which seems like a bit too much.
On top of that there is also VU meter feedback for all the inputs/outputs, as well as general fader controls for each output and global output configs and status. I'm not sure about the VU meters, but everything else sounds like it would map fine to normal mixer controls.
Is there some recommended way to expose this kind of matrix mixer interface to userspace?
I don't know if it's recommended practice, but the echo3g driver may be a prior art as it has a working matrix mixer for the Layla, and the corresponding echomixer in alsa-tools.
I think for something like this you pretty much have to rely on device-specific tools to make the UX manageable
For a full UX, perhaps. But the concept of a matrix mixer itself is general enough.
Perhaps there could be a UI tool which is self contained with one purpose, to control a 'standard' matrix mixer. It could be invoked alongside alsamixer or other mixer tool, and multiple times if there are multiple matrix mixers on a piece of hardware.
On Thu, 4 Jul 2024, at 4:55 AM, Mark Hills wrote: [...]
I think for something like this you pretty much have to rely on device-specific tools to make the UX manageable
For a full UX, perhaps. But the concept of a matrix mixer itself is general enough.
Perhaps there could be a UI tool which is self contained with one purpose, to control a 'standard' matrix mixer. It could be invoked alongside alsamixer or other mixer tool, and multiple times if there are multiple matrix mixers on a piece of hardware.
If this were generalised, it might be possible for the functionality to be exposed via the audio server as well, which allows for things like sandboxing, etc. to work nicely too.
-- Arun
participants (6)
-
Arun Raghavan -
Asahi Lina -
Geoffrey D. Bennett -
Mark Hills -
Takashi Iwai -
Takashi Sakamoto