[alsa-devel] Jack event API - decision needed
Hi,
I'm still new in the community in that sense that I'm not sure how decisions are made. But I could use the outcome of such a decision.
Background: I'm trying to pull together the missing pieces of jack detection on both kernel / plumbing / application layers, so that when you plug something in (headset, microphone etc), userspace is notified and can take appropriate actions (e g routing decisions).
As part of that I wrote a udev patch a few days ago, which nobody commented on in alsa-devel [1], but was somewhat disliked by at least Kay Sievers who maintains udev [2], who preferred we would rewrite our input layer to do something else within ALSA.
So before I proceed further I'd like to know if
1) We're continuing the path with /dev/input devices
2a) We'll rewrite these devices to be read-only ALSA mixer controls
2b) We'll rewrite these devices to be something within ALSA, but not exactly mixer controls.
For options 2a) and 2b) I guess the existing /dev/input thing should be deprecated and/or removed. So part of decision should maybe be based on information about how widespread the usage of these devices are currently...?
On Mon, Jun 20, 2011 at 03:37:25PM +0200, David Henningsson wrote:
I'm still new in the community in that sense that I'm not sure how decisions are made. But I could use the outcome of such a decision.
So, I'm still not 100% sure what the actual technical issue is here?
As part of that I wrote a udev patch a few days ago, which nobody commented on in alsa-devel [1], but was somewhat disliked by at
Having dug out the mailing list archive I guess the fact that you just posted it to the list and didn't CC anyone on the patch didn't help here. Looking at the patch the main thing that jumps out at me without any knowledge of the udev code is that the patch will end up classifying video output jacks as audio even if they've no audio capability which is obviously not correct.
I still don't entirely understand the technical issue you're trying to address here - this doesn't seem specific to audio. As far as I can tell the issue is that some of the input devices on the system aren't being made available to the console user, presumably because there are some that shouldn't be made available to them, so the issue is that the heuristics that udev uses to decide if an input device should be root only aren't working correctly in at least this case. It feels like if we understood why the heuristics are making a bad call here we might be able to come up with a better solution.
least Kay Sievers who maintains udev [2], who preferred we would rewrite our input layer to do something else within ALSA.
It seems like Kay's issues are based on a misunderstanding about what a jack might be.
For options 2a) and 2b) I guess the existing /dev/input thing should be deprecated and/or removed. So part of decision should maybe be based on information about how widespread the usage of these devices are currently...?
There's a reasonable amount of usage in the embedded space.
At Mon, 20 Jun 2011 18:07:40 +0100, Mark Brown wrote:
On Mon, Jun 20, 2011 at 03:37:25PM +0200, David Henningsson wrote:
I'm still new in the community in that sense that I'm not sure how decisions are made. But I could use the outcome of such a decision.
So, I'm still not 100% sure what the actual technical issue is here?
The device permissions. They are set inaccessible from the normal desktop user as default.
...
For options 2a) and 2b) I guess the existing /dev/input thing should be deprecated and/or removed. So part of decision should maybe be based on information about how widespread the usage of these devices are currently...?
There's a reasonable amount of usage in the embedded space.
Do they use udev? And don't they need to tweak the device permissions of these input-jack devices on the fly per user?
thanks,
Takashi
On Mon, Jun 20, 2011 at 07:12:21PM +0200, Takashi Iwai wrote:
Mark Brown wrote:
On Mon, Jun 20, 2011 at 03:37:25PM +0200, David Henningsson wrote:
I'm still new in the community in that sense that I'm not sure how decisions are made. But I could use the outcome of such a decision.
So, I'm still not 100% sure what the actual technical issue is here?
The device permissions. They are set inaccessible from the normal desktop user as default.
Right, I got that much but what I was trying to say in the rest of the paragraph was that it wasn't immediately obvious to me that this was due to needing to do something special for audio jacks. That's certainly a symptom but what's not clear to me is why we think the current setup is a good call for the generic input device that udev didn't have specific magic for. We seem to have jumped straight into some very non-generic solutions, skipping quite a few steps in understanding what the current situation is.
There's a reasonable amount of usage in the embedded space.
Do they use udev? And don't they need to tweak the device permissions of these input-jack devices on the fly per user?
No and no. The physical user owns the whole device and typically there aren't user accounts in that sense on the system.
At Mon, 20 Jun 2011 18:31:01 +0100, Mark Brown wrote:
On Mon, Jun 20, 2011 at 07:12:21PM +0200, Takashi Iwai wrote:
Mark Brown wrote:
On Mon, Jun 20, 2011 at 03:37:25PM +0200, David Henningsson wrote:
I'm still new in the community in that sense that I'm not sure how decisions are made. But I could use the outcome of such a decision.
So, I'm still not 100% sure what the actual technical issue is here?
The device permissions. They are set inaccessible from the normal desktop user as default.
Right, I got that much but what I was trying to say in the rest of the paragraph was that it wasn't immediately obvious to me that this was due to needing to do something special for audio jacks. That's certainly a symptom but what's not clear to me is why we think the current setup is a good call for the generic input device that udev didn't have specific magic for. We seem to have jumped straight into some very non-generic solutions, skipping quite a few steps in understanding what the current situation is.
True. OTOH, the input-jack devices of the sound driver belong to the sound device (from device-tree POV), so it's natural to take the similar control for them as the sound devices. So, I initially thought it'd be OK in that way. But, maybe the situation will change when the jack layer is used more widely for other device classes.
There's a reasonable amount of usage in the embedded space.
Do they use udev? And don't they need to tweak the device permissions of these input-jack devices on the fly per user?
No and no. The physical user owns the whole device and typically there aren't user accounts in that sense on the system.
OK, thanks for clarification.
Takashi
On 2011-06-20 19:07, Mark Brown wrote:
On Mon, Jun 20, 2011 at 03:37:25PM +0200, David Henningsson wrote:
As part of that I wrote a udev patch a few days ago, which nobody commented on in alsa-devel [1], but was somewhat disliked by at
Having dug out the mailing list archive I guess the fact that you just posted it to the list and didn't CC anyone on the patch didn't help here.
Therefore I cc:ed more people on this thread, which seems to at least got more attention, but risk falling into the ditch of endless discussion instead. Please prove me wrong :-)
Looking at the patch the main thing that jumps out at me without any knowledge of the udev code is that the patch will end up classifying video output jacks as audio even if they've no audio capability which is obviously not correct.
In suggested implementation, they will only be marked audio jacks if their parent object is a sound card.
I still don't entirely understand the technical issue you're trying to address here - this doesn't seem specific to audio.
I think this is a difference between embedded space and desktop space. In embedded space, a hardware designer can decide to connect the "take a camera picture" button to "line in jack detect" just because that saves him an extra GPIO chip (and "line in" isn't connected anyway). Those things don't happen on normal PC [1], but instead, things must be auto-detectable.
As far as I can tell the issue is that some of the input devices on the system aren't being made available to the console user, presumably because there are some that shouldn't be made available to them, so the issue is that the heuristics that udev uses to decide if an input device should be root only aren't working correctly in at least this case. It feels like if we understood why the heuristics are making a bad call here we might be able to come up with a better solution.
AFAICT, the current "heuristics", is to assign all input devices to root and root only.
For options 2a) and 2b) I guess the existing /dev/input thing should be deprecated and/or removed. So part of decision should maybe be based on information about how widespread the usage of these devices are currently...?
There's a reasonable amount of usage in the embedded space.
But maintaining two different implementations of input jacks without at least strongly deprecating one of them, lead to application programmers being confused, kernel being big and bloated, and so on...
On Mon, Jun 20, 2011 at 08:53:46PM +0200, David Henningsson wrote:
On 2011-06-20 19:07, Mark Brown wrote:
On Mon, Jun 20, 2011 at 03:37:25PM +0200, David Henningsson wrote:
Looking at the patch the main thing that jumps out at me without any knowledge of the udev code is that the patch will end up classifying video output jacks as audio even if they've no audio capability which is obviously not correct.
In suggested implementation, they will only be marked audio jacks if their parent object is a sound card.
Oh, then in that case we've got another issue in that jacks that happen not to be associated directly with a sound card for some reason (they're implemented by the embedded controller and exposed via ACPI for example) won't be made available to applications.
I still don't entirely understand the technical issue you're trying to address here - this doesn't seem specific to audio.
I think this is a difference between embedded space and desktop space. In embedded space, a hardware designer can decide to connect the "take a camera picture" button to "line in jack detect" just because that saves him an extra GPIO chip (and "line in" isn't connected anyway). Those things don't happen on normal PC [1], but instead, things must be auto-detectable.
I'm sorry but I'm not sure I follow the connection between the text you've written above and the point made in the text you're quoting. The physical implementation of the hardware doesn't seem strongly related to how Linux distributions manage permissions for the interfaces exposed by the drivers that manage that hardware.
If you're talking about the support for buttons that's nothing to do with wiring random unrelated controls to jack detection circuits (which would be highly unusual as pretty much any detection specific electronics are highly specialised and difficult to use for anything else). It's there because most headsets have at least one button on them, implemented by shorting the mic to ground and used for play/pause/call functionality, and some have more complex arrangements. I've got a laptop sitting on this table which implements such functionality.
As far as the implementation stuff goes you do get all sorts of odd stuff on PCs, anything you've done that involves ACPI interaction (like the Thinkpad extra volume control that was being discussed recently) is going to be that sort of oddity.
only aren't working correctly in at least this case. It feels like if we understood why the heuristics are making a bad call here we might be able to come up with a better solution.
AFAICT, the current "heuristics", is to assign all input devices to root and root only.
OK, well that doesn't seem like an immediately obvious choice. I can imagine there might be some concern around keyboards due to passwords but in general it doesn't seem obvious that the console user shouldn't be able to read physical input devices on the box (which is the case we're trying to implement here; we don't need write support). Do all classes of input device currently have some root only service to mediate access to them, and if that is the model we're using perhaps it'd make sense to have one for this with a dbus interface or whatever that PulseAudio can talk do rather than to have it talking direct to the device?
For options 2a) and 2b) I guess the existing /dev/input thing should be deprecated and/or removed. So part of decision should maybe be based on information about how widespread the usage of these devices are currently...?
There's a reasonable amount of usage in the embedded space.
But maintaining two different implementations of input jacks without at least strongly deprecating one of them, lead to application programmers being confused, kernel being big and bloated, and so on...
"But"?
On 2011-06-21 01:40, Mark Brown wrote:
On Mon, Jun 20, 2011 at 08:53:46PM +0200, David Henningsson wrote:
On 2011-06-20 19:07, Mark Brown wrote:
On Mon, Jun 20, 2011 at 03:37:25PM +0200, David Henningsson wrote:
Looking at the patch the main thing that jumps out at me without any knowledge of the udev code is that the patch will end up classifying video output jacks as audio even if they've no audio capability which is obviously not correct.
In suggested implementation, they will only be marked audio jacks if their parent object is a sound card.
Oh, then in that case we've got another issue in that jacks that happen not to be associated directly with a sound card for some reason (they're implemented by the embedded controller and exposed via ACPI for example) won't be made available to applications.
Which is not a problem IMO, because in the desktop world this does not happen (or at least I've never seen it), and in the embedded world you have PA running as root anyway.
I still don't entirely understand the technical issue you're trying to address here - this doesn't seem specific to audio.
I think this is a difference between embedded space and desktop space. In embedded space, a hardware designer can decide to connect the "take a camera picture" button to "line in jack detect" just because that saves him an extra GPIO chip (and "line in" isn't connected anyway). Those things don't happen on normal PC [1], but instead, things must be auto-detectable.
I'm sorry but I'm not sure I follow the connection between the text you've written above and the point made in the text you're quoting. The physical implementation of the hardware doesn't seem strongly related to how Linux distributions manage permissions for the interfaces exposed by the drivers that manage that hardware.
If you're talking about the support for buttons that's nothing to do with wiring random unrelated controls to jack detection circuits (which would be highly unusual as pretty much any detection specific electronics are highly specialised and difficult to use for anything else). It's there because most headsets have at least one button on them, implemented by shorting the mic to ground and used for play/pause/call functionality, and some have more complex arrangements. I've got a laptop sitting on this table which implements such functionality.
As far as the implementation stuff goes you do get all sorts of odd stuff on PCs, anything you've done that involves ACPI interaction (like the Thinkpad extra volume control that was being discussed recently) is going to be that sort of oddity.
If you ask me, I think the Thinkpad stuff should be implemented with connections between the hda-intel and thinkpad-acpi driver inside the kernel, and we can leave userspace out of it - userspace would just see the hda-intel card, possibly with an extra mute or volume control. But that's another story.
only aren't working correctly in at least this case. It feels like if we understood why the heuristics are making a bad call here we might be able to come up with a better solution.
AFAICT, the current "heuristics", is to assign all input devices to root and root only.
OK, well that doesn't seem like an immediately obvious choice. I can imagine there might be some concern around keyboards due to passwords but in general it doesn't seem obvious that the console user shouldn't be able to read physical input devices on the box (which is the case we're trying to implement here; we don't need write support). Do all classes of input device currently have some root only service to mediate access to them, and if that is the model we're using perhaps it'd make sense to have one for this with a dbus interface or whatever that PulseAudio can talk do rather than to have it talking direct to the device?
If I look at what /dev/input devices I have here, that's keyboard, mouse, and ACPI buttons (power, lid close etc). AFAIK, all of those go through the X input layer. Play/pause keys should go through there as well, but are you saying that audio jack events should also go through the X input layer?
If you are, you're welcome to try to fit it in the 248 keycodes that are already occupied, design a new X input extension, or whatever is the right solution for that. I don't know myself.
If you're not, then that's why this is specific to audio.
For options 2a) and 2b) I guess the existing /dev/input thing should be deprecated and/or removed. So part of decision should maybe be based on information about how widespread the usage of these devices are currently...?
There's a reasonable amount of usage in the embedded space.
But maintaining two different implementations of input jacks without at least strongly deprecating one of them, lead to application programmers being confused, kernel being big and bloated, and so on...
"But"?
Do you agree that maintaining two different implementations of input jacks is something we should avoid?
On Tue, Jun 21, 2011 at 02:11:15PM +0200, David Henningsson wrote:
On 2011-06-21 01:40, Mark Brown wrote:
Oh, then in that case we've got another issue in that jacks that happen not to be associated directly with a sound card for some reason (they're implemented by the embedded controller and exposed via ACPI for example) won't be made available to applications.
Which is not a problem IMO, because in the desktop world this does not happen (or at least I've never seen it), and in the embedded world you have PA running as root anyway.
This seems rather short sighted - the desktop and embedded worlds aren't as clearly divided as all that, you've already got things like the Genesi systems on the market at the minute running Ubuntu on ARM hardware for example and you've also got a bunch of laptop manufacturers putting ARM systems in their systems to dual boot into.
OK, well that doesn't seem like an immediately obvious choice. I can imagine there might be some concern around keyboards due to passwords but in general it doesn't seem obvious that the console user shouldn't be able to read physical input devices on the box (which is the case we're trying to implement here; we don't need write support). Do all classes of input device currently have some root only service to mediate access to them, and if that is the model we're using perhaps it'd make sense to have one for this with a dbus interface or whatever that PulseAudio can talk do rather than to have it talking direct to the device?
If I look at what /dev/input devices I have here, that's keyboard, mouse, and ACPI buttons (power, lid close etc). AFAIK, all of those go through the X input layer. Play/pause keys should go through there as well, but are you saying that audio jack events should also go through the X input layer?
I'm asking what the sensible model for handling this stuff is and why we need to use a different model for this. Perhaps this should go through the X server, perhaps somewhere else.
It does strike me that the UI may want to be able to join the buttons up with the particular jack - for example, if you've got a headset that you've got assigned to telephony functions only perhaps the mic short button on that headset should only do telephony things.
If you are, you're welcome to try to fit it in the 248 keycodes that are already occupied, design a new X input extension, or whatever is the right solution for that. I don't know myself.
If you're not, then that's why this is specific to audio.
It doesn't strike me as an obviously awful solution if that's what we're using for things like the ACPI buttons which seem pretty similar - like I say one of my questions here is why we need to do something domain specific here. There are certainly a bunch of other switch types in the input API which look like the desktop might want to look at them (things like SW_DOCK and SW_ROTATE_LOCK for example) so presumably we ought to handle them somehow too.
It does sound like we need some input from the udev and more general userspace stack people about how this stuff is usually handled.
For options 2a) and 2b) I guess the existing /dev/input thing should be deprecated and/or removed. So part of decision should maybe be based on information about how widespread the usage of these devices are currently...?
There's a reasonable amount of usage in the embedded space.
But maintaining two different implementations of input jacks without at least strongly deprecating one of them, lead to application programmers being confused, kernel being big and bloated, and so on...
"But"?
Do you agree that maintaining two different implementations of input jacks is something we should avoid?
Show me the interfaces and I'll tell you if they look like a bad idea. Do we have two identical APIs or do we have two APIs that sit next to each other, more providing more detail on the data from the other?
I don't see what that's got to do with what I said above, though?
On 2011-06-21 14:39, Mark Brown wrote:
On Tue, Jun 21, 2011 at 02:11:15PM +0200, David Henningsson wrote:
On 2011-06-21 01:40, Mark Brown wrote:
Oh, then in that case we've got another issue in that jacks that happen not to be associated directly with a sound card for some reason (they're implemented by the embedded controller and exposed via ACPI for example) won't be made available to applications.
Which is not a problem IMO, because in the desktop world this does not happen (or at least I've never seen it), and in the embedded world you have PA running as root anyway.
This seems rather short sighted - the desktop and embedded worlds aren't as clearly divided as all that, you've already got things like the Genesi systems on the market at the minute running Ubuntu on ARM hardware for example and you've also got a bunch of laptop manufacturers putting ARM systems in their systems to dual boot into.
If you have a better idea about how to determine if a SW_VIDEOOUT jack is actually an audio jack or not, let me know.
OK, well that doesn't seem like an immediately obvious choice. I can imagine there might be some concern around keyboards due to passwords but in general it doesn't seem obvious that the console user shouldn't be able to read physical input devices on the box (which is the case we're trying to implement here; we don't need write support). Do all classes of input device currently have some root only service to mediate access to them, and if that is the model we're using perhaps it'd make sense to have one for this with a dbus interface or whatever that PulseAudio can talk do rather than to have it talking direct to the device?
If I look at what /dev/input devices I have here, that's keyboard, mouse, and ACPI buttons (power, lid close etc). AFAIK, all of those go through the X input layer. Play/pause keys should go through there as well, but are you saying that audio jack events should also go through the X input layer?
I'm asking what the sensible model for handling this stuff is and why we need to use a different model for this. Perhaps this should go through the X server, perhaps somewhere else.
It does strike me that the UI may want to be able to join the buttons up with the particular jack - for example, if you've got a headset that you've got assigned to telephony functions only perhaps the mic short button on that headset should only do telephony things.
If you are, you're welcome to try to fit it in the 248 keycodes that are already occupied, design a new X input extension, or whatever is the right solution for that. I don't know myself.
If you're not, then that's why this is specific to audio.
It doesn't strike me as an obviously awful solution if that's what we're using for things like the ACPI buttons which seem pretty similar - like I say one of my questions here is why we need to do something domain specific here. There are certainly a bunch of other switch types in the input API which look like the desktop might want to look at them (things like SW_DOCK and SW_ROTATE_LOCK for example) so presumably we ought to handle them somehow too.
It does sound like we need some input from the udev and more general userspace stack people about how this stuff is usually handled.
I was hoping for Lennart or Kay to answer here as they are more "udev and general userspace stack people" than I am, but lacking that, I can only notice that on my machine, the only processes listening to /dev/input files are upowerd (for the lid switch) and Xorg (for everything else). [1] Acpid seems to be able to listen to these events too, but does not seem to do that on my machine (as determined by "sudo fuser -v").
Xorg can only handle keypresses and mouse events, and AFAICT don't care about switch events, so, if the way things are usually handled is to add another u-audiojack-d just to forward events to pulseaudio, I would prefer giving the user access to the relevant input devices directly, out of pure simplicity.
On Wed, Jun 22, 2011 at 12:47:35PM +0200, David Henningsson wrote:
On 2011-06-21 14:39, Mark Brown wrote:
On Tue, Jun 21, 2011 at 02:11:15PM +0200, David Henningsson wrote:
[Using the parent device to work out if a jack is an audio jack.]
If you have a better idea about how to determine if a SW_VIDEOOUT jack is actually an audio jack or not, let me know.
I'd have done this by checking for any audio capabilities - if the jack can detect an audio input or output then it's an audio capable jack.
It does sound like we need some input from the udev and more general userspace stack people about how this stuff is usually handled.
I was hoping for Lennart or Kay to answer here as they are more "udev and general userspace stack people" than I am, but lacking
Yes, me too. Kay, does any of this ring any bells from a udev point of view?
that, I can only notice that on my machine, the only processes listening to /dev/input files are upowerd (for the lid switch) and Xorg (for everything else). [1] Acpid seems to be able to listen to these events too, but does not seem to do that on my machine (as determined by "sudo fuser -v").
Hrm, right. So upowerd is an example of a little daemon running as root which provides an interface to the applications running as users separately to the X server, and it was added recently too so it should be an example of current best practice. However with the lid there is an additional motivation for doing this as a system daemon as we need to do power handling even when there is nobody logged in which is less of an issue for audio especially with the current model we have.
Xorg can only handle keypresses and mouse events, and AFAICT don't care about switch events, so, if the way things are usually handled is to add another u-audiojack-d just to forward events to pulseaudio, I would prefer giving the user access to the relevant input devices directly, out of pure simplicity.
I tend to agree. I think my preferences would be to open up access to input (or possibly just switch) devices in general, or if there's a good reason not to do that then do the daemon thing.
On Wed, Jun 22, 2011 at 13:48, Mark Brown broonie@opensource.wolfsonmicro.com wrote:
On Wed, Jun 22, 2011 at 12:47:35PM +0200, David Henningsson wrote:
On 2011-06-21 14:39, Mark Brown wrote:
On Tue, Jun 21, 2011 at 02:11:15PM +0200, David Henningsson wrote:
[Using the parent device to work out if a jack is an audio jack.]
If you have a better idea about how to determine if a SW_VIDEOOUT jack is actually an audio jack or not, let me know.
I'd have done this by checking for any audio capabilities - if the jack can detect an audio input or output then it's an audio capable jack.
It does sound like we need some input from the udev and more general userspace stack people about how this stuff is usually handled.
I was hoping for Lennart or Kay to answer here as they are more "udev and general userspace stack people" than I am, but lacking
Yes, me too. Â Kay, does any of this ring any bells from a udev point of view?
At this point, I wouldn't really expect udev to be directly involved. Udev does not really know anything about audio, video, buttons or input devices.
I'm still pretty sure that there should be no fake input devices involved for any jack related things. Especially not in the simple and common desktop case, where the input devices are created by, and direct child devices of the ALSA card device (like Intel HDA). I'm sure, such things just belong to the ALSA control device directly.
We should get the needed patches Takashi already has, refreshed and merged. For PulseAudio we would just add the support for the control device instead of input device support. Udev would not do anything special here, just manage the access permissions for the control device like it already does since years.
There might be odd use cases or use cases where multiple device classes are involved for the same type of jacks, which can probably not be covered by the ALSA-only control device, and where some video/multimedia event channel makes sense too. I don't necessarily see a conflict with the ALSA control device event here, not even with events for the jacks coming out of ALSA control and the multimedia device at the same time.
To me the ALSA control device events for audio-related jack events seems like the natural interface and the best option currently. Unless there are good technical reasons not to do that, I like to see them made working, instead of adding support for input devices, or wait for some (magic) meta multimedia device to be defined, or an additional userspace event daemon requirement.
Thanks, Kay
On Wed, Jun 22, 2011 at 02:50:59PM +0200, Kay Sievers wrote:
I'm still pretty sure that there should be no fake input devices involved for any jack related things. Especially not in the simple and common desktop case, where the input devices are created by, and direct child devices of the ALSA card device (like Intel HDA). I'm sure, such things just belong to the ALSA control device directly.
It would really helpful if you could engage with some of the discussion about this... Apart from anything else "fake" seems a bit misleading, a jack is a physical thing which really exists and which I can point to on the system and in may cases has buttons on it.
There might be odd use cases or use cases where multiple device classes are involved for the same type of jacks, which can probably not be covered by the ALSA-only control device, and where some
Like I said in reply to your earlier mail on this subject that's not a terribly unusual case for PC class hardware, HDMI is very widely deployed and obviously carries both audio and video over a single physical connector.
To me the ALSA control device events for audio-related jack events seems like the natural interface and the best option currently. Unless there are good technical reasons not to do that, I like to see them made working, instead of adding support for input devices, or wait for some (magic) meta multimedia device to be defined, or an additional userspace event daemon requirement.
To summarise some of the issues from the rest of the thread:
- This isn't a new interface at all, it's been implemented in the kernel for quite some time now (the first switch was added in 2.6.18). All that's changed here is that PulseAudio is trying to use the information that the kernel is exporting. - Mixed function jacks aren't that unusual; if anything they're more common on PCs than anything else. HDMI is the obvious one for PC class hardware and where these things happen we do need to be able to join the audio and the video up with each other. - Many jacks include buttons so need an input device anyway which again we want to join up with the other functions. - We still need to handle cases where the jack isn't particularly closely connected to the audio subsystem. - There are other non-audio non-jack things being exposed via the same interface which we need to have some method for handling even if we end up doing something audio specific for some subset of jacks.
and probably some others I forget.
On Wed, Jun 22, 2011 at 15:25, Mark Brown broonie@opensource.wolfsonmicro.com wrote:
On Wed, Jun 22, 2011 at 02:50:59PM +0200, Kay Sievers wrote:
I'm still pretty sure that there should be no fake input devices involved for any jack related things. Especially not in the simple and common desktop case, where the input devices are created by, and direct child devices of the ALSA card device (like Intel HDA). I'm sure, such things just belong to the ALSA control device directly.
It would really helpful if you could engage with some of the discussion about this... Â Apart from anything else "fake" seems a bit misleading, a jack is a physical thing which really exists and which I can point to on the system and in may cases has buttons on it.
Physical does not imply button, and even then, button does not imply input. It's really a domain of the individual subsystem it is associated with. Cdrom media eject buttons are no input devices, the events come out of SCSI.
We should stop misusing input for anything that isn't user a real input device like a keyboard button, mouse, joystick, screen, tablet like interface. Otherwise, I fear, with that logic anything with a physical state change could become an input device. USB would then make a good case too, to have all ports as individual input devices, telling us if anything is plugged in or out, and so on ...
The kernel's input just has some sensible event relay interface, that should not be the reason to put everything that has a state and can change its state as an input device. People should come up with their own event channel instead of just lazily misuse input for unrelated things.
There might be odd use cases or use cases where multiple device classes are involved for the same type of jacks, which can probably not be covered by the ALSA-only control device, and where some
Like I said in reply to your earlier mail on this subject that's not a terribly unusual case for PC class hardware, HDMI is very widely deployed and obviously carries both audio and video over a single physical connector.
HDMI has no buttons, no virtual ones no physical ones. :)
HDMI exposes an ALSA interface if audio is involved, and that ALSA can send events, just as the associated video interface can send video related events. That there is a single connector does again not make a case to use input for it, it's an hot-pluggable audio/video connection, not an input device.
To me the ALSA control device events for audio-related jack events seems like the natural interface and the best option currently. Unless there are good technical reasons not to do that, I like to see them made working, instead of adding support for input devices, or wait for some (magic) meta multimedia device to be defined, or an additional userspace event daemon requirement.
To summarise some of the issues from the rest of the thread:
- This isn't a new interface at all, it's been implemented in the  kernel for quite some time now (the first switch was added in  2.6.18).
Which does not make it correct, or any more tasteful. It's just wrong to misuse input for subsystem specific plugging events not related to input.
If these devices should be removed ever, or at what time if so, is nothing we need to discuss now. But we should not start using them in _new_ stuff. They are just wrong from the beginning.
All that's changed here is that PulseAudio is trying to  use the information that the kernel is exporting.
Right, And we don't want to use mis-designed interfaces if we don't need to. And as we are not in a hurry and have the option of a native ALSA control interface, we should focus on that.
- Mixed function jacks aren't that unusual; if anything they're more  common on PCs than anything else.  HDMI is the obvious one for PC  class hardware and where these things happen we do need to be able to  join the audio and the video up with each other.
Sure. But again, HDMI has in no way any button or input. This is about hot-plugging of audio/video devices. Just let ALSA _and_ the video device send the event.
- Many jacks include buttons so need an input device anyway which again  we want to join up with the other functions.
I disagree. They are states for the associated class of devices, they are not sensible input devices today and probably will never really be such.
- We still need to handle cases where the jack isn't particularly  closely connected to the audio subsystem.
Sure, that might need to be solved by some mapping, still makes no case to misuse input. In many modern connector cases the jack senses are not even buttons anymore, and they should not become buttons for userspace.
- There are other non-audio non-jack things being exposed via the same  interface which we need to have some method for handling even if we  end up doing something audio specific for some subset of jacks.
and probably some others I forget.
Sure, they all might need to rethink their use of input, and ideally stop doing that. I expect most of them are not right doing what they do.
Thanks, Kay
On Wed, Jun 22, 2011 at 03:55:46PM +0200, Kay Sievers wrote:
On Wed, Jun 22, 2011 at 15:25, Mark Brown
It would really helpful if you could engage with some of the discussion about this... Â Apart from anything else "fake" seems a bit misleading, a jack is a physical thing which really exists and which I can point to on the system and in may cases has buttons on it.
We should stop misusing input for anything that isn't user a real input device like a keyboard button, mouse, joystick, screen, tablet like interface. Otherwise, I fear, with that logic anything with a physical state change could become an input device. USB would then make a good case too, to have all ports as individual input devices, telling us if anything is plugged in or out, and so on ...
The idea of reporting presence status for USB ports doesn't seem obviously bad.
The kernel's input just has some sensible event relay interface, that should not be the reason to put everything that has a state and can change its state as an input device. People should come up with their own event channel instead of just lazily misuse input for unrelated things.
So if what you want to do is create a new API for this sort of switch rather than do some audio specific thing like you keep saying I guess we could merge some version of the switch API that the Android guys wrote because they didn't notice the existing functionality. Currently it'd need quite a bit of work from what I remember, it's very basic, and there's going to be a transition period where we move all or most of the various existing users of switches out of the input API.
One can, of course, flip this around and say that what we should really be doing is factoring the event delivery mechanism out of the input layer and into a generic subsystem where it can be reused so we don't have to reinvent the wheel. The transition issues do apply there too but probably less so and it seems like we'd end up with less redundant code.
If we did one of those two things we would still need to deal with all the same permission management issues that we've got for input devices at some point.
HDMI exposes an ALSA interface if audio is involved, and that ALSA can send events, just as the associated video interface can send video related events. That there is a single connector does again not make a case to use input for it, it's an hot-pluggable audio/video connection, not an input device.
There's two orthogonal issues here which you keep confusing.
One is that you keep saying to add a new audio specific API to replace the existing API. This seems like a clear loss to me, we're currently able to represent multi function jacks as a single object to the application layer and we'd loose that ability. That seems like a failure.
The other is that you don't like the fact that the input layer has been used for this stuff. This seems like a more valid point, the main issues are implementation and deployment, but you keep suggesting an audio specific solution.
- This isn't a new interface at all, it's been implemented in the  kernel for quite some time now (the first switch was added in  2.6.18).
Which does not make it correct, or any more tasteful. It's just wrong to misuse input for subsystem specific plugging events not related to input.
You keep saying subsystem specific here - like I keep on saying I don't think that reflects the reality of the hardware we've got currently.
If these devices should be removed ever, or at what time if so, is nothing we need to discuss now. But we should not start using them in _new_ stuff. They are just wrong from the beginning.
Well, deciding that we want to ditch the current API doesn't really help people like David who are trying to use the current kernel - userspace has a lot of the information right now, it's just having a hard time figuring out how to talk to itself. That's a third mostly orthogonal problem.
All that's changed here is that PulseAudio is trying to  use the information that the kernel is exporting.
Right, And we don't want to use mis-designed interfaces if we don't need to. And as we are not in a hurry and have the option of a native ALSA control interface, we should focus on that.
I don't think it makes any sense to push for an audio specific solution.
- Mixed function jacks aren't that unusual; if anything they're more  common on PCs than anything else.  HDMI is the obvious one for PC  class hardware and where these things happen we do need to be able to  join the audio and the video up with each other.
Sure. But again, HDMI has in no way any button or input. This is about hot-plugging of audio/video devices. Just let ALSA _and_ the video device send the event.
I'm not so sure about that, I believe there's some mechanism for propagating events through HDMI links for things like volume changes which we don't currently support and I'd not be surprised if they didn't look like buttons. Besides if you don't like HDMI as an example there's also four pole analogue jacks that can do video, audio and buttons. I'm mostly just mentioning HDMI because it's a blatantly obvious example seen on PCs.
Besides, userspace needs some way to figure out that the two events are connected and that they're also connected with anything else that appared as part of the same accessory detection.
- Many jacks include buttons so need an input device anyway which again  we want to join up with the other functions.
I disagree. They are states for the associated class of devices, they are not sensible input devices today and probably will never really be such.
Please read what I wrote, this is for *buttons*. For example, I'm sitting here with a headset that has play/pause, rewind and fast forward buttons on it. Pretty much every headset used with phones has at least one such button.
- We still need to handle cases where the jack isn't particularly  closely connected to the audio subsystem.
Sure, that might need to be solved by some mapping, still makes no
So again why the push to come up with an audio specific solution? We can see right now that it's got some issues with current hardware.
case to misuse input. In many modern connector cases the jack senses are not even buttons anymore, and they should not become buttons for userspace.
I do have some passing familiarity with the physical implementations, thanks. Note that the presence detection stuff is all reported as *switches* rather than buttons.
On Wed, Jun 22, 2011 at 04:11:30PM +0100, Mark Brown wrote:
On Wed, Jun 22, 2011 at 03:55:46PM +0200, Kay Sievers wrote:
On Wed, Jun 22, 2011 at 15:25, Mark Brown
It would really helpful if you could engage with some of the discussion about this... Â Apart from anything else "fake" seems a bit misleading, a jack is a physical thing which really exists and which I can point to on the system and in may cases has buttons on it.
We should stop misusing input for anything that isn't user a real input device like a keyboard button, mouse, joystick, screen, tablet like interface. Otherwise, I fear, with that logic anything with a physical state change could become an input device. USB would then make a good case too, to have all ports as individual input devices, telling us if anything is plugged in or out, and so on ...
The idea of reporting presence status for USB ports doesn't seem obviously bad.
And so is reporting whether network cable is plugged in. Does not mean that it should be routed through input though.
The kernel's input just has some sensible event relay interface, that should not be the reason to put everything that has a state and can change its state as an input device. People should come up with their own event channel instead of just lazily misuse input for unrelated things.
So if what you want to do is create a new API for this sort of switch rather than do some audio specific thing like you keep saying I guess we could merge some version of the switch API that the Android guys wrote because they didn't notice the existing functionality. Currently it'd need quite a bit of work from what I remember, it's very basic, and there's going to be a transition period where we move all or most of the various existing users of switches out of the input API.
I do not think you want to have switch API, it is more a 'connection' API that is needed (i.e. we need a way to report and query whether something is connected to something else or not).
One can, of course, flip this around and say that what we should really be doing is factoring the event delivery mechanism out of the input layer and into a generic subsystem where it can be reused so we don't have to reinvent the wheel. The transition issues do apply there too but probably less so and it seems like we'd end up with less redundant code.
If we did one of those two things we would still need to deal with all the same permission management issues that we've got for input devices at some point.
Hmm, the connection changes should normally be infrequent; maybe delivering them over netlink in the same fashion we deliver uevents woudl make sense. In fact, maybe uevents are all that is needed here.
HDMI exposes an ALSA interface if audio is involved, and that ALSA can send events, just as the associated video interface can send video related events. That there is a single connector does again not make a case to use input for it, it's an hot-pluggable audio/video connection, not an input device.
There's two orthogonal issues here which you keep confusing.
One is that you keep saying to add a new audio specific API to replace the existing API. This seems like a clear loss to me, we're currently able to represent multi function jacks as a single object to the application layer and we'd loose that ability. That seems like a failure.
The other is that you don't like the fact that the input layer has been used for this stuff. This seems like a more valid point, the main issues are implementation and deployment, but you keep suggesting an audio specific solution.
- This isn't a new interface at all, it's been implemented in the  kernel for quite some time now (the first switch was added in  2.6.18).
Which does not make it correct, or any more tasteful. It's just wrong to misuse input for subsystem specific plugging events not related to input.
You keep saying subsystem specific here - like I keep on saying I don't think that reflects the reality of the hardware we've got currently.
If these devices should be removed ever, or at what time if so, is nothing we need to discuss now. But we should not start using them in _new_ stuff. They are just wrong from the beginning.
Well, deciding that we want to ditch the current API doesn't really help people like David who are trying to use the current kernel - userspace has a lot of the information right now, it's just having a hard time figuring out how to talk to itself. That's a third mostly orthogonal problem.
All that's changed here is that PulseAudio is trying to  use the information that the kernel is exporting.
Right, And we don't want to use mis-designed interfaces if we don't need to. And as we are not in a hurry and have the option of a native ALSA control interface, we should focus on that.
I don't think it makes any sense to push for an audio specific solution.
- Mixed function jacks aren't that unusual; if anything they're more  common on PCs than anything else.  HDMI is the obvious one for PC  class hardware and where these things happen we do need to be able to  join the audio and the video up with each other.
Sure. But again, HDMI has in no way any button or input. This is about hot-plugging of audio/video devices. Just let ALSA _and_ the video device send the event.
I'm not so sure about that, I believe there's some mechanism for propagating events through HDMI links for things like volume changes which we don't currently support and I'd not be surprised if they didn't look like buttons. Besides if you don't like HDMI as an example there's also four pole analogue jacks that can do video, audio and buttons. I'm mostly just mentioning HDMI because it's a blatantly obvious example seen on PCs.
Besides, userspace needs some way to figure out that the two events are connected and that they're also connected with anything else that appared as part of the same accessory detection.
- Many jacks include buttons so need an input device anyway which again  we want to join up with the other functions.
I disagree. They are states for the associated class of devices, they are not sensible input devices today and probably will never really be such.
Please read what I wrote, this is for *buttons*. For example, I'm sitting here with a headset that has play/pause, rewind and fast forward buttons on it. Pretty much every headset used with phones has at least one such button.
- We still need to handle cases where the jack isn't particularly  closely connected to the audio subsystem.
Sure, that might need to be solved by some mapping, still makes no
So again why the push to come up with an audio specific solution? We can see right now that it's got some issues with current hardware.
case to misuse input. In many modern connector cases the jack senses are not even buttons anymore, and they should not become buttons for userspace.
I do have some passing familiarity with the physical implementations, thanks. Note that the presence detection stuff is all reported as *switches* rather than buttons.
Buttons or switches does not really make much difference. They are not really HID devices (unless there are physical switches that can be toggled by the user while device is still plugged into a socket).
On Wed, Jun 22, 2011 at 02:41:54PM -0700, Dmitry Torokhov wrote:
On Wed, Jun 22, 2011 at 04:11:30PM +0100, Mark Brown wrote:
The idea of reporting presence status for USB ports doesn't seem obviously bad.
And so is reporting whether network cable is plugged in. Does not mean that it should be routed through input though.
Like I said in the message you're replying to I don't really disagree with that. There's a whole section of the mail (which you quoted but I didn't respond to) where I tried to clarify the several issues here. Just to reiterate once more it's not the fact that people don't like the input API, it's the fact that people are proposing solutions which are obviously less functional than what we've got right now.
So if what you want to do is create a new API for this sort of switch rather than do some audio specific thing like you keep saying I guess we could merge some version of the switch API that the Android guys wrote
I do not think you want to have switch API, it is more a 'connection' API that is needed (i.e. we need a way to report and query whether something is connected to something else or not).
The reason I called it switch is because that's what both the existing Android API and the events within input are called. It's also not just a case of reporting if things are connected, we need to group these things together into bundles and link them to multiple other devices in the system.
If we did one of those two things we would still need to deal with all the same permission management issues that we've got for input devices at some point.
Hmm, the connection changes should normally be infrequent; maybe delivering them over netlink in the same fashion we deliver uevents woudl make sense. In fact, maybe uevents are all that is needed here.
uevents are what the Android switch API uses. I think that would have some of the daemon issues that David raised, though?
case to misuse input. In many modern connector cases the jack senses are not even buttons anymore, and they should not become buttons for userspace.
I do have some passing familiarity with the physical implementations, thanks. Note that the presence detection stuff is all reported as *switches* rather than buttons.
Buttons or switches does not really make much difference. They are not really HID devices (unless there are physical switches that can be toggled by the user while device is still plugged into a socket).
There are such switches on a vast proportion of jacks out there, typically presented physically as a button, so we really do want input devices in the mix. To repeat again I'm not saying that they need to be the only way a jack is represented.
On Thu, Jun 23, 2011 at 01:15:01AM +0100, Mark Brown wrote:
On Wed, Jun 22, 2011 at 02:41:54PM -0700, Dmitry Torokhov wrote:
On Wed, Jun 22, 2011 at 04:11:30PM +0100, Mark Brown wrote:
The idea of reporting presence status for USB ports doesn't seem obviously bad.
And so is reporting whether network cable is plugged in. Does not mean that it should be routed through input though.
Like I said in the message you're replying to I don't really disagree with that. There's a whole section of the mail (which you quoted but I didn't respond to)
That is because I mostly agree with you saying that it should be a generic API, not alsa or other subsystem-specific solution.
where I tried to clarify the several issues here. Just to reiterate once more it's not the fact that people don't like the input API, it's the fact that people are proposing solutions which are obviously less functional than what we've got right now.
So if what you want to do is create a new API for this sort of switch rather than do some audio specific thing like you keep saying I guess we could merge some version of the switch API that the Android guys wrote
I do not think you want to have switch API, it is more a 'connection' API that is needed (i.e. we need a way to report and query whether something is connected to something else or not).
The reason I called it switch is because that's what both the existing Android API and the events within input are called.
And I think we should stop calling them switches, if only to avoid confusion. Input layer originally had notion of switches representing physical objects that could be toggled on an off and retain their state (unlike a button that is expected to auto-release). Later jacks were added as "switches" (which is something I never liked but end up adding snice we did not have anything better at the time - still don't).
It's also not just a case of reporting if things are connected, we need to group these things together into bundles and link them to multiple other devices in the system.
Hmm, I am not sure I follow. We usually need to calssify and group devices; there is nothing new here...
If we did one of those two things we would still need to deal with all the same permission management issues that we've got for input devices at some point.
Hmm, the connection changes should normally be infrequent; maybe delivering them over netlink in the same fashion we deliver uevents woudl make sense. In fact, maybe uevents are all that is needed here.
uevents are what the Android switch API uses. I think that would have some of the daemon issues that David raised, though?
I'll have to locate original email; I am not subscribed to alsa list.
case to misuse input. In many modern connector cases the jack senses are not even buttons anymore, and they should not become buttons for userspace.
I do have some passing familiarity with the physical implementations, thanks. Note that the presence detection stuff is all reported as *switches* rather than buttons.
Buttons or switches does not really make much difference. They are not really HID devices (unless there are physical switches that can be toggled by the user while device is still plugged into a socket).
There are such switches on a vast proportion of jacks out there, typically presented physically as a button, so we really do want input devices in the mix. To repeat again I'm not saying that they need to be the only way a jack is represented.
And I agree here as well.
Thanks.
On Thu, Jun 23, 2011 at 01:42:36AM -0700, Dmitry Torokhov wrote:
On Thu, Jun 23, 2011 at 01:15:01AM +0100, Mark Brown wrote:
a case of reporting if things are connected, we need to group these things together into bundles and link them to multiple other devices in the system.
Hmm, I am not sure I follow. We usually need to calssify and group devices; there is nothing new here...
Usually we do this per device type and by control path but what we need to do here is say that a given jack has audio, video and input functionality - it's a different grouping to our normal ones.
uevents are what the Android switch API uses. I think that would have some of the daemon issues that David raised, though?
I'll have to locate original email; I am not subscribed to alsa list.
The issue was that he would really prefer PulseAudio (which runs as a regular user) to be able to read the events directly.
On Wed, 22.06.11 14:25, Mark Brown (broonie@opensource.wolfsonmicro.com) wrote:
On Wed, Jun 22, 2011 at 02:50:59PM +0200, Kay Sievers wrote:
I'm still pretty sure that there should be no fake input devices involved for any jack related things. Especially not in the simple and common desktop case, where the input devices are created by, and direct child devices of the ALSA card device (like Intel HDA). I'm sure, such things just belong to the ALSA control device directly.
It would really helpful if you could engage with some of the discussion about this... Apart from anything else "fake" seems a bit misleading, a jack is a physical thing which really exists and which I can point to on the system and in may cases has buttons on it.
I fully agree with Kay here btw.
Input devices should really just be used for input, i.e. interfacing with humans via keyboards, mice, joystick, touchscreens and suchlike. However Jack sensing does not qualify as such. The main reason why input device exist to do jack sensing right now appears to be that there was no other infrastructure readily available for it. But that is a poor excuse, which might be acceptable early in the game where things aren't really clear yet but not for the long run. On top of that there actually always has been a more appropriate place for signalling audio jack sensing events: the alsa control layer. Because that is what Jack sensing ultimately is: audio control. And if you have it in audio control this has many benefits: for example it's in the same namespace as the other audio controls (a very weakly defined namespace, but still the same namespace), so you could by employing some rules match up mute and volume controls with their matching jacks.
On top of that Takashi has patches ready to port HDA over to jack sensing via control devices, and hence I believe this is definitely the way to go.
Jack sensing for video plugs should similarly be handled by the video layer -- and actually is handled like that for VGA hotplug already.
HDMI is a composite technology, but I don't think this should be used as excuse to implement jack sensing via an independent subsystem. If it is composite it should simply use the native sensing interfaces for its technologies, i.e. send plug notifications via both video and audio control, the way they want.
There are a number of other non-input technologies that have been bolted into the input subsystem, but that is really not an excuse to bolt even more into them.
To summarise some of the issues from the rest of the thread:
- This isn't a new interface at all, it's been implemented in the kernel for quite some time now (the first switch was added in 2.6.18). All that's changed here is that PulseAudio is trying to use the information that the kernel is exporting.
It's not a new interface, but it will be used for the first time in a big way that will find its way into generic distros. That means it is still time to get this right before this is too late.
You should also not forget that having a fucked up kernel interface usually means a lot of additional work to work around that in userspace: i.e. in PA we need some way to match up input devices with alsa control devices and specific controls. That is messy, requires knowledge and logic that would be much much simpler if we could just access jack sensing via the same interface as the rest of the audio control stuff we do.
- Mixed function jacks aren't that unusual; if anything they're more common on PCs than anything else. HDMI is the obvious one for PC class hardware and where these things happen we do need to be able to join the audio and the video up with each other.
As mentioned this is not really an excuse as nothing stops you from sending out plug events via both the audio and the video subsystem
- Many jacks include buttons so need an input device anyway which again we want to join up with the other functions.
Well, if you are speaking of headsets here with specific Play and Stop buttons then I believe those actually qualify as proper input keys, and it is the right thing to route them via the input layer. And if we do X and GNOME and Rhythmbox will actually already handle them as they should.
- We still need to handle cases where the jack isn't particularly closely connected to the audio subsystem.
Well, I am tempted to say that this should be handled in kernel. I am not convinced that it is a good idea to expose particularities of the hw design too much in userspace. In fact, I agree with David that the thinkpad laptop driver would best integrate directly with HDA so that in userspace no knowledge about their relation would be necessary.
- There are other non-audio non-jack things being exposed via the same interface which we need to have some method for handling even if we end up doing something audio specific for some subset of jacks.
This exists for VGA hotplug at least. I see no reason why this shouldn't be available for HDMI as well.
Lennart
Lennart Poettering wrote at Wednesday, June 22, 2011 3:02 PM:
On Wed, 22.06.11 14:25, Mark Brown (broonie@opensource.wolfsonmicro.com) wrote:
On Wed, Jun 22, 2011 at 02:50:59PM +0200, Kay Sievers wrote:
I'm still pretty sure that there should be no fake input devices involved for any jack related things. Especially not in the simple and common desktop case, where the input devices are created by, and direct child devices of the ALSA card device (like Intel HDA). I'm sure, such things just belong to the ALSA control device directly.
It would really helpful if you could engage with some of the discussion about this... Apart from anything else "fake" seems a bit misleading, a jack is a physical thing which really exists and which I can point to on the system and in may cases has buttons on it.
I fully agree with Kay here btw.
Input devices should really just be used for input, i.e. interfacing with humans via keyboards, mice, joystick, touchscreens and suchlike. However Jack sensing does not qualify as such. The main reason why input device exist to do jack sensing right now appears to be that there was no other infrastructure readily available for it. But that is a poor excuse, which might be acceptable early in the game where things aren't really clear yet but not for the long run. On top of that there actually always has been a more appropriate place for signalling audio jack sensing events: the alsa control layer. Because that is what Jack sensing ultimately is: audio control. And if you have it in audio control this has many benefits: for example it's in the same namespace as the other audio controls (a very weakly defined namespace, but still the same namespace), so you could by employing some rules match up mute and volume controls with their matching jacks.
On top of that Takashi has patches ready to port HDA over to jack sensing via control devices, and hence I believe this is definitely the way to go.
Jack sensing for video plugs should similarly be handled by the video layer -- and actually is handled like that for VGA hotplug already.
HDMI is a composite technology, but I don't think this should be used as excuse to implement jack sensing via an independent subsystem. If it is composite it should simply use the native sensing interfaces for its technologies, i.e. send plug notifications via both video and audio control, the way they want.
For composite devices such as HDMI, with audio and video "sub ports", user space needs to know which audio port and which video port are the same physical port. For example, when displaying UI to select an audio output device, this information is needed to display an X display ID, monitor name, etc. as an option, which is much more informative to a user than some random GPU's ALSA PCM device name/number.
It seems like this correlation would be easier to represent with a single unified jack model, with "sub ports" on each jack for audio, video, etc.
However, if the jack information is pushed into separate APIs, can they please at least have some unified naming for the physical ports, so that user-space can determine which audio jacks are the same as which video jacks etc.
Thanks!
On Wed, Jun 22, 2011 at 11:01:31PM +0200, Lennart Poettering wrote:
On Wed, 22.06.11 14:25, Mark Brown (broonie@opensource.wolfsonmicro.com) wrote:
On Wed, Jun 22, 2011 at 02:50:59PM +0200, Kay Sievers wrote:
Input devices should really just be used for input, i.e. interfacing with humans via keyboards, mice, joystick, touchscreens and
Once more: there are several mostly orthogonal issues here which are getting conflated a bit. The main ones are:
- We need to represent the fact that multiple subsystems are working with the same jack. - People don't like the fact that we're currently reporting state via the input API. - Even if we invent new interfaces for this we really ought to be able to teach userspace about existing kernels.
Like Stephen says the first point is vital and it's the major thing I'm missing with the idea of doing everything per subsystem but it's the one which you and Kay aren't really addressing. We need some way for userspace to figure out that a jack exists and that it contains multiple functionalities from multiple subsystems. If we have a proposal for how we do that which seems viable then great, but we don't have that yet.
suchlike. However Jack sensing does not qualify as such. The main reason why input device exist to do jack sensing right now appears to be that there was no other infrastructure readily available for it. But that is
No, that's not the case. If you look at the changelogs when this was added the Zaurus systems that were the main driving force had a bunch of switches in them, one of which was a microswitch for detecting physical insertion of jacks, so implementing them as switches was fairly natural. When I did the more detailed jack support within ALSA and ASoC I looked at what we were doing at the time, didn't see any massive problems with it (it probably wouldn't have been my first choice) and just worked with the existing ABI.
Personally it's not using the input API in particular that I'm concerned about, it's the fact that the way we're using it at the minute gives us a single thing to point at in userspace. Although that said some of the discussion came from the fact that looking at this does make me think we have several other problems with how we're using and handling input in both kernel and userspace right now and perhaps it's best to fix up the current situation up (at least in userspace) even if we change things later.
aren't really clear yet but not for the long run. On top of that there actually always has been a more appropriate place for signalling audio jack sensing events: the alsa control layer. Because that is what Jack sensing ultimately is: audio control. And if you have it in audio
No, not at all - it's easy to make this assumption in the basic PC space (though you do still have to dismiss HDMI) but as soon as you start to look at a wider range of systems systems you just start to see issues. If you've got a jack which only carries audio data then obviously that's all it's there for but as you start to carry additional functionality over the one connector it becomes more and more difficult to take that view.
On top of that Takashi has patches ready to port HDA over to jack sensing via control devices, and hence I believe this is definitely the way to go.
I really hope it's not just for HDA, that would again be a loss.
Jack sensing for video plugs should similarly be handled by the video layer -- and actually is handled like that for VGA hotplug already.
Actually one other issue here is that physically the jack sensing is generally done over the jack as a whole in some way so even if we don't make it user visible we're going to want some general interface within the kernel to glue stuff together.
HDMI is a composite technology, but I don't think this should be used as excuse to implement jack sensing via an independent subsystem. If it is composite it should simply use the native sensing interfaces for its technologies, i.e. send plug notifications via both video and audio control, the way they want.
That's certainly simpler for programs which only work within a single subsystem but as soon as you care about the system as a whole it becomes less clear, especially given that the proposals to do subsystem specific things haven't included any suggestion about how userspace should work out that the different objects are connected to each other.
One of the use cases I thought about with this was that if you're on a phone call with headset and simultaneously playing a movie via HDMI it's going to be important to make sure that you work out which jack any button press events came from in order to decide if you should pause the move or hang up the call.
To summarise some of the issues from the rest of the thread:
- This isn't a new interface at all, it's been implemented in the kernel for quite some time now (the first switch was added in 2.6.18). All that's changed here is that PulseAudio is trying to use the information that the kernel is exporting.
It's not a new interface, but it will be used for the first time in a big way that will find its way into generic distros. That means it is still time to get this right before this is too late.
Generic *desktop* distros. There's also the issue I see with other non-jack but similar functionality which is currently made available via the same mechanism - the question which I was asking earlier was how things currently work for them or if we're just happening to look at this one problem first.
You should also not forget that having a fucked up kernel interface usually means a lot of additional work to work around that in userspace: i.e. in PA we need some way to match up input devices with alsa control devices and specific controls. That is messy, requires knowledge and logic that would be much much simpler if we could just access jack sensing via the same interface as the rest of the audio control stuff we do.
Right, but hopefully this means you can appreciate the concerns that I and Stephen have expressed about being able to join the different bits of the jack up with each other for programs which do care about things over more than one subsystem?
- Many jacks include buttons so need an input device anyway which again we want to join up with the other functions.
Well, if you are speaking of headsets here with specific Play and Stop buttons then I believe those actually qualify as proper input keys, and it is the right thing to route them via the input layer. And if we do X and GNOME and Rhythmbox will actually already handle them as they should.
Yes, they do - on a system wide basis.
- We still need to handle cases where the jack isn't particularly closely connected to the audio subsystem.
Well, I am tempted to say that this should be handled in kernel. I am not convinced that it is a good idea to expose particularities of the hw design too much in userspace. In fact, I agree with David that the thinkpad laptop driver would best integrate directly with HDA so that in userspace no knowledge about their relation would be necessary.
Yeah, that's just one example - I was actually thinking of the sort of large many function connectors in the style of docking stations you can get (and I guess docking stations too) and need to get to play together.
- There are other non-audio non-jack things being exposed via the same interface which we need to have some method for handling even if we end up doing something audio specific for some subset of jacks.
This exists for VGA hotplug at least. I see no reason why this shouldn't be available for HDMI as well.
That's not my point - I'm talking about things like the docking station or front proximity detection that are clearly not at all related to jacks but are going out via the same interface. If jacks don't currently work in the application layer due to the way input is handled then like I said above it looks like we have a bunch of other issues we also need to cope with.
Mark Brown wrote:
We need some way for userspace to figure out that a jack exists and that it contains multiple functionalities from multiple subsystems. If we have a proposal for how we do that which seems viable then great, but we don't have that yet.
Last November I posted a patch to add ALSA entity types to the media API; it stalled due to lack of time. What's still missing is a mechanism to associate entities with ALSA controls.
Regards, Clemens
At Thu, 23 Jun 2011 09:01:33 +0200, Clemens Ladisch wrote:
Mark Brown wrote:
We need some way for userspace to figure out that a jack exists and that it contains multiple functionalities from multiple subsystems. If we have a proposal for how we do that which seems viable then great, but we don't have that yet.
Last November I posted a patch to add ALSA entity types to the media API; it stalled due to lack of time. What's still missing is a mechanism to associate entities with ALSA controls.
The association of ALSA stuff is the biggest obstacle, IMHO. The implementation of the notification mechanism itself is trivial, no matter what way is used. But, it's pretty new that you create the association between jacks and various components (PCM, control elements, whatever). Such information is completely missing in the current driver, thus this must be added newly from scratch.
Basically this was the reason I didn't go further for the jack notification implementation with ALSA control API (besides the lack of time). Which information should be coupled and how should be -- this has to be discussed in details beforehand.
thanks,
Takashi
On Thu, 23.06.11 02:10, Mark Brown (broonie@opensource.wolfsonmicro.com) wrote:
On Wed, Jun 22, 2011 at 11:01:31PM +0200, Lennart Poettering wrote:
On Wed, 22.06.11 14:25, Mark Brown (broonie@opensource.wolfsonmicro.com) wrote:
On Wed, Jun 22, 2011 at 02:50:59PM +0200, Kay Sievers wrote:
Input devices should really just be used for input, i.e. interfacing with humans via keyboards, mice, joystick, touchscreens and
Once more: there are several mostly orthogonal issues here which are getting conflated a bit. The main ones are:
- We need to represent the fact that multiple subsystems are working with the same jack.
Tbh I am not entirely convinced that this is really such an important thing. We can't even map audio controls to PCM devices which would be vastly more interesting, and we can't even do that.
I can give you a thousand of real-life usecases for wanting to match up PCM devices with controls, but the one for wanting to match up HDMI audio with HDMI video is much weaker, since machines usually have multiple PCM streams, but not multiple HDMI ports.
I am not saying that such a match-up shouldn't be possible, but I'd say it could be relatively easy to implement. One option would be to go by names. i.e. simply say that if an alsa control device is called "HDMI1 Jack Sensing", and an X11 XRANDR port is called "HDMI1", then they should be the same, and apps should comapre the first words of these names, and that both can be traced to the same udev originating device.
In fact, something similar is now already in place for external usb speakers with built-in volume keys to map X11 XInput keypresses up to PA audio devices so that we can map volume keypresses to the appropriate audio device in gnome-settings-daemon (note that the latter is not making use of this yet, as the infrastructure is still very new): for each keypress we can find the originating XInput device, from that we can query the kernel device, which we can then find in sysfs. Similarly we can find the sysfs device from PA and then do a match-up.
So, what I am saying here is that doing this cross-subsystem match-up in other areas is already possible. It's not beautiful, but it works.
If you are thinking about matching up devics across subsystems, you should not limit your focus to jack sensing events only: the above mentioned key-press use is a lot more interesting -- and is solved.
Also, I'd claim that adding an additional subsystem that covers only jack sensing would complicate things even further, since now you have to match up audio, video and input devices with this jack sensing device, instead of just matching them up directly.
- Even if we invent new interfaces for this we really ought to be able to teach userspace about existing kernels.
See, I don't buy this actually. It wouldn't be a regression if we don't support the input device based scheme in PA.
- There are other non-audio non-jack things being exposed via the same interface which we need to have some method for handling even if we end up doing something audio specific for some subset of jacks.
This exists for VGA hotplug at least. I see no reason why this shouldn't be available for HDMI as well.
That's not my point - I'm talking about things like the docking station or front proximity detection that are clearly not at all related to jacks but are going out via the same interface. If jacks don't currently work in the application layer due to the way input is handled then like I said above it looks like we have a bunch of other issues we also need to cope with.
I think bolting proximity detection and docking station stuff into the input layer is very wrong too. Both deserve probably their own kind of class devices.
Lennart
On Thu, Jun 23, 2011 at 11:49:25AM +0200, Lennart Poettering wrote:
On Thu, 23.06.11 02:10, Mark Brown (broonie@opensource.wolfsonmicro.com) wrote:
- We need to represent the fact that multiple subsystems are working with the same jack.
Tbh I am not entirely convinced that this is really such an important thing. We can't even map audio controls to PCM devices which would be vastly more interesting, and we can't even do that.
No disagreement that being able to expose the audio routing within the devices is needed.
I can give you a thousand of real-life usecases for wanting to match up PCM devices with controls, but the one for wanting to match up HDMI audio with HDMI video is much weaker, since machines usually have multiple PCM streams, but not multiple HDMI ports.
Apparently that's becoming an issue on desktops with nVidia chipsets - there's been a moderate amount of discussion on that recently on the list.
I am not saying that such a match-up shouldn't be possible, but I'd say it could be relatively easy to implement. One option would be to go by names. i.e. simply say that if an alsa control device is called "HDMI1 Jack Sensing", and an X11 XRANDR port is called "HDMI1", then they should be the same, and apps should comapre the first words of these names, and that both can be traced to the same udev originating device.
You certainly can't assume that the same device will originate all the functionality on the jack - in embedded systems the way functionality is built up from blocks on the SoC means that the control bus routing is of essentially no use in working out how the system looks to the user.
In fact, something similar is now already in place for external usb speakers with built-in volume keys to map X11 XInput keypresses up to PA audio devices so that we can map volume keypresses to the appropriate audio device in gnome-settings-daemon (note that the latter is not making use of this yet, as the infrastructure is still very new): for each keypress we can find the originating XInput device, from that we can query the kernel device, which we can then find in sysfs. Similarly we can find the sysfs device from PA and then do a match-up.
OK, can you point us to a description of what the actual mechanism is here? This sounds hopeful, though the fact that nobody knows about it isn't inspiring.
If you are thinking about matching up devics across subsystems, you should not limit your focus to jack sensing events only: the above mentioned key-press use is a lot more interesting -- and is solved.
I have *repeatedly* talked about the need to hook the button presses up with the rest of the jacks. Including in the mail you're replying to.
Also, I'd claim that adding an additional subsystem that covers only jack sensing would complicate things even further, since now you have to match up audio, video and input devices with this jack sensing device, instead of just matching them up directly.
On the other hand it also means you now have to work through any number of separate APIs (the jack could acquire other functionality beyond those three) and manually build up a picture of what's there. I like the idea that we can point application developers to a thing rather than them having to infer what's going on from a bunch of separate devices - the fact that nobody else seems to know about the support you mention is one of the potential issues with this.
- Even if we invent new interfaces for this we really ought to be able to teach userspace about existing kernels.
See, I don't buy this actually. It wouldn't be a regression if we don't support the input device based scheme in PA.
On the other hand if we go for doing something completely new it's going to push back support by I'd guess at least six months. Which doesn't seem great.
currently work in the application layer due to the way input is handled then like I said above it looks like we have a bunch of other issues we also need to cope with.
I think bolting proximity detection and docking station stuff into the input layer is very wrong too. Both deserve probably their own kind of class devices.
I don't really disagree, I'm just saying that we have a broader problem which we should probably look at those too.
Lennart Poettering wrote at Thursday, June 23, 2011 3:49 AM:
On Thu, 23.06.11 02:10, Mark Brown (broonie@opensource.wolfsonmicro.com) wrote: I can give you a thousand of real-life usecases for wanting to match up PCM devices with controls, but the one for wanting to match up HDMI audio with HDMI video is much weaker, since machines usually have multiple PCM streams, but not multiple HDMI ports.
As an example, all NVIDIA GPUs, and some chipsets, built within the last perhaps 2 or 3 years contain an HD-audio controller with 4 PCM devices. Any/all of these could be routed out to actual connectors on the board. For audio purposes, HDMI, DVI (connected to an HDMI sink), and DisplayPort connectors all can carry audio and end up being exposed identically by ALSA. In practice, it's very common to have at least 2 such digital connectors on a GPU.
I believe both Intel and AMD/ATI chipsets and GPUs that support audio are in exactly the same boat, although perhaps only supporting up to 2 or 3 PCMs total?
I am not saying that such a match-up shouldn't be possible, but I'd say it could be relatively easy to implement. One option would be to go by names. i.e. simply say that if an alsa control device is called "HDMI1 Jack Sensing", and an X11 XRANDR port is called "HDMI1", then they should be the same, and apps should comapre the first words of these names, and that both can be traced to the same udev originating device.
It'll be challenging to implement this based on device name alone. The X driver probably has enough information to know the difference between DVI, HDMI, and DP ports and hence could name ports appropriately. However, the generic HD-audio HDMI driver has basically zero knowledge of which HDA pin complexes are routed to connectors at all, or for those that are, whether they're routed to DVI, HDMI, or DP ports, and equally no idea what port number X might choose if there are say 2 DVI ports.
(well, HDMI vs. DP is pin complex property, but HDMI vs. DVI certainly isn't, and numbering order when there are n HDMI pins isn't defined, and only very recent NVIDIA GPUs mark unconnected pin complexes)
This is what I attempted to start talking about in my somewhat recent email titled " EDID-like data for audio: Port correlation between ALSA and X"; see:
http://mailman.alsa-project.org/pipermail/alsa-devel/2011-May/040186.html
The basic idea: This X vs. ALSA correlation is probably going to be through ELD data, at least internally to the kernel.
On Mon, Jun 20, 2011 at 03:37:25PM +0200, David Henningsson wrote:
- We're continuing the path with /dev/input devices
2a) We'll rewrite these devices to be read-only ALSA mixer controls
2b) We'll rewrite these devices to be something within ALSA, but not exactly mixer controls.
For options 2a) and 2b) I guess the existing /dev/input thing should be deprecated and/or removed. So part of decision should maybe be based on information about how widespread the usage of these devices are currently...?
So, this discussion seems to have ground to a halt. Summarising briefly it seems that Lennart and Kay both seem adamant that they don't want a unified jack object visible to userspace, they want to glue the objects together via some implicit means. Lennart mentioned that there is some support for at least some of this functionality in PulseAudio already but didn't provide any details yet - does anyone else know what happens here or have time to go look at the code?
In kernel I'm concerned that we'll still need to have a unified object due to the need to share the detection between multiple subsystems, if one driver can report for multiple subsystems. This seems likely to mean that we'll end up with some sort of in kernel object for many if not all jacks.
I'm also concerned that an implicit system may be hard to use from userspace, if only in terms of figuring out what the rules are for matching things up.
'Twas brillig, and Mark Brown at 27/06/11 13:07 did gyre and gimble:
On Mon, Jun 20, 2011 at 03:37:25PM +0200, David Henningsson wrote:
- We're continuing the path with /dev/input devices
2a) We'll rewrite these devices to be read-only ALSA mixer controls
2b) We'll rewrite these devices to be something within ALSA, but not exactly mixer controls.
For options 2a) and 2b) I guess the existing /dev/input thing should be deprecated and/or removed. So part of decision should maybe be based on information about how widespread the usage of these devices are currently...?
So, this discussion seems to have ground to a halt. Summarising briefly it seems that Lennart and Kay both seem adamant that they don't want a unified jack object visible to userspace, they want to glue the objects together via some implicit means. Lennart mentioned that there is some support for at least some of this functionality in PulseAudio already but didn't provide any details yet - does anyone else know what happens here or have time to go look at the code?
So AFAIK, there is no support in PulseAudio per-se for this.
My understanding is that the component which deals with handling the volume keyboard controls (e.g. gnome-settings-daemon?) is the bit that actually does this glueing....
I believe it works like this:
1. It gets an keyboard event for VOLUME_UP. 2. It checks which device this keypress came from. 3. If it's a generic keyboard or the laptop keyboard, then it likely just picks the "default" sound device. 4. If it's a USB HID that happens to have an "Originating Device" then g-s-d looks through the various PA sinks and see is one of them has the same "Originating Device". If we get a match, then this is the device we'll use. 5. g-s-d issues a volume increment or decrement to PA with the appropriate device.
The above is a very high level, hand wavey description. I don't think PA has (or needs) specific support to tie these things together.
I'm not 100% sure what elements of a sink is used to do this matching but I'd take a wild stab in the dark guess at the "device.bus_path" entry in the proplis which can look something like: device.bus_path = "pci-0000:00:1d.7-usb-0:7.1:1.0"
But I really don't know what the xinput2 or gtk3 side of things looks like for this, so I'm just going on descriptions Lennart has given in the past.
I'm also concerned that an implicit system may be hard to use from userspace, if only in terms of figuring out what the rules are for matching things up.
I doubt this will be much of a problem once a few clean examples are out there. The logic to match things up should be pretty trivial.
Col
On Mon, Jun 27, 2011 at 06:01:55PM +0100, Colin Guthrie wrote:
I'm not 100% sure what elements of a sink is used to do this matching but I'd take a wild stab in the dark guess at the "device.bus_path" entry in the proplis which can look something like: device.bus_path = "pci-0000:00:1d.7-usb-0:7.1:1.0"
Ah, if it's doing that that's not going to work in general - the jack may not be part of the same device as the thing we're reporting on.
I'm also concerned that an implicit system may be hard to use from userspace, if only in terms of figuring out what the rules are for matching things up.
I doubt this will be much of a problem once a few clean examples are out there. The logic to match things up should be pretty trivial.
One of my biggest concerns here is people even finding the relevant bit of the stack, let alone working out what to do with it - bear in mind that my main focus is on the embedded side so people may not be familiar with Linux or using the standard desktop stack at all.
On Mon, Jun 27, 2011 at 06:01:55PM +0100, Colin Guthrie wrote:
'Twas brillig, and Mark Brown at 27/06/11 13:07 did gyre and gimble:
So, this discussion seems to have ground to a halt. Summarising briefly it seems that Lennart and Kay both seem adamant that they don't want a unified jack object visible to userspace, they want to glue the objects together via some implicit means. Lennart mentioned that there is some support for at least some of this functionality in PulseAudio already but didn't provide any details yet - does anyone else know what happens here or have time to go look at the code?
So AFAIK, there is no support in PulseAudio per-se for this.
Hrm, nobody else seems to have any idea...
I'm not 100% sure what elements of a sink is used to do this matching but I'd take a wild stab in the dark guess at the "device.bus_path" entry in the proplis which can look something like: device.bus_path = "pci-0000:00:1d.7-usb-0:7.1:1.0"
...this being the key bit.
On 2011-06-27 13:07, Mark Brown wrote:
On Mon, Jun 20, 2011 at 03:37:25PM +0200, David Henningsson wrote:
- We're continuing the path with /dev/input devices
2a) We'll rewrite these devices to be read-only ALSA mixer controls
2b) We'll rewrite these devices to be something within ALSA, but not exactly mixer controls.
For options 2a) and 2b) I guess the existing /dev/input thing should be deprecated and/or removed. So part of decision should maybe be based on information about how widespread the usage of these devices are currently...?
So, this discussion seems to have ground to a halt.
Yes, unfortunately, and without a clear consensus. That puts me in a difficult position, because I'm trying to get the job done. And very preferrable, in the next release of Ubuntu (which is to be released on October this year). I've been talking to a few of my colleagues here at Canonical, and here's how we reason currently:
We have the input layer. We also have a set of patches for supporting this in PulseAudio (although currently unmerged and I'm not sure about their current quality/state). We don't have the alternative Alsa Control API discussed in this thread. Without taking a stance of whether that would be a better solution or not, designing it will take some time - somewhat depending on the set of problems we're aiming to solve. Regardless, it will definitely not reach the 3.0 kernel (which is what we will ship in the next release of Ubuntu).
As such, it makes the most sense for me to continue working on the existing input layer API for the time being. If or when a new API is announced and finished, rewriting the pulseaudio patches to target that API will probably make sense. But we're not there today, and the time schedule for getting there is unknown.
If upstream udev/PulseAudio would be willing to merge my (existing and upcoming) patches, I would appreciate that, as I believe that would make both our lives easier. If not, well at least make a note that I /tried/ to do things the right way, and to make everybody happy - which is not always possible.
On 28/06/11 17:27, David Henningsson wrote:
On 2011-06-27 13:07, Mark Brown wrote:
On Mon, Jun 20, 2011 at 03:37:25PM +0200, David Henningsson wrote:
- We're continuing the path with /dev/input devices
2a) We'll rewrite these devices to be read-only ALSA mixer controls
2b) We'll rewrite these devices to be something within ALSA, but not exactly mixer controls.
For options 2a) and 2b) I guess the existing /dev/input thing should be deprecated and/or removed. So part of decision should maybe be based on information about how widespread the usage of these devices are currently...?
So, this discussion seems to have ground to a halt.
Yes, unfortunately, and without a clear consensus. That puts me in a difficult position, because I'm trying to get the job done. And very preferrable, in the next release of Ubuntu (which is to be released on October this year). I've been talking to a few of my colleagues here at Canonical, and here's how we reason currently:
We have the input layer. We also have a set of patches for supporting this in PulseAudio (although currently unmerged and I'm not sure about their current quality/state).
I know that these are working on Panda and that Graeme is addressing review comments for upstream. Don't know his ETA yet though.
Liam
On Tue, Jun 28, 2011 at 05:27:15PM +0100, David Henningsson wrote:
As such, it makes the most sense for me to continue working on the existing input layer API for the time being. If or when a new API is announced and finished, rewriting the pulseaudio patches to target that API will probably make sense. But we're not there today, and the time schedule for getting there is unknown.
If upstream udev/PulseAudio would be willing to merge my (existing and upcoming) patches, I would appreciate that, as I believe that would make both our lives easier. If not, well at least make a note that I /tried/ to do things the right way, and to make everybody happy - which is not always possible.
FWIW I tend to agree that pragmatically it'd make a lot of sense to support the current kernel interfaces in userspace even if we're also going to decide to deprecate them, if only for the deployment reasons you mention.
At Tue, 28 Jun 2011 17:27:15 +0100, David Henningsson wrote:
On 2011-06-27 13:07, Mark Brown wrote:
On Mon, Jun 20, 2011 at 03:37:25PM +0200, David Henningsson wrote:
- We're continuing the path with /dev/input devices
2a) We'll rewrite these devices to be read-only ALSA mixer controls
2b) We'll rewrite these devices to be something within ALSA, but not exactly mixer controls.
For options 2a) and 2b) I guess the existing /dev/input thing should be deprecated and/or removed. So part of decision should maybe be based on information about how widespread the usage of these devices are currently...?
So, this discussion seems to have ground to a halt.
Yes, unfortunately, and without a clear consensus. That puts me in a difficult position, because I'm trying to get the job done. And very preferrable, in the next release of Ubuntu (which is to be released on October this year). I've been talking to a few of my colleagues here at Canonical, and here's how we reason currently:
We have the input layer. We also have a set of patches for supporting this in PulseAudio (although currently unmerged and I'm not sure about their current quality/state).
So, the targets are already defined, i.e. what information is actually required? This wasn't very clear to me until now.
We don't have the alternative Alsa Control API discussed in this thread. Without taking a stance of whether that would be a better solution or not, designing it will take some time - somewhat depending on the set of problems we're aiming to solve. Regardless, it will definitely not reach the 3.0 kernel (which is what we will ship in the next release of Ubuntu).
If only the same functionality is required as currently done in the input-jack layer, re-implementing the jack-detection elements in ALSA control API is pretty easy. It means that the control element would have some jack name with a location prefix or such, reports the current jack status, and notifies the jack change. That's all. Optionally, it can have a TLV data giving the HD-audio jack attribute, too.
As already mentioned, the difficulties of implementation mainly depends on the requirements. If just a flat array without structural connection is enough, it's easy, of course.
But, it definitely won't reach to 3.0 kernel. So, maybe this won't help in your case, anyway...
thanks,
Takashi
On Tue, Jun 28, 2011 at 06:35:33PM +0200, Takashi Iwai wrote:
If only the same functionality is required as currently done in the input-jack layer, re-implementing the jack-detection elements in ALSA control API is pretty easy. It means that the control element would have some jack name with a location prefix or such, reports the current jack status, and notifies the jack change. That's all. Optionally, it can have a TLV data giving the HD-audio jack attribute, too.
It needs a subset of the current information - it should report only audio events, so pretty much only headphone, microphone or line out presence. Anything else needs to be reported via a different API and figured out by userspace, and the input device should stay there for button press events.
At Tue, 28 Jun 2011 19:59:18 -0700, Mark Brown wrote:
On Tue, Jun 28, 2011 at 06:35:33PM +0200, Takashi Iwai wrote:
If only the same functionality is required as currently done in the input-jack layer, re-implementing the jack-detection elements in ALSA control API is pretty easy. It means that the control element would have some jack name with a location prefix or such, reports the current jack status, and notifies the jack change. That's all. Optionally, it can have a TLV data giving the HD-audio jack attribute, too.
It needs a subset of the current information - it should report only audio events, so pretty much only headphone, microphone or line out presence.
Is that really all for PA, for now, and even for future? That's my primary question. Was this clarified in the thread...?
Anything else needs to be reported via a different API and figured out by userspace, and the input device should stay there for button press events.
The first and the second part are independent, IMO. That is, whether using the input-jack device in future is an open question. It might be better to re-implement in a new API in a unified way, for example. Of course, keeping the old API is mandatory, so we'll still keep the input-jack code in future, though.
thanks,
Takashi
On Wed, Jun 29, 2011 at 07:34:11AM +0200, Takashi Iwai wrote:
Mark Brown wrote:
It needs a subset of the current information - it should report only audio events, so pretty much only headphone, microphone or line out presence.
Is that really all for PA, for now, and even for future? That's my primary question. Was this clarified in the thread...?
Well, that was what Kay and Lennart were keen on - the information for a given subsystem should be reported via that subsystem so all the other information we're reporting currently should be going via some other subsystem.
Like I keep saying I still don't have a clear picture as to how this would actually be implemented in userspace and how practical it is.
Anything else needs to be reported via a different API and figured out by userspace, and the input device should stay there for button press events.
The first and the second part are independent, IMO. That is, whether using the input-jack device in future is an open question. It might be better to re-implement in a new API in a unified way, for example.
You're missing my point here. The point is not the switches, the point is the buttons on the jacks - they need to go via the input API and not the ALSA API.
Of course, keeping the old API is mandatory, so we'll still keep the input-jack code in future, though.
That too.
At Tue, 28 Jun 2011 23:59:41 -0700, Mark Brown wrote:
Anything else needs to be reported via a different API and figured out by userspace, and the input device should stay there for button press events.
The first and the second part are independent, IMO. That is, whether using the input-jack device in future is an open question. It might be better to re-implement in a new API in a unified way, for example.
You're missing my point here. The point is not the switches, the point is the buttons on the jacks - they need to go via the input API and not the ALSA API.
OK, buttons. How confusing terminologies are...
thanks,
Takashi
On 2011-06-28 17:35, Takashi Iwai wrote:
At Tue, 28 Jun 2011 17:27:15 +0100, David Henningsson wrote:
On 2011-06-27 13:07, Mark Brown wrote:
On Mon, Jun 20, 2011 at 03:37:25PM +0200, David Henningsson wrote:
- We're continuing the path with /dev/input devices
2a) We'll rewrite these devices to be read-only ALSA mixer controls
2b) We'll rewrite these devices to be something within ALSA, but not exactly mixer controls.
For options 2a) and 2b) I guess the existing /dev/input thing should be deprecated and/or removed. So part of decision should maybe be based on information about how widespread the usage of these devices are currently...?
So, this discussion seems to have ground to a halt.
Yes, unfortunately, and without a clear consensus. That puts me in a difficult position, because I'm trying to get the job done. And very preferrable, in the next release of Ubuntu (which is to be released on October this year). I've been talking to a few of my colleagues here at Canonical, and here's how we reason currently:
We have the input layer. We also have a set of patches for supporting this in PulseAudio (although currently unmerged and I'm not sure about their current quality/state).
So, the targets are already defined, i.e. what information is actually required? This wasn't very clear to me until now.
Well, for basic jack detection the current input layer will work. We might want to add some more information to it (e g by changing the name string or use the additional fields available), but that should not be very controversial, I think.
However, if we manage to expose all the routing between PCMs, volume controls and ports, we can make an even better PulseAudio, because we can solve other problems as well, such as the volume control naming problem. But figuring out how to do that in the best way API-wise and so on, I guess that will take some time.
We don't have the alternative Alsa Control API discussed in this thread. Without taking a stance of whether that would be a better solution or not, designing it will take some time - somewhat depending on the set of problems we're aiming to solve. Regardless, it will definitely not reach the 3.0 kernel (which is what we will ship in the next release of Ubuntu).
If only the same functionality is required as currently done in the input-jack layer, re-implementing the jack-detection elements in ALSA control API is pretty easy.
Yes, but it is not clear to me that this is the decision, and what we actually want to do. So far I have only seen Kay and Lennart advocating for it and Mark against it. I started off this thread because I needed a decision, but so far no clear decision has been made.
It means that the control element would have some jack name with a location prefix or such, reports the current jack status, and notifies the jack change. That's all. Optionally, it can have a TLV data giving the HD-audio jack attribute, too.
I guess this will require support in alsa-lib as well then? At least if you want to pass through new types of TLV data.
As already mentioned, the difficulties of implementation mainly depends on the requirements. If just a flat array without structural connection is enough, it's easy, of course.
But, it definitely won't reach to 3.0 kernel. So, maybe this won't help in your case, anyway...
Well, if the changes are not too invasive, there is also the possibility to backport things into the Ubuntu version of the 3.0 kernel as well.
On Wed, Jun 29, 2011 at 08:13:42AM +0100, David Henningsson wrote:
However, if we manage to expose all the routing between PCMs, volume controls and ports, we can make an even better PulseAudio, because we can solve other problems as well, such as the volume control naming problem. But figuring out how to do that in the best way API-wise and so on, I guess that will take some time.
This is really a totally separate question, there's so much routing within the devices that the jacks are just not a particularly interesting part of it.
If only the same functionality is required as currently done in the input-jack layer, re-implementing the jack-detection elements in ALSA control API is pretty easy.
Yes, but it is not clear to me that this is the decision, and what we actually want to do. So far I have only seen Kay and Lennart advocating for it and Mark against it. I started off this thread because I needed a decision, but so far no clear decision has been made.
I'm not against having the information in ALSA, or really against using separate APIs. My main concern is being able to group the objects back together again for use both in kernel (when the hardware overlaps) and in userspace in an understandable fashion and I've not seen anything that I'm as comfortable with as a directly visible object exported from the kernel. If ALSA chooses to export some subset of the information that's kind of orthogonal to that issue.
I do have to say I like not having to use alsa-lib to get the information but that's even less of a big deal.
On 2011-06-29 08:21, Mark Brown wrote:
On Wed, Jun 29, 2011 at 08:13:42AM +0100, David Henningsson wrote:
However, if we manage to expose all the routing between PCMs, volume controls and ports, we can make an even better PulseAudio, because we can solve other problems as well, such as the volume control naming problem. But figuring out how to do that in the best way API-wise and so on, I guess that will take some time.
This is really a totally separate question,
The question is not totally separate as if we design an ALSA API, we might have this in the back of our minds in order to not build an API when adding this information later will be very difficult.
there's so much routing within the devices that the jacks are just not a particularly interesting part of it.
I'm probably misunderstanding you here, because that sounds crazy. What sense would it make to display a routing graph if the jacks are not part of that graph?
If only the same functionality is required as currently done in the input-jack layer, re-implementing the jack-detection elements in ALSA control API is pretty easy.
Yes, but it is not clear to me that this is the decision, and what we actually want to do. So far I have only seen Kay and Lennart advocating for it and Mark against it. I started off this thread because I needed a decision, but so far no clear decision has been made.
I'm not against having the information in ALSA, or really against using separate APIs.
Ok.
My main concern is being able to group the objects back together again for use both in kernel (when the hardware overlaps) and in userspace in an understandable fashion and I've not seen anything that I'm as comfortable with as a directly visible object exported from the kernel.
I'm not following. For userspace, you need to match the input layer against an ALSA sound card. You need to do that regardless of whether the buttons and the jacks are in the input layer, or if only the buttons are there. No difference in difficulty as I see it.
For the kernel part, I guess that if ASoC core (or ALSA core) is responsible for the splitting, and that the actual driver has an object covering both types of input (i e both jacks and buttons), that would be the most obvious way to solve the problem.
I do have to say I like not having to use alsa-lib to get the information but that's even less of a big deal.
Ok, is that because Android does not use alsa-lib?
On Wed, Jun 29, 2011 at 09:52:13AM +0100, David Henningsson wrote:
On 2011-06-29 08:21, Mark Brown wrote:
However, if we manage to expose all the routing between PCMs, volume controls and ports, we can make an even better PulseAudio, because we can solve other problems as well, such as the volume control naming problem. But figuring out how to do that in the best way API-wise and so on, I guess that will take some time.
This is really a totally separate question,
The question is not totally separate as if we design an ALSA API, we might have this in the back of our minds in order to not build an API when adding this information later will be very difficult.
I think if we make that difficult we've messed up massively anyway.
there's so much routing within the devices that the jacks are just not a particularly interesting part of it.
I'm probably misunderstanding you here, because that sounds crazy. What sense would it make to display a routing graph if the jacks are not part of that graph?
I said they weren't an interesting part of it, not that they're not part of it. It's a very small detail relative to all the other stuff we need to worry about.
My main concern is being able to group the objects back together again for use both in kernel (when the hardware overlaps) and in userspace in an understandable fashion and I've not seen anything that I'm as comfortable with as a directly visible object exported from the kernel.
I'm not following. For userspace, you need to match the input layer against an ALSA sound card. You need to do that regardless of whether the buttons and the jacks are in the input layer, or if only the buttons are there. No difference in difficulty as I see it.
As repeatedly discussed we need to do at least a three way match - currently we need to also glue both input and audio to graphics as well. It seems likely we'll end up with other things on there too.
For the kernel part, I guess that if ASoC core (or ALSA core) is responsible for the splitting, and that the actual driver has an object covering both types of input (i e both jacks and buttons), that would be the most obvious way to solve the problem.
The multiple subsystems thing also applies here - you really can't rely on doing this all in one driver.
I do have to say I like not having to use alsa-lib to get the information but that's even less of a big deal.
Ok, is that because Android does not use alsa-lib?
No, it's because it's a pretty big library with it's own programming interface that's more complicated than your standard poll and read so it's not quite so natural to integrate into your system management daemon (that looks after the system as a whole) as it could be.
participants (10)
-
Clemens Ladisch -
Colin Guthrie -
David Henningsson -
Dmitry Torokhov -
Kay Sievers -
Lennart Poettering -
Liam Girdwood -
Mark Brown -
Stephen Warren -
Takashi Iwai