[alsa-devel] Software mixer at kernel level
Hi, I was wondering if it could be possible to implement a low-level software mixer (like OSS4's vmix) since almost everybody needs it and PulseAudio causes too much overhead on not-so-old systems.
_________________________________________________________________ Explore the seven wonders of the world http://search.msn.com/results.aspx?q=7+wonders+world&mkt=en-US&form=...
Hi,
On Mon, Feb 9, 2009 at 9:02 PM, Mauricio Nuñez mauricio_n2000@hotmail.com wrote:
Hi, I was wondering if it could be possible to implement a low-level software mixer (like OSS4's vmix) since almost everybody needs it and PulseAudio causes too much overhead on not-so-old systems.
A few points that have been raised in past discussions of kernel-mode software mixing:
1. You're stuck with integer precision. Floating point arithmetic in the kernel is a BIG no-no. 2. Kernel core developers (including Linus) have made it clear that this kind of math operation should be done in user mode. The reasoning is that any bug in the kernel can bring down the whole system, or at least, an entire subsystem (audio, graphics, etc). The point here is that software mixing is aptly named, in that the mixing is done in /software/, while the kernel's primary function is to interface with /hardware/. The kernel needs special privileged memory access, firmware access, etc. to carry out these operations, but you don't need any special access beyond what the ALSA pcm interface already provides, to effectively do software mixing in userspace. From a perspective of security alone, we would want to follow the policy of minimal privilege level wherever possible: software should not run with more privileges than it needs to carry out its function. You can't effectively limit permissions in kernel mode due to the hardware design.
I would also add to the discussion that, unlike you insinuated, doing software mixing in the kernel would not reduce its overhead significantly. The CPU resources needed by PulseAudio are mainly due to functions such as resampling, which, if implemented, the ALSA kernel modules would have to do under software mixing. These algorithms are quite optimal already given current computer science theory, and the code is operating without much runtime overhead (it's written in C).
However, the point about "everybody needs it" is a valid one. If I had my druthers, it would be impossible for anyone in usermode to initialize an ALSA pcm in a mode which does not provide software mixing. We can still provide the software mixing in usermode, but my reasoning is that
(a) Not using software mixing (i.e., hogging the sound card) makes your app a poor desktop citizen, because it effectively causes other programs to appear to malfunction from the user's point of view. (b) There is very little justification for using the functions that are part of the "unsafe" ALSA subset. By contrast, if everyone only used the "safe" ALSA subset, then requiring software mixing (such as dmix) would be fine. It's only when people want features such as mmap() that we run into problems with software mixing. (c) By requiring software mixing at the ALSA-lib user-mode layer, we can prevent those poor citizens from affecting other desktop apps; we can guarantee user experience; and we can help establish a broader consensus as to the sound APIs that should be used on the free desktop.
Of course, PulseAudio can be said to require software mixing: you can't be a PulseAudio client without your data stream being subject to its mixing routines. So they've already done what I just proposed. The only difference is that, since it's built on top of ALSA, the same issues are still present within ALSA as have always been there. It's just that, for backwards compatibility's sake, we can't rightly change our mind on these things now, unless we can live without things such as JACK, Ardour, Audacity, Teamspeak, and a host of others.
At some basic level, software mixing requires proper tuning of the kernel; a fairly recent system; and a "sane" compromise between desired computational complexity and actually available computing resources. For instance, trying to use the highest quality floating point resampling in the libspeex library on a first-generation Pentium 4 might cause some sound quality problems when the system is under load. That's because you're asking a lot of the CPU to make all those floating point calculations. The mixing can be done more cheaply, but you have to pay, to some extent, with quality: either integer samples, or faster and less accurate algorithms.
If the existing software mixing solutions in usermode are not performing to your liking, then you may consider upgrading your system so that these solutions perform adequately for you; or, live with less expensive resampling algorithms (if you're using PulseAudio). OTOH, if you are simply dreaming of a day when software mixing is taken for granted... I think the only way to get there is to keep educating developers that they need to write their software with a "soft" PCM in mind, rather than assuming they are talking directly to hardware.
But I can not see how kernel mode mixing will get us there any faster or easier; on the contrary, it is likely to introduce instability and become a maintenance nightmare.
HTH,
Sean
P.S. - Not a knock on OSS4 at all, but I have had my system kernel panic with OSS4 under many different kernels and hardwares over the years. I sort of liked it for a while when it worked well, but ALSA is leaps and bounds more stable. This could be a sort of case study, if you're interested, as evidence supporting the theory that kernel mode mixing is a bad idea.
Explore the seven wonders of the world http://search.msn.com/results.aspx?q=7+wonders+world&mkt=en-US&form=... _______________________________________________ Alsa-devel mailing list Alsa-devel@alsa-project.org http://mailman.alsa-project.org/mailman/listinfo/alsa-devel
Sean McNamara wrote:
Hi,
However, the point about "everybody needs it" is a valid one. If I had my druthers, it would be impossible for anyone in usermode to initialize an ALSA pcm in a mode which does not provide software mixing. We can still provide the software mixing in usermode, but my reasoning is that
<rant> I disagree. I have one of those applications that doesn't do softwae mixing, deliberately because I want it to have good sound (it's a brainwave entrainment app). It calls functions in alsa that *can't* be accessed through pulse, and allows users to set the frame rate to any hardware rate their card supports (in pulse, one size fits all).
Pulse is great for those who want its functionality, and I would love to see it replace dmix in alsa so that when it received a request for a function it doesn't support it would have the ability to step aside and allow the application to take over the sound device.
Linux is about freedom, and I think you are wanting to restrict that freedom for me to use my hardware the way I want. If you want to put a switch in place that does that, but allows it to be turned off, great, but not hard wired to only behave a single way. Wait! We already have such a switch, it's called /home/.asoundrc.
You are welcome to do whatever you want on your system, but don't assume that everyone wants what you want. </rant>
On Mon, Feb 9, 2009 at 10:57 PM, stan ghjeold_i_mwee@cox.net wrote:
Sean McNamara wrote:
Hi,
However, the point about "everybody needs it" is a valid one. If I had my druthers, it would be impossible for anyone in usermode to initialize an ALSA pcm in a mode which does not provide software mixing. We can still provide the software mixing in usermode, but my reasoning is that
<rant> I disagree. I have one of those applications that doesn't do softwae mixing, deliberately because I want it to have good sound (it's a brainwave entrainment app). It calls functions in alsa that *can't* be accessed through pulse, and allows users to set the frame rate to any hardware rate their card supports (in pulse, one size fits all).
Pulse is great for those who want its functionality, and I would love to see it replace dmix in alsa so that when it received a request for a function it doesn't support it would have the ability to step aside and allow the application to take over the sound device.
Linux is about freedom, and I think you are wanting to restrict that freedom for me to use my hardware the way I want. If you want to put a switch in place that does that, but allows it to be turned off, great, but not hard wired to only behave a single way. Wait! We already have such a switch, it's called /home/.asoundrc.
Sorry, but I think you misinterpreted my suggestion.
Let me sort of imagineer the kind of functionality I was discussing in a bit more detail, and then you can re-evaluate whether I'm trying to restrict things in a way that you need to rant about, ok?
First, let's review how it is today.
Let's say I'm using dmix, and it's in my /etc/asound.conf as the "default" PCM. I am happily going along, starting applications that obey the tacit convention of opening the device named "default", which by convention is supposed to contain whatever PCM pluggage that the user wants.
Now let's say I open some application that, for some reason, wants to access hw:0. As soon as I try to do this while dmix-using apps aren't playing sound, hw:0 is now taken, and the dmix apps can no longer play sound.
This, to me, is a very bad idea for the use case of the desktop. This should be worked from two sides. Most importantly, we need to make sure that desktop apps (i.e., the vast majority of apps) should be able to use dmix or maybe even PulseAudio through the ALSA plug layer. But an adjunct possibility would be to have a setting somewhere in ALSA which _prevents_ applications from directly accessing hw:0 in a way that does not use software mixing. We can then define a set of plugins (jack, pulse, dmix, dsnoop... any others?) which safely enable software mixing, and maybe we can have policy exceptions for individual binaries, to support daemons.
With the changes I was referring to when I wrote my original post, the new functionality would be thus:
1. [Prerequisite] Start PulseAudio or JACK if desired; if not, set up dmix as the pcm.!default. 2. Set an option somewhere in the ALSA configs that says "Apps [which are not in the exception list] may no longer hog the soundcard." 3. Apps that want to play nice with the desktop will access dmix/pulseaudio/JACK using the "safe" ALSA subset and will open the "default" device, and not hw:0. 4. An app that wants to open a direct PCM (ignoring the "default" device) tries to access hw:0 while the other apps are temporarily quiet. alsa-lib immediately returns an error, whether or not the device is in use. 5. The apps designed to use software mixing continue on their way uninterrupted.
We could also have a distro-supplied hook which displays a GUI when an app tries to directly access hw:0. It will be disallowed regardless of whether the device is busy or not, if the configuration bit is set.
This behavior has the benefit of increasing the reliability of apps which _do_ use software mixing, because it ensures they will never be blocked by an app that directly accesses "hw:x". For desktop software that _should_ be compatible with software mixing but is not, it has the side-effect of bringing them into the limelight as ones that the community should try to fix, if possible, to live with the safe subset and open "default" instead of "hw:x". But as you can see, it in no way entails that you will no longer be able to open a direct hw PCM.
I guess I can see how you /could/ interpret my original post as implying some sort of draconian prevention of direct PCM access (with no way around it), but please, let's be charitable: this is the FOSS world, and wherever there is need for a bit-flip on points of disagreement, there usually is one.
The change I've recommended here is merely air code, but its intention is to suggest one way of making the desktop sound experience more reliable, _not_ to close off paths for very special use cases like brainwave entrainment. I'm sure you already realize the consequences of hogging the "hw:x" devices, so it goes without saying that your app can't work well in a desktop environment using sounds from many programs at once. But that's a design decision you have planned for and accepted, and as such, your program is not designed to be desktop software on the order of Firefox, Rhythmbox, Pidgin, Evolution, Phonon, and so on. Since your requirements are so very different, we can not ignore either the desktop use case or your own special use case. But that doesn't mean our hands are tied in terms of trying to make the desktop experience better.
How many times have users wondered why their Pidgin/Skype/Rhythmbox/Flash stuff won't play sound after they fire up Audacity, or Second Life? This _is_ a problem with the applications and not ALSA, but unless we try to encourage use of software mixing, the issues don't seem to be getting addressed.
Anyway, back on topic to the OP, this matter of "firmer" software mixing is the only valid point I could salvage from their idea, so hopefully you'll take me with a grain of salt for talking through his issue to its logical conclusion :)
Thanks,
Sean
You are welcome to do whatever you want on your system, but don't assume that everyone wants what you want.
</rant>
Sean McNamara wrote:
You are welcome to do whatever you want on your system, but don't assume that everyone wants what you want.
</rant> _______________________________________________
Or you could just avoid all this mess with Linux audio for the time being and use a hardware mixing device instead. I realize laptop users are going to bitchslap me for saying that but its true.
The whole buggy software mixing situation with regards to applications not working correctly or as intended is frustrating. OSS4 does it at the kernel level because it tries to be transparent to all the applications which think the device has hardware mixing capabilities. I realize OSS4 can be buggy for some but its implementation is far more compatible instead of the cocktail of sound servers and software mixing routines we have in ALSA right now.
Some apps out there require direct access to /dev/dsp or whatever, what are we going to do? Hope somebody updates the program? (usually not going to happen with older closed source games or whatnot). Take UT99 for example, rarely does this ever work with any software mixing period.
At Tue, 10 Feb 2009 01:50:54 -0500, Brendan Pike wrote:
Sean McNamara wrote:
You are welcome to do whatever you want on your system, but don't assume that everyone wants what you want.
</rant> _______________________________________________
Or you could just avoid all this mess with Linux audio for the time being and use a hardware mixing device instead. I realize laptop users are going to bitchslap me for saying that but its true.
The whole buggy software mixing situation with regards to applications not working correctly or as intended is frustrating. OSS4 does it at the kernel level because it tries to be transparent to all the applications which think the device has hardware mixing capabilities. I realize OSS4 can be buggy for some but its implementation is far more compatible instead of the cocktail of sound servers and software mixing routines we have in ALSA right now.
Some apps out there require direct access to /dev/dsp or whatever, what are we going to do? Hope somebody updates the program? (usually not going to happen with older closed source games or whatnot). Take UT99 for example, rarely does this ever work with any software mixing period.
Well, we can categorize the demands:
A. OSS-style access needed just because of fixed, binary-only things B. OSS-style access needed because of old written codes C. OSS-style access needed because of resource reduction (without server)
The original question was about C. However, in this case, it's not guaranteed whether a kernel-side mixing requires less resources. A server solution could be in less amount, depending on the implementation. Nevertheless, one certain thing is that this question is not for generic desktop uses, but for a pretty limited use-case.
Honestly, I don't mind if anyone introduces a kernel-side thing for that limited purpose like C for his own project. But, this won't hit the mainline, anyway...
For other cases, the resource usage isn't *THAT* big problem at all. The only question is whether it works reliably or not.
It seems that hooking via LD_PRELOAD doesn't work always. Another interesting approach for the OSS fixed devices is to route via a new FUSE char-driver extension by Tejun. This is of course more layer onto the existing setup, but it just works for cases A & B.
Takashi
'Twas brillig, and stan at 10/02/09 03:57 did gyre and gimble:
(in pulse, one size fits all).
While I don't want to comment on the finer points of this thread, I will say that pulse 0.9.15 will be able to switch "profiles" on the fly.
I'm not 100% sure if it currently supports switching sample rates, but this should be trivial to implement now if it's not.
This should allow for an app which requires high quality output to request a specific hardware sample rate. Not sure how this would work in practice in terms of what apps are allowed to do this etc. but it should prevent software resampling kicking in in some circumstances.
Apologies if I've misinterpreted this new functionality (and it is a little theoretical in terms of putting it into practice), but I think i'm right :)
Col
On Mon, Feb 9, 2009 at 9:02 PM, Mauricio Nuñez mauricio_n2000@hotmail.com wrote:
Hi, I was wondering if it could be possible to implement a low-level software mixer (like OSS4's vmix) since almost everybody needs it and PulseAudio causes too much overhead on not-so-old systems.
What kind of overhead? Which systems? Details please.
It works perfectly for me with Ubuntu 8.10 on my work PC, with a relatively old and buggy version of Pulse.
Lee
participants (7)
-
Brendan Pike -
Colin Guthrie -
Lee Revell -
Mauricio Nuñez -
Sean McNamara -
stan -
Takashi Iwai