At Tue, 16 Jun 2009 13:59:45 +0200, Clemens Ladisch wrote:
Robert Schweikert wrote:
Currently ALSA is a trial use module for LSB (http://www.linuxfoundation.org/collaborate/workgroups/lsb) 4.0 and it is the intention to make ALSA mandatory with the LSB 4.1 release targeted for Q1 2010. One of the requirements of a mandatory module is the existence of a test suite for the module that covers a good chunk of the interfaces provided by the LSB spec. An initial exploratory look at the existing test cases showed that tests appear to be tied to hardware drivers and are interactive.
Is this initial impression correct?
Yes.
From an LSB testing perspective, interface testing is very important; i.e. I call the interface with the arguments required and I get the expected result back.
In the case of a hardware interface library like libasound, there is not always a result that comes back where it could by easily tested.
While many parts of the library can be tested without special hardware (config, sequencer+MIDI, timers), the parts that are most likely to be used (PCM, mixer) require at least some kernel driver and contain many functions that are used to handle hardware differences.
True. That's the very high hurdle that an automatic testing would face.
1.) Is there interest from the community side to participate in this effort and accept patches?
I'm just an individual and cannot speak for "the community", whatever that is, but there is certainly interest to accept patches.
Good to hear :)
2.) Is it reasonable to expect that the existing tests can be used in some way by using a dummy sound device or a sound loop device to verify output?
There is a loopback driver, but it is not part of the LSB specification. This means that a test using this driver would not work with any alternate ALSA implementation.
Yes, I mentioned about this in the last conf call with LSB guys.
Another candidate is alsa-lib file plugin (and null plugin). They can be used to simulate in a certain level of I/O.
I can easily imagine LSB-compliant computers that do not have any sound card (e.g., most servers). Even if the ALSA interface (i.e., libasound) is installed, some parts cannot be used in any meaningful way. What is a test supposed to do in this case? It could just exit with "pass", but this wouldn't actually _test_ much.
The test can never cover 100% all real cases. So, the primary question is: what kind of test do we need.
From the LSB perspective, I suppose, the API compatibility is the biggest issue. The functionality, either working or not on the real hardware, is a subsystem bug. And, the subtle tests would be likely needed done manually with the real hardware.
(Hmmm, a dummy ALSA implementation that just returns -ENODEV when the application tries to open a device would be compliant, because it cannot be distinguished from the 'real' ALSA with no installed sound card. :-)
You mean the dummy sound driver?
3.) Would the community be more comfortable if this is an effort that creates new tests separate from the existing HW focused tests?
The current interactive tests are mostly intended to test drivers, not the interface. I don't think it would be desirable or even possible to integrate them into a test suite with strict pass/fail requirements.
An automatic test would be hard, yes. But, including "manual tests" by human makes a lot sense. For example, providing a set of WAV files with different format, or giving a script to handle with sox, etc.
In my case, I've been testing like the following:
- Mixer adjustment via alsamixer - Play with different format bits (8, 16, 24, 32, float), sample rates, and channels via aplay - Play with different access methods (mmap, r/w) via aplay - Speaker configuration tests via speaker-test - Buffer size, period size tests via aplay - Recording via arecord; this is really hard to automate right now... - SPDIF AC3 output via ac3dec to a surround receiver - SPDIF input recording by cabling to SPDIF input
So, most of them are real hardware tests...
4.) Is anyone interested in helping with this effort, writing tests, answering questions about ALSA for those not familiar with the interface and or sound in general?
Yes.
Since most of libasound is somewhat undocumented (the doxygen reference is just a skeleton), I'm wondering who is going to write the tests ...
Heh, first response, first victim? :)
As mentioned, we need to focus on the definition of what kind of tests at first.
thanks,
Takashi