Re: [alsa-devel] [RFC] ALSA vs. dedicated char device for a USB Audio Class gadget driver
Hi Hal,
On Friday 15 May 2009 22:15:18 Hal Murray wrote:
If I use a synchronous endpoint, isn't the number of samples per frame determined by the nominal sampling rate and the nominal SOF frequency ? With the SOF clock running at 1kHz, I expect a synchronous endpoint for a 16 bits mono 48kHz stream to deliver exactly 48 frames (96 bytes) per USB frame.
Beware of the "exact" in there. In real life, crystals have a tolerance. The USB clock will not match the audio clock perfectly. What should be 1 kHz might be 1.00003 kHz or it might be 0.99997 kHz.
If the USB clock in this example is slightly fast, you will get occasional times when there are only 47 samples ready. If it's slightly slow, you will occasionally have 49 samples.
That's slow/fast relative to the audio clock. If the USB clock is 0.001% slow but the audio clock is 0.002% slow, then the USB clock will be fast for this discussion.
Except that the SOF clock *is* the reference clock. It will be slower or faster than the audio clock, and the userspace application will have to perform sample rate matching so that packets will be exactly 48 bytes long.
This is a common problem in communications. There are several techniques to cope with it.
The simplest is to make the transport mechanism have a bit of extra bandwidth so you can always keep up. It's just a matter of how often you use that extra bandwidth. In this case, you would allocate enough bandwidth for 49 samples and only use 48 most of the time. An occasional USB frame would have 47 or 49.
That's what would happen with an asynchronous endpoint. The hard bit is to find out when to send that occasional packet, as the driver doesn't have access to the audio clock.
Another approach is to use big enough FIFO so you never get in trouble. For this to work, you need a limit on the length of the data stream. For example, suppose the longest song you want to send is 1000 seconds and both clocks are accurate to 50 ppm (parts per million). The max difference in clocks is 100 ppm. 48 k samples/second * 1000 seconds is 48 million samples total. 100 ppm means that the worst difference would be 4800 samples. So if the USB side waits until it gets 4800 samples before it starts forwarding data, it won't run out if the audio clock is worst-case slow and the USB clock is worst-case fast. You also need buffering for another 4800 samples in case the clock differences are reversed. This adds a delay to get started while you collect the 4800 samples.
That won't work here, the stream can last forever (well, not quite, but still a long time).
You can just drop or duplicate a sample whenever the clocks slip.
You can derive both clocks from a common source.
You can lock one clock to the other with a PLL (Phase Locked Loop).
The SOF clock is driven by the USB host, and the audio clock is driven by the audio codec. I can't lock one to another or derive both of them from a common source. There could even be no audio clock at all if I stream audio data from a file.
There are probably other approaches.
The applicable techniques require knowledge of both the audio clock and the SOF clock in a common place. My driver has no access to the audio clock. All it knows about is the SOF clock.
There are only two options I can think of.
The first one is to use an asynchronous endpoint and sent the occasional smaller or bigger packet (or duplicate/drop one sample). As the driver can't access the audio clock it needs to derive the information from the amount of data present in the ALSA ring buffer. To be honest I'm not sure if that will be possible at all, as the application will write data at a non-constant rate.
The second one, which sounds easier, at least on the driver side, is to use a synchronous endpoint with a fixed packet size. The application will perform rate matching (duplicating/dropping a sample or resampling the audio stream) using the audio clock and the SOF clock. What I'm still unsure about is how the application can access the audio clock and the SOF clock through ALSA, but I suppose that's possible.
Best regards,
Laurent Pinchart
On Sun, 17 May 2009, Laurent Pinchart wrote:
The applicable techniques require knowledge of both the audio clock and the SOF clock in a common place. My driver has no access to the audio clock. All it knows about is the SOF clock.
This whole discussion is a little puzzling.
The Gadget API doesn't provide any way to tie the buffer contents of Isochronous usb_request's to the frame number. Here's what I mean: Suppose you want to transfer a new audio data buffer every frame. You queue some requests, let's say
R0 for frame F0 R1 for frame F1 R2 for frame F2 etc.
But what happens if a communications error prevents R0 from being delivered during F0? That's the sort of thing you expect to happen from time to time, and Isochronous streams are supposed to handle such errors by simply ignoring them. So ideally you'd like to forget about the missing data, and go ahead with R1 during F1 and so on.
But as far as I can see, the Gadget API doesn't provide any way to do this! Depending on the implementation of the device controller driver, you might end up transferring R0 during F1, R1 during F2, and so on. Everything would be misaligned from then on. I don't see any solution to this problem.
I'd like to answer your questions about synchronizing the USB audio stream with an ALSA audio stream, but since I don't know anything about how either audio protocol is supposed to work, I can't. Suppose you were trying to write a normal program that accepted data from an ALSA microphone and sent it to an ALSA speaker; how would such a program synchronize the input and output streams?
Alan Stern
Hi Alan,
On Sunday 17 May 2009 23:28:20 Alan Stern wrote:
On Sun, 17 May 2009, Laurent Pinchart wrote:
The applicable techniques require knowledge of both the audio clock and the SOF clock in a common place. My driver has no access to the audio clock. All it knows about is the SOF clock.
This whole discussion is a little puzzling.
The Gadget API doesn't provide any way to tie the buffer contents of Isochronous usb_request's to the frame number. Here's what I mean: Suppose you want to transfer a new audio data buffer every frame. You queue some requests, let's say
R0 for frame F0 R1 for frame F1 R2 for frame F2 etc.
But what happens if a communications error prevents R0 from being delivered during F0? That's the sort of thing you expect to happen from time to time, and Isochronous streams are supposed to handle such errors by simply ignoring them. So ideally you'd like to forget about the missing data, and go ahead with R1 during F1 and so on.
But as far as I can see, the Gadget API doesn't provide any way to do this! Depending on the implementation of the device controller driver, you might end up transferring R0 during F1, R1 during F2, and so on. Everything would be misaligned from then on. I don't see any solution to this problem.
Your analysis is right, but I think I can work around the problem by checking the SOF counter in the USB request completion handlers. If the counter is too much ahead its ideal value I can assume that at least one packet failed to transfer and resulted in a time shift. I will then just drop one packet. Instead of the ideal R0/F0, --/F1, R2/F2, R3/F3, R4/F4, R5/F5, ... situation I will get R0/F0, --/F1, R1/F2, R2/F3, R4/F4, R5/F5, ...
I'd like to answer your questions about synchronizing the USB audio stream with an ALSA audio stream, but since I don't know anything about how either audio protocol is supposed to work, I can't. Suppose you were trying to write a normal program that accepted data from an ALSA microphone and sent it to an ALSA speaker; how would such a program synchronize the input and output streams?
That's a very good question. If an ALSA guru could answer it I would (hopefully) get one step closer to a solution. I can't imagine that ALSA wouldn't support this use case.
Best regards,
Laurent Pinchart
Laurent Pinchart wrote:
The applicable techniques require knowledge of both the audio clock and the SOF clock in a common place. My driver has no access to the audio clock. All it knows about is the SOF clock.
There are only two options I can think of.
The first one is to use an asynchronous endpoint and sent the occasional smaller or bigger packet (or duplicate/drop one sample). As the driver can't access the audio clock it needs to derive the information from the amount of data present in the ALSA ring buffer. To be honest I'm not sure if that will be possible at all, as the application will write data at a non-constant rate.
The ALSA API assumes that the device controls the audio clock and that the application derives its own speed from that, not the other way round.
The second one, which sounds easier, at least on the driver side, is to use a synchronous endpoint with a fixed packet size. The application will perform rate matching (duplicating/dropping a sample or resampling the audio stream) using the audio clock and the SOF clock. What I'm still unsure about is how the application can access the audio clock and the SOF clock through ALSA, but I suppose that's possible.
The ALSA API doesn't give you access to the SOF clock.
What you want to do is not possible (or impracticably hard) with the ALSA API; I strongly suggest that you define your own API for your audio gadget driver so that SOF clock information is made available to the application, and that packet sizes can be directly controlled by the application.
Best regards, Clemens
Hi Clemens,
On Monday 18 May 2009 10:39:24 Clemens Ladisch wrote:
Laurent Pinchart wrote:
The applicable techniques require knowledge of both the audio clock and the SOF clock in a common place. My driver has no access to the audio clock. All it knows about is the SOF clock.
There are only two options I can think of.
The first one is to use an asynchronous endpoint and sent the occasional smaller or bigger packet (or duplicate/drop one sample). As the driver can't access the audio clock it needs to derive the information from the amount of data present in the ALSA ring buffer. To be honest I'm not sure if that will be possible at all, as the application will write data at a non-constant rate.
The ALSA API assumes that the device controls the audio clock and that the application derives its own speed from that, not the other way round.
That's a sensible requirement for a sound card. I now know that my use case is out of ALSA's bounds :-)
The second one, which sounds easier, at least on the driver side, is to use a synchronous endpoint with a fixed packet size. The application will perform rate matching (duplicating/dropping a sample or resampling the audio stream) using the audio clock and the SOF clock. What I'm still unsure about is how the application can access the audio clock and the SOF clock through ALSA, but I suppose that's possible.
The ALSA API doesn't give you access to the SOF clock.
Does the ALSA API give applications access to the audio clock ? As Alan Stern mention in his e-mail, does ALSA allow an application to resample audio from an ALSA source (microphone on sound card 1) before sending it to an ALSA sink (speaker on sound card 2) ?
What you want to do is not possible (or impracticably hard) with the ALSA API; I strongly suggest that you define your own API for your audio gadget driver so that SOF clock information is made available to the application, and that packet sizes can be directly controlled by the application.
I'll have a look at that. ALSA was appealing because of the existing user base and its ability to transfer audio data from/to userspace applications. I knew it hasn't been designed with my use case in mind, so I didn't close to door to using a custom API, but I still wanted to give it a try.
Best regards,
Laurent Pinchart
Laurent Pinchart wrote:
On Monday 18 May 2009 10:39:24 Clemens Ladisch wrote:
The ALSA API doesn't give you access to the SOF clock.
Does the ALSA API give applications access to the audio clock ?
ALSA wakes up the application (if it's blocked) whenever a sound card interrupt happens, i.e., at period boundaries. At all times, you can get the current sample position by calling snd_pcm_delay().
As Alan Stern mention in his e-mail, does ALSA allow an application to resample audio from an ALSA source (microphone on sound card 1) before sending it to an ALSA sink (speaker on sound card 2) ?
It is possible, if the application monitors the speed of both devices.
Best regards, Clemens
participants (3)
-
Alan Stern -
Clemens Ladisch -
Laurent Pinchart