Hi Alan, Richard,

Many thanks for your responses!

It is probably reasonable to consider Audacity not a focus of concern, since it simply uses Portaudio. [...] enlist help just from Alsa-devel,

[...] agree with Alan that you need to take as much as possible out of the equation to start with

• and that certainly means Audacity.

Fully agreed on that; the only reason I mentioned Audacity, was (more or less) "search engine optimization": I initially met the problem there - and looking up for "Audacity full duplex drop" didn't give me much at the time. And as it took me quite some time to dig up where the problem was coming from - I thought mentioning Audacity as a lead into the problem, may save others looking up something similar in the future. Unfortunately, sometimes providing too much detail may shift the focus in the problem - it was absolutely not my intention to blame Audacity for anything [in fact, I know it is my own driver coding skills to blame :) ]; just to have a mention for a future reference, and maybe to see if some from that community have had similar experiences.

I am including audacity-dev in this reply, just so my response above is logged; however, I've modified the reply-to field in this post, to point only to portaudio and alsa-devel - so hopefully any future responses will be taken there.

... describing the hardware and ideally presenting your driver code.

I have already coded a virtual (that is, independent of any hardware) ALSA driver, and an example PortAudio program, that demonstrate the exact same problem (as I originally met with hardware and Audacity); and the driver, program and test scripts are posted at the link:

http://sdaaubckp.sourceforge.net/post/alsa-patest-duplex/

... since I expect problems, if I try to attach all those files in a post to mailing list(s).

I think there are too many areas embroiled in this, and they need to be separated out. If you have issues writing an Alsa driver, don't rely on Portaudio for testing, but use a range of audio players, or other means, [...]

I have used the default ALSA tools, `arecord` in parallel with `aplay`, and full-duplex in that way goes without any unexpected problems. It is only PortAudio when using a single/"wire" callback (and thus Audacity) where I experience the full-duplex "drop input" problem.

Have you read Takashi Iwai's notes:

http://www.alsa-project.org/~tiwai/writing-an-alsa-driver/index.html ?

Many times! :) My difficulties there occur because the notes are written in respect to hypothetical devices, which I assume are PCI - and I don't really understand PCI. So I'm trying to translate the meanings when reading those notes to virtual drivers, and that sometimes works for me - sometimes, unfortunately, it doesn't (like with this problem).

I suspect the 'dummy device' may not be a good model [...]

Thanks for mentioning this - so far, I haven't read any comment online, on whether `dummy` is a good model of an actual hardware driver; judging from the way the timer functions are coded there, it seemed reasonable to me, so I assumed it is. Then again, I've tried to read `snd-hda-intel`'s driver, and I have a hard time relating it to `dummy`, so it certainly doesn't model all details; but I hoped at least the stream buffer positions (as eventually presented to user-space software) are correct. If anyone can confirm that `dummy` is not reliable in this, that would be great.

[...] and you are not correctly reporting the number of samples transferred to the hardware?

I agree that it must be the core of the problem - but I have problem understanding why, given I currently perceive that I'm doing everything right: I know I have a rate of 44100 frames per second; I choose either a period for timer functions, and calculate bytes per period to match the rate, or vice versa; and in each period, I increase stream buffer positions for that bytes per period amount (taking care of buffer wrapping). But PortAudio, when using a single callback for full-duplex, will still detect a "drop input" - and so, I'm at a loss in figuring out what is being done wrong.

If you think there is a problem with Portaudio, run tests with standard sound hardware such as on-board, PCI or USB units known to work with Alsa, and take the focus off your own driver

I have already tried the PortAudio program with the onboard `snd-hda-intel` and its driver, and it doesn't do a full-duplex drop. However, when at the time I tried to plot the PortAudio variables I captured, I remember I got some weird plots (e.g. the cumulative count of framesGot in the PortAudio callback made a curved line, not a straight one). I should probably try this test again, though.

(since others will not be able to reproduce your custom results anyway).

If at all possible, use current versions of software, and a set-up that others can replicate as much as possible.

Well, I was hoping - given that the problem in OP is expressed in terms of an interaction of a virtual driver (no hardware) and a (relatively simple) PortAudio program - that it would be possible for others to reproduce the problem (at least on a matching kernel and PortAudio version), as it's all in software.

However, I'll also admit that my original post is somewhat dense - and it may not have been obvious from first reading, that there is example code available that should run on any typical Linux PC (without need for additional card hardware).

There are few within Portaudio who are familiar with the Alsa code but we will try to assist if needed (I think that list has a 20K size limit IIRC).

Thanks for that, good to keep in mind. Well, I'm really hoping for a response from the alsa-dev folks, then - but given that this is a problem that is triggered (as far as I know) only in a specific section in PortAudio (even if by my own faulty driver code), I hope it would be OK to keep `portaudio` list in CC. I was also hoping that some Portaudio folks may have encountered the "full duplex drop" in their own development (even if on another OS, and with actual soundcards), and gained some insight with it they'd be willing to share.

Please don't use such an old version of Audacity - we have updated our Portaudio snapshot significantly since then. Bugs in obsolete code really aren't interesting.

Note that Audacity does not utilise the latest Portaudio code available, so consider getting a more recent version.

The thing is, this work is part of a bigger project of mine, and in the name of reproducibility, I wanted to fix all my deliverables to the OSs I started working with (which is why I still use old versions). Given the "full duplex drop" is not a problem in PortAudio per se - but a problem with my ALSA driver code - I don't see a reason why the driver shouldn't work with these older versions, when/if it gets fixed.

...

In Audacity, as far as I could see, there is only one PortAudio callback defined (audacity-1.3.13/src/AudioIO.cpp has audacityAudioCallback;), so Audacity basically has a single "wire" callback for full-duplex.

Correct.

Thanks for confirming that - that would mean (hopefully) that my PortAudio program `patest_duplex_wire.c`, when using a single "wire" callback, should be a somewhat accurate approximation for Audacity behavior; and therefore, if the driver problem gets solved with `patest_duplex_wire.c` - it should also be solved in Audacity.

Hope this can help to move your problem forward.

I think I definitely should try running the experiments on the onboard "HDA Intel" soundcard, and try getting some more insight in how its driver manages its stream buffer positions - but I still hope some from alsa-devel may recognize the problem that I'm dealing with, and let me know where am I going wrong in buffer position calculation in my virtual driver (`dummy-2.6.32-patest.c`).

Many thanks again for the responses, Cheers!

Hi Clemens,

Many thanks for your reply - and apologies it took me a while to write back (and for a longish email again). Since reading your reply, I've spent most of my time coding a new test case for discussion, now posted here (see Readme for more):

http://sdaaubckp.sourceforge.net/post/alsa-capttest/ http://sdaaubckp.sourceforge.net/post/alsa-capttest/Readme

I took the previous advice; and this is an ALSA-only (and capture only, to keep in simple) test, trying to explore what happens when two `snd_pcm_readi` commands:

ret1 = snd_pcm_readi(capture_pcm_handle, audiobuf, period_32_frames); ret2 = snd_pcm_readi(capture_pcm_handle, audiobuf, period_32_frames);

... are ran in succession, for 44100Hz/16b/stereo (with period_size=32 and buffer_size=64 frames, resulting with period time of 725.6 microseconds) in two contexts: 1) with my onboard PCI 'snd_hda_intel' card; and 2) with the virtual 'snd_dummy' driver. Hopefully it will help me get some questions I have clarified - and eventually result with a virtual ALSA driver, that does not trigger the full-duplex drop in PortAudio.

On 2013-07-25 10:37, Clemens Ladisch wrote:

...

...

[...] and you are not correctly reporting the number of samples transferred to the hardware?

I agree that it must be the core of the problem - but I have problem understanding why, given I currently perceive that I'm doing everything right: I know I have a rate of 44100 frames per second; I choose either a period for timer functions, and calculate bytes per period to match the rate, or vice versa; and in each period, I increase stream buffer positions for that bytes per period amount (taking care of buffer wrapping).

Your driver's .pointer callback must report the *actual* position at which the hardware has finished reading from the buffer. You *must* read some hardware register of your DMA controller for this.

I understand this - and agree with it, if I had such a case case, where my driver would talk to an actual hardware card. However, since here I'm interested in the operation of a virtual (platform) driver, which talks to _no_ soundcard hardware - how could I possibly read a hardware register, related to a card that doesn't exist? (Maybe it's the "...transferred to the hardware..." mention at the start of the quote, that gave the wrong impression of my focus in this case? My statement in the quote, refers to what I'm trying to do with the _virtual_ "dummy" driver.)

Anyways, now that it's mentioned, I wanted to make sure I've understood the reporting of the actual position at "which the hardware has finished reading from the buffer" conceptually, in context of actual soundcard hardware - so here is a diagram based on my onboard PCI 'hda-intel' card (replace ".png" with ".pdf" in link, to get a text-searchable PDF version):

http://sdaaubckp.sourceforge.net/post/alsa-capttest/montage-hda-intel.png

Here's what I'm trying to show on it: I'm assuming that the card has it's own intern capture buffer memory on board; and ALSA (the hda_intel driver, actually) manages the equivalent of `substream->runtime->dma_area` (the hda_intel driver actually manages it's own `area` pointer, as part of a `chip` structure) as capture buffer memory in RAM. The main purpose of the ALSA driver, then, is to manage the copying the data from the intern card capture buffer, to the `dma_area` capture buffer in RAM of the PC; once the data is in the `dma_area`, the rest of the ALSA engine will make sure that data ends up in `audiobuf` in user-space, upon a call to `snd_pcm_readi` as given above.

What is intended to be shown, is: the card starts filling its intern capture buffer, soon after `snd_pcm_start`; since the period is set to 32 frames, when the card reaches this boundary in its buffer, it generates an interrupt ("CardIRQ?"); the kernel reacts to this by handling this hardware interrupt, by eventually calling the `azx_interrupt()` handler of the hda-intel driver. [[[~ Obviously, I cannot measure the actual interrupts generated by the card, so the "CardIRQ?" positions are interpolated - based on reported kernel interrupt entries, but only where the `azx_interrupt` handler has been called (since it's also possible to capture interrupt entries for power, for instance); the shown filling of the buffer is then interpolated based on this. ~]]]

On the PC side, the driver's .pointer callback can be triggered both by userspace call to `snd_pcm_readi`, and (apparently) independently of it - but (surprising for me) it is not necessarily periodic! The `dma_area` filling shown is based on actual position returned by the .pointer callback (as much as space allows). Looking from a distance, it looks like the .pointer position returned, seems to track well the (idealized) filling of the intern capture buffer. [[[~ however, this may also be due to the .pointer callback (`azx_pcm_pointer()`) being occasionally called in quick succession (apparently in context of `snd_pcm_capture_ioctl()` function). ~]]]

Is the above understood correctly? And does the observation, that (apparently): * the "filling" of `dma_area` buffer on the PC side "tracks well" the "filling" of the intern capture buffer on the card side; ... illustrate the nature of .pointer "reporting of the actual position at which the hardware has finished reading from the buffer" correctly?

It is not possible to deduce this from the current time because the clocks do not run at the same speed, and any kind of buffering will introduce more errors.

Thanks for mentioning this - I had otherwise completely forgotten about clock domains; so this was the comment that got me working on this test round! First, I'd like to make sure I understand "the clocks do not run at the same speed" properly:

In the `montage-hda-intel.png` diagram above, there are three time axes, shown vertically. It is assumed that the card hardware has its own separate crystal oscillator (XOcard), in addition to the PC having its own crystal (XOpc) - as clock sources; consequently, they have their separate time axes "(Card) Time" and "(PC) Time", given there will always be some mismatch between the frequencies they generate. The leftmost axis is what I've called "(Real) time", and is used for no other reason, than being a reminder; I guess it would represent the clock of an "independent observer", or the "developer clock" - or the "global date & time clock" (such as retrieved by `ntpdate`). By the way, when I see "wall clock" referred to in code, does that refer to this, which I've called "Real Time"?

The diagram takes the "(Real) Time" axis to have a time unit == 1; the "(Card) Time" is set to have a 1.5% smaller (faster) unit than 1; and the "(PC) Time" is set to have a 1.5% larger (slower) unit than 1. The value of 1.5% is chosen arbitrarily, so the mismatch is more obviously visible through the diagram axes' ticks. Does this illustrate the nature of "the clocks do not run at the same speed" properly (in general)?

Anyways - I do understand the impossibility of deduction of the (XOcard) timing, based solely on algorithms running on (XOpc) timing. And I agree it would have been a problem, if I was in a context of working with actual soundcard hardware.

However, since I'm inquiring about a virtual driver - there is no actual card hardware, and consequently no actual (Xocard) crystal oscillator/clock; as you've noted:

The dummy driver uses a timer because there is no actual hardware.

Right - I have tried to visualise this on the diagram below (again, replace `.png` with `.pdf` for a PDF version):

http://sdaaubckp.sourceforge.net/post/alsa-capttest/montage-dummy.png

That diagram should clearly show, that in a context of a virtual (platform) "dummy" driver, there is no actual card hardware targetted - nor a corresponding oscillator (with a corresponding independent clock domain). The large red arrows, that used to indicate the hardware IRQ in the `hda-intel` diagram, now indicate the timer functions softirq entry - and since they originate from the CPU itself, they now point from the other side (note also I'm reusing the "CardIRQ?" engine, to also render these interrupt entries on the card axis, under the cross-out). There is only one buffer (`dma_area`'s) filling shown - and as there is no intern card buffer now, there is nothing to compare this `dma_area`'s filling process to; the filling shown is based solely on what the .pointer callback returns.

And herein lies the crux of my inquiery: given there is no hardware targetted with a virtual driver, I can in principle return whatever I want as a .pointer position. Logic would say that the value returned from pointer, should increase (in this case) by 32 (frames) each PCM period (726 microseconds) - and it is with this in mind, that the timer functions are ran in the dummy driver. Userspace, in principle, deals only with this layer of information - so I should be able to simulate a proper operation to userspace, just by increasing this .pointer value properly. However, even if I do that, I still manage to somehow trigger a full-duplex drop in the PortAudio userspace layer - and that means, I'm still going wrong somewhere with the .pointer position calculation, even if I believe I'm doing it right.

The full-duplex part, of course, is not handled in this test, being capture only; however, I can notice some things, that may eventually have an influence:

* The .pointer callback can be called in quick succession, in context of `snd_pcm_capture_ioctl`, with both `dummy` and `hda-intel` drivers. * In both cases, right after `snd_pcm_start` is called, .pointer is called, returning zero, BUT: ** with `hda-intel` driver, the card immediately raises an interrupt here - making for a total of 3 interrupts in a 2ms capture; ** with `dummy` driver, the timer function is just scheduled at the start - but does not fire at the start; it fires first after a period has expired - making for a total of 2 interrupts in a 2 ms capture.

Also, just by looking at `montage-hda-intel.png` vs. `montage-dummy.png`, one would gather that the hardware IRQ runs slightly faster than the expected 726 μs period; while the timer function softirq runs somewhat slower than that. However, that is inaccurate - both drivers' timing IRQ's can jitter in either direction; this is especially obvious if several captures are made, and then the corresponding plots animated, as shown on the animated .gif here:

http://sdaaubckp.sourceforge.net/post/alsa-capttest/capttest.gif

In that gif, the plots which were vertical in the montages, are now shown horizontally; 2ms capture of `dummy` is shown on top, while the capture of `hda_intel` is shown below it. It is noticeable that, first, focusing on the "(Card) Time" track, `hda-inter` fires an additional "start" interrupt, which has no counterpart in `dummy`. However, it is also noticeable that:

* Focusing on the "(Card) Time" track, the `hda-intel` hardware interrupts are much less jittery (and follow the expected 726 us period more closely) than the timer interrupts of `dummy`;

I'm not sure if this is the expected behaviour. As far as I know, the order priority of interrupts in Linux is (crudely):

hardware IRQ > softIRQ (sofftware IRQ) > task switching/scheduling > everything else

The card then, uses it's own clock, which is not burdened with anything else but filling buffers, meaning we can expect tight timing here; and when it generates an IRQ, it is handled by kernel with highest priority - ergo, not so much jitter. Timer functions, on the other hand, run in softIRQ context - meaning they (and their scheduling) could be preempted by the hardware IRQ of any other device on the system; ergo, more jitter. Is this reasonable to assume?

* When .pointer runs in quick succession, that usually results with a "correction" for `hda-intel` - but `dummy`'s position remains the same in the same situation

I just noticed this while writing this mail, otherwise I didn't pay much attention to it. But, I just remembered that the original `dummy` driver, calculates delta and returns pointer position based on it in the .pointer callback itself:

dummy_hrtimer_pointer(struct snd_pcm_substream *substream) { ... delta = ktime_us_delta(hrtimer_cb_get_time(&dpcm->timer), dpcm->base_time); ... div_u64_rem(delta, runtime->buffer_size, &pos); // this sets pos return pos; }

... and in that, could simulate the "correction" that `hda-intel` also does, when called in quick succession.

However, in my version of the `dummy` driver, I'm also trying to write a few pulses to the `dma_area` - therefore I actually manage the position (returned by .pointer) in the hrtimer tasklet in a variable `pcm_buf_pos`:

static void dummy_hrtimer_pcm_elapsed(unsigned long priv) // this is the tasklet { ... delta = ktime_us_delta(hrtimer_cb_get_time(&dpcm->timer), dpcm->base_time); ... div_u64_rem(delta, runtime->buffer_size, &pos); // this sets pos ... dpcm->pcm_buf_pos = frames_to_bytes(runtime, pos); ... }

... and in the .pointer, I simply return this number:

dummy_hrtimer_pointer(struct snd_pcm_substream *substream) { ... pos = bytes_to_frames(runtime, dpcm->pcm_buf_pos); return pos; }

Now, note that the `capttest.gif` animation shows the jitter of the timer *(soft)IRQ entry*; however, the timer function in itself just schedules the tasklet to run even later - and this is also visible in `montage-dummy.png`, where it can be seen that the tasklet `dummy_hrtimer_pcm_elapsed` usually occurs up to some 100 us μs *after* the timer IRQ entry! This probably has an influence on the .pointer position calculation - but can it be to such a degree, to cause a PortAudio drop in full-duplex mode?

Since I've mentioned writing in the `dma_area`: `hda-intel` probably schedules the DMA controller, to transfer the data from the intern capture memory, to RAM of the PC - and as such, the transfer/copy uses no CPU cycles (CPU time). While in my `dummy`, just by trying to `memset` (not even copy) a few bytes, I'm using extra CPU cycles - could this also have an influence on increased jitter?

And now that DMA is mentioned, I might as well ask again:

* What is the meaning of MMAP in context of SNDRV_PCM_INFO_MMAP? Is it: ** A memory map from the card's internal buffer, via DMA, to the `dma_area` in PC RAM; or ** A memory map from `dma_area` in kernel space, to whatever buffer is referred to in user space?

Well, I hope someone will be able to confirm, if I am right in my understanding so far - and point out where am I otherwise wrong...

Thanks in advance for any feedback, Cheers!

On 08/06/2013 12:59 PM, Clemens Ladisch wrote:

Smilen Dimitrov wrote:

...

On 2013-07-25 10:37, Clemens Ladisch wrote:

...

Your driver's .pointer callback must report the *actual* position at which the hardware has finished reading from the buffer

... for a playback stream, or finished reading, for a capture stream.

Hi Clemens,

I'm not involved with Smilen but still find the questions interesting, so as always, thanks for sharing your knowledge :-)

What if the pointer granularity is very coarse? E g, some hardware might only be able what period you're in (IIRC, I've seen this on the Tegra platform), rather than the actual sample. Would you recommend to report the latest period boundary in that case, or interpolating it with timers?

This is also interesting to PulseAudio which likes to rewind buffers and so on, and relies on a good pointer granularity.

...

The card then, uses it's own clock, which is not burdened with anything else but filling buffers, meaning we can expect tight timing here; and when it generates an IRQ, it is handled by kernel with highest priority - ergo, not so much jitter. Timer functions, on the other hand, run in softIRQ context - meaning they (and their scheduling) could be preempted by the hardware IRQ of any other device on the system; ergo, more jitter. Is this reasonable to assume?

Yes, but other hardware interrupts interfere only if other devices are used at the same time.

Also, it applies to kernel-space only. If you want to process anything in userspace, you can still be interrupted by any kernel process - hardIRQ, softIRQ or even other kernel tasks.

Hi Clemens, David,

Many thanks for bearing with me, and your feedback - I truly appreciate it! Apologies for a verbose response again; I have tried to organize this time, and push my more verbose snippets to the end of the mail; hope that helps with readability.

On 2013-08-06 12:59, Clemens Ladisch wrote:

Smilen Dimitrov wrote:

...

On 2013-07-25 10:37, Clemens Ladisch wrote:

...

I'm interested in the operation of a virtual (platform) driver, which talks to _no_ soundcard hardware

And what does this driver do? What is your goal?

For now, my goal is to develop a virtual driver (or rather, make minimal modifications to the existing `dummy` driver), which will a) work at CD-quality (44100/16b/2ch) and not trigger the full-duplex drop in PortAudio and b) write something trivial (like "pulses" at period and buffer boundary) in the capture buffer - just so I can load the driver, fire up Audacity, load a playback file, press record (which will start full-duplex in overdub mode), and see something expected be captured (without drops, and repeatedly).

The driver's purpose, in my case, would be to 1) learn what it takes for a virtual driver to not trigger full-duplex drop in PortAudio, and then 2) serve as a sort of a comparison basis (or a benchmark), against which I'd compare the operation of a similar (with timer functions) driver. I have moved the more verbose explanation in comment (*c1) at end of this mail.

...

I'm assuming that the card has it's own intern capture buffer memory on board;

No modern card has this. All data is immediately read from/written to main memory.

Thanks for noting this - I wish I knew better :) ( By the way, can anyone recommend any references, which would get me up to speed on the evolution of soundcard hardware? )

My immediate question here is - why, then, do I observe IRQ's at period boundary with `hda-intel`? But then:

...

and ALSA manages the equivalent of `substream->runtime->dma_area` as capture buffer memory in RAM. The main purpose of the ALSA driver, then, is to manage the copying the data from the intern card capture buffer, to the `dma_area` capture buffer in RAM of the PC;

This is handled by the hardware's DMA.

Aha - ok, I'll try to speculate here, to make sure I now have a more acceptable model of operation for a proper DMA card; based on the simplified DMA schematic in the previously mentioned http://sdaaubckp.sf.net/post/alsa-capttest/montage-hda-intel.png - again just discussing capture, let's say 44100/16b/2ch:

* Card uses its XO to derive a sampling clock as close as possible to 44100 Hz; * When this clock hits, card has to perform an ADC - this means it has to store 16 bits per channel somewhere (in this case, 4 bytes for the two channels) * The card, having sampled, triggers a request on the DMA bus (possibly, by "raising" DREQ) ** Since this signal doesn't utilize the CPU - it is **not** registered as an interrupt (IRQ) ** The DMA controller then makes sure control and data bus are switched soon enough, so these 4 bytes are stored in main memory, possibly asserting DACK0 for acknowledgment afterwards ** Upon ACK, card internally accumulates +4 on its capture "buffer pointer" counter ** (Thus, this DMA "interrupt"/request, (sort of) functions as the sampling rate "timer"/trigger in the context of the PC as a whole - but not in the context of the CPU directly) * When card realizes "buffer pointer" counter is >= period_size, it triggers an IRQ proper, that interrupts the CPU - not to initiate a copy from "intern capture buffer memory" to main RAM - but to inform the OS, that now +period_size frames (bytes) are available in main RAM memory ** Driver reacts to IRQ, then eventually raises `snd_pcm_period_elapsed` (and rest of ALSA takes it from there)

Now, I would call those 4 bytes I speculate about "intern capture memory", although it's definitely not a "buffer" - maybe the more proper term for them would be "registers" (as they are few in number, and likely fast)?. Or do modern cards, for instance, hook the ADC output directly to DMA bus - so not even those 4 bytes are present as "intern capture memory" on card?

In either case - could the speculative "breakdown" above, be taken to be a more accurate approximation of the capture process with a modern card? If so, then maybe the `montage-hda-intel.png` image can still be considered somewhat applicable for modern cards - as in: the period boundary IRQ is raised _as if_ there was an intern card capture buffer of the given period/buffer size, signalling a need for copy to main memory - except there is no actual copy, since there is no intern capture buffer (and capture data goes directly to main RAM) on modern cards.

...

And herein lies the crux of my inquiery: given there is no hardware targetted with a virtual driver, I can in principle return whatever I want as a .pointer position.

If you are actually transferring sample from/to somewhere, you must return the status of these transfers.

Agreed - and my bad for saying "whatever I want"; I guess, what I was trying to emphasize, comes from my experience with AudioArduino (and conversely, lack of experience with actual PCI/DMA cards). Namely, with AudioArduino (see also (*c1)), essentially I only had to relate to one formula:

bytes_per_period = (rate_bytes/1[s])*(period_time[s])

... which is what was being returned (as increase) in .pointer there - moved the verbose to comment (*c2) at end of this mail.

In other words, I am assuming that _solely_ by increasing pointers by bytes_per_period in period_time, the driver should be able to persuade ALSA (and userspace) that transfers are going fine; I am cheating ALSA with that approach, but I'm not cheating full-duplex PortAudio :) However, it seems the PortAudio full-duplex drop is triggered by a `snd_pcm_delay` check, see comment (*c4); but even `snd_pcm_delay` depends only on appl_ptr and hw_ptr - and ultimately, on .pointer position.

...

I should be able to simulate a proper operation to userspace, just by increasing this .pointer value properly. However, even if I do that, I still manage to somehow trigger a full-duplex drop in the PortAudio userspace layer

A buffer length of about 1 ms is very likely to result in over/underruns, regardless of what your .pointer callback does.

Better try with a 32000-frame buffer first.

I think this is a key comment - I'll have to try and understand it better, because buffer sizes that big didn't really occur to me. And that is because, even if I multiply both sides of the above formula with a constant A > 1:

A*bytes_per_period = (rate_bytes/1[s])*(A*period_time[s])

... the ratio doesn't change; so it's not immediately obvious to me, why a very large buffer would have helped (I have tried with period_size = 1024 frames, and buffer apparently 2x that - that still does a full duplex drop).

The only thing I can think of is the jitter (shown on the .gifs): the (hr)timer functions can easily be off by 30 or more microseconds, which is on the order of a sample (frame) period (1/44100 ~= 22.6e-06). So for a period on order of 1ms, error would be 22e-6/1e-3 = 0.022 => 2.2%; while for period of 16000 frames, period time is 16000/44100 = 0.362812 = 362.8 ms; so error there is less: 22e-6/362e-3 = 6.07735e-05 => 0.006%. Beyond this, is there any other reason why small buffer/period size is likely to result in over/underruns?

...

...

...

...

Your driver's .pointer callback must report the *actual* position at which the hardware has finished reading from the buffer

... for a playback stream, or finished reading, for a capture stream.

What if the pointer granularity is very coarse? E g, some hardware might only be able what period you're in (IIRC, I've seen this on the Tegra platform), rather than the actual sample. Would you recommend to report the latest period boundary in that case, or interpolating it with timers?

By reporting position x, the driver guarantees that the device has finished reading (for a playback stream) before x, and that the application is allowed to overwrite the buffer before x with new sample data.

When the driver does not know the current position of the DMA controller, it must report the last known 'safe' position (and set SNDRV_PCM_INFO_BLOCK_TRANSFER).

An essential comment, much appreciated - I was not clearly aware of the "finished" aspect, and in particular of the "overwrite" aspect; will definitely take that into account from now on.

With that in mind, could the following be said?: In a virtual driver context, given that there is no underlying hardware to speak of, a report of .pointer at position x is merely an _assertion_ from the driver to ALSA: "Hey, I checked the hardware, this position x has already been processed by the stream"

...

On the PC side, the driver's .pointer callback can be triggered both by userspace call to `snd_pcm_readi`, and (apparently) independently of it - but (surprising for me) it is not necessarily periodic!

The .pointer callback is triggered by snd_pcm_period_elapsed() (because some more data, or even another period, might have been transferred in the meantime), whenever userspace writes or reads samples, or whenever userspace feels like asking for the current position.

Good to have this confirmed! By the way, I was experimenting with capturing the behavior of the original `dummy` in the meantime - and I think it nicely illustrates "some more data, ... might have been transferred in the meantime", but in a virtual driver. Here is an animated gif:

http://sdaaubckp.sourceforge.net/post/alsa-capttest/capttest_03.gif http://sdaaubckp.sourceforge.net/post/alsa-capttest/_cappics03/ (source images/PDFs)

More verbose discussion in comment (*c3) at end of this mail.

...

once the data is in the `dma_area`, the rest of the ALSA engine will make sure that data ends up in `audiobuf` in user-space, upon a call to `snd_pcm_readi` as given above.

Yes. (When using snd_pcm_mmap_*, the dma_area is mapped to userspace, and ALSA itself will never access the contents of the buffer.)

Thanks for this - it's great to have confirmed, what is supposed to be mapped to what! :)

By the way, I'm still not sure if mmap could also influence the occurence of these full duplex drops - while I can enforce use of `snd_pcm_readi` (as in the `captmini.c` test), the `dummy` driver does declare SNDRV_PCM_INFO_MMAP | SNDRV_PCM_INFO_MMAP_VALID - and I'm not sure if this may be a signal to PortAudio (when using the `patest_duplex_wire.c` test) to use the mmap'd versions of snd_pcm_ functions; I should check. Since in my version `dummy-2.6.32-patest.c`, I insist on memsetting the capture `dma_area` from the timer function tasklet; I guess it could well be, that interferes with proper mmap operation - thus making that version of the driver less robust (as mentioned in comment (*c3)) to "full-duplex drops", than the original `dummy`?

...

...

The card then, uses [its] own clock, which is not burdened with anything else but filling buffers, meaning we can expect tight timing here; and when it generates an IRQ, it is handled by kernel with highest priority - ergo, not so much jitter. Timer functions, on the other hand, run in softIRQ context - meaning they (and their scheduling) could be preempted by the hardware IRQ of any other device on the system; ergo, more jitter. Is this reasonable to assume?

Yes, but other hardware interrupts interfere only if other devices are used at the same time.

Also, it applies to kernel-space only. If you want to process anything in userspace, you can still be interrupted by any kernel process - hardIRQ, softIRQ or even other kernel tasks.

Got it - I take my development PC into consideration, and it's a netbook with a touchpad, USB mouse and ethernet network; I guess any of this could, in principle, cause hardware IRQ interference - however, I do take that as not very likely (or at least, not crucial to the full-duplex drop problem).

...

[...] where the `azx_interrupt` handler has been called (since it's also possible to capture interrupt entries for power, for instance);

The stream's SD_STS register tells whether this is an interrupt because a period boundary has been crossed.

Thanks for this too - as I don't really understand `hda-intel.c`, the only reason I emphasized this function is that I saw `snd_pcm_period_elapsed` called from there (it's also called from `azx_irq_pending_work`, but judging by that name, I doubted it'd have shown any periodic behavior on a plot). Now reading the `azx_interrupt` function makes a lot more sense.

...

when I see "wall clock" referred to in code, does that refer to this, which I've called "Real Time"?

It's called "wall clock" because Intel named the register this way; actually, it's the device's sample clock.

Heh - thanks for this, would never have guessed! :)

By the way, I just realized that `snd_pcm_update_hw_ptr0` in `sound/core/pcm_lib.c`, also refers to a SNDRV_PCM_INFO_HAS_WALL_CLOCK, (which in `include/uapi/sound/asound.h` has comment: "/* has audio wall clock for audio/system time sync */") and `substream->ops->wall_clock` (as part of `snd_pcm_ops` in `sound/pcm.h`). This part of the structure would again refer to a device's sample clock (if the device declares _HAS_WALL_CLOCK), right?

Before I wrap up, just to mention that I experimented a bit with PortAudio (`patest_duplex_wire.c`) and original `dummy` driver; and have a bit more on the conditions that trigger the full-duplex drop in comment (*c4) - it seems `snd_pcm_delay` is critical there, but even that can apparently be boiled down to .pointer positions.

Many thanks again for the excellent discussion; and I really hope also this mail will attract the same level of scrutiny - the feedback on where I'm going wrong, has really helped me get some of my misconceptions cleared,

Cheers!

:: Verbose comments:

(*c1) I've been working on http://imi.aau.dk/~sd/phd/index.php?title=AudioArduino , which should be understood as an academic exercise, simplified enough to serve as a practical introduction to soundcard operation - even if one misses a lot of details, like proper understanding of, say, PCI. That driver works using timer functions, and by basically trusting that `ftdi_write()` and the `ftdi_process_packet()` will do their thing in time: when the timer function hits for playback, I `ftdi_write` ammount of bytes per period from `dma_area` and consider it "played"; `ftdi_process_packet` is fired as interrupt from `ftdi_sio`, I collect that data in intermediary buffer - and when timer hits for capture, I `memcpy` ammount of bytes per period from intermediary to `dma_area`. All I've had to pay attention to here was memory allocations, and having proper buffer/period wrap pointer arithmetic. And even with this simplified understanding, I can program the Arduino to echo back every b yte it received from serial (what I call "digital duplex test"), load up a file in Audacity, press record, and see the played file be echoed back inside in the capture - never experienced any full-duplex drops (like here). And that driver doesn't even use hrtimer - it uses systimers, which can be unreliable up to a period of a jiffy (4ms in my case)! However, that driver works only with 44100/8bit/mono streams.

I now want to see if I can do this "digital duplex" test, but for 44100/16b/2ch (assuming the 2 Mbps bandwidth of `ftdi_sio` and Arduino will be enough). After realizing that I cannot use systimers anymore ( see http://stackoverflow.com/q/16920238/277826 ), and the switch to hrtimers, I started getting these full-duplex drops from PortAudio - even if observing the stream through an analyzer (at points TX and RX on the Arduino) would reveal that there are no interruptions in the stream, and that each byte in the played sequence is correctly "reflected"! So I thought - OK, must be something in my .pointer arithmetic is wrong, let's compare to something that works. So I tried `hda-intel`, it indeed doesn't do full-duplex drop - but since I don't really know what sort of a hardware it is, reading its driver's source doesn't help me much. Then I thought - well, let's try `dummy` - given that I took the timer approach from there, analyzing it will hopefully reveal what is wrong wit h my driver. So I just made the small modification described earlier (.pointer position calculated in tasklet, and writing of pulses in capture `dma_area`), and tried it. And imagine my surprise when `dummy` turned out to trigger these full-duplex drops in PortAudio as well! Today I just tried the original `dummy` 2.6.32 (which doesn't manipulate `dma_area` at all, and calcs .pointer position in .pointer) - with just an extra `trace_printk()` in .pointer - and while far more robust that my modified version (e.g. my version drops any time I switch workspace in Gnome with Ctrl-Alt-arrows; the original doesn't), it _still_ triggers a full-duplex drop!

So, now that my idea of using `dummy` as a comparison point has broken down, I'm actually genuinely interested in `dummy` for its own sake: given that it does nothing but increase .pointer (thus, very little CPU overhead which could influence things), and there is no hardware in respect to which we would calculate an (im)proper operation - how on earth can it trigger a full-duplex drop in PortAudio at all? And why - what is the condition that triggers it, then? Of course, I eventually hope that by understanding this, I'll be able to apply the conclusions to my 44100/16b/2ch AudioArduino case - but for now, I'd really like to have a better understanding of why this drop occurs in the context of a virtual driver to begin with. One problem could be, that so far I've taken that userspace has to decide whether operation is proper solely based on what .pointer reports - there are likely other variables at play here, too; most importantly, `snd_pcm_delay` -see (*c4).

(*c2) I guess, what I was trying to emphasize, comes from my experience with AudioArduino (and conversely, lack of experience with actual PCI/DMA cards). Namely, with AudioArduino (see also (*c1)), essentially I only had to relate to one formula:

bytes_per_period = (rate_bytes/1[s])*(period_time[s])

So, in that case, I knew I had 44100Hz/8b/mono, which translates to rate of 44100 Bytes/s. I also knew I was going to use systimer functions, set at a period of a jiffy (on my platform, 4ms) -> so, bytes_per_period = 44100*4e-3 = 176.4 ~= 176. So, at each timer function:

* for playback - I `ftdi_write` 176 bytes from `dma_area`, and increase `->pcm_buf_pos` by 176 * for capture - I memcpy 176 bytes from intermediate buffer to `dma_area`, and increase `->pcm_buf_pos` by 176

... and in .pointer, I return `bytes_to_frames(->pcm_buf_pos)` for either direction.

So - as I saw it at that point - there is no other information about the status of the transfers, other than the increase of the .pointer positions! Now, it's well a standing question if this is the _best_ way to solve this driver - for instance, maybe I should have used .copy/.silence callbacks (instead of dealing with `dma_area` inside the timer functions). However, it was _good enough_, in that I never experienced a full-duplex drop (nor other problems) in Audacity with it. So I reasoned: it must be that ALSA keeps track of time; then the only thing it needs, so as to check whether constant rate is held, is to see that pointer positions increase for the right ammount. And since here I explicitly write that ammount in each direction (playback or capture) in the driver - ALSA is kept "happy", and that propagates to userspace.

Note that from this, to a pure virtual driver, there is only one step - one simply stops using the `ftdi_` functions, and stops manipulating the `dma_area`s. (not surprising, given that the whole timer function technique I saw first in `dummy`). That is what made me believe, that there is nothing else but the .pointer positions, that have an influence the on a proper streaming operation (as ALSA would see it); when I said I can "return whatever I want", I really meant - I can explicitly return a position increase, that should match the requested transfer rate, as per the above formula (which, as I see it, should "cheat" ALSA that all is fine).

However, I make a virtual driver with timer functions, set it to 44100Hz/16b/stereo (= 176000 Bytes/s, so four times the transfer rate of AudioArduino) - and I observe a full-duplex drop in PortAudio. So, obviously the model, where the only thing that matters is the above formula in context of timer functions, breaks down for this rate (on my platform at least) - apparently starting to break some timing constraints, maybe not necesarilly in ALSA, but definitely in PortAudio. And I'd like to learn more precisely what condition causes this breakage. In (*c4) I can see `snd_pcm_delay` has an influence - but seemingly, it also relies in great part on reported .pointer position.

(*c3) A capture of the original `dummy` driver, with `captmini.c`:

http://sdaaubckp.sourceforge.net/post/alsa-capttest/capttest_03.gif http://sdaaubckp.sourceforge.net/post/alsa-capttest/_cappics03/ (source images/PDFs)

Basically ((in simplified terms, given `snd_pcm_update_hw_ptr0` is more complex, and does delay and XRUN calculations)): when .pointer is called from `snd_pcm_update_hw_ptr0`, it seems it is called repeatedly in quick succession, until at least `hw_ptr` for the substream matches the .pointer position. With the original `dummy` driver, since it calculates delays directly in the .pointer callback - it's values can keep on increasing by 1 or frames, while the rest of the code is updating `hw_ptr`; and so .pointer called to update again, and this goes on (apparently) until `snd_pcm_update_hw_ptr0` is satisfied - that it got enough frames according to the time expired, - which can be 5 or more times in quick succession. My modification `dummy-2.6.32-patest.c`, since it calculates the .pointer position in the timer tasklet, typically stays unchanged when `snd_pcm_update_hw_ptr0` inquires, so .pointer from there is called max 2 times in quick succession ( e.g. as on http://sdaaubckp .sf.net/post/alsa-capttest/montage-dummy.png ).

But then, there is something strange again - if you focus on the top part of capttest_03.gif, which shows the behavior of the original `dummy` driver, it shows a somewhat comparable jitter to `hda-intel` - which is apparently much better than what my modified `dummy-2.6.32-patest.c` showed (on http://sdaaubckp.sourceforge.net/post/alsa-capttest/capttest.gif top part). I find this surprising, because - regardless of a) if the .pointer position is calculated in the tasklet or in .pointer; or b) if the tasklet had anything more to do than just call `snd_pcm_period_elapsed` - this should not have an influence on when the timer softirq first fires, since _both_ drivers re-schedule the timer function (`dummy_hrtimer_callback`) as first thing when it enters, and only then schedule a tasklet?

Having noted this, and looking back at http://sdaaubckp.sf.net/post/alsa-capttest/montage-hda-intel.png , I would have found the triplet of `.pointer` at about 1.20 ms also somewhat strange - namely, the .pointer has increased from 41 to 49 (causing the third call to .pointer from `snd_pcm_update_hw_ptr0`, until it ultimately syncs) - but there is no card IRQ running at the time? Then what could have changed the .pointer, which for `hda-intel` could be something like `azx_dev->posbuf`? This, of course ties in with my earlier (wrong) understanding that only card IRQ initiates a copy to main capture buffer; but if this "copy" happens transparently via DMA (like in my speculative breakdown above), then I see where this update - "more data transferred in the meantime" - could come from.

(*c4) Having experimented with `patest_duplex_wire.c` (set to 512 frames per period), a debug version of PortAudio, and the original `dummy` 2.6.32 driver (with but a `trace_printk` in its .pointer function); I realized that when a full-duplex drop occurs, the PortAudio log looks like this:

... Pa_IsStreamActive returned: PaError: 1 ( Invalid error code (value greater than zero) ) ContinuePoll: Trying to poll again for playback frames, pollTimeout: 6 dly:507 mrg:251 <<<<<<<< ContinuePoll: Trying to poll again for playback frames, pollTimeout: 6 dly:498 mrg:242 ContinuePoll: Trying to poll again for playback frames, pollTimeout: 6 dly:495 mrg:239 ContinuePoll: Trying to poll again for playback frames, pollTimeout: 6 dly:492 mrg:236 ContinuePoll: Trying to poll again for playback frames, pollTimeout: 6 dly:490 mrg:234 ContinuePoll: Trying to poll again for playback frames, pollTimeout: 6 dly:487 mrg:231 ContinuePoll: Trying to poll again for playback frames, pollTimeout: 6 dly:484 mrg:228 ContinuePoll: Trying to poll again for playback frames, pollTimeout: 6 dly:481 mrg:225 ContinuePoll: Trying to poll again for playback frames, pollTimeout: 6 dly:478 mrg:222 ContinuePoll: Trying to poll again for playback frames, pollTimeout: 5 dly:475 mrg:219 ContinuePoll: Trying to poll again for playback frames, pollTimeout: 5 dly:472 mrg:216 ... ContinuePoll: Trying to poll again for playback frames, pollTimeout: 1 dly:261 mrg:5 ContinuePoll: Trying to poll again for playback frames, pollTimeout: 1 dly:258 mrg:2 ContinuePoll: Stopping poll for playback PaAlsaStream_WaitForFrames: full-duplex (not xrun): Drop input, a period's worth - fra:769 ContinuePoll: Stopping poll for capture CallbackThreadFunc: Input underflow fra:777 urn:0 orn:0 CallbackThreadFunc: Input underflow fra:265 urn:0 orn:0 >>>>>>>> play index = 45056 ; rec/capt index = 88064 Pa_IsStreamActive called: PaStream* stream: 0x0x99037e0 Pa_IsStreamActive returned: PaError: 1 ( Invalid error code (value greater than zero) ) ...

When a `patest_duplex_wire.c` completes successfully (without a drop), it turns out the section between the ">>>" and "<<<" is never in the logs - meaning the "ContinuePoll" messages, in addition to the "Drop input" and input underflow messages, are also a sign of a drop. (NB: It seems there is also such correlation between `ContinuePoll` and the full-duplex drop in logs from starting post of this thread; although `ContinuePoll` there can also appear in a context of an XRUN.)

So I looked into `src/hostapi/alsa/pa_linux_alsa.c`, and `ContinuePoll` does this:

... snd_pcm_delay( otherComponent->pcm, &delay ) ... ... if( StreamDirection_Out == streamDir ) { /* Number of eligible frames before capture overrun */ delay = otherComponent->bufferSize - delay; } margin = delay - otherComponent->framesPerBuffer / 2; if( margin < 0 ) { ... PA_DEBUG(( "%s: Stopping poll .... *continuePoll = 0; } else if( margin < otherComponent->framesPerBuffer ) { *pollTimeout = CalculatePollTimeout( stream, margin ); PA_DEBUG(( "%s: Trying to poll again for %s frames, pollTimeout: %d dly:%d mrg:%d\n", __FUNCTION__, StreamDirection_In == streamDir ? "capture" : "playback", *pollTimeout , delay, margin )); // modded }

So, apparently `snd_pcm_delay` is being used for calculation of `margin` and `delay` - and should the `margin` drop to below zero, the poll is stopped. Apparently, this polling is started, ended and continued from `PaAlsaStream_WaitForFrames`, where there is the following loop:

while( pollPlayback || pollCapture ) { ... /* @concern FullDuplex If only one of two pcms is ready we may want to compromise between the two. * If there is less than half a period's worth of samples left of frames in the other pcm's buffer we will * stop polling. */ if( self->capture.pcm && self->playback.pcm ) { if( pollCapture && !pollPlayback ) { PA_ENSURE( ContinuePoll( self, StreamDirection_In, &pollTimeout, &pollCapture ) ); } else if( pollPlayback && !pollCapture ) { PA_ENSURE( ContinuePoll( self, StreamDirection_Out, &pollTimeout, &pollPlayback ) ); } } } // end while

So, `ContinuePoll` can set `pollCapture` or `pollPlayback` (via `*continuePoll`) to 0, which will break the while loop - and right after this while loop, is the `if( !xrun ) ...` check, cited in the starting post of this thread, which determines the full-duplex drop in PortAudio.

In other words, now it looks like it is a `snd_pcm_delay` check failure, that triggers the full-duplex drop in PortAudio. And as a reminder, this check fails for "otherComponent" stream - specifically, full-duplex drop happens if we're in full-duplex mode (so both capture and playback are running), and **playback** is not ready; since the condition for the full-duplex drop is: `if( self->capture.pcm && self->playback.pcm ) { if( !self->playback.ready && !self->neverDropInput ) ...`. (( I'm still not clear on what sets `playback.ready` to 0 - `ContinuePoll` apparently doesn't ))

So, given I haven't met `snd_pcm_delay` by now, I think I should look more into it. The docs say: "For playback ... It is as such the overall latency from the write call to the final DAC. For capture ... It is as such the overall latency from the initial ADC to the read call.", which I have a problem translating to virtual driver context (given there is no actual ADCs nor DACs). However, I found this thread:

"[alsa-devel] What does snd_pcm_delay() actually return?" http://mailman.alsa-project.org/pipermail/alsa-devel/2008-June/008421.html > In the driver implementation level, snd_pcm_delay() simply returns the > difference between appl_ptr and hw_ptr. It means how many samples are > ahead on the buffer from the point currently being played. > However, if you stop feeding samples now, snd_pcm_delay() returns the > least time XRUN occurs. [...] > The implementation of snd_pcm_delay() (at least in the driver level) > purely depends on the accuracy of PCM pointer callback of each > driver. So, if the driver returns more accurate hw_ptr via pointer > callback, you'll get more accurate value of snd_pcm_delay(). In the > worst case, it may be bigger up to one period size than the real > delay.

... so one way or another, it boils down to appl_ptr, hw_ptr - and what is being returned as .pointer position. I think the next thing, is to see what triggers the "ContinuePoll" altogether - since its seems its presence in PortAudio debug logs is not really a good sign.

While I have learned that modern cards do not have on-board buffers,

Take a look at Intel® High Definition Audio Specification Document Change Notification 4.6 Energy Efficient HD Audio (EEAudio) Mechanism

There is controller (HW) buffer <local fifo> in the Figure 14 HD Audio DMA and buffering

Hi all,

Since the PortAudio full-duplex problem I have, depends on both the capture and playback direction - I thought I'd also look into full-duplex, at least from the perspective of ALSA, and post a writeup on that. This is again done by comparing the `hda_intel` and `dummy` ALSA drivers, but ran under a different set of userspace programs. Even though I'll have specific questions, please let me know if anything pokes your eye as wrong in this email; and as always, apologies for the verbosity.

First of all, I wondered if there is a "plain ALSA" program, which would be an equivalent to PortAudio's `patest(_duplex)_wire.c`, and could be used to do a full-duplex test. I never would have guessed by the name alone, but it seems there is - in form of `alsa-lib`'s `test/latency.c`. I had quite a hard time figuring out exactly how this `latency.c` is supposed to behave - so after playing with it a bit, I took the liberty to post a page about it on the wiki:

http://www.alsa-project.org/main/index.php/Test_latency.c

... which also includes some valuable snippets from `alsa-devel` discussions - hope that's OK. While the snippets go a long way in explaining a lot (such as the latency being visible as the playback stream being a number of frames ahead of the capture one), what confuses me is this:

* I specify `-m 128 -M 128` as arguments, which forces latency=128 (frames) in the code * The program confirms that, saying: "Trying latency 128 frames ..." * Before the read/write loop, `writebuf(phandle, buffer, latency ...)` is called **twice**, and yet: ** the very first `appl_ptr` obtained from a playback .pointer is 128 (in most of my captures) ** At "Success" end, we may have Playback: *** frames = 44288 *** and Capture: *** frames = 44160 *** (like in the output under #Usage on the wiki page); and their difference is 44288-44160 = 128

So my question is: how is it possible to do a write of 128 frames *twice*; and yet still get both the first `appl_ptr`, and p/c stream difference to be 128? Shouldn't it be 2*128 = 256, if we started with writing 128 frames twice?

Then, I wondered if there is an ALSA equivalent to PortAudio's use of a single/"wire" callback to specify "proper"/"hard" full-duplex operation. To me, it seems that it is the command `snd_pcm_link`, for which the docs say: "The two PCMs will start/stop/prepare in sync". Is this what defines a proper full-duplex operation - that the playback and capture streams are running in sync? A confirmation for this can be seen to come from portaudio-v19/src/hostapi/alsa/pa_linux_alsa.c:

/* this will cause the two streams to automatically start/stop/prepare in sync. * We only need to execute these operations on one of the pair. * A: We don't want to do this on a blocking stream. */ if( self->callbackMode && self->capture.pcm && self->playback.pcm ) { int err = snd_pcm_link( self->capture.pcm, self->playback.pcm ); if( err == 0 ) self->pcmsSynced = 1; else PA_DEBUG(( "%s: Unable to sync pcms: %s\n", __FUNCTION__, snd_strerror( err ) )); }

... where we have the only instance of `snd_pcm_link`, being used only when capture and playback streams both exist in the same context (besides the callbackMode) - which, I gather, is only possible when a single/"wire" callback is used; is this correct? But then, why wouldn't we want to do that on a blocking stream? Also, would that "blocking" refer to actual blocking file descriptors only - or can it also be understood to include polling waits as well?

Anyways, I have again posted the scripts for, and debug log acquisitions + visualisations of, the `latency` tests in:

http://sdaaubckp.sourceforge.net/post/alsa-capttest/

There is a modified version of `latency.c` used there, called `latency-mod.c`; the drivers are supposed to have `trace_printk`s as per `latency-drv.patch`; the related scripts are `run-alsa-lattest.sh`, `traceFGLatLogfile2Csv.py`, `traceFGLatLogGraph.gp` and `lat-anim.pl`. The debug logs are in `captures-alsa-captlat.tar.xz`, and their PDF plots are in the `_cappics05/` subfolder. Please see the end of the updated `Readme`, in the same directory, for more information about these.

To begin with, I tried acquiring the operation of `hda-intel` under `latency-mod.c` with various parameters, to see how it would behave - these parameters are encoded in the respective file/directory names. First, here is `hda-intel` driver (and card), under `latency-mod.c` without poll (`np`), without Round Robin scheduling (`ns`) and without blocking (`nb`), with latency 128 frames, and duration of test 256 frames:

http://sdaaubckp.sf.net/post/alsa-capttest/captlat-hda-128-256-np-0-ns-nb.gi...

(( Note that all of the provided acquisitions are without Round Robin scheduling (`ns`), with latency 128 frames, and duration of test 256 frames. Also most of the acquisitions represent successfully completed `latency` runs - except for the last two, which are indicated with an `-f` in their names ))

The .gif, as usual, is not very legible - but you can view the PDFs of the individual frames given in the `_cappics05` subfolder. What is clear though, is that .pointer is being called very frequently - and the reason for this, are the very frequent calls to `snd_pcm_readi`; which are frequent because neither blocking, nor polling is used (and so `readbuf` in `latency-mod.c` spins in its loop as fast as it can).

Once we start using polling (`yp`) with `latency`, the situation settles down; note that "polling" in `latency.c` uses a `snd_pcm_wait` to wait a given time (in milliseconds) on the capture file handle. As I had some problems with poll at first, I thought it was due to the poll waiting time, which was fixed to 1000 ms in the original `latency.c` - so I added a `--polltime` option in `latency-mod.c`. After some reboots, I couldn't reproduce that problem any more, and it looks like the operation with polling is more-less the same with 1 ms wait time vs. 1000 ms wait time:

http://sdaaubckp.sf.net/post/alsa-capttest/captlat-hda-128-256-yp-1-ns-nb.gi... http://sdaaubckp.sf.net/post/alsa-capttest/captlat-hda-128-256-yp-1000-ns-nb...

Finally, I tried using blocking (`yb`) with `latency` instead - and it turns out, it doesn't look all that much different from polling with `hda-intel`:

http://sdaaubckp.sf.net/post/alsa-capttest/captlat-hda-128-256-np-0-ns-yb.gi...

... and so I finally settled on using these `latency` settings (128-256-np-0-ns-yb) for later acquisition of `dummy` operation as well. Anyways, in terms of `hda-intel`, I gather the following can be said generally for the operation under `latency(-mod).c`:

* The operation starts with two or three violet "CardIRQ?"s, spanning approx a quarter of the buffer period (the time corresponding to period_size) ** (violet "CardIRQ?" means that an audio driver interrupt handler function, here `azx_interrupt`, has been detected - but no .pointer was called in its context) * Approx a whole buffer period later from the very first violet "CardIRQ?", the first "proper" "CardIRQ?" is fired, and it is capture (blue) * Approx a quarter of the buffer period later from the first capture "CardIRQ?", the first playback (red) "CardIRQ?" fires * The capture/playback "CardIRQ?" then repeat after a buffer period respectively (keeping the quarter buffer period phase between them) until the end of the program

This seems to match the behavior of the separate capture and playback "montage"s for `hda-intel`, discussed earlier in the thread. The interesting thing for me here is: even if the `latency` program starts by doing two `snd_pcm_writei`s - the driver operation still starts with a capture IRQ, and the playback series is delayed in respect to capture for a quarter buffer period; is this correctly surmised?

For a more detailed analysis, the `lat-anim.pl` script can be used to render a single debug acquisition as an animation; here is one such example for `hda-intel` under `latency`:

http://sdaaubckp.sf.net/post/alsa-capttest/captlat-2013-09-02-23-30-38-hda-1...

... try it with:

vlc --repeat --rate 3.0 captlat-2013-09-02-23-30-38-hda-128-256-np-0-ns-yb.mpg

Again, the supplied `.mpg`s are not very legible - however, you can use `lat-anim.pl` and the provided debug logs to generate the high-res animation frames (and videos) yourself; although note that rendering frames from a debug acquisition like the above takes about an hour on my machine (see the `Readme` for more). By default, an animation frame is 800x600, and in that format looks like this:

http://sdaaubckp.sf.net/post/alsa-capttest/captlat-2013-09-11-00-36-17-duM-1...

On the left hand side, there is visualization of playback (left block) and capture (right block) pointers - from top to bottom:

* For playback, `appl_ptr` changes first - `_pointer` and then `hw_ptr` try to follow it * For capture, `_pointer` changes first - `hw_ptr` and then `appl_ptr` try to follow it

On the right hand side, there is the "Card Time" lane overlay, showing the a repeat of the `_pointer` values (violet); and interpolated playback (red) and capture (blue) position (again, see `Readme` for more). The interpolated positions start with the first proper capture or playback "cardIRQ?", and then grow as per sampling rate period (here 1/44100), so they simulate the card sampling clock. The animation in this lane shows that:

* The capture and playback streams indeed seem to be close to synchronized - as expected from `snd_pcm_link` - by looking at their interpolated positions (the small delta between those values is likely due to calculation errors) * The respective `_pointers` do seem to track closely the interpolated positions - and they typically cross about half a buffer (in this case, a period_size), when they update (close to their respective "cardIRQ?"s)

The interesting thing here for me, is the fact that the _pointer/interpolated positions indeed seem to be synchronized - even if the capture and playback IRQ sequences are quarter buffer period apart! At first, I expected that since the c/p IRQ sequences are quarter buffer period apart - then also the respective interpolated/_pointer values would also have to be quarter buffer_size (half period_size) apart!? But seen like this, it seems that at the very first (violet) cardIRQ, the sampling clock is started synchronously for both playback and capture on the card - and so, _even_ if the playback IRQs hit a little bit later, the PC still is informed of relatively accurate card values (esp. given that, as surmised earlier in the thread, the `_pointer` values seem to propagate from the card to the PC via DMA, and thus do not rely on interrupts "reporting" them). Is this somewhat correctly understood?

[[ This confused me greatly, especially since one of the very first captures (which I also used for developing the scripts) was this:

http://sdaaubckp.sf.net/post/alsa-capttest/_cappics05/captlat-2013-08-20-09-...

There the very first playback _pointer has a value of 48 - barely 200 μs (duration of some 8 frames) after the `snd_pcm_start`! However, it is about 48/128 = (1/2.66667)th of the buffer_size, whereas a quarter buffer_size is 32 frames - and since I got this in no other debug log, I guess this was an anomaly/error of sorts (even if `latency` reported successful completion). ]]

Further on to the `dummy` driver, the original version (with the extra `trace_printk`), behaves like this for a blocking `latency` test:

http://sdaaubckp.sf.net/post/alsa-capttest/captlat-dum-128-256-np-0-ns-yb.gi...

Here, even with blocking, .pointer is quite frequently called - however, as noted earlier in this thread, that is because this driver version re-calculates the _pointer value in the .pointer function; and so the value of _pointer changes by the time `hw_ptr` is set to _pointer's last value. And since in the capture direction, ALSA insists having `hw_ptr` set to _pointer, it repeats this process.

The behavior of the modified `dummy` driver (`dummy-2.6.32-patest.c` with a respective `trace_printk`) looks like this:

http://sdaaubckp.sf.net/post/alsa-capttest/captlat-duM-128-256-np-0-ns-yb.gi...

Since `dummy-2.6.32-patest` recalculates _pointer values in the hrtimer callbacks (and the .pointer function simply returns it) - the ALSA engine can quickly update `hw_ptr` to the _pointer value without many repetitions; and so it resembles the behavior `hda-intel` a bit more. Apparently, due to the `snd_pcm_link`, the hrtimer callbacks for playback and capture schedule very close to one another - causing the future callbacks to be answered in the context of a single IRQ; however, also here, the capture callback seems to be answered first. Looking at one of its debug logs as an animation:

http://sdaaubckp.sf.net/post/alsa-capttest/captlat-2013-09-11-02-21-08-duM-1...

... also the update behavior of the pointers and the interpolated positions seems close to the behavior of `hda-intel`.

While also `hda-intel`can fail (with an XRUN) in a `latency` test like above, I was more interested in a failure of the modified `dummy-2.6.32-patest` - so here is one such debug plot:

http://sdaaubckp.sf.net/post/alsa-capttest/_cappics05/captlat-2013-09-11-13-...

... and as an animation:

http://sdaaubckp.sf.net/post/alsa-capttest/captlat-2013-09-11-13-46-43-duM-1...

At end of this run, `latency-mod.c` would have reported playback/capture frames at end, and an array logging the status of each `readbuf`/`writebuf` call in `latency-mod.c`:

* 192/128 xrun: [rd wr] 0: [64 64] 1: [64 -32] 2: [0 0] 3: [0 0] ...

... which would mean that the second `writebuf` failed. However, in the PDF plot, we can see the first `_readi` called right at start; after the first "cardIRQ?", first `_writei` and second `_readi` are called. In the successful acquisitions, this pattern repeats to the end - but in this failing one, the failing `_writei` is called after an `xrun()` is decided in context of the third playback "cardIRQ?" (this is not shown on the PDF - you'll have to look at the `.csv` file to see the `xrun()`). This means that - for some reason - both `_writei and `_readi` are *skipped* after the second "cardIRQ?"; and I cannot really tell what would be the reason for it. I may try to look into this further - but as I remember, the PortAudio full-duplex problem was not a typical XRUN, so maybe I'll go back directly to PortAudio from this point.

Thanks for bearing with me so far - and I'm looking forward to any answers/corrections, Cheers!

Hi list(s),

I hope I'll be forgiven for bumping all lists again - just wanted to confirm that indeed, the problem I stated in the start of this thread, was not with Audacity nor PortAudio nor ALSA as such (even with the older versions I've used); the problem was my modification of the ALSA `dummy` driver, submitted previously as `dummy-2.6.32-patest-fix.c` (or "dummy-mod" for short).

I'd also announce, that it seems that the driver `dummy-2.6.32-patest-fix.c` (or "dummy-fix"), uploaded in this directory:

http://sdaaubckp.sourceforge.net/post/alsa-patest-duplex/fix/

... fixes the problems with the full-duplex "drop input" I experienced in Audacity - but I'll still use this opportunity to ask some questions. I'll try to be as brief as I can here, some more info is in the `Readme` in the `fix` directory.

As a reminder, the full-duplex "drop input" looked like this for me:

http://sdaaubckp.sourceforge.net/post/alsa-patest-duplex/fix/dummy-mod-fddro...

... or, as the screenshot shows, very soon after a start of capture in full-duplex mode, `dummy-mod` would trigger a full-duplex "drop input" in PortAudio, which would propagate to Audacity.

The behavior of `dummy-fix` now is like this:

http://sdaaubckp.sourceforge.net/post/alsa-patest-duplex/fix/dummy-fix-ok.pn...

... or, as the screenshot shows, Audacity can now run for 10 mins in full-duplex capture mode with `dummy-fix`, without a full-duplex "drop input" being triggered; which is as good as I need it, I guess.

First, though, I have a question on the "nature" of full-duplex. The way I see it now, there are two distinct contexts for full-duplex use, which I'd call:

* monitoring - you want to listen on the speakers, what is recorded on the microphone (playback of the capture stream) * studio/overlay recording - you want to play a background track, and you want to record the singer singing to that track "in sync" with the played track

In the monitoring case, I guess one doesn't care much for stream synchronization - input stream will arrive when it arrives (after inherent latencies of the system); in the mean time you can just play silence - and as soon as input data is available, you can play that too; it is "full-duplex" only in the sense that the playback and capture streams are running "at the same time" generally. On the other hand, for "overlay" recording, one would probably want the recorded stream as closely synchronized as possible to the playback stream. Is this correctly understood?

From this, I guess that ALSA's `latency.c` achieves the full-duplex synchronization (of the "overlay" kind) by calling `snd_pcm_link`, *and* by writing a 2*period_size worth of playback data (let's call this playback pre-buffer) *before* the full-duplex operation starts. However, I couldn't see anything like this "playback pre-buffer" in PortAudio, even if `pa_linux_alsa.c` does call `snd_pcm_link`. Then, I couldn't see a "playback pre-buffer" in PortAudio's `patest_duplex.c` either, but I thought maybe this program is meant to demonstrate a full-duplex of the monitoring kind (and thus it doesn't need such prebuffering). So my question is - does PortAudio do this kind of playback pre-buffer that I may have missed; and if it doesn't, does Audacity do it?

Back to topic - so, while I suspected anything from the massive printouts from ALSA/PortAudio debugs and kernel message printouts to (un)reliability of hrtimers in the Linux kernel as the cause of trouble, it turns out that isn't the problem - the issue got solved as soon as I managed to simulate the IRQ .pointer behavior of `hda-intel`, as timer callback .pointer behavior within `dummy-fix`.

First of all, the full-duplex "drop input" seems to be triggered, initially, by a polling error of the playback stream in PortAudio. I'm still not exactly clear on which stream it is (due to the PortAudio code using "thisComponent" and "otherComponent"), but both the error condition of `snd_pcm_playback_poll` in ALSA, and further behavior of the `margin` variable in the PortAudio code in `ContinuePoll`, seem to indicate that a hw_ptr is not increasing. While the original `snd-dummy` always recalculates the .pointer position in the .pointer function - I had moved that calculation in the timer callback in `dummy-mod`, and the .pointer function then simply returned the last calculated value. So in `dummy-fix`, .pointer function again recalculates the position (almost) every call - but this was not the entirety of the fix.

The fix is in simulation of this behavior of `hda-intel`:

For period sizes > 64 frames; the period IRQ (or timer function) for the playback stream should be delayed early for some 48 frames (at CD quality, 48/44100 = 1.088 ms); however it should return the proper expected .pointer position (at periods, that is typically N*period_size+1 in frames, where N=0,1,(2..))

Now this is what puzzles me most: _why_ should the playback stream (in particular) be delayed early? I noticed this behavior by first analyzing the period IRQ positions of `hda-intel` (the plot shows use of both ALSA `latency-mod.c`, and PortAudio `patest_duplex_wire.c`, as user-space programs):

http://sdaaubckp.sourceforge.net/post/alsa-patest-duplex/fix/collectmirq_hda...

Note here that: even as period sizes increase - the playback (red) is delayed early from the capture (blue), for approximately the same amount of time. As this plots (as closely as possible) the IRQs the card issues, that means that the card hardware actually issues the playback interrupts early. Why?

In comparison, in `dummy-mod` there were no discernible time offsets between capture and playback timer callbacks:

http://sdaaubckp.sourceforge.net/post/alsa-patest-duplex/fix/collectmirq_duM...

... and seemingly, this is what caused the full-duplex drop. Now, `dummy-fix` behaves rather similarly to `hda-intel` in that respect:

http://sdaaubckp.sourceforge.net/post/alsa-patest-duplex/fix/collectmirq_duF...

... and here is a plot, that shows how `dummy-fix` approximates `hda-intel` a bit more closely - for period_size 256 and buffer_size 512 frames:

http://sdaaubckp.sourceforge.net/post/alsa-patest-duplex/fix/cmirq_hda_duF_5...

Another interesting thing is, that `hda-intel` does not behave the same for period_size <= 64 frames; in that case, the playback is delayed late, not early. In earlier mails in this thread, I tried to analyze smaller period sizes (so as to limit the ammount of kernel data to be analyzed and plotted) - and this made me interpret the offsets as "quarter period"; obviously that approach failed. (Other problems I had was 16UL*100000000UL not actually fitting in unsigned long, but requiring unsigned long long; and a bug in Gnuplot when using palette, which inverted the capture and playback colors in the plots, making me code the wrong offsets). The interesting thing, though, is that when I try to simulate that behavior in `dummy-fix`:

http://sdaaubckp.sourceforge.net/post/alsa-patest-duplex/fix/cmirq_hda_duF_1...

... even if the behavior is quite close (left half is `hda-intel`, right half is `dummy-fix`), the `dummy-fix` tends to XRUN a _lot_ in that case; however, it should be said that `hda-intel` also tends to XRUN quite a bit (though not as much) for period_size 64 frames. Going back to running the periods (timer callbacks) of capture and playback streams without significant offsets (close to each other):

http://sdaaubckp.sourceforge.net/post/alsa-patest-duplex/fix/cmirq_hda_duF_1...

... seems to make `dummy-fix` much more reliable (very few XRUNs). Why?

Finally, there is also some test code, that allows for acquiring .pointer positions with Audacity - this code is somewhat simpler though, and renders using the timestamps of the .pointer printouts (not the timestamps of the causing IRQ/timers, which would have happened a bit earlier), but still looks good enough, I guess. This is the comparison of `hda-intel` and `dummy-fix` in full-duplex mode:

http://sdaaubckp.sourceforge.net/post/alsa-patest-duplex/fix/collectmirq_aci...

... but interestingly, Audacity (or PortAudio) seems to settle on slightly different period_sizes for `hda-intel` vs. `dummy-fix` in capture-only mode:

http://sdaaubckp.sourceforge.net/post/alsa-patest-duplex/fix/collectmirq_aci...

... or period_size / buffer_size (period_per_buffer) as a table (for my dev platform, at least):

| audacity | capture-only | full-duplex | |----------------------------------------------| |dummy-fix | 1102 / 4408 (4) | 2048 / 4096 (2) | |hda-intel | 1088 / 4352 (4) | 2048 / 4096 (2) | ------------------------------------------------

Would anyone have an idea, why would Audacity (or PortAudio?) choose the same settings for the two drivers in full-duplex mode, but differing settings in capture-only mode?

To summarize, I haven't really found the exact conditions which trigger the full-duplex drop input detection in PortAudio - but it seems I've fixed the problem, by replicating the early delay of playback vs. capture timers behavior of `hda-intel`; hope it's robust enough, so I don't come back crying to the list(s) about new significant bugs found `:)` However, I'd still love to hear if anyone has answers to my questions above - or to a more simplified understanding of what condition actually triggers this drop (or, indeed, any comments `:)`).

Many thanks for all the responses in this thread so far (most of it found on alsa-devel) - I doubt I would have arrived at this point without that help; much appreciated, Cheers!