[alsa-devel] SALSA-Lib: Playback overrun on initial PCM start
Dear List,
Using SALSA-Lib 0.0.3, I call:
snd_pcm_open (&playback_handle, argv[1], SND_PCM_STREAM_PLAYBACK, 0); snd_pcm_hw_params_malloc (&hw_params); snd_pcm_hw_params_any (playback_handle, hw_params); snd_pcm_hw_params_set_access (playback_handle, hw_params, SND_PCM_ACCESS_RW_INTERLEAVED); snd_pcm_hw_params_set_format (playback_handle, hw_params, SND_PCM_FORMAT_S16_LE)); snd_pcm_hw_params_set_rate_near (playback_handle, hw_params, 44100, 0); snd_pcm_hw_params_set_channels (playback_handle, hw_params, 2); snd_pcm_hw_params_set_buffer_size_last (playback_handle, hw_params, &bfrsize); snd_pcm_hw_params_get_buffer_size (hw_params, &bfrsize); snd_pcm_hw_params (playback_handle, hw_params); snd_pcm_hw_params_free (hw_params); snd_pcm_sw_params_malloc (&sw_params); snd_pcm_sw_params_current (playback_handle, sw_params); snd_pcm_sw_params_set_avail_min (playback_handle, sw_params, 4096); snd_pcm_sw_params_set_start_threshold (playback_handle, sw_params, 10000U); snd_pcm_sw_params_set_xfer_align (playback_handle, sw_params, 1); snd_pcm_sw_params (playback_handle, sw_params); state = snd_pcm_state (playback_handle); frames_to_deliver = snd_pcm_avail_update (playback_handle);
while ((frames_to_deliver = snd_pcm_avail_update (playback_handle)) > 4096) { snd_pcm_writei (playback_handle, &buf, 4096) }
As soon as the buffer start threshold is reached, or alternatively if I manually start PCM stream, the PCM stream reports an overrun (i.e. "Broken Pipe"). If I then examine the PCM state, it is in the overrun (XRUN) state.
The code is being cross-compiled and tested on an ARM processor (PXA270) which is running linux kernel 2.6.21. This same code sequence -does- operate properly (in the same ARM test environment) when compiled against AlsaLib 1.0.13.
Another apparent (and perhaps relevent) difference between AlsaLib and SALSA-Lib is that if I open the PCM stream with O_NONBLOCK and then call snd_pcm_poll_descriptors, AlsaLib gives me a FD = 4, whereas SALSA-Lib reports an FD = 3;
Thanks, Scott.
At Thu, 28 Jun 2007 15:01:08 -0400, J. Scott Merritt wrote:
Dear List,
Using SALSA-Lib 0.0.3, I call:
snd_pcm_open (&playback_handle, argv[1], SND_PCM_STREAM_PLAYBACK, 0); snd_pcm_hw_params_malloc (&hw_params); snd_pcm_hw_params_any (playback_handle, hw_params); snd_pcm_hw_params_set_access (playback_handle, hw_params, SND_PCM_ACCESS_RW_INTERLEAVED); snd_pcm_hw_params_set_format (playback_handle, hw_params, SND_PCM_FORMAT_S16_LE)); snd_pcm_hw_params_set_rate_near (playback_handle, hw_params, 44100, 0); snd_pcm_hw_params_set_channels (playback_handle, hw_params, 2); snd_pcm_hw_params_set_buffer_size_last (playback_handle, hw_params, &bfrsize); snd_pcm_hw_params_get_buffer_size (hw_params, &bfrsize); snd_pcm_hw_params (playback_handle, hw_params); snd_pcm_hw_params_free (hw_params);
snd_pcm_sw_params_malloc (&sw_params); snd_pcm_sw_params_current (playback_handle, sw_params); snd_pcm_sw_params_set_avail_min (playback_handle, sw_params, 4096); snd_pcm_sw_params_set_start_threshold (playback_handle, sw_params, 10000U); snd_pcm_sw_params_set_xfer_align (playback_handle, sw_params, 1); snd_pcm_sw_params (playback_handle, sw_params);
state = snd_pcm_state (playback_handle); frames_to_deliver = snd_pcm_avail_update (playback_handle);
while ((frames_to_deliver = snd_pcm_avail_update (playback_handle)) > 4096) { snd_pcm_writei (playback_handle, &buf, 4096) }
As soon as the buffer start threshold is reached, or alternatively if I manually start PCM stream, the PCM stream reports an overrun (i.e. "Broken Pipe"). If I then examine the PCM state, it is in the overrun (XRUN) state.
What are the buffer and period sizes? Did you try alsa-lib with hw, too, right?
The code is being cross-compiled and tested on an ARM processor (PXA270) which is running linux kernel 2.6.21. This same code sequence -does- operate properly (in the same ARM test environment) when compiled against AlsaLib 1.0.13.
Well, it might not be a bug. SALSA-lib is more straightforward communication to the driver, and skips many layers. Thus it may behave differently.
Another apparent (and perhaps relevent) difference between AlsaLib and SALSA-Lib is that if I open the PCM stream with O_NONBLOCK and then call snd_pcm_poll_descriptors, AlsaLib gives me a FD = 4, whereas SALSA-Lib reports an FD = 3;
Ditto.
Takashi
On Fri, 29 Jun 2007 11:25:16 +0200 Takashi Iwai tiwai@suse.de wrote:
At Thu, 28 Jun 2007 15:01:08 -0400, J. Scott Merritt wrote:
Dear List,
Using SALSA-Lib 0.0.3, I call:
snd_pcm_open (&playback_handle, argv[1], SND_PCM_STREAM_PLAYBACK, 0); snd_pcm_hw_params_malloc (&hw_params); snd_pcm_hw_params_any (playback_handle, hw_params); snd_pcm_hw_params_set_access (playback_handle, hw_params, SND_PCM_ACCESS_RW_INTERLEAVED); snd_pcm_hw_params_set_format (playback_handle, hw_params, SND_PCM_FORMAT_S16_LE)); snd_pcm_hw_params_set_rate_near (playback_handle, hw_params, 44100, 0); snd_pcm_hw_params_set_channels (playback_handle, hw_params, 2); snd_pcm_hw_params_set_buffer_size_last (playback_handle, hw_params, &bfrsize); snd_pcm_hw_params_get_buffer_size (hw_params, &bfrsize); snd_pcm_hw_params (playback_handle, hw_params); snd_pcm_hw_params_free (hw_params);
snd_pcm_sw_params_malloc (&sw_params); snd_pcm_sw_params_current (playback_handle, sw_params); snd_pcm_sw_params_set_avail_min (playback_handle, sw_params, 4096); snd_pcm_sw_params_set_start_threshold (playback_handle, sw_params, 10000U); snd_pcm_sw_params_set_xfer_align (playback_handle, sw_params, 1); snd_pcm_sw_params (playback_handle, sw_params);
state = snd_pcm_state (playback_handle); frames_to_deliver = snd_pcm_avail_update (playback_handle);
while ((frames_to_deliver = snd_pcm_avail_update (playback_handle)) > 4096) { snd_pcm_writei (playback_handle, &buf, 4096) }
As soon as the buffer start threshold is reached, or alternatively if I manually start PCM stream, the PCM stream reports an overrun (i.e. "Broken Pipe"). If I then examine the PCM state, it is in the overrun (XRUN) state.
What are the buffer and period sizes? Did you try alsa-lib with hw, too, right?
In sample program above, buffer size is set to "last", which results in a buffer size of 32768 (which matches the allocation in the kernel driver). The sample programs above does not establish a period size and is unknown.
I added code to set the period size to 1024 frames and experienced the same (overrun) result. I also forced the buffer size to 8192 and lowered the sample rate to 16 kHz and did not see any improvement.
Yes, the same exact code runs properly on the target hardware with alsa-lib 1.0.13.
The code is being cross-compiled and tested on an ARM processor (PXA270) which is running linux kernel 2.6.21. This same code sequence -does- operate properly (in the same ARM test environment) when compiled against AlsaLib 1.0.13.
Well, it might not be a bug. SALSA-lib is more straightforward communication to the driver, and skips many layers. Thus it may behave differently.
Do you have any suggestions on how to debug an overrun problem that occurs whenever PCM is started ?
Absent further suggestions or requests, I will simply switch back to alsa-lib and give up on saving the extra 1/2 Mbyte of RAM.
Thanks, Scott.
At Fri, 29 Jun 2007 11:24:09 -0400, J. Scott Merritt wrote:
On Fri, 29 Jun 2007 11:25:16 +0200 Takashi Iwai tiwai@suse.de wrote:
At Thu, 28 Jun 2007 15:01:08 -0400, J. Scott Merritt wrote:
Dear List,
Using SALSA-Lib 0.0.3, I call:
snd_pcm_open (&playback_handle, argv[1], SND_PCM_STREAM_PLAYBACK, 0); snd_pcm_hw_params_malloc (&hw_params); snd_pcm_hw_params_any (playback_handle, hw_params); snd_pcm_hw_params_set_access (playback_handle, hw_params, SND_PCM_ACCESS_RW_INTERLEAVED); snd_pcm_hw_params_set_format (playback_handle, hw_params, SND_PCM_FORMAT_S16_LE)); snd_pcm_hw_params_set_rate_near (playback_handle, hw_params, 44100, 0); snd_pcm_hw_params_set_channels (playback_handle, hw_params, 2); snd_pcm_hw_params_set_buffer_size_last (playback_handle, hw_params, &bfrsize); snd_pcm_hw_params_get_buffer_size (hw_params, &bfrsize); snd_pcm_hw_params (playback_handle, hw_params); snd_pcm_hw_params_free (hw_params);
snd_pcm_sw_params_malloc (&sw_params); snd_pcm_sw_params_current (playback_handle, sw_params); snd_pcm_sw_params_set_avail_min (playback_handle, sw_params, 4096); snd_pcm_sw_params_set_start_threshold (playback_handle, sw_params, 10000U); snd_pcm_sw_params_set_xfer_align (playback_handle, sw_params, 1); snd_pcm_sw_params (playback_handle, sw_params);
state = snd_pcm_state (playback_handle); frames_to_deliver = snd_pcm_avail_update (playback_handle);
while ((frames_to_deliver = snd_pcm_avail_update (playback_handle)) > 4096) { snd_pcm_writei (playback_handle, &buf, 4096) }
As soon as the buffer start threshold is reached, or alternatively if I manually start PCM stream, the PCM stream reports an overrun (i.e. "Broken Pipe"). If I then examine the PCM state, it is in the overrun (XRUN) state.
What are the buffer and period sizes? Did you try alsa-lib with hw, too, right?
In sample program above, buffer size is set to "last", which results in a buffer size of 32768 (which matches the allocation in the kernel driver). The sample programs above does not establish a period size and is unknown.
I added code to set the period size to 1024 frames and experienced the same (overrun) result. I also forced the buffer size to 8192 and lowered the sample rate to 16 kHz and did not see any improvement.
Yes, the same exact code runs properly on the target hardware with alsa-lib 1.0.13.
OK, looks like a bug in salsa-lib, then. I fixed some remaining bugs and release 0.0.4 now. Please give it a try. (Note that it might take some time until the ftp server is exported / mirrored.)
thanks,
Takashi
On Fri, 29 Jun 2007 18:07:20 +0200 Takashi Iwai tiwai@suse.de wrote:
At Fri, 29 Jun 2007 11:24:09 -0400, J. Scott Merritt wrote:
On Fri, 29 Jun 2007 11:25:16 +0200 Takashi Iwai tiwai@suse.de wrote:
At Thu, 28 Jun 2007 15:01:08 -0400, J. Scott Merritt wrote:
Dear List,
Using SALSA-Lib 0.0.3, I call:
snd_pcm_open (&playback_handle, argv[1], SND_PCM_STREAM_PLAYBACK, 0); snd_pcm_hw_params_malloc (&hw_params); snd_pcm_hw_params_any (playback_handle, hw_params); snd_pcm_hw_params_set_access (playback_handle, hw_params, SND_PCM_ACCESS_RW_INTERLEAVED); snd_pcm_hw_params_set_format (playback_handle, hw_params, SND_PCM_FORMAT_S16_LE)); snd_pcm_hw_params_set_rate_near (playback_handle, hw_params, 44100, 0); snd_pcm_hw_params_set_channels (playback_handle, hw_params, 2); snd_pcm_hw_params_set_buffer_size_last (playback_handle, hw_params, &bfrsize); snd_pcm_hw_params_get_buffer_size (hw_params, &bfrsize); snd_pcm_hw_params (playback_handle, hw_params); snd_pcm_hw_params_free (hw_params);
snd_pcm_sw_params_malloc (&sw_params); snd_pcm_sw_params_current (playback_handle, sw_params); snd_pcm_sw_params_set_avail_min (playback_handle, sw_params, 4096); snd_pcm_sw_params_set_start_threshold (playback_handle, sw_params, 10000U); snd_pcm_sw_params_set_xfer_align (playback_handle, sw_params, 1); snd_pcm_sw_params (playback_handle, sw_params);
state = snd_pcm_state (playback_handle); frames_to_deliver = snd_pcm_avail_update (playback_handle);
while ((frames_to_deliver = snd_pcm_avail_update (playback_handle)) > 4096) { snd_pcm_writei (playback_handle, &buf, 4096) }
As soon as the buffer start threshold is reached, or alternatively if I manually start PCM stream, the PCM stream reports an overrun (i.e. "Broken Pipe"). If I then examine the PCM state, it is in the overrun (XRUN) state.
What are the buffer and period sizes? Did you try alsa-lib with hw, too, right?
In sample program above, buffer size is set to "last", which results in a buffer size of 32768 (which matches the allocation in the kernel driver). The sample programs above does not establish a period size and is unknown.
I added code to set the period size to 1024 frames and experienced the same (overrun) result. I also forced the buffer size to 8192 and lowered the sample rate to 16 kHz and did not see any improvement.
Yes, the same exact code runs properly on the target hardware with alsa-lib 1.0.13.
OK, looks like a bug in salsa-lib, then. I fixed some remaining bugs and release 0.0.4 now. Please give it a try. (Note that it might take some time until the ftp server is exported / mirrored.)
Much better :)
I no longer get the PlayBack overrun on PCM start. However, it appears that "poll" or the snd_pcm_sw_params_set_avail_min (=4096) is not being properly honored. With the normal alsa-lib, when the "poll" returns snd_pcm_avail_update reports that space is available for 4096 frames. However, in my initial testing, when running with SALSA lib, only about 100 frames or so are available when the poll returns.
I will examine this more closely.
At Fri, 29 Jun 2007 12:26:45 -0400, J. Scott Merritt wrote:
On Fri, 29 Jun 2007 18:07:20 +0200 Takashi Iwai tiwai@suse.de wrote:
At Fri, 29 Jun 2007 11:24:09 -0400, J. Scott Merritt wrote:
On Fri, 29 Jun 2007 11:25:16 +0200 Takashi Iwai tiwai@suse.de wrote:
At Thu, 28 Jun 2007 15:01:08 -0400, J. Scott Merritt wrote:
Dear List,
Using SALSA-Lib 0.0.3, I call:
snd_pcm_open (&playback_handle, argv[1], SND_PCM_STREAM_PLAYBACK, 0); snd_pcm_hw_params_malloc (&hw_params); snd_pcm_hw_params_any (playback_handle, hw_params); snd_pcm_hw_params_set_access (playback_handle, hw_params, SND_PCM_ACCESS_RW_INTERLEAVED); snd_pcm_hw_params_set_format (playback_handle, hw_params, SND_PCM_FORMAT_S16_LE)); snd_pcm_hw_params_set_rate_near (playback_handle, hw_params, 44100, 0); snd_pcm_hw_params_set_channels (playback_handle, hw_params, 2); snd_pcm_hw_params_set_buffer_size_last (playback_handle, hw_params, &bfrsize); snd_pcm_hw_params_get_buffer_size (hw_params, &bfrsize); snd_pcm_hw_params (playback_handle, hw_params); snd_pcm_hw_params_free (hw_params);
snd_pcm_sw_params_malloc (&sw_params); snd_pcm_sw_params_current (playback_handle, sw_params); snd_pcm_sw_params_set_avail_min (playback_handle, sw_params, 4096); snd_pcm_sw_params_set_start_threshold (playback_handle, sw_params, 10000U); snd_pcm_sw_params_set_xfer_align (playback_handle, sw_params, 1); snd_pcm_sw_params (playback_handle, sw_params);
state = snd_pcm_state (playback_handle); frames_to_deliver = snd_pcm_avail_update (playback_handle);
while ((frames_to_deliver = snd_pcm_avail_update (playback_handle)) > 4096) { snd_pcm_writei (playback_handle, &buf, 4096) }
As soon as the buffer start threshold is reached, or alternatively if I manually start PCM stream, the PCM stream reports an overrun (i.e. "Broken Pipe"). If I then examine the PCM state, it is in the overrun (XRUN) state.
What are the buffer and period sizes? Did you try alsa-lib with hw, too, right?
In sample program above, buffer size is set to "last", which results in a buffer size of 32768 (which matches the allocation in the kernel driver). The sample programs above does not establish a period size and is unknown.
I added code to set the period size to 1024 frames and experienced the same (overrun) result. I also forced the buffer size to 8192 and lowered the sample rate to 16 kHz and did not see any improvement.
Yes, the same exact code runs properly on the target hardware with alsa-lib 1.0.13.
OK, looks like a bug in salsa-lib, then. I fixed some remaining bugs and release 0.0.4 now. Please give it a try. (Note that it might take some time until the ftp server is exported / mirrored.)
Much better :)
I no longer get the PlayBack overrun on PCM start. However, it appears that "poll" or the snd_pcm_sw_params_set_avail_min (=4096) is not being properly honored. With the normal alsa-lib, when the "poll" returns snd_pcm_avail_update reports that space is available for 4096 frames. However, in my initial testing, when running with SALSA lib, only about 100 frames or so are available when the poll returns.
I will examine this more closely.
Found out another bug. Try the patch below.
Thanks for reporting!
Takashi
diff -r ad2c81608e10 src/pcm.c --- a/src/pcm.c Fri Jun 29 17:59:52 2007 +0200 +++ b/src/pcm.c Fri Jun 29 19:12:04 2007 +0200 @@ -857,6 +857,7 @@ static int snd_pcm_hw_mmap_status(snd_pc pcm->mmap_status = NULL; goto no_mmap; } + pcm->mmap_control->avail_min = 1; return 0;
no_mmap: @@ -865,6 +866,7 @@ static int snd_pcm_hw_mmap_status(snd_pc return -ENOMEM; pcm->mmap_status = &pcm->sync_ptr->s.status; pcm->mmap_control = &pcm->sync_ptr->c.control; + pcm->mmap_control->avail_min = 1; _snd_pcm_sync_ptr(pcm, 0); return 0; } diff -r ad2c81608e10 src/pcm_params.c --- a/src/pcm_params.c Fri Jun 29 17:59:52 2007 +0200 +++ b/src/pcm_params.c Fri Jun 29 19:12:04 2007 +0200 @@ -1052,6 +1052,7 @@ int snd_pcm_sw_params(snd_pcm_t *pcm, sn if (ioctl(pcm->fd, SNDRV_PCM_IOCTL_SW_PARAMS, params) < 0) return -errno; pcm->sw_params = *params; + pcm->mmap_control->avail_min = params->avail_min; return 0; }
On Fri, 29 Jun 2007 19:15:00 +0200 Takashi Iwai tiwai@suse.de wrote:
At Fri, 29 Jun 2007 12:26:45 -0400, J. Scott Merritt wrote:
I no longer get the PlayBack overrun on PCM start. However, it appears that "poll" or the snd_pcm_sw_params_set_avail_min (=4096) is not being properly honored. With the normal alsa-lib, when the "poll" returns snd_pcm_avail_update reports that space is available for 4096 frames. However, in my initial testing, when running with SALSA lib, only about 100 frames or so are available when the poll returns.
I will examine this more closely.
Found out another bug. Try the patch below.
Thanks for reporting!
Takashi
diff -r ad2c81608e10 src/pcm.c --- a/src/pcm.c Fri Jun 29 17:59:52 2007 +0200 +++ b/src/pcm.c Fri Jun 29 19:12:04 2007 +0200 @@ -857,6 +857,7 @@ static int snd_pcm_hw_mmap_status(snd_pc pcm->mmap_status = NULL; goto no_mmap; }
pcm->mmap_control->avail_min = 1; return 0;
no_mmap:
@@ -865,6 +866,7 @@ static int snd_pcm_hw_mmap_status(snd_pc return -ENOMEM; pcm->mmap_status = &pcm->sync_ptr->s.status; pcm->mmap_control = &pcm->sync_ptr->c.control;
- pcm->mmap_control->avail_min = 1; _snd_pcm_sync_ptr(pcm, 0); return 0;
} diff -r ad2c81608e10 src/pcm_params.c --- a/src/pcm_params.c Fri Jun 29 17:59:52 2007 +0200 +++ b/src/pcm_params.c Fri Jun 29 19:12:04 2007 +0200 @@ -1052,6 +1052,7 @@ int snd_pcm_sw_params(snd_pcm_t *pcm, sn if (ioctl(pcm->fd, SNDRV_PCM_IOCTL_SW_PARAMS, params) < 0) return -errno; pcm->sw_params = *params;
- pcm->mmap_control->avail_min = params->avail_min; return 0;
}
That did the trick :)
Many thanks for the extremely quick and accurate assistance !
Best regards, Scott.
Dear List,
Using SALSA-Lib 0.0.4, I call:
snd_pcm_open (&playback_handle, argv[1], SND_PCM_STREAM_PLAYBACK, O_NONBLOCK); snd_pcm_hw_params_malloc (&hw_params) snd_pcm_hw_params_any (playback_handle, hw_params) snd_pcm_hw_params_set_access (playback_handle, hw_params, SND_PCM_ACCESS_RW_INTERLEAVED) snd_pcm_hw_params_set_format (playback_handle, hw_params, SND_PCM_FORMAT_S16_LE) snd_pcm_hw_params_set_rate (playback_handle, hw_params, 16000, 0) snd_pcm_hw_params_set_channels (playback_handle, hw_params, 2) snd_pcm_hw_params_set_buffer_size_last (playback_handle, hw_params, &bfrsize) snd_pcm_hw_params_set_period_size (playback_handle, hw_params, 1024, 0) snd_pcm_hw_params (playback_handle, hw_params) snd_pcm_hw_params_free (hw_params); snd_pcm_sw_params_malloc (&sw_params) snd_pcm_sw_params_current (playback_handle, sw_params) snd_pcm_sw_params_set_avail_min (playback_handle, sw_params, 1024) snd_pcm_sw_params_set_start_threshold (playback_handle, sw_params, 4096U) snd_pcm_sw_params (playback_handle, sw_params) snd_pcm_prepare (playback_handle)
int pdcount = snd_pcm_poll_descriptors_count (playback_handle); struct pollfd fds [10]; pdcount = snd_pcm_poll_descriptors (playback_handle, fds, 10);
while ((frames_to_deliver = snd_pcm_avail_update (playback_handle)) > 1024) { snd_pcm_writei (playback_handle, &buf, 1024) }
while (1) { retval = poll (fds, 1, 5000);
frames_to_deliver = snd_pcm_avail_update (playback_handle); fprintf (stderr, "Frames to deliver was: %ld\n", frames_to_deliver);
frames_to_deliver = frames_to_deliver > 4096 ? 4096 : frames_to_deliver; playback_callback (frames_to_deliver); }
The "retval = poll (fds, 1, 5000)" statement does not appear to wait for the playback buffer to be sufficiently emptied. When this same program is run with the standard alsa-lib, the POLL statement will wait until there is room in the buffer to accomodate 1024 additional samples. When run with SALSA-lib, it appears to return immediately and snd_pcm_avail_update indicates that there is only space for about 80 more samples.
The code is being cross-compiled and tested on an ARM processor (PXA270) which is running linux kernel 2.6.21. This same code sequence -does- operate properly (in the same ARM test environment) when compiled against AlsaLib 1.0.13.
Another apparent (and perhaps relevent) difference between AlsaLib and SALSA-Lib is that if I open the PCM stream with O_NONBLOCK and then call snd_pcm_poll_descriptors, AlsaLib gives me a FD = 4, whereas SALSA-Lib reports an FD = 3;
Thanks, Scott.
On Fri, 29 Jun 2007 18:07:20 +0200 Takashi Iwai tiwai@suse.de wrote:
OK, looks like a bug in salsa-lib, then. I fixed some remaining bugs and release 0.0.4 now. Please give it a try. (Note that it might take some time until the ftp server is exported / mirrored.)
Much better :)
I no longer get the PlayBack overrun on PCM start. However, it appears that "poll" or the snd_pcm_sw_params_set_avail_min (=4096) is not being properly honored. With the normal alsa-lib, when the "poll" returns snd_pcm_avail_update reports that space is available for 4096 frames. However, in my initial testing, when running with SALSA lib, only about 100 frames or so are available when the poll returns.
I will examine this more closely.
participants (3)
-
J. Scott Merritt -
J. Scott Merritt -
Takashi Iwai