[alsa-devel] [PATCH 0/2] RFC: support for audio wall clock
V2 of patches to show how the audio wall clock (eg. the HDAudio one) can be used to provide precise audio timestamps and track the drift between audio time and system time. On my laptop I am able to track a 7ppm delta. I will explain in more details how this can be used during LPC in San Diego, but with the summer coming I'd like to get feedback as there are changes to the core/alsa-lib that haven't been modified in years.
TODO: - 64-bit alignment, verification of pcm_compat.c changes - handling of digital inputs, need to disable wall-clock timestamps in that case (feedback from Clemens) - check differences in protocol - change timecounter to avoid accumulation of rounding errors in cycle->ns conversions - check overflows - rework the timecounter init on trigger
Pierre-Louis Bossart (2): ALSA: core: add hooks for audio timestamps read from WALLCLOCK ALSA: hda: support for wallclock timestamps
include/sound/asound.h | 7 ++++- include/sound/pcm.h | 2 + sound/core/pcm_compat.c | 13 +++++++++- sound/core/pcm_lib.c | 15 ++++++++++- sound/core/pcm_native.c | 2 + sound/pci/hda/hda_intel.c | 54 ++++++++++++++++++++++++++++++++++++++++++++- 6 files changed, 86 insertions(+), 7 deletions(-)
Add new .audio_wallclock routine to enable fine-grain synchronization between monotonic system time and audio hardware time. Prior to this patch, the clock drift estimation was handled in user-space by comparing frames and system time. Using the wallclock, if supported in hardware, allows for a much better sub-microsecond precision and a common drift tracking for all devices sharing the same wall clock (master clock).
TODO: handle 64-bit alignment
Signed-off-by: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com --- include/sound/asound.h | 7 +++++-- include/sound/pcm.h | 2 ++ sound/core/pcm_compat.c | 13 +++++++++++-- sound/core/pcm_lib.c | 15 +++++++++++++-- sound/core/pcm_native.c | 2 ++ 5 files changed, 33 insertions(+), 6 deletions(-)
diff --git a/include/sound/asound.h b/include/sound/asound.h index 0876a1e..b889585 100644 --- a/include/sound/asound.h +++ b/include/sound/asound.h @@ -152,7 +152,7 @@ struct snd_hwdep_dsp_image { * * *****************************************************************************/
-#define SNDRV_PCM_VERSION SNDRV_PROTOCOL_VERSION(2, 0, 10) +#define SNDRV_PCM_VERSION SNDRV_PROTOCOL_VERSION(2, 0, 11)
typedef unsigned long snd_pcm_uframes_t; typedef signed long snd_pcm_sframes_t; @@ -274,6 +274,7 @@ typedef int __bitwise snd_pcm_subformat_t; #define SNDRV_PCM_INFO_JOINT_DUPLEX 0x00200000 /* playback and capture stream are somewhat correlated */ #define SNDRV_PCM_INFO_SYNC_START 0x00400000 /* pcm support some kind of sync go */ #define SNDRV_PCM_INFO_NO_PERIOD_WAKEUP 0x00800000 /* period wakeup can be disabled */ +#define SNDRV_PCM_INFO_HAS_WALL_CLOCK 0x01000000 /* has audio wall clock for audio/system time sync */ #define SNDRV_PCM_INFO_FIFO_IN_FRAMES 0x80000000 /* internal kernel flag - FIFO size is in frames */
typedef int __bitwise snd_pcm_state_t; @@ -422,7 +423,8 @@ struct snd_pcm_status { snd_pcm_uframes_t avail_max; /* max frames available on hw since last status */ snd_pcm_uframes_t overrange; /* count of ADC (capture) overrange detections from last status */ snd_pcm_state_t suspended_state; /* suspended stream state */ - unsigned char reserved[60]; /* must be filled with zero */ + struct timespec audio_tstamp; /* audio wall clock timestamp */ + unsigned char reserved[60-sizeof(struct timespec)]; /* must be filled with zero */ };
struct snd_pcm_mmap_status { @@ -431,6 +433,7 @@ struct snd_pcm_mmap_status { snd_pcm_uframes_t hw_ptr; /* RO: hw ptr (0...boundary-1) */ struct timespec tstamp; /* Timestamp */ snd_pcm_state_t suspended_state; /* RO: suspended stream state */ + struct timespec audio_tstamp; /* audio wall clock timestamp */ };
struct snd_pcm_mmap_control { diff --git a/include/sound/pcm.h b/include/sound/pcm.h index 68372bc..d4d65bf 100644 --- a/include/sound/pcm.h +++ b/include/sound/pcm.h @@ -71,6 +71,8 @@ struct snd_pcm_ops { int (*prepare)(struct snd_pcm_substream *substream); int (*trigger)(struct snd_pcm_substream *substream, int cmd); snd_pcm_uframes_t (*pointer)(struct snd_pcm_substream *substream); + int (*wall_clock)(struct snd_pcm_substream *substream, + struct timespec *audio_ts); int (*copy)(struct snd_pcm_substream *substream, int channel, snd_pcm_uframes_t pos, void __user *buf, snd_pcm_uframes_t count); diff --git a/sound/core/pcm_compat.c b/sound/core/pcm_compat.c index 91cdf94..ab4c953 100644 --- a/sound/core/pcm_compat.c +++ b/sound/core/pcm_compat.c @@ -190,7 +190,8 @@ struct snd_pcm_status32 { u32 avail_max; u32 overrange; s32 suspended_state; - unsigned char reserved[60]; + struct timespec audio_tstamp; + unsigned char reserved[60-sizeof(struct timespec)]; } __attribute__((packed));
@@ -215,7 +216,9 @@ static int snd_pcm_status_user_compat(struct snd_pcm_substream *substream, put_user(status.avail, &src->avail) || put_user(status.avail_max, &src->avail_max) || put_user(status.overrange, &src->overrange) || - put_user(status.suspended_state, &src->suspended_state)) + put_user(status.suspended_state, &src->suspended_state) || + put_user(status.audio_tstamp.tv_sec, &src->audio_tstamp.tv_sec) || + put_user(status.audio_tstamp.tv_nsec, &src->audio_tstamp.tv_nsec)) return -EFAULT;
return err; @@ -364,6 +367,7 @@ struct snd_pcm_mmap_status32 { u32 hw_ptr; struct compat_timespec tstamp; s32 suspended_state; + struct compat_timespec audio_tstamp; } __attribute__((packed));
struct snd_pcm_mmap_control32 { @@ -426,12 +430,17 @@ static int snd_pcm_ioctl_sync_ptr_compat(struct snd_pcm_substream *substream, sstatus.hw_ptr = status->hw_ptr % boundary; sstatus.tstamp = status->tstamp; sstatus.suspended_state = status->suspended_state; + sstatus.audio_tstamp = status->audio_tstamp; snd_pcm_stream_unlock_irq(substream); if (put_user(sstatus.state, &src->s.status.state) || put_user(sstatus.hw_ptr, &src->s.status.hw_ptr) || put_user(sstatus.tstamp.tv_sec, &src->s.status.tstamp.tv_sec) || put_user(sstatus.tstamp.tv_nsec, &src->s.status.tstamp.tv_nsec) || put_user(sstatus.suspended_state, &src->s.status.suspended_state) || + put_user(sstatus.audio_tstamp.tv_sec, + &src->s.status.audio_tstamp.tv_sec) || + put_user(sstatus.audio_tstamp.tv_nsec, + &src->s.status.audio_tstamp.tv_nsec) || put_user(scontrol.appl_ptr, &src->c.control.appl_ptr) || put_user(scontrol.avail_min, &src->c.control.avail_min)) return -EFAULT; diff --git a/sound/core/pcm_lib.c b/sound/core/pcm_lib.c index 8f312fa..beece7d 100644 --- a/sound/core/pcm_lib.c +++ b/sound/core/pcm_lib.c @@ -315,6 +315,7 @@ static int snd_pcm_update_hw_ptr0(struct snd_pcm_substream *substream, unsigned long jdelta; unsigned long curr_jiffies; struct timespec curr_tstamp; + struct timespec audio_tstamp;
old_hw_ptr = runtime->status->hw_ptr;
@@ -326,9 +327,17 @@ static int snd_pcm_update_hw_ptr0(struct snd_pcm_substream *substream, */ pos = substream->ops->pointer(substream); curr_jiffies = jiffies; - if (runtime->tstamp_mode == SNDRV_PCM_TSTAMP_ENABLE) + if (runtime->tstamp_mode == SNDRV_PCM_TSTAMP_ENABLE) { snd_pcm_gettime(runtime, (struct timespec *)&curr_tstamp);
+ if ((runtime->hw.info & SNDRV_PCM_INFO_HAS_WALL_CLOCK) && + (substream->ops->wall_clock)) + substream->ops->wall_clock(substream, &audio_tstamp); + else + /* no audio tstamp available, initialize anyway */ + audio_tstamp = curr_tstamp; + } + if (pos == SNDRV_PCM_POS_XRUN) { xrun(substream); return -EPIPE; @@ -506,8 +515,10 @@ static int snd_pcm_update_hw_ptr0(struct snd_pcm_substream *substream, runtime->hw_ptr_base = hw_base; runtime->status->hw_ptr = new_hw_ptr; runtime->hw_ptr_jiffies = curr_jiffies; - if (runtime->tstamp_mode == SNDRV_PCM_TSTAMP_ENABLE) + if (runtime->tstamp_mode == SNDRV_PCM_TSTAMP_ENABLE) { runtime->status->tstamp = curr_tstamp; + runtime->status->audio_tstamp = audio_tstamp; + }
return snd_pcm_update_state(substream, runtime); } diff --git a/sound/core/pcm_native.c b/sound/core/pcm_native.c index 53b5ada..c5206f4 100644 --- a/sound/core/pcm_native.c +++ b/sound/core/pcm_native.c @@ -594,6 +594,8 @@ int snd_pcm_status(struct snd_pcm_substream *substream, snd_pcm_update_hw_ptr(substream); if (runtime->tstamp_mode == SNDRV_PCM_TSTAMP_ENABLE) { status->tstamp = runtime->status->tstamp; + status->audio_tstamp = + runtime->status->audio_tstamp; goto _tstamp_end; } }
reuse code from clocksource to handle wall clock counter. Since wrapparound occurs, the timestamps are reinitialized with monotonic time on a trigger.
TODO: - only re-init timecounter if there was no device active. - Keep the same timestamp for all devices on same chip. - make sure no overflow occurs in the 125/3 scaling implementation
Signed-off-by: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com --- sound/pci/hda/hda_intel.c | 54 ++++++++++++++++++++++++++++++++++++++++++++- 1 files changed, 53 insertions(+), 1 deletions(-)
diff --git a/sound/pci/hda/hda_intel.c b/sound/pci/hda/hda_intel.c index 86758dd..9bb795b 100644 --- a/sound/pci/hda/hda_intel.c +++ b/sound/pci/hda/hda_intel.c @@ -46,6 +46,10 @@ #include <linux/mutex.h> #include <linux/reboot.h> #include <linux/io.h> + +#include <linux/clocksource.h> +#include <linux/time.h> + #ifdef CONFIG_X86 /* for snoop control */ #include <asm/pgtable.h> @@ -496,6 +500,10 @@ struct azx {
/* reboot notifier (for mysterious hangup problem at power-down) */ struct notifier_block reboot_notifier; + + struct timecounter azx_tc; + struct cyclecounter azx_cc; + };
/* driver types */ @@ -1700,6 +1708,45 @@ static inline void azx_release_device(struct azx_dev *azx_dev) azx_dev->opened = 0; }
+static cycle_t azx_cc_read(const struct cyclecounter *cc) +{ + struct azx *chip = container_of(cc, struct azx, azx_cc); + + return azx_readl(chip, WALLCLK); +} + +static void azx_timecounter_init(struct azx *chip) +{ + struct timecounter *tc = &chip->azx_tc; + struct cyclecounter *cc = &chip->azx_cc; + struct timespec ts; + u64 nsec; + + cc->read = azx_cc_read; + cc->mask = CLOCKSOURCE_MASK(32); + +#define NBITS_NS 12 /* accuracy issues in cyc2ns with lower number of bits */ + cc->mult = clocksource_khz2mult(24000, NBITS_NS); + cc->shift = NBITS_NS; + + ktime_get_ts(&ts); + nsec = timespec_to_ns(&ts); + timecounter_init(tc, cc, nsec); +} + +static int azx_get_wallclock_tstamp(struct snd_pcm_substream *substream, + struct timespec *ts) +{ + struct azx_pcm *apcm = snd_pcm_substream_chip(substream); + struct azx *chip = apcm->chip; + u64 nsec; + + nsec = timecounter_read(&chip->azx_tc); + *ts = ns_to_timespec(nsec); + + return 0; +} + static struct snd_pcm_hardware azx_pcm_hw = { .info = (SNDRV_PCM_INFO_MMAP | SNDRV_PCM_INFO_INTERLEAVED | @@ -1709,6 +1756,7 @@ static struct snd_pcm_hardware azx_pcm_hw = { /* SNDRV_PCM_INFO_RESUME |*/ SNDRV_PCM_INFO_PAUSE | SNDRV_PCM_INFO_SYNC_START | + SNDRV_PCM_INFO_HAS_WALL_CLOCK | SNDRV_PCM_INFO_NO_PERIOD_WAKEUP), .formats = SNDRV_PCM_FMTBIT_S16_LE, .rates = SNDRV_PCM_RATE_48000, @@ -1982,8 +2030,10 @@ static int azx_pcm_trigger(struct snd_pcm_substream *substream, int cmd) } spin_unlock(&chip->reg_lock); if (start) { - if (nsync == 1) + if (nsync == 1) { + azx_timecounter_init(chip); return 0; + } /* wait until all FIFOs get ready */ for (timeout = 5000; timeout; timeout--) { nwait = 0; @@ -1999,6 +2049,7 @@ static int azx_pcm_trigger(struct snd_pcm_substream *substream, int cmd) break; cpu_relax(); } + azx_timecounter_init(chip); } else { /* wait until all RUN bits are cleared */ for (timeout = 5000; timeout; timeout--) { @@ -2240,6 +2291,7 @@ static struct snd_pcm_ops azx_pcm_ops = { .prepare = azx_pcm_prepare, .trigger = azx_pcm_trigger, .pointer = azx_pcm_pointer, + .wall_clock = azx_get_wallclock_tstamp, .mmap = azx_pcm_mmap, .page = snd_pcm_sgbuf_ops_page, };
2012/6/29 Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com:
V2 of patches to show how the audio wall clock (eg. the HDAudio one) can be used to provide precise audio timestamps and track the drift between audio time and system time. On my laptop I am able to track a 7ppm delta. I will explain in more details how this can be used during LPC in San Diego, but with the summer coming I'd like to get feedback as there are changes to the core/alsa-lib that haven't been modified in years.
TODO:
- 64-bit alignment, verification of pcm_compat.c changes
- handling of digital inputs, need to disable wall-clock timestamps in that case (feedback from Clemens)
- check differences in protocol
- change timecounter to avoid accumulation of rounding errors in cycle->ns conversions
- check overflows
- rework the timecounter init on trigger
Do suspsend/resume affect the usage of the wall clock ?
Those au88x0 also have a 32 bit register which return the ticks from a 12.288 Mz oscillator (i.e. 256 x 48000Hz) , this register wrap around in around 349.5 seconds
Can this clock be used to determine the drift too ?
On 6/29/2012 8:03 PM, Raymond Yau wrote:
Do suspsend/resume affect the usage of the wall clock ? Those au88x0 also have a 32 bit register which return the ticks from a 12.288 Mz oscillator (i.e. 256 x 48000Hz) , this register wrap around in around 349.5 seconds Can this clock be used to determine the drift too ?
In theory any clock can be used, as long as it's synchronous with the bitclock used in the serial stream. I am planning to use our 19.2 MHz clock on non HDAudio platforms, a 12.288 would work at well. The cycle count will be translated to ns, the initial clock base has a marginal impact on the result. I haven't validated anything related to suspend/resume. It should work if the trigger is invoked on resume where the audio wallclock is reinitialized with the system time -Pierre
At Thu, 28 Jun 2012 16:12:33 -0500, Pierre-Louis Bossart wrote:
V2 of patches to show how the audio wall clock (eg. the HDAudio one) can be used to provide precise audio timestamps and track the drift between audio time and system time. On my laptop I am able to track a 7ppm delta. I will explain in more details how this can be used during LPC in San Diego, but with the summer coming I'd like to get feedback as there are changes to the core/alsa-lib that haven't been modified in years.
Sorry for the late reply. I've looked over the things around my vacation...
This version looks better, so if other people have no objection, I can merge it. The extension of mmap_status seems still fitting with 64bit char size, so it should be OK.
But, one thing -- can't we reuse the normal tstamp field for the audio wallclock? That is, add a new tstamp mode, and then put wallclock tstamp there instead of the normal system tstamp. Just an idea.
thanks,
Takashi
TODO:
- 64-bit alignment, verification of pcm_compat.c changes
- handling of digital inputs, need to disable wall-clock timestamps in that case (feedback from Clemens)
- check differences in protocol
- change timecounter to avoid accumulation of rounding errors in cycle->ns conversions
- check overflows
- rework the timecounter init on trigger
Pierre-Louis Bossart (2): ALSA: core: add hooks for audio timestamps read from WALLCLOCK ALSA: hda: support for wallclock timestamps
include/sound/asound.h | 7 ++++- include/sound/pcm.h | 2 + sound/core/pcm_compat.c | 13 +++++++++- sound/core/pcm_lib.c | 15 ++++++++++- sound/core/pcm_native.c | 2 + sound/pci/hda/hda_intel.c | 54 ++++++++++++++++++++++++++++++++++++++++++++- 6 files changed, 86 insertions(+), 7 deletions(-)
-- 1.7.6.5
Alsa-devel mailing list Alsa-devel@alsa-project.org http://mailman.alsa-project.org/mailman/listinfo/alsa-devel
On 7/18/2012 12:10 PM, Takashi Iwai wrote:
Sorry for the late reply. I've looked over the things around my vacation... This version looks better, so if other people have no objection, I can merge it. The extension of mmap_status seems still fitting with 64bit char size, so it should be OK. But, one thing -- can't we reuse the normal tstamp field for the audio wallclock? That is, add a new tstamp mode, and then put wallclock tstamp there instead of the normal system tstamp. Just an idea. thanks, Takashi
Thanks for reviewing the updated patches. No, the audio wall clock needs to come in addition to the regular system tstamp. This is needed for applications such as PulseAudio who try to estimate the drift between system time and audio time. If you remove the system time information, they can't program a timer any longer. Better wait a bit to merge, I need to work on all the TODOs, complete some additional validation for corner cases and get your feedback live at LPC. -Pierre
At Fri, 20 Jul 2012 14:47:24 -0500, Pierre-Louis Bossart wrote:
On 7/18/2012 12:10 PM, Takashi Iwai wrote:
Sorry for the late reply. I've looked over the things around my vacation... This version looks better, so if other people have no objection, I can merge it. The extension of mmap_status seems still fitting with 64bit char size, so it should be OK. But, one thing -- can't we reuse the normal tstamp field for the audio wallclock? That is, add a new tstamp mode, and then put wallclock tstamp there instead of the normal system tstamp. Just an idea. thanks, Takashi
Thanks for reviewing the updated patches. No, the audio wall clock needs to come in addition to the regular system tstamp. This is needed for applications such as PulseAudio who try to estimate the drift between system time and audio time. If you remove the system time information, they can't program a timer any longer. Better wait a bit to merge, I need to work on all the TODOs, complete some additional validation for corner cases and get your feedback live at LPC.
Alright. Let's cook a bit longer.
Takashi
participants (3)
-
Pierre-Louis Bossart -
Raymond Yau -
Takashi Iwai