On Tue, 2019-07-30 at 10:45 -0700, Jon Flatley wrote:
On Mon, Jul 29, 2019 at 7:23 PM Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com wrote:
On 7/29/19 7:53 PM, Ranjani Sridharan wrote:
On Mon, 2019-07-29 at 18:02 -0500, Pierre-Louis Bossart wrote:
On 7/29/19 4:53 PM, Jon Flatley wrote:
I've been working on upstreaming the bdw-rt5650 machine driver for the Acer Chromebase 24 (buddy). There seems to be an issue when first setting the hardware controls that appears to be crashing the DSP:
[ 51.424554] haswell-pcm-audio haswell-pcm-audio: FW loaded, mailbox readback FW info: type 01, - version: 00.00, build 77, source commit id: 876ac6906f31a43b6772b23c7c983ce9dcb18a19 ... [ 84.924666] haswell-pcm-audio haswell-pcm-audio: error: audio DSP boot timeout IPCD 0x0 IPCX 0x0 [ 85.260655] haswell-pcm-audio haswell-pcm-audio: ipc: -- message timeout-- ipcx 0x83000000 isr 0x00000000 ipcd 0x00000000 imrx 0x7fff0000 [ 85.273609] haswell-pcm-audio haswell-pcm-audio: error: stream commit failed [ 85.279746] System PCM: error: failed to commit stream -110 [ 85.285388] haswell-pcm-audio haswell-pcm-audio: ASoC: haswell-pcm-audio hw params failed: -110 [ 85.293963] System PCM: ASoC: hw_params FE failed -110
This happens roughly 50% of the time when first setting hardware controls after a reboot. The other 50% of the time the DSP comes up just fine and audio works fine thereafter. Adding "#define DEBUG 1" to sound/soc/intel/haswell/sst-haswell-ipc.c makes the issue occur much less frequently in my testing. Seems like a subtle timing issue.
There were timing issues encountered during the bringup of the 2015 chromebook pixel (samus) which uses the bdw-rt5677 machine driver. Those were slightly different, and manifested during repeated arecords. Both devices use the same revision of the sst2 firmware.
Any ideas for how to debug this?
this could be trying to send an IPC while you are already waiting for one to complete. we've seen this before with SOF, if the IPCs are not strictly serialized then things go in the weeds and timeout.
Pierre/Jon
In this case it looks like the DSP boot failed leading to the IPC timeout? WOndering if increasing the boot timeout would help?
I did actually try this without success.
Yes, that too. The boot timeout is typically experimentally defined, and never decreasing due to platform variations... I am still leaning more on the side of an side effect between two IPCs, the added DEBUG points to the printk which solves timing issues. The boot timeout would typically not be impacted by such changes.
I think the real struggle I'm having is finding a good debugging method that doesn't impact the timing of the IPCs significantly (as adding DEBUG seems to). This could maybe be overcome with using a stress test to reproduce. The crash only seems to occur when first booting the DSP, and so far I've been testing this by completely power cycling the machine on every test, which is very slow and tedious. So maybe the issue with DEBUG defined occurs 1 in 20 reboots rather than 1 in 2, I wouldn't know. If there's a way to reboot the DSP and reproduce this crash without rebooting the entire device that would be very helpful to me.
Maybe you've already tried this. But, how about blacklisting the audio driver and then trying a modprobe/rmmod to insert and remove themodule. This should attempt to boot the DSP upon every modprobe. But what I am not sure about is whether the rmmod would succeed if the IPC times out because the DSP has crashed.
Thanks, Ranjani
Thanks, Jon