[RFC PATCH 00/14] Introduce QC USB SND audio offloading support
Several Qualcomm based chipsets can support USB audio offloading to a dedicated audio DSP, which can take over issuing transfers to the USB host controller. The intention is to reduce the load on the main processors in the SoC, and allow them to be placed into lower power modes. There are several parts to this design: 1. Adding ASoC binding layer 2. Create a USB backend for Q6DSP 3. Introduce XHCI interrupter support 4. Create vendor ops for the USB SND driver
Adding ASoC binding layer: soc-usb: Intention is to treat a USB port similar to a headphone jack. The port is always present on the device, but cable/pin status can be enabled/disabled. Expose mechanisms for USB backend ASoC drivers to communicate with USB SND.
Create a USB backend for Q6DSP: q6usb: Basic backend driver that will be responsible for maintaining the resources needed to initiate a playback stream using the Q6DSP. Will be the entity that checks to make sure the connected USB audio device supports the requested PCM format. If it does not, the PCM open call will fail, and userpsace ALSA can take action accordingly.
Introduce XHCI interrupter support: XHCI HCD supports multiple interrupters, which allows for events to be routed to different event rings. This is determined by "Interrupter Target" field specified in Section "6.4.1.1 Normal TRB" of the XHCI specification.
Events in the offloading case will be routed to an event ring that is assigned to the audio DSP.
Create vendor ops for the USB SND driver: qc_audio_offload: This particular driver has several components associated with it: - QMI stream request handler - XHCI interrupter and resource management - audio DSP memory management
When the audio DSP wants to enable a playback stream, the request is first received by the ASoC platform sound card. Depending on the selected route, ASoC will bring up the individual DAIs in the path. The Q6USB backend DAI will send an AFE port start command (with enabling the USB playback path), and the audio DSP will handle the request accordingly.
Part of the AFE USB port start handling will have an exchange of control messages using the QMI protocol. The qc_audio_offload driver will populate the buffer information: - Event ring base address - EP transfer ring base address
and pass it along to the audio DSP. All endpoint management will now be handed over to the DSP, and the main processor is not involved in transfers.
Overall, implementing this feature will still expose separate sound card and PCM devices for both the platorm card and USB audio device: 0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D SM8250-MTP-WCD9380-WSA8810-VA-DMIC 1 [Audio ]: USB-Audio - USB Audio Generic USB Audio at usb-xhci-hcd.1.auto-1.4, high speed
This is to ensure that userspace ALSA entities can decide which route to take when executing the audio playback. In the above, if card#1 is selected, then USB audio data will take the legacy path over the USB PCM drivers, etc...
This feature was validated using: - tinymix: set/enable the multimedia path to route to USB backend - tinyplay: issue playback on platform card
Wesley Cheng (14): ASoC: Add SOC USB APIs for adding an USB backend ASoC: qcom: qdsp6: Introduce USB AFE port to q6dsp ASoC: qcom: Add USB backend ASoC driver for Q6 sound: usb: card: Introduce USB SND vendor op callbacks sound: usb: Export USB SND APIs for modules usb: core: hcd: Introduce USB HCD APIs for interrupter management usb: host: xhci: Add XHCI secondary interrupter support usb: dwc3: Add DT parameter to specify maximum number of interrupters sound: usb: Introduce QC USB SND offloading support sound: usb: card: Check for support for requested audio format sound: soc: soc-usb: Add PCM format check API for USB backend sound: soc: qcom: qusb6: Ensure PCM format is supported by USB audio device ASoC: dt-bindings: Add Q6USB backend bindings ASoC: dt-bindings: Update example for enabling USB offload on SM8250
.../bindings/sound/qcom,q6usb-dais.yaml | 55 + .../bindings/sound/qcom,sm8250.yaml | 13 + drivers/usb/core/hcd.c | 86 + drivers/usb/dwc3/core.c | 12 + drivers/usb/dwc3/core.h | 2 + drivers/usb/dwc3/host.c | 5 +- drivers/usb/host/xhci-mem.c | 219 ++- drivers/usb/host/xhci-plat.c | 2 + drivers/usb/host/xhci.c | 169 +- drivers/usb/host/xhci.h | 15 + .../sound/qcom,q6dsp-lpass-ports.h | 1 + include/linux/usb.h | 7 + include/linux/usb/hcd.h | 16 +- include/sound/q6usboffload.h | 20 + include/sound/soc-usb.h | 34 + sound/soc/Makefile | 2 +- sound/soc/qcom/Kconfig | 4 + sound/soc/qcom/qdsp6/Makefile | 1 + sound/soc/qcom/qdsp6/q6afe-dai.c | 47 + sound/soc/qcom/qdsp6/q6afe.c | 183 ++ sound/soc/qcom/qdsp6/q6afe.h | 46 +- sound/soc/qcom/qdsp6/q6dsp-lpass-ports.c | 23 + sound/soc/qcom/qdsp6/q6dsp-lpass-ports.h | 1 + sound/soc/qcom/qdsp6/q6routing.c | 8 + sound/soc/qcom/qdsp6/q6usb.c | 239 +++ sound/soc/soc-usb.c | 79 + sound/usb/Kconfig | 14 + sound/usb/Makefile | 2 +- sound/usb/card.c | 43 + sound/usb/card.h | 10 + sound/usb/endpoint.c | 2 + sound/usb/helper.c | 1 + sound/usb/pcm.c | 9 +- sound/usb/pcm.h | 12 + sound/usb/qcom/Makefile | 2 + sound/usb/qcom/qc_audio_offload.c | 1610 +++++++++++++++++ sound/usb/qcom/usb_audio_qmi_v01.c | 892 +++++++++ sound/usb/qcom/usb_audio_qmi_v01.h | 162 ++ 38 files changed, 3998 insertions(+), 50 deletions(-) create mode 100644 Documentation/devicetree/bindings/sound/qcom,q6usb-dais.yaml create mode 100644 include/sound/q6usboffload.h create mode 100644 include/sound/soc-usb.h create mode 100644 sound/soc/qcom/qdsp6/q6usb.c create mode 100644 sound/soc/soc-usb.c create mode 100644 sound/usb/qcom/Makefile create mode 100644 sound/usb/qcom/qc_audio_offload.c create mode 100644 sound/usb/qcom/usb_audio_qmi_v01.c create mode 100644 sound/usb/qcom/usb_audio_qmi_v01.h
Some platforms may want to register its USB port to be handled by the ASoC framework. Audio playback/capture support is also handled entirely by the vendor ASoC drivers.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com --- include/sound/soc-usb.h | 31 +++++++++++++++++++ sound/soc/Makefile | 2 +- sound/soc/soc-usb.c | 66 +++++++++++++++++++++++++++++++++++++++++ 3 files changed, 98 insertions(+), 1 deletion(-) create mode 100644 include/sound/soc-usb.h create mode 100644 sound/soc/soc-usb.c
diff --git a/include/sound/soc-usb.h b/include/sound/soc-usb.h new file mode 100644 index 000000000000..7d52e5d2371c --- /dev/null +++ b/include/sound/soc-usb.h @@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: GPL-2.0 + * + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef __LINUX_SND_SOC_USB_H +#define __LINUX_SND_SOC_USB_H + +/** + * struct snd_soc_usb + * @component - Reference to DAPM component + * @connection_status_cb - callback to notify connection events + * @priv_data - vendor data + **/ +struct snd_soc_usb { + struct snd_soc_component *component; + int (*connection_status_cb)(struct snd_soc_usb *usb, int card_idx, + int connected); + void *priv_data; +}; + +int snd_soc_usb_connect(int card_idx); +int snd_soc_usb_disconnect(void); +void snd_soc_usb_set_priv_data(void *priv); +void *snd_soc_usb_get_priv_data(void); + +struct snd_soc_usb *snd_soc_usb_add_port(struct device *dev, + int (*connection_cb)(struct snd_soc_usb *usb, int card_idx, + int connected)); +int snd_soc_usb_remove_port(void); +#endif diff --git a/sound/soc/Makefile b/sound/soc/Makefile index 507eaed1d6a1..3305ceb59d84 100644 --- a/sound/soc/Makefile +++ b/sound/soc/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 -snd-soc-core-objs := soc-core.o soc-dapm.o soc-jack.o soc-utils.o soc-dai.o soc-component.o +snd-soc-core-objs := soc-core.o soc-dapm.o soc-jack.o soc-usb.o soc-utils.o soc-dai.o soc-component.o snd-soc-core-objs += soc-pcm.o soc-devres.o soc-ops.o soc-link.o soc-card.o snd-soc-core-$(CONFIG_SND_SOC_COMPRESS) += soc-compress.o
diff --git a/sound/soc/soc-usb.c b/sound/soc/soc-usb.c new file mode 100644 index 000000000000..c6c376960e4d --- /dev/null +++ b/sound/soc/soc-usb.c @@ -0,0 +1,66 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ +#include <linux/usb.h> +#include <sound/soc.h> +#include <sound/soc-usb.h> +#include "../usb/card.h" + +struct snd_soc_usb *ctx; + +void *snd_soc_usb_get_priv_data(void) +{ + if (ctx) + return ctx->priv_data; + + return 0; +} +EXPORT_SYMBOL_GPL(snd_soc_usb_get_priv_data); + +void snd_soc_usb_set_priv_data(void *priv) +{ + if (ctx) + ctx->priv_data = priv; +} +EXPORT_SYMBOL_GPL(snd_soc_usb_set_priv_data); + +struct snd_soc_usb *snd_soc_usb_add_port(struct device *dev, + int (*connection_cb)(struct snd_soc_usb *usb, int card_idx, + int connected)) +{ + struct snd_soc_usb *usb; + + usb = devm_kzalloc(dev, sizeof(*usb), GFP_KERNEL); + if (!usb) + return ERR_PTR(-ENOMEM); + + usb->connection_status_cb = connection_cb; + ctx = usb; + + return usb; +} +EXPORT_SYMBOL_GPL(snd_soc_usb_add_port); + +int snd_soc_usb_remove_port(void) +{ + ctx = NULL; + return 0; +} +EXPORT_SYMBOL_GPL(snd_soc_usb_remove_port); + +int snd_soc_usb_connect(int card_idx) +{ + if (ctx && ctx->connection_status_cb) + ctx->connection_status_cb(ctx, card_idx, 1); + return 0; +} +EXPORT_SYMBOL_GPL(snd_soc_usb_connect); + +int snd_soc_usb_disconnect(void) +{ + if (ctx && ctx->connection_status_cb) + ctx->connection_status_cb(ctx, -1, 0); + return 0; +} +EXPORT_SYMBOL_GPL(snd_soc_usb_disconnect);
On Fri, Dec 23, 2022 at 03:31:47PM -0800, Wesley Cheng wrote:
diff --git a/sound/soc/soc-usb.c b/sound/soc/soc-usb.c new file mode 100644 index 000000000000..c6c376960e4d --- /dev/null +++ b/sound/soc/soc-usb.c @@ -0,0 +1,66 @@ +// SPDX-License-Identifier: GPL-2.0 +/*
- Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
- */
+#include <linux/usb.h> +#include <sound/soc.h> +#include <sound/soc-usb.h> +#include "../usb/card.h"
+struct snd_soc_usb *ctx;
Note, this will not work. You can not only have "one" state for a system like this. That just broke any system with more than one controller, of which we have millions.
This has to be dynamic for any number of controllers in the system, like the sound and USB core can handle. Any requirement of "there can be only one!" will obviously never be acceptable as that is not how Linux works.
thanks,
greg k-h
The QC ADSP is able to support USB playback and capture, so that the main application processor can be placed into lower CPU power modes. This adds the required AFE port configurations and port start command to start an audio session.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com --- .../sound/qcom,q6dsp-lpass-ports.h | 1 + sound/soc/qcom/qdsp6/q6afe-dai.c | 47 +++++ sound/soc/qcom/qdsp6/q6afe.c | 183 ++++++++++++++++++ sound/soc/qcom/qdsp6/q6afe.h | 46 ++++- sound/soc/qcom/qdsp6/q6dsp-lpass-ports.c | 23 +++ sound/soc/qcom/qdsp6/q6dsp-lpass-ports.h | 1 + sound/soc/qcom/qdsp6/q6routing.c | 8 + 7 files changed, 308 insertions(+), 1 deletion(-)
diff --git a/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h b/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h index 9f7c5103bc82..746bc462bb2e 100644 --- a/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h +++ b/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h @@ -131,6 +131,7 @@ #define RX_CODEC_DMA_RX_7 126 #define QUINARY_MI2S_RX 127 #define QUINARY_MI2S_TX 128 +#define USB_RX 129
#define LPASS_CLK_ID_PRI_MI2S_IBIT 1 #define LPASS_CLK_ID_PRI_MI2S_EBIT 2 diff --git a/sound/soc/qcom/qdsp6/q6afe-dai.c b/sound/soc/qcom/qdsp6/q6afe-dai.c index 8bb7452b8f18..7fdee8cbd4de 100644 --- a/sound/soc/qcom/qdsp6/q6afe-dai.c +++ b/sound/soc/qcom/qdsp6/q6afe-dai.c @@ -111,6 +111,40 @@ static int q6hdmi_hw_params(struct snd_pcm_substream *substream, return 0; }
+static int q6usb_hw_params(struct snd_pcm_substream *substream, + struct snd_pcm_hw_params *params, + struct snd_soc_dai *dai) +{ + struct q6afe_dai_data *dai_data = dev_get_drvdata(dai->dev); + int channels = params_channels(params); + int rate = params_rate(params); + struct q6afe_usb_cfg *usb = &dai_data->port_config[dai->id].usb_audio; + + usb->sample_rate = rate; + usb->num_channels = channels; + + switch (params_format(params)) { + case SNDRV_PCM_FORMAT_U16_LE: + case SNDRV_PCM_FORMAT_S16_LE: + case SNDRV_PCM_FORMAT_SPECIAL: + usb->bit_width = 16; + break; + case SNDRV_PCM_FORMAT_S24_LE: + case SNDRV_PCM_FORMAT_S24_3LE: + usb->bit_width = 24; + break; + case SNDRV_PCM_FORMAT_S32_LE: + usb->bit_width = 32; + break; + default: + dev_err(dai->dev, "%s: invalid format %d\n", + __func__, params_format(params)); + return -EINVAL; + } + + return 0; +} + static int q6i2s_hw_params(struct snd_pcm_substream *substream, struct snd_pcm_hw_params *params, struct snd_soc_dai *dai) @@ -411,6 +445,10 @@ static int q6afe_dai_prepare(struct snd_pcm_substream *substream, q6afe_cdc_dma_port_prepare(dai_data->port[dai->id], &dai_data->port_config[dai->id].dma_cfg); break; + case USB_RX: + q6afe_usb_port_prepare(dai_data->port[dai->id], + &dai_data->port_config[dai->id].usb_audio); + break; default: return -EINVAL; } @@ -495,6 +533,7 @@ static int q6afe_mi2s_set_sysclk(struct snd_soc_dai *dai, }
static const struct snd_soc_dapm_route q6afe_dapm_routes[] = { + {"USB Playback", NULL, "USB_RX"}, {"HDMI Playback", NULL, "HDMI_RX"}, {"Display Port Playback", NULL, "DISPLAY_PORT_RX"}, {"Slimbus Playback", NULL, "SLIMBUS_0_RX"}, @@ -639,6 +678,12 @@ static const struct snd_soc_dapm_route q6afe_dapm_routes[] = { {"RX_CODEC_DMA_RX_7 Playback", NULL, "RX_CODEC_DMA_RX_7"}, };
+static const struct snd_soc_dai_ops q6usb_ops = { + .prepare = q6afe_dai_prepare, + .hw_params = q6usb_hw_params, + .shutdown = q6afe_dai_shutdown, +}; + static const struct snd_soc_dai_ops q6hdmi_ops = { .prepare = q6afe_dai_prepare, .hw_params = q6hdmi_hw_params, @@ -703,6 +748,7 @@ static int msm_dai_q6_dai_remove(struct snd_soc_dai *dai) }
static const struct snd_soc_dapm_widget q6afe_dai_widgets[] = { + SND_SOC_DAPM_AIF_IN("USB_RX", NULL, 0, SND_SOC_NOPM, 0, 0), SND_SOC_DAPM_AIF_IN("HDMI_RX", NULL, 0, SND_SOC_NOPM, 0, 0), SND_SOC_DAPM_AIF_IN("SLIMBUS_0_RX", NULL, 0, SND_SOC_NOPM, 0, 0), SND_SOC_DAPM_AIF_IN("SLIMBUS_1_RX", NULL, 0, SND_SOC_NOPM, 0, 0), @@ -1068,6 +1114,7 @@ static int q6afe_dai_dev_probe(struct platform_device *pdev) cfg.q6i2s_ops = &q6i2s_ops; cfg.q6tdm_ops = &q6tdm_ops; cfg.q6dma_ops = &q6dma_ops; + cfg.q6usb_ops = &q6usb_ops; dais = q6dsp_audio_ports_set_config(dev, &cfg, &num_dais);
return devm_snd_soc_register_component(dev, &q6afe_dai_component, dais, num_dais); diff --git a/sound/soc/qcom/qdsp6/q6afe.c b/sound/soc/qcom/qdsp6/q6afe.c index 919e326b9462..2054f5723e03 100644 --- a/sound/soc/qcom/qdsp6/q6afe.c +++ b/sound/soc/qcom/qdsp6/q6afe.c @@ -34,6 +34,8 @@ #define AFE_MODULE_TDM 0x0001028A
#define AFE_PARAM_ID_CDC_SLIMBUS_SLAVE_CFG 0x00010235 +#define AFE_PARAM_ID_USB_AUDIO_DEV_PARAMS 0x000102A5 +#define AFE_PARAM_ID_USB_AUDIO_DEV_LPCM_FMT 0x000102AA
#define AFE_PARAM_ID_LPAIF_CLK_CONFIG 0x00010238 #define AFE_PARAM_ID_INT_DIGITAL_CDC_CLK_CONFIG 0x00010239 @@ -43,6 +45,7 @@ #define AFE_PARAM_ID_TDM_CONFIG 0x0001029D #define AFE_PARAM_ID_PORT_SLOT_MAPPING_CONFIG 0x00010297 #define AFE_PARAM_ID_CODEC_DMA_CONFIG 0x000102B8 +#define AFE_PARAM_ID_USB_AUDIO_CONFIG 0x000102A4 #define AFE_CMD_REMOTE_LPASS_CORE_HW_VOTE_REQUEST 0x000100f4 #define AFE_CMD_RSP_REMOTE_LPASS_CORE_HW_VOTE_REQUEST 0x000100f5 #define AFE_CMD_REMOTE_LPASS_CORE_HW_DEVOTE_REQUEST 0x000100f6 @@ -71,12 +74,16 @@ #define AFE_PORT_CONFIG_I2S_WS_SRC_INTERNAL 0x1 #define AFE_LINEAR_PCM_DATA 0x0
+#define AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG 0x1
/* Port IDs */ #define AFE_API_VERSION_HDMI_CONFIG 0x1 #define AFE_PORT_ID_MULTICHAN_HDMI_RX 0x100E #define AFE_PORT_ID_HDMI_OVER_DP_RX 0x6020
+/* USB AFE port */ +#define AFE_PORT_ID_USB_RX 0x7000 + #define AFE_API_VERSION_SLIMBUS_CONFIG 0x1 /* Clock set API version */ #define AFE_API_VERSION_CLOCK_SET 1 @@ -512,12 +519,109 @@ struct afe_param_id_cdc_dma_cfg { u16 active_channels_mask; } __packed;
+struct afe_param_id_usb_cfg { +/* Minor version used for tracking USB audio device configuration. + * Supported values: AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG + */ + u32 cfg_minor_version; +/* Sampling rate of the port. + * Supported values: + * - AFE_PORT_SAMPLE_RATE_8K + * - AFE_PORT_SAMPLE_RATE_11025 + * - AFE_PORT_SAMPLE_RATE_12K + * - AFE_PORT_SAMPLE_RATE_16K + * - AFE_PORT_SAMPLE_RATE_22050 + * - AFE_PORT_SAMPLE_RATE_24K + * - AFE_PORT_SAMPLE_RATE_32K + * - AFE_PORT_SAMPLE_RATE_44P1K + * - AFE_PORT_SAMPLE_RATE_48K + * - AFE_PORT_SAMPLE_RATE_96K + * - AFE_PORT_SAMPLE_RATE_192K + */ + u32 sample_rate; +/* Bit width of the sample. + * Supported values: 16, 24 + */ + u16 bit_width; +/* Number of channels. + * Supported values: 1 and 2 + */ + u16 num_channels; +/* Data format supported by the USB. The supported value is + * 0 (#AFE_USB_AUDIO_DATA_FORMAT_LINEAR_PCM). + */ + u16 data_format; +/* this field must be 0 */ + u16 reserved; +/* device token of actual end USB aduio device */ + u32 dev_token; +/* endianness of this interface */ + u32 endian; +/* service interval */ + u32 service_interval; +} __packed; + +/** + * struct afe_param_id_usb_audio_dev_params + * @cfg_minor_version: Minor version used for tracking USB audio device + * configuration. + * Supported values: + * AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG + * @dev_token: device token of actual end USB aduio device + **/ +struct afe_param_id_usb_audio_dev_params { + u32 cfg_minor_version; + u32 dev_token; +} __packed; + +/** + * struct afe_param_id_usb_audio_dev_lpcm_fmt + * @cfg_minor_version: Minor version used for tracking USB audio device + * configuration. + * Supported values: + * AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG + * @endian: endianness of this interface + **/ +struct afe_param_id_usb_audio_dev_lpcm_fmt { + u32 cfg_minor_version; + u32 endian; +} __packed; + +/** + * struct afe_param_id_usb_audio_dev_latency_mode + * @cfg_minor_version: Minor version used for tracking USB audio device + * configuration. + * Supported values: + * AFE_API_MINOR_VERSION_USB_AUDIO_LATENCY_MODE + * @mode: latency mode for the USB audio device + **/ +struct afe_param_id_usb_audio_dev_latency_mode { + u32 minor_version; + u32 mode; +} __packed; + +#define AFE_PARAM_ID_USB_AUDIO_SVC_INTERVAL 0x000102B7 + +/** + * struct afe_param_id_usb_audio_svc_interval + * @cfg_minor_version: Minor version used for tracking USB audio device + * configuration. + * Supported values: + * AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG + * @svc_interval: service interval + **/ +struct afe_param_id_usb_audio_svc_interval { + u32 cfg_minor_version; + u32 svc_interval; +} __packed; + union afe_port_config { struct afe_param_id_hdmi_multi_chan_audio_cfg hdmi_multi_ch; struct afe_param_id_slimbus_cfg slim_cfg; struct afe_param_id_i2s_cfg i2s_cfg; struct afe_param_id_tdm_cfg tdm_cfg; struct afe_param_id_cdc_dma_cfg dma_cfg; + struct afe_param_id_usb_cfg usb_cfg; } __packed;
@@ -577,6 +681,7 @@ struct afe_port_map { */
static struct afe_port_map port_maps[AFE_PORT_MAX] = { + [USB_RX] = { AFE_PORT_ID_USB_RX, USB_RX, 1, 1}, [HDMI_RX] = { AFE_PORT_ID_MULTICHAN_HDMI_RX, HDMI_RX, 1, 1}, [SLIMBUS_0_RX] = { AFE_PORT_ID_SLIMBUS_MULTI_CHAN_0_RX, SLIMBUS_0_RX, 1, 1}, @@ -1289,6 +1394,82 @@ void q6afe_tdm_port_prepare(struct q6afe_port *port, } EXPORT_SYMBOL_GPL(q6afe_tdm_port_prepare);
+int afe_port_send_usb_dev_param(struct q6afe_port *port, struct q6afe_usb_cfg *cfg) +{ + union afe_port_config *pcfg = &port->port_cfg; + struct afe_param_id_usb_audio_dev_params usb_dev; + struct afe_param_id_usb_audio_dev_lpcm_fmt lpcm_fmt; + struct afe_param_id_usb_audio_svc_interval svc_int; + int ret = 0; + + if (!pcfg) { + pr_err("%s: Error, no configuration data\n", __func__); + ret = -EINVAL; + goto exit; + } + + memset(&usb_dev, 0, sizeof(usb_dev)); + memset(&lpcm_fmt, 0, sizeof(lpcm_fmt)); + + usb_dev.cfg_minor_version = AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG; + q6afe_port_set_param_v2(port, &usb_dev, + AFE_PARAM_ID_USB_AUDIO_DEV_PARAMS, + AFE_MODULE_AUDIO_DEV_INTERFACE, sizeof(usb_dev)); + if (ret) { + pr_err("%s: AFE device param cmd failed %d\n", + __func__, ret); + goto exit; + } + + lpcm_fmt.cfg_minor_version = AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG; + lpcm_fmt.endian = pcfg->usb_cfg.endian; + ret = q6afe_port_set_param_v2(port, &lpcm_fmt, + AFE_PARAM_ID_USB_AUDIO_DEV_LPCM_FMT, + AFE_MODULE_AUDIO_DEV_INTERFACE, sizeof(lpcm_fmt)); + if (ret) { + pr_err("%s: AFE device param cmd LPCM_FMT failed %d\n", + __func__, ret); + goto exit; + } + + svc_int.cfg_minor_version = + AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG; + svc_int.svc_interval = pcfg->usb_cfg.service_interval; + ret = q6afe_port_set_param_v2(port, &svc_int, + AFE_PARAM_ID_USB_AUDIO_SVC_INTERVAL, + AFE_MODULE_AUDIO_DEV_INTERFACE, sizeof(svc_int)); + if (ret) { + pr_err("%s: AFE device param cmd svc_interval failed %d\n", + __func__, ret); + ret = -EINVAL; + goto exit; + } +exit: + return ret; +} + +/** + * q6afe_usb_port_prepare() - Prepare usb afe port. + * + * @port: Instance of afe port + * @cfg: USB configuration for the afe port + * + */ +void q6afe_usb_port_prepare(struct q6afe_port *port, + struct q6afe_usb_cfg *cfg) +{ + union afe_port_config *pcfg = &port->port_cfg; + + pcfg->usb_cfg.cfg_minor_version = + AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG; + pcfg->usb_cfg.sample_rate = cfg->sample_rate; + pcfg->usb_cfg.num_channels = cfg->num_channels; + pcfg->usb_cfg.bit_width = cfg->bit_width; + + afe_port_send_usb_dev_param(port, cfg); +} +EXPORT_SYMBOL_GPL(q6afe_usb_port_prepare); + /** * q6afe_hdmi_port_prepare() - Prepare hdmi afe port. * @@ -1611,6 +1792,8 @@ struct q6afe_port *q6afe_port_get_from_id(struct device *dev, int id) break; case AFE_PORT_ID_WSA_CODEC_DMA_RX_0 ... AFE_PORT_ID_RX_CODEC_DMA_RX_7: cfg_type = AFE_PARAM_ID_CODEC_DMA_CONFIG; + case AFE_PORT_ID_USB_RX: + cfg_type = AFE_PARAM_ID_USB_AUDIO_CONFIG; break; default: dev_err(dev, "Invalid port id 0x%x\n", port_id); diff --git a/sound/soc/qcom/qdsp6/q6afe.h b/sound/soc/qcom/qdsp6/q6afe.h index 30fd77e2f458..88550a08e57d 100644 --- a/sound/soc/qcom/qdsp6/q6afe.h +++ b/sound/soc/qcom/qdsp6/q6afe.h @@ -5,7 +5,7 @@
#include <dt-bindings/sound/qcom,q6afe.h>
-#define AFE_PORT_MAX 129 +#define AFE_PORT_MAX 130
#define MSM_AFE_PORT_TYPE_RX 0 #define MSM_AFE_PORT_TYPE_TX 1 @@ -205,6 +205,47 @@ struct q6afe_cdc_dma_cfg { u16 active_channels_mask; };
+/** + * struct q6afe_usb_cfg + * @cfg_minor_version: Minor version used for tracking USB audio device + * configuration. + * Supported values: + * AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG + * @sample_rate: Sampling rate of the port + * Supported values: + * AFE_PORT_SAMPLE_RATE_8K + * AFE_PORT_SAMPLE_RATE_11025 + * AFE_PORT_SAMPLE_RATE_12K + * AFE_PORT_SAMPLE_RATE_16K + * AFE_PORT_SAMPLE_RATE_22050 + * AFE_PORT_SAMPLE_RATE_24K + * AFE_PORT_SAMPLE_RATE_32K + * AFE_PORT_SAMPLE_RATE_44P1K + * AFE_PORT_SAMPLE_RATE_48K + * AFE_PORT_SAMPLE_RATE_96K + * AFE_PORT_SAMPLE_RATE_192K + * @bit_width: Bit width of the sample. + * Supported values: 16, 24 + * @num_channels: Number of channels + * Supported values: 1, 2 + * @data_format: Data format supported by the USB + * Supported values: 0 + * @reserved: this field must be 0 + * @dev_token: device token of actual end USB aduio device + * @endian: endianness of this interface + * @service_interval: service interval + **/ +struct q6afe_usb_cfg { + u32 cfg_minor_version; + u32 sample_rate; + u16 bit_width; + u16 num_channels; + u16 data_format; + u16 reserved; + u32 dev_token; + u32 endian; + u32 service_interval; +};
struct q6afe_port_config { struct q6afe_hdmi_cfg hdmi; @@ -212,6 +253,7 @@ struct q6afe_port_config { struct q6afe_i2s_cfg i2s_cfg; struct q6afe_tdm_cfg tdm; struct q6afe_cdc_dma_cfg dma_cfg; + struct q6afe_usb_cfg usb_audio; };
struct q6afe_port; @@ -221,6 +263,8 @@ int q6afe_port_start(struct q6afe_port *port); int q6afe_port_stop(struct q6afe_port *port); void q6afe_port_put(struct q6afe_port *port); int q6afe_get_port_id(int index); +void q6afe_usb_port_prepare(struct q6afe_port *port, + struct q6afe_usb_cfg *cfg); void q6afe_hdmi_port_prepare(struct q6afe_port *port, struct q6afe_hdmi_cfg *cfg); void q6afe_slim_port_prepare(struct q6afe_port *port, diff --git a/sound/soc/qcom/qdsp6/q6dsp-lpass-ports.c b/sound/soc/qcom/qdsp6/q6dsp-lpass-ports.c index f67c16fd90b9..39719c3f1767 100644 --- a/sound/soc/qcom/qdsp6/q6dsp-lpass-ports.c +++ b/sound/soc/qcom/qdsp6/q6dsp-lpass-ports.c @@ -81,6 +81,26 @@
static struct snd_soc_dai_driver q6dsp_audio_fe_dais[] = { + { + .playback = { + .stream_name = "USB Playback", + .rates = SNDRV_PCM_RATE_8000 | SNDRV_PCM_RATE_11025 | + SNDRV_PCM_RATE_16000 | SNDRV_PCM_RATE_22050 | + SNDRV_PCM_RATE_32000 | SNDRV_PCM_RATE_44100 | + SNDRV_PCM_RATE_48000 | SNDRV_PCM_RATE_96000 | + SNDRV_PCM_RATE_192000, + .formats = SNDRV_PCM_FMTBIT_S16_LE | SNDRV_PCM_FMTBIT_S16_BE | + SNDRV_PCM_FMTBIT_U16_LE | SNDRV_PCM_FMTBIT_U16_BE | + SNDRV_PCM_FMTBIT_S24_LE | SNDRV_PCM_FMTBIT_S24_BE | + SNDRV_PCM_FMTBIT_U24_LE | SNDRV_PCM_FMTBIT_U24_BE, + .channels_min = 1, + .channels_max = 2, + .rate_min = 8000, + .rate_max = 192000, + }, + .id = USB_RX, + .name = "USB_RX", + }, { .playback = { .stream_name = "HDMI Playback", @@ -616,6 +636,9 @@ struct snd_soc_dai_driver *q6dsp_audio_ports_set_config(struct device *dev, case WSA_CODEC_DMA_RX_0 ... RX_CODEC_DMA_RX_7: q6dsp_audio_fe_dais[i].ops = cfg->q6dma_ops; break; + case USB_RX: + q6dsp_audio_fe_dais[i].ops = cfg->q6usb_ops; + break; default: break; } diff --git a/sound/soc/qcom/qdsp6/q6dsp-lpass-ports.h b/sound/soc/qcom/qdsp6/q6dsp-lpass-ports.h index 7f052c8a1257..d8dde6dd0aca 100644 --- a/sound/soc/qcom/qdsp6/q6dsp-lpass-ports.h +++ b/sound/soc/qcom/qdsp6/q6dsp-lpass-ports.h @@ -11,6 +11,7 @@ struct q6dsp_audio_port_dai_driver_config { const struct snd_soc_dai_ops *q6i2s_ops; const struct snd_soc_dai_ops *q6tdm_ops; const struct snd_soc_dai_ops *q6dma_ops; + const struct snd_soc_dai_ops *q6usb_ops; };
struct snd_soc_dai_driver *q6dsp_audio_ports_set_config(struct device *dev, diff --git a/sound/soc/qcom/qdsp6/q6routing.c b/sound/soc/qcom/qdsp6/q6routing.c index 928fd23e2c27..683ae2ae8e50 100644 --- a/sound/soc/qcom/qdsp6/q6routing.c +++ b/sound/soc/qcom/qdsp6/q6routing.c @@ -514,6 +514,9 @@ static int msm_routing_put_audio_mixer(struct snd_kcontrol *kcontrol, return 1; }
+static const struct snd_kcontrol_new usb_mixer_controls[] = { + Q6ROUTING_RX_MIXERS(USB_RX) }; + static const struct snd_kcontrol_new hdmi_mixer_controls[] = { Q6ROUTING_RX_MIXERS(HDMI_RX) };
@@ -733,6 +736,10 @@ static const struct snd_kcontrol_new mmul8_mixer_controls[] = {
static const struct snd_soc_dapm_widget msm_qdsp6_widgets[] = { /* Mixer definitions */ + SND_SOC_DAPM_MIXER("USB Mixer", SND_SOC_NOPM, 0, 0, + usb_mixer_controls, + ARRAY_SIZE(usb_mixer_controls)), + SND_SOC_DAPM_MIXER("HDMI Mixer", SND_SOC_NOPM, 0, 0, hdmi_mixer_controls, ARRAY_SIZE(hdmi_mixer_controls)), @@ -952,6 +959,7 @@ static const struct snd_soc_dapm_widget msm_qdsp6_widgets[] = { };
static const struct snd_soc_dapm_route intercon[] = { + Q6ROUTING_RX_DAPM_ROUTE("USB Mixer", "USB_RX"), Q6ROUTING_RX_DAPM_ROUTE("HDMI Mixer", "HDMI_RX"), Q6ROUTING_RX_DAPM_ROUTE("DISPLAY_PORT_RX Audio Mixer", "DISPLAY_PORT_RX"),
On 12/23/22 17:31, Wesley Cheng wrote:
The QC ADSP is able to support USB playback and capture, so that the main application processor can be placed into lower CPU power modes. This adds the required AFE port configurations and port start command to start an audio session.
It would be good to clarify what sort of endpoints can be supported. I presume the SOF-synchronous case is handled, but how would you deal with async endpoints with feedback (be it explicit or implicit)?
Note that it's very hard to make the decision not to support async endpoints, there are quite a few devices that are exposed as async to work around an obscure legacy issue on Windows but are really sof-synchronous endpoints that never ask for any change of pace.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com
.../sound/qcom,q6dsp-lpass-ports.h | 1 + sound/soc/qcom/qdsp6/q6afe-dai.c | 47 +++++ sound/soc/qcom/qdsp6/q6afe.c | 183 ++++++++++++++++++ sound/soc/qcom/qdsp6/q6afe.h | 46 ++++- sound/soc/qcom/qdsp6/q6dsp-lpass-ports.c | 23 +++ sound/soc/qcom/qdsp6/q6dsp-lpass-ports.h | 1 + sound/soc/qcom/qdsp6/q6routing.c | 8 + 7 files changed, 308 insertions(+), 1 deletion(-)
diff --git a/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h b/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h index 9f7c5103bc82..746bc462bb2e 100644 --- a/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h +++ b/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h @@ -131,6 +131,7 @@ #define RX_CODEC_DMA_RX_7 126 #define QUINARY_MI2S_RX 127 #define QUINARY_MI2S_TX 128 +#define USB_RX 129
the commit message says the DSP can support Playback and capture, but here there's capture only ...
static const struct snd_soc_dapm_route q6afe_dapm_routes[] = {
- {"USB Playback", NULL, "USB_RX"},
... but here RX means playback?
I am not sure I get the convention on directions and what is actually supported?
+struct afe_param_id_usb_cfg { +/* Minor version used for tracking USB audio device configuration.
- Supported values: AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG
- */
- u32 cfg_minor_version;
+/* Sampling rate of the port.
- Supported values:
- AFE_PORT_SAMPLE_RATE_8K
- AFE_PORT_SAMPLE_RATE_11025
- AFE_PORT_SAMPLE_RATE_12K
- AFE_PORT_SAMPLE_RATE_16K
- AFE_PORT_SAMPLE_RATE_22050
- AFE_PORT_SAMPLE_RATE_24K
- AFE_PORT_SAMPLE_RATE_32K
- AFE_PORT_SAMPLE_RATE_44P1K
- AFE_PORT_SAMPLE_RATE_48K
- AFE_PORT_SAMPLE_RATE_96K
- AFE_PORT_SAMPLE_RATE_192K
- */
- u32 sample_rate;
+/* Bit width of the sample.
- Supported values: 16, 24
- */
- u16 bit_width;
+/* Number of channels.
- Supported values: 1 and 2
that aligns with my feedback on the cover letter, if you connect a device that can support from than 2 channels should the DSP even expose this DSP-optimized path?
Oh and I forgot, what happens if there are multiple audio devices connected, can the DSP deal with all of them? If not, how is this handled?
- */
- u16 num_channels;
+/* Data format supported by the USB. The supported value is
- 0 (#AFE_USB_AUDIO_DATA_FORMAT_LINEAR_PCM).
- */
- u16 data_format;
+/* this field must be 0 */
- u16 reserved;
+/* device token of actual end USB aduio device */
typo: audio
- u32 dev_token;
+/* endianness of this interface */
- u32 endian;
Is this a USB concept? I can't recall having seen any parts of the USB audio class spec that the data can be big or little endian?
+/* service interval */
- u32 service_interval;
+} __packed;
+int afe_port_send_usb_dev_param(struct q6afe_port *port, struct q6afe_usb_cfg *cfg) +{
- union afe_port_config *pcfg = &port->port_cfg;
- struct afe_param_id_usb_audio_dev_params usb_dev;
- struct afe_param_id_usb_audio_dev_lpcm_fmt lpcm_fmt;
- struct afe_param_id_usb_audio_svc_interval svc_int;
- int ret = 0;
- if (!pcfg) {
pr_err("%s: Error, no configuration data\n", __func__);
can you use a dev_err() here and the rest of the code?
ret = -EINVAL;
goto exit;
- }
- memset(&usb_dev, 0, sizeof(usb_dev));
- memset(&lpcm_fmt, 0, sizeof(lpcm_fmt));
- usb_dev.cfg_minor_version = AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG;
- q6afe_port_set_param_v2(port, &usb_dev,
AFE_PARAM_ID_USB_AUDIO_DEV_PARAMS,
AFE_MODULE_AUDIO_DEV_INTERFACE, sizeof(usb_dev));
- if (ret) {
pr_err("%s: AFE device param cmd failed %d\n",
__func__, ret);
goto exit;
- }
- lpcm_fmt.cfg_minor_version = AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG;
- lpcm_fmt.endian = pcfg->usb_cfg.endian;
- ret = q6afe_port_set_param_v2(port, &lpcm_fmt,
AFE_PARAM_ID_USB_AUDIO_DEV_LPCM_FMT,
AFE_MODULE_AUDIO_DEV_INTERFACE, sizeof(lpcm_fmt));
- if (ret) {
pr_err("%s: AFE device param cmd LPCM_FMT failed %d\n",
__func__, ret);
goto exit;
- }
- svc_int.cfg_minor_version =
AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG;
- svc_int.svc_interval = pcfg->usb_cfg.service_interval;
- ret = q6afe_port_set_param_v2(port, &svc_int,
AFE_PARAM_ID_USB_AUDIO_SVC_INTERVAL,
AFE_MODULE_AUDIO_DEV_INTERFACE, sizeof(svc_int));
- if (ret) {
pr_err("%s: AFE device param cmd svc_interval failed %d\n",
__func__, ret);
ret = -EINVAL;
goto exit;
- }
+exit:
- return ret;
+}
Hi Pierre,
On 1/4/2023 3:33 PM, Pierre-Louis Bossart wrote:
On 12/23/22 17:31, Wesley Cheng wrote:
The QC ADSP is able to support USB playback and capture, so that the main application processor can be placed into lower CPU power modes. This adds the required AFE port configurations and port start command to start an audio session.
It would be good to clarify what sort of endpoints can be supported. I presume the SOF-synchronous case is handled, but how would you deal with async endpoints with feedback (be it explicit or implicit)?
Sure, both types of feedback endpoints are expected to be supported by the audio DSP, as well as sync eps. We have the logic there to modify the audio sample size accordingly.
Note that it's very hard to make the decision not to support async endpoints, there are quite a few devices that are exposed as async to work around an obscure legacy issue on Windows but are really sof-synchronous endpoints that never ask for any change of pace.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com
.../sound/qcom,q6dsp-lpass-ports.h | 1 + sound/soc/qcom/qdsp6/q6afe-dai.c | 47 +++++ sound/soc/qcom/qdsp6/q6afe.c | 183 ++++++++++++++++++ sound/soc/qcom/qdsp6/q6afe.h | 46 ++++- sound/soc/qcom/qdsp6/q6dsp-lpass-ports.c | 23 +++ sound/soc/qcom/qdsp6/q6dsp-lpass-ports.h | 1 + sound/soc/qcom/qdsp6/q6routing.c | 8 + 7 files changed, 308 insertions(+), 1 deletion(-)
diff --git a/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h b/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h index 9f7c5103bc82..746bc462bb2e 100644 --- a/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h +++ b/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h @@ -131,6 +131,7 @@ #define RX_CODEC_DMA_RX_7 126 #define QUINARY_MI2S_RX 127 #define QUINARY_MI2S_TX 128 +#define USB_RX 129
the commit message says the DSP can support Playback and capture, but here there's capture only ...
Sorry I will update the commit message properly. At the moment we've only verified audio playback.
static const struct snd_soc_dapm_route q6afe_dapm_routes[] = {
- {"USB Playback", NULL, "USB_RX"},
... but here RX means playback?
I am not sure I get the convention on directions and what is actually supported?
The notation is based on the direction of which the audio data is sourced or pushed on to the DSP. So in playback, the DSP is receiving audio data to send, and capture, it is transmitting audio data received. (although we do not support capture yet)
+struct afe_param_id_usb_cfg { +/* Minor version used for tracking USB audio device configuration.
- Supported values: AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG
- */
- u32 cfg_minor_version;
+/* Sampling rate of the port.
- Supported values:
- AFE_PORT_SAMPLE_RATE_8K
- AFE_PORT_SAMPLE_RATE_11025
- AFE_PORT_SAMPLE_RATE_12K
- AFE_PORT_SAMPLE_RATE_16K
- AFE_PORT_SAMPLE_RATE_22050
- AFE_PORT_SAMPLE_RATE_24K
- AFE_PORT_SAMPLE_RATE_32K
- AFE_PORT_SAMPLE_RATE_44P1K
- AFE_PORT_SAMPLE_RATE_48K
- AFE_PORT_SAMPLE_RATE_96K
- AFE_PORT_SAMPLE_RATE_192K
- */
- u32 sample_rate;
+/* Bit width of the sample.
- Supported values: 16, 24
- */
- u16 bit_width;
+/* Number of channels.
- Supported values: 1 and 2
that aligns with my feedback on the cover letter, if you connect a device that can support from than 2 channels should the DSP even expose this DSP-optimized path?
My assumption is that I programmed the DAIs w/ PCM formats supported by the DSP, so I think the ASoC core should not allow userspace to choose that path if the hw params don't fit/match.
Oh and I forgot, what happens if there are multiple audio devices connected, can the DSP deal with all of them? If not, how is this handled?
This is one topic that we were pretty open ended on. At least on our implementation, only one audio device can be supported at a time. We choose the latest device that was plugged in or discovered by the USB SND class driver.
- */
- u16 num_channels;
+/* Data format supported by the USB. The supported value is
- 0 (#AFE_USB_AUDIO_DATA_FORMAT_LINEAR_PCM).
- */
- u16 data_format;
+/* this field must be 0 */
- u16 reserved;
+/* device token of actual end USB aduio device */
typo: audio
Thanks
- u32 dev_token;
+/* endianness of this interface */
- u32 endian;
Is this a USB concept? I can't recall having seen any parts of the USB audio class spec that the data can be big or little endian?
No, this is probably just something our audio DSP uses on the AFE commands that it receives.
+/* service interval */
- u32 service_interval;
+} __packed;
+int afe_port_send_usb_dev_param(struct q6afe_port *port, struct q6afe_usb_cfg *cfg) +{
- union afe_port_config *pcfg = &port->port_cfg;
- struct afe_param_id_usb_audio_dev_params usb_dev;
- struct afe_param_id_usb_audio_dev_lpcm_fmt lpcm_fmt;
- struct afe_param_id_usb_audio_svc_interval svc_int;
- int ret = 0;
- if (!pcfg) {
pr_err("%s: Error, no configuration data\n", __func__);
can you use a dev_err() here and the rest of the code?
Sure.
Thanks Wesley Cheng
The QC ADSP is able to support USB playback and capture, so that the main application processor can be placed into lower CPU power modes. This adds the required AFE port configurations and port start command to start an audio session.
It would be good to clarify what sort of endpoints can be supported. I presume the SOF-synchronous case is handled, but how would you deal with async endpoints with feedback (be it explicit or implicit)?
Sure, both types of feedback endpoints are expected to be supported by the audio DSP, as well as sync eps. We have the logic there to modify the audio sample size accordingly.
did you mean modify samples per USB frame (or uframe), so as to change the pace at which data is transferred? If yes it'd be the same for Intel.
static const struct snd_soc_dapm_route q6afe_dapm_routes[] = { + {"USB Playback", NULL, "USB_RX"},
... but here RX means playback?
I am not sure I get the convention on directions and what is actually supported?
The notation is based on the direction of which the audio data is sourced or pushed on to the DSP. So in playback, the DSP is receiving audio data to send, and capture, it is transmitting audio data received. (although we do not support capture yet)
ok, it'd be good to add a comment on this convention. Usually RX/TX is bus-centric.
+struct afe_param_id_usb_cfg { +/* Minor version used for tracking USB audio device configuration.
- Supported values: AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG
- */
+ u32 cfg_minor_version; +/* Sampling rate of the port.
- Supported values:
- AFE_PORT_SAMPLE_RATE_8K
- AFE_PORT_SAMPLE_RATE_11025
- AFE_PORT_SAMPLE_RATE_12K
- AFE_PORT_SAMPLE_RATE_16K
- AFE_PORT_SAMPLE_RATE_22050
- AFE_PORT_SAMPLE_RATE_24K
- AFE_PORT_SAMPLE_RATE_32K
- AFE_PORT_SAMPLE_RATE_44P1K
- AFE_PORT_SAMPLE_RATE_48K
- AFE_PORT_SAMPLE_RATE_96K
- AFE_PORT_SAMPLE_RATE_192K
- */
+ u32 sample_rate; +/* Bit width of the sample.
- Supported values: 16, 24
- */
+ u16 bit_width; +/* Number of channels.
- Supported values: 1 and 2
that aligns with my feedback on the cover letter, if you connect a device that can support from than 2 channels should the DSP even expose this DSP-optimized path?
My assumption is that I programmed the DAIs w/ PCM formats supported by the DSP, so I think the ASoC core should not allow userspace to choose that path if the hw params don't fit/match.
Right, but the point I was trying to make is that if the device can do more, why create this DSP path at all?
Oh and I forgot, what happens if there are multiple audio devices connected, can the DSP deal with all of them? If not, how is this handled?
This is one topic that we were pretty open ended on. At least on our implementation, only one audio device can be supported at a time. We choose the latest device that was plugged in or discovered by the USB SND class driver.
Similar case for Intel. I have to revisit this, I don't recall the details.
+ u32 dev_token; +/* endianness of this interface */ + u32 endian;
Is this a USB concept? I can't recall having seen any parts of the USB audio class spec that the data can be big or little endian?
No, this is probably just something our audio DSP uses on the AFE commands that it receives.
ok.
Hi Pierre,
On 1/6/2023 8:09 AM, Pierre-Louis Bossart wrote:
The QC ADSP is able to support USB playback and capture, so that the main application processor can be placed into lower CPU power modes. This adds the required AFE port configurations and port start command to start an audio session.
It would be good to clarify what sort of endpoints can be supported. I presume the SOF-synchronous case is handled, but how would you deal with async endpoints with feedback (be it explicit or implicit)?
Sure, both types of feedback endpoints are expected to be supported by the audio DSP, as well as sync eps. We have the logic there to modify the audio sample size accordingly.
did you mean modify samples per USB frame (or uframe), so as to change the pace at which data is transferred? If yes it'd be the same for Intel.
Yes, sorry for not being clear. Your understanding is correct.
static const struct snd_soc_dapm_route q6afe_dapm_routes[] = { + {"USB Playback", NULL, "USB_RX"},
... but here RX means playback?
I am not sure I get the convention on directions and what is actually supported?
The notation is based on the direction of which the audio data is sourced or pushed on to the DSP. So in playback, the DSP is receiving audio data to send, and capture, it is transmitting audio data received. (although we do not support capture yet)
ok, it'd be good to add a comment on this convention. Usually RX/TX is bus-centric.
Sure, will do.
+struct afe_param_id_usb_cfg { +/* Minor version used for tracking USB audio device configuration.
- Supported values: AFE_API_MINOR_VERSION_USB_AUDIO_CONFIG
- */
+ u32 cfg_minor_version; +/* Sampling rate of the port.
- Supported values:
- AFE_PORT_SAMPLE_RATE_8K
- AFE_PORT_SAMPLE_RATE_11025
- AFE_PORT_SAMPLE_RATE_12K
- AFE_PORT_SAMPLE_RATE_16K
- AFE_PORT_SAMPLE_RATE_22050
- AFE_PORT_SAMPLE_RATE_24K
- AFE_PORT_SAMPLE_RATE_32K
- AFE_PORT_SAMPLE_RATE_44P1K
- AFE_PORT_SAMPLE_RATE_48K
- AFE_PORT_SAMPLE_RATE_96K
- AFE_PORT_SAMPLE_RATE_192K
- */
+ u32 sample_rate; +/* Bit width of the sample.
- Supported values: 16, 24
- */
+ u16 bit_width; +/* Number of channels.
- Supported values: 1 and 2
that aligns with my feedback on the cover letter, if you connect a device that can support from than 2 channels should the DSP even expose this DSP-optimized path?
My assumption is that I programmed the DAIs w/ PCM formats supported by the DSP, so I think the ASoC core should not allow userspace to choose that path if the hw params don't fit/match.
Right, but the point I was trying to make is that if the device can do more, why create this DSP path at all?
Yeah, I think this brings me back to needing to understand a bit more of how the userspace chooses which PCM device to use. At least for our current use cases, userspace would always route through the offload path, regardless of if the device can do more. It will just select a lower audio profile if so.
Oh and I forgot, what happens if there are multiple audio devices connected, can the DSP deal with all of them? If not, how is this handled?
This is one topic that we were pretty open ended on. At least on our implementation, only one audio device can be supported at a time. We choose the latest device that was plugged in or discovered by the USB SND class driver.
Similar case for Intel. I have to revisit this, I don't recall the details.
Got it.
Thanks Wesley Cheng
On 24/12/2022 00:31, Wesley Cheng wrote:
The QC ADSP is able to support USB playback and capture, so that the main application processor can be placed into lower CPU power modes. This adds the required AFE port configurations and port start command to start an audio session.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com
.../sound/qcom,q6dsp-lpass-ports.h | 1 + sound/soc/qcom/qdsp6/q6afe-dai.c | 47 +++++ sound/soc/qcom/qdsp6/q6afe.c | 183 ++++++++++++++++++ sound/soc/qcom/qdsp6/q6afe.h | 46 ++++- sound/soc/qcom/qdsp6/q6dsp-lpass-ports.c | 23 +++ sound/soc/qcom/qdsp6/q6dsp-lpass-ports.h | 1 + sound/soc/qcom/qdsp6/q6routing.c | 8 + 7 files changed, 308 insertions(+), 1 deletion(-)
diff --git a/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h b/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h index 9f7c5103bc82..746bc462bb2e 100644 --- a/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h +++ b/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h @@ -131,6 +131,7 @@ #define RX_CODEC_DMA_RX_7 126 #define QUINARY_MI2S_RX 127 #define QUINARY_MI2S_TX 128 +#define USB_RX 129
#define LPASS_CLK_ID_PRI_MI2S_IBIT 1
Bindings are separate patches. Please split.
Best regards, Krzysztof
Hi Krzysztof,
On 1/5/2023 10:09 AM, Krzysztof Kozlowski wrote:
On 24/12/2022 00:31, Wesley Cheng wrote:
The QC ADSP is able to support USB playback and capture, so that the main application processor can be placed into lower CPU power modes. This adds the required AFE port configurations and port start command to start an audio session.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com
.../sound/qcom,q6dsp-lpass-ports.h | 1 + sound/soc/qcom/qdsp6/q6afe-dai.c | 47 +++++ sound/soc/qcom/qdsp6/q6afe.c | 183 ++++++++++++++++++ sound/soc/qcom/qdsp6/q6afe.h | 46 ++++- sound/soc/qcom/qdsp6/q6dsp-lpass-ports.c | 23 +++ sound/soc/qcom/qdsp6/q6dsp-lpass-ports.h | 1 + sound/soc/qcom/qdsp6/q6routing.c | 8 + 7 files changed, 308 insertions(+), 1 deletion(-)
diff --git a/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h b/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h index 9f7c5103bc82..746bc462bb2e 100644 --- a/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h +++ b/include/dt-bindings/sound/qcom,q6dsp-lpass-ports.h @@ -131,6 +131,7 @@ #define RX_CODEC_DMA_RX_7 126 #define QUINARY_MI2S_RX 127 #define QUINARY_MI2S_TX 128 +#define USB_RX 129
#define LPASS_CLK_ID_PRI_MI2S_IBIT 1
Bindings are separate patches. Please split.
Thanks for catching this. Will do.
Thanks Wesley Cheng
Create a USB BE component that will register a new USB port to the ASoC USB framework. This will handle determination on if the requested audio profile is supported by the USB device currently selected.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com --- include/sound/q6usboffload.h | 20 +++ sound/soc/qcom/Kconfig | 4 + sound/soc/qcom/qdsp6/Makefile | 1 + sound/soc/qcom/qdsp6/q6usb.c | 232 ++++++++++++++++++++++++++++++++++ 4 files changed, 257 insertions(+) create mode 100644 include/sound/q6usboffload.h create mode 100644 sound/soc/qcom/qdsp6/q6usb.c
diff --git a/include/sound/q6usboffload.h b/include/sound/q6usboffload.h new file mode 100644 index 000000000000..e576808901d9 --- /dev/null +++ b/include/sound/q6usboffload.h @@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: GPL-2.0 + * + * linux/sound/q6usboffload.h -- QDSP6 USB offload + * + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +/** + * struct q6usb_offload + * @dev - dev handle to usb be + * @sid - streamID for iommu + * @intr_num - usb interrupter number + * @domain - allocated iommu domain + **/ +struct q6usb_offload { + struct device *dev; + u32 sid; + u32 intr_num; + struct iommu_domain *domain; +}; diff --git a/sound/soc/qcom/Kconfig b/sound/soc/qcom/Kconfig index 8c7398bc1ca8..d65c365116e5 100644 --- a/sound/soc/qcom/Kconfig +++ b/sound/soc/qcom/Kconfig @@ -111,6 +111,9 @@ config SND_SOC_QDSP6_APM config SND_SOC_QDSP6_PRM_LPASS_CLOCKS tristate
+config SND_SOC_QDSP6_USB + tristate + config SND_SOC_QDSP6_PRM tristate select SND_SOC_QDSP6_PRM_LPASS_CLOCKS @@ -131,6 +134,7 @@ config SND_SOC_QDSP6 select SND_SOC_TOPOLOGY select SND_SOC_QDSP6_APM select SND_SOC_QDSP6_PRM + select SND_SOC_QDSP6_USB help To add support for MSM QDSP6 Soc Audio. This will enable sound soc platform specific diff --git a/sound/soc/qcom/qdsp6/Makefile b/sound/soc/qcom/qdsp6/Makefile index 3963bf234664..c9457ee898d0 100644 --- a/sound/soc/qcom/qdsp6/Makefile +++ b/sound/soc/qcom/qdsp6/Makefile @@ -17,3 +17,4 @@ obj-$(CONFIG_SND_SOC_QDSP6_APM_DAI) += q6apm-dai.o obj-$(CONFIG_SND_SOC_QDSP6_APM_LPASS_DAI) += q6apm-lpass-dais.o obj-$(CONFIG_SND_SOC_QDSP6_PRM) += q6prm.o obj-$(CONFIG_SND_SOC_QDSP6_PRM_LPASS_CLOCKS) += q6prm-clocks.o +obj-$(CONFIG_SND_SOC_QDSP6_USB) += q6usb.o diff --git a/sound/soc/qcom/qdsp6/q6usb.c b/sound/soc/qcom/qdsp6/q6usb.c new file mode 100644 index 000000000000..a9da6dec6c6f --- /dev/null +++ b/sound/soc/qcom/qdsp6/q6usb.c @@ -0,0 +1,232 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include <linux/err.h> +#include <linux/init.h> +#include <linux/module.h> +#include <linux/device.h> +#include <linux/platform_device.h> +#include <linux/slab.h> +#include <linux/iommu.h> +#include <linux/dma-mapping.h> +#include <linux/dma-map-ops.h> + +#include <sound/pcm.h> +#include <sound/soc.h> +#include <sound/soc-usb.h> +#include <sound/pcm_params.h> +#include <sound/asound.h> +#include <sound/q6usboffload.h> + +#include "q6dsp-lpass-ports.h" +#include "q6afe.h" + +struct q6usb_port_data { + struct q6afe_usb_cfg usb_cfg; + struct snd_soc_usb *usb; + struct q6usb_offload priv; + int active_idx; +}; + +static const struct snd_soc_dapm_widget q6usb_dai_widgets[] = { + SND_SOC_DAPM_HP("USB_RX_BE", NULL), +}; + +static const struct snd_soc_dapm_route q6usb_dapm_routes[] = { + {"USB Playback", NULL, "USB_RX_BE"}, +}; + +static int q6usb_hw_params(struct snd_pcm_substream *substream, + struct snd_pcm_hw_params *params, + struct snd_soc_dai *dai) +{ + return 0; +} +static const struct snd_soc_dai_ops q6usb_ops = { + .hw_params = q6usb_hw_params, +}; + +static struct snd_soc_dai_driver q6usb_be_dais[] = { + { + .playback = { + .stream_name = "USB BE RX", + .rates = SNDRV_PCM_RATE_8000 | SNDRV_PCM_RATE_11025 | + SNDRV_PCM_RATE_16000 | SNDRV_PCM_RATE_22050 | + SNDRV_PCM_RATE_32000 | SNDRV_PCM_RATE_44100 | + SNDRV_PCM_RATE_48000 | SNDRV_PCM_RATE_96000 | + SNDRV_PCM_RATE_192000, + .formats = SNDRV_PCM_FMTBIT_S16_LE | SNDRV_PCM_FMTBIT_S16_BE | + SNDRV_PCM_FMTBIT_U16_LE | SNDRV_PCM_FMTBIT_U16_BE | + SNDRV_PCM_FMTBIT_S24_LE | SNDRV_PCM_FMTBIT_S24_BE | + SNDRV_PCM_FMTBIT_U24_LE | SNDRV_PCM_FMTBIT_U24_BE, + .channels_min = 1, + .channels_max = 2, + .rate_max = 192000, + .rate_min = 8000, + }, + .id = USB_RX, + .name = "USB_RX_BE", + .ops = &q6usb_ops, + }, +}; + +int q6usb_audio_ports_of_xlate_dai_name(struct snd_soc_component *component, + const struct of_phandle_args *args, + const char **dai_name) +{ + int id = args->args[0]; + int ret = -EINVAL; + int i; + + for (i = 0; i < ARRAY_SIZE(q6usb_be_dais); i++) { + if (q6usb_be_dais[i].id == id) { + *dai_name = q6usb_be_dais[i].name; + ret = 0; + break; + } + } + + return ret; +} + +static int q6usb_component_probe(struct snd_soc_component *component) +{ + struct q6usb_port_data *data = dev_get_drvdata(component->dev); + struct snd_soc_dapm_context *dapm = snd_soc_component_get_dapm(component); + + data->usb->component = component; + + snd_soc_dapm_disable_pin(dapm, "USB_RX_BE"); + snd_soc_dapm_sync(dapm); + + return 0; +} + +static const struct snd_soc_component_driver q6usb_dai_component = { + .probe = q6usb_component_probe, + .name = "q6usb-dai-component", + .dapm_widgets = q6usb_dai_widgets, + .num_dapm_widgets = ARRAY_SIZE(q6usb_dai_widgets), + .dapm_routes = q6usb_dapm_routes, + .num_dapm_routes = ARRAY_SIZE(q6usb_dapm_routes), + .of_xlate_dai_name = q6usb_audio_ports_of_xlate_dai_name, +}; + +int q6usb_alsa_connection_cb(struct snd_soc_usb *usb, int card_idx, + int connected) +{ + struct snd_soc_dapm_context *dapm; + struct q6usb_port_data *data; + + if (!usb->component) + return 0; + + dapm = snd_soc_component_get_dapm(usb->component); + data = dev_get_drvdata(usb->component->dev); + + if (connected) { + snd_soc_dapm_enable_pin(dapm, "USB_RX_BE"); + /* We only track the latest USB headset plugged in */ + data->active_idx = card_idx; + } else { + snd_soc_dapm_disable_pin(dapm, "USB_RX_BE"); + } + snd_soc_dapm_sync(dapm); + + return 0; +} + +static int q6usb_dai_dev_probe(struct platform_device *pdev) +{ + struct device_node *node = pdev->dev.of_node; + struct q6usb_port_data *data; + struct device *dev = &pdev->dev; + int ret; + + data = devm_kzalloc(dev, sizeof(*data), GFP_KERNEL); + if (!data) + return -ENOMEM; + + ret = of_property_read_u32(node, "qcom,usb-audio-stream-id", + &data->priv.sid); + if (ret) { + dev_err(&pdev->dev, "failed to read sid.\n"); + return -ENODEV; + } + + ret = of_property_read_u32(node, "qcom,usb-audio-intr-num", + &data->priv.intr_num); + if (ret) { + dev_err(&pdev->dev, "failed to read intr num.\n"); + return -ENODEV; + } + + data->priv.domain = iommu_domain_alloc(pdev->dev.bus); + if (!data->priv.domain) { + dev_err(&pdev->dev, "failed to allocate iommu domain\n"); + return -ENODEV; + } + + /* attach to external processor iommu */ + ret = iommu_attach_device(data->priv.domain, &pdev->dev); + if (ret) { + dev_err(&pdev->dev, "failed to attach device ret = %d\n", ret); + goto free_domain; + } + + data->usb = snd_soc_usb_add_port(dev, q6usb_alsa_connection_cb); + if (IS_ERR(data->usb)) { + dev_err(&pdev->dev, "failed to add usb port\n"); + goto detach_device; + } + + data->priv.dev = dev; + dev_set_drvdata(dev, data); + devm_snd_soc_register_component(dev, &q6usb_dai_component, + q6usb_be_dais, ARRAY_SIZE(q6usb_be_dais)); + snd_soc_usb_set_priv_data(&data->priv); + + return 0; + +detach_device: + iommu_detach_device(data->priv.domain, &pdev->dev); +free_domain: + iommu_domain_free(data->priv.domain); + + return ret; +} + +static int q6usb_dai_dev_remove(struct platform_device *pdev) +{ + struct q6usb_port_data *data = platform_get_drvdata(pdev); + + iommu_detach_device(data->priv.domain, &pdev->dev); + iommu_domain_free(data->priv.domain); + + snd_soc_usb_remove_port(); + + return 0; +} + +#ifdef CONFIG_OF +static const struct of_device_id q6usb_dai_device_id[] = { + { .compatible = "qcom,q6usb-dais" }, + {}, +}; +MODULE_DEVICE_TABLE(of, q6afe_dai_device_id); +#endif + +static struct platform_driver q6usb_dai_platform_driver = { + .driver = { + .name = "q6usb-dai", + .of_match_table = of_match_ptr(q6usb_dai_device_id), + }, + .probe = q6usb_dai_dev_probe, + .remove = q6usb_dai_dev_remove, +}; +module_platform_driver(q6usb_dai_platform_driver); + +MODULE_DESCRIPTION("Q6 USB backend dai driver"); +MODULE_LICENSE("GPL");
On Fri, Dec 23, 2022 at 03:31:49PM -0800, Wesley Cheng wrote:
Create a USB BE component that will register a new USB port to the ASoC USB framework. This will handle determination on if the requested audio profile is supported by the USB device currently selected.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com
include/sound/q6usboffload.h | 20 +++ sound/soc/qcom/Kconfig | 4 + sound/soc/qcom/qdsp6/Makefile | 1 + sound/soc/qcom/qdsp6/q6usb.c | 232 ++++++++++++++++++++++++++++++++++ 4 files changed, 257 insertions(+) create mode 100644 include/sound/q6usboffload.h create mode 100644 sound/soc/qcom/qdsp6/q6usb.c
diff --git a/include/sound/q6usboffload.h b/include/sound/q6usboffload.h new file mode 100644 index 000000000000..e576808901d9 --- /dev/null +++ b/include/sound/q6usboffload.h @@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: GPL-2.0
- linux/sound/q6usboffload.h -- QDSP6 USB offload
- Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
- */
+/**
- struct q6usb_offload
- @dev - dev handle to usb be
"be"? What is that?
- @sid - streamID for iommu
- @intr_num - usb interrupter number
- @domain - allocated iommu domain
- **/
+struct q6usb_offload {
- struct device *dev;
Do you properly reference count this?
- u32 sid;
- u32 intr_num;
- struct iommu_domain *domain;
+};
What is the lifetime of this structure? Who owns it? Where is the lock for accessing it?
diff --git a/sound/soc/qcom/Kconfig b/sound/soc/qcom/Kconfig index 8c7398bc1ca8..d65c365116e5 100644 --- a/sound/soc/qcom/Kconfig +++ b/sound/soc/qcom/Kconfig @@ -111,6 +111,9 @@ config SND_SOC_QDSP6_APM config SND_SOC_QDSP6_PRM_LPASS_CLOCKS tristate
+config SND_SOC_QDSP6_USB
- tristate
config SND_SOC_QDSP6_PRM tristate select SND_SOC_QDSP6_PRM_LPASS_CLOCKS @@ -131,6 +134,7 @@ config SND_SOC_QDSP6 select SND_SOC_TOPOLOGY select SND_SOC_QDSP6_APM select SND_SOC_QDSP6_PRM
- select SND_SOC_QDSP6_USB help To add support for MSM QDSP6 Soc Audio. This will enable sound soc platform specific
diff --git a/sound/soc/qcom/qdsp6/Makefile b/sound/soc/qcom/qdsp6/Makefile index 3963bf234664..c9457ee898d0 100644 --- a/sound/soc/qcom/qdsp6/Makefile +++ b/sound/soc/qcom/qdsp6/Makefile @@ -17,3 +17,4 @@ obj-$(CONFIG_SND_SOC_QDSP6_APM_DAI) += q6apm-dai.o obj-$(CONFIG_SND_SOC_QDSP6_APM_LPASS_DAI) += q6apm-lpass-dais.o obj-$(CONFIG_SND_SOC_QDSP6_PRM) += q6prm.o obj-$(CONFIG_SND_SOC_QDSP6_PRM_LPASS_CLOCKS) += q6prm-clocks.o +obj-$(CONFIG_SND_SOC_QDSP6_USB) += q6usb.o diff --git a/sound/soc/qcom/qdsp6/q6usb.c b/sound/soc/qcom/qdsp6/q6usb.c new file mode 100644 index 000000000000..a9da6dec6c6f --- /dev/null +++ b/sound/soc/qcom/qdsp6/q6usb.c @@ -0,0 +1,232 @@ +// SPDX-License-Identifier: GPL-2.0 +/*
- Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
All of the code in this patch series is older than 2022 as I know it has been in shipping devices for many years. Please use the proper copyright year to make your lawyers happy...
- */
+#include <linux/err.h> +#include <linux/init.h> +#include <linux/module.h> +#include <linux/device.h> +#include <linux/platform_device.h> +#include <linux/slab.h> +#include <linux/iommu.h> +#include <linux/dma-mapping.h> +#include <linux/dma-map-ops.h>
+#include <sound/pcm.h> +#include <sound/soc.h> +#include <sound/soc-usb.h> +#include <sound/pcm_params.h> +#include <sound/asound.h> +#include <sound/q6usboffload.h>
+#include "q6dsp-lpass-ports.h" +#include "q6afe.h"
+struct q6usb_port_data {
- struct q6afe_usb_cfg usb_cfg;
- struct snd_soc_usb *usb;
- struct q6usb_offload priv;
- int active_idx;
+};
+static const struct snd_soc_dapm_widget q6usb_dai_widgets[] = {
- SND_SOC_DAPM_HP("USB_RX_BE", NULL),
+};
+static const struct snd_soc_dapm_route q6usb_dapm_routes[] = {
- {"USB Playback", NULL, "USB_RX_BE"},
+};
No terminating entry? How does this not break? Why do you need to specify the size of the array, that feels like a design bug somewhere.
+static int q6usb_hw_params(struct snd_pcm_substream *substream,
struct snd_pcm_hw_params *params,
struct snd_soc_dai *dai)
+{
- return 0;
+} +static const struct snd_soc_dai_ops q6usb_ops = {
- .hw_params = q6usb_hw_params,
+};
+static struct snd_soc_dai_driver q6usb_be_dais[] = {
- {
.playback = {
.stream_name = "USB BE RX",
.rates = SNDRV_PCM_RATE_8000 | SNDRV_PCM_RATE_11025 |
SNDRV_PCM_RATE_16000 | SNDRV_PCM_RATE_22050 |
SNDRV_PCM_RATE_32000 | SNDRV_PCM_RATE_44100 |
SNDRV_PCM_RATE_48000 | SNDRV_PCM_RATE_96000 |
SNDRV_PCM_RATE_192000,
.formats = SNDRV_PCM_FMTBIT_S16_LE | SNDRV_PCM_FMTBIT_S16_BE |
SNDRV_PCM_FMTBIT_U16_LE | SNDRV_PCM_FMTBIT_U16_BE |
SNDRV_PCM_FMTBIT_S24_LE | SNDRV_PCM_FMTBIT_S24_BE |
SNDRV_PCM_FMTBIT_U24_LE | SNDRV_PCM_FMTBIT_U24_BE,
.channels_min = 1,
.channels_max = 2,
.rate_max = 192000,
.rate_min = 8000,
},
.id = USB_RX,
.name = "USB_RX_BE",
.ops = &q6usb_ops,
- },
+};
+int q6usb_audio_ports_of_xlate_dai_name(struct snd_soc_component *component,
const struct of_phandle_args *args,
const char **dai_name)
+{
- int id = args->args[0];
- int ret = -EINVAL;
- int i;
- for (i = 0; i < ARRAY_SIZE(q6usb_be_dais); i++) {
if (q6usb_be_dais[i].id == id) {
*dai_name = q6usb_be_dais[i].name;
ret = 0;
break;
}
- }
- return ret;
+}
+static int q6usb_component_probe(struct snd_soc_component *component) +{
- struct q6usb_port_data *data = dev_get_drvdata(component->dev);
- struct snd_soc_dapm_context *dapm = snd_soc_component_get_dapm(component);
- data->usb->component = component;
- snd_soc_dapm_disable_pin(dapm, "USB_RX_BE");
- snd_soc_dapm_sync(dapm);
- return 0;
+}
+static const struct snd_soc_component_driver q6usb_dai_component = {
- .probe = q6usb_component_probe,
- .name = "q6usb-dai-component",
- .dapm_widgets = q6usb_dai_widgets,
- .num_dapm_widgets = ARRAY_SIZE(q6usb_dai_widgets),
- .dapm_routes = q6usb_dapm_routes,
- .num_dapm_routes = ARRAY_SIZE(q6usb_dapm_routes),
- .of_xlate_dai_name = q6usb_audio_ports_of_xlate_dai_name,
+};
+int q6usb_alsa_connection_cb(struct snd_soc_usb *usb, int card_idx,
int connected)
+{
- struct snd_soc_dapm_context *dapm;
- struct q6usb_port_data *data;
- if (!usb->component)
return 0;
How can this happen?
Why is this not an error?
- dapm = snd_soc_component_get_dapm(usb->component);
- data = dev_get_drvdata(usb->component->dev);
- if (connected) {
snd_soc_dapm_enable_pin(dapm, "USB_RX_BE");
/* We only track the latest USB headset plugged in */
data->active_idx = card_idx;
- } else {
snd_soc_dapm_disable_pin(dapm, "USB_RX_BE");
- }
- snd_soc_dapm_sync(dapm);
- return 0;
+}
+static int q6usb_dai_dev_probe(struct platform_device *pdev) +{
- struct device_node *node = pdev->dev.of_node;
- struct q6usb_port_data *data;
- struct device *dev = &pdev->dev;
- int ret;
- data = devm_kzalloc(dev, sizeof(*data), GFP_KERNEL);
- if (!data)
return -ENOMEM;
- ret = of_property_read_u32(node, "qcom,usb-audio-stream-id",
&data->priv.sid);
- if (ret) {
dev_err(&pdev->dev, "failed to read sid.\n");
return -ENODEV;
- }
- ret = of_property_read_u32(node, "qcom,usb-audio-intr-num",
&data->priv.intr_num);
- if (ret) {
dev_err(&pdev->dev, "failed to read intr num.\n");
return -ENODEV;
- }
- data->priv.domain = iommu_domain_alloc(pdev->dev.bus);
- if (!data->priv.domain) {
dev_err(&pdev->dev, "failed to allocate iommu domain\n");
return -ENODEV;
- }
- /* attach to external processor iommu */
- ret = iommu_attach_device(data->priv.domain, &pdev->dev);
- if (ret) {
dev_err(&pdev->dev, "failed to attach device ret = %d\n", ret);
goto free_domain;
- }
- data->usb = snd_soc_usb_add_port(dev, q6usb_alsa_connection_cb);
- if (IS_ERR(data->usb)) {
dev_err(&pdev->dev, "failed to add usb port\n");
goto detach_device;
- }
- data->priv.dev = dev;
- dev_set_drvdata(dev, data);
- devm_snd_soc_register_component(dev, &q6usb_dai_component,
q6usb_be_dais, ARRAY_SIZE(q6usb_be_dais));
Very odd indentation. Please do this properly everywhere.
- snd_soc_usb_set_priv_data(&data->priv);
- return 0;
+detach_device:
- iommu_detach_device(data->priv.domain, &pdev->dev);
+free_domain:
- iommu_domain_free(data->priv.domain);
- return ret;
+}
+static int q6usb_dai_dev_remove(struct platform_device *pdev) +{
- struct q6usb_port_data *data = platform_get_drvdata(pdev);
- iommu_detach_device(data->priv.domain, &pdev->dev);
- iommu_domain_free(data->priv.domain);
- snd_soc_usb_remove_port();
- return 0;
+}
+#ifdef CONFIG_OF
Is this really needed still?
thanks,
greg k-h
On Sat, Dec 24, 2022 at 10:02:59AM +0100, Greg KH wrote:
On Fri, Dec 23, 2022 at 03:31:49PM -0800, Wesley Cheng wrote:
- struct q6usb_offload
- @dev - dev handle to usb be
"be"? What is that?
Back end. This is a concept in DPCM which should be reasonably discoverable to people working on the audio portions of this code.
+// SPDX-License-Identifier: GPL-2.0 +/*
- Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
All of the code in this patch series is older than 2022 as I know it has been in shipping devices for many years. Please use the proper copyright year to make your lawyers happy...
Are you *positive* about this. Based on some preparatory discussions the Qualcomm people had with Takashi and I it seemed like this was a new version of existing concepts. I'm sure they had something already but it's not obvious to me that they're just posting the same code.
+static const struct snd_soc_dapm_route q6usb_dapm_routes[] = {
- {"USB Playback", NULL, "USB_RX_BE"},
+};
No terminating entry? How does this not break? Why do you need to specify the size of the array, that feels like a design bug somewhere.
It's how the interface works, the number of entries is passed in when adding routes.
On Tue, Dec 27, 2022 at 01:04:55PM +0000, Mark Brown wrote:
On Sat, Dec 24, 2022 at 10:02:59AM +0100, Greg KH wrote:
On Fri, Dec 23, 2022 at 03:31:49PM -0800, Wesley Cheng wrote:
- struct q6usb_offload
- @dev - dev handle to usb be
"be"? What is that?
Back end. This is a concept in DPCM which should be reasonably discoverable to people working on the audio portions of this code.
Ok, then how is the reference counting logic handled here? USB devices can be removed from the system at any point in time...
+// SPDX-License-Identifier: GPL-2.0 +/*
- Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
All of the code in this patch series is older than 2022 as I know it has been in shipping devices for many years. Please use the proper copyright year to make your lawyers happy...
Are you *positive* about this. Based on some preparatory discussions the Qualcomm people had with Takashi and I it seemed like this was a new version of existing concepts. I'm sure they had something already but it's not obvious to me that they're just posting the same code.
I thought that this same code has been shipping in devices for a few years now in the last few Samsung phone models. Is this not the same code that is in those devices?
If not, why not, what happened to that codebase that makes it not worthy of being submitted upstream?
+static const struct snd_soc_dapm_route q6usb_dapm_routes[] = {
- {"USB Playback", NULL, "USB_RX_BE"},
+};
No terminating entry? How does this not break? Why do you need to specify the size of the array, that feels like a design bug somewhere.
It's how the interface works, the number of entries is passed in when adding routes.
Ah, you all might want to change that to make it simpler :)
thanks,
greg k-h
On Tue, 27 Dec 2022 14:45:13 +0100, Greg KH wrote:
On Tue, Dec 27, 2022 at 01:04:55PM +0000, Mark Brown wrote:
On Sat, Dec 24, 2022 at 10:02:59AM +0100, Greg KH wrote:
On Fri, Dec 23, 2022 at 03:31:49PM -0800, Wesley Cheng wrote:
- struct q6usb_offload
- @dev - dev handle to usb be
"be"? What is that?
Back end. This is a concept in DPCM which should be reasonably discoverable to people working on the audio portions of this code.
Ok, then how is the reference counting logic handled here? USB devices can be removed from the system at any point in time...
The whole picture is fairly complex, and this patch is a part belonging to the ASoC machine driver -- that is, it's bound to the Qualcomm host, and there can be only one on a system.
OTOH, USB audio devices are still managed by the existing USB audio driver as is, and they can be multiple and any devices. The basic idea here is a hijack of the USB data processing in USB audio driver with the offloading mechanism by this ASoC driver (only if the condition met).
thanks,
Takashi
On Tue, Dec 27, 2022 at 03:02:31PM +0100, Takashi Iwai wrote:
Greg KH wrote:
On Tue, Dec 27, 2022 at 01:04:55PM +0000, Mark Brown wrote:
On Sat, Dec 24, 2022 at 10:02:59AM +0100, Greg KH wrote:
"be"? What is that?
Back end. This is a concept in DPCM which should be reasonably discoverable to people working on the audio portions of this code.
Ok, then how is the reference counting logic handled here? USB devices can be removed from the system at any point in time...
The whole picture is fairly complex, and this patch is a part belonging to the ASoC machine driver -- that is, it's bound to the Qualcomm host, and there can be only one on a system.
OTOH, USB audio devices are still managed by the existing USB audio driver as is, and they can be multiple and any devices. The basic idea here is a hijack of the USB data processing in USB audio driver with the offloading mechanism by this ASoC driver (only if the condition met).
Right. I haven't even begun to look at the actual code here, just triaging my inbox, so I've got no thoughts on if things work or not.
On Tue, Dec 27, 2022 at 02:45:13PM +0100, Greg KH wrote:
On Tue, Dec 27, 2022 at 01:04:55PM +0000, Mark Brown wrote:
On Sat, Dec 24, 2022 at 10:02:59AM +0100, Greg KH wrote:
- Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
All of the code in this patch series is older than 2022 as I know it has been in shipping devices for many years. Please use the proper copyright year to make your lawyers happy...
Are you *positive* about this. Based on some preparatory discussions the Qualcomm people had with Takashi and I it seemed like this was a new version of existing concepts. I'm sure they had something already but it's not obvious to me that they're just posting the same code.
I thought that this same code has been shipping in devices for a few years now in the last few Samsung phone models. Is this not the same code that is in those devices?
If not, why not, what happened to that codebase that makes it not worthy of being submitted upstream?
I don't specifically know anything about that code but I'd expect that for out of tree code breaking new ground like this there'd be a strong likelyhood that there'd be design level issues and that's what the pre submission discussions were all about - how to fit the concept into the kernel subsystems in a way that might be maintainable. There's also been the whole transition to their new DSP firmware going on. It's possible that what was shipped was implemented in the same way with the same code but I'd not assume that this is the case without actually comparing the two.
Hi Mark/Greg,
On 12/27/2022 7:11 AM, Mark Brown wrote:
On Tue, Dec 27, 2022 at 02:45:13PM +0100, Greg KH wrote:
On Tue, Dec 27, 2022 at 01:04:55PM +0000, Mark Brown wrote:
On Sat, Dec 24, 2022 at 10:02:59AM +0100, Greg KH wrote:
- Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
All of the code in this patch series is older than 2022 as I know it has been in shipping devices for many years. Please use the proper copyright year to make your lawyers happy...
Are you *positive* about this. Based on some preparatory discussions the Qualcomm people had with Takashi and I it seemed like this was a new version of existing concepts. I'm sure they had something already but it's not obvious to me that they're just posting the same code.
I thought that this same code has been shipping in devices for a few years now in the last few Samsung phone models. Is this not the same code that is in those devices?
If not, why not, what happened to that codebase that makes it not worthy of being submitted upstream?
I don't specifically know anything about that code but I'd expect that for out of tree code breaking new ground like this there'd be a strong likelyhood that there'd be design level issues and that's what the pre submission discussions were all about - how to fit the concept into the kernel subsystems in a way that might be maintainable. There's also been the whole transition to their new DSP firmware going on. It's possible that what was shipped was implemented in the same way with the same code but I'd not assume that this is the case without actually comparing the two.
It's correct that all the ASoC related patches are new, and didn't exist previously in QC products. It is due to the fact that Android has a different ALSA userspace concept which allowed for a lot of the Q6 audio front end (AFE) communication to be done from userspace, I believe. This is the reason that we had to introduce this new ASoC based design.
I can't comment much more about the Android ALSA design, maybe @Patrick Lai can.
Thanks Wesley Cheng
Hi Greg,
On 12/24/2022 1:02 AM, Greg KH wrote:
On Fri, Dec 23, 2022 at 03:31:49PM -0800, Wesley Cheng wrote:
Create a USB BE component that will register a new USB port to the ASoC USB framework. This will handle determination on if the requested audio profile is supported by the USB device currently selected.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com
include/sound/q6usboffload.h | 20 +++ sound/soc/qcom/Kconfig | 4 + sound/soc/qcom/qdsp6/Makefile | 1 + sound/soc/qcom/qdsp6/q6usb.c | 232 ++++++++++++++++++++++++++++++++++ 4 files changed, 257 insertions(+) create mode 100644 include/sound/q6usboffload.h create mode 100644 sound/soc/qcom/qdsp6/q6usb.c
diff --git a/include/sound/q6usboffload.h b/include/sound/q6usboffload.h new file mode 100644 index 000000000000..e576808901d9 --- /dev/null +++ b/include/sound/q6usboffload.h @@ -0,0 +1,20 @@ +/* SPDX-License-Identifier: GPL-2.0
- linux/sound/q6usboffload.h -- QDSP6 USB offload
- Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
- */
+/**
- struct q6usb_offload
- @dev - dev handle to usb be
"be"? What is that?
- @sid - streamID for iommu
- @intr_num - usb interrupter number
- @domain - allocated iommu domain
- **/
+struct q6usb_offload {
- struct device *dev;
Do you properly reference count this?
- u32 sid;
- u32 intr_num;
- struct iommu_domain *domain;
+};
What is the lifetime of this structure? Who owns it? Where is the lock for accessing it?
Owner of this structure is the USB backend. If the USB backend is removed, then the qc_audio_offload driver would never receive the QMI request to enable the audio stream path from the audio dsp. (where this is referenced) It will exist as long as the USB BE device exists.
One thing I will need to follow up on is if an ASoC backend device is removed while an audio playback is happening how it would handle it.
diff --git a/sound/soc/qcom/Kconfig b/sound/soc/qcom/Kconfig index 8c7398bc1ca8..d65c365116e5 100644 --- a/sound/soc/qcom/Kconfig +++ b/sound/soc/qcom/Kconfig @@ -111,6 +111,9 @@ config SND_SOC_QDSP6_APM config SND_SOC_QDSP6_PRM_LPASS_CLOCKS tristate
+config SND_SOC_QDSP6_USB
- tristate
- config SND_SOC_QDSP6_PRM tristate select SND_SOC_QDSP6_PRM_LPASS_CLOCKS
@@ -131,6 +134,7 @@ config SND_SOC_QDSP6 select SND_SOC_TOPOLOGY select SND_SOC_QDSP6_APM select SND_SOC_QDSP6_PRM
- select SND_SOC_QDSP6_USB help To add support for MSM QDSP6 Soc Audio. This will enable sound soc platform specific
diff --git a/sound/soc/qcom/qdsp6/Makefile b/sound/soc/qcom/qdsp6/Makefile index 3963bf234664..c9457ee898d0 100644 --- a/sound/soc/qcom/qdsp6/Makefile +++ b/sound/soc/qcom/qdsp6/Makefile @@ -17,3 +17,4 @@ obj-$(CONFIG_SND_SOC_QDSP6_APM_DAI) += q6apm-dai.o obj-$(CONFIG_SND_SOC_QDSP6_APM_LPASS_DAI) += q6apm-lpass-dais.o obj-$(CONFIG_SND_SOC_QDSP6_PRM) += q6prm.o obj-$(CONFIG_SND_SOC_QDSP6_PRM_LPASS_CLOCKS) += q6prm-clocks.o +obj-$(CONFIG_SND_SOC_QDSP6_USB) += q6usb.o diff --git a/sound/soc/qcom/qdsp6/q6usb.c b/sound/soc/qcom/qdsp6/q6usb.c new file mode 100644 index 000000000000..a9da6dec6c6f --- /dev/null +++ b/sound/soc/qcom/qdsp6/q6usb.c @@ -0,0 +1,232 @@ +// SPDX-License-Identifier: GPL-2.0 +/*
- Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved.
All of the code in this patch series is older than 2022 as I know it has been in shipping devices for many years. Please use the proper copyright year to make your lawyers happy...
- */
+#include <linux/err.h> +#include <linux/init.h> +#include <linux/module.h> +#include <linux/device.h> +#include <linux/platform_device.h> +#include <linux/slab.h> +#include <linux/iommu.h> +#include <linux/dma-mapping.h> +#include <linux/dma-map-ops.h>
+#include <sound/pcm.h> +#include <sound/soc.h> +#include <sound/soc-usb.h> +#include <sound/pcm_params.h> +#include <sound/asound.h> +#include <sound/q6usboffload.h>
+#include "q6dsp-lpass-ports.h" +#include "q6afe.h"
+struct q6usb_port_data {
- struct q6afe_usb_cfg usb_cfg;
- struct snd_soc_usb *usb;
- struct q6usb_offload priv;
- int active_idx;
+};
+static const struct snd_soc_dapm_widget q6usb_dai_widgets[] = {
- SND_SOC_DAPM_HP("USB_RX_BE", NULL),
+};
+static const struct snd_soc_dapm_route q6usb_dapm_routes[] = {
- {"USB Playback", NULL, "USB_RX_BE"},
+};
No terminating entry? How does this not break? Why do you need to specify the size of the array, that feels like a design bug somewhere.
+static int q6usb_hw_params(struct snd_pcm_substream *substream,
struct snd_pcm_hw_params *params,
struct snd_soc_dai *dai)
+{
- return 0;
+} +static const struct snd_soc_dai_ops q6usb_ops = {
- .hw_params = q6usb_hw_params,
+};
+static struct snd_soc_dai_driver q6usb_be_dais[] = {
- {
.playback = {
.stream_name = "USB BE RX",
.rates = SNDRV_PCM_RATE_8000 | SNDRV_PCM_RATE_11025 |
SNDRV_PCM_RATE_16000 | SNDRV_PCM_RATE_22050 |
SNDRV_PCM_RATE_32000 | SNDRV_PCM_RATE_44100 |
SNDRV_PCM_RATE_48000 | SNDRV_PCM_RATE_96000 |
SNDRV_PCM_RATE_192000,
.formats = SNDRV_PCM_FMTBIT_S16_LE | SNDRV_PCM_FMTBIT_S16_BE |
SNDRV_PCM_FMTBIT_U16_LE | SNDRV_PCM_FMTBIT_U16_BE |
SNDRV_PCM_FMTBIT_S24_LE | SNDRV_PCM_FMTBIT_S24_BE |
SNDRV_PCM_FMTBIT_U24_LE | SNDRV_PCM_FMTBIT_U24_BE,
.channels_min = 1,
.channels_max = 2,
.rate_max = 192000,
.rate_min = 8000,
},
.id = USB_RX,
.name = "USB_RX_BE",
.ops = &q6usb_ops,
- },
+};
+int q6usb_audio_ports_of_xlate_dai_name(struct snd_soc_component *component,
const struct of_phandle_args *args,
const char **dai_name)
+{
- int id = args->args[0];
- int ret = -EINVAL;
- int i;
- for (i = 0; i < ARRAY_SIZE(q6usb_be_dais); i++) {
if (q6usb_be_dais[i].id == id) {
*dai_name = q6usb_be_dais[i].name;
ret = 0;
break;
}
- }
- return ret;
+}
+static int q6usb_component_probe(struct snd_soc_component *component) +{
- struct q6usb_port_data *data = dev_get_drvdata(component->dev);
- struct snd_soc_dapm_context *dapm = snd_soc_component_get_dapm(component);
- data->usb->component = component;
- snd_soc_dapm_disable_pin(dapm, "USB_RX_BE");
- snd_soc_dapm_sync(dapm);
- return 0;
+}
+static const struct snd_soc_component_driver q6usb_dai_component = {
- .probe = q6usb_component_probe,
- .name = "q6usb-dai-component",
- .dapm_widgets = q6usb_dai_widgets,
- .num_dapm_widgets = ARRAY_SIZE(q6usb_dai_widgets),
- .dapm_routes = q6usb_dapm_routes,
- .num_dapm_routes = ARRAY_SIZE(q6usb_dapm_routes),
- .of_xlate_dai_name = q6usb_audio_ports_of_xlate_dai_name,
+};
+int q6usb_alsa_connection_cb(struct snd_soc_usb *usb, int card_idx,
int connected)
+{
- struct snd_soc_dapm_context *dapm;
- struct q6usb_port_data *data;
- if (!usb->component)
return 0;
How can this happen?
Why is this not an error?
If the ASoC platform card that this USB BE is a child of, is not yet registered then the component probe hasn't happened. However, that is independent of when the USB connect/disconnect events can happen.
- dapm = snd_soc_component_get_dapm(usb->component);
- data = dev_get_drvdata(usb->component->dev);
- if (connected) {
snd_soc_dapm_enable_pin(dapm, "USB_RX_BE");
/* We only track the latest USB headset plugged in */
data->active_idx = card_idx;
- } else {
snd_soc_dapm_disable_pin(dapm, "USB_RX_BE");
- }
- snd_soc_dapm_sync(dapm);
- return 0;
+}
+static int q6usb_dai_dev_probe(struct platform_device *pdev) +{
- struct device_node *node = pdev->dev.of_node;
- struct q6usb_port_data *data;
- struct device *dev = &pdev->dev;
- int ret;
- data = devm_kzalloc(dev, sizeof(*data), GFP_KERNEL);
- if (!data)
return -ENOMEM;
- ret = of_property_read_u32(node, "qcom,usb-audio-stream-id",
&data->priv.sid);
- if (ret) {
dev_err(&pdev->dev, "failed to read sid.\n");
return -ENODEV;
- }
- ret = of_property_read_u32(node, "qcom,usb-audio-intr-num",
&data->priv.intr_num);
- if (ret) {
dev_err(&pdev->dev, "failed to read intr num.\n");
return -ENODEV;
- }
- data->priv.domain = iommu_domain_alloc(pdev->dev.bus);
- if (!data->priv.domain) {
dev_err(&pdev->dev, "failed to allocate iommu domain\n");
return -ENODEV;
- }
- /* attach to external processor iommu */
- ret = iommu_attach_device(data->priv.domain, &pdev->dev);
- if (ret) {
dev_err(&pdev->dev, "failed to attach device ret = %d\n", ret);
goto free_domain;
- }
- data->usb = snd_soc_usb_add_port(dev, q6usb_alsa_connection_cb);
- if (IS_ERR(data->usb)) {
dev_err(&pdev->dev, "failed to add usb port\n");
goto detach_device;
- }
- data->priv.dev = dev;
- dev_set_drvdata(dev, data);
- devm_snd_soc_register_component(dev, &q6usb_dai_component,
q6usb_be_dais, ARRAY_SIZE(q6usb_be_dais));
Very odd indentation. Please do this properly everywhere.
Got it...will fix everywhere
- snd_soc_usb_set_priv_data(&data->priv);
- return 0;
+detach_device:
- iommu_detach_device(data->priv.domain, &pdev->dev);
+free_domain:
- iommu_domain_free(data->priv.domain);
- return ret;
+}
+static int q6usb_dai_dev_remove(struct platform_device *pdev) +{
- struct q6usb_port_data *data = platform_get_drvdata(pdev);
- iommu_detach_device(data->priv.domain, &pdev->dev);
- iommu_domain_free(data->priv.domain);
- snd_soc_usb_remove_port();
- return 0;
+}
+#ifdef CONFIG_OF
Is this really needed still?
Will remove this
Thanks Wesley Cheng
+int q6usb_alsa_connection_cb(struct snd_soc_usb *usb, int card_idx,
int connected)
+{
- struct snd_soc_dapm_context *dapm;
- struct q6usb_port_data *data;
- if (!usb->component)
return 0;
- dapm = snd_soc_component_get_dapm(usb->component);
- data = dev_get_drvdata(usb->component->dev);
- if (connected) {
snd_soc_dapm_enable_pin(dapm, "USB_RX_BE");
/* We only track the latest USB headset plugged in */
that answers to my earlier question on how to deal with multiple devices, but is this a desirable policy? This could lead to a lot of confusion. If there are restrictions to a single device, then it might be more interesting for userspace or the user to indicate which USB device gets to use USB offload and all others use legacy.
data->active_idx = card_idx;
- } else {
snd_soc_dapm_disable_pin(dapm, "USB_RX_BE");
- }
- snd_soc_dapm_sync(dapm);
- return 0;
+}
Hi Pierre,
On 1/4/2023 3:41 PM, Pierre-Louis Bossart wrote:
+int q6usb_alsa_connection_cb(struct snd_soc_usb *usb, int card_idx,
int connected)
+{
- struct snd_soc_dapm_context *dapm;
- struct q6usb_port_data *data;
- if (!usb->component)
return 0;
- dapm = snd_soc_component_get_dapm(usb->component);
- data = dev_get_drvdata(usb->component->dev);
- if (connected) {
snd_soc_dapm_enable_pin(dapm, "USB_RX_BE");
/* We only track the latest USB headset plugged in */
that answers to my earlier question on how to deal with multiple devices, but is this a desirable policy? This could lead to a lot of confusion. If there are restrictions to a single device, then it might be more interesting for userspace or the user to indicate which USB device gets to use USB offload and all others use legacy.
Yeah, as mentioned its still pretty open ended. I think from the feedback received from Mark/Takashi, this was a viable option for now. Would you happen to have any insight/input on how the userspace can pass down that selection to the ASoC framework? Maybe some kind of PCM IOCTL call?
Thanks Wesley Cheng
On 1/5/23 19:05, Wesley Cheng wrote:
Hi Pierre,
On 1/4/2023 3:41 PM, Pierre-Louis Bossart wrote:
+int q6usb_alsa_connection_cb(struct snd_soc_usb *usb, int card_idx, + int connected) +{ + struct snd_soc_dapm_context *dapm; + struct q6usb_port_data *data;
+ if (!usb->component) + return 0;
+ dapm = snd_soc_component_get_dapm(usb->component); + data = dev_get_drvdata(usb->component->dev);
+ if (connected) { + snd_soc_dapm_enable_pin(dapm, "USB_RX_BE"); + /* We only track the latest USB headset plugged in */
that answers to my earlier question on how to deal with multiple devices, but is this a desirable policy? This could lead to a lot of confusion. If there are restrictions to a single device, then it might be more interesting for userspace or the user to indicate which USB device gets to use USB offload and all others use legacy.
Yeah, as mentioned its still pretty open ended. I think from the feedback received from Mark/Takashi, this was a viable option for now. Would you happen to have any insight/input on how the userspace can pass down that selection to the ASoC framework? Maybe some kind of PCM IOCTL call?
I don't have a turn-key solution either :-) We'd need userspace to make one device as 'preferred' or 'optimized' and give it a priority somehow. It can't be a PCM IOCTL, it has to be at the device level.
It's really a second-level optimization that can be parked for now, the bulk of the work is really the interaction between USB audio and ASoC stacks, we should probably focus on that BIG topic with a design that can be shared across implementations.
Allow for different vendors to be notified on USB SND connect/disconnect seqeunces. This allows for vendor USB SND modules to properly initialize and populate internal structures with references to the USB SND chip device.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com --- sound/usb/card.c | 22 ++++++++++++++++++++++ sound/usb/card.h | 7 +++++++ 2 files changed, 29 insertions(+)
diff --git a/sound/usb/card.c b/sound/usb/card.c index 26268ffb8274..212f55a7683c 100644 --- a/sound/usb/card.c +++ b/sound/usb/card.c @@ -117,6 +117,21 @@ MODULE_PARM_DESC(skip_validation, "Skip unit descriptor validation (default: no) static DEFINE_MUTEX(register_mutex); static struct snd_usb_audio *usb_chip[SNDRV_CARDS]; static struct usb_driver usb_audio_driver; +static struct snd_usb_vendor_ops *vendor_ops; + +int snd_usb_register_vendor_ops(struct snd_usb_vendor_ops *ops) +{ + vendor_ops = ops; + return 0; +} +EXPORT_SYMBOL_GPL(snd_usb_register_vendor_ops); + +int snd_usb_unregister_vendor_ops(void) +{ + vendor_ops = NULL; + return 0; +} +EXPORT_SYMBOL_GPL(snd_usb_unregister_vendor_ops);
/* * disconnect streams @@ -910,6 +925,10 @@ static int usb_audio_probe(struct usb_interface *intf, usb_set_intfdata(intf, chip); atomic_dec(&chip->active); mutex_unlock(®ister_mutex); + + if (vendor_ops->connect_cb) + vendor_ops->connect_cb(intf, chip); + return 0;
__error: @@ -943,6 +962,9 @@ static void usb_audio_disconnect(struct usb_interface *intf) if (chip == USB_AUDIO_IFACE_UNUSED) return;
+ if (vendor_ops->disconnect_cb) + vendor_ops->disconnect_cb(intf); + card = chip->card;
mutex_lock(®ister_mutex); diff --git a/sound/usb/card.h b/sound/usb/card.h index 40061550105a..a785bb256b0d 100644 --- a/sound/usb/card.h +++ b/sound/usb/card.h @@ -206,4 +206,11 @@ struct snd_usb_stream { struct list_head list; };
+struct snd_usb_vendor_ops { + void (*connect_cb)(struct usb_interface *intf, struct snd_usb_audio *chip); + void (*disconnect_cb)(struct usb_interface *intf); +}; + +int snd_usb_register_vendor_ops(struct snd_usb_vendor_ops *ops); +int snd_usb_unregister_vendor_ops(void); #endif /* __USBAUDIO_CARD_H */
Hi,
On Sat, 24 Dec 2022 at 01:33, Wesley Cheng quic_wcheng@quicinc.com wrote:
Allow for different vendors to be notified on USB SND connect/disconnect seqeunces. This allows for vendor USB SND modules to properly initialize and populate internal structures with references to the USB SND chip device.
The commit message definitely needs some improvement. We do not notify vendors on SND connect/disconnect events.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com
sound/usb/card.c | 22 ++++++++++++++++++++++ sound/usb/card.h | 7 +++++++ 2 files changed, 29 insertions(+)
diff --git a/sound/usb/card.c b/sound/usb/card.c index 26268ffb8274..212f55a7683c 100644 --- a/sound/usb/card.c +++ b/sound/usb/card.c @@ -117,6 +117,21 @@ MODULE_PARM_DESC(skip_validation, "Skip unit descriptor validation (default: no) static DEFINE_MUTEX(register_mutex); static struct snd_usb_audio *usb_chip[SNDRV_CARDS]; static struct usb_driver usb_audio_driver; +static struct snd_usb_vendor_ops *vendor_ops;
+int snd_usb_register_vendor_ops(struct snd_usb_vendor_ops *ops)
platform ops?
+{
vendor_ops = ops;
return 0;
+} +EXPORT_SYMBOL_GPL(snd_usb_register_vendor_ops);
What happens if several platforms try to register different ops? I saw from the patch 09/14 that you register these ops unconditionally. If other devices follow your approach there is an obvious conflict.
+int snd_usb_unregister_vendor_ops(void) +{
vendor_ops = NULL;
return 0;
+} +EXPORT_SYMBOL_GPL(snd_usb_unregister_vendor_ops);
/*
- disconnect streams
@@ -910,6 +925,10 @@ static int usb_audio_probe(struct usb_interface *intf, usb_set_intfdata(intf, chip); atomic_dec(&chip->active); mutex_unlock(®ister_mutex);
if (vendor_ops->connect_cb)
vendor_ops->connect_cb(intf, chip);
return 0;
__error:
@@ -943,6 +962,9 @@ static void usb_audio_disconnect(struct usb_interface *intf) if (chip == USB_AUDIO_IFACE_UNUSED) return;
if (vendor_ops->disconnect_cb)
vendor_ops->disconnect_cb(intf);
card = chip->card; mutex_lock(®ister_mutex);
diff --git a/sound/usb/card.h b/sound/usb/card.h index 40061550105a..a785bb256b0d 100644 --- a/sound/usb/card.h +++ b/sound/usb/card.h @@ -206,4 +206,11 @@ struct snd_usb_stream { struct list_head list; };
+struct snd_usb_vendor_ops {
void (*connect_cb)(struct usb_interface *intf, struct snd_usb_audio *chip);
void (*disconnect_cb)(struct usb_interface *intf);
+};
+int snd_usb_register_vendor_ops(struct snd_usb_vendor_ops *ops); +int snd_usb_unregister_vendor_ops(void); #endif /* __USBAUDIO_CARD_H */
-- With best wishes
Dmitry
Hi Dmitry,
On 12/24/2022 3:03 AM, Dmitry Baryshkov wrote:
Hi,
On Sat, 24 Dec 2022 at 01:33, Wesley Cheng quic_wcheng@quicinc.com wrote:
Allow for different vendors to be notified on USB SND connect/disconnect seqeunces. This allows for vendor USB SND modules to properly initialize and populate internal structures with references to the USB SND chip device.
The commit message definitely needs some improvement. We do not notify vendors on SND connect/disconnect events.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com
sound/usb/card.c | 22 ++++++++++++++++++++++ sound/usb/card.h | 7 +++++++ 2 files changed, 29 insertions(+)
diff --git a/sound/usb/card.c b/sound/usb/card.c index 26268ffb8274..212f55a7683c 100644 --- a/sound/usb/card.c +++ b/sound/usb/card.c @@ -117,6 +117,21 @@ MODULE_PARM_DESC(skip_validation, "Skip unit descriptor validation (default: no) static DEFINE_MUTEX(register_mutex); static struct snd_usb_audio *usb_chip[SNDRV_CARDS]; static struct usb_driver usb_audio_driver; +static struct snd_usb_vendor_ops *vendor_ops;
+int snd_usb_register_vendor_ops(struct snd_usb_vendor_ops *ops)
platform ops?
Will change it.
+{
vendor_ops = ops;
return 0;
+} +EXPORT_SYMBOL_GPL(snd_usb_register_vendor_ops);
What happens if several platforms try to register different ops? I saw from the patch 09/14 that you register these ops unconditionally. If other devices follow your approach there is an obvious conflict.
Thank you for the review.
That is true. I don't think there is a proper need to have multiple vendor ops being registered, so maybe just returning an error for if ops are already registered is sufficient.
Thanks Wesley Cheng
On 27/12/2022 23:07, Wesley Cheng wrote:
Hi Dmitry,
On 12/24/2022 3:03 AM, Dmitry Baryshkov wrote:
Hi,
On Sat, 24 Dec 2022 at 01:33, Wesley Cheng quic_wcheng@quicinc.com wrote:
Allow for different vendors to be notified on USB SND connect/disconnect seqeunces. This allows for vendor USB SND modules to properly initialize and populate internal structures with references to the USB SND chip device.
The commit message definitely needs some improvement. We do not notify vendors on SND connect/disconnect events.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com
sound/usb/card.c | 22 ++++++++++++++++++++++ sound/usb/card.h | 7 +++++++ 2 files changed, 29 insertions(+)
diff --git a/sound/usb/card.c b/sound/usb/card.c index 26268ffb8274..212f55a7683c 100644 --- a/sound/usb/card.c +++ b/sound/usb/card.c @@ -117,6 +117,21 @@ MODULE_PARM_DESC(skip_validation, "Skip unit descriptor validation (default: no) static DEFINE_MUTEX(register_mutex); static struct snd_usb_audio *usb_chip[SNDRV_CARDS]; static struct usb_driver usb_audio_driver; +static struct snd_usb_vendor_ops *vendor_ops;
+int snd_usb_register_vendor_ops(struct snd_usb_vendor_ops *ops)
platform ops?
Will change it.
+{ + vendor_ops = ops; + return 0; +} +EXPORT_SYMBOL_GPL(snd_usb_register_vendor_ops);
What happens if several platforms try to register different ops? I saw from the patch 09/14 that you register these ops unconditionally. If other devices follow your approach there is an obvious conflict.
Thank you for the review.
That is true. I don't think there is a proper need to have multiple vendor ops being registered, so maybe just returning an error for if ops are already registered is sufficient.
This would be a required step. And also you have to check the running platform before registering your ops unconditionally. Ideally this should be done as a part of the device's driver, so that we can control registration of the platform ops using the usual interface.
Thanks Wesley Cheng
On 24.12.22 00:31, Wesley Cheng wrote:
Allow for different vendors to be notified on USB SND connect/disconnect seqeunces. This allows for vendor USB SND modules to properly initialize and populate internal structures with references to the USB SND chip device.
Hi,
this raises a design question. If the system is suspending or, worse, hibernating, how do you make sure the offloader and the device are suspended in the correct order? And what happens if you need to go into reset_resume() when resuming?
Regards Oliver
On Thu, 29 Dec 2022 14:49:21 +0100, Oliver Neukum wrote:
On 24.12.22 00:31, Wesley Cheng wrote:
Allow for different vendors to be notified on USB SND connect/disconnect seqeunces. This allows for vendor USB SND modules to properly initialize and populate internal structures with references to the USB SND chip device.
Hi,
this raises a design question. If the system is suspending or, worse, hibernating, how do you make sure the offloader and the device are suspended in the correct order? And what happens if you need to go into reset_resume() when resuming?
I guess we'd need to establish a device link when the binding from the offload driver is done. Then the PM order will be assured.
Takashi
Hi,
On 12/29/2022 6:20 AM, Takashi Iwai wrote:
On Thu, 29 Dec 2022 14:49:21 +0100, Oliver Neukum wrote:
On 24.12.22 00:31, Wesley Cheng wrote:
Allow for different vendors to be notified on USB SND connect/disconnect seqeunces. This allows for vendor USB SND modules to properly initialize and populate internal structures with references to the USB SND chip device.
Hi,
this raises a design question. If the system is suspending or, worse, hibernating, how do you make sure the offloader and the device are suspended in the correct order? And what happens if you need to go into reset_resume() when resuming?
It may depend on how the offloading is implemented, but we do have a mechanism to force the audio stream off from the qc_usb_audio_offload. Regardless of if the UDEV is suspended first, or the USB backend, as long as we ensure that the offloading is disabled before entering suspend, I think that should be sufficient. I would need to add some suspend handling in the offload driver to issue the command to stop the offloading.
As for the resume path, is there a concern if either device is resumed first? The only scenario where maybe it could cause some mishandling is if the USB backend is resumed before the offload driver is connected/resumed. This means that the userspace ALSA would have access to the platform sound card, and could potentially attempt to route audio streams to it. I think in worst case, if we were going through a reset_resume() we would end up rejecting that request coming from the audio DSP to enable the stream. However, userspace entities would be resumed/unfrozen last, so not sure if that would ever be a problem.
The reset_resume() path is fine. Bus reset is going to cause a disconnect() callback in the offload driver, in which we already have the proper handling for ensuring the offload path is halted, and we reject any incoming stream start requests.
Thanks Wesley Cheng
On 30.12.22 08:10, Wesley Cheng wrote:
It may depend on how the offloading is implemented, but we do have a mechanism to force the audio stream off from the qc_usb_audio_offload. Regardless of if the UDEV is suspended first, or the USB backend, as long as we ensure that the offloading is disabled before entering suspend, I think that should be sufficient.
You would presumably output garbage, if the UDEV is asleep but the backend is not.
The reset_resume() path is fine. Bus reset is going to cause a disconnect() callback in the offload driver, in which we already have the proper handling for ensuring the offload path is halted, and we reject any incoming stream start requests.
How? If we go the reset_resume() code path, we find that usb-audio does not make a difference between regular resume() and reset_resume()
Regards Oliver
On Tue, 03 Jan 2023 13:20:48 +0100, Oliver Neukum wrote:
On 30.12.22 08:10, Wesley Cheng wrote:
It may depend on how the offloading is implemented, but we do have a mechanism to force the audio stream off from the qc_usb_audio_offload. Regardless of if the UDEV is suspended first, or the USB backend, as long as we ensure that the offloading is disabled before entering suspend, I think that should be sufficient.
You would presumably output garbage, if the UDEV is asleep but the backend is not.
The reset_resume() path is fine. Bus reset is going to cause a disconnect() callback in the offload driver, in which we already have the proper handling for ensuring the offload path is halted, and we reject any incoming stream start requests.
How? If we go the reset_resume() code path, we find that usb-audio does not make a difference between regular resume() and reset_resume()
Note that, for USB audio, there is no much difference between resume() and reset_resume(), especially about the PCM stream handling that is the main target for the offload (the mixer isn't handled there). And for the PCM, we just set the power state for UAC3, and that's all. All the rest is handled by the PCM core handler as usual.
Takashi
Hi Oliver,
On 1/3/2023 4:49 AM, Takashi Iwai wrote:
On Tue, 03 Jan 2023 13:20:48 +0100, Oliver Neukum wrote:
On 30.12.22 08:10, Wesley Cheng wrote:
It may depend on how the offloading is implemented, but we do have a mechanism to force the audio stream off from the qc_usb_audio_offload. Regardless of if the UDEV is suspended first, or the USB backend, as long as we ensure that the offloading is disabled before entering suspend, I think that should be sufficient.
You would presumably output garbage, if the UDEV is asleep but the backend is not.
As long as the stream is halted, i.e. the audio DSP doesn't execute further transfers on the bus, there shouldn't be any noise/static that will continue to be outputted. When I mentioned that we have a mechanism to force for the offloading to be disabled to the audio DSP side, it will no longer submit any audio data to the USB bus.
The reset_resume() path is fine. Bus reset is going to cause a disconnect() callback in the offload driver, in which we already have the proper handling for ensuring the offload path is halted, and we reject any incoming stream start requests.
How? If we go the reset_resume() code path, we find that usb-audio does not make a difference between regular resume() and reset_resume()
Note that, for USB audio, there is no much difference between resume() and reset_resume(), especially about the PCM stream handling that is the main target for the offload (the mixer isn't handled there). And for the PCM, we just set the power state for UAC3, and that's all. All the rest is handled by the PCM core handler as usual.
Sorry, I was under the impression that the USB SND class driver didn't register a reset_resume() callback, which Takashi helped clarify that it does indeed do. (if no callback is registered, then USB interface is re-binded in the resume path) However, it doesn't explicitly treat the reset_resume differently than a normal resume.
For the offload path, we don't need to do anything special either - if we have ensured that the stream was stopped in the suspend path. (to be added) It would be up to the userspace to restart the ASoC PCM stream, which would cause another stream request enable QMI command handshake to happen before any transfers would start.
One thing that I could add to protect the situation where the USB ASoC backend is resumed before UDEV, is to check the chip->system_suspend state. Normally, the offload driver needs to ensure the bus is in U0 before starting the audio stream, but it is done using runtime PM:
static int enable_audio_stream(struct snd_usb_substream *subs, snd_pcm_format_t pcm_format, unsigned int channels, unsigned int cur_rate, int datainterval) { ... pm_runtime_barrier(&chip->intf[0]->dev); snd_usb_autoresume(chip);
In case we're in the PM resume path, I don't believe PM runtime can be triggered to resume a device. In this case, the snd_usb_autoresume() would return w/o ensuring the USB SND device is fully exited from PM suspend. Although, this situation would be a corner case, as userspace entities (userspace ALSA) are going to be unfrozen after kernel devices are resumed, so most likely there should be no request to enable the audio stream if kernel devices are still resuming.
I don't see an issue with the sequence of UDEV being resumed before USB backend. In this case, the USB bus is ready to go, and able to handle stream enable requests.
Thanks Wesley Cheng
Some vendor modules will utilize useful parsing and endpoint management APIs to start audio playback/capture.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com --- sound/usb/card.c | 2 ++ sound/usb/endpoint.c | 2 ++ sound/usb/helper.c | 1 + sound/usb/pcm.c | 9 ++++++--- sound/usb/pcm.h | 12 ++++++++++++ 5 files changed, 23 insertions(+), 3 deletions(-)
diff --git a/sound/usb/card.c b/sound/usb/card.c index 212f55a7683c..396e5a34e23b 100644 --- a/sound/usb/card.c +++ b/sound/usb/card.c @@ -1068,6 +1068,7 @@ int snd_usb_autoresume(struct snd_usb_audio *chip) } return 0; } +EXPORT_SYMBOL(snd_usb_autoresume);
void snd_usb_autosuspend(struct snd_usb_audio *chip) { @@ -1081,6 +1082,7 @@ void snd_usb_autosuspend(struct snd_usb_audio *chip) for (i = 0; i < chip->num_interfaces; i++) usb_autopm_put_interface(chip->intf[i]); } +EXPORT_SYMBOL(snd_usb_autosuspend);
static int usb_audio_suspend(struct usb_interface *intf, pm_message_t message) { diff --git a/sound/usb/endpoint.c b/sound/usb/endpoint.c index 310cd6fb0038..1663f9e9cb4b 100644 --- a/sound/usb/endpoint.c +++ b/sound/usb/endpoint.c @@ -858,6 +858,7 @@ snd_usb_endpoint_open(struct snd_usb_audio *chip, mutex_unlock(&chip->mutex); return ep; } +EXPORT_SYMBOL_GPL(snd_usb_endpoint_open);
/* * snd_usb_endpoint_set_sync: Link data and sync endpoints @@ -1525,6 +1526,7 @@ int snd_usb_endpoint_get_clock_rate(struct snd_usb_audio *chip, int clock) mutex_unlock(&chip->mutex); return rate; } +EXPORT_SYMBOL(snd_usb_endpoint_prepare);
/** * snd_usb_endpoint_start: start an snd_usb_endpoint diff --git a/sound/usb/helper.c b/sound/usb/helper.c index a4410267bf70..b4ed9ef3eeb3 100644 --- a/sound/usb/helper.c +++ b/sound/usb/helper.c @@ -62,6 +62,7 @@ void *snd_usb_find_csint_desc(void *buffer, int buflen, void *after, u8 dsubtype } return NULL; } +EXPORT_SYMBOL_GPL(snd_usb_find_csint_desc);
/* * Wrapper for usb_control_msg(). diff --git a/sound/usb/pcm.c b/sound/usb/pcm.c index 8ed165f036a0..624f555c6657 100644 --- a/sound/usb/pcm.c +++ b/sound/usb/pcm.c @@ -87,7 +87,7 @@ static snd_pcm_uframes_t snd_usb_pcm_pointer(struct snd_pcm_substream *substream /* * find a matching audio format */ -static const struct audioformat * +const struct audioformat * find_format(struct list_head *fmt_list_head, snd_pcm_format_t format, unsigned int rate, unsigned int channels, bool strict_match, struct snd_usb_substream *subs) @@ -147,8 +147,9 @@ find_format(struct list_head *fmt_list_head, snd_pcm_format_t format, } return found; } +EXPORT_SYMBOL(find_format);
-static const struct audioformat * +const struct audioformat * find_substream_format(struct snd_usb_substream *subs, const struct snd_pcm_hw_params *params) { @@ -156,6 +157,7 @@ find_substream_format(struct snd_usb_substream *subs, params_rate(params), params_channels(params), true, subs); } +EXPORT_SYMBOL(find_substream_format);
static int init_pitch_v1(struct snd_usb_audio *chip, int ep) { @@ -418,7 +420,7 @@ int snd_usb_pcm_resume(struct snd_usb_stream *as) return 0; }
-static void close_endpoints(struct snd_usb_audio *chip, +void close_endpoints(struct snd_usb_audio *chip, struct snd_usb_substream *subs) { if (subs->data_endpoint) { @@ -432,6 +434,7 @@ static void close_endpoints(struct snd_usb_audio *chip, subs->sync_endpoint = NULL; } } +EXPORT_SYMBOL(close_endpoints);
/* * hw_params callback diff --git a/sound/usb/pcm.h b/sound/usb/pcm.h index 493a4e34d78d..43a4a03dfce7 100644 --- a/sound/usb/pcm.h +++ b/sound/usb/pcm.h @@ -13,4 +13,16 @@ void snd_usb_preallocate_buffer(struct snd_usb_substream *subs); int snd_usb_audioformat_set_sync_ep(struct snd_usb_audio *chip, struct audioformat *fmt);
+void close_endpoints(struct snd_usb_audio *chip, + struct snd_usb_substream *subs); +int configure_endpoints(struct snd_usb_audio *chip, + struct snd_usb_substream *subs); + +const struct audioformat * +find_format(struct list_head *fmt_list_head, snd_pcm_format_t format, + unsigned int rate, unsigned int channels, bool strict_match, + struct snd_usb_substream *subs); +const struct audioformat * +find_substream_format(struct snd_usb_substream *subs, + const struct snd_pcm_hw_params *params); #endif /* __USBAUDIO_PCM_H */
On Fri, Dec 23, 2022 at 03:31:51PM -0800, Wesley Cheng wrote:
+EXPORT_SYMBOL(snd_usb_autoresume);
EXPORT_SYMBOL_GPL()? I have to ask...
Same for the other ones here.
+EXPORT_SYMBOL(find_format);
That is not a valid global symbol name to use, you know better than this...
thanks,
greg k-h
For USB HCDs that can support multiple USB interrupters, expose functions that class drivers can utilize for setting up secondary interrupters. Class drivers can pass this information to its respective clients, i.e. a dedicated DSP.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com --- drivers/usb/core/hcd.c | 86 +++++++++++++++++++++++++++++++++++++++++ include/linux/usb.h | 7 ++++ include/linux/usb/hcd.h | 16 +++++++- 3 files changed, 108 insertions(+), 1 deletion(-)
diff --git a/drivers/usb/core/hcd.c b/drivers/usb/core/hcd.c index 8300baedafd2..90ead90faf1d 100644 --- a/drivers/usb/core/hcd.c +++ b/drivers/usb/core/hcd.c @@ -1579,6 +1579,92 @@ int usb_hcd_submit_urb (struct urb *urb, gfp_t mem_flags) return status; }
+/** + * usb_free_interrupter - free an interrupter + * @udev: usb device which requested the interrupter + * @intr_num: interrupter number to free + * + * Release the USB HCD interrupter that was reserved by a USB class driver. + **/ +int usb_free_interrupter(struct usb_device *udev, int intr_num) +{ + struct usb_hcd *hcd = bus_to_hcd(udev->bus); + int ret = 0; + + if (hcd->driver->free_interrupter) + ret = hcd->driver->free_interrupter(hcd, intr_num); + + return ret; +} +EXPORT_SYMBOL_GPL(usb_free_interrupter); + +/** + * usb_set_interruper - Reserve an interrupter + * @udev: usb device which requested the interrupter + * @intr_num: interrupter number to reserve + * @dma: iova address to event ring + * + * Request for a specific interrupter to be reserved for a USB class driver. + * This will return the base address to the event ring that was allocated to + * the specific interrupter. + **/ +phys_addr_t usb_set_interruper(struct usb_device *udev, int intr_num, + dma_addr_t *dma) +{ + struct usb_hcd *hcd; + phys_addr_t pa = 0; + + hcd = bus_to_hcd(udev->bus); + if (hcd->driver->update_interrupter) + pa = hcd->driver->update_interrupter(hcd, intr_num, dma); + + return pa; +} +EXPORT_SYMBOL_GPL(usb_set_interruper); + +/** + * usb_hcd_get_transfer_resource - Retrieve USB EP resource information + * @udev: usb device + * @ep: usb ep to retrieve resource information + * @dma: iova address to transfer resource + * + * Request for USB EP transfer resource which is used for submitting + * transfers to the USB HCD. + **/ +phys_addr_t usb_hcd_get_transfer_resource(struct usb_device *udev, + struct usb_host_endpoint *ep, dma_addr_t *dma) +{ + struct usb_hcd *hcd; + phys_addr_t pa = 0; + + hcd = bus_to_hcd(udev->bus); + if (hcd->driver->get_transfer_resource) + pa = hcd->driver->get_transfer_resource(udev, ep, dma); + + return pa; +} +EXPORT_SYMBOL_GPL(usb_hcd_get_transfer_resource); + +/** + * usb_hcd_stop_endpoint - Halt USB EP transfers + * @udev: usb device + * @ep: usb ep to stop + * + * Stop pending transfers on a specific USB endpoint. + **/ +int usb_hcd_stop_endpoint(struct usb_device *udev, + struct usb_host_endpoint *ep) +{ + struct usb_hcd *hcd = bus_to_hcd(udev->bus); + int ret = 0; + + if (hcd->driver->stop_endpoint) + ret = hcd->driver->stop_endpoint(hcd, udev, ep); + + return ret; +} +EXPORT_SYMBOL_GPL(usb_hcd_stop_endpoint); + /*-------------------------------------------------------------------------*/
/* this makes the hcd giveback() the urb more quickly, by kicking it diff --git a/include/linux/usb.h b/include/linux/usb.h index d4afeeec1e1a..2f71cd4fb6e0 100644 --- a/include/linux/usb.h +++ b/include/linux/usb.h @@ -1724,6 +1724,13 @@ static inline void usb_fill_int_urb(struct urb *urb, urb->start_frame = -1; }
+extern int usb_free_interrupter(struct usb_device *udev, int intr_num); +extern phys_addr_t usb_set_interruper(struct usb_device *udev, + int intr_num, dma_addr_t *dma); +extern int usb_hcd_stop_endpoint(struct usb_device *udev, + struct usb_host_endpoint *ep); +extern phys_addr_t usb_hcd_get_transfer_resource(struct usb_device *udev, + struct usb_host_endpoint *ep, dma_addr_t *dma); extern void usb_init_urb(struct urb *urb); extern struct urb *usb_alloc_urb(int iso_packets, gfp_t mem_flags); extern void usb_free_urb(struct urb *urb); diff --git a/include/linux/usb/hcd.h b/include/linux/usb/hcd.h index b51c07111729..f5bce80b2e40 100644 --- a/include/linux/usb/hcd.h +++ b/include/linux/usb/hcd.h @@ -343,7 +343,21 @@ struct hc_driver { int (*free_streams)(struct usb_hcd *hcd, struct usb_device *udev, struct usb_host_endpoint **eps, unsigned int num_eps, gfp_t mem_flags); - + /* Frees an interrupter from the current client, and makes it available + * for use. + */ + int (*free_interrupter)(struct usb_hcd *hcd, int intr_num); + /* Request an interrupter from the current allocated pool. Will provide + * the address to the allocated ring. + */ + phys_addr_t (*update_interrupter)(struct usb_hcd *hcd, int intr_num, + dma_addr_t *dma); + /* Fetch transfer/ep ring address */ + phys_addr_t (*get_transfer_resource)(struct usb_device *udev, + struct usb_host_endpoint *ep, dma_addr_t *dma); + /* Stop transfers on a particular endpoint */ + int (*stop_endpoint)(struct usb_hcd *hcd, struct usb_device *udev, + struct usb_host_endpoint *ep); /* Bandwidth computation functions */ /* Note that add_endpoint() can only be called once per endpoint before * check_bandwidth() or reset_bandwidth() must be called.
On Fri, Dec 23, 2022 at 03:31:52PM -0800, Wesley Cheng wrote:
For USB HCDs that can support multiple USB interrupters, expose functions that class drivers can utilize for setting up secondary interrupters. Class drivers can pass this information to its respective clients, i.e. a dedicated DSP.
Where is the locking here that seems to be required when a hcd is removed from the system and you have data in flight? What am I missing here in the design of this?
And yes, HCDs get removed all the time, and will happen more and more in the future with the design of more systems moving to Thunderbolt/PCIe designs due to the simplicity of it.
+/**
- usb_set_interruper - Reserve an interrupter
Where is an "interrupter" defined? I don't know what this term means sorry, is this in the USB specification somewhere?
- @udev: usb device which requested the interrupter
- @intr_num: interrupter number to reserve
- @dma: iova address to event ring
- Request for a specific interrupter to be reserved for a USB class driver.
- This will return the base address to the event ring that was allocated to
- the specific interrupter.
- **/
+phys_addr_t usb_set_interruper(struct usb_device *udev, int intr_num,
dma_addr_t *dma)
+{
- struct usb_hcd *hcd;
- phys_addr_t pa = 0;
- hcd = bus_to_hcd(udev->bus);
- if (hcd->driver->update_interrupter)
pa = hcd->driver->update_interrupter(hcd, intr_num, dma);
- return pa;
Wait, you return a physical address? What are you going to do with that? And what guarantees that the address is valid after you return it (again, remember memory and devices can be removed at any point in time.
thanks,
greg k-h
Hi Greg,
On 12/24/2022 12:54 AM, Greg KH wrote:
On Fri, Dec 23, 2022 at 03:31:52PM -0800, Wesley Cheng wrote:
For USB HCDs that can support multiple USB interrupters, expose functions that class drivers can utilize for setting up secondary interrupters. Class drivers can pass this information to its respective clients, i.e. a dedicated DSP.
Where is the locking here that seems to be required when a hcd is removed from the system and you have data in flight? What am I missing here in the design of this?
The XHCI driver is the one that maintains the list of interrupters that are available, so the locking was placed in the XHCI driver versus adding it in the core hcd layer.
And yes, HCDs get removed all the time, and will happen more and more in the future with the design of more systems moving to Thunderbolt/PCIe designs due to the simplicity of it.
As part of the HCD removal, it has to first ensure that class driver interfaces, and the connected udevs are removed first. qc_audio_offload will first handle the transfer cleanup and stopping of the audio stream before returning from the disconnect callback. (this includes ensuring that the interrupter is released)
This concept is how all usb class drivers are currently implemented. When the HCD remove occurs, the class drivers are the ones responsible for ensuring that all URBs are stopped, and removed before it returns from its respective disconnect callback.
+/**
- usb_set_interruper - Reserve an interrupter
Where is an "interrupter" defined? I don't know what this term means sorry, is this in the USB specification somewhere?
Interrupter is defined in the XHCI spec, refer to "section 4.17 Interrupters"
- @udev: usb device which requested the interrupter
- @intr_num: interrupter number to reserve
- @dma: iova address to event ring
- Request for a specific interrupter to be reserved for a USB class driver.
- This will return the base address to the event ring that was allocated to
- the specific interrupter.
- **/
+phys_addr_t usb_set_interruper(struct usb_device *udev, int intr_num,
dma_addr_t *dma)
+{
- struct usb_hcd *hcd;
- phys_addr_t pa = 0;
- hcd = bus_to_hcd(udev->bus);
- if (hcd->driver->update_interrupter)
pa = hcd->driver->update_interrupter(hcd, intr_num, dma);
- return pa;
Wait, you return a physical address? What are you going to do with that? And what guarantees that the address is valid after you return it (again, remember memory and devices can be removed at any point in time.
The interrupter is basically another event ring which is the buffer allocated for the controller to copy events into. Since the audio dsp now takes over handling of the endpoint events, then it needs to know where the buffer resides. Will fix the interruper typo as well.
The allocation and freeing of this event ring follows how XHCI allocates and frees the main event ring as well. This API just reserves the interrupter for the class driver, and return the previously allocated (during XHCI init) memory address.
Thanks Wesley Cheng
On Fri, Dec 23, 2022 at 03:31:52PM -0800, Wesley Cheng wrote:
For USB HCDs that can support multiple USB interrupters, expose functions that class drivers can utilize for setting up secondary interrupters. Class drivers can pass this information to its respective clients, i.e. a dedicated DSP.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com
drivers/usb/core/hcd.c | 86 +++++++++++++++++++++++++++++++++++++++++ include/linux/usb.h | 7 ++++ include/linux/usb/hcd.h | 16 +++++++- 3 files changed, 108 insertions(+), 1 deletion(-)
diff --git a/drivers/usb/core/hcd.c b/drivers/usb/core/hcd.c index 8300baedafd2..90ead90faf1d 100644 --- a/drivers/usb/core/hcd.c +++ b/drivers/usb/core/hcd.c
+/**
- usb_hcd_stop_endpoint - Halt USB EP transfers
- @udev: usb device
- @ep: usb ep to stop
- Stop pending transfers on a specific USB endpoint.
- **/
+int usb_hcd_stop_endpoint(struct usb_device *udev,
struct usb_host_endpoint *ep)
+{
- struct usb_hcd *hcd = bus_to_hcd(udev->bus);
- int ret = 0;
- if (hcd->driver->stop_endpoint)
ret = hcd->driver->stop_endpoint(hcd, udev, ep);
- return ret;
+} +EXPORT_SYMBOL_GPL(usb_hcd_stop_endpoint);
You know, there already is a function that does this. It's named usb_hcd_flush_endpoint(). No need to add another function that does the same thing.
Alan Stern
Hi Alan,
On 12/24/2022 7:29 AM, Alan Stern wrote:
On Fri, Dec 23, 2022 at 03:31:52PM -0800, Wesley Cheng wrote:
For USB HCDs that can support multiple USB interrupters, expose functions that class drivers can utilize for setting up secondary interrupters. Class drivers can pass this information to its respective clients, i.e. a dedicated DSP.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com
drivers/usb/core/hcd.c | 86 +++++++++++++++++++++++++++++++++++++++++ include/linux/usb.h | 7 ++++ include/linux/usb/hcd.h | 16 +++++++- 3 files changed, 108 insertions(+), 1 deletion(-)
diff --git a/drivers/usb/core/hcd.c b/drivers/usb/core/hcd.c index 8300baedafd2..90ead90faf1d 100644 --- a/drivers/usb/core/hcd.c +++ b/drivers/usb/core/hcd.c
+/**
- usb_hcd_stop_endpoint - Halt USB EP transfers
- @udev: usb device
- @ep: usb ep to stop
- Stop pending transfers on a specific USB endpoint.
- **/
+int usb_hcd_stop_endpoint(struct usb_device *udev,
struct usb_host_endpoint *ep)
+{
- struct usb_hcd *hcd = bus_to_hcd(udev->bus);
- int ret = 0;
- if (hcd->driver->stop_endpoint)
ret = hcd->driver->stop_endpoint(hcd, udev, ep);
- return ret;
+} +EXPORT_SYMBOL_GPL(usb_hcd_stop_endpoint);
You know, there already is a function that does this. It's named usb_hcd_flush_endpoint(). No need to add another function that does the same thing.
Thanks for the suggestion and review.
Hmmm...maybe I should change the name of the API then to avoid the confusion. Yes, usb_hcd_flush_endpoint() does ensure that URBs submitted to the EP are stopped. However, with this offloading concept, we aren't actually submitting URBs from the main processor, so the ep->urb_list will be empty.
This means the usb_hcd_flush_endpoint() API won't actually do anything. What we need is to ensure that we send a XHCI stop ep command to the controller.
Thanks Wesley Cheng
On 27.12.22 22:07, Wesley Cheng wrote:
Hmmm...maybe I should change the name of the API then to avoid the confusion. Yes, usb_hcd_flush_endpoint() does ensure that URBs submitted to the EP are stopped. However, with this offloading concept, we aren't actually submitting URBs from the main processor, so the ep->urb_list will be empty.
This means the usb_hcd_flush_endpoint() API won't actually do anything. What we need is to ensure that we send a XHCI stop ep command to the controller.
That is a concept specific to XHCI, yet you are adding a generic API. The namin should reflect that. usb_quiesce_endpoint() ?
Regards Oliver
On Wed, Dec 28, 2022 at 09:59:03AM +0100, Oliver Neukum wrote:
On 27.12.22 22:07, Wesley Cheng wrote:
Hmmm...maybe I should change the name of the API then to avoid the confusion. Yes, usb_hcd_flush_endpoint() does ensure that URBs submitted to the EP are stopped. However, with this offloading concept, we aren't actually submitting URBs from the main processor, so the ep->urb_list will be empty.
This means the usb_hcd_flush_endpoint() API won't actually do anything. What we need is to ensure that we send a XHCI stop ep command to the controller.
That is a concept specific to XHCI, yet you are adding a generic API. The namin should reflect that. usb_quiesce_endpoint() ?
Or even xhci_send_stop_ep_cmd(), which is what the routine is intended to do.
Alan Stern
Hi Alan,
On 12/28/2022 7:16 AM, Alan Stern wrote:
On Wed, Dec 28, 2022 at 09:59:03AM +0100, Oliver Neukum wrote:
On 27.12.22 22:07, Wesley Cheng wrote:
Hmmm...maybe I should change the name of the API then to avoid the confusion. Yes, usb_hcd_flush_endpoint() does ensure that URBs submitted to the EP are stopped. However, with this offloading concept, we aren't actually submitting URBs from the main processor, so the ep->urb_list will be empty.
This means the usb_hcd_flush_endpoint() API won't actually do anything. What we need is to ensure that we send a XHCI stop ep command to the controller.
That is a concept specific to XHCI, yet you are adding a generic API. The namin should reflect that. usb_quiesce_endpoint() ?
Or even xhci_send_stop_ep_cmd(), which is what the routine is intended to do.
Just to clarify, you're talking about renaming the API that was added in the XHCI driver, correct?
Thanks Wesley Cheng
On Wed, Dec 28, 2022 at 12:31:16PM -0800, Wesley Cheng wrote:
Hi Alan,
On 12/28/2022 7:16 AM, Alan Stern wrote:
On Wed, Dec 28, 2022 at 09:59:03AM +0100, Oliver Neukum wrote:
On 27.12.22 22:07, Wesley Cheng wrote:
Hmmm...maybe I should change the name of the API then to avoid the confusion. Yes, usb_hcd_flush_endpoint() does ensure that URBs submitted to the EP are stopped. However, with this offloading concept, we aren't actually submitting URBs from the main processor, so the ep->urb_list will be empty.
This means the usb_hcd_flush_endpoint() API won't actually do anything. What we need is to ensure that we send a XHCI stop ep command to the controller.
That is a concept specific to XHCI, yet you are adding a generic API. The namin should reflect that. usb_quiesce_endpoint() ?
Or even xhci_send_stop_ep_cmd(), which is what the routine is intended to do.
Just to clarify, you're talking about renaming the API that was added in the XHCI driver, correct?
To be precise, we're talking about renaming your usb_hcd_stop_endpoint() function, although similar arguments probably apply to your usb_free_interrupter(), usb_set_interrupter(), and usb_hcd_get_transfer_resource() routines.
You wrote earlier:
The XHCI driver is the one that maintains the list of interrupters that are available, so the locking was placed in the XHCI driver versus adding it in the core hcd layer.
The "stop ep" functionality and other interrupter management things you want to add seem a lot like this locking stuff. Since you decided to put the locking in the xhci-hcd driver instead of the core HCD layer, it would be logical to do the same with the "stop ep" and other routines. Which means there shouldn't be any need to make changes to hcd.c or include/linux/usb/hcd.h.
Alan Stern
Implement the XHCI operations for allocating and requesting for a secondary interrupter. The secondary interrupter can allow for events for a particular endpoint to be routed to a separate event ring. The event routing is defined when submitting a transfer descriptor to the USB HW. There is a specific field which denotes which interrupter ring to route the event to when the transfer is completed.
An example use case, such as audio packet offloading can utilize a separate event ring, so that these events can be routed to a different processor within the system. The processor would be able to independently submit transfers and handle its completions without intervention from the main processor.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com --- drivers/usb/host/xhci-mem.c | 219 ++++++++++++++++++++++++++++------- drivers/usb/host/xhci-plat.c | 2 + drivers/usb/host/xhci.c | 169 ++++++++++++++++++++++++++- drivers/usb/host/xhci.h | 15 +++ 4 files changed, 363 insertions(+), 42 deletions(-)
diff --git a/drivers/usb/host/xhci-mem.c b/drivers/usb/host/xhci-mem.c index 81ca2bc1f0be..d5cb4b82ad3d 100644 --- a/drivers/usb/host/xhci-mem.c +++ b/drivers/usb/host/xhci-mem.c @@ -1835,6 +1835,7 @@ void xhci_free_erst(struct xhci_hcd *xhci, struct xhci_erst *erst) void xhci_mem_cleanup(struct xhci_hcd *xhci) { struct device *dev = xhci_to_hcd(xhci)->self.sysdev; + struct xhci_sec *sec, *tmp; int i, j, num_ports;
cancel_delayed_work_sync(&xhci->cmd_timer); @@ -1846,6 +1847,16 @@ void xhci_mem_cleanup(struct xhci_hcd *xhci) xhci->event_ring = NULL; xhci_dbg_trace(xhci, trace_xhci_dbg_init, "Freed event ring");
+ list_for_each_entry_safe(sec, tmp, &xhci->xhci_sec_list, list) { + list_del(&sec->list); + if (sec->event_ring) { + xhci_ring_free(xhci, sec->event_ring); + xhci_dbg_trace(xhci, trace_xhci_dbg_init, + "Freed secondary ring %d", sec->intr_num); + } + kfree(sec); + } + if (xhci->cmd_ring) xhci_ring_free(xhci, xhci->cmd_ring); xhci->cmd_ring = NULL; @@ -2087,18 +2098,18 @@ static int xhci_check_trb_in_td_math(struct xhci_hcd *xhci) return 0; }
-static void xhci_set_hc_event_deq(struct xhci_hcd *xhci) +static void xhci_set_hc_event_deq(struct xhci_hcd *xhci, struct xhci_ring *er, + struct xhci_intr_reg __iomem *ir_set) { u64 temp; dma_addr_t deq;
- deq = xhci_trb_virt_to_dma(xhci->event_ring->deq_seg, - xhci->event_ring->dequeue); + deq = xhci_trb_virt_to_dma(er->deq_seg, er->dequeue); if (!deq) xhci_warn(xhci, "WARN something wrong with SW event ring " "dequeue ptr.\n"); /* Update HC event ring dequeue pointer */ - temp = xhci_read_64(xhci, &xhci->ir_set->erst_dequeue); + temp = xhci_read_64(xhci, &ir_set->erst_dequeue); temp &= ERST_PTR_MASK; /* Don't clear the EHB bit (which is RW1C) because * there might be more events to service. @@ -2108,7 +2119,7 @@ static void xhci_set_hc_event_deq(struct xhci_hcd *xhci) "// Write event ring dequeue pointer, " "preserving EHB bit"); xhci_write_64(xhci, ((u64) deq & (u64) ~ERST_PTR_MASK) | temp, - &xhci->ir_set->erst_dequeue); + &ir_set->erst_dequeue); }
static void xhci_add_in_port(struct xhci_hcd *xhci, unsigned int num_ports, @@ -2375,10 +2386,159 @@ static int xhci_setup_port_arrays(struct xhci_hcd *xhci, gfp_t flags) return 0; }
+static void xhci_reset_evt_ring_base(struct xhci_hcd *xhci, struct xhci_ring *er, + struct xhci_erst *erst, struct xhci_intr_reg __iomem *ir_set) +{ + u64 val_64; + + xhci_dbg_trace(xhci, trace_xhci_dbg_init, + "// Set ERST entries to point to event ring."); + /* set the segment table base address */ + xhci_dbg_trace(xhci, trace_xhci_dbg_init, + "// Set ERST base address for ir_set 0 = 0x%llx", + (unsigned long long)erst->erst_dma_addr); + val_64 = xhci_read_64(xhci, &ir_set->erst_base); + val_64 &= ERST_PTR_MASK; + pr_err("after clearing ptr = 0x%llx\n", val_64); + val_64 |= (erst->erst_dma_addr & (u64) ~ERST_PTR_MASK); + pr_err("after clearing ptr = 0x%llx\n", val_64); + + xhci_write_64(xhci, val_64, &ir_set->erst_base); + xhci_dbg_trace(xhci, trace_xhci_dbg_init, + "done."); + /* Set the event ring dequeue address */ + xhci_set_hc_event_deq(xhci, er, ir_set); + xhci_dbg_trace(xhci, trace_xhci_dbg_init, + "Wrote ERST address to ir_set 0."); +} + +void xhci_handle_sec_intr_events(struct xhci_hcd *xhci, struct xhci_ring *ring, + struct xhci_intr_reg __iomem *ir_set, struct xhci_sec *sec) +{ + union xhci_trb *erdp_trb, *current_trb; + struct xhci_segment *seg; + u64 erdp_reg; + u32 iman_reg; + dma_addr_t deq; + unsigned long segment_offset; + struct xhci_erst *erst = &sec->erst; + + /* disable irq, ack pending interrupt and ack all pending events */ + + iman_reg = readl_relaxed(&ir_set->irq_pending); + iman_reg &= ~IMAN_IE; + writel_relaxed(iman_reg, &ir_set->irq_pending); + iman_reg = readl_relaxed(&ir_set->irq_pending); + if (iman_reg & IMAN_IP) + writel_relaxed(iman_reg, &ir_set->irq_pending); + + /* last acked event trb is in erdp reg */ + erdp_reg = xhci_read_64(xhci, &ir_set->erst_dequeue); + deq = (dma_addr_t)(erdp_reg & ~ERST_PTR_MASK); + if (!deq) { + pr_debug("%s: event ring handling not required\n", __func__); + return; + } + + seg = ring->first_seg; + segment_offset = deq - seg->dma; + + /* find out virtual address of the last acked event trb */ + erdp_trb = current_trb = &seg->trbs[0] + + (segment_offset/sizeof(*current_trb)); + + /* read cycle state of the last acked trb to find out CCS */ + ring->cycle_state = le32_to_cpu(current_trb->event_cmd.flags) & TRB_CYCLE; + + while (1) { + /* last trb of the event ring: toggle cycle state */ + if (current_trb == &seg->trbs[TRBS_PER_SEGMENT - 1]) { + ring->cycle_state ^= 1; + current_trb = &seg->trbs[0]; + } else { + current_trb++; + } + + /* cycle state transition */ + if ((le32_to_cpu(current_trb->event_cmd.flags) & TRB_CYCLE) != + ring->cycle_state) + break; + } + + if (erdp_trb != current_trb) { + deq = xhci_trb_virt_to_dma(ring->deq_seg, current_trb); + if (deq == 0) + xhci_warn(xhci, + "WARN invalid SW event ring dequeue ptr.\n"); + /* Update HC event ring dequeue pointer */ + erdp_reg &= ERST_PTR_MASK; + erdp_reg |= ((u64) deq & (u64) ~ERST_PTR_MASK); + } + + /* Clear the event handler busy flag (RW1C); event ring is empty. */ + erdp_reg |= ERST_EHB; + xhci_write_64(xhci, erdp_reg, &ir_set->erst_dequeue); + + xhci_reset_evt_ring_base(xhci, ring, erst, ir_set); +} + +static int xhci_event_ring_setup(struct xhci_hcd *xhci, struct xhci_ring **er, + struct xhci_erst *erst, struct xhci_intr_reg __iomem *ir_set, + unsigned int intr_num, gfp_t flags) +{ + int ret; + unsigned int val; + + /* + * Event ring setup: Allocate a normal ring, but also setup + * the event ring segment table (ERST). Section 4.9.3. + */ + xhci_dbg_trace(xhci, trace_xhci_dbg_init, "// Allocating event ring"); + *er = xhci_ring_alloc(xhci, ERST_NUM_SEGS, 1, TYPE_EVENT, + 0, flags); + if (!*er) + return -ENOMEM; + + ret = xhci_alloc_erst(xhci, *er, erst, flags); + if (ret) + return ret; + + /* set ERST count with the number of entries in the segment table */ + val = readl(&ir_set->erst_size); + val &= ERST_SIZE_MASK; + val |= ERST_NUM_SEGS; + xhci_dbg_trace(xhci, trace_xhci_dbg_init, + "// Write ERST size = %i to ir_set 0 (some bits preserved)", + val); + writel(val, &ir_set->erst_size); + + xhci_reset_evt_ring_base(xhci, *er, erst, ir_set); + + return 0; +} + +int xhci_mem_sec_init(struct xhci_hcd *xhci, struct xhci_sec *sec) +{ + int ret; + + sec->ir_set = &xhci->run_regs->ir_set[sec->intr_num]; + ret = xhci_event_ring_setup(xhci, &sec->event_ring, + &sec->erst, sec->ir_set, sec->intr_num, GFP_KERNEL); + if (ret) { + xhci_err(xhci, "sec event ring setup failed inter#%d\n", + sec->intr_num); + return ret; + } + sec->xhci = 0; + + return 0; +} + int xhci_mem_init(struct xhci_hcd *xhci, gfp_t flags) { dma_addr_t dma; struct device *dev = xhci_to_hcd(xhci)->self.sysdev; + struct xhci_sec *sec; unsigned int val, val2; u64 val_64; u32 page_size, temp; @@ -2497,46 +2657,23 @@ int xhci_mem_init(struct xhci_hcd *xhci, gfp_t flags) /* Set ir_set to interrupt register set 0 */ xhci->ir_set = &xhci->run_regs->ir_set[0];
- /* - * Event ring setup: Allocate a normal ring, but also setup - * the event ring segment table (ERST). Section 4.9.3. - */ - xhci_dbg_trace(xhci, trace_xhci_dbg_init, "// Allocating event ring"); - xhci->event_ring = xhci_ring_alloc(xhci, ERST_NUM_SEGS, 1, TYPE_EVENT, - 0, flags); - if (!xhci->event_ring) - goto fail; - if (xhci_check_trb_in_td_math(xhci) < 0) + ret = xhci_event_ring_setup(xhci, &xhci->event_ring, &xhci->erst, xhci->ir_set, 0, flags); + if (ret < 0) goto fail;
- ret = xhci_alloc_erst(xhci, xhci->event_ring, &xhci->erst, flags); - if (ret) + if (xhci_check_trb_in_td_math(xhci) < 0) goto fail;
- /* set ERST count with the number of entries in the segment table */ - val = readl(&xhci->ir_set->erst_size); - val &= ERST_SIZE_MASK; - val |= ERST_NUM_SEGS; - xhci_dbg_trace(xhci, trace_xhci_dbg_init, - "// Write ERST size = %i to ir_set 0 (some bits preserved)", - val); - writel(val, &xhci->ir_set->erst_size); - - xhci_dbg_trace(xhci, trace_xhci_dbg_init, - "// Set ERST entries to point to event ring."); - /* set the segment table base address */ - xhci_dbg_trace(xhci, trace_xhci_dbg_init, - "// Set ERST base address for ir_set 0 = 0x%llx", - (unsigned long long)xhci->erst.erst_dma_addr); - val_64 = xhci_read_64(xhci, &xhci->ir_set->erst_base); - val_64 &= ERST_PTR_MASK; - val_64 |= (xhci->erst.erst_dma_addr & (u64) ~ERST_PTR_MASK); - xhci_write_64(xhci, val_64, &xhci->ir_set->erst_base); - - /* Set the event ring dequeue address */ - xhci_set_hc_event_deq(xhci); - xhci_dbg_trace(xhci, trace_xhci_dbg_init, - "Wrote ERST address to ir_set 0."); + /* Setup secondary interrupters (if any) */ + INIT_LIST_HEAD(&xhci->xhci_sec_list); + for (i = 0; i < xhci->max_interrupters; i++) { + sec = kzalloc(sizeof(struct xhci_sec), GFP_KERNEL); + if (sec) { + sec->intr_num = i + 1; + xhci_mem_sec_init(xhci, sec); + list_add_tail(&sec->list, &xhci->xhci_sec_list); + } + }
xhci->isoc_bei_interval = AVOID_BEI_INTERVAL_MAX;
diff --git a/drivers/usb/host/xhci-plat.c b/drivers/usb/host/xhci-plat.c index 5fb55bf19493..a1b6c17ecf74 100644 --- a/drivers/usb/host/xhci-plat.c +++ b/drivers/usb/host/xhci-plat.c @@ -303,6 +303,8 @@ static int xhci_plat_probe(struct platform_device *pdev)
device_property_read_u32(tmpdev, "imod-interval-ns", &xhci->imod_interval); + device_property_read_u8(tmpdev, "num-hc-interrupters", + &xhci->max_interrupters); }
hcd->usb_phy = devm_usb_get_phy_by_phandle(sysdev, "usb-phy", 0); diff --git a/drivers/usb/host/xhci.c b/drivers/usb/host/xhci.c index 79d7931c048a..353b06df2000 100644 --- a/drivers/usb/host/xhci.c +++ b/drivers/usb/host/xhci.c @@ -2101,6 +2101,64 @@ int xhci_add_endpoint(struct usb_hcd *hcd, struct usb_device *udev, } EXPORT_SYMBOL_GPL(xhci_add_endpoint);
+static int xhci_stop_endpoint(struct usb_hcd *hcd, struct usb_device *udev, + struct usb_host_endpoint *ep) +{ + struct xhci_hcd *xhci = hcd_to_xhci(hcd); + unsigned int ep_index; + struct xhci_virt_device *virt_dev; + struct xhci_command *cmd; + unsigned long flags; + int ret = 0; + + ret = xhci_check_args(hcd, udev, ep, 1, true, __func__); + if (ret <= 0) + return ret; + + cmd = xhci_alloc_command(xhci, true, GFP_NOIO); + if (!cmd) + return -ENOMEM; + + spin_lock_irqsave(&xhci->lock, flags); + virt_dev = xhci->devs[udev->slot_id]; + if (!virt_dev) { + ret = -ENODEV; + goto err; + } + + ep_index = xhci_get_endpoint_index(&ep->desc); + if (virt_dev->eps[ep_index].ring && + virt_dev->eps[ep_index].ring->dequeue) { + ret = xhci_queue_stop_endpoint(xhci, cmd, udev->slot_id, + ep_index, 0); + if (ret) + goto err; + + xhci_ring_cmd_db(xhci); + spin_unlock_irqrestore(&xhci->lock, flags); + + /* Wait for stop endpoint command to finish */ + wait_for_completion(cmd->completion); + + if (cmd->status == COMP_COMMAND_ABORTED || + cmd->status == COMP_STOPPED) { + xhci_warn(xhci, + "stop endpoint command timeout for ep%d%s\n", + usb_endpoint_num(&ep->desc), + usb_endpoint_dir_in(&ep->desc) ? "in" : "out"); + ret = -ETIME; + } + goto free_cmd; + } + +err: + spin_unlock_irqrestore(&xhci->lock, flags); +free_cmd: + xhci_free_command(xhci, cmd); + + return ret; +} + static void xhci_zero_in_ctx(struct xhci_hcd *xhci, struct xhci_virt_device *virt_dev) { struct xhci_input_control_ctx *ctrl_ctx; @@ -5278,6 +5336,110 @@ static void xhci_hcd_init_usb3_data(struct xhci_hcd *xhci, struct usb_hcd *hcd) xhci->usb3_rhub.hcd = hcd; }
+/* + * Free a XHCI interrupter that was previously allocated. Ensure that the + * event ring which was used is reset to the proper state, and recycle the + * interrupter for next use by clearing the XHCI reference. + */ +static int xhci_free_interrupter(struct usb_hcd *hcd, int intr_num) +{ + struct xhci_hcd *xhci = hcd_to_xhci(hcd); + struct xhci_sec *sec; + + mutex_lock(&xhci->mutex); + list_for_each_entry(sec, &xhci->xhci_sec_list, list) { + if (sec->intr_num == intr_num) { + xhci_handle_sec_intr_events(xhci, sec->event_ring, sec->ir_set, sec); + sec->xhci = 0; + } + } + mutex_unlock(&xhci->mutex); + + return 0; +} + +/* + * Reserve a XHCI interrupter, and pass the base address of the event ring for + * this pariticular interrupter back to the client. + */ +static phys_addr_t xhci_update_interrupter(struct usb_hcd *hcd, int intr_num, + dma_addr_t *dma) +{ + struct xhci_hcd *xhci = hcd_to_xhci(hcd); + struct device *dev = hcd->self.sysdev; + struct sg_table sgt; + phys_addr_t pa; + struct xhci_sec *sec; + + if (!HCD_RH_RUNNING(hcd) || + (xhci->xhc_state & XHCI_STATE_HALTED)) + return 0; + + mutex_lock(&xhci->mutex); + list_for_each_entry(sec, &xhci->xhci_sec_list, list) { + if (sec->intr_num == intr_num) { + dma_get_sgtable(dev, &sgt, sec->event_ring->first_seg->trbs, + sec->event_ring->first_seg->dma, TRB_SEGMENT_SIZE); + + *dma = sec->event_ring->first_seg->dma; + + pa = page_to_phys(sg_page(sgt.sgl)); + sg_free_table(&sgt); + sec->xhci = xhci; + mutex_unlock(&xhci->mutex); + + return pa; + } + } + mutex_unlock(&xhci->mutex); + + return 0; +} + +/* Retrieve the transfer ring base address for a specific endpoint. */ +static phys_addr_t xhci_get_xfer_resource(struct usb_device *udev, + struct usb_host_endpoint *ep, dma_addr_t *dma) +{ + struct usb_hcd *hcd = bus_to_hcd(udev->bus); + struct device *dev = hcd->self.sysdev; + struct xhci_hcd *xhci = hcd_to_xhci(hcd); + struct sg_table sgt; + phys_addr_t pa; + int ret; + unsigned int ep_index; + struct xhci_virt_device *virt_dev; + + if (!HCD_RH_RUNNING(hcd)) + return 0; + + ret = xhci_check_args(hcd, udev, ep, 1, true, __func__); + if (ret <= 0) { + xhci_err(xhci, "%s: invalid args\n", __func__); + return 0; + } + + virt_dev = xhci->devs[udev->slot_id]; + ep_index = xhci_get_endpoint_index(&ep->desc); + + if (virt_dev->eps[ep_index].ring && + virt_dev->eps[ep_index].ring->first_seg) { + + dma_get_sgtable(dev, &sgt, + virt_dev->eps[ep_index].ring->first_seg->trbs, + virt_dev->eps[ep_index].ring->first_seg->dma, + TRB_SEGMENT_SIZE); + + *dma = virt_dev->eps[ep_index].ring->first_seg->dma; + + pa = page_to_phys(sg_page(sgt.sgl)); + sg_free_table(&sgt); + + return pa; + } + + return 0; +} + int xhci_gen_setup(struct usb_hcd *hcd, xhci_get_quirks_t get_quirks) { struct xhci_hcd *xhci; @@ -5321,6 +5483,8 @@ int xhci_gen_setup(struct usb_hcd *hcd, xhci_get_quirks_t get_quirks) xhci->hcc_params2 = readl(&xhci->cap_regs->hcc_params2);
xhci->quirks |= quirks; + xhci->max_interrupters = min_t(u32, HCS_MAX_INTRS(xhci->hcs_params1), + xhci->max_interrupters);
get_quirks(dev, xhci);
@@ -5454,7 +5618,10 @@ static const struct hc_driver xhci_hc_driver = { .enable_device = xhci_enable_device, .update_hub_device = xhci_update_hub_device, .reset_device = xhci_discover_or_reset_device, - + .update_interrupter = xhci_update_interrupter, + .free_interrupter = xhci_free_interrupter, + .get_transfer_resource = xhci_get_xfer_resource, + .stop_endpoint = xhci_stop_endpoint, /* * scheduling support */ diff --git a/drivers/usb/host/xhci.h b/drivers/usb/host/xhci.h index c9f06c5e4e9d..2e694686c849 100644 --- a/drivers/usb/host/xhci.h +++ b/drivers/usb/host/xhci.h @@ -1926,6 +1926,7 @@ struct xhci_hcd { struct dentry *debugfs_root; struct dentry *debugfs_slots; struct list_head regset_list; + struct list_head xhci_sec_list;
void *dbc; /* platform-specific data -- must come last */ @@ -1945,6 +1946,17 @@ struct xhci_driver_overrides { void (*reset_bandwidth)(struct usb_hcd *, struct usb_device *); };
+struct xhci_sec { + struct xhci_ring *event_ring; + struct xhci_erst erst; + /* secondary interrupter */ + struct xhci_intr_reg __iomem *ir_set; + struct xhci_hcd *xhci; + int intr_num; + + struct list_head list; +}; + #define XHCI_CFC_DELAY 10
/* convert between an HCD pointer and the corresponding EHCI_HCD */ @@ -2034,6 +2046,9 @@ void xhci_dbg_trace(struct xhci_hcd *xhci, void (*trace)(struct va_format *), /* xHCI memory management */ void xhci_mem_cleanup(struct xhci_hcd *xhci); int xhci_mem_init(struct xhci_hcd *xhci, gfp_t flags); +void xhci_handle_sec_intr_events(struct xhci_hcd *xhci, + struct xhci_ring *ring, struct xhci_intr_reg __iomem *ir_set, struct xhci_sec *sec); +int xhci_mem_sec_init(struct xhci_hcd *xhci, struct xhci_sec *sec); void xhci_free_virt_device(struct xhci_hcd *xhci, int slot_id); int xhci_alloc_virt_device(struct xhci_hcd *xhci, int slot_id, struct usb_device *udev, gfp_t flags); int xhci_setup_addressable_virt_dev(struct xhci_hcd *xhci, struct usb_device *udev);
On Fri, Dec 23, 2022 at 03:31:53PM -0800, Wesley Cheng wrote:
Implement the XHCI operations for allocating and requesting for a secondary interrupter. The secondary interrupter can allow for events for a particular endpoint to be routed to a separate event ring. The event routing is defined when submitting a transfer descriptor to the USB HW. There is a specific field which denotes which interrupter ring to route the event to when the transfer is completed.
An example use case, such as audio packet offloading can utilize a separate event ring, so that these events can be routed to a different processor within the system. The processor would be able to independently submit transfers and handle its completions without intervention from the main processor.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com
drivers/usb/host/xhci-mem.c | 219 ++++++++++++++++++++++++++++------- drivers/usb/host/xhci-plat.c | 2 + drivers/usb/host/xhci.c | 169 ++++++++++++++++++++++++++- drivers/usb/host/xhci.h | 15 +++ 4 files changed, 363 insertions(+), 42 deletions(-)
diff --git a/drivers/usb/host/xhci-mem.c b/drivers/usb/host/xhci-mem.c index 81ca2bc1f0be..d5cb4b82ad3d 100644 --- a/drivers/usb/host/xhci-mem.c +++ b/drivers/usb/host/xhci-mem.c @@ -1835,6 +1835,7 @@ void xhci_free_erst(struct xhci_hcd *xhci, struct xhci_erst *erst) void xhci_mem_cleanup(struct xhci_hcd *xhci) { struct device *dev = xhci_to_hcd(xhci)->self.sysdev;
struct xhci_sec *sec, *tmp; int i, j, num_ports;
cancel_delayed_work_sync(&xhci->cmd_timer);
@@ -1846,6 +1847,16 @@ void xhci_mem_cleanup(struct xhci_hcd *xhci) xhci->event_ring = NULL; xhci_dbg_trace(xhci, trace_xhci_dbg_init, "Freed event ring");
- list_for_each_entry_safe(sec, tmp, &xhci->xhci_sec_list, list) {
list_del(&sec->list);
if (sec->event_ring) {
xhci_ring_free(xhci, sec->event_ring);
xhci_dbg_trace(xhci, trace_xhci_dbg_init,
"Freed secondary ring %d", sec->intr_num);
Odd indentation :(
On 24.12.2022 1.31, Wesley Cheng wrote:
Implement the XHCI operations for allocating and requesting for a secondary interrupter. The secondary interrupter can allow for events for a particular endpoint to be routed to a separate event ring. The event routing is defined when submitting a transfer descriptor to the USB HW. There is a specific field which denotes which interrupter ring to route the event to when the transfer is completed.
An example use case, such as audio packet offloading can utilize a separate event ring, so that these events can be routed to a different processor within the system. The processor would be able to independently submit transfers and handle its completions without intervention from the main processor.
Adding support for more xHCI interrupters than just the primary one make sense for both the offloading and virtualization cases.
xHCI support for several interrupters was probably added to support virtualization, to hand over usb devices to virtual machines and give them their own event ring and MSI/MSI-X vector.
In this offloading case you probably want to avoid xHC interrupts from this device completely, making sure it doesn't wake up the main CPU unnecessarily.
So is the idea here to let xhci driver set up the new interrupter, its event ring, and the endpoint transfer rings. Then pass the address of the endpoint transfer rings and the new event ring to the separate processor.
This separate processor then both polls the event ring for new events, sets its dequeue pointer, clears EHB bit, and queues new TRBs on the transfer ring.
so xhci driver does not handle any events for the audio part, and no audio data URBs are sent to usb core?
How about the control part? Is the control endpoint for this device still handled normally by usb core/xhci?
For the xhci parts I think we should start start by adding generic support for several interrupters, then add parts needed for offloading.
Thanks Mathias
Hi Mathias,
On 12/28/2022 7:47 AM, Mathias Nyman wrote:
On 24.12.2022 1.31, Wesley Cheng wrote:
Implement the XHCI operations for allocating and requesting for a secondary interrupter. The secondary interrupter can allow for events for a particular endpoint to be routed to a separate event ring. The event routing is defined when submitting a transfer descriptor to the USB HW. There is a specific field which denotes which interrupter ring to route the event to when the transfer is completed.
An example use case, such as audio packet offloading can utilize a separate event ring, so that these events can be routed to a different processor within the system. The processor would be able to independently submit transfers and handle its completions without intervention from the main processor.
Adding support for more xHCI interrupters than just the primary one make sense for both the offloading and virtualization cases.
xHCI support for several interrupters was probably added to support virtualization, to hand over usb devices to virtual machines and give them their own event ring and MSI/MSI-X vector.
In this offloading case you probably want to avoid xHC interrupts from this device completely, making sure it doesn't wake up the main CPU unnecessarily.
So is the idea here to let xhci driver set up the new interrupter, its event ring, and the endpoint transfer rings. Then pass the address of the endpoint transfer rings and the new event ring to the separate processor.
This separate processor then both polls the event ring for new events, sets its dequeue pointer, clears EHB bit, and queues new TRBs on the transfer ring.
so xhci driver does not handle any events for the audio part, and no audio data URBs are sent to usb core?
Your entire description is correct. To clarify, the interfaces which are non-audio will still be handled by the main processor. For example, a USB headset can have a HID interface as well for volume control. The HID interface will still be handled by the main processor, and events routed to the main event ring.
How about the control part? Is the control endpoint for this device still handled normally by usb core/xhci?
Control transfers are always handled on the main processor. Only audio interface's endpoints.
For the xhci parts I think we should start start by adding generic support for several interrupters, then add parts needed for offloading.
I can split up the patchsets to add interrupters first, then adding the offloading APIs in a separate patch.
Thanks Wesley Cheng
On 29.12.2022 23.14, Wesley Cheng wrote:
Hi Mathias,
On 12/28/2022 7:47 AM, Mathias Nyman wrote:
On 24.12.2022 1.31, Wesley Cheng wrote:
Implement the XHCI operations for allocating and requesting for a secondary interrupter. The secondary interrupter can allow for events for a particular endpoint to be routed to a separate event ring. The event routing is defined when submitting a transfer descriptor to the USB HW. There is a specific field which denotes which interrupter ring to route the event to when the transfer is completed.
An example use case, such as audio packet offloading can utilize a separate event ring, so that these events can be routed to a different processor within the system. The processor would be able to independently submit transfers and handle its completions without intervention from the main processor.
Adding support for more xHCI interrupters than just the primary one make sense for both the offloading and virtualization cases.
xHCI support for several interrupters was probably added to support virtualization, to hand over usb devices to virtual machines and give them their own event ring and MSI/MSI-X vector.
In this offloading case you probably want to avoid xHC interrupts from this device completely, making sure it doesn't wake up the main CPU unnecessarily.
So is the idea here to let xhci driver set up the new interrupter, its event ring, and the endpoint transfer rings. Then pass the address of the endpoint transfer rings and the new event ring to the separate processor.
This separate processor then both polls the event ring for new events, sets its dequeue pointer, clears EHB bit, and queues new TRBs on the transfer ring.
so xhci driver does not handle any events for the audio part, and no audio data URBs are sent to usb core?
Your entire description is correct. To clarify, the interfaces which are non-audio will still be handled by the main processor. For example, a USB headset can have a HID interface as well for volume control. The HID interface will still be handled by the main processor, and events routed to the main event ring.
How about the control part? Is the control endpoint for this device still handled normally by usb core/xhci?
Control transfers are always handled on the main processor. Only audio interface's endpoints.
Good to know, that means interrupter should be chosen per endpoint, not per device.
For the xhci parts I think we should start start by adding generic support for several interrupters, then add parts needed for offloading.
I can split up the patchsets to add interrupters first, then adding the offloading APIs in a separate patch.
I started looking at supporting secondary interrupters myself. Let me work on that part a bit first. We have a bit different end goals. I want to handle interrupts from a secondary interrupter, while this audio offload really just wants to mask some interrupts.
Thanks Mathias
Hi Mathias,
On 1/2/2023 8:38 AM, Mathias Nyman wrote:
On 29.12.2022 23.14, Wesley Cheng wrote:
Hi Mathias,
On 12/28/2022 7:47 AM, Mathias Nyman wrote:
On 24.12.2022 1.31, Wesley Cheng wrote:
Implement the XHCI operations for allocating and requesting for a secondary interrupter. The secondary interrupter can allow for events for a particular endpoint to be routed to a separate event ring. The event routing is defined when submitting a transfer descriptor to the USB HW. There is a specific field which denotes which interrupter ring to route the event to when the transfer is completed.
An example use case, such as audio packet offloading can utilize a separate event ring, so that these events can be routed to a different processor within the system. The processor would be able to independently submit transfers and handle its completions without intervention from the main processor.
Adding support for more xHCI interrupters than just the primary one make sense for both the offloading and virtualization cases.
xHCI support for several interrupters was probably added to support virtualization, to hand over usb devices to virtual machines and give them their own event ring and MSI/MSI-X vector.
In this offloading case you probably want to avoid xHC interrupts from this device completely, making sure it doesn't wake up the main CPU unnecessarily.
So is the idea here to let xhci driver set up the new interrupter, its event ring, and the endpoint transfer rings. Then pass the address of the endpoint transfer rings and the new event ring to the separate processor.
This separate processor then both polls the event ring for new events, sets its dequeue pointer, clears EHB bit, and queues new TRBs on the transfer ring.
so xhci driver does not handle any events for the audio part, and no audio data URBs are sent to usb core?
Your entire description is correct. To clarify, the interfaces which are non-audio will still be handled by the main processor. For example, a USB headset can have a HID interface as well for volume control. The HID interface will still be handled by the main processor, and events routed to the main event ring.
How about the control part? Is the control endpoint for this device still handled normally by usb core/xhci?
Control transfers are always handled on the main processor. Only audio interface's endpoints.
Good to know, that means interrupter should be chosen per endpoint, not per device.
For the xhci parts I think we should start start by adding generic support for several interrupters, then add parts needed for offloading.
I can split up the patchsets to add interrupters first, then adding the offloading APIs in a separate patch.
I started looking at supporting secondary interrupters myself. Let me work on that part a bit first. We have a bit different end goals. I want to handle interrupts from a secondary interrupter, while this audio offload really just wants to mask some interrupts.
I was looking at how we could possibly split up the XHCI secondary interrupter, and offloading parts. Since the XHCI secondary interrupter is a feature that is defined in the XHCI spec (and we aren't doing anything outside of what is defined), I was thinking of having a separate XHCI driver (ie xhci-sec.c/h) that can be used to define all APIs related to setting up the event ring and ring management. (interrupt support can be added here) This aligns a bit with what Alan suggested, and removing the APIs in the USB HCD, since this is XHCI specific stuff. ( https://lore.kernel.org/linux-usb/Y6zwZOquZOTZfnvP@rowland.harvard.edu/ )
For the offloading part, I think this is a bit more dependent on how different platforms implement it. To use more of a generic approach like how Albert suggested here:
https://patchwork.kernel.org/project/linux-usb/list/?series=704174
Basically to give vendors the ability to define their own sequences/callbacks, and from which the XHCI driver will call into. (if needed) These would need to be a separate set of XHCI drivers as well.
Do you think this is a proper model for us to go with, so that we can allow for vendors to easily add functionality? Appreciate the inputs.
Thanks Wesley Cheng
On 9.1.2023 22.24, Wesley Cheng wrote:
Hi Mathias,
On 1/2/2023 8:38 AM, Mathias Nyman wrote:
On 29.12.2022 23.14, Wesley Cheng wrote:
Hi Mathias,
On 12/28/2022 7:47 AM, Mathias Nyman wrote:
On 24.12.2022 1.31, Wesley Cheng wrote:
Implement the XHCI operations for allocating and requesting for a secondary interrupter. The secondary interrupter can allow for events for a particular endpoint to be routed to a separate event ring. The event routing is defined when submitting a transfer descriptor to the USB HW. There is a specific field which denotes which interrupter ring to route the event to when the transfer is completed.
An example use case, such as audio packet offloading can utilize a separate event ring, so that these events can be routed to a different processor within the system. The processor would be able to independently submit transfers and handle its completions without intervention from the main processor.
Adding support for more xHCI interrupters than just the primary one make sense for both the offloading and virtualization cases.
xHCI support for several interrupters was probably added to support virtualization, to hand over usb devices to virtual machines and give them their own event ring and MSI/MSI-X vector.
In this offloading case you probably want to avoid xHC interrupts from this device completely, making sure it doesn't wake up the main CPU unnecessarily.
So is the idea here to let xhci driver set up the new interrupter, its event ring, and the endpoint transfer rings. Then pass the address of the endpoint transfer rings and the new event ring to the separate processor.
This separate processor then both polls the event ring for new events, sets its dequeue pointer, clears EHB bit, and queues new TRBs on the transfer ring.
so xhci driver does not handle any events for the audio part, and no audio data URBs are sent to usb core?
Your entire description is correct. To clarify, the interfaces which are non-audio will still be handled by the main processor. For example, a USB headset can have a HID interface as well for volume control. The HID interface will still be handled by the main processor, and events routed to the main event ring.
How about the control part? Is the control endpoint for this device still handled normally by usb core/xhci?
Control transfers are always handled on the main processor. Only audio interface's endpoints.
Good to know, that means interrupter should be chosen per endpoint, not per device.
For the xhci parts I think we should start start by adding generic support for several interrupters, then add parts needed for offloading.
I can split up the patchsets to add interrupters first, then adding the offloading APIs in a separate patch.
I started looking at supporting secondary interrupters myself. Let me work on that part a bit first. We have a bit different end goals. I want to handle interrupts from a secondary interrupter, while this audio offload really just wants to mask some interrupts.
I was looking at how we could possibly split up the XHCI secondary interrupter, and offloading parts. Since the XHCI secondary interrupter is a feature that is defined in the XHCI spec (and we aren't doing anything outside of what is defined), I was thinking of having a separate XHCI driver (ie xhci-sec.c/h) that can be used to define all APIs related to setting up the event ring and ring management. (interrupt support can be added here) This aligns a bit with what Alan suggested, and removing the APIs in the USB HCD, since this is XHCI specific stuff. ( https://lore.kernel.org/linux-usb/Y6zwZOquZOTZfnvP@rowland.harvard.edu/ )
Already started working on the interrupter, that part fits well into current driver.
Code (untested, will be randomly rebased etc) can be found in my feature_interrupters branch: git://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git feature_interrupters https://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git/log/?h=featu...
First step turns current event ring into a primary interrupter. last patch is a test implementation for creating and freeing new secondary interrupters.
For the offloading part, I think this is a bit more dependent on how different platforms implement it. To use more of a generic approach like how Albert suggested here:
https://patchwork.kernel.org/project/linux-usb/list/?series=704174
Basically to give vendors the ability to define their own sequences/callbacks, and from which the XHCI driver will call into. (if needed) These would need to be a separate set of XHCI drivers as well.
Do you think this is a proper model for us to go with, so that we can allow for vendors to easily add functionality? Appreciate the inputs.
I'm not convinced that overriding different xhci memory allocation functions is the best solution. I think xhci driver will need to know which endpoints are offloaded. maybe usb class driver could register an "offloader" with xhci for a usb device.
Trying to figure out what this xhci offload API would look like. The dsp needs at least dma address of an event ring, and offloaded endpoint rings. Is there anything else that the dsp would directly need to take care of, or can we just export some xhci functions for starting/stopping endpoints, and update event deq?
Thanks -Mathias
Hi Mathias,
On 1/10/2023 11:47 AM, Mathias Nyman wrote:
On 9.1.2023 22.24, Wesley Cheng wrote:
Hi Mathias,
On 1/2/2023 8:38 AM, Mathias Nyman wrote:
On 29.12.2022 23.14, Wesley Cheng wrote:
Hi Mathias,
On 12/28/2022 7:47 AM, Mathias Nyman wrote:
On 24.12.2022 1.31, Wesley Cheng wrote:
Implement the XHCI operations for allocating and requesting for a secondary interrupter. The secondary interrupter can allow for events for a particular endpoint to be routed to a separate event ring. The event routing is defined when submitting a transfer descriptor to the USB HW. There is a specific field which denotes which interrupter ring to route the event to when the transfer is completed.
An example use case, such as audio packet offloading can utilize a separate event ring, so that these events can be routed to a different processor within the system. The processor would be able to independently submit transfers and handle its completions without intervention from the main processor.
Adding support for more xHCI interrupters than just the primary one make sense for both the offloading and virtualization cases.
xHCI support for several interrupters was probably added to support virtualization, to hand over usb devices to virtual machines and give them their own event ring and MSI/MSI-X vector.
In this offloading case you probably want to avoid xHC interrupts from this device completely, making sure it doesn't wake up the main CPU unnecessarily.
So is the idea here to let xhci driver set up the new interrupter, its event ring, and the endpoint transfer rings. Then pass the address of the endpoint transfer rings and the new event ring to the separate processor.
This separate processor then both polls the event ring for new events, sets its dequeue pointer, clears EHB bit, and queues new TRBs on the transfer ring.
so xhci driver does not handle any events for the audio part, and no audio data URBs are sent to usb core?
Your entire description is correct. To clarify, the interfaces which are non-audio will still be handled by the main processor. For example, a USB headset can have a HID interface as well for volume control. The HID interface will still be handled by the main processor, and events routed to the main event ring.
How about the control part? Is the control endpoint for this device still handled normally by usb core/xhci?
Control transfers are always handled on the main processor. Only audio interface's endpoints.
Good to know, that means interrupter should be chosen per endpoint, not per device.
For the xhci parts I think we should start start by adding generic support for several interrupters, then add parts needed for offloading.
I can split up the patchsets to add interrupters first, then adding the offloading APIs in a separate patch.
I started looking at supporting secondary interrupters myself. Let me work on that part a bit first. We have a bit different end goals. I want to handle interrupts from a secondary interrupter, while this audio offload really just wants to mask some interrupts.
I was looking at how we could possibly split up the XHCI secondary interrupter, and offloading parts. Since the XHCI secondary interrupter is a feature that is defined in the XHCI spec (and we aren't doing anything outside of what is defined), I was thinking of having a separate XHCI driver (ie xhci-sec.c/h) that can be used to define all APIs related to setting up the event ring and ring management. (interrupt support can be added here) This aligns a bit with what Alan suggested, and removing the APIs in the USB HCD, since this is XHCI specific stuff. ( https://lore.kernel.org/linux-usb/Y6zwZOquZOTZfnvP@rowland.harvard.edu/ )
Already started working on the interrupter, that part fits well into current driver.
Code (untested, will be randomly rebased etc) can be found in my feature_interrupters branch: git://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git feature_interrupters https://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git/log/?h=featu...
Oh perfect, let me take a look. Thanks for this!
First step turns current event ring into a primary interrupter. last patch is a test implementation for creating and freeing new secondary interrupters.
For the offloading part, I think this is a bit more dependent on how different platforms implement it. To use more of a generic approach like how Albert suggested here:
https://patchwork.kernel.org/project/linux-usb/list/?series=704174
Basically to give vendors the ability to define their own sequences/callbacks, and from which the XHCI driver will call into. (if needed) These would need to be a separate set of XHCI drivers as well.
Do you think this is a proper model for us to go with, so that we can allow for vendors to easily add functionality? Appreciate the inputs.
I'm not convinced that overriding different xhci memory allocation functions is the best solution. I think xhci driver will need to know which endpoints are offloaded. maybe usb class driver could register an "offloader" with xhci for a usb device.
Trying to figure out what this xhci offload API would look like. The dsp needs at least dma address of an event ring, and offloaded endpoint rings. Is there anything else that the dsp would directly need to take care of, or can we just export some xhci functions for starting/stopping endpoints, and update event deq?
I'm still working it out with Albert as well to see what they actually require, and if "hooking" into these XHCI apis is really necessary. I wouldn't follow the path of hooking into existing XHCI APIs either, since I'm sure most of it will be duplicate code that is already done by the XHCI drivers. I'll follow up a bit more to get a better understanding, or if Albert wants to chime in here that would be helpful also, so everyone is on the same page.
In the QC implementation, we only need the transfer and event ring address, skipping pending events in the secondary event ring (similar to update event deq), and starting/stopping eps. Exporting APIs would work for us.
Thanks Wesley Cheng
Hi Mathias,
On 1/10/2023 12:03 PM, Wesley Cheng wrote:
Hi Mathias,
On 1/10/2023 11:47 AM, Mathias Nyman wrote:
On 9.1.2023 22.24, Wesley Cheng wrote:
Hi Mathias,
On 1/2/2023 8:38 AM, Mathias Nyman wrote:
On 29.12.2022 23.14, Wesley Cheng wrote:
Hi Mathias,
On 12/28/2022 7:47 AM, Mathias Nyman wrote:
On 24.12.2022 1.31, Wesley Cheng wrote: > Implement the XHCI operations for allocating and requesting for a > secondary > interrupter. The secondary interrupter can allow for events for a > particular endpoint to be routed to a separate event ring. The > event > routing is defined when submitting a transfer descriptor to the > USB HW. > There is a specific field which denotes which interrupter ring to > route the > event to when the transfer is completed. > > An example use case, such as audio packet offloading can utilize > a separate > event ring, so that these events can be routed to a different > processor > within the system. The processor would be able to independently > submit > transfers and handle its completions without intervention from > the main > processor. >
Adding support for more xHCI interrupters than just the primary one make sense for both the offloading and virtualization cases.
xHCI support for several interrupters was probably added to support virtualization, to hand over usb devices to virtual machines and give them their own event ring and MSI/MSI-X vector.
In this offloading case you probably want to avoid xHC interrupts from this device completely, making sure it doesn't wake up the main CPU unnecessarily.
So is the idea here to let xhci driver set up the new interrupter, its event ring, and the endpoint transfer rings. Then pass the address of the endpoint transfer rings and the new event ring to the separate processor.
This separate processor then both polls the event ring for new events, sets its dequeue pointer, clears EHB bit, and queues new TRBs on the transfer ring.
so xhci driver does not handle any events for the audio part, and no audio data URBs are sent to usb core?
Your entire description is correct. To clarify, the interfaces which are non-audio will still be handled by the main processor. For example, a USB headset can have a HID interface as well for volume control. The HID interface will still be handled by the main processor, and events routed to the main event ring.
How about the control part? Is the control endpoint for this device still handled normally by usb core/xhci?
Control transfers are always handled on the main processor. Only audio interface's endpoints.
Good to know, that means interrupter should be chosen per endpoint, not per device.
For the xhci parts I think we should start start by adding generic support for several interrupters, then add parts needed for offloading.
I can split up the patchsets to add interrupters first, then adding the offloading APIs in a separate patch.
I started looking at supporting secondary interrupters myself. Let me work on that part a bit first. We have a bit different end goals. I want to handle interrupts from a secondary interrupter, while this audio offload really just wants to mask some interrupts.
I was looking at how we could possibly split up the XHCI secondary interrupter, and offloading parts. Since the XHCI secondary interrupter is a feature that is defined in the XHCI spec (and we aren't doing anything outside of what is defined), I was thinking of having a separate XHCI driver (ie xhci-sec.c/h) that can be used to define all APIs related to setting up the event ring and ring management. (interrupt support can be added here) This aligns a bit with what Alan suggested, and removing the APIs in the USB HCD, since this is XHCI specific stuff. ( https://lore.kernel.org/linux-usb/Y6zwZOquZOTZfnvP@rowland.harvard.edu/ )
Already started working on the interrupter, that part fits well into current driver.
Code (untested, will be randomly rebased etc) can be found in my feature_interrupters branch: git://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git feature_interrupters https://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git/log/?h=featu...
Oh perfect, let me take a look. Thanks for this!
I actually tried to see if I could get our audio offloading to work with your current series. (I understand its still work in progress) I did have to make some changes to expose the APIs to our class driver, but I wanted to let you know about one of the issues I saw when developing my implementation, because I am seeing the same behavior w/ yours. (and there's a discrepancy w/ what's stated in the XHCI spec :))
So the reason why my initial submission did the event ring allocation and set up before the run/stop bit was set, is that I found that when writing to the ir_set->erst_base in this scenario (for the secondary interrupter), it lead to a SMMU fault from the DWC3 controller. One thing I noticed, was that the SMMU fault address was the lower 32 bits of the segment table base address allocated. The XHCI driver utilizes the xhci_write_64() api which first writes the lower 32 bits then the upper 32 bits. The XHCI spec states that:
Table 5-41: Event Ring Segment Table Base Address Register Bit Definitions (ERSTBA)
"Event Ring Segment Table Base Address Register – RW. Default = ‘0’. This field defines the high order bits of the start address of the Event Ring Segment Table. Writing this register sets the Event Ring State Machine:EREP Advancement to the Start state. Refer to Figure 4-12 for more information. **For Secondary Interrupters: This field may be modified at any time.**"
I'm not sure if this is an issue with the specific controller we're using, so maybe I will wait until you can give this a try on your set up. However, it doesn't seem to be true that we can write the ERSTBA any time we want to. My assumption is that once I made the lower 32 bit write, the controller attempted to enable the Event Ring State machine (Figure 4-12), and this led to a SMMU fault, since the upper 64 bits haven't been written. I also did some bit banging manually as well (using devmem) and any time I write to the secondary ring ERSTBA register it generates a fault. (before any offloading has started)
Thanks Wesley Cheng
On 11.1.2023 5.11, Wesley Cheng wrote:
Hi Mathias,
On 1/10/2023 12:03 PM, Wesley Cheng wrote:
Hi Mathias,
On 1/10/2023 11:47 AM, Mathias Nyman wrote:
On 9.1.2023 22.24, Wesley Cheng wrote:
Hi Mathias,
On 1/2/2023 8:38 AM, Mathias Nyman wrote:
On 29.12.2022 23.14, Wesley Cheng wrote:
Hi Mathias,
On 12/28/2022 7:47 AM, Mathias Nyman wrote: > On 24.12.2022 1.31, Wesley Cheng wrote: >> Implement the XHCI operations for allocating and requesting for a secondary >> interrupter. The secondary interrupter can allow for events for a >> particular endpoint to be routed to a separate event ring. The event >> routing is defined when submitting a transfer descriptor to the USB HW. >> There is a specific field which denotes which interrupter ring to route the >> event to when the transfer is completed. >> >> An example use case, such as audio packet offloading can utilize a separate >> event ring, so that these events can be routed to a different processor >> within the system. The processor would be able to independently submit >> transfers and handle its completions without intervention from the main >> processor. >> > > Adding support for more xHCI interrupters than just the primary one make sense for > both the offloading and virtualization cases. > > xHCI support for several interrupters was probably added to support virtualization, > to hand over usb devices to virtual machines and give them their own event ring and > MSI/MSI-X vector. > > In this offloading case you probably want to avoid xHC interrupts from this device > completely, making sure it doesn't wake up the main CPU unnecessarily. > > So is the idea here to let xhci driver set up the new interrupter, its event ring, > and the endpoint transfer rings. Then pass the address of the endpoint transfer rings > and the new event ring to the separate processor. > > This separate processor then both polls the event ring for new events, sets its dequeue > pointer, clears EHB bit, and queues new TRBs on the transfer ring. > > so xhci driver does not handle any events for the audio part, and no audio data URBs > are sent to usb core?
Your entire description is correct. To clarify, the interfaces which are non-audio will still be handled by the main processor. For example, a USB headset can have a HID interface as well for volume control. The HID interface will still be handled by the main processor, and events routed to the main event ring.
> > How about the control part? > Is the control endpoint for this device still handled normally by usb core/xhci? >
Control transfers are always handled on the main processor. Only audio interface's endpoints.
Good to know, that means interrupter should be chosen per endpoint, not per device.
> For the xhci parts I think we should start start by adding generic support for several > interrupters, then add parts needed for offloading.
I can split up the patchsets to add interrupters first, then adding the offloading APIs in a separate patch.
I started looking at supporting secondary interrupters myself. Let me work on that part a bit first. We have a bit different end goals. I want to handle interrupts from a secondary interrupter, while this audio offload really just wants to mask some interrupts.
I was looking at how we could possibly split up the XHCI secondary interrupter, and offloading parts. Since the XHCI secondary interrupter is a feature that is defined in the XHCI spec (and we aren't doing anything outside of what is defined), I was thinking of having a separate XHCI driver (ie xhci-sec.c/h) that can be used to define all APIs related to setting up the event ring and ring management. (interrupt support can be added here) This aligns a bit with what Alan suggested, and removing the APIs in the USB HCD, since this is XHCI specific stuff. ( https://lore.kernel.org/linux-usb/Y6zwZOquZOTZfnvP@rowland.harvard.edu/ )
Already started working on the interrupter, that part fits well into current driver.
Code (untested, will be randomly rebased etc) can be found in my feature_interrupters branch: git://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git feature_interrupters https://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git/log/?h=featu...
Oh perfect, let me take a look. Thanks for this!
I actually tried to see if I could get our audio offloading to work with your current series. (I understand its still work in progress) I did have to make some changes to expose the APIs to our class driver, but I wanted to let you know about one of the issues I saw when developing my implementation, because I am seeing the same behavior w/ yours. (and there's a discrepancy w/ what's stated in the XHCI spec :))
So the reason why my initial submission did the event ring allocation and set up before the run/stop bit was set, is that I found that when writing to the ir_set->erst_base in this scenario (for the secondary interrupter), it lead to a SMMU fault from the DWC3 controller. One thing I noticed, was that the SMMU fault address was the lower 32 bits of the segment table base address allocated. The XHCI driver utilizes the xhci_write_64() api which first writes the lower 32 bits then the upper 32 bits. The XHCI spec states that:
Table 5-41: Event Ring Segment Table Base Address Register Bit Definitions (ERSTBA)
"Event Ring Segment Table Base Address Register – RW. Default = ‘0’. This field defines the high order bits of the start address of the Event Ring Segment Table. Writing this register sets the Event Ring State Machine:EREP Advancement to the Start state. Refer to Figure 4-12 for more information. **For Secondary Interrupters: This field may be modified at any time.**"
I'm not sure if this is an issue with the specific controller we're using, so maybe I will wait until you can give this a try on your set up. However, it doesn't seem to be true that we can write the ERSTBA any time we want to. My assumption is that once I made the lower 32 bit write, the controller attempted to enable the Event Ring State machine (Figure 4-12), and this led to a SMMU fault, since the upper 64 bits haven't been written. I also did some bit banging manually as well (using devmem) and any time I write to the secondary ring ERSTBA register it generates a fault. (before any offloading has started)
Tried on an Intel host and it seems to work fine. I created a few secondary interrupters while xHC was running without issues. DMA mask is 64 bits. Only created the interrupters, no events on those new event rings, and didn't actually check that the values written to ERSTBA were 64 bit.
Does temporarily setting DMA mask to 32 bits while allocating erst help in your case?
Thanks Mathias
Hi Mathias,
On 1/12/2023 1:24 AM, Mathias Nyman wrote:
On 11.1.2023 5.11, Wesley Cheng wrote:
Hi Mathias,
On 1/10/2023 12:03 PM, Wesley Cheng wrote:
Hi Mathias,
On 1/10/2023 11:47 AM, Mathias Nyman wrote:
On 9.1.2023 22.24, Wesley Cheng wrote:
Hi Mathias,
On 1/2/2023 8:38 AM, Mathias Nyman wrote:
On 29.12.2022 23.14, Wesley Cheng wrote: > Hi Mathias, > > On 12/28/2022 7:47 AM, Mathias Nyman wrote: >> On 24.12.2022 1.31, Wesley Cheng wrote: >>> Implement the XHCI operations for allocating and requesting for >>> a secondary >>> interrupter. The secondary interrupter can allow for events for a >>> particular endpoint to be routed to a separate event ring. The >>> event >>> routing is defined when submitting a transfer descriptor to the >>> USB HW. >>> There is a specific field which denotes which interrupter ring >>> to route the >>> event to when the transfer is completed. >>> >>> An example use case, such as audio packet offloading can >>> utilize a separate >>> event ring, so that these events can be routed to a different >>> processor >>> within the system. The processor would be able to >>> independently submit >>> transfers and handle its completions without intervention from >>> the main >>> processor. >>> >> >> Adding support for more xHCI interrupters than just the primary >> one make sense for >> both the offloading and virtualization cases. >> >> xHCI support for several interrupters was probably added to >> support virtualization, >> to hand over usb devices to virtual machines and give them their >> own event ring and >> MSI/MSI-X vector. >> >> In this offloading case you probably want to avoid xHC >> interrupts from this device >> completely, making sure it doesn't wake up the main CPU >> unnecessarily. >> >> So is the idea here to let xhci driver set up the new >> interrupter, its event ring, >> and the endpoint transfer rings. Then pass the address of the >> endpoint transfer rings >> and the new event ring to the separate processor. >> >> This separate processor then both polls the event ring for new >> events, sets its dequeue >> pointer, clears EHB bit, and queues new TRBs on the transfer ring. >> >> so xhci driver does not handle any events for the audio part, >> and no audio data URBs >> are sent to usb core? > > Your entire description is correct. To clarify, the interfaces > which are non-audio will still be handled by the main processor. > For example, a USB headset can have a HID interface as well for > volume control. The HID interface will still be handled by the > main processor, and events routed to the main event ring. > >> >> How about the control part? >> Is the control endpoint for this device still handled normally >> by usb core/xhci? >> > > Control transfers are always handled on the main processor. Only > audio interface's endpoints.
Good to know, that means interrupter should be chosen per endpoint, not per device.
> >> For the xhci parts I think we should start start by adding >> generic support for several >> interrupters, then add parts needed for offloading. > I can split up the patchsets to add interrupters first, then adding the offloading APIs in a separate patch.
I started looking at supporting secondary interrupters myself. Let me work on that part a bit first. We have a bit different end goals. I want to handle interrupts from a secondary interrupter, while this audio offload really just wants to mask some interrupts.
I was looking at how we could possibly split up the XHCI secondary interrupter, and offloading parts. Since the XHCI secondary interrupter is a feature that is defined in the XHCI spec (and we aren't doing anything outside of what is defined), I was thinking of having a separate XHCI driver (ie xhci-sec.c/h) that can be used to define all APIs related to setting up the event ring and ring management. (interrupt support can be added here) This aligns a bit with what Alan suggested, and removing the APIs in the USB HCD, since this is XHCI specific stuff. ( https://lore.kernel.org/linux-usb/Y6zwZOquZOTZfnvP@rowland.harvard.edu/ )
Already started working on the interrupter, that part fits well into current driver.
Code (untested, will be randomly rebased etc) can be found in my feature_interrupters branch: git://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git feature_interrupters https://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git/log/?h=featu...
Oh perfect, let me take a look. Thanks for this!
I actually tried to see if I could get our audio offloading to work with your current series. (I understand its still work in progress) I did have to make some changes to expose the APIs to our class driver, but I wanted to let you know about one of the issues I saw when developing my implementation, because I am seeing the same behavior w/ yours. (and there's a discrepancy w/ what's stated in the XHCI spec :))
So the reason why my initial submission did the event ring allocation and set up before the run/stop bit was set, is that I found that when writing to the ir_set->erst_base in this scenario (for the secondary interrupter), it lead to a SMMU fault from the DWC3 controller. One thing I noticed, was that the SMMU fault address was the lower 32 bits of the segment table base address allocated. The XHCI driver utilizes the xhci_write_64() api which first writes the lower 32 bits then the upper 32 bits. The XHCI spec states that:
Table 5-41: Event Ring Segment Table Base Address Register Bit Definitions (ERSTBA)
"Event Ring Segment Table Base Address Register – RW. Default = ‘0’. This field defines the high order bits of the start address of the Event Ring Segment Table. Writing this register sets the Event Ring State Machine:EREP Advancement to the Start state. Refer to Figure 4-12 for more information. **For Secondary Interrupters: This field may be modified at any time.**"
I'm not sure if this is an issue with the specific controller we're using, so maybe I will wait until you can give this a try on your set up. However, it doesn't seem to be true that we can write the ERSTBA any time we want to. My assumption is that once I made the lower 32 bit write, the controller attempted to enable the Event Ring State machine (Figure 4-12), and this led to a SMMU fault, since the upper 64 bits haven't been written. I also did some bit banging manually as well (using devmem) and any time I write to the secondary ring ERSTBA register it generates a fault. (before any offloading has started)
Tried on an Intel host and it seems to work fine. I created a few secondary interrupters while xHC was running without issues. DMA mask is 64 bits. Only created the interrupters, no events on those new event rings, and didn't actually check that the values written to ERSTBA were 64 bit.
Does temporarily setting DMA mask to 32 bits while allocating erst help in your case?
Yes, that works fine. Another thing I found which works is that, the XHCI spec mentions that we should always write the lower 32 bits first before the upper bits for a 64 bit register. Just as an experiment, I flipped the write order (upper first, lower second), and that seemed to not trigger the event ring state machine, and I was able to verify that our audio offloading was working.
At the moment, I exposed/exported the xhci_remove_secondary_interrupter() and xhci_create_secondary_interrupter() into a header file (include/linux/usb/xhci-intr.h), along with some other APIs to fetch the transfer resources. (same mechanism I have in my patchset) This allowed me to only reference the required APIs from the USB audio offload class driver w/o having to include all of xhci.h (where some of the things are currently defined in your changes)
Thanks Wesley Cheng
Allow for the DWC3 host driver to pass along a XHCI property that defines how many interrupters to allocate. This is in relation for the number of event rings that can be potentially used by other processors within the system.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com --- drivers/usb/dwc3/core.c | 12 ++++++++++++ drivers/usb/dwc3/core.h | 2 ++ drivers/usb/dwc3/host.c | 5 ++++- 3 files changed, 18 insertions(+), 1 deletion(-)
diff --git a/drivers/usb/dwc3/core.c b/drivers/usb/dwc3/core.c index 476b63618511..67d6f0ae81d2 100644 --- a/drivers/usb/dwc3/core.c +++ b/drivers/usb/dwc3/core.c @@ -1446,6 +1446,7 @@ static void dwc3_get_properties(struct dwc3 *dwc) u8 tx_thr_num_pkt_prd = 0; u8 tx_max_burst_prd = 0; u8 tx_fifo_resize_max_num; + u8 num_hc_interrupters; const char *usb_psy_name; int ret;
@@ -1468,6 +1469,9 @@ static void dwc3_get_properties(struct dwc3 *dwc) */ tx_fifo_resize_max_num = 6;
+ /* default to a single XHCI interrupter */ + num_hc_interrupters = 1; + dwc->maximum_speed = usb_get_maximum_speed(dev); dwc->max_ssp_rate = usb_get_maximum_ssp_rate(dev); dwc->dr_mode = usb_get_dr_mode(dev); @@ -1511,6 +1515,12 @@ static void dwc3_get_properties(struct dwc3 *dwc) &tx_thr_num_pkt_prd); device_property_read_u8(dev, "snps,tx-max-burst-prd", &tx_max_burst_prd); + device_property_read_u8(dev, "snps,num-hc-interrupters", + &num_hc_interrupters); + /* DWC3 core allowed to have a max of 8 interrupters */ + if (num_hc_interrupters > 8) + num_hc_interrupters = 8; + dwc->do_fifo_resize = device_property_read_bool(dev, "tx-fifo-resize"); if (dwc->do_fifo_resize) @@ -1589,6 +1599,8 @@ static void dwc3_get_properties(struct dwc3 *dwc) dwc->imod_interval = 0;
dwc->tx_fifo_resize_max_num = tx_fifo_resize_max_num; + + dwc->num_hc_interrupters = num_hc_interrupters; }
/* check whether the core supports IMOD */ diff --git a/drivers/usb/dwc3/core.h b/drivers/usb/dwc3/core.h index 8f9959ba9fd4..09037299da53 100644 --- a/drivers/usb/dwc3/core.h +++ b/drivers/usb/dwc3/core.h @@ -1050,6 +1050,7 @@ struct dwc3_scratchpad_array { * @tx_max_burst_prd: max periodic ESS transmit burst size * @tx_fifo_resize_max_num: max number of fifos allocated during txfifo resize * @clear_stall_protocol: endpoint number that requires a delayed status phase + * @num_hc_interrupters: number of host controller interrupters * @hsphy_interface: "utmi" or "ulpi" * @connected: true when we're connected to a host, false otherwise * @softconnect: true when gadget connect is called, false when disconnect runs @@ -1275,6 +1276,7 @@ struct dwc3 { u8 tx_max_burst_prd; u8 tx_fifo_resize_max_num; u8 clear_stall_protocol; + u8 num_hc_interrupters;
const char *hsphy_interface;
diff --git a/drivers/usb/dwc3/host.c b/drivers/usb/dwc3/host.c index f6f13e7f1ba1..52a284fdd704 100644 --- a/drivers/usb/dwc3/host.c +++ b/drivers/usb/dwc3/host.c @@ -66,7 +66,7 @@ static int dwc3_host_get_irq(struct dwc3 *dwc)
int dwc3_host_init(struct dwc3 *dwc) { - struct property_entry props[4]; + struct property_entry props[5]; struct platform_device *xhci; int ret, irq; int prop_idx = 0; @@ -112,6 +112,9 @@ int dwc3_host_init(struct dwc3 *dwc) if (DWC3_VER_IS_WITHIN(DWC3, ANY, 300A)) props[prop_idx++] = PROPERTY_ENTRY_BOOL("quirk-broken-port-ped");
+ props[prop_idx++] = PROPERTY_ENTRY_U8("num-hc-interrupters", + dwc->num_hc_interrupters); + if (prop_idx) { ret = device_create_managed_software_node(&xhci->dev, props, NULL); if (ret) {
On Sat, 24 Dec 2022 at 01:33, Wesley Cheng quic_wcheng@quicinc.com wrote:
Allow for the DWC3 host driver to pass along a XHCI property that defines how many interrupters to allocate. This is in relation for the number of event rings that can be potentially used by other processors within the system.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com
drivers/usb/dwc3/core.c | 12 ++++++++++++ drivers/usb/dwc3/core.h | 2 ++ drivers/usb/dwc3/host.c | 5 ++++- 3 files changed, 18 insertions(+), 1 deletion(-)
diff --git a/drivers/usb/dwc3/core.c b/drivers/usb/dwc3/core.c index 476b63618511..67d6f0ae81d2 100644 --- a/drivers/usb/dwc3/core.c +++ b/drivers/usb/dwc3/core.c @@ -1446,6 +1446,7 @@ static void dwc3_get_properties(struct dwc3 *dwc) u8 tx_thr_num_pkt_prd = 0; u8 tx_max_burst_prd = 0; u8 tx_fifo_resize_max_num;
u8 num_hc_interrupters; const char *usb_psy_name; int ret;
@@ -1468,6 +1469,9 @@ static void dwc3_get_properties(struct dwc3 *dwc) */ tx_fifo_resize_max_num = 6;
/* default to a single XHCI interrupter */
num_hc_interrupters = 1;
dwc->maximum_speed = usb_get_maximum_speed(dev); dwc->max_ssp_rate = usb_get_maximum_ssp_rate(dev); dwc->dr_mode = usb_get_dr_mode(dev);
@@ -1511,6 +1515,12 @@ static void dwc3_get_properties(struct dwc3 *dwc) &tx_thr_num_pkt_prd); device_property_read_u8(dev, "snps,tx-max-burst-prd", &tx_max_burst_prd);
device_property_read_u8(dev, "snps,num-hc-interrupters",
&num_hc_interrupters);
bindings change?
/* DWC3 core allowed to have a max of 8 interrupters */
if (num_hc_interrupters > 8)
num_hc_interrupters = 8;
dwc->do_fifo_resize = device_property_read_bool(dev, "tx-fifo-resize"); if (dwc->do_fifo_resize)
@@ -1589,6 +1599,8 @@ static void dwc3_get_properties(struct dwc3 *dwc) dwc->imod_interval = 0;
dwc->tx_fifo_resize_max_num = tx_fifo_resize_max_num;
dwc->num_hc_interrupters = num_hc_interrupters;
}
/* check whether the core supports IMOD */ diff --git a/drivers/usb/dwc3/core.h b/drivers/usb/dwc3/core.h index 8f9959ba9fd4..09037299da53 100644 --- a/drivers/usb/dwc3/core.h +++ b/drivers/usb/dwc3/core.h @@ -1050,6 +1050,7 @@ struct dwc3_scratchpad_array {
- @tx_max_burst_prd: max periodic ESS transmit burst size
- @tx_fifo_resize_max_num: max number of fifos allocated during txfifo resize
- @clear_stall_protocol: endpoint number that requires a delayed status phase
- @num_hc_interrupters: number of host controller interrupters
- @hsphy_interface: "utmi" or "ulpi"
- @connected: true when we're connected to a host, false otherwise
- @softconnect: true when gadget connect is called, false when disconnect runs
@@ -1275,6 +1276,7 @@ struct dwc3 { u8 tx_max_burst_prd; u8 tx_fifo_resize_max_num; u8 clear_stall_protocol;
u8 num_hc_interrupters; const char *hsphy_interface;
diff --git a/drivers/usb/dwc3/host.c b/drivers/usb/dwc3/host.c index f6f13e7f1ba1..52a284fdd704 100644 --- a/drivers/usb/dwc3/host.c +++ b/drivers/usb/dwc3/host.c @@ -66,7 +66,7 @@ static int dwc3_host_get_irq(struct dwc3 *dwc)
int dwc3_host_init(struct dwc3 *dwc) {
struct property_entry props[4];
struct property_entry props[5]; struct platform_device *xhci; int ret, irq; int prop_idx = 0;
@@ -112,6 +112,9 @@ int dwc3_host_init(struct dwc3 *dwc) if (DWC3_VER_IS_WITHIN(DWC3, ANY, 300A)) props[prop_idx++] = PROPERTY_ENTRY_BOOL("quirk-broken-port-ped");
props[prop_idx++] = PROPERTY_ENTRY_U8("num-hc-interrupters",
dwc->num_hc_interrupters);
if (prop_idx) { ret = device_create_managed_software_node(&xhci->dev, props, NULL); if (ret) {
On 24/12/2022 12:13, Dmitry Baryshkov wrote:
@@ -1468,6 +1469,9 @@ static void dwc3_get_properties(struct dwc3 *dwc) */ tx_fifo_resize_max_num = 6;
/* default to a single XHCI interrupter */
num_hc_interrupters = 1;
dwc->maximum_speed = usb_get_maximum_speed(dev); dwc->max_ssp_rate = usb_get_maximum_ssp_rate(dev); dwc->dr_mode = usb_get_dr_mode(dev);
@@ -1511,6 +1515,12 @@ static void dwc3_get_properties(struct dwc3 *dwc) &tx_thr_num_pkt_prd); device_property_read_u8(dev, "snps,tx-max-burst-prd", &tx_max_burst_prd);
device_property_read_u8(dev, "snps,num-hc-interrupters",
&num_hc_interrupters);
bindings change?
Undocumented bindings change :(
Best regards, Krzysztof
Hi Dmitry,
On 12/24/2022 3:13 AM, Dmitry Baryshkov wrote:
On Sat, 24 Dec 2022 at 01:33, Wesley Cheng quic_wcheng@quicinc.com wrote:
Allow for the DWC3 host driver to pass along a XHCI property that defines how many interrupters to allocate. This is in relation for the number of event rings that can be potentially used by other processors within the system.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com
drivers/usb/dwc3/core.c | 12 ++++++++++++ drivers/usb/dwc3/core.h | 2 ++ drivers/usb/dwc3/host.c | 5 ++++- 3 files changed, 18 insertions(+), 1 deletion(-)
diff --git a/drivers/usb/dwc3/core.c b/drivers/usb/dwc3/core.c index 476b63618511..67d6f0ae81d2 100644 --- a/drivers/usb/dwc3/core.c +++ b/drivers/usb/dwc3/core.c @@ -1446,6 +1446,7 @@ static void dwc3_get_properties(struct dwc3 *dwc) u8 tx_thr_num_pkt_prd = 0; u8 tx_max_burst_prd = 0; u8 tx_fifo_resize_max_num;
u8 num_hc_interrupters; const char *usb_psy_name; int ret;
@@ -1468,6 +1469,9 @@ static void dwc3_get_properties(struct dwc3 *dwc) */ tx_fifo_resize_max_num = 6;
/* default to a single XHCI interrupter */
num_hc_interrupters = 1;
dwc->maximum_speed = usb_get_maximum_speed(dev); dwc->max_ssp_rate = usb_get_maximum_ssp_rate(dev); dwc->dr_mode = usb_get_dr_mode(dev);
@@ -1511,6 +1515,12 @@ static void dwc3_get_properties(struct dwc3 *dwc) &tx_thr_num_pkt_prd); device_property_read_u8(dev, "snps,tx-max-burst-prd", &tx_max_burst_prd);
device_property_read_u8(dev, "snps,num-hc-interrupters",
&num_hc_interrupters);
bindings change?
Will add one. Thanks!
Thanks Wesley Cheng
Several Qualcomm SoCs have a dedicated audio DSP, which has the ability to support USB sound devices. This vendor driver will implement the required handshaking with the DSP, in order to pass along required resources that will be utilized by the DSP's USB SW. The communication channel used for this handshaking will be using the QMI protocol. Required resources include: - Allocated secondary event ring address - EP transfer ring address - Interrupter number
The above information will allow for the audio DSP to execute USB transfers over the USB bus. It will also be able to support devices that have an implicit feedback and sync endpoint as well. Offloading these data transfers will allow the main/applications processor to enter lower CPU power modes, and sustain a longer duration in those modes.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com --- sound/usb/Kconfig | 14 + sound/usb/Makefile | 2 +- sound/usb/qcom/Makefile | 2 + sound/usb/qcom/qc_audio_offload.c | 1610 ++++++++++++++++++++++++++++ sound/usb/qcom/usb_audio_qmi_v01.c | 892 +++++++++++++++ sound/usb/qcom/usb_audio_qmi_v01.h | 162 +++ 6 files changed, 2681 insertions(+), 1 deletion(-) create mode 100644 sound/usb/qcom/Makefile create mode 100644 sound/usb/qcom/qc_audio_offload.c create mode 100644 sound/usb/qcom/usb_audio_qmi_v01.c create mode 100644 sound/usb/qcom/usb_audio_qmi_v01.h
diff --git a/sound/usb/Kconfig b/sound/usb/Kconfig index 059242f15d75..18d65a0d905a 100644 --- a/sound/usb/Kconfig +++ b/sound/usb/Kconfig @@ -165,6 +165,20 @@ config SND_BCD2000 To compile this driver as a module, choose M here: the module will be called snd-bcd2000.
+config QC_USB_AUDIO_OFFLOAD + tristate "Qualcomm Audio Offload driver" + select SND_PCM + help + Say Y here to enable the Qualcomm USB audio offloading feature + + This module sets up the required QMI stream enable/disable + responses to requests generated by the audio DSP. It passes the + USB transfer resource references, so that the audio DSP can issue + USB transfers to the host controller. + + To compile this driver as a module, choose M here: the module + will be called qc-audio-offload. + source "sound/usb/line6/Kconfig"
endif # SND_USB diff --git a/sound/usb/Makefile b/sound/usb/Makefile index 9ccb21a4ff8a..2243ae333ec9 100644 --- a/sound/usb/Makefile +++ b/sound/usb/Makefile @@ -33,5 +33,5 @@ obj-$(CONFIG_SND_USB_UA101) += snd-usbmidi-lib.o obj-$(CONFIG_SND_USB_USX2Y) += snd-usbmidi-lib.o obj-$(CONFIG_SND_USB_US122L) += snd-usbmidi-lib.o
-obj-$(CONFIG_SND) += misc/ usx2y/ caiaq/ 6fire/ hiface/ bcd2000/ +obj-$(CONFIG_SND) += misc/ usx2y/ caiaq/ 6fire/ hiface/ bcd2000/ qcom/ obj-$(CONFIG_SND_USB_LINE6) += line6/ diff --git a/sound/usb/qcom/Makefile b/sound/usb/qcom/Makefile new file mode 100644 index 000000000000..d27d39beb8ce --- /dev/null +++ b/sound/usb/qcom/Makefile @@ -0,0 +1,2 @@ +snd-usb-audio-qmi-objs := usb_audio_qmi_v01.o qc_audio_offload.o +obj-$(CONFIG_QC_USB_AUDIO_OFFLOAD) += snd-usb-audio-qmi.o \ No newline at end of file diff --git a/sound/usb/qcom/qc_audio_offload.c b/sound/usb/qcom/qc_audio_offload.c new file mode 100644 index 000000000000..a3a91c72c684 --- /dev/null +++ b/sound/usb/qcom/qc_audio_offload.c @@ -0,0 +1,1610 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include <linux/ctype.h> +#include <linux/moduleparam.h> +#include <linux/module.h> +#include <linux/usb.h> +#include <linux/init.h> +#include <linux/usb/quirks.h> +#include <linux/usb/audio.h> +#include <linux/usb/audio-v2.h> +#include <linux/usb/audio-v3.h> +#include <linux/soc/qcom/qmi.h> +#include <linux/iommu.h> +#include <linux/dma-mapping.h> +#include <linux/dma-map-ops.h> +#include <sound/q6usboffload.h> + +#include <sound/control.h> +#include <sound/core.h> +#include <sound/info.h> +#include <sound/pcm.h> +#include <sound/pcm_params.h> +#include <sound/initval.h> + +#include <sound/soc.h> +#include <sound/soc-usb.h> +#include "../usbaudio.h" +#include "../card.h" +#include "../midi.h" +#include "../mixer.h" +#include "../proc.h" +#include "../quirks.h" +#include "../endpoint.h" +#include "../helper.h" +#include "../pcm.h" +#include "../format.h" +#include "../power.h" +#include "../stream.h" +#include "../media.h" +#include "usb_audio_qmi_v01.h" + +#define DEV_RELEASE_WAIT_TIMEOUT 10000 /* in ms */ + +#define BUS_INTERVAL_FULL_SPEED 1000 /* in us */ +#define BUS_INTERVAL_HIGHSPEED_AND_ABOVE 125 /* in us */ +#define MAX_BINTERVAL_ISOC_EP 16 + +#define SND_PCM_CARD_NUM_MASK 0xffff0000 +#define SND_PCM_DEV_NUM_MASK 0xff00 +#define SND_PCM_STREAM_DIRECTION 0xff + +#define PREPEND_SID_TO_IOVA(iova, sid) ((u64)(((u64)(iova)) | \ + (((u64)sid) << 32))) + +/* event ring iova base address */ +#define IOVA_BASE 0x1000 + +#define IOVA_XFER_RING_BASE (IOVA_BASE + PAGE_SIZE * (SNDRV_CARDS + 1)) +#define IOVA_XFER_BUF_BASE (IOVA_XFER_RING_BASE + PAGE_SIZE * SNDRV_CARDS * 32) +#define IOVA_XFER_RING_MAX (IOVA_XFER_BUF_BASE - PAGE_SIZE) +#define IOVA_XFER_BUF_MAX (0xfffff000 - PAGE_SIZE) + +#define MAX_XFER_BUFF_LEN (24 * PAGE_SIZE) + +struct iova_info { + struct list_head list; + unsigned long start_iova; + size_t size; + bool in_use; +}; + +struct intf_info { + unsigned long data_xfer_ring_va; + size_t data_xfer_ring_size; + unsigned long sync_xfer_ring_va; + size_t sync_xfer_ring_size; + unsigned long xfer_buf_va; + size_t xfer_buf_size; + phys_addr_t xfer_buf_pa; + unsigned int data_ep_pipe; + unsigned int sync_ep_pipe; + u8 *xfer_buf; + u8 intf_num; + u8 pcm_card_num; + u8 pcm_dev_num; + u8 direction; + bool in_use; +}; + +struct uaudio_qmi_dev { + struct device *dev; + u32 sid; + u32 intr_num; + struct xhci_ring *sec_ring; + struct iommu_domain *domain; + + /* list to keep track of available iova */ + struct list_head xfer_ring_list; + size_t xfer_ring_iova_size; + unsigned long curr_xfer_ring_iova; + struct list_head xfer_buf_list; + size_t xfer_buf_iova_size; + unsigned long curr_xfer_buf_iova; + + /* bit fields representing pcm card enabled */ + unsigned long card_slot; + /* indicate event ring mapped or not */ + bool er_mapped; + /* reference count to number of possible consumers */ + atomic_t qdev_in_use; + /* idx to last udev card number plugged in */ + unsigned int last_card_num; +}; + +struct uaudio_dev { + struct usb_device *udev; + /* audio control interface */ + struct usb_host_interface *ctrl_intf; + unsigned int card_num; + unsigned int usb_core_id; + atomic_t in_use; + struct kref kref; + wait_queue_head_t disconnect_wq; + + /* interface specific */ + int num_intf; + struct intf_info *info; + struct snd_usb_audio *chip; +}; + +static struct uaudio_dev uadev[SNDRV_CARDS]; +static struct uaudio_qmi_dev *uaudio_qdev; +static struct uaudio_qmi_svc *uaudio_svc; + +struct uaudio_qmi_svc { + struct qmi_handle *uaudio_svc_hdl; + struct work_struct qmi_disconnect_work; + struct workqueue_struct *uaudio_wq; + struct sockaddr_qrtr client_sq; + bool client_connected; +}; + +enum mem_type { + MEM_EVENT_RING, + MEM_XFER_RING, + MEM_XFER_BUF, +}; + +enum usb_qmi_audio_format { + USB_QMI_PCM_FORMAT_S8 = 0, + USB_QMI_PCM_FORMAT_U8, + USB_QMI_PCM_FORMAT_S16_LE, + USB_QMI_PCM_FORMAT_S16_BE, + USB_QMI_PCM_FORMAT_U16_LE, + USB_QMI_PCM_FORMAT_U16_BE, + USB_QMI_PCM_FORMAT_S24_LE, + USB_QMI_PCM_FORMAT_S24_BE, + USB_QMI_PCM_FORMAT_U24_LE, + USB_QMI_PCM_FORMAT_U24_BE, + USB_QMI_PCM_FORMAT_S24_3LE, + USB_QMI_PCM_FORMAT_S24_3BE, + USB_QMI_PCM_FORMAT_U24_3LE, + USB_QMI_PCM_FORMAT_U24_3BE, + USB_QMI_PCM_FORMAT_S32_LE, + USB_QMI_PCM_FORMAT_S32_BE, + USB_QMI_PCM_FORMAT_U32_LE, + USB_QMI_PCM_FORMAT_U32_BE, +}; + +static void uaudio_iommu_unmap(enum mem_type mtype, unsigned long va, + size_t iova_size, size_t mapped_iova_size); +static void uaudio_dev_cleanup(struct uaudio_dev *dev); +static void disable_audio_stream(struct snd_usb_substream *subs); +static struct snd_usb_substream *find_substream(unsigned int card_num, + unsigned int pcm_idx, unsigned int direction); + +static void qmi_disconnect_work(struct work_struct *w) +{ + struct intf_info *info; + int idx, if_idx; + struct snd_usb_substream *subs; + struct snd_usb_audio *chip; + + /* find all active intf for set alt 0 and cleanup usb audio dev */ + for (idx = 0; idx < SNDRV_CARDS; idx++) { + if (!atomic_read(&uadev[idx].in_use)) + continue; + + for (if_idx = 0; if_idx < uadev[idx].num_intf; if_idx++) { + if (!uadev[idx].info || !uadev[idx].info[if_idx].in_use) + continue; + info = &uadev[idx].info[if_idx]; + subs = find_substream(info->pcm_card_num, + info->pcm_dev_num, + info->direction); + chip = uadev[idx].chip; + if (!subs || !chip || atomic_read(&chip->shutdown)) { + pr_err("no subs for c#%u, dev#%u dir%u\n", + info->pcm_card_num, + info->pcm_dev_num, + info->direction); + continue; + } + disable_audio_stream(subs); + } + atomic_set(&uadev[idx].in_use, 0); + uaudio_dev_cleanup(&uadev[idx]); + } +} + +static void qmi_bye_cb(struct qmi_handle *handle, unsigned int node) +{ + struct uaudio_qmi_svc *svc = uaudio_svc; + + if (svc->uaudio_svc_hdl != handle) { + pr_err("handle mismatch\n"); + return; + } + + if (svc->client_connected && svc->client_sq.sq_node == node) { + pr_err("node: %d\n", node); + queue_work(svc->uaudio_wq, &svc->qmi_disconnect_work); + svc->client_sq.sq_node = 0; + svc->client_sq.sq_port = 0; + svc->client_sq.sq_family = 0; + svc->client_connected = false; + } +} + +static void qmi_svc_disconnect_cb(struct qmi_handle *handle, + unsigned int node, unsigned int port) +{ + struct uaudio_qmi_svc *svc; + + if (uaudio_svc == NULL) + return; + + svc = uaudio_svc; + if (svc->uaudio_svc_hdl != handle) { + pr_err("handle mismatch\n"); + return; + } + + if (svc->client_connected && svc->client_sq.sq_node == node && + svc->client_sq.sq_port == port) { + pr_debug("client node:%x port:%x\n", node, port); + queue_work(svc->uaudio_wq, &svc->qmi_disconnect_work); + svc->client_sq.sq_node = 0; + svc->client_sq.sq_port = 0; + svc->client_sq.sq_family = 0; + svc->client_connected = false; + } +} + +static struct qmi_ops uaudio_svc_ops_options = { + .bye = qmi_bye_cb, + .del_client = qmi_svc_disconnect_cb, +}; + +static enum usb_audio_device_speed_enum_v01 +get_speed_info(enum usb_device_speed udev_speed) +{ + switch (udev_speed) { + case USB_SPEED_LOW: + return USB_AUDIO_DEVICE_SPEED_LOW_V01; + case USB_SPEED_FULL: + return USB_AUDIO_DEVICE_SPEED_FULL_V01; + case USB_SPEED_HIGH: + return USB_AUDIO_DEVICE_SPEED_HIGH_V01; + case USB_SPEED_SUPER: + return USB_AUDIO_DEVICE_SPEED_SUPER_V01; + case USB_SPEED_SUPER_PLUS: + return USB_AUDIO_DEVICE_SPEED_SUPER_PLUS_V01; + default: + pr_err("udev speed %d\n", udev_speed); + return USB_AUDIO_DEVICE_SPEED_INVALID_V01; + } +} + +static unsigned long uaudio_get_iova(unsigned long *curr_iova, + size_t *curr_iova_size, struct list_head *head, size_t size) +{ + struct iova_info *info, *new_info = NULL; + struct list_head *curr_head; + unsigned long va = 0; + size_t tmp_size = size; + bool found = false; + + if (size % PAGE_SIZE) { + pr_err("size %zu is not page size multiple\n", size); + goto done; + } + + if (size > *curr_iova_size) { + pr_err("size %zu > curr size %zu\n", size, *curr_iova_size); + goto done; + } + if (*curr_iova_size == 0) { + pr_err("iova mapping is full\n"); + goto done; + } + + list_for_each_entry(info, head, list) { + /* exact size iova_info */ + if (!info->in_use && info->size == size) { + info->in_use = true; + va = info->start_iova; + *curr_iova_size -= size; + found = true; + pr_debug("exact size: %zu found\n", size); + goto done; + } else if (!info->in_use && tmp_size >= info->size) { + if (!new_info) + new_info = info; + pr_debug("partial size: %zu found\n", info->size); + tmp_size -= info->size; + if (tmp_size) + continue; + + va = new_info->start_iova; + for (curr_head = &new_info->list; curr_head != + &info->list; curr_head = curr_head->next) { + new_info = list_entry(curr_head, struct + iova_info, list); + new_info->in_use = true; + } + info->in_use = true; + *curr_iova_size -= size; + found = true; + goto done; + } else { + /* iova region in use */ + new_info = NULL; + tmp_size = size; + } + } + + info = kzalloc(sizeof(struct iova_info), GFP_KERNEL); + if (!info) { + va = 0; + goto done; + } + + va = info->start_iova = *curr_iova; + info->size = size; + info->in_use = true; + *curr_iova += size; + *curr_iova_size -= size; + found = true; + list_add_tail(&info->list, head); + +done: + if (!found) + pr_err("unable to find %zu size iova\n", size); + else + pr_debug("va:0x%08lx curr_iova:0x%08lx curr_iova_size:%zu\n", + va, *curr_iova, *curr_iova_size); + + return va; +} + +static unsigned long uaudio_iommu_map(enum mem_type mtype, bool dma_coherent, + phys_addr_t pa, size_t size, struct sg_table *sgt) +{ + unsigned long va_sg, va = 0; + bool map = true; + int i, ret; + size_t sg_len, total_len = 0; + struct scatterlist *sg; + phys_addr_t pa_sg; + int prot = IOMMU_READ | IOMMU_WRITE; + + if (dma_coherent) + prot |= IOMMU_CACHE; + + switch (mtype) { + case MEM_EVENT_RING: + va = IOVA_BASE; + /* er already mapped */ + if (uaudio_qdev->er_mapped) + map = false; + break; + case MEM_XFER_RING: + va = uaudio_get_iova(&uaudio_qdev->curr_xfer_ring_iova, + &uaudio_qdev->xfer_ring_iova_size, &uaudio_qdev->xfer_ring_list, + size); + break; + case MEM_XFER_BUF: + va = uaudio_get_iova(&uaudio_qdev->curr_xfer_buf_iova, + &uaudio_qdev->xfer_buf_iova_size, &uaudio_qdev->xfer_buf_list, + size); + break; + default: + pr_err("unknown mem type %d\n", mtype); + } + + if (!va || !map) + goto done; + + if (!sgt) + goto skip_sgt_map; + + va_sg = va; + for_each_sg(sgt->sgl, sg, sgt->nents, i) { + sg_len = PAGE_ALIGN(sg->offset + sg->length); + pa_sg = page_to_phys(sg_page(sg)); + ret = iommu_map(uaudio_qdev->domain, va_sg, pa_sg, sg_len, + prot); + if (ret) { + pr_err("mapping failed ret%d\n", ret); + pr_err("type:%d, pa:%pa iova:0x%08lx sg_len:%zu\n", + mtype, &pa_sg, va_sg, sg_len); + uaudio_iommu_unmap(MEM_XFER_BUF, va, size, total_len); + va = 0; + goto done; + } + pr_debug("type:%d map pa:%pa to iova:0x%08lx len:%zu offset:%u\n", + mtype, &pa_sg, va_sg, sg_len, sg->offset); + va_sg += sg_len; + total_len += sg_len; + } + + if (size != total_len) { + pr_err("iova size %zu != mapped iova size %zu\n", size, + total_len); + uaudio_iommu_unmap(MEM_XFER_BUF, va, size, total_len); + va = 0; + } + return va; + +skip_sgt_map: + pr_debug("type:%d map pa:%pa to iova:0x%08lx size:%zu\n", mtype, &pa, + va, size); + + ret = iommu_map(uaudio_qdev->domain, va, pa, size, prot); + if (ret) + pr_err("failed to map pa:%pa iova:0x%lx type:%d ret:%d\n", + &pa, va, mtype, ret); +done: + return va; +} + +static void uaudio_put_iova(unsigned long va, size_t size, struct list_head + *head, size_t *curr_iova_size) +{ + struct iova_info *info; + size_t tmp_size = size; + bool found = false; + + list_for_each_entry(info, head, list) { + if (info->start_iova == va) { + if (!info->in_use) { + pr_err("va %lu is not in use\n", va); + return; + } + found = true; + info->in_use = false; + if (info->size == size) + goto done; + } + + if (found && tmp_size >= info->size) { + info->in_use = false; + tmp_size -= info->size; + if (!tmp_size) + goto done; + } + } + + if (!found) { + pr_err("unable to find the va %lu\n", va); + return; + } +done: + *curr_iova_size += size; + pr_debug("curr_iova_size %zu\n", *curr_iova_size); +} + +static void uaudio_iommu_unmap(enum mem_type mtype, unsigned long va, + size_t iova_size, size_t mapped_iova_size) +{ + size_t umap_size; + bool unmap = true; + + if (!va || !iova_size) + return; + + switch (mtype) { + case MEM_EVENT_RING: + if (uaudio_qdev->er_mapped) + uaudio_qdev->er_mapped = false; + else + unmap = false; + break; + + case MEM_XFER_RING: + uaudio_put_iova(va, iova_size, &uaudio_qdev->xfer_ring_list, + &uaudio_qdev->xfer_ring_iova_size); + break; + case MEM_XFER_BUF: + uaudio_put_iova(va, iova_size, &uaudio_qdev->xfer_buf_list, + &uaudio_qdev->xfer_buf_iova_size); + break; + default: + pr_err("unknown mem type %d\n", mtype); + unmap = false; + } + + if (!unmap || !mapped_iova_size) + return; + + pr_debug("type %d: unmap iova 0x%08lx size %zu\n", mtype, va, + mapped_iova_size); + + umap_size = iommu_unmap(uaudio_qdev->domain, va, mapped_iova_size); + if (umap_size != mapped_iova_size) + pr_err("unmapped size %zu for iova 0x%08lx of mapped size %zu\n", + umap_size, va, mapped_iova_size); +} + +/* looks up alias, if any, for controller DT node and returns the index */ +static int usb_get_controller_id(struct usb_device *udev) +{ + if (!udev || !udev->bus->sysdev || !udev->bus->sysdev->of_node) + pr_err("UDEV NULL\n"); + + if (udev->bus->sysdev && udev->bus->sysdev->of_node) + return of_alias_get_id(udev->bus->sysdev->of_node, "usb"); + + return -ENODEV; +} + +static void uaudio_dev_intf_cleanup(struct usb_device *udev, + struct intf_info *info) +{ + uaudio_iommu_unmap(MEM_XFER_RING, info->data_xfer_ring_va, + info->data_xfer_ring_size, info->data_xfer_ring_size); + info->data_xfer_ring_va = 0; + info->data_xfer_ring_size = 0; + + uaudio_iommu_unmap(MEM_XFER_RING, info->sync_xfer_ring_va, + info->sync_xfer_ring_size, info->sync_xfer_ring_size); + info->sync_xfer_ring_va = 0; + info->sync_xfer_ring_size = 0; + + uaudio_iommu_unmap(MEM_XFER_BUF, info->xfer_buf_va, + info->xfer_buf_size, info->xfer_buf_size); + info->xfer_buf_va = 0; + + usb_free_coherent(udev, info->xfer_buf_size, + info->xfer_buf, info->xfer_buf_pa); + info->xfer_buf_size = 0; + info->xfer_buf = NULL; + info->xfer_buf_pa = 0; + + info->in_use = false; +} + +static void uaudio_event_ring_cleanup_free(struct uaudio_dev *dev) +{ + clear_bit(dev->card_num, &uaudio_qdev->card_slot); + /* all audio devices are disconnected */ + if (!uaudio_qdev->card_slot) { + uaudio_iommu_unmap(MEM_EVENT_RING, IOVA_BASE, PAGE_SIZE, + PAGE_SIZE); + usb_free_interrupter(dev->udev, uaudio_qdev->intr_num); + pr_debug("all audio devices disconnected\n"); + } +} + +static void uaudio_dev_release(struct kref *kref) +{ + struct uaudio_dev *dev = container_of(kref, struct uaudio_dev, kref); + + uaudio_event_ring_cleanup_free(dev); + atomic_set(&dev->in_use, 0); + wake_up(&dev->disconnect_wq); +} + +static struct snd_usb_substream *find_substream(unsigned int card_num, + unsigned int pcm_idx, unsigned int direction) +{ + struct snd_usb_stream *as; + struct snd_usb_substream *subs = NULL; + struct snd_usb_audio *chip; + + chip = uadev[card_num].chip; + if (!chip || atomic_read(&chip->shutdown)) { + pr_debug("%s: instance of usb card # %d does not exist\n", + __func__, card_num); + goto done; + } + + if (pcm_idx >= chip->pcm_devs) { + pr_err("%s: invalid pcm dev number %u > %d\n", __func__, + pcm_idx, chip->pcm_devs); + goto done; + } + + if (direction > SNDRV_PCM_STREAM_CAPTURE) { + pr_err("%s: invalid direction %u\n", __func__, direction); + goto done; + } + + list_for_each_entry(as, &chip->pcm_list, list) { + if (as->pcm_index == pcm_idx) { + subs = &as->substream[direction]; + goto done; + } + } + +done: + if (!subs) + pr_err("%s: substream instance not found\n", __func__); + return subs; +} + +static int info_idx_from_ifnum(int card_num, int intf_num, bool enable) +{ + int i; + + /* + * default index 0 is used when info is allocated upon + * first enable audio stream req for a pcm device + */ + if (enable && !uadev[card_num].info) + return 0; + + for (i = 0; i < uadev[card_num].num_intf; i++) { + if (enable && !uadev[card_num].info[i].in_use) + return i; + else if (!enable && + uadev[card_num].info[i].intf_num == intf_num) + return i; + } + + return -EINVAL; +} + +static int get_data_interval_from_si(struct snd_usb_substream *subs, + u32 service_interval) +{ + unsigned int bus_intval, bus_intval_mult, binterval; + + if (subs->dev->speed >= USB_SPEED_HIGH) + bus_intval = BUS_INTERVAL_HIGHSPEED_AND_ABOVE; + else + bus_intval = BUS_INTERVAL_FULL_SPEED; + + if (service_interval % bus_intval) + return -EINVAL; + + bus_intval_mult = service_interval / bus_intval; + binterval = ffs(bus_intval_mult); + if (!binterval || binterval > MAX_BINTERVAL_ISOC_EP) + return -EINVAL; + + /* check if another bit is set then bail out */ + bus_intval_mult = bus_intval_mult >> binterval; + if (bus_intval_mult) + return -EINVAL; + + return (binterval - 1); +} + +/* maps audio format received over QMI to asound.h based pcm format */ +static snd_pcm_format_t map_pcm_format(enum usb_qmi_audio_format fmt_received) +{ + switch (fmt_received) { + case USB_QMI_PCM_FORMAT_S8: + return SNDRV_PCM_FORMAT_S8; + case USB_QMI_PCM_FORMAT_U8: + return SNDRV_PCM_FORMAT_U8; + case USB_QMI_PCM_FORMAT_S16_LE: + return SNDRV_PCM_FORMAT_S16_LE; + case USB_QMI_PCM_FORMAT_S16_BE: + return SNDRV_PCM_FORMAT_S16_BE; + case USB_QMI_PCM_FORMAT_U16_LE: + return SNDRV_PCM_FORMAT_U16_LE; + case USB_QMI_PCM_FORMAT_U16_BE: + return SNDRV_PCM_FORMAT_U16_BE; + case USB_QMI_PCM_FORMAT_S24_LE: + return SNDRV_PCM_FORMAT_S24_LE; + case USB_QMI_PCM_FORMAT_S24_BE: + return SNDRV_PCM_FORMAT_S24_BE; + case USB_QMI_PCM_FORMAT_U24_LE: + return SNDRV_PCM_FORMAT_U24_LE; + case USB_QMI_PCM_FORMAT_U24_BE: + return SNDRV_PCM_FORMAT_U24_BE; + case USB_QMI_PCM_FORMAT_S24_3LE: + return SNDRV_PCM_FORMAT_S24_3LE; + case USB_QMI_PCM_FORMAT_S24_3BE: + return SNDRV_PCM_FORMAT_S24_3BE; + case USB_QMI_PCM_FORMAT_U24_3LE: + return SNDRV_PCM_FORMAT_U24_3LE; + case USB_QMI_PCM_FORMAT_U24_3BE: + return SNDRV_PCM_FORMAT_U24_3BE; + case USB_QMI_PCM_FORMAT_S32_LE: + return SNDRV_PCM_FORMAT_S32_LE; + case USB_QMI_PCM_FORMAT_S32_BE: + return SNDRV_PCM_FORMAT_S32_BE; + case USB_QMI_PCM_FORMAT_U32_LE: + return SNDRV_PCM_FORMAT_U32_LE; + case USB_QMI_PCM_FORMAT_U32_BE: + return SNDRV_PCM_FORMAT_U32_BE; + default: + /* + * We expect the caller to do input validation so we should + * never hit this. But we do have to return a proper + * snd_pcm_format_t value due to the __bitwise attribute; so + * just return the equivalent of 0 in case of bad input. + */ + return SNDRV_PCM_FORMAT_S8; + } +} + +static int snd_interval_refine_set(struct snd_interval *i, unsigned int val) +{ + struct snd_interval t; + + t.empty = 0; + t.min = t.max = val; + t.openmin = t.openmax = 0; + t.integer = 1; + return snd_interval_refine(i, &t); +} + +static int _snd_pcm_hw_param_set(struct snd_pcm_hw_params *params, + snd_pcm_hw_param_t var, unsigned int val, + int dir) +{ + int changed; + + if (hw_is_mask(var)) { + struct snd_mask *m = hw_param_mask(params, var); + + if (val == 0 && dir < 0) { + changed = -EINVAL; + snd_mask_none(m); + } else { + if (dir > 0) + val++; + else if (dir < 0) + val--; + changed = snd_mask_refine_set( + hw_param_mask(params, var), val); + } + } else if (hw_is_interval(var)) { + struct snd_interval *i = hw_param_interval(params, var); + + if (val == 0 && dir < 0) { + changed = -EINVAL; + snd_interval_none(i); + } else if (dir == 0) + changed = snd_interval_refine_set(i, val); + else { + struct snd_interval t; + + t.openmin = 1; + t.openmax = 1; + t.empty = 0; + t.integer = 0; + if (dir < 0) { + t.min = val - 1; + t.max = val; + } else { + t.min = val; + t.max = val+1; + } + changed = snd_interval_refine(i, &t); + } + } else + return -EINVAL; + if (changed) { + params->cmask |= 1 << var; + params->rmask |= 1 << var; + } + return changed; +} + +static void disable_audio_stream(struct snd_usb_substream *subs) +{ + struct snd_usb_audio *chip = subs->stream->chip; + + if (subs->data_endpoint || subs->sync_endpoint) { + close_endpoints(chip, subs); + + mutex_lock(&chip->mutex); + subs->cur_audiofmt = NULL; + mutex_unlock(&chip->mutex); + } + + snd_usb_autosuspend(chip); +} + +static int enable_audio_stream(struct snd_usb_substream *subs, + snd_pcm_format_t pcm_format, + unsigned int channels, unsigned int cur_rate, + int datainterval) +{ + struct snd_usb_audio *chip = subs->stream->chip; + struct snd_pcm_hw_params params; + const struct audioformat *fmt; + int ret; + + _snd_pcm_hw_params_any(¶ms); + _snd_pcm_hw_param_set(¶ms, SNDRV_PCM_HW_PARAM_FORMAT, + pcm_format, 0); + _snd_pcm_hw_param_set(¶ms, SNDRV_PCM_HW_PARAM_CHANNELS, + channels, 0); + _snd_pcm_hw_param_set(¶ms, SNDRV_PCM_HW_PARAM_RATE, + cur_rate, 0); + + pm_runtime_barrier(&chip->intf[0]->dev); + snd_usb_autoresume(chip); + + fmt = find_format(&subs->fmt_list, pcm_format, cur_rate, + channels, datainterval, subs); + if (!fmt) { + dev_err(&subs->dev->dev, "cannot find format: format = %#x, rate = %d, channels = %d\n", + pcm_format, cur_rate, channels); + return -EINVAL; + } + + if (atomic_read(&chip->shutdown)) { + pr_err("chip already shutdown\n"); + ret = -ENODEV; + } else { + if (subs->data_endpoint) + close_endpoints(chip, subs); + + subs->data_endpoint = snd_usb_endpoint_open(chip, fmt, + ¶ms, false); + if (!subs->data_endpoint) { + pr_err("failed to open data endpoint\n"); + return -EINVAL; + } + + if (fmt->sync_ep) { + subs->sync_endpoint = snd_usb_endpoint_open(chip, + fmt, ¶ms, true); + if (!subs->sync_endpoint) { + pr_err("failed to open sync endpoint\n"); + return -EINVAL; + } + + subs->data_endpoint->sync_source = subs->sync_endpoint; + } + + mutex_lock(&chip->mutex); + subs->cur_audiofmt = fmt; + mutex_unlock(&chip->mutex); + + if (subs->sync_endpoint) { + ret = snd_usb_endpoint_prepare(chip, subs->sync_endpoint); + if (ret < 0) + return ret; + } + + ret = snd_usb_endpoint_prepare(chip, subs->data_endpoint); + if (ret < 0) + return ret; + + pr_info("selected %s iface:%d altsetting:%d datainterval:%dus\n", + subs->direction ? "capture" : "playback", + fmt->iface, fmt->altsetting, + (1 << fmt->datainterval) * + (subs->dev->speed >= USB_SPEED_HIGH ? + BUS_INTERVAL_HIGHSPEED_AND_ABOVE : + BUS_INTERVAL_FULL_SPEED)); + } + + return 0; +} + +static int prepare_qmi_response(struct snd_usb_substream *subs, + struct qmi_uaudio_stream_req_msg_v01 *req_msg, + struct qmi_uaudio_stream_resp_msg_v01 *resp, int info_idx) +{ + struct usb_interface *iface; + struct usb_host_interface *alts; + struct usb_interface_descriptor *altsd; + struct usb_interface_assoc_descriptor *assoc; + struct usb_host_endpoint *ep; + struct uac_format_type_i_continuous_descriptor *fmt; + struct uac_format_type_i_discrete_descriptor *fmt_v1; + struct uac_format_type_i_ext_descriptor *fmt_v2; + struct uac1_as_header_descriptor *as; + int ret; + int protocol, card_num, pcm_dev_num; + void *hdr_ptr; + u8 *xfer_buf; + unsigned int data_ep_pipe = 0, sync_ep_pipe = 0; + u32 len, mult, remainder, xfer_buf_len; + unsigned long va, tr_data_va = 0, tr_sync_va = 0; + phys_addr_t xhci_pa, xfer_buf_pa, tr_data_pa = 0, tr_sync_pa = 0; + dma_addr_t dma; + struct sg_table sgt; + bool dma_coherent; + + iface = usb_ifnum_to_if(subs->dev, subs->cur_audiofmt->iface); + if (!iface) { + pr_err("interface # %d does not exist\n", subs->cur_audiofmt->iface); + ret = -ENODEV; + goto err; + } + + assoc = iface->intf_assoc; + pcm_dev_num = (req_msg->usb_token & SND_PCM_DEV_NUM_MASK) >> 8; + xfer_buf_len = req_msg->xfer_buff_size; + card_num = uaudio_qdev->last_card_num; + + alts = &iface->altsetting[subs->cur_audiofmt->altset_idx]; + altsd = get_iface_desc(alts); + protocol = altsd->bInterfaceProtocol; + + /* get format type */ + if (protocol != UAC_VERSION_3) { + fmt = snd_usb_find_csint_desc(alts->extra, alts->extralen, NULL, + UAC_FORMAT_TYPE); + if (!fmt) { + pr_err("%u:%d : no UAC_FORMAT_TYPE desc\n", + subs->cur_audiofmt->iface, + subs->cur_audiofmt->altset_idx); + ret = -ENODEV; + goto err; + } + } + + if (!uadev[card_num].ctrl_intf) { + pr_err("audio ctrl intf info not cached\n"); + ret = -ENODEV; + goto err; + } + + if (protocol != UAC_VERSION_3) { + hdr_ptr = snd_usb_find_csint_desc(uadev[card_num].ctrl_intf->extra, + uadev[card_num].ctrl_intf->extralen, NULL, + UAC_HEADER); + if (!hdr_ptr) { + pr_err("no UAC_HEADER desc\n"); + ret = -ENODEV; + goto err; + } + } + + if (protocol == UAC_VERSION_1) { + struct uac1_ac_header_descriptor *uac1_hdr = hdr_ptr; + + as = snd_usb_find_csint_desc(alts->extra, alts->extralen, NULL, + UAC_AS_GENERAL); + if (!as) { + pr_err("%u:%d : no UAC_AS_GENERAL desc\n", + subs->cur_audiofmt->iface, + subs->cur_audiofmt->altset_idx); + ret = -ENODEV; + goto err; + } + resp->data_path_delay = as->bDelay; + resp->data_path_delay_valid = 1; + fmt_v1 = (struct uac_format_type_i_discrete_descriptor *)fmt; + resp->usb_audio_subslot_size = fmt_v1->bSubframeSize; + resp->usb_audio_subslot_size_valid = 1; + + resp->usb_audio_spec_revision = le16_to_cpu(uac1_hdr->bcdADC); + resp->usb_audio_spec_revision_valid = 1; + } else if (protocol == UAC_VERSION_2) { + struct uac2_ac_header_descriptor *uac2_hdr = hdr_ptr; + + fmt_v2 = (struct uac_format_type_i_ext_descriptor *)fmt; + resp->usb_audio_subslot_size = fmt_v2->bSubslotSize; + resp->usb_audio_subslot_size_valid = 1; + + resp->usb_audio_spec_revision = le16_to_cpu(uac2_hdr->bcdADC); + resp->usb_audio_spec_revision_valid = 1; + } else if (protocol == UAC_VERSION_3) { + if (assoc->bFunctionSubClass == + UAC3_FUNCTION_SUBCLASS_FULL_ADC_3_0) { + pr_err("full adc is not supported\n"); + ret = -EINVAL; + } + + switch (le16_to_cpu(get_endpoint(alts, 0)->wMaxPacketSize)) { + case UAC3_BADD_EP_MAXPSIZE_SYNC_MONO_16: + case UAC3_BADD_EP_MAXPSIZE_SYNC_STEREO_16: + case UAC3_BADD_EP_MAXPSIZE_ASYNC_MONO_16: + case UAC3_BADD_EP_MAXPSIZE_ASYNC_STEREO_16: { + resp->usb_audio_subslot_size = 0x2; + break; + } + + case UAC3_BADD_EP_MAXPSIZE_SYNC_MONO_24: + case UAC3_BADD_EP_MAXPSIZE_SYNC_STEREO_24: + case UAC3_BADD_EP_MAXPSIZE_ASYNC_MONO_24: + case UAC3_BADD_EP_MAXPSIZE_ASYNC_STEREO_24: { + resp->usb_audio_subslot_size = 0x3; + break; + } + + default: + pr_err("%d: %u: Invalid wMaxPacketSize\n", + subs->cur_audiofmt->iface, + subs->cur_audiofmt->altset_idx); + ret = -EINVAL; + goto err; + } + resp->usb_audio_subslot_size_valid = 1; + } else { + pr_err("unknown protocol version %x\n", protocol); + ret = -ENODEV; + goto err; + } + + resp->slot_id = subs->dev->slot_id; + resp->slot_id_valid = 1; + + memcpy(&resp->std_as_opr_intf_desc, &alts->desc, sizeof(alts->desc)); + resp->std_as_opr_intf_desc_valid = 1; + + ep = usb_pipe_endpoint(subs->dev, subs->data_endpoint->pipe); + if (!ep) { + pr_err("data ep # %d context is null\n", + subs->data_endpoint->ep_num); + ret = -ENODEV; + goto err; + } + data_ep_pipe = subs->data_endpoint->pipe; + memcpy(&resp->std_as_data_ep_desc, &ep->desc, sizeof(ep->desc)); + resp->std_as_data_ep_desc_valid = 1; + + tr_data_pa = usb_hcd_get_transfer_resource(subs->dev, ep, &dma); + if (!tr_data_pa) { + pr_err("failed to get data ep ring dma address\n"); + ret = -ENODEV; + goto err; + } + resp->xhci_mem_info.tr_data.pa = dma; + + if (subs->sync_endpoint) { + ep = usb_pipe_endpoint(subs->dev, subs->sync_endpoint->pipe); + if (!ep) { + pr_err("implicit fb on data ep\n"); + goto skip_sync_ep; + } + sync_ep_pipe = subs->sync_endpoint->pipe; + memcpy(&resp->std_as_sync_ep_desc, &ep->desc, sizeof(ep->desc)); + resp->std_as_sync_ep_desc_valid = 1; + + tr_sync_pa = usb_hcd_get_transfer_resource(subs->dev, ep, &dma); + if (!tr_sync_pa) { + pr_err("failed to get sync ep ring dma address\n"); + ret = -ENODEV; + goto err; + } + resp->xhci_mem_info.tr_sync.pa = dma; + } + +skip_sync_ep: + resp->interrupter_num = uaudio_qdev->intr_num; + resp->interrupter_num_valid = 1; + resp->controller_num_valid = 0; + ret = usb_get_controller_id(subs->dev); + if (ret >= 0) { + resp->controller_num = ret; + resp->controller_num_valid = 1; + } + /* map xhci data structures PA memory to iova */ + dma_coherent = dev_is_dma_coherent(subs->dev->bus->sysdev); + + /* event ring */ + xhci_pa = usb_set_interruper(subs->dev, resp->interrupter_num, &dma); + if (!xhci_pa) { + pr_err("failed to get sec event ring dma address\n"); + ret = -ENODEV; + goto free_sec_ring; + } + + va = uaudio_iommu_map(MEM_EVENT_RING, dma_coherent, xhci_pa, PAGE_SIZE, + NULL); + if (!va) { + ret = -ENOMEM; + goto free_sec_ring; + } + + resp->xhci_mem_info.evt_ring.va = PREPEND_SID_TO_IOVA(va, + uaudio_qdev->sid); + resp->xhci_mem_info.evt_ring.pa = dma; + resp->xhci_mem_info.evt_ring.size = PAGE_SIZE; + uaudio_qdev->er_mapped = true; + + resp->speed_info = get_speed_info(subs->dev->speed); + if (resp->speed_info == USB_AUDIO_DEVICE_SPEED_INVALID_V01) { + ret = -ENODEV; + goto unmap_er; + } + + resp->speed_info_valid = 1; + + /* data transfer ring */ + va = uaudio_iommu_map(MEM_XFER_RING, dma_coherent, tr_data_pa, + PAGE_SIZE, NULL); + if (!va) { + ret = -ENOMEM; + goto unmap_er; + } + + tr_data_va = va; + resp->xhci_mem_info.tr_data.va = PREPEND_SID_TO_IOVA(va, + uaudio_qdev->sid); + resp->xhci_mem_info.tr_data.size = PAGE_SIZE; + + /* sync transfer ring */ + if (!resp->xhci_mem_info.tr_sync.pa) + goto skip_sync; + + xhci_pa = resp->xhci_mem_info.tr_sync.pa; + va = uaudio_iommu_map(MEM_XFER_RING, dma_coherent, tr_sync_pa, + PAGE_SIZE, NULL); + if (!va) { + ret = -ENOMEM; + goto unmap_data; + } + + tr_sync_va = va; + resp->xhci_mem_info.tr_sync.va = PREPEND_SID_TO_IOVA(va, + uaudio_qdev->sid); + resp->xhci_mem_info.tr_sync.size = PAGE_SIZE; + +skip_sync: + /* xfer buffer, multiple of 4K only */ + if (!xfer_buf_len) + xfer_buf_len = PAGE_SIZE; + + mult = xfer_buf_len / PAGE_SIZE; + remainder = xfer_buf_len % PAGE_SIZE; + len = mult * PAGE_SIZE; + len += remainder ? PAGE_SIZE : 0; + + if (len > MAX_XFER_BUFF_LEN) { + pr_err("req buf len %d > max buf len %lu, setting %lu\n", + len, MAX_XFER_BUFF_LEN, MAX_XFER_BUFF_LEN); + len = MAX_XFER_BUFF_LEN; + } + + xfer_buf = usb_alloc_coherent(subs->dev, len, GFP_KERNEL, &xfer_buf_pa); + if (!xfer_buf) { + ret = -ENOMEM; + goto unmap_sync; + } + + dma_get_sgtable(subs->dev->bus->sysdev, &sgt, xfer_buf, xfer_buf_pa, + len); + va = uaudio_iommu_map(MEM_XFER_BUF, dma_coherent, xfer_buf_pa, len, + &sgt); + if (!va) { + ret = -ENOMEM; + goto unmap_sync; + } + + resp->xhci_mem_info.xfer_buff.pa = xfer_buf_pa; + resp->xhci_mem_info.xfer_buff.size = len; + + resp->xhci_mem_info.xfer_buff.va = PREPEND_SID_TO_IOVA(va, + uaudio_qdev->sid); + + resp->xhci_mem_info_valid = 1; + + sg_free_table(&sgt); + + if (!atomic_read(&uadev[card_num].in_use)) { + kref_init(&uadev[card_num].kref); + init_waitqueue_head(&uadev[card_num].disconnect_wq); + uadev[card_num].num_intf = + subs->dev->config->desc.bNumInterfaces; + uadev[card_num].info = kcalloc(uadev[card_num].num_intf, + sizeof(struct intf_info), GFP_KERNEL); + if (!uadev[card_num].info) { + ret = -ENOMEM; + goto unmap_sync; + } + uadev[card_num].udev = subs->dev; + atomic_set(&uadev[card_num].in_use, 1); + } else { + kref_get(&uadev[card_num].kref); + } + + uadev[card_num].card_num = card_num; + uadev[card_num].usb_core_id = resp->controller_num; + + /* cache intf specific info to use it for unmap and free xfer buf */ + uadev[card_num].info[info_idx].data_xfer_ring_va = tr_data_va; + uadev[card_num].info[info_idx].data_xfer_ring_size = PAGE_SIZE; + uadev[card_num].info[info_idx].sync_xfer_ring_va = tr_sync_va; + uadev[card_num].info[info_idx].sync_xfer_ring_size = PAGE_SIZE; + uadev[card_num].info[info_idx].xfer_buf_va = va; + uadev[card_num].info[info_idx].xfer_buf_pa = xfer_buf_pa; + uadev[card_num].info[info_idx].xfer_buf_size = len; + uadev[card_num].info[info_idx].data_ep_pipe = data_ep_pipe; + uadev[card_num].info[info_idx].sync_ep_pipe = sync_ep_pipe; + uadev[card_num].info[info_idx].xfer_buf = xfer_buf; + uadev[card_num].info[info_idx].pcm_card_num = card_num; + uadev[card_num].info[info_idx].pcm_dev_num = pcm_dev_num; + uadev[card_num].info[info_idx].direction = subs->direction; + uadev[card_num].info[info_idx].intf_num = subs->cur_audiofmt->iface; + uadev[card_num].info[info_idx].in_use = true; + + set_bit(card_num, &uaudio_qdev->card_slot); + + return 0; + +unmap_sync: + usb_free_coherent(subs->dev, len, xfer_buf, xfer_buf_pa); + uaudio_iommu_unmap(MEM_XFER_RING, tr_sync_va, PAGE_SIZE, PAGE_SIZE); +unmap_data: + uaudio_iommu_unmap(MEM_XFER_RING, tr_data_va, PAGE_SIZE, PAGE_SIZE); +unmap_er: + uaudio_iommu_unmap(MEM_EVENT_RING, IOVA_BASE, PAGE_SIZE, PAGE_SIZE); +free_sec_ring: + usb_free_interrupter(subs->dev, uaudio_qdev->intr_num); +err: + return ret; +} + +static void handle_uaudio_stream_req(struct qmi_handle *handle, + struct sockaddr_qrtr *sq, + struct qmi_txn *txn, + const void *decoded_msg) +{ + struct qmi_uaudio_stream_req_msg_v01 *req_msg; + struct qmi_uaudio_stream_resp_msg_v01 resp = {{0}, 0}; + struct snd_usb_substream *subs; + struct snd_usb_audio *chip = NULL; + struct uaudio_qmi_svc *svc = uaudio_svc; + struct intf_info *info; + struct usb_host_endpoint *ep; + u8 pcm_card_num, pcm_dev_num, direction; + int info_idx = -EINVAL, datainterval = -EINVAL, ret = 0; + + if (!svc->client_connected) { + svc->client_sq = *sq; + svc->client_connected = true; + } + + req_msg = (struct qmi_uaudio_stream_req_msg_v01 *)decoded_msg; + if (!req_msg->audio_format_valid || !req_msg->bit_rate_valid || + !req_msg->number_of_ch_valid || !req_msg->xfer_buff_size_valid) { + pr_err("invalid request msg\n"); + ret = -EINVAL; + goto response; + } + + direction = (req_msg->usb_token & SND_PCM_STREAM_DIRECTION); + pcm_dev_num = (req_msg->usb_token & SND_PCM_DEV_NUM_MASK) >> 8; + pcm_card_num = req_msg->enable ? uaudio_qdev->last_card_num : + ffs(uaudio_qdev->card_slot) - 1; + + pr_info("card#:%d dev#:%d dir:%d en:%d fmt:%d rate:%d #ch:%d\n", + pcm_card_num, pcm_dev_num, (req_msg->usb_token & SND_PCM_STREAM_DIRECTION), + req_msg->enable, req_msg->audio_format, req_msg->bit_rate, + req_msg->number_of_ch); + + if (pcm_card_num >= SNDRV_CARDS) { + pr_err("invalid card # %u", pcm_card_num); + ret = -EINVAL; + goto response; + } + + if (req_msg->audio_format > USB_QMI_PCM_FORMAT_U32_BE) { + pr_err("unsupported pcm format received %d\n", + req_msg->audio_format); + ret = -EINVAL; + goto response; + } + + subs = find_substream(pcm_card_num, pcm_dev_num, direction); + chip = uadev[pcm_card_num].chip; + if (!subs || !chip || atomic_read(&chip->shutdown)) { + pr_err("can't find substream for card# %u, dev# %u dir%u\n", + pcm_card_num, pcm_dev_num, direction); + ret = -ENODEV; + goto response; + } + + info_idx = info_idx_from_ifnum(pcm_card_num, subs->cur_audiofmt ? + subs->cur_audiofmt->iface : -1, req_msg->enable); + if (atomic_read(&chip->shutdown) || !subs->stream || !subs->stream->pcm + || !subs->stream->chip) { + pr_err("chip or sub not available: shutdown:%d stream:%p pcm:%p chip:%p\n", + atomic_read(&chip->shutdown), subs->stream, + subs->stream->pcm, subs->stream->chip); + ret = -ENODEV; + goto response; + } + + if (req_msg->enable) { + if (info_idx < 0) { + pr_err("interface# %d already in use card# %d\n", + subs->cur_audiofmt->iface, pcm_card_num); + ret = -EBUSY; + goto response; + } + } + + if (req_msg->service_interval_valid) { + ret = get_data_interval_from_si(subs, + req_msg->service_interval); + if (ret == -EINVAL) { + pr_err("invalid service interval %u\n", + req_msg->service_interval); + goto response; + } + + datainterval = ret; + } + + uadev[pcm_card_num].ctrl_intf = chip->ctrl_intf; + + if (req_msg->enable) { + ret = enable_audio_stream(subs, + map_pcm_format(req_msg->audio_format), + req_msg->number_of_ch, req_msg->bit_rate, + datainterval); + + if (!ret) + ret = prepare_qmi_response(subs, req_msg, &resp, + info_idx); + } else { + info = &uadev[pcm_card_num].info[info_idx]; + if (info->data_ep_pipe) { + ep = usb_pipe_endpoint(uadev[pcm_card_num].udev, + info->data_ep_pipe); + if (!ep) + pr_err("no data ep\n"); + else + usb_hcd_stop_endpoint(uadev[pcm_card_num].udev, + ep); + info->data_ep_pipe = 0; + } + + if (info->sync_ep_pipe) { + ep = usb_pipe_endpoint(uadev[pcm_card_num].udev, + info->sync_ep_pipe); + if (!ep) + pr_err("no sync ep\n"); + else + usb_hcd_stop_endpoint(uadev[pcm_card_num].udev, + ep); + info->sync_ep_pipe = 0; + } + + disable_audio_stream(subs); + } + +response: + if (!req_msg->enable && ret != -EINVAL && ret != -ENODEV) { + mutex_lock(&chip->mutex); + if (info_idx >= 0) { + info = &uadev[pcm_card_num].info[info_idx]; + uaudio_dev_intf_cleanup( + uadev[pcm_card_num].udev, + info); + pr_info("release resources: intf# %d card# %d\n", + info->intf_num, pcm_card_num); + } + if (atomic_read(&uadev[pcm_card_num].in_use)) + kref_put(&uadev[pcm_card_num].kref, + uaudio_dev_release); + mutex_unlock(&chip->mutex); + } + + resp.usb_token = req_msg->usb_token; + resp.usb_token_valid = 1; + resp.internal_status = ret; + resp.internal_status_valid = 1; + resp.status = ret ? USB_AUDIO_STREAM_REQ_FAILURE_V01 : ret; + resp.status_valid = 1; + ret = qmi_send_response(svc->uaudio_svc_hdl, sq, txn, + QMI_UAUDIO_STREAM_RESP_V01, + QMI_UAUDIO_STREAM_RESP_MSG_V01_MAX_MSG_LEN, + qmi_uaudio_stream_resp_msg_v01_ei, &resp); +} + +static struct qmi_msg_handler uaudio_stream_req_handlers = { + .type = QMI_REQUEST, + .msg_id = QMI_UAUDIO_STREAM_REQ_V01, + .ei = qmi_uaudio_stream_req_msg_v01_ei, + .decoded_size = QMI_UAUDIO_STREAM_REQ_MSG_V01_MAX_MSG_LEN, + .fn = handle_uaudio_stream_req, +}; + +int qc_usb_audio_offload_init_qmi_dev(void) +{ + struct q6usb_offload *data; + + uaudio_qdev = kzalloc(sizeof(struct uaudio_qmi_dev), + GFP_KERNEL); + if (!uaudio_qdev) + return -ENOMEM; + + /* initialize xfer ring and xfer buf iova list */ + INIT_LIST_HEAD(&uaudio_qdev->xfer_ring_list); + uaudio_qdev->curr_xfer_ring_iova = IOVA_XFER_RING_BASE; + uaudio_qdev->xfer_ring_iova_size = + IOVA_XFER_RING_MAX - IOVA_XFER_RING_BASE; + + INIT_LIST_HEAD(&uaudio_qdev->xfer_buf_list); + uaudio_qdev->curr_xfer_buf_iova = IOVA_XFER_BUF_BASE; + uaudio_qdev->xfer_buf_iova_size = + IOVA_XFER_BUF_MAX - IOVA_XFER_BUF_BASE; + + data = snd_soc_usb_get_priv_data(); + if (data) { + uaudio_qdev->domain = data->domain; + uaudio_qdev->intr_num = data->intr_num; + uaudio_qdev->sid = data->sid; + uaudio_qdev->dev = data->dev; + } + + return 0; +} + +void qc_usb_audio_offload_probe(struct usb_interface *intf, struct snd_usb_audio *chip) +{ + if (!uaudio_qdev) + qc_usb_audio_offload_init_qmi_dev(); + + atomic_inc(&uaudio_qdev->qdev_in_use); + uadev[chip->card->number].chip = chip; + uaudio_qdev->last_card_num = chip->card->number; + snd_soc_usb_connect(chip->index); +} + +static void uaudio_dev_cleanup(struct uaudio_dev *dev) +{ + int if_idx; + + if (!dev->udev) { + pr_info("USB audio device memory is already freed.\n"); + return; + } + + /* free xfer buffer and unmap xfer ring and buf per interface */ + for (if_idx = 0; if_idx < dev->num_intf; if_idx++) { + if (!dev->info[if_idx].in_use) + continue; + uaudio_dev_intf_cleanup(dev->udev, &dev->info[if_idx]); + pr_debug("release resources: intf# %d card# %d\n", + dev->info[if_idx].intf_num, dev->card_num); + } + + dev->num_intf = 0; + + /* free interface info */ + kfree(dev->info); + dev->info = NULL; + uaudio_event_ring_cleanup_free(dev); + dev->udev = NULL; +} + +static void qc_usb_audio_cleanup_qmi_dev(void) +{ + kfree(uaudio_qdev); + uaudio_qdev = NULL; +} + +void qc_usb_audio_offload_disconnect(struct usb_interface *intf) +{ + int ret; + struct snd_usb_audio *chip = usb_get_intfdata(intf); + struct uaudio_dev *dev; + int card_num; + struct uaudio_qmi_svc *svc = uaudio_svc; + struct qmi_uaudio_stream_ind_msg_v01 disconnect_ind = {0}; + + if (!chip) { + pr_err("chip is NULL\n"); + return; + } + + card_num = chip->card->number; + pr_debug("intf: %s: %p chip: %p card: %d\n", dev_name(&intf->dev), + intf, chip, card_num); + + if (card_num >= SNDRV_CARDS) { + pr_err("invalid card number\n"); + return; + } + + dev = &uadev[card_num]; + + /* clean up */ + if (!dev->udev) { + pr_debug("no clean up required\n"); + goto done; + } + + if (atomic_read(&dev->in_use)) { + pr_debug("sending qmi indication disconnect\n"); + pr_debug("sq->sq_family:%x sq->sq_node:%x sq->sq_port:%x\n", + svc->client_sq.sq_family, + svc->client_sq.sq_node, svc->client_sq.sq_port); + disconnect_ind.dev_event = USB_AUDIO_DEV_DISCONNECT_V01; + disconnect_ind.slot_id = dev->udev->slot_id; + disconnect_ind.controller_num = dev->usb_core_id; + disconnect_ind.controller_num_valid = 1; + ret = qmi_send_indication(svc->uaudio_svc_hdl, &svc->client_sq, + QMI_UAUDIO_STREAM_IND_V01, + QMI_UAUDIO_STREAM_IND_MSG_V01_MAX_MSG_LEN, + qmi_uaudio_stream_ind_msg_v01_ei, + &disconnect_ind); + if (ret < 0) + pr_err("qmi send failed with err: %d\n", ret); + + ret = wait_event_interruptible_timeout(dev->disconnect_wq, + !atomic_read(&dev->in_use), + msecs_to_jiffies(DEV_RELEASE_WAIT_TIMEOUT)); + if (!ret) { + pr_err("timeout while waiting for dev_release\n"); + atomic_set(&dev->in_use, 0); + } else if (ret < 0) { + pr_err("failed with ret %d\n", ret); + atomic_set(&dev->in_use, 0); + } + } + + uaudio_dev_cleanup(dev); +done: + uadev[card_num].chip = NULL; + + atomic_dec(&uaudio_qdev->qdev_in_use); + if (!atomic_read(&uaudio_qdev->qdev_in_use)) { + snd_soc_usb_disconnect(); + qc_usb_audio_cleanup_qmi_dev(); + } +} + +struct snd_usb_vendor_ops offload_ops = { + .connect_cb = qc_usb_audio_offload_probe, + .disconnect_cb = qc_usb_audio_offload_disconnect, +}; + +static int __init qc_usb_audio_offload_init(void) +{ + struct uaudio_qmi_svc *svc; + int ret; + + ret = snd_usb_register_vendor_ops(&offload_ops); + if (ret < 0) + return ret; + + svc = kzalloc(sizeof(struct uaudio_qmi_svc), GFP_KERNEL); + if (!svc) { + ret = -ENOMEM; + goto unreg_ops; + } + + svc->uaudio_wq = create_singlethread_workqueue("uaudio_svc"); + if (!svc->uaudio_wq) { + ret = -ENOMEM; + goto free_svc; + } + + svc->uaudio_svc_hdl = kzalloc(sizeof(struct qmi_handle), GFP_KERNEL); + if (!svc->uaudio_svc_hdl) { + ret = -ENOMEM; + goto free_wq; + } + + ret = qmi_handle_init(svc->uaudio_svc_hdl, + QMI_UAUDIO_STREAM_REQ_MSG_V01_MAX_MSG_LEN, + &uaudio_svc_ops_options, + &uaudio_stream_req_handlers); + ret = qmi_add_server(svc->uaudio_svc_hdl, UAUDIO_STREAM_SERVICE_ID_V01, + UAUDIO_STREAM_SERVICE_VERS_V01, 0); + + INIT_WORK(&svc->qmi_disconnect_work, qmi_disconnect_work); + uaudio_svc = svc; + + return 0; + +free_wq: + destroy_workqueue(svc->uaudio_wq); +free_svc: + kfree(svc); +unreg_ops: + snd_usb_unregister_vendor_ops(); + + return ret; +} + +static void __exit qc_usb_audio_offload_exit(void) +{ + struct uaudio_qmi_svc *svc = uaudio_svc; + + qmi_handle_release(svc->uaudio_svc_hdl); + flush_workqueue(svc->uaudio_wq); + destroy_workqueue(svc->uaudio_wq); + kfree(svc); + uaudio_svc = NULL; + snd_usb_unregister_vendor_ops(); +} + +module_init(qc_usb_audio_offload_init); +module_exit(qc_usb_audio_offload_exit); + +MODULE_DESCRIPTION("QC USB Audio Offloading"); +MODULE_LICENSE("GPL"); diff --git a/sound/usb/qcom/usb_audio_qmi_v01.c b/sound/usb/qcom/usb_audio_qmi_v01.c new file mode 100644 index 000000000000..95ae434f0a41 --- /dev/null +++ b/sound/usb/qcom/usb_audio_qmi_v01.c @@ -0,0 +1,892 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#include <linux/soc/qcom/qmi.h> + +#include "usb_audio_qmi_v01.h" + +static struct qmi_elem_info mem_info_v01_ei[] = { + { + .data_type = QMI_UNSIGNED_8_BYTE, + .elem_len = 1, + .elem_size = sizeof(u64), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct mem_info_v01, va), + }, + { + .data_type = QMI_UNSIGNED_8_BYTE, + .elem_len = 1, + .elem_size = sizeof(u64), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct mem_info_v01, pa), + }, + { + .data_type = QMI_UNSIGNED_4_BYTE, + .elem_len = 1, + .elem_size = sizeof(u32), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct mem_info_v01, size), + }, + { + .data_type = QMI_EOTI, + .array_type = NO_ARRAY, + .tlv_type = QMI_COMMON_TLV_TYPE, + }, +}; + +static struct qmi_elem_info apps_mem_info_v01_ei[] = { + { + .data_type = QMI_STRUCT, + .elem_len = 1, + .elem_size = sizeof(struct mem_info_v01), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct apps_mem_info_v01, evt_ring), + .ei_array = mem_info_v01_ei, + }, + { + .data_type = QMI_STRUCT, + .elem_len = 1, + .elem_size = sizeof(struct mem_info_v01), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct apps_mem_info_v01, tr_data), + .ei_array = mem_info_v01_ei, + }, + { + .data_type = QMI_STRUCT, + .elem_len = 1, + .elem_size = sizeof(struct mem_info_v01), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct apps_mem_info_v01, tr_sync), + .ei_array = mem_info_v01_ei, + }, + { + .data_type = QMI_STRUCT, + .elem_len = 1, + .elem_size = sizeof(struct mem_info_v01), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct apps_mem_info_v01, xfer_buff), + .ei_array = mem_info_v01_ei, + }, + { + .data_type = QMI_STRUCT, + .elem_len = 1, + .elem_size = sizeof(struct mem_info_v01), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct apps_mem_info_v01, dcba), + .ei_array = mem_info_v01_ei, + }, + { + .data_type = QMI_EOTI, + .array_type = NO_ARRAY, + .tlv_type = QMI_COMMON_TLV_TYPE, + }, +}; + +static struct qmi_elem_info usb_endpoint_descriptor_v01_ei[] = { + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct usb_endpoint_descriptor_v01, + bLength), + }, + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct usb_endpoint_descriptor_v01, + bDescriptorType), + }, + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct usb_endpoint_descriptor_v01, + bEndpointAddress), + }, + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct usb_endpoint_descriptor_v01, + bmAttributes), + }, + { + .data_type = QMI_UNSIGNED_2_BYTE, + .elem_len = 1, + .elem_size = sizeof(u16), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct usb_endpoint_descriptor_v01, + wMaxPacketSize), + }, + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct usb_endpoint_descriptor_v01, + bInterval), + }, + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct usb_endpoint_descriptor_v01, + bRefresh), + }, + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct usb_endpoint_descriptor_v01, + bSynchAddress), + }, + { + .data_type = QMI_EOTI, + .array_type = NO_ARRAY, + .tlv_type = QMI_COMMON_TLV_TYPE, + }, +}; + +static struct qmi_elem_info usb_interface_descriptor_v01_ei[] = { + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct usb_interface_descriptor_v01, + bLength), + }, + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct usb_interface_descriptor_v01, + bDescriptorType), + }, + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct usb_interface_descriptor_v01, + bInterfaceNumber), + }, + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct usb_interface_descriptor_v01, + bAlternateSetting), + }, + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct usb_interface_descriptor_v01, + bNumEndpoints), + }, + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct usb_interface_descriptor_v01, + bInterfaceClass), + }, + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct usb_interface_descriptor_v01, + bInterfaceSubClass), + }, + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct usb_interface_descriptor_v01, + bInterfaceProtocol), + }, + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0, + .offset = offsetof(struct usb_interface_descriptor_v01, + iInterface), + }, + { + .data_type = QMI_EOTI, + .array_type = NO_ARRAY, + .tlv_type = QMI_COMMON_TLV_TYPE, + }, +}; + +struct qmi_elem_info qmi_uaudio_stream_req_msg_v01_ei[] = { + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x01, + .offset = offsetof(struct qmi_uaudio_stream_req_msg_v01, + enable), + }, + { + .data_type = QMI_UNSIGNED_4_BYTE, + .elem_len = 1, + .elem_size = sizeof(u32), + .array_type = NO_ARRAY, + .tlv_type = 0x02, + .offset = offsetof(struct qmi_uaudio_stream_req_msg_v01, + usb_token), + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x10, + .offset = offsetof(struct qmi_uaudio_stream_req_msg_v01, + audio_format_valid), + }, + { + .data_type = QMI_UNSIGNED_4_BYTE, + .elem_len = 1, + .elem_size = sizeof(u32), + .array_type = NO_ARRAY, + .tlv_type = 0x10, + .offset = offsetof(struct qmi_uaudio_stream_req_msg_v01, + audio_format), + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x11, + .offset = offsetof(struct qmi_uaudio_stream_req_msg_v01, + number_of_ch_valid), + }, + { + .data_type = QMI_UNSIGNED_4_BYTE, + .elem_len = 1, + .elem_size = sizeof(u32), + .array_type = NO_ARRAY, + .tlv_type = 0x11, + .offset = offsetof(struct qmi_uaudio_stream_req_msg_v01, + number_of_ch), + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x12, + .offset = offsetof(struct qmi_uaudio_stream_req_msg_v01, + bit_rate_valid), + }, + { + .data_type = QMI_UNSIGNED_4_BYTE, + .elem_len = 1, + .elem_size = sizeof(u32), + .array_type = NO_ARRAY, + .tlv_type = 0x12, + .offset = offsetof(struct qmi_uaudio_stream_req_msg_v01, + bit_rate), + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x13, + .offset = offsetof(struct qmi_uaudio_stream_req_msg_v01, + xfer_buff_size_valid), + }, + { + .data_type = QMI_UNSIGNED_4_BYTE, + .elem_len = 1, + .elem_size = sizeof(u32), + .array_type = NO_ARRAY, + .tlv_type = 0x13, + .offset = offsetof(struct qmi_uaudio_stream_req_msg_v01, + xfer_buff_size), + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x14, + .offset = offsetof(struct qmi_uaudio_stream_req_msg_v01, + service_interval_valid), + }, + { + .data_type = QMI_UNSIGNED_4_BYTE, + .elem_len = 1, + .elem_size = sizeof(u32), + .array_type = NO_ARRAY, + .tlv_type = 0x14, + .offset = offsetof(struct qmi_uaudio_stream_req_msg_v01, + service_interval), + }, + { + .data_type = QMI_EOTI, + .array_type = NO_ARRAY, + .tlv_type = QMI_COMMON_TLV_TYPE, + }, +}; + +struct qmi_elem_info qmi_uaudio_stream_resp_msg_v01_ei[] = { + { + .data_type = QMI_STRUCT, + .elem_len = 1, + .elem_size = sizeof(struct qmi_response_type_v01), + .array_type = NO_ARRAY, + .tlv_type = 0x02, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + resp), + .ei_array = qmi_response_type_v01_ei, + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x10, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + status_valid), + }, + { + .data_type = QMI_SIGNED_4_BYTE_ENUM, + .elem_len = 1, + .elem_size = sizeof(enum usb_audio_stream_status_enum_v01), + .array_type = NO_ARRAY, + .tlv_type = 0x10, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + status), + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x11, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + internal_status_valid), + }, + { + .data_type = QMI_UNSIGNED_4_BYTE, + .elem_len = 1, + .elem_size = sizeof(u32), + .array_type = NO_ARRAY, + .tlv_type = 0x11, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + internal_status), + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x12, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + slot_id_valid), + }, + { + .data_type = QMI_UNSIGNED_4_BYTE, + .elem_len = 1, + .elem_size = sizeof(u32), + .array_type = NO_ARRAY, + .tlv_type = 0x12, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + slot_id), + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x13, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + usb_token_valid), + }, + { + .data_type = QMI_UNSIGNED_4_BYTE, + .elem_len = 1, + .elem_size = sizeof(u32), + .array_type = NO_ARRAY, + .tlv_type = 0x13, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + usb_token), + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x14, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + std_as_opr_intf_desc_valid), + }, + { + .data_type = QMI_STRUCT, + .elem_len = 1, + .elem_size = sizeof(struct usb_interface_descriptor_v01), + .array_type = NO_ARRAY, + .tlv_type = 0x14, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + std_as_opr_intf_desc), + .ei_array = usb_interface_descriptor_v01_ei, + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x15, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + std_as_data_ep_desc_valid), + }, + { + .data_type = QMI_STRUCT, + .elem_len = 1, + .elem_size = sizeof(struct usb_endpoint_descriptor_v01), + .array_type = NO_ARRAY, + .tlv_type = 0x15, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + std_as_data_ep_desc), + .ei_array = usb_endpoint_descriptor_v01_ei, + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x16, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + std_as_sync_ep_desc_valid), + }, + { + .data_type = QMI_STRUCT, + .elem_len = 1, + .elem_size = sizeof(struct usb_endpoint_descriptor_v01), + .array_type = NO_ARRAY, + .tlv_type = 0x16, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + std_as_sync_ep_desc), + .ei_array = usb_endpoint_descriptor_v01_ei, + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x17, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + usb_audio_spec_revision_valid), + }, + { + .data_type = QMI_UNSIGNED_2_BYTE, + .elem_len = 1, + .elem_size = sizeof(u16), + .array_type = NO_ARRAY, + .tlv_type = 0x17, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + usb_audio_spec_revision), + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x18, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + data_path_delay_valid), + }, + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x18, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + data_path_delay), + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x19, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + usb_audio_subslot_size_valid), + }, + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x19, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + usb_audio_subslot_size), + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x1A, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + xhci_mem_info_valid), + }, + { + .data_type = QMI_STRUCT, + .elem_len = 1, + .elem_size = sizeof(struct apps_mem_info_v01), + .array_type = NO_ARRAY, + .tlv_type = 0x1A, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + xhci_mem_info), + .ei_array = apps_mem_info_v01_ei, + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x1B, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + interrupter_num_valid), + }, + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x1B, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + interrupter_num), + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x1C, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + speed_info_valid), + }, + { + .data_type = QMI_SIGNED_4_BYTE_ENUM, + .elem_len = 1, + .elem_size = sizeof(enum usb_audio_device_speed_enum_v01), + .array_type = NO_ARRAY, + .tlv_type = 0x1C, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + speed_info), + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x1D, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + controller_num_valid), + }, + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x1D, + .offset = offsetof( + struct qmi_uaudio_stream_resp_msg_v01, + controller_num), + }, + { + .data_type = QMI_EOTI, + .array_type = NO_ARRAY, + .tlv_type = QMI_COMMON_TLV_TYPE, + }, +}; + +struct qmi_elem_info qmi_uaudio_stream_ind_msg_v01_ei[] = { + { + .data_type = QMI_SIGNED_4_BYTE_ENUM, + .elem_len = 1, + .elem_size = sizeof( + enum usb_audio_device_indication_enum_v01), + .array_type = NO_ARRAY, + .tlv_type = 0x01, + .offset = offsetof(struct qmi_uaudio_stream_ind_msg_v01, + dev_event), + }, + { + .data_type = QMI_UNSIGNED_4_BYTE, + .elem_len = 1, + .elem_size = sizeof(u32), + .array_type = NO_ARRAY, + .tlv_type = 0x02, + .offset = offsetof(struct qmi_uaudio_stream_ind_msg_v01, + slot_id), + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x10, + .offset = offsetof(struct qmi_uaudio_stream_ind_msg_v01, + usb_token_valid), + }, + { + .data_type = QMI_UNSIGNED_4_BYTE, + .elem_len = 1, + .elem_size = sizeof(u32), + .array_type = NO_ARRAY, + .tlv_type = 0x10, + .offset = offsetof(struct qmi_uaudio_stream_ind_msg_v01, + usb_token), + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x11, + .offset = offsetof(struct qmi_uaudio_stream_ind_msg_v01, + std_as_opr_intf_desc_valid), + }, + { + .data_type = QMI_STRUCT, + .elem_len = 1, + .elem_size = sizeof(struct usb_interface_descriptor_v01), + .array_type = NO_ARRAY, + .tlv_type = 0x11, + .offset = offsetof(struct qmi_uaudio_stream_ind_msg_v01, + std_as_opr_intf_desc), + .ei_array = usb_interface_descriptor_v01_ei, + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x12, + .offset = offsetof(struct qmi_uaudio_stream_ind_msg_v01, + std_as_data_ep_desc_valid), + }, + { + .data_type = QMI_STRUCT, + .elem_len = 1, + .elem_size = sizeof(struct usb_endpoint_descriptor_v01), + .array_type = NO_ARRAY, + .tlv_type = 0x12, + .offset = offsetof(struct qmi_uaudio_stream_ind_msg_v01, + std_as_data_ep_desc), + .ei_array = usb_endpoint_descriptor_v01_ei, + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x13, + .offset = offsetof(struct qmi_uaudio_stream_ind_msg_v01, + std_as_sync_ep_desc_valid), + }, + { + .data_type = QMI_STRUCT, + .elem_len = 1, + .elem_size = sizeof(struct usb_endpoint_descriptor_v01), + .array_type = NO_ARRAY, + .tlv_type = 0x13, + .offset = offsetof(struct qmi_uaudio_stream_ind_msg_v01, + std_as_sync_ep_desc), + .ei_array = usb_endpoint_descriptor_v01_ei, + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x14, + .offset = offsetof(struct qmi_uaudio_stream_ind_msg_v01, + usb_audio_spec_revision_valid), + }, + { + .data_type = QMI_UNSIGNED_2_BYTE, + .elem_len = 1, + .elem_size = sizeof(u16), + .array_type = NO_ARRAY, + .tlv_type = 0x14, + .offset = offsetof(struct qmi_uaudio_stream_ind_msg_v01, + usb_audio_spec_revision), + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x15, + .offset = offsetof(struct qmi_uaudio_stream_ind_msg_v01, + data_path_delay_valid), + }, + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x15, + .offset = offsetof(struct qmi_uaudio_stream_ind_msg_v01, + data_path_delay), + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x16, + .offset = offsetof(struct qmi_uaudio_stream_ind_msg_v01, + usb_audio_subslot_size_valid), + }, + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x16, + .offset = offsetof(struct qmi_uaudio_stream_ind_msg_v01, + usb_audio_subslot_size), + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x17, + .offset = offsetof(struct qmi_uaudio_stream_ind_msg_v01, + xhci_mem_info_valid), + }, + { + .data_type = QMI_STRUCT, + .elem_len = 1, + .elem_size = sizeof(struct apps_mem_info_v01), + .array_type = NO_ARRAY, + .tlv_type = 0x17, + .offset = offsetof(struct qmi_uaudio_stream_ind_msg_v01, + xhci_mem_info), + .ei_array = apps_mem_info_v01_ei, + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x18, + .offset = offsetof(struct qmi_uaudio_stream_ind_msg_v01, + interrupter_num_valid), + }, + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x18, + .offset = offsetof(struct qmi_uaudio_stream_ind_msg_v01, + interrupter_num), + }, + { + .data_type = QMI_OPT_FLAG, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x19, + .offset = offsetof(struct qmi_uaudio_stream_ind_msg_v01, + controller_num_valid), + }, + { + .data_type = QMI_UNSIGNED_1_BYTE, + .elem_len = 1, + .elem_size = sizeof(u8), + .array_type = NO_ARRAY, + .tlv_type = 0x19, + .offset = offsetof(struct qmi_uaudio_stream_ind_msg_v01, + controller_num), + }, + { + .data_type = QMI_EOTI, + .array_type = NO_ARRAY, + .tlv_type = QMI_COMMON_TLV_TYPE, + }, +}; diff --git a/sound/usb/qcom/usb_audio_qmi_v01.h b/sound/usb/qcom/usb_audio_qmi_v01.h new file mode 100644 index 000000000000..4e9b5f0bcddf --- /dev/null +++ b/sound/usb/qcom/usb_audio_qmi_v01.h @@ -0,0 +1,162 @@ +/* SPDX-License-Identifier: GPL-2.0 + * + * Copyright (c) 2022 Qualcomm Innovation Center, Inc. All rights reserved. + */ + +#ifndef USB_QMI_V01_H +#define USB_QMI_V01_H + +#define UAUDIO_STREAM_SERVICE_ID_V01 0x41D +#define UAUDIO_STREAM_SERVICE_VERS_V01 0x01 + +#define QMI_UAUDIO_STREAM_RESP_V01 0x0001 +#define QMI_UAUDIO_STREAM_REQ_V01 0x0001 +#define QMI_UAUDIO_STREAM_IND_V01 0x0001 + + +struct mem_info_v01 { + u64 va; + u64 pa; + u32 size; +}; + +struct apps_mem_info_v01 { + struct mem_info_v01 evt_ring; + struct mem_info_v01 tr_data; + struct mem_info_v01 tr_sync; + struct mem_info_v01 xfer_buff; + struct mem_info_v01 dcba; +}; + +struct usb_endpoint_descriptor_v01 { + u8 bLength; + u8 bDescriptorType; + u8 bEndpointAddress; + u8 bmAttributes; + u16 wMaxPacketSize; + u8 bInterval; + u8 bRefresh; + u8 bSynchAddress; +}; + +struct usb_interface_descriptor_v01 { + u8 bLength; + u8 bDescriptorType; + u8 bInterfaceNumber; + u8 bAlternateSetting; + u8 bNumEndpoints; + u8 bInterfaceClass; + u8 bInterfaceSubClass; + u8 bInterfaceProtocol; + u8 iInterface; +}; + +enum usb_audio_stream_status_enum_v01 { + USB_AUDIO_STREAM_STATUS_ENUM_MIN_VAL_V01 = INT_MIN, + USB_AUDIO_STREAM_REQ_SUCCESS_V01 = 0, + USB_AUDIO_STREAM_REQ_FAILURE_V01 = 1, + USB_AUDIO_STREAM_REQ_FAILURE_NOT_FOUND_V01 = 2, + USB_AUDIO_STREAM_REQ_FAILURE_INVALID_PARAM_V01 = 3, + USB_AUDIO_STREAM_REQ_FAILURE_MEMALLOC_V01 = 4, + USB_AUDIO_STREAM_STATUS_ENUM_MAX_VAL_V01 = INT_MAX, +}; + +enum usb_audio_device_indication_enum_v01 { + USB_AUDIO_DEVICE_INDICATION_ENUM_MIN_VAL_V01 = INT_MIN, + USB_AUDIO_DEV_CONNECT_V01 = 0, + USB_AUDIO_DEV_DISCONNECT_V01 = 1, + USB_AUDIO_DEV_SUSPEND_V01 = 2, + USB_AUDIO_DEV_RESUME_V01 = 3, + USB_AUDIO_DEVICE_INDICATION_ENUM_MAX_VAL_V01 = INT_MAX, +}; + +enum usb_audio_device_speed_enum_v01 { + USB_AUDIO_DEVICE_SPEED_ENUM_MIN_VAL_V01 = INT_MIN, + USB_AUDIO_DEVICE_SPEED_INVALID_V01 = 0, + USB_AUDIO_DEVICE_SPEED_LOW_V01 = 1, + USB_AUDIO_DEVICE_SPEED_FULL_V01 = 2, + USB_AUDIO_DEVICE_SPEED_HIGH_V01 = 3, + USB_AUDIO_DEVICE_SPEED_SUPER_V01 = 4, + USB_AUDIO_DEVICE_SPEED_SUPER_PLUS_V01 = 5, + USB_AUDIO_DEVICE_SPEED_ENUM_MAX_VAL_V01 = INT_MAX, +}; + +struct qmi_uaudio_stream_req_msg_v01 { + u8 enable; + u32 usb_token; + u8 audio_format_valid; + u32 audio_format; + u8 number_of_ch_valid; + u32 number_of_ch; + u8 bit_rate_valid; + u32 bit_rate; + u8 xfer_buff_size_valid; + u32 xfer_buff_size; + u8 service_interval_valid; + u32 service_interval; +}; +#define QMI_UAUDIO_STREAM_REQ_MSG_V01_MAX_MSG_LEN 46 +extern struct qmi_elem_info qmi_uaudio_stream_req_msg_v01_ei[]; + +struct qmi_uaudio_stream_resp_msg_v01 { + struct qmi_response_type_v01 resp; + u8 status_valid; + enum usb_audio_stream_status_enum_v01 status; + u8 internal_status_valid; + u32 internal_status; + u8 slot_id_valid; + u32 slot_id; + u8 usb_token_valid; + u32 usb_token; + u8 std_as_opr_intf_desc_valid; + struct usb_interface_descriptor_v01 std_as_opr_intf_desc; + u8 std_as_data_ep_desc_valid; + struct usb_endpoint_descriptor_v01 std_as_data_ep_desc; + u8 std_as_sync_ep_desc_valid; + struct usb_endpoint_descriptor_v01 std_as_sync_ep_desc; + u8 usb_audio_spec_revision_valid; + u16 usb_audio_spec_revision; + u8 data_path_delay_valid; + u8 data_path_delay; + u8 usb_audio_subslot_size_valid; + u8 usb_audio_subslot_size; + u8 xhci_mem_info_valid; + struct apps_mem_info_v01 xhci_mem_info; + u8 interrupter_num_valid; + u8 interrupter_num; + u8 speed_info_valid; + enum usb_audio_device_speed_enum_v01 speed_info; + u8 controller_num_valid; + u8 controller_num; +}; +#define QMI_UAUDIO_STREAM_RESP_MSG_V01_MAX_MSG_LEN 202 +extern struct qmi_elem_info qmi_uaudio_stream_resp_msg_v01_ei[]; + +struct qmi_uaudio_stream_ind_msg_v01 { + enum usb_audio_device_indication_enum_v01 dev_event; + u32 slot_id; + u8 usb_token_valid; + u32 usb_token; + u8 std_as_opr_intf_desc_valid; + struct usb_interface_descriptor_v01 std_as_opr_intf_desc; + u8 std_as_data_ep_desc_valid; + struct usb_endpoint_descriptor_v01 std_as_data_ep_desc; + u8 std_as_sync_ep_desc_valid; + struct usb_endpoint_descriptor_v01 std_as_sync_ep_desc; + u8 usb_audio_spec_revision_valid; + u16 usb_audio_spec_revision; + u8 data_path_delay_valid; + u8 data_path_delay; + u8 usb_audio_subslot_size_valid; + u8 usb_audio_subslot_size; + u8 xhci_mem_info_valid; + struct apps_mem_info_v01 xhci_mem_info; + u8 interrupter_num_valid; + u8 interrupter_num; + u8 controller_num_valid; + u8 controller_num; +}; +#define QMI_UAUDIO_STREAM_IND_MSG_V01_MAX_MSG_LEN 181 +extern struct qmi_elem_info qmi_uaudio_stream_ind_msg_v01_ei[]; + +#endif
On Sat, 24 Dec 2022 00:31:55 +0100, Wesley Cheng wrote:
Several Qualcomm SoCs have a dedicated audio DSP, which has the ability to support USB sound devices. This vendor driver will implement the required handshaking with the DSP, in order to pass along required resources that will be utilized by the DSP's USB SW. The communication channel used for this handshaking will be using the QMI protocol. Required resources include:
- Allocated secondary event ring address
- EP transfer ring address
- Interrupter number
The above information will allow for the audio DSP to execute USB transfers over the USB bus. It will also be able to support devices that have an implicit feedback and sync endpoint as well. Offloading these data transfers will allow the main/applications processor to enter lower CPU power modes, and sustain a longer duration in those modes.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com
Hmm, this must be the main part that works to bypass the normal USB packet handling in USB audio driver but hooks to the own offload one, but there is no description how to take over and manage. A missing "big picture" makes it difficult to understand and review.
Also, since both drivers are asynchronous, we may need some proper locking.
More on the code change:
+static int snd_interval_refine_set(struct snd_interval *i, unsigned int val) +{
- struct snd_interval t;
- t.empty = 0;
- t.min = t.max = val;
- t.openmin = t.openmax = 0;
- t.integer = 1;
- return snd_interval_refine(i, &t);
+}
+static int _snd_pcm_hw_param_set(struct snd_pcm_hw_params *params,
snd_pcm_hw_param_t var, unsigned int val,
int dir)
+{
- int changed;
- if (hw_is_mask(var)) {
struct snd_mask *m = hw_param_mask(params, var);
if (val == 0 && dir < 0) {
changed = -EINVAL;
snd_mask_none(m);
} else {
if (dir > 0)
val++;
else if (dir < 0)
val--;
changed = snd_mask_refine_set(
hw_param_mask(params, var), val);
}
- } else if (hw_is_interval(var)) {
struct snd_interval *i = hw_param_interval(params, var);
if (val == 0 && dir < 0) {
changed = -EINVAL;
snd_interval_none(i);
} else if (dir == 0)
changed = snd_interval_refine_set(i, val);
else {
struct snd_interval t;
t.openmin = 1;
t.openmax = 1;
t.empty = 0;
t.integer = 0;
if (dir < 0) {
t.min = val - 1;
t.max = val;
} else {
t.min = val;
t.max = val+1;
}
changed = snd_interval_refine(i, &t);
}
- } else
return -EINVAL;
- if (changed) {
params->cmask |= 1 << var;
params->rmask |= 1 << var;
- }
- return changed;
+}
Those are taken from sound/core/oss/pcm_oss.c? We may put to the common PCM helper instead of duplication.
+static void disable_audio_stream(struct snd_usb_substream *subs) +{
- struct snd_usb_audio *chip = subs->stream->chip;
- if (subs->data_endpoint || subs->sync_endpoint) {
close_endpoints(chip, subs);
mutex_lock(&chip->mutex);
subs->cur_audiofmt = NULL;
mutex_unlock(&chip->mutex);
- }
- snd_usb_autosuspend(chip);
+}
+static int enable_audio_stream(struct snd_usb_substream *subs,
snd_pcm_format_t pcm_format,
unsigned int channels, unsigned int cur_rate,
int datainterval)
+{
- struct snd_usb_audio *chip = subs->stream->chip;
- struct snd_pcm_hw_params params;
- const struct audioformat *fmt;
- int ret;
- _snd_pcm_hw_params_any(¶ms);
- _snd_pcm_hw_param_set(¶ms, SNDRV_PCM_HW_PARAM_FORMAT,
pcm_format, 0);
- _snd_pcm_hw_param_set(¶ms, SNDRV_PCM_HW_PARAM_CHANNELS,
channels, 0);
- _snd_pcm_hw_param_set(¶ms, SNDRV_PCM_HW_PARAM_RATE,
cur_rate, 0);
What about other parameters like period / buffer sizes?
+struct qmi_uaudio_stream_req_msg_v01 {
- u8 enable;
- u32 usb_token;
- u8 audio_format_valid;
- u32 audio_format;
- u8 number_of_ch_valid;
- u32 number_of_ch;
- u8 bit_rate_valid;
- u32 bit_rate;
- u8 xfer_buff_size_valid;
- u32 xfer_buff_size;
- u8 service_interval_valid;
- u32 service_interval;
+};
Are this and the other structs a part of DSP ABI? Or is it a definition only used in kernel? I'm asking because __packed attribute is required for most of ABI definitions with different field types.
thanks,
Takashi
Hi Takashi,
On 1/2/2023 9:28 AM, Takashi Iwai wrote:
On Sat, 24 Dec 2022 00:31:55 +0100, Wesley Cheng wrote:
Several Qualcomm SoCs have a dedicated audio DSP, which has the ability to support USB sound devices. This vendor driver will implement the required handshaking with the DSP, in order to pass along required resources that will be utilized by the DSP's USB SW. The communication channel used for this handshaking will be using the QMI protocol. Required resources include:
- Allocated secondary event ring address
- EP transfer ring address
- Interrupter number
The above information will allow for the audio DSP to execute USB transfers over the USB bus. It will also be able to support devices that have an implicit feedback and sync endpoint as well. Offloading these data transfers will allow the main/applications processor to enter lower CPU power modes, and sustain a longer duration in those modes.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com
Hmm, this must be the main part that works to bypass the normal USB packet handling in USB audio driver but hooks to the own offload one, but there is no description how to take over and manage. A missing "big picture" makes it difficult to understand and review.
Technically, we are not taking over the functionality of the USB SND, as we still want the normal path to be accessible in case there is an audio profile/format that can't be supported by the audio DSP. I can add some more information on how this offload driver co-exists with the USB SND.
Also, since both drivers are asynchronous, we may need some proper locking.
Yes, I think locking is needed in some places. Will add that in the next revision.
More on the code change:
+static int snd_interval_refine_set(struct snd_interval *i, unsigned int val) +{
- struct snd_interval t;
- t.empty = 0;
- t.min = t.max = val;
- t.openmin = t.openmax = 0;
- t.integer = 1;
- return snd_interval_refine(i, &t);
+}
+static int _snd_pcm_hw_param_set(struct snd_pcm_hw_params *params,
snd_pcm_hw_param_t var, unsigned int val,
int dir)
+{
- int changed;
- if (hw_is_mask(var)) {
struct snd_mask *m = hw_param_mask(params, var);
if (val == 0 && dir < 0) {
changed = -EINVAL;
snd_mask_none(m);
} else {
if (dir > 0)
val++;
else if (dir < 0)
val--;
changed = snd_mask_refine_set(
hw_param_mask(params, var), val);
}
- } else if (hw_is_interval(var)) {
struct snd_interval *i = hw_param_interval(params, var);
if (val == 0 && dir < 0) {
changed = -EINVAL;
snd_interval_none(i);
} else if (dir == 0)
changed = snd_interval_refine_set(i, val);
else {
struct snd_interval t;
t.openmin = 1;
t.openmax = 1;
t.empty = 0;
t.integer = 0;
if (dir < 0) {
t.min = val - 1;
t.max = val;
} else {
t.min = val;
t.max = val+1;
}
changed = snd_interval_refine(i, &t);
}
- } else
return -EINVAL;
- if (changed) {
params->cmask |= 1 << var;
params->rmask |= 1 << var;
- }
- return changed;
+}
Those are taken from sound/core/oss/pcm_oss.c? We may put to the common PCM helper instead of duplication.
Sure, I can do that.
+static void disable_audio_stream(struct snd_usb_substream *subs) +{
- struct snd_usb_audio *chip = subs->stream->chip;
- if (subs->data_endpoint || subs->sync_endpoint) {
close_endpoints(chip, subs);
mutex_lock(&chip->mutex);
subs->cur_audiofmt = NULL;
mutex_unlock(&chip->mutex);
- }
- snd_usb_autosuspend(chip);
+}
+static int enable_audio_stream(struct snd_usb_substream *subs,
snd_pcm_format_t pcm_format,
unsigned int channels, unsigned int cur_rate,
int datainterval)
+{
- struct snd_usb_audio *chip = subs->stream->chip;
- struct snd_pcm_hw_params params;
- const struct audioformat *fmt;
- int ret;
- _snd_pcm_hw_params_any(¶ms);
- _snd_pcm_hw_param_set(¶ms, SNDRV_PCM_HW_PARAM_FORMAT,
pcm_format, 0);
- _snd_pcm_hw_param_set(¶ms, SNDRV_PCM_HW_PARAM_CHANNELS,
channels, 0);
- _snd_pcm_hw_param_set(¶ms, SNDRV_PCM_HW_PARAM_RATE,
cur_rate, 0);
What about other parameters like period / buffer sizes?
I don't think we will need those parameters on the audio DSP. The "params" here is used to pass the pcm format into the qmi response.
+struct qmi_uaudio_stream_req_msg_v01 {
- u8 enable;
- u32 usb_token;
- u8 audio_format_valid;
- u32 audio_format;
- u8 number_of_ch_valid;
- u32 number_of_ch;
- u8 bit_rate_valid;
- u32 bit_rate;
- u8 xfer_buff_size_valid;
- u32 xfer_buff_size;
- u8 service_interval_valid;
- u32 service_interval;
+};
Are this and the other structs a part of DSP ABI? Or is it a definition only used in kernel? I'm asking because __packed attribute is required for most of ABI definitions with different field types.
This would be in the kernel only.
Thanks Wesley Cheng
On 12/23/22 17:31, Wesley Cheng wrote:
Several Qualcomm SoCs have a dedicated audio DSP, which has the ability to support USB sound devices. This vendor driver will implement the required handshaking with the DSP, in order to pass along required resources that will be utilized by the DSP's USB SW. The communication channel used for this handshaking will be using the QMI protocol. Required resources include:
- Allocated secondary event ring address
- EP transfer ring address
- Interrupter number
The above information will allow for the audio DSP to execute USB transfers over the USB bus. It will also be able to support devices that have an implicit feedback and sync endpoint as well. Offloading these data transfers will allow the main/applications processor to enter lower CPU power modes, and sustain a longer duration in those modes.
Are you suggesting that the entire feedback loop be handled in the DSP? It's not clear what "Offloading these data transfers" refers to, the data part or the feedback path?
Comments are almost inexistent in this patch so it's hard to figure out what it really does.
Hi Pierre,
On 1/4/2023 3:51 PM, Pierre-Louis Bossart wrote:
On 12/23/22 17:31, Wesley Cheng wrote:
Several Qualcomm SoCs have a dedicated audio DSP, which has the ability to support USB sound devices. This vendor driver will implement the required handshaking with the DSP, in order to pass along required resources that will be utilized by the DSP's USB SW. The communication channel used for this handshaking will be using the QMI protocol. Required resources include:
- Allocated secondary event ring address
- EP transfer ring address
- Interrupter number
The above information will allow for the audio DSP to execute USB transfers over the USB bus. It will also be able to support devices that have an implicit feedback and sync endpoint as well. Offloading these data transfers will allow the main/applications processor to enter lower CPU power modes, and sustain a longer duration in those modes.
Are you suggesting that the entire feedback loop be handled in the DSP? It's not clear what "Offloading these data transfers" refers to, the data part or the feedback path?
Yes, as mentioned in the cover letter response, we'll handle the feedback endpoints. (feedback eps are part of the audio data interface)
Comments are almost inexistent in this patch so it's hard to figure out what it really does.
OK, will add some more comments.
Thanks Wesley Cheng
Allow for checks on a specific USB audio device to see if a requested PCM format is supported. This is needed for support for when playback is initiated by the ASoC USB backend path.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com --- sound/usb/card.c | 19 +++++++++++++++++++ sound/usb/card.h | 3 +++ 2 files changed, 22 insertions(+)
diff --git a/sound/usb/card.c b/sound/usb/card.c index 396e5a34e23b..9b8d2ed308c8 100644 --- a/sound/usb/card.c +++ b/sound/usb/card.c @@ -133,6 +133,25 @@ int snd_usb_unregister_vendor_ops(void) } EXPORT_SYMBOL_GPL(snd_usb_unregister_vendor_ops);
+struct snd_usb_stream *snd_usb_find_suppported_substream(int card_idx, + struct snd_pcm_hw_params *params, int direction) +{ + struct snd_usb_stream *as; + struct snd_usb_substream *subs = NULL; + const struct audioformat *fmt; + + if (usb_chip[card_idx] && enable[card_idx]) { + list_for_each_entry(as, &usb_chip[card_idx]->pcm_list, list) { + subs = &as->substream[direction]; + fmt = find_substream_format(subs, params); + if (fmt) + return as; + } + } + + return 0; +} + /* * disconnect streams * called from usb_audio_disconnect() diff --git a/sound/usb/card.h b/sound/usb/card.h index a785bb256b0d..d4936b6054fc 100644 --- a/sound/usb/card.h +++ b/sound/usb/card.h @@ -213,4 +213,7 @@ struct snd_usb_vendor_ops {
int snd_usb_register_vendor_ops(struct snd_usb_vendor_ops *ops); int snd_usb_unregister_vendor_ops(void); + +struct snd_usb_stream *snd_usb_find_suppported_substream(int card_idx, + struct snd_pcm_hw_params *params, int direction); #endif /* __USBAUDIO_CARD_H */
On Fri, Dec 23, 2022 at 03:31:56PM -0800, Wesley Cheng wrote:
Allow for checks on a specific USB audio device to see if a requested PCM format is supported. This is needed for support for when playback is initiated by the ASoC USB backend path.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com
sound/usb/card.c | 19 +++++++++++++++++++ sound/usb/card.h | 3 +++ 2 files changed, 22 insertions(+)
diff --git a/sound/usb/card.c b/sound/usb/card.c index 396e5a34e23b..9b8d2ed308c8 100644 --- a/sound/usb/card.c +++ b/sound/usb/card.c @@ -133,6 +133,25 @@ int snd_usb_unregister_vendor_ops(void) } EXPORT_SYMBOL_GPL(snd_usb_unregister_vendor_ops);
+struct snd_usb_stream *snd_usb_find_suppported_substream(int card_idx,
struct snd_pcm_hw_params *params, int direction)
+{
- struct snd_usb_stream *as;
- struct snd_usb_substream *subs = NULL;
- const struct audioformat *fmt;
- if (usb_chip[card_idx] && enable[card_idx]) {
list_for_each_entry(as, &usb_chip[card_idx]->pcm_list, list) {
subs = &as->substream[direction];
fmt = find_substream_format(subs, params);
if (fmt)
return as;
}
- }
Where is the locking here? How can you walk a list that can be changed as you walk it?
And what about reference counting? You are returning a pointer to a structure, who now "owns" it? What happens if it is removed from the system after you return it?
- return 0;
Didn't sparse complain about this? You can't return "0" as a pointer, it should be NULL.
Always run basic tools like sparse on code before submitting it so that we don't have to find errors like this.
thanks,
greg k-h
Hi Greg,
On 12/24/2022 12:59 AM, Greg KH wrote:
On Fri, Dec 23, 2022 at 03:31:56PM -0800, Wesley Cheng wrote:
Allow for checks on a specific USB audio device to see if a requested PCM format is supported. This is needed for support for when playback is initiated by the ASoC USB backend path.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com
sound/usb/card.c | 19 +++++++++++++++++++ sound/usb/card.h | 3 +++ 2 files changed, 22 insertions(+)
diff --git a/sound/usb/card.c b/sound/usb/card.c index 396e5a34e23b..9b8d2ed308c8 100644 --- a/sound/usb/card.c +++ b/sound/usb/card.c @@ -133,6 +133,25 @@ int snd_usb_unregister_vendor_ops(void) } EXPORT_SYMBOL_GPL(snd_usb_unregister_vendor_ops);
+struct snd_usb_stream *snd_usb_find_suppported_substream(int card_idx,
struct snd_pcm_hw_params *params, int direction)
+{
- struct snd_usb_stream *as;
- struct snd_usb_substream *subs = NULL;
- const struct audioformat *fmt;
- if (usb_chip[card_idx] && enable[card_idx]) {
list_for_each_entry(as, &usb_chip[card_idx]->pcm_list, list) {
subs = &as->substream[direction];
fmt = find_substream_format(subs, params);
if (fmt)
return as;
}
- }
Where is the locking here? How can you walk a list that can be changed as you walk it?
And what about reference counting? You are returning a pointer to a structure, who now "owns" it? What happens if it is removed from the system after you return it?
- return 0;
Didn't sparse complain about this? You can't return "0" as a pointer, it should be NULL.
Always run basic tools like sparse on code before submitting it so that we don't have to find errors like this.
Got it...I didn't get a chance to run that, but will do it on future submissions. Will also address the locking and pointer reference you mentioned.
Thanks Wesley Cheng
Introduce a check for if a particular PCM format is supported by the USB audio device connected. If the USB audio device does not have an audio profile which can support the requested format, then notify the USB backend.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com --- include/sound/soc-usb.h | 3 +++ sound/soc/soc-usb.c | 13 +++++++++++++ 2 files changed, 16 insertions(+)
diff --git a/include/sound/soc-usb.h b/include/sound/soc-usb.h index 7d52e5d2371c..36df6f4fe093 100644 --- a/include/sound/soc-usb.h +++ b/include/sound/soc-usb.h @@ -19,6 +19,9 @@ struct snd_soc_usb { void *priv_data; };
+int snd_soc_usb_find_format(int card_idx, struct snd_pcm_hw_params *params, + int direction); + int snd_soc_usb_connect(int card_idx); int snd_soc_usb_disconnect(void); void snd_soc_usb_set_priv_data(void *priv); diff --git a/sound/soc/soc-usb.c b/sound/soc/soc-usb.c index c6c376960e4d..021380a022bc 100644 --- a/sound/soc/soc-usb.c +++ b/sound/soc/soc-usb.c @@ -25,6 +25,19 @@ void snd_soc_usb_set_priv_data(void *priv) } EXPORT_SYMBOL_GPL(snd_soc_usb_set_priv_data);
+int snd_soc_usb_find_format(int card_idx, struct snd_pcm_hw_params *params, + int direction) +{ + struct snd_usb_stream *as; + + as = snd_usb_find_suppported_substream(card_idx, params, direction); + if (!as) + return -EOPNOTSUPP; + + return 0; +} +EXPORT_SYMBOL_GPL(snd_soc_usb_find_format); + struct snd_soc_usb *snd_soc_usb_add_port(struct device *dev, int (*connection_cb)(struct snd_soc_usb *usb, int card_idx, int connected))
Check for if the PCM format is supported during the hw_params callback. If the profile is not supported then the userspace ALSA entity will receive an error, and can take further action.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com --- sound/soc/qcom/qdsp6/q6usb.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/sound/soc/qcom/qdsp6/q6usb.c b/sound/soc/qcom/qdsp6/q6usb.c index a9da6dec6c6f..128e0974db4e 100644 --- a/sound/soc/qcom/qdsp6/q6usb.c +++ b/sound/soc/qcom/qdsp6/q6usb.c @@ -42,7 +42,14 @@ static int q6usb_hw_params(struct snd_pcm_substream *substream, struct snd_pcm_hw_params *params, struct snd_soc_dai *dai) { - return 0; + struct q6usb_port_data *data = dev_get_drvdata(dai->dev); + int direction = substream->stream; + int ret; + + ret = snd_soc_usb_find_format(data->active_idx, params, direction); + + return ret; + } static const struct snd_soc_dai_ops q6usb_ops = { .hw_params = q6usb_hw_params,
Hello!
On 12/24/22 2:31 AM, Wesley Cheng wrote:
Check for if the PCM format is supported during the hw_params callback. If the profile is not supported then the userspace ALSA entity will receive an error, and can take further action.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com
sound/soc/qcom/qdsp6/q6usb.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/sound/soc/qcom/qdsp6/q6usb.c b/sound/soc/qcom/qdsp6/q6usb.c index a9da6dec6c6f..128e0974db4e 100644 --- a/sound/soc/qcom/qdsp6/q6usb.c +++ b/sound/soc/qcom/qdsp6/q6usb.c @@ -42,7 +42,14 @@ static int q6usb_hw_params(struct snd_pcm_substream *substream, struct snd_pcm_hw_params *params, struct snd_soc_dai *dai) {
- return 0;
- struct q6usb_port_data *data = dev_get_drvdata(dai->dev);
- int direction = substream->stream;
- int ret;
You don't seem to need this variable, just use *return* snd_soc_usb_find_format(...).
- ret = snd_soc_usb_find_format(data->active_idx, params, direction);
- return ret;
} static const struct snd_soc_dai_ops q6usb_ops = { .hw_params = q6usb_hw_params,
MBR, Sergey
Hi Sergey,
On 12/24/2022 12:19 AM, Sergey Shtylyov wrote:
Hello!
On 12/24/22 2:31 AM, Wesley Cheng wrote:
Check for if the PCM format is supported during the hw_params callback. If the profile is not supported then the userspace ALSA entity will receive an error, and can take further action.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com
sound/soc/qcom/qdsp6/q6usb.c | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/sound/soc/qcom/qdsp6/q6usb.c b/sound/soc/qcom/qdsp6/q6usb.c index a9da6dec6c6f..128e0974db4e 100644 --- a/sound/soc/qcom/qdsp6/q6usb.c +++ b/sound/soc/qcom/qdsp6/q6usb.c @@ -42,7 +42,14 @@ static int q6usb_hw_params(struct snd_pcm_substream *substream, struct snd_pcm_hw_params *params, struct snd_soc_dai *dai) {
- return 0;
- struct q6usb_port_data *data = dev_get_drvdata(dai->dev);
- int direction = substream->stream;
- int ret;
You don't seem to need this variable, just use *return*
snd_soc_usb_find_format(...) >
Thanks for catching this... Will fix it in the next submission I make. Happy holidays!
Thanks Wesley Cheng
On Fri, Dec 23, 2022 at 03:31:58PM -0800, Wesley Cheng wrote:
Check for if the PCM format is supported during the hw_params callback. If the profile is not supported then the userspace ALSA entity will receive an error, and can take further action.
Ideally we'd wire up constraints for this but that gets complicated with DPCM so it's probably disproportionate effort. Otherwise other than the subject lines not using ASoC on this and the previous change I don't have any issues that other people didn't raise, but then most of the complication is in the USB bits.
Add a dt-binding to describe the definition of enabling the Q6 USB backend device for audio offloading. The node carries information, which is passed along to the QC USB SND class driver counterpart. These parameters will be utilized during QMI stream enable requests.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com --- .../bindings/sound/qcom,q6usb-dais.yaml | 55 +++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 Documentation/devicetree/bindings/sound/qcom,q6usb-dais.yaml
diff --git a/Documentation/devicetree/bindings/sound/qcom,q6usb-dais.yaml b/Documentation/devicetree/bindings/sound/qcom,q6usb-dais.yaml new file mode 100644 index 000000000000..e24b4d52fa7e --- /dev/null +++ b/Documentation/devicetree/bindings/sound/qcom,q6usb-dais.yaml @@ -0,0 +1,55 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/sound/qcom,q6usb-dais.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml# + +title: Qualcomm ASoC USB backend DAI + +maintainers: + - Wesley Cheng quic_wcheng@quicinc.com + +description: + The Q6USB backend is a supported AFE port on the Q6DSP. This backend + driver will communicate the required settings to the QC USB SND class + driver for properly enabling the audio stream. Parameters defined + under this node will carry settings, which will be passed along during + the QMI stream enable request. + +properties: + compatible: + enum: + - qcom,q6usb-dais + + iommus: + maxItems: 1 + + "#sound-dai-cells": + const: 1 + + qcom,usb-audio-stream-id: + description: + SID for the Q6DSP processor for IOMMU mapping. + $ref: /schemas/types.yaml#/definitions/uint32 + + qcom,usb-audio-intr-num: + description: + Desired XHCI interrupter number to use. + $ref: /schemas/types.yaml#/definitions/uint32 + +required: + - compatible + - '#sound-dai-cells' + - qcom,usb-audio-intr-num + +additionalProperties: false + +examples: + - | + usbdai: usbd { + compatible = "qcom,q6usb-dais"; + #sound-dai-cells = <1>; + iommus = <&apps_smmu 0x180f 0x0>; + qcom,usb-audio-stream-id = <0xf>; + qcom,usb-audio-intr-num = <2>; + };
On 24/12/2022 00:31, Wesley Cheng wrote:
Add a dt-binding to describe the definition of enabling the Q6 USB backend device for audio offloading. The node carries information, which is passed along to the QC USB SND class driver counterpart. These parameters will be utilized during QMI stream enable requests.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com
.../bindings/sound/qcom,q6usb-dais.yaml | 55 +++++++++++++++++++ 1 file changed, 55 insertions(+) create mode 100644 Documentation/devicetree/bindings/sound/qcom,q6usb-dais.yaml
diff --git a/Documentation/devicetree/bindings/sound/qcom,q6usb-dais.yaml b/Documentation/devicetree/bindings/sound/qcom,q6usb-dais.yaml new file mode 100644 index 000000000000..e24b4d52fa7e --- /dev/null +++ b/Documentation/devicetree/bindings/sound/qcom,q6usb-dais.yaml @@ -0,0 +1,55 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) +%YAML 1.2 +--- +$id: http://devicetree.org/schemas/sound/qcom,q6usb-dais.yaml# +$schema: http://devicetree.org/meta-schemas/core.yaml#
+title: Qualcomm ASoC USB backend DAI
What is "ASoC"? Does not look like Qualcomm name for hardware, but rather Linux, so should be dropped or changed to SoC or ADSP etc.
+maintainers:
- Wesley Cheng quic_wcheng@quicinc.com
+description:
- The Q6USB backend is a supported AFE port on the Q6DSP. This backend
What does "supported AFE port" mean? Can an AFE port be unsupported?
- driver will communicate the required settings to the QC USB SND class
If you mean Linux driver, then please drop entirely all references to drivers. In other case, please explain more.
- driver for properly enabling the audio stream. Parameters defined
- under this node will carry settings, which will be passed along during
- the QMI stream enable request.
+properties:
- compatible:
- enum:
- qcom,q6usb-dais
- iommus:
- maxItems: 1
- "#sound-dai-cells":
- const: 1
- qcom,usb-audio-stream-id:
- description:
SID for the Q6DSP processor for IOMMU mapping.
Why this is not part of regular iommus property? It's the first time something like this appears in Qualcomm hardware, so I wonder why this DAI is special?
- $ref: /schemas/types.yaml#/definitions/uint32
- qcom,usb-audio-intr-num:
- description:
Desired XHCI interrupter number to use.
What does it mean? Interrupts are defined with "interrupts" property, so you need to explain more.
- $ref: /schemas/types.yaml#/definitions/uint32
+required:
- compatible
- '#sound-dai-cells'
Keep consistent quotes - either ' or "
- qcom,usb-audio-intr-num
+additionalProperties: false
+examples:
- |
- usbdai: usbd {
Generic node names, so dai or dais
compatible = "qcom,q6usb-dais";
#sound-dai-cells = <1>;
iommus = <&apps_smmu 0x180f 0x0>;
qcom,usb-audio-stream-id = <0xf>;
qcom,usb-audio-intr-num = <2>;
- };
Best regards, Krzysztof
Add an example on enabling of USB offload for the Q6DSP. The routing can be done by the mixer, which can pass the multimedia stream to the USB backend.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com --- .../devicetree/bindings/sound/qcom,sm8250.yaml | 13 +++++++++++++ 1 file changed, 13 insertions(+)
diff --git a/Documentation/devicetree/bindings/sound/qcom,sm8250.yaml b/Documentation/devicetree/bindings/sound/qcom,sm8250.yaml index 70080d04ddc9..60cd84e6727a 100644 --- a/Documentation/devicetree/bindings/sound/qcom,sm8250.yaml +++ b/Documentation/devicetree/bindings/sound/qcom,sm8250.yaml @@ -216,6 +216,19 @@ examples: sound-dai = <&vamacro 0>; }; }; + + usb-dai-link { + link-name = "USB Playback"; + cpu { + sound-dai = <&q6afedai USB_RX>; + }; + codec { + sound-dai = <&usbdai USB_RX>; + }; + platform { + sound-dai = <&q6routing>; + }; + }; };
- |
On 24/12/2022 00:32, Wesley Cheng wrote:
Add an example on enabling of USB offload for the Q6DSP. The routing can be done by the mixer, which can pass the multimedia stream to the USB backend.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com
.../devicetree/bindings/sound/qcom,sm8250.yaml | 13 +++++++++++++ 1 file changed, 13 insertions(+)
diff --git a/Documentation/devicetree/bindings/sound/qcom,sm8250.yaml b/Documentation/devicetree/bindings/sound/qcom,sm8250.yaml index 70080d04ddc9..60cd84e6727a 100644 --- a/Documentation/devicetree/bindings/sound/qcom,sm8250.yaml +++ b/Documentation/devicetree/bindings/sound/qcom,sm8250.yaml @@ -216,6 +216,19 @@ examples: sound-dai = <&vamacro 0>; }; };
usb-dai-link {
link-name = "USB Playback";
cpu {
sound-dai = <&q6afedai USB_RX>;
Hmm, that makes me wonder if you really tested the bindings before sending? If yes, where is the USB_RX defined?
Best regards, Krzysztof
On Mon, Dec 26, 2022 at 01:27:21PM +0100, Krzysztof Kozlowski wrote:
On 24/12/2022 00:32, Wesley Cheng wrote:
link-name = "USB Playback";
cpu {
sound-dai = <&q6afedai USB_RX>;
Hmm, that makes me wonder if you really tested the bindings before sending? If yes, where is the USB_RX defined?
It was added in patch 2, it's in include/dt-bindings.
On 03/01/2023 18:46, Mark Brown wrote:
On Mon, Dec 26, 2022 at 01:27:21PM +0100, Krzysztof Kozlowski wrote:
On 24/12/2022 00:32, Wesley Cheng wrote:
link-name = "USB Playback";
cpu {
sound-dai = <&q6afedai USB_RX>;
Hmm, that makes me wonder if you really tested the bindings before sending? If yes, where is the USB_RX defined?
It was added in patch 2, it's in include/dt-bindings.
Thanks, indeed, I was looking for another bindings patch but this was squashed with a driver.
Best regards, Krzysztof
On Fri, 23 Dec 2022 15:32:00 -0800, Wesley Cheng wrote:
Add an example on enabling of USB offload for the Q6DSP. The routing can be done by the mixer, which can pass the multimedia stream to the USB backend.
Signed-off-by: Wesley Cheng quic_wcheng@quicinc.com
.../devicetree/bindings/sound/qcom,sm8250.yaml | 13 +++++++++++++ 1 file changed, 13 insertions(+)
Acked-by: Rob Herring robh@kernel.org
On Fri, Dec 23, 2022 at 03:31:46PM -0800, Wesley Cheng wrote:
Several Qualcomm based chipsets can support USB audio offloading to a dedicated audio DSP, which can take over issuing transfers to the USB host controller. The intention is to reduce the load on the main processors in the SoC, and allow them to be placed into lower power modes. There are several parts to this design:
- Adding ASoC binding layer
- Create a USB backend for Q6DSP
- Introduce XHCI interrupter support
- Create vendor ops for the USB SND driver
Adding ASoC binding layer: soc-usb: Intention is to treat a USB port similar to a headphone jack. The port is always present on the device, but cable/pin status can be enabled/disabled. Expose mechanisms for USB backend ASoC drivers to communicate with USB SND.
Create a USB backend for Q6DSP: q6usb: Basic backend driver that will be responsible for maintaining the resources needed to initiate a playback stream using the Q6DSP. Will be the entity that checks to make sure the connected USB audio device supports the requested PCM format. If it does not, the PCM open call will fail, and userpsace ALSA can take action accordingly.
Introduce XHCI interrupter support: XHCI HCD supports multiple interrupters, which allows for events to be routed to different event rings. This is determined by "Interrupter Target" field specified in Section "6.4.1.1 Normal TRB" of the XHCI specification.
Events in the offloading case will be routed to an event ring that is assigned to the audio DSP.
Create vendor ops for the USB SND driver: qc_audio_offload: This particular driver has several components associated with it:
- QMI stream request handler
- XHCI interrupter and resource management
- audio DSP memory management
When the audio DSP wants to enable a playback stream, the request is first received by the ASoC platform sound card. Depending on the selected route, ASoC will bring up the individual DAIs in the path. The Q6USB backend DAI will send an AFE port start command (with enabling the USB playback path), and the audio DSP will handle the request accordingly.
Part of the AFE USB port start handling will have an exchange of control messages using the QMI protocol. The qc_audio_offload driver will populate the buffer information:
- Event ring base address
- EP transfer ring base address
and pass it along to the audio DSP. All endpoint management will now be handed over to the DSP, and the main processor is not involved in transfers.
Overall, implementing this feature will still expose separate sound card and PCM devices for both the platorm card and USB audio device: 0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D SM8250-MTP-WCD9380-WSA8810-VA-DMIC 1 [Audio ]: USB-Audio - USB Audio Generic USB Audio at usb-xhci-hcd.1.auto-1.4, high speed
This is to ensure that userspace ALSA entities can decide which route to take when executing the audio playback. In the above, if card#1 is selected, then USB audio data will take the legacy path over the USB PCM drivers, etc...
This feature was validated using:
- tinymix: set/enable the multimedia path to route to USB backend
- tinyplay: issue playback on platform card
This looks to duplicate a bunch of the same things that a number of different google developers have posted recently. Please work with them to come up with a unified set of patches that you all can agree with, AND get them to sign off on the changes before resubmitting them.
This uncoordinated drip of patches from different people doing the same thing is almost impossible to review from our side, as I'm sure you can imagine.
That being said, thank you finally for at least submitting all of the needed changes together as one patch set. That's a first, and something we had been asking for for years.
Have a good holiday break,
greg k-h
Hi Greg,
On 12/23/2022 10:45 PM, Greg KH wrote:
On Fri, Dec 23, 2022 at 03:31:46PM -0800, Wesley Cheng wrote:
Several Qualcomm based chipsets can support USB audio offloading to a dedicated audio DSP, which can take over issuing transfers to the USB host controller. The intention is to reduce the load on the main processors in the SoC, and allow them to be placed into lower power modes. There are several parts to this design:
- Adding ASoC binding layer
- Create a USB backend for Q6DSP
- Introduce XHCI interrupter support
- Create vendor ops for the USB SND driver
Adding ASoC binding layer: soc-usb: Intention is to treat a USB port similar to a headphone jack. The port is always present on the device, but cable/pin status can be enabled/disabled. Expose mechanisms for USB backend ASoC drivers to communicate with USB SND.
Create a USB backend for Q6DSP: q6usb: Basic backend driver that will be responsible for maintaining the resources needed to initiate a playback stream using the Q6DSP. Will be the entity that checks to make sure the connected USB audio device supports the requested PCM format. If it does not, the PCM open call will fail, and userpsace ALSA can take action accordingly.
Introduce XHCI interrupter support: XHCI HCD supports multiple interrupters, which allows for events to be routed to different event rings. This is determined by "Interrupter Target" field specified in Section "6.4.1.1 Normal TRB" of the XHCI specification.
Events in the offloading case will be routed to an event ring that is assigned to the audio DSP.
Create vendor ops for the USB SND driver: qc_audio_offload: This particular driver has several components associated with it:
- QMI stream request handler
- XHCI interrupter and resource management
- audio DSP memory management
When the audio DSP wants to enable a playback stream, the request is first received by the ASoC platform sound card. Depending on the selected route, ASoC will bring up the individual DAIs in the path. The Q6USB backend DAI will send an AFE port start command (with enabling the USB playback path), and the audio DSP will handle the request accordingly.
Part of the AFE USB port start handling will have an exchange of control messages using the QMI protocol. The qc_audio_offload driver will populate the buffer information:
- Event ring base address
- EP transfer ring base address
and pass it along to the audio DSP. All endpoint management will now be handed over to the DSP, and the main processor is not involved in transfers.
Overall, implementing this feature will still expose separate sound card and PCM devices for both the platorm card and USB audio device: 0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D SM8250-MTP-WCD9380-WSA8810-VA-DMIC 1 [Audio ]: USB-Audio - USB Audio Generic USB Audio at usb-xhci-hcd.1.auto-1.4, high speed
This is to ensure that userspace ALSA entities can decide which route to take when executing the audio playback. In the above, if card#1 is selected, then USB audio data will take the legacy path over the USB PCM drivers, etc...
This feature was validated using:
- tinymix: set/enable the multimedia path to route to USB backend
- tinyplay: issue playback on platform card
This looks to duplicate a bunch of the same things that a number of different google developers have posted recently. Please work with them to come up with a unified set of patches that you all can agree with, AND get them to sign off on the changes before resubmitting them.
This uncoordinated drip of patches from different people doing the same thing is almost impossible to review from our side, as I'm sure you can imagine.
I saw some of the Google patchsets submitted awhile back, but didn't really get a chance to look at them in detail. Let me reach out to Albert Wang to see if we can come to a solution that works for both implementations.
From the looks of it (at least from the XHCI HCD changes), it seems that a different set of resources is required for the Google implementation to work. I'll need to ask for a bit more details before I can comment further...
That being said, thank you finally for at least submitting all of the needed changes together as one patch set. That's a first, and something we had been asking for for years.
Have a good holiday break,
Thanks for the quick in-depth review, and the feedback. Gives me some more things to think about improving over the break :). Happy holidays!
Thanks Wesley Cheng
On Sat, Dec 24, 2022 at 07:45:38AM +0100, Greg KH wrote:
On Fri, Dec 23, 2022 at 03:31:46PM -0800, Wesley Cheng wrote:
soc-usb: Intention is to treat a USB port similar to a headphone jack. The port is always present on the device, but cable/pin status can be enabled/disabled. Expose mechanisms for USB backend ASoC drivers to communicate with USB SND.
Create a USB backend for Q6DSP: q6usb: Basic backend driver that will be responsible for maintaining the resources needed to initiate a playback stream using the Q6DSP. Will
This looks to duplicate a bunch of the same things that a number of different google developers have posted recently. Please work with them to come up with a unified set of patches that you all can agree with, AND get them to sign off on the changes before resubmitting them.
This uncoordinated drip of patches from different people doing the same thing is almost impossible to review from our side, as I'm sure you can imagine.
I have to say this is the first I've heard of any such patches other than from the Qualcomm people and I can't immediately see anything that was on the list either, though I might be missing something since I don't have the subject or anything. If other people send things again it's probably good to suggest they copy in audio people and lists.
On 12/23/22 17:31, Wesley Cheng wrote:
Several Qualcomm based chipsets can support USB audio offloading to a dedicated audio DSP, which can take over issuing transfers to the USB host controller. The intention is to reduce the load on the main processors in the SoC, and allow them to be placed into lower power modes.
It would be nice to clarify what you want to offload a) audio data transfers for isoc ports b) control for e.g. volume settings (those go to endpoint 0 IIRC) c) Both?
This has a lot of implications on the design. ASoC/DPCM is mainly intended for audio data transfers, control is a separate problem with configurations handled with register settings or bus-specific commands.
There are several parts to this design:
- Adding ASoC binding layer
- Create a USB backend for Q6DSP
- Introduce XHCI interrupter support
- Create vendor ops for the USB SND driver
Adding ASoC binding layer: soc-usb: Intention is to treat a USB port similar to a headphone jack. The port is always present on the device, but cable/pin status can be enabled/disabled. Expose mechanisms for USB backend ASoC drivers to communicate with USB SND.
Create a USB backend for Q6DSP: q6usb: Basic backend driver that will be responsible for maintaining the resources needed to initiate a playback stream using the Q6DSP. Will be the entity that checks to make sure the connected USB audio device supports the requested PCM format. If it does not, the PCM open call will fail, and userpsace ALSA can take action accordingly.
Introduce XHCI interrupter support: XHCI HCD supports multiple interrupters, which allows for events to be routed to different event rings. This is determined by "Interrupter Target" field specified in Section "6.4.1.1 Normal TRB" of the XHCI specification.
Events in the offloading case will be routed to an event ring that is assigned to the audio DSP.
Create vendor ops for the USB SND driver: qc_audio_offload: This particular driver has several components associated with it:
- QMI stream request handler
- XHCI interrupter and resource management
- audio DSP memory management
When the audio DSP wants to enable a playback stream, the request is first received by the ASoC platform sound card. Depending on the selected route, ASoC will bring up the individual DAIs in the path. The Q6USB backend DAI will send an AFE port start command (with enabling the USB playback path), and the audio DSP will handle the request accordingly.
Part of the AFE USB port start handling will have an exchange of control messages using the QMI protocol. The qc_audio_offload driver will populate the buffer information:
- Event ring base address
- EP transfer ring base address
and pass it along to the audio DSP. All endpoint management will now be handed over to the DSP, and the main processor is not involved in transfers.
Overall, implementing this feature will still expose separate sound card and PCM devices for both the platorm card and USB audio device: 0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D SM8250-MTP-WCD9380-WSA8810-VA-DMIC 1 [Audio ]: USB-Audio - USB Audio Generic USB Audio at usb-xhci-hcd.1.auto-1.4, high speed
This is to ensure that userspace ALSA entities can decide which route to take when executing the audio playback. In the above, if card#1 is selected, then USB audio data will take the legacy path over the USB PCM drivers, etc...
You would still need some sort of mutual exclusion to make sure the isoc endpoints are not used concurrently by the two cards. Relying on userspace intelligence to enforce that exclusion is not safe IMHO.
Intel looked at this sort of offload support a while ago and our directions were very different - for a variety of reasons USB offload is enabled on Windows platforms but remains a TODO for Linux. Rather than having two cards, you could have a single card and addition subdevices that expose the paths through the DSP. The benefits were that there was a single set of controls that userspace needed to know about, and volume settings were the same no matter which path you used (legacy or DSP-optimized paths). That's consistent with the directions to use 'Deep Buffer' PCM paths for local playback, it's the same idea of reducing power consumption with optimized routing.
Another point is that there may be cases where the DSP paths are not available if the DSP memory and MCPS budget is exceeded. In those cases, the DSP parts needs the ability to notify userspace that the legacy path should be used.
Another case to handle is that some USB devices can handle way more data than DSPs can chew, for example Pro audio boxes that can deal with 8ch 192kHz will typically use the legacy paths. Some also handle specific formats such as DSD over PCM. So it's quite likely that PCM devices for card0 and card1 above do NOT expose support for the same formats, or put differently that only a subset of the USB device capabilities are handled through the DSP.
And last, power optimizations with DSPs typically come from additional latency helping put the SoC in low-power modes. That's not necessarily ideal for all usages, e.g. for music recording and mixing I am not convinced the DSP path would help at all.
This feature was validated using:
- tinymix: set/enable the multimedia path to route to USB backend
- tinyplay: issue playback on platform card
Hi Pierre,
On 1/4/2023 3:19 PM, Pierre-Louis Bossart wrote:
On 12/23/22 17:31, Wesley Cheng wrote:
Several Qualcomm based chipsets can support USB audio offloading to a dedicated audio DSP, which can take over issuing transfers to the USB host controller. The intention is to reduce the load on the main processors in the SoC, and allow them to be placed into lower power modes.
It would be nice to clarify what you want to offload a) audio data transfers for isoc ports b) control for e.g. volume settings (those go to endpoint 0 IIRC) c) Both?
Thanks for sharing your experience, and inputs!
It would be the audio related endpoints only, so ISOC and potentially feedback ep.
This has a lot of implications on the design. ASoC/DPCM is mainly intended for audio data transfers, control is a separate problem with configurations handled with register settings or bus-specific commands.
Control would still be handled by the main processor.
There are several parts to this design:
- Adding ASoC binding layer
- Create a USB backend for Q6DSP
- Introduce XHCI interrupter support
- Create vendor ops for the USB SND driver
Adding ASoC binding layer: soc-usb: Intention is to treat a USB port similar to a headphone jack. The port is always present on the device, but cable/pin status can be enabled/disabled. Expose mechanisms for USB backend ASoC drivers to communicate with USB SND.
Create a USB backend for Q6DSP: q6usb: Basic backend driver that will be responsible for maintaining the resources needed to initiate a playback stream using the Q6DSP. Will be the entity that checks to make sure the connected USB audio device supports the requested PCM format. If it does not, the PCM open call will fail, and userpsace ALSA can take action accordingly.
Introduce XHCI interrupter support: XHCI HCD supports multiple interrupters, which allows for events to be routed to different event rings. This is determined by "Interrupter Target" field specified in Section "6.4.1.1 Normal TRB" of the XHCI specification.
Events in the offloading case will be routed to an event ring that is assigned to the audio DSP.
Create vendor ops for the USB SND driver: qc_audio_offload: This particular driver has several components associated with it:
- QMI stream request handler
- XHCI interrupter and resource management
- audio DSP memory management
When the audio DSP wants to enable a playback stream, the request is first received by the ASoC platform sound card. Depending on the selected route, ASoC will bring up the individual DAIs in the path. The Q6USB backend DAI will send an AFE port start command (with enabling the USB playback path), and the audio DSP will handle the request accordingly.
Part of the AFE USB port start handling will have an exchange of control messages using the QMI protocol. The qc_audio_offload driver will populate the buffer information:
- Event ring base address
- EP transfer ring base address
and pass it along to the audio DSP. All endpoint management will now be handed over to the DSP, and the main processor is not involved in transfers.
Overall, implementing this feature will still expose separate sound card and PCM devices for both the platorm card and USB audio device: 0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D SM8250-MTP-WCD9380-WSA8810-VA-DMIC 1 [Audio ]: USB-Audio - USB Audio Generic USB Audio at usb-xhci-hcd.1.auto-1.4, high speed
This is to ensure that userspace ALSA entities can decide which route to take when executing the audio playback. In the above, if card#1 is selected, then USB audio data will take the legacy path over the USB PCM drivers, etc...
You would still need some sort of mutual exclusion to make sure the isoc endpoints are not used concurrently by the two cards. Relying on userspace intelligence to enforce that exclusion is not safe IMHO.
Sure, I think we can make the USB card as being used if the offloading path is currently being enabled. Kernel could return an error to userspace when this situation happens.
Intel looked at this sort of offload support a while ago and our directions were very different - for a variety of reasons USB offload is enabled on Windows platforms but remains a TODO for Linux. Rather than having two cards, you could have a single card and addition subdevices that expose the paths through the DSP. The benefits were that there was a single set of controls that userspace needed to know about, and volume settings were the same no matter which path you used (legacy or DSP-optimized paths). That's consistent with the directions to use 'Deep Buffer' PCM paths for local playback, it's the same idea of reducing power consumption with optimized routing.
Volume control would still be done through the legacy path as mentioned above. For example, if a USB headset w/ a HID interface exposed (for volume control) was connected, those HID events would be routed to userspace to adjust volume accordingly on the main processor. (although you're right about having separate controls still present - one for the ASoC card and another for USB card)
Another point is that there may be cases where the DSP paths are not available if the DSP memory and MCPS budget is exceeded. In those cases, the DSP parts needs the ability to notify userspace that the legacy path should be used.
If we ran into this scenario, the audio DSP AFE port start command would fail, and this would be propagated to the userspace entity. It could then potentially re-route to the legacy/non-offload path.
Another case to handle is that some USB devices can handle way more data than DSPs can chew, for example Pro audio boxes that can deal with 8ch 192kHz will typically use the legacy paths. Some also handle specific formats such as DSD over PCM. So it's quite likely that PCM devices for card0 and card1 above do NOT expose support for the same formats, or put differently that only a subset of the USB device capabilities are handled through the DSP.
Same as the above. We have programmed the USB backend to support the profiles that the audio DSP can handle. I assume if there was any other request, the userspace entity would fail the PCM open for that requested profile.
And last, power optimizations with DSPs typically come from additional latency helping put the SoC in low-power modes. That's not necessarily ideal for all usages, e.g. for music recording and mixing I am not convinced the DSP path would help at all.
That's true. At the same time, this feature is more for power related benefits, not specifically for performance. (although we haven't seen any performance related issues w/ this approach on the audio profiles the DSP supports) I think if its an audio profile that supports a high sample rate and large number of channels, then the DSP wouldn't be able to support it anyway, and userspace could still use the legacy path. This would allow for those high-performance audio devices to not be affected.
Thanks Wesley Cheng
On 12/23/22 17:31, Wesley Cheng wrote:
Several Qualcomm based chipsets can support USB audio offloading to a dedicated audio DSP, which can take over issuing transfers to the USB host controller. The intention is to reduce the load on the main processors in the SoC, and allow them to be placed into lower power modes.
It would be nice to clarify what you want to offload a) audio data transfers for isoc ports b) control for e.g. volume settings (those go to endpoint 0 IIRC) c) Both?
Thanks for sharing your experience, and inputs!
It would be the audio related endpoints only, so ISOC and potentially feedback ep.
That's good, that means there's a common basis for at least two separate hardware implementations.
This has a lot of implications on the design. ASoC/DPCM is mainly intended for audio data transfers, control is a separate problem with configurations handled with register settings or bus-specific commands.
Control would still be handled by the main processor.
Excellent, one more thing in common. Maintainers like this sort of alignment :-)
There are several parts to this design: 1. Adding ASoC binding layer 2. Create a USB backend for Q6DSP 3. Introduce XHCI interrupter support 4. Create vendor ops for the USB SND driver
Adding ASoC binding layer: soc-usb: Intention is to treat a USB port similar to a headphone jack. The port is always present on the device, but cable/pin status can be enabled/disabled. Expose mechanisms for USB backend ASoC drivers to communicate with USB SND.
Create a USB backend for Q6DSP: q6usb: Basic backend driver that will be responsible for maintaining the resources needed to initiate a playback stream using the Q6DSP. Will be the entity that checks to make sure the connected USB audio device supports the requested PCM format. If it does not, the PCM open call will fail, and userpsace ALSA can take action accordingly.
Introduce XHCI interrupter support: XHCI HCD supports multiple interrupters, which allows for events to be routed to different event rings. This is determined by "Interrupter Target" field specified in Section "6.4.1.1 Normal TRB" of the XHCI specification.
Events in the offloading case will be routed to an event ring that is assigned to the audio DSP.
To the best of my knowledge this isn't needed on Intel platforms, but that's something we would need to double-check.
Create vendor ops for the USB SND driver: qc_audio_offload: This particular driver has several components associated with it:
- QMI stream request handler
- XHCI interrupter and resource management
- audio DSP memory management
When the audio DSP wants to enable a playback stream, the request is first received by the ASoC platform sound card. Depending on the selected route, ASoC will bring up the individual DAIs in the path. The Q6USB backend DAI will send an AFE port start command (with enabling the USB playback path), and the audio DSP will handle the request accordingly.
Part of the AFE USB port start handling will have an exchange of control messages using the QMI protocol. The qc_audio_offload driver will populate the buffer information:
- Event ring base address
- EP transfer ring base address
and pass it along to the audio DSP. All endpoint management will now be handed over to the DSP, and the main processor is not involved in transfers.
Overall, implementing this feature will still expose separate sound card and PCM devices for both the platorm card and USB audio device: 0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D SM8250-MTP-WCD9380-WSA8810-VA-DMIC 1 [Audio ]: USB-Audio - USB Audio Generic USB Audio at usb-xhci-hcd.1.auto-1.4, high speed
This is to ensure that userspace ALSA entities can decide which route to take when executing the audio playback. In the above, if card#1 is selected, then USB audio data will take the legacy path over the USB PCM drivers, etc...
You would still need some sort of mutual exclusion to make sure the isoc endpoints are not used concurrently by the two cards. Relying on userspace intelligence to enforce that exclusion is not safe IMHO.
Sure, I think we can make the USB card as being used if the offloading path is currently being enabled. Kernel could return an error to userspace when this situation happens.
It's problematic for servers such as PipeWire/PulseAudio that open all possible PCMs to figure out what they support in terms of formats. I am not sure we can enforce a user-space serialization when discovering capabilities?
Intel looked at this sort of offload support a while ago and our directions were very different - for a variety of reasons USB offload is enabled on Windows platforms but remains a TODO for Linux. Rather than having two cards, you could have a single card and addition subdevices that expose the paths through the DSP. The benefits were that there was a single set of controls that userspace needed to know about, and volume settings were the same no matter which path you used (legacy or DSP-optimized paths). That's consistent with the directions to use 'Deep Buffer' PCM paths for local playback, it's the same idea of reducing power consumption with optimized routing.
Volume control would still be done through the legacy path as mentioned above. For example, if a USB headset w/ a HID interface exposed (for volume control) was connected, those HID events would be routed to userspace to adjust volume accordingly on the main processor. (although you're right about having separate controls still present - one for the ASoC card and another for USB card)
The two sets of controls implied by the use of two cards is really problematic IMHO. This adds complexity for userspace to figure out that the controls are really the same and synchronize/mirror changes.
The premise of offload is that it should really not get in the way of user-experience, design constructs that result in delayed starts/stop, changed volumes or quality differences should be avoided, or users/distros will disable this optimization.
One card with additional DSP-based PCM devices seems simpler to me in terms of usage, but it's not without technical challenges either: with the use of the ASoC topology framework we only know what the DSP supports when registering a card and probing the ASoC components.
The interaction between USB audio and ASoC would also be at least as complicated as display audio, in that it needs to work no matter what the probe order is, and even survive the Linux device/driver model requirement that there are no timing dependencies in the driver bind/unbind sequences.
Another point is that there may be cases where the DSP paths are not available if the DSP memory and MCPS budget is exceeded. In those cases, the DSP parts needs the ability to notify userspace that the legacy path should be used.
If we ran into this scenario, the audio DSP AFE port start command would fail, and this would be propagated to the userspace entity. It could then potentially re-route to the legacy/non-offload path.
'start' or 'open'? This is a rather important design difference. Usually we try to make decisions in the .open or .hw_params stage. The 'start' or 'trigger' are usually not meant to fail due to unavailable resources in ALSA.
Another case to handle is that some USB devices can handle way more data than DSPs can chew, for example Pro audio boxes that can deal with 8ch 192kHz will typically use the legacy paths. Some also handle specific formats such as DSD over PCM. So it's quite likely that PCM devices for card0 and card1 above do NOT expose support for the same formats, or put differently that only a subset of the USB device capabilities are handled through the DSP.
Same as the above. We have programmed the USB backend to support the profiles that the audio DSP can handle. I assume if there was any other request, the userspace entity would fail the PCM open for that requested profile.
What's not clear to me is whether there's any matching process between the DSP capabilities and what the USB device exposes? if the USB device is already way more complicated that what the ASoC back-end can deal with, why expose a card?
And last, power optimizations with DSPs typically come from additional latency helping put the SoC in low-power modes. That's not necessarily ideal for all usages, e.g. for music recording and mixing I am not convinced the DSP path would help at all.
That's true. At the same time, this feature is more for power related benefits, not specifically for performance. (although we haven't seen any performance related issues w/ this approach on the audio profiles the DSP supports) I think if its an audio profile that supports a high sample rate and large number of channels, then the DSP wouldn't be able to support it anyway, and userspace could still use the legacy path. This would allow for those high-performance audio devices to not be affected.
ok, we are aligned as well here. Excellent. With the on-going work to introduce 'Deep Buffer' capabilities, we'll have a need to tag PCM devices with a usage or 'modifier', or have this information in UCM/topology. Logic will have to be introduced in userspace to use the best routing, I would think this work can be reused for USB cases to indicate the offload solution is geared to power optimizations.
Hi Pierre,
On 1/6/2023 7:57 AM, Pierre-Louis Bossart wrote:
On 12/23/22 17:31, Wesley Cheng wrote:
Several Qualcomm based chipsets can support USB audio offloading to a dedicated audio DSP, which can take over issuing transfers to the USB host controller. The intention is to reduce the load on the main processors in the SoC, and allow them to be placed into lower power modes.
It would be nice to clarify what you want to offload a) audio data transfers for isoc ports b) control for e.g. volume settings (those go to endpoint 0 IIRC) c) Both?
Thanks for sharing your experience, and inputs!
It would be the audio related endpoints only, so ISOC and potentially feedback ep.
That's good, that means there's a common basis for at least two separate hardware implementations.
This has a lot of implications on the design. ASoC/DPCM is mainly intended for audio data transfers, control is a separate problem with configurations handled with register settings or bus-specific commands.
Control would still be handled by the main processor.
Excellent, one more thing in common. Maintainers like this sort of alignment :-)
There are several parts to this design: 1. Adding ASoC binding layer 2. Create a USB backend for Q6DSP 3. Introduce XHCI interrupter support 4. Create vendor ops for the USB SND driver
Adding ASoC binding layer: soc-usb: Intention is to treat a USB port similar to a headphone jack. The port is always present on the device, but cable/pin status can be enabled/disabled. Expose mechanisms for USB backend ASoC drivers to communicate with USB SND.
Create a USB backend for Q6DSP: q6usb: Basic backend driver that will be responsible for maintaining the resources needed to initiate a playback stream using the Q6DSP. Will be the entity that checks to make sure the connected USB audio device supports the requested PCM format. If it does not, the PCM open call will fail, and userpsace ALSA can take action accordingly.
Introduce XHCI interrupter support: XHCI HCD supports multiple interrupters, which allows for events to be routed to different event rings. This is determined by "Interrupter Target" field specified in Section "6.4.1.1 Normal TRB" of the XHCI specification.
Events in the offloading case will be routed to an event ring that is assigned to the audio DSP.
To the best of my knowledge this isn't needed on Intel platforms, but that's something we would need to double-check.
I think Mathias mentioned that he was looking into adding some XHCI secondary interrupter support as well. However, it did have some slightly different requirements compared to what this offloading feature is trying to do.
I'll first have to split up the XHCI/HCD changes into separate parts (interrupter specific and offloading specific), and then I'll work with him to see what can be improved from there.
Create vendor ops for the USB SND driver: qc_audio_offload: This particular driver has several components associated with it:
- QMI stream request handler
- XHCI interrupter and resource management
- audio DSP memory management
When the audio DSP wants to enable a playback stream, the request is first received by the ASoC platform sound card. Depending on the selected route, ASoC will bring up the individual DAIs in the path. The Q6USB backend DAI will send an AFE port start command (with enabling the USB playback path), and the audio DSP will handle the request accordingly.
Part of the AFE USB port start handling will have an exchange of control messages using the QMI protocol. The qc_audio_offload driver will populate the buffer information:
- Event ring base address
- EP transfer ring base address
and pass it along to the audio DSP. All endpoint management will now be handed over to the DSP, and the main processor is not involved in transfers.
Overall, implementing this feature will still expose separate sound card and PCM devices for both the platorm card and USB audio device: 0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D SM8250-MTP-WCD9380-WSA8810-VA-DMIC 1 [Audio ]: USB-Audio - USB Audio Generic USB Audio at usb-xhci-hcd.1.auto-1.4, high speed
This is to ensure that userspace ALSA entities can decide which route to take when executing the audio playback. In the above, if card#1 is selected, then USB audio data will take the legacy path over the USB PCM drivers, etc...
You would still need some sort of mutual exclusion to make sure the isoc endpoints are not used concurrently by the two cards. Relying on userspace intelligence to enforce that exclusion is not safe IMHO.
Sure, I think we can make the USB card as being used if the offloading path is currently being enabled. Kernel could return an error to userspace when this situation happens.
It's problematic for servers such as PipeWire/PulseAudio that open all possible PCMs to figure out what they support in terms of formats. I am not sure we can enforce a user-space serialization when discovering capabilities?
I see...I'm not too familiar yet with all the different implementations from userspace yet, so that is something I'll need to look up on the side. Takashi, would you happen to have any inputs with regards to how flexible PCM device selection can be from the userspace level? If the offload PCM device can't be supported, can it fallback to another PCM device?
Intel looked at this sort of offload support a while ago and our directions were very different - for a variety of reasons USB offload is enabled on Windows platforms but remains a TODO for Linux. Rather than having two cards, you could have a single card and addition subdevices that expose the paths through the DSP. The benefits were that there was a single set of controls that userspace needed to know about, and volume settings were the same no matter which path you used (legacy or DSP-optimized paths). That's consistent with the directions to use 'Deep Buffer' PCM paths for local playback, it's the same idea of reducing power consumption with optimized routing.
Volume control would still be done through the legacy path as mentioned above. For example, if a USB headset w/ a HID interface exposed (for volume control) was connected, those HID events would be routed to userspace to adjust volume accordingly on the main processor. (although you're right about having separate controls still present - one for the ASoC card and another for USB card)
The two sets of controls implied by the use of two cards is really problematic IMHO. This adds complexity for userspace to figure out that the controls are really the same and synchronize/mirror changes.
The premise of offload is that it should really not get in the way of
user-experience, design constructs that result in delayed starts/stop, changed volumes or quality differences should be avoided, or users/distros will disable this optimization.
Makes sense. I think in terms of controls, we know that for an USB audio device, anything will still be handled through the USB card. Again, since I'm not too familiar yet with all the userspace implementations, does it have mechanisms to treat the control and data interfaces separately?
One card with additional DSP-based PCM devices seems simpler to me in terms of usage, but it's not without technical challenges either: with the use of the ASoC topology framework we only know what the DSP supports when registering a card and probing the ASoC components.
The interaction between USB audio and ASoC would also be at least as complicated as display audio, in that it needs to work no matter what the probe order is, and even survive the Linux device/driver model requirement that there are no timing dependencies in the driver bind/unbind sequences.
Yes, this was my initial approach as well, but from the technical perspective it was very very messy, and potentially could have affected functionality on certain devices if not handled correctly. I think the difficult part was that the USB SND framework itself is an independent entity, and it was tough to dissect the portions which created PCM/sound card devices.
I don't think that was something which would have gone well if introduced all at once. It would require a lot of discussion before getting the proper implementation. At least this series introduces the initial communication between ASoC and USB SND, and maybe as use cases become clearer we can always improve/build on top of it.
Another point is that there may be cases where the DSP paths are not available if the DSP memory and MCPS budget is exceeded. In those cases, the DSP parts needs the ability to notify userspace that the legacy path should be used.
If we ran into this scenario, the audio DSP AFE port start command would fail, and this would be propagated to the userspace entity. It could then potentially re-route to the legacy/non-offload path.
'start' or 'open'? This is a rather important design difference. Usually we try to make decisions in the .open or .hw_params stage. The 'start' or 'trigger' are usually not meant to fail due to unavailable resources in ALSA.
This happens during the .prepare() phase.
Another case to handle is that some USB devices can handle way more data than DSPs can chew, for example Pro audio boxes that can deal with 8ch 192kHz will typically use the legacy paths. Some also handle specific formats such as DSD over PCM. So it's quite likely that PCM devices for card0 and card1 above do NOT expose support for the same formats, or put differently that only a subset of the USB device capabilities are handled through the DSP.
Same as the above. We have programmed the USB backend to support the profiles that the audio DSP can handle. I assume if there was any other request, the userspace entity would fail the PCM open for that requested profile.
What's not clear to me is whether there's any matching process between the DSP capabilities and what the USB device exposes? if the USB device is already way more complicated that what the ASoC back-end can deal with, why expose a card?
That's something I thought was done by the ASoC core. I can check that and see if that's the case. There is a check added in hw_params of our ASoC component where we do query the USB audio descriptors to ensure that the PCM format being used is supported by the device. I guess this is when the DSP capabilities are better than what the headset can support :).
And last, power optimizations with DSPs typically come from additional latency helping put the SoC in low-power modes. That's not necessarily ideal for all usages, e.g. for music recording and mixing I am not convinced the DSP path would help at all.
That's true. At the same time, this feature is more for power related benefits, not specifically for performance. (although we haven't seen any performance related issues w/ this approach on the audio profiles the DSP supports) I think if its an audio profile that supports a high sample rate and large number of channels, then the DSP wouldn't be able to support it anyway, and userspace could still use the legacy path. This would allow for those high-performance audio devices to not be affected.
ok, we are aligned as well here. Excellent. With the on-going work to introduce 'Deep Buffer' capabilities, we'll have a need to tag PCM devices with a usage or 'modifier', or have this information in UCM/topology. Logic will have to be introduced in userspace to use the best routing, I would think this work can be reused for USB cases to indicate the offload solution is geared to power optimizations.
Great, I like that idea to see if we can help userspace choose the desired path based on what the overall system is looking for. I wonder if that would also potentially help with some of the PCM device selection complexities you brought up as well. If the system just wants best possible performance then it would just completely ignore the power optimized (offload) path for any device.
Thanks Wesley Cheng
participants (8)
-
Alan Stern
-
Dmitry Baryshkov
-
Greg KH
-
Krzysztof Kozlowski
-
Mark Brown
-
Mathias Nyman
-
Oliver Neukum
-
Pierre-Louis Bossart
-
Rob Herring
-
Sergey Shtylyov
-
Takashi Iwai
-
Wesley Cheng