[alsa-devel] [PATCH v10 0/4] Media Device Allocator API
Media Device Allocator API to allows multiple drivers share a media device. This API solves a very common use-case for media devices where one physical device (an USB stick) provides both audio and video. When such media device exposes a standard USB Audio class, a proprietary Video class, two or more independent drivers will share a single physical USB bridge. In such cases, it is necessary to coordinate access to the shared resource.
Using this API, drivers can allocate a media device with the shared struct device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.
- This patch series is tested on 5.0-rc3 and addresses comments on v9 series from Hans Verkuil. - v9 was tested on 4.20-rc6. - Tested sharing resources with kaffeine, vlc, xawtv, tvtime, and arecord. When analog is streaming, digital and audio user-space applications detect that the tuner is busy and exit. When digital is streaming, analog and audio applications detect that the tuner is busy and exit. When arecord is owns the tuner, digital and analog detect that the tuner is busy and exit. - Tested media device allocator API with bind/unbind testing on snd-usb-audio and au0828 drivers to make sure /dev/mediaX is released only when the last driver is unbound. - Addressed review comments from Hans on the RFC v8 (rebased on 4.19) - Updated change log to describe the use-case more clearly. - No changes to 0001,0002 code since the v7 referenced below. - 0003 is a new patch to enable ALSA defines that have been disabled for kernel between 4.9 and 4.19. - Minor merge conflict resolution in 0004. - Added SPDX to new files.
Changes since v9: - Patch 1: Fix mutex assert warning from find_module() calls. This code was written before the change to find_module() that requires callers to hold module_mutex. I missed this during my testing on 4.20-rc6. Hans Verkuil reported the problem. - Patch 4: sound/usb: Initializes all the entities it can before registering the device based on comments from Hans Verkuil - Carried Reviewed-by tag from Takashi Iwai for the sound from v9. - No changes to Patches 2 and 3.
References: https://lkml.org/lkml/2018/11/2/169 https://www.mail-archive.com/linux-media@vger.kernel.org/msg105854.html
Shuah Khan (4): media: Media Device Allocator API media: change au0828 to use Media Device Allocator API media: media.h: Enable ALSA MEDIA_INTF_T* interface types sound/usb: Use Media Controller API to share media resources
Documentation/media/kapi/mc-core.rst | 41 ++++ drivers/media/Makefile | 4 + drivers/media/media-dev-allocator.c | 144 +++++++++++ drivers/media/usb/au0828/au0828-core.c | 12 +- drivers/media/usb/au0828/au0828.h | 1 + include/media/media-dev-allocator.h | 53 ++++ include/uapi/linux/media.h | 25 +- sound/usb/Kconfig | 4 + sound/usb/Makefile | 2 + sound/usb/card.c | 14 ++ sound/usb/card.h | 3 + sound/usb/media.c | 327 +++++++++++++++++++++++++ sound/usb/media.h | 74 ++++++ sound/usb/mixer.h | 3 + sound/usb/pcm.c | 29 ++- sound/usb/quirks-table.h | 1 + sound/usb/stream.c | 2 + sound/usb/usbaudio.h | 6 + 18 files changed, 723 insertions(+), 22 deletions(-) create mode 100644 drivers/media/media-dev-allocator.c create mode 100644 include/media/media-dev-allocator.h create mode 100644 sound/usb/media.c create mode 100644 sound/usb/media.h
Media Device Allocator API to allows multiple drivers share a media device. This API solves a very common use-case for media devices where one physical device (an USB stick) provides both audio and video. When such media device exposes a standard USB Audio class, a proprietary Video class, two or more independent drivers will share a single physical USB bridge. In such cases, it is necessary to coordinate access to the shared resource.
Using this API, drivers can allocate a media device with the shared struct device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.
Signed-off-by: Shuah Khan shuah@kernel.org --- Documentation/media/kapi/mc-core.rst | 41 ++++++++ drivers/media/Makefile | 4 + drivers/media/media-dev-allocator.c | 144 +++++++++++++++++++++++++++ include/media/media-dev-allocator.h | 53 ++++++++++ 4 files changed, 242 insertions(+) create mode 100644 drivers/media/media-dev-allocator.c create mode 100644 include/media/media-dev-allocator.h
diff --git a/Documentation/media/kapi/mc-core.rst b/Documentation/media/kapi/mc-core.rst index 0bcfeadbc52d..07f2a6a90af2 100644 --- a/Documentation/media/kapi/mc-core.rst +++ b/Documentation/media/kapi/mc-core.rst @@ -259,6 +259,45 @@ Subsystems should facilitate link validation by providing subsystem specific helper functions to provide easy access for commonly needed information, and in the end provide a way to use driver-specific callbacks.
+Media Controller Device Allocator API +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +When the media device belongs to more than one driver, the shared media +device is allocated with the shared struct device as the key for look ups. + +The shared media device should stay in registered state until the last +driver unregisters it. In addition, the media device should be released when +all the references are released. Each driver gets a reference to the media +device during probe, when it allocates the media device. If media device is +already allocated, the allocate API bumps up the refcount and returns the +existing media device. The driver puts the reference back in its disconnect +routine when it calls :c:func:`media_device_delete()`. + +The media device is unregistered and cleaned up from the kref put handler to +ensure that the media device stays in registered state until the last driver +unregisters the media device. + +**Driver Usage** + +Drivers should use the appropriate media-core routines to manage the shared +media device life-time handling the two states: +1. allocate -> register -> delete +2. get reference to already registered device -> delete + +call :c:func:`media_device_delete()` routine to make sure the shared media +device delete is handled correctly. + +**driver probe:** +Call :c:func:`media_device_usb_allocate()` to allocate or get a reference +Call :c:func:`media_device_register()`, if media devnode isn't registered + +**driver disconnect:** +Call :c:func:`media_device_delete()` to free the media_device. Freeing is +handled by the kref put handler. + +API Definitions +^^^^^^^^^^^^^^^ + .. kernel-doc:: include/media/media-device.h
.. kernel-doc:: include/media/media-devnode.h @@ -266,3 +305,5 @@ in the end provide a way to use driver-specific callbacks. .. kernel-doc:: include/media/media-entity.h
.. kernel-doc:: include/media/media-request.h + +.. kernel-doc:: include/media/media-dev-allocator.h diff --git a/drivers/media/Makefile b/drivers/media/Makefile index 985d35ec6b29..1d7653318af6 100644 --- a/drivers/media/Makefile +++ b/drivers/media/Makefile @@ -6,6 +6,10 @@ media-objs := media-device.o media-devnode.o media-entity.o \ media-request.o
+ifeq ($(CONFIG_USB),y) + media-objs += media-dev-allocator.o +endif + # # I2C drivers should come before other drivers, otherwise they'll fail # when compiled as builtin drivers diff --git a/drivers/media/media-dev-allocator.c b/drivers/media/media-dev-allocator.c new file mode 100644 index 000000000000..4606456c1e86 --- /dev/null +++ b/drivers/media/media-dev-allocator.c @@ -0,0 +1,144 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * media-dev-allocator.c - Media Controller Device Allocator API + * + * Copyright (c) 2018 Shuah Khan shuah@kernel.org + * + * Credits: Suggested by Laurent Pinchart laurent.pinchart@ideasonboard.com + */ + +/* + * This file adds a global refcounted Media Controller Device Instance API. + * A system wide global media device list is managed and each media device + * includes a kref count. The last put on the media device releases the media + * device instance. + * + */ + +#include <linux/kref.h> +#include <linux/module.h> +#include <linux/slab.h> +#include <linux/usb.h> + +#include <media/media-device.h> + +static LIST_HEAD(media_device_list); +static DEFINE_MUTEX(media_device_lock); + +struct media_device_instance { + struct media_device mdev; + struct module *owner; + struct list_head list; + struct kref refcount; +}; + +static inline struct media_device_instance * +to_media_device_instance(struct media_device *mdev) +{ + return container_of(mdev, struct media_device_instance, mdev); +} + +static void media_device_instance_release(struct kref *kref) +{ + struct media_device_instance *mdi = + container_of(kref, struct media_device_instance, refcount); + + dev_dbg(mdi->mdev.dev, "%s: mdev=%p\n", __func__, &mdi->mdev); + + mutex_lock(&media_device_lock); + + media_device_unregister(&mdi->mdev); + media_device_cleanup(&mdi->mdev); + + list_del(&mdi->list); + mutex_unlock(&media_device_lock); + + kfree(mdi); +} + +/* Callers should hold media_device_lock when calling this function */ +static struct media_device *__media_device_get(struct device *dev, + const char *module_name, + struct module *modp) +{ + struct media_device_instance *mdi; + + list_for_each_entry(mdi, &media_device_list, list) { + + if (mdi->mdev.dev != dev) + continue; + + kref_get(&mdi->refcount); + + /* get module reference for the media_device owner */ + if (modp != mdi->owner && !try_module_get(mdi->owner)) + dev_err(dev, "%s: try_module_get() error\n", __func__); + dev_dbg(dev, "%s: get mdev=%p module_name %s\n", + __func__, &mdi->mdev, module_name); + return &mdi->mdev; + } + + mdi = kzalloc(sizeof(*mdi), GFP_KERNEL); + if (!mdi) + return NULL; + + mdi->owner = modp; + kref_init(&mdi->refcount); + list_add_tail(&mdi->list, &media_device_list); + + dev_dbg(dev, "%s: alloc mdev=%p module_name %s\n", __func__, + &mdi->mdev, module_name); + return &mdi->mdev; +} + +#if IS_ENABLED(CONFIG_USB) +struct media_device *media_device_usb_allocate(struct usb_device *udev, + const char *module_name) +{ + struct media_device *mdev; + struct module *modptr; + + mutex_lock(&module_mutex); + modptr = find_module(module_name); + mutex_unlock(&module_mutex); + + mutex_lock(&media_device_lock); + mdev = __media_device_get(&udev->dev, module_name, modptr); + if (!mdev) { + mutex_unlock(&media_device_lock); + return ERR_PTR(-ENOMEM); + } + + /* check if media device is already initialized */ + if (!mdev->dev) + __media_device_usb_init(mdev, udev, udev->product, + module_name); + mutex_unlock(&media_device_lock); + return mdev; +} +EXPORT_SYMBOL_GPL(media_device_usb_allocate); +#endif + +void media_device_delete(struct media_device *mdev, const char *module_name) +{ + struct media_device_instance *mdi = to_media_device_instance(mdev); + struct module *modptr; + + dev_dbg(mdi->mdev.dev, "%s: mdev=%p module_name %s\n", + __func__, &mdi->mdev, module_name); + + mutex_lock(&module_mutex); + modptr = find_module(module_name); + mutex_unlock(&module_mutex); + + mutex_lock(&media_device_lock); + /* put module reference if media_device owner is not THIS_MODULE */ + if (mdi->owner != modptr) { + module_put(mdi->owner); + dev_dbg(mdi->mdev.dev, + "%s decremented owner module reference\n", __func__); + } + mutex_unlock(&media_device_lock); + kref_put(&mdi->refcount, media_device_instance_release); +} +EXPORT_SYMBOL_GPL(media_device_delete); diff --git a/include/media/media-dev-allocator.h b/include/media/media-dev-allocator.h new file mode 100644 index 000000000000..9164795e911c --- /dev/null +++ b/include/media/media-dev-allocator.h @@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* + * media-dev-allocator.h - Media Controller Device Allocator API + * + * Copyright (c) 2018 Shuah Khan shuah@kernel.org + * + * Credits: Suggested by Laurent Pinchart laurent.pinchart@ideasonboard.com + */ + +/* + * This file adds a global ref-counted Media Controller Device Instance API. + * A system wide global media device list is managed and each media device + * includes a kref count. The last put on the media device releases the media + * device instance. + */ + +#ifndef _MEDIA_DEV_ALLOCTOR_H +#define _MEDIA_DEV_ALLOCTOR_H + +struct usb_device; + +#if defined(CONFIG_MEDIA_CONTROLLER) && defined(CONFIG_USB) +/** + * media_device_usb_allocate() - Allocate and return struct &media device + * + * @udev: struct &usb_device pointer + * @module_name: should be filled with %KBUILD_MODNAME + * + * This interface should be called to allocate a Media Device when multiple + * drivers share usb_device and the media device. This interface allocates + * &media_device structure and calls media_device_usb_init() to initialize + * it. + * + */ +struct media_device *media_device_usb_allocate(struct usb_device *udev, + char *module_name); +/** + * media_device_delete() - Release media device. Calls kref_put(). + * + * @mdev: struct &media_device pointer + * @module_name: should be filled with %KBUILD_MODNAME + * + * This interface should be called to put Media Device Instance kref. + */ +void media_device_delete(struct media_device *mdev, char *module_name); +#else +static inline struct media_device *media_device_usb_allocate( + struct usb_device *udev, char *module_name) + { return NULL; } +static inline void media_device_delete( + struct media_device *mdev, char *module_name) { } +#endif /* CONFIG_MEDIA_CONTROLLER */ +#endif
Hi Shuah,
On Thu, Jan 24, 2019 at 01:32:38PM -0700, Shuah Khan wrote:
Media Device Allocator API to allows multiple drivers share a media device. This API solves a very common use-case for media devices where one physical device (an USB stick) provides both audio and video. When such media device exposes a standard USB Audio class, a proprietary Video class, two or more independent drivers will share a single physical USB bridge. In such cases, it is necessary to coordinate access to the shared resource.
Using this API, drivers can allocate a media device with the shared struct device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.
Are there real, non-USB devices that could use the same API?
Signed-off-by: Shuah Khan shuah@kernel.org
Documentation/media/kapi/mc-core.rst | 41 ++++++++ drivers/media/Makefile | 4 + drivers/media/media-dev-allocator.c | 144 +++++++++++++++++++++++++++ include/media/media-dev-allocator.h | 53 ++++++++++ 4 files changed, 242 insertions(+) create mode 100644 drivers/media/media-dev-allocator.c create mode 100644 include/media/media-dev-allocator.h
diff --git a/Documentation/media/kapi/mc-core.rst b/Documentation/media/kapi/mc-core.rst index 0bcfeadbc52d..07f2a6a90af2 100644 --- a/Documentation/media/kapi/mc-core.rst +++ b/Documentation/media/kapi/mc-core.rst @@ -259,6 +259,45 @@ Subsystems should facilitate link validation by providing subsystem specific helper functions to provide easy access for commonly needed information, and in the end provide a way to use driver-specific callbacks.
+Media Controller Device Allocator API +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+When the media device belongs to more than one driver, the shared media +device is allocated with the shared struct device as the key for look ups.
+The shared media device should stay in registered state until the last +driver unregisters it. In addition, the media device should be released when +all the references are released. Each driver gets a reference to the media +device during probe, when it allocates the media device. If media device is +already allocated, the allocate API bumps up the refcount and returns the +existing media device. The driver puts the reference back in its disconnect +routine when it calls :c:func:`media_device_delete()`.
+The media device is unregistered and cleaned up from the kref put handler to +ensure that the media device stays in registered state until the last driver +unregisters the media device.
+**Driver Usage**
+Drivers should use the appropriate media-core routines to manage the shared +media device life-time handling the two states: +1. allocate -> register -> delete +2. get reference to already registered device -> delete
+call :c:func:`media_device_delete()` routine to make sure the shared media +device delete is handled correctly.
+**driver probe:** +Call :c:func:`media_device_usb_allocate()` to allocate or get a reference +Call :c:func:`media_device_register()`, if media devnode isn't registered
+**driver disconnect:** +Call :c:func:`media_device_delete()` to free the media_device. Freeing is +handled by the kref put handler.
+API Definitions +^^^^^^^^^^^^^^^
.. kernel-doc:: include/media/media-device.h
.. kernel-doc:: include/media/media-devnode.h @@ -266,3 +305,5 @@ in the end provide a way to use driver-specific callbacks. .. kernel-doc:: include/media/media-entity.h
.. kernel-doc:: include/media/media-request.h
+.. kernel-doc:: include/media/media-dev-allocator.h diff --git a/drivers/media/Makefile b/drivers/media/Makefile index 985d35ec6b29..1d7653318af6 100644 --- a/drivers/media/Makefile +++ b/drivers/media/Makefile @@ -6,6 +6,10 @@ media-objs := media-device.o media-devnode.o media-entity.o \ media-request.o
+ifeq ($(CONFIG_USB),y)
- media-objs += media-dev-allocator.o
+endif
# # I2C drivers should come before other drivers, otherwise they'll fail # when compiled as builtin drivers diff --git a/drivers/media/media-dev-allocator.c b/drivers/media/media-dev-allocator.c new file mode 100644 index 000000000000..4606456c1e86 --- /dev/null +++ b/drivers/media/media-dev-allocator.c @@ -0,0 +1,144 @@ +// SPDX-License-Identifier: GPL-2.0-or-later
GPL-2.0+
+/*
- media-dev-allocator.c - Media Controller Device Allocator API
- Copyright (c) 2018 Shuah Khan shuah@kernel.org
- Credits: Suggested by Laurent Pinchart laurent.pinchart@ideasonboard.com
- */
+/*
- This file adds a global refcounted Media Controller Device Instance API.
- A system wide global media device list is managed and each media device
- includes a kref count. The last put on the media device releases the media
- device instance.
- */
+#include <linux/kref.h> +#include <linux/module.h> +#include <linux/slab.h> +#include <linux/usb.h>
+#include <media/media-device.h>
+static LIST_HEAD(media_device_list); +static DEFINE_MUTEX(media_device_lock);
+struct media_device_instance {
- struct media_device mdev;
- struct module *owner;
- struct list_head list;
- struct kref refcount;
+};
+static inline struct media_device_instance * +to_media_device_instance(struct media_device *mdev) +{
- return container_of(mdev, struct media_device_instance, mdev);
+}
+static void media_device_instance_release(struct kref *kref) +{
- struct media_device_instance *mdi =
container_of(kref, struct media_device_instance, refcount);- dev_dbg(mdi->mdev.dev, "%s: mdev=%p\n", __func__, &mdi->mdev);
- mutex_lock(&media_device_lock);
- media_device_unregister(&mdi->mdev);
- media_device_cleanup(&mdi->mdev);
This is a problem, albeit not really more of a problem than it is in a driver. The refcounting changes can be made here instead. I'll take this into account in the media device refcounting series I'm planning to start working on again; would you be perhaps able to help testing with this device once I have patches in that shape? I have no access to the hardware.
- list_del(&mdi->list);
- mutex_unlock(&media_device_lock);
- kfree(mdi);
+}
+/* Callers should hold media_device_lock when calling this function */ +static struct media_device *__media_device_get(struct device *dev,
const char *module_name,struct module *modp)+{
- struct media_device_instance *mdi;
- list_for_each_entry(mdi, &media_device_list, list) {
if (mdi->mdev.dev != dev)continue;kref_get(&mdi->refcount);/* get module reference for the media_device owner */if (modp != mdi->owner && !try_module_get(mdi->owner))dev_err(dev, "%s: try_module_get() error\n", __func__);dev_dbg(dev, "%s: get mdev=%p module_name %s\n",__func__, &mdi->mdev, module_name);return &mdi->mdev;- }
- mdi = kzalloc(sizeof(*mdi), GFP_KERNEL);
- if (!mdi)
return NULL;- mdi->owner = modp;
- kref_init(&mdi->refcount);
- list_add_tail(&mdi->list, &media_device_list);
- dev_dbg(dev, "%s: alloc mdev=%p module_name %s\n", __func__,
&mdi->mdev, module_name);- return &mdi->mdev;
+}
+#if IS_ENABLED(CONFIG_USB)
You already compile the file only if CONFIG_USB is enabled. I think you could remove this.
+struct media_device *media_device_usb_allocate(struct usb_device *udev,
const char *module_name)
I'd like to suggest working based on usb_interface instead of usb_device here: that object already exists and you can find out the device based on it. It seems all callers of this function already have the usb_interface around.
+{
- struct media_device *mdev;
- struct module *modptr;
- mutex_lock(&module_mutex);
- modptr = find_module(module_name);
- mutex_unlock(&module_mutex);
- mutex_lock(&media_device_lock);
- mdev = __media_device_get(&udev->dev, module_name, modptr);
- if (!mdev) {
mutex_unlock(&media_device_lock);return ERR_PTR(-ENOMEM);- }
- /* check if media device is already initialized */
- if (!mdev->dev)
__media_device_usb_init(mdev, udev, udev->product,module_name);- mutex_unlock(&media_device_lock);
- return mdev;
+} +EXPORT_SYMBOL_GPL(media_device_usb_allocate); +#endif
+void media_device_delete(struct media_device *mdev, const char *module_name)
Same here. The use of the module name seems a bit hackish to me, albeit I suppose it'd work, too.
+{
- struct media_device_instance *mdi = to_media_device_instance(mdev);
- struct module *modptr;
- dev_dbg(mdi->mdev.dev, "%s: mdev=%p module_name %s\n",
__func__, &mdi->mdev, module_name);- mutex_lock(&module_mutex);
- modptr = find_module(module_name);
- mutex_unlock(&module_mutex);
- mutex_lock(&media_device_lock);
- /* put module reference if media_device owner is not THIS_MODULE */
- if (mdi->owner != modptr) {
module_put(mdi->owner);dev_dbg(mdi->mdev.dev,"%s decremented owner module reference\n", __func__);- }
- mutex_unlock(&media_device_lock);
- kref_put(&mdi->refcount, media_device_instance_release);
+} +EXPORT_SYMBOL_GPL(media_device_delete); diff --git a/include/media/media-dev-allocator.h b/include/media/media-dev-allocator.h new file mode 100644 index 000000000000..9164795e911c --- /dev/null +++ b/include/media/media-dev-allocator.h @@ -0,0 +1,53 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/*
- media-dev-allocator.h - Media Controller Device Allocator API
- Copyright (c) 2018 Shuah Khan shuah@kernel.org
- Credits: Suggested by Laurent Pinchart laurent.pinchart@ideasonboard.com
- */
+/*
- This file adds a global ref-counted Media Controller Device Instance API.
- A system wide global media device list is managed and each media device
- includes a kref count. The last put on the media device releases the media
- device instance.
- */
+#ifndef _MEDIA_DEV_ALLOCTOR_H +#define _MEDIA_DEV_ALLOCTOR_H
+struct usb_device;
+#if defined(CONFIG_MEDIA_CONTROLLER) && defined(CONFIG_USB) +/**
- media_device_usb_allocate() - Allocate and return struct &media device
- @udev: struct &usb_device pointer
- @module_name: should be filled with %KBUILD_MODNAME
- This interface should be called to allocate a Media Device when multiple
- drivers share usb_device and the media device. This interface allocates
- &media_device structure and calls media_device_usb_init() to initialize
- it.
- */
+struct media_device *media_device_usb_allocate(struct usb_device *udev,
char *module_name);+/**
- media_device_delete() - Release media device. Calls kref_put().
- @mdev: struct &media_device pointer
- @module_name: should be filled with %KBUILD_MODNAME
- This interface should be called to put Media Device Instance kref.
- */
+void media_device_delete(struct media_device *mdev, char *module_name); +#else +static inline struct media_device *media_device_usb_allocate(
struct usb_device *udev, char *module_name){ return NULL; }+static inline void media_device_delete(
struct media_device *mdev, char *module_name) { }+#endif /* CONFIG_MEDIA_CONTROLLER */ +#endif
Hi Sakari,
On 1/25/19 8:38 AM, Sakari Ailus wrote:
Hi Shuah,
On Thu, Jan 24, 2019 at 01:32:38PM -0700, Shuah Khan wrote:
Media Device Allocator API to allows multiple drivers share a media device. This API solves a very common use-case for media devices where one physical device (an USB stick) provides both audio and video. When such media device exposes a standard USB Audio class, a proprietary Video class, two or more independent drivers will share a single physical USB bridge. In such cases, it is necessary to coordinate access to the shared resource.
Using this API, drivers can allocate a media device with the shared struct device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.
Are there real, non-USB devices that could use the same API?
There might be. I don't have any to test. This patch is restricted to USB at the moment.
Signed-off-by: Shuah Khan shuah@kernel.org
Documentation/media/kapi/mc-core.rst | 41 ++++++++ drivers/media/Makefile | 4 + drivers/media/media-dev-allocator.c | 144 +++++++++++++++++++++++++++ include/media/media-dev-allocator.h | 53 ++++++++++ 4 files changed, 242 insertions(+) create mode 100644 drivers/media/media-dev-allocator.c create mode 100644 include/media/media-dev-allocator.h
diff --git a/Documentation/media/kapi/mc-core.rst b/Documentation/media/kapi/mc-core.rst index 0bcfeadbc52d..07f2a6a90af2 100644 --- a/Documentation/media/kapi/mc-core.rst +++ b/Documentation/media/kapi/mc-core.rst @@ -259,6 +259,45 @@ Subsystems should facilitate link validation by providing subsystem specific helper functions to provide easy access for commonly needed information, and in the end provide a way to use driver-specific callbacks.
+Media Controller Device Allocator API +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+When the media device belongs to more than one driver, the shared media +device is allocated with the shared struct device as the key for look ups.
+The shared media device should stay in registered state until the last +driver unregisters it. In addition, the media device should be released when +all the references are released. Each driver gets a reference to the media +device during probe, when it allocates the media device. If media device is +already allocated, the allocate API bumps up the refcount and returns the +existing media device. The driver puts the reference back in its disconnect +routine when it calls :c:func:`media_device_delete()`.
+The media device is unregistered and cleaned up from the kref put handler to +ensure that the media device stays in registered state until the last driver +unregisters the media device.
+**Driver Usage**
+Drivers should use the appropriate media-core routines to manage the shared +media device life-time handling the two states: +1. allocate -> register -> delete +2. get reference to already registered device -> delete
+call :c:func:`media_device_delete()` routine to make sure the shared media +device delete is handled correctly.
+**driver probe:** +Call :c:func:`media_device_usb_allocate()` to allocate or get a reference +Call :c:func:`media_device_register()`, if media devnode isn't registered
+**driver disconnect:** +Call :c:func:`media_device_delete()` to free the media_device. Freeing is +handled by the kref put handler.
+API Definitions +^^^^^^^^^^^^^^^
.. kernel-doc:: include/media/media-device.h
.. kernel-doc:: include/media/media-devnode.h
@@ -266,3 +305,5 @@ in the end provide a way to use driver-specific callbacks. .. kernel-doc:: include/media/media-entity.h
.. kernel-doc:: include/media/media-request.h
+.. kernel-doc:: include/media/media-dev-allocator.h diff --git a/drivers/media/Makefile b/drivers/media/Makefile index 985d35ec6b29..1d7653318af6 100644 --- a/drivers/media/Makefile +++ b/drivers/media/Makefile @@ -6,6 +6,10 @@ media-objs := media-device.o media-devnode.o media-entity.o \ media-request.o
+ifeq ($(CONFIG_USB),y)
- media-objs += media-dev-allocator.o
+endif
- # # I2C drivers should come before other drivers, otherwise they'll fail # when compiled as builtin drivers
diff --git a/drivers/media/media-dev-allocator.c b/drivers/media/media-dev-allocator.c new file mode 100644 index 000000000000..4606456c1e86 --- /dev/null +++ b/drivers/media/media-dev-allocator.c @@ -0,0 +1,144 @@ +// SPDX-License-Identifier: GPL-2.0-or-later
GPL-2.0+
I would like to address this in a follow-on patch instead of re-doing the series again. Hope that is okay.
+/*
- media-dev-allocator.c - Media Controller Device Allocator API
- Copyright (c) 2018 Shuah Khan shuah@kernel.org
- Credits: Suggested by Laurent Pinchart laurent.pinchart@ideasonboard.com
- */
+/*
- This file adds a global refcounted Media Controller Device Instance API.
- A system wide global media device list is managed and each media device
- includes a kref count. The last put on the media device releases the media
- device instance.
- */
+#include <linux/kref.h> +#include <linux/module.h> +#include <linux/slab.h> +#include <linux/usb.h>
+#include <media/media-device.h>
+static LIST_HEAD(media_device_list); +static DEFINE_MUTEX(media_device_lock);
+struct media_device_instance {
- struct media_device mdev;
- struct module *owner;
- struct list_head list;
- struct kref refcount;
+};
+static inline struct media_device_instance * +to_media_device_instance(struct media_device *mdev) +{
- return container_of(mdev, struct media_device_instance, mdev);
+}
+static void media_device_instance_release(struct kref *kref) +{
- struct media_device_instance *mdi =
container_of(kref, struct media_device_instance, refcount);- dev_dbg(mdi->mdev.dev, "%s: mdev=%p\n", __func__, &mdi->mdev);
- mutex_lock(&media_device_lock);
- media_device_unregister(&mdi->mdev);
- media_device_cleanup(&mdi->mdev);
This is a problem, albeit not really more of a problem than it is in a driver.
Okay good to know that it can be addressed in your media device refcounting series.
The refcounting changes can be made here instead. I'll take this
into account in the media device refcounting series I'm planning to start working on again; would you be perhaps able to help testing with this device once I have patches in that shape? I have no access to the hardware.
Absolutely. I will be happy to help you with testing on the same hardware, I used for this series.
- list_del(&mdi->list);
- mutex_unlock(&media_device_lock);
- kfree(mdi);
+}
+/* Callers should hold media_device_lock when calling this function */ +static struct media_device *__media_device_get(struct device *dev,
const char *module_name,struct module *modp)+{
- struct media_device_instance *mdi;
- list_for_each_entry(mdi, &media_device_list, list) {
if (mdi->mdev.dev != dev)continue;kref_get(&mdi->refcount);/* get module reference for the media_device owner */if (modp != mdi->owner && !try_module_get(mdi->owner))dev_err(dev, "%s: try_module_get() error\n", __func__);dev_dbg(dev, "%s: get mdev=%p module_name %s\n",__func__, &mdi->mdev, module_name);return &mdi->mdev;- }
- mdi = kzalloc(sizeof(*mdi), GFP_KERNEL);
- if (!mdi)
return NULL;- mdi->owner = modp;
- kref_init(&mdi->refcount);
- list_add_tail(&mdi->list, &media_device_list);
- dev_dbg(dev, "%s: alloc mdev=%p module_name %s\n", __func__,
&mdi->mdev, module_name);- return &mdi->mdev;
+}
+#if IS_ENABLED(CONFIG_USB)
You already compile the file only if CONFIG_USB is enabled. I think you could remove this.
+struct media_device *media_device_usb_allocate(struct usb_device *udev,
const char *module_name)I'd like to suggest working based on usb_interface instead of usb_device here: that object already exists and you can find out the device based on it. It seems all callers of this function already have the usb_interface around.
Is there an advantage to using interface instead of the parent device? Is there a problem doing it this way?
+{
- struct media_device *mdev;
- struct module *modptr;
- mutex_lock(&module_mutex);
- modptr = find_module(module_name);
- mutex_unlock(&module_mutex);
- mutex_lock(&media_device_lock);
- mdev = __media_device_get(&udev->dev, module_name, modptr);
- if (!mdev) {
mutex_unlock(&media_device_lock);return ERR_PTR(-ENOMEM);- }
- /* check if media device is already initialized */
- if (!mdev->dev)
__media_device_usb_init(mdev, udev, udev->product,module_name);- mutex_unlock(&media_device_lock);
- return mdev;
+} +EXPORT_SYMBOL_GPL(media_device_usb_allocate); +#endif
+void media_device_delete(struct media_device *mdev, const char *module_name)
Same here. The use of the module name seems a bit hackish to me, albeit I suppose it'd work, too.
What is hackish about it? I found it very useful to use it in debug and error messages that are user-friendly.
thanks, -- Shuah
Media Device Allocator API to allows multiple drivers share a media device. This API solves a very common use-case for media devices where one physical device (an USB stick) provides both audio and video. When such media device exposes a standard USB Audio class, a proprietary Video class, two or more independent drivers will share a single physical USB bridge. In such cases, it is necessary to coordinate access to the shared resource.
Using this API, drivers can allocate a media device with the shared struct device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.
Change au0828 to use Media Device Allocator API to allocate media device with the parent usb struct device as the key, so it can be shared with the snd_usb_audio driver.
Signed-off-by: Shuah Khan shuah@kernel.org --- drivers/media/usb/au0828/au0828-core.c | 12 ++++-------- drivers/media/usb/au0828/au0828.h | 1 + 2 files changed, 5 insertions(+), 8 deletions(-)
diff --git a/drivers/media/usb/au0828/au0828-core.c b/drivers/media/usb/au0828/au0828-core.c index 1fdb1601dc65..4b0a395d59aa 100644 --- a/drivers/media/usb/au0828/au0828-core.c +++ b/drivers/media/usb/au0828/au0828-core.c @@ -155,9 +155,7 @@ static void au0828_unregister_media_device(struct au0828_dev *dev) dev->media_dev->disable_source = NULL; mutex_unlock(&mdev->graph_mutex);
- media_device_unregister(dev->media_dev); - media_device_cleanup(dev->media_dev); - kfree(dev->media_dev); + media_device_delete(dev->media_dev, KBUILD_MODNAME); dev->media_dev = NULL; #endif } @@ -210,14 +208,10 @@ static int au0828_media_device_init(struct au0828_dev *dev, #ifdef CONFIG_MEDIA_CONTROLLER struct media_device *mdev;
- mdev = kzalloc(sizeof(*mdev), GFP_KERNEL); + mdev = media_device_usb_allocate(udev, KBUILD_MODNAME); if (!mdev) return -ENOMEM;
- /* check if media device is already initialized */ - if (!mdev->dev) - media_device_usb_init(mdev, udev, udev->product); - dev->media_dev = mdev; #endif return 0; @@ -480,6 +474,8 @@ static int au0828_media_device_register(struct au0828_dev *dev, /* register media device */ ret = media_device_register(dev->media_dev); if (ret) { + media_device_delete(dev->media_dev, KBUILD_MODNAME); + dev->media_dev = NULL; dev_err(&udev->dev, "Media Device Register Error: %d\n", ret); return ret; diff --git a/drivers/media/usb/au0828/au0828.h b/drivers/media/usb/au0828/au0828.h index 004eadef55c7..7dbe3db15ebe 100644 --- a/drivers/media/usb/au0828/au0828.h +++ b/drivers/media/usb/au0828/au0828.h @@ -31,6 +31,7 @@ #include <media/v4l2-ctrls.h> #include <media/v4l2-fh.h> #include <media/media-device.h> +#include <media/media-dev-allocator.h>
/* DVB */ #include <media/demux.h>
Move PCM_CAPTURE, PCM_PLAYBACK, and CONTROL ALSA MEDIA_INTF_T* interface types back into __KERNEL__ scope to get ready for adding ALSA support for these to the media controller.
Signed-off-by: Shuah Khan shuah@kernel.org --- include/uapi/linux/media.h | 25 +++++++++++++++---------- 1 file changed, 15 insertions(+), 10 deletions(-)
diff --git a/include/uapi/linux/media.h b/include/uapi/linux/media.h index e5d0c5c611b5..9aedb187bc48 100644 --- a/include/uapi/linux/media.h +++ b/include/uapi/linux/media.h @@ -262,6 +262,11 @@ struct media_links_enum { #define MEDIA_INTF_T_V4L_SWRADIO (MEDIA_INTF_T_V4L_BASE + 4) #define MEDIA_INTF_T_V4L_TOUCH (MEDIA_INTF_T_V4L_BASE + 5)
+#define MEDIA_INTF_T_ALSA_BASE 0x00000300 +#define MEDIA_INTF_T_ALSA_PCM_CAPTURE (MEDIA_INTF_T_ALSA_BASE) +#define MEDIA_INTF_T_ALSA_PCM_PLAYBACK (MEDIA_INTF_T_ALSA_BASE + 1) +#define MEDIA_INTF_T_ALSA_CONTROL (MEDIA_INTF_T_ALSA_BASE + 2) + #if defined(__KERNEL__)
/* @@ -413,19 +418,19 @@ struct media_v2_topology { #define MEDIA_ENT_F_DTV_DECODER MEDIA_ENT_F_DV_DECODER
/* - * There is still no ALSA support in the media controller. These + * There is still no full ALSA support in the media controller. These * defines should not have been added and we leave them here only * in case some application tries to use these defines. + * + * The ALSA defines that are in use have been moved into __KERNEL__ + * scope. As support gets added to these interface types, they should + * be moved into __KERNEL__ scope with the code that uses them. */ -#define MEDIA_INTF_T_ALSA_BASE 0x00000300 -#define MEDIA_INTF_T_ALSA_PCM_CAPTURE (MEDIA_INTF_T_ALSA_BASE) -#define MEDIA_INTF_T_ALSA_PCM_PLAYBACK (MEDIA_INTF_T_ALSA_BASE + 1) -#define MEDIA_INTF_T_ALSA_CONTROL (MEDIA_INTF_T_ALSA_BASE + 2) -#define MEDIA_INTF_T_ALSA_COMPRESS (MEDIA_INTF_T_ALSA_BASE + 3) -#define MEDIA_INTF_T_ALSA_RAWMIDI (MEDIA_INTF_T_ALSA_BASE + 4) -#define MEDIA_INTF_T_ALSA_HWDEP (MEDIA_INTF_T_ALSA_BASE + 5) -#define MEDIA_INTF_T_ALSA_SEQUENCER (MEDIA_INTF_T_ALSA_BASE + 6) -#define MEDIA_INTF_T_ALSA_TIMER (MEDIA_INTF_T_ALSA_BASE + 7) +#define MEDIA_INTF_T_ALSA_COMPRESS (MEDIA_INTF_T_ALSA_BASE + 3) +#define MEDIA_INTF_T_ALSA_RAWMIDI (MEDIA_INTF_T_ALSA_BASE + 4) +#define MEDIA_INTF_T_ALSA_HWDEP (MEDIA_INTF_T_ALSA_BASE + 5) +#define MEDIA_INTF_T_ALSA_SEQUENCER (MEDIA_INTF_T_ALSA_BASE + 6) +#define MEDIA_INTF_T_ALSA_TIMER (MEDIA_INTF_T_ALSA_BASE + 7)
/* Obsolete symbol for media_version, no longer used in the kernel */ #define MEDIA_API_VERSION ((0 << 16) | (1 << 8) | 0)
Media Device Allocator API to allows multiple drivers share a media device. This API solves a very common use-case for media devices where one physical device (an USB stick) provides both audio and video. When such media device exposes a standard USB Audio class, a proprietary Video class, two or more independent drivers will share a single physical USB bridge. In such cases, it is necessary to coordinate access to the shared resource.
Using this API, drivers can allocate a media device with the shared struct device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.
Change the ALSA driver to use the Media Controller API to share media resources with DVB, and V4L2 drivers on a AU0828 media device.
The Media Controller specific initialization is done after sound card is registered. ALSA creates Media interface and entity function graph nodes for Control, Mixer, PCM Playback, and PCM Capture devices.
snd_usb_hw_params() will call Media Controller enable source handler interface to request the media resource. If resource request is granted, it will release it from snd_usb_hw_free(). If resource is busy, -EBUSY is returned.
Media specific cleanup is done in usb_audio_disconnect().
Reviewed-by: Takashi Iwai tiwai@suse.de Signed-off-by: Shuah Khan shuah@kernel.org --- sound/usb/Kconfig | 4 + sound/usb/Makefile | 2 + sound/usb/card.c | 14 ++ sound/usb/card.h | 3 + sound/usb/media.c | 327 +++++++++++++++++++++++++++++++++++++++ sound/usb/media.h | 74 +++++++++ sound/usb/mixer.h | 3 + sound/usb/pcm.c | 29 +++- sound/usb/quirks-table.h | 1 + sound/usb/stream.c | 2 + sound/usb/usbaudio.h | 6 + 11 files changed, 461 insertions(+), 4 deletions(-) create mode 100644 sound/usb/media.c create mode 100644 sound/usb/media.h
diff --git a/sound/usb/Kconfig b/sound/usb/Kconfig index f61b5662bb89..6319b544ba3a 100644 --- a/sound/usb/Kconfig +++ b/sound/usb/Kconfig @@ -15,6 +15,7 @@ config SND_USB_AUDIO select SND_RAWMIDI select SND_PCM select BITREVERSE + select SND_USB_AUDIO_USE_MEDIA_CONTROLLER if MEDIA_CONTROLLER && (MEDIA_SUPPORT=y || MEDIA_SUPPORT=SND_USB_AUDIO) help Say Y here to include support for USB audio and USB MIDI devices. @@ -22,6 +23,9 @@ config SND_USB_AUDIO To compile this driver as a module, choose M here: the module will be called snd-usb-audio.
+config SND_USB_AUDIO_USE_MEDIA_CONTROLLER + bool + config SND_USB_UA101 tristate "Edirol UA-101/UA-1000 driver" select SND_PCM diff --git a/sound/usb/Makefile b/sound/usb/Makefile index d330f74c90e6..e1ce257ab705 100644 --- a/sound/usb/Makefile +++ b/sound/usb/Makefile @@ -18,6 +18,8 @@ snd-usb-audio-objs := card.o \ quirks.o \ stream.o
+snd-usb-audio-$(CONFIG_SND_USB_AUDIO_USE_MEDIA_CONTROLLER) += media.o + snd-usbmidi-lib-objs := midi.o
# Toplevel Module Dependency diff --git a/sound/usb/card.c b/sound/usb/card.c index 746a72e23cf9..cc6b23e282ec 100644 --- a/sound/usb/card.c +++ b/sound/usb/card.c @@ -68,6 +68,7 @@ #include "format.h" #include "power.h" #include "stream.h" +#include "media.h"
MODULE_AUTHOR("Takashi Iwai tiwai@suse.de"); MODULE_DESCRIPTION("USB Audio"); @@ -673,6 +674,11 @@ static int usb_audio_probe(struct usb_interface *intf, if (err < 0) goto __error;
+ if (quirk && quirk->shares_media_device) { + /* don't want to fail when snd_media_device_create() fails */ + snd_media_device_create(chip, intf); + } + usb_chip[chip->index] = chip; chip->num_interfaces++; usb_set_intfdata(intf, chip); @@ -732,6 +738,14 @@ static void usb_audio_disconnect(struct usb_interface *intf) list_for_each(p, &chip->midi_list) { snd_usbmidi_disconnect(p); } + /* + * Nice to check quirk && quirk->shares_media_device and + * then call the snd_media_device_delete(). Don't have + * access to the quirk here. snd_media_device_delete() + * accesses mixer_list + */ + snd_media_device_delete(chip); + /* release mixer resources */ list_for_each_entry(mixer, &chip->mixer_list, list) { snd_usb_mixer_disconnect(mixer); diff --git a/sound/usb/card.h b/sound/usb/card.h index ac785d15ced4..5dd3538ed6b5 100644 --- a/sound/usb/card.h +++ b/sound/usb/card.h @@ -108,6 +108,8 @@ struct snd_usb_endpoint { struct list_head list; };
+struct media_ctl; + struct snd_usb_substream { struct snd_usb_stream *stream; struct usb_device *dev; @@ -160,6 +162,7 @@ struct snd_usb_substream { } dsd_dop;
bool trigger_tstamp_pending_update; /* trigger timestamp being updated from initial estimate */ + struct media_ctl *media_ctl; };
struct snd_usb_stream { diff --git a/sound/usb/media.c b/sound/usb/media.c new file mode 100644 index 000000000000..c657ed45a703 --- /dev/null +++ b/sound/usb/media.c @@ -0,0 +1,327 @@ +// SPDX-License-Identifier: GPL-2.0-or-later +/* + * media.c - Media Controller specific ALSA driver code + * + * Copyright (c) 2018 Shuah Khan shuah@kernel.org + * + */ + +/* + * This file adds Media Controller support to the ALSA driver + * to use the Media Controller API to share the tuner with DVB + * and V4L2 drivers that control the media device. + * + * The media device is created based on the existing quirks framework. + * Using this approach, the media controller API usage can be added for + * a specific device. + */ + +#include <linux/init.h> +#include <linux/list.h> +#include <linux/mutex.h> +#include <linux/slab.h> +#include <linux/usb.h> + +#include <sound/pcm.h> +#include <sound/core.h> + +#include "usbaudio.h" +#include "card.h" +#include "mixer.h" +#include "media.h" + +int snd_media_stream_init(struct snd_usb_substream *subs, struct snd_pcm *pcm, + int stream) +{ + struct media_device *mdev; + struct media_ctl *mctl; + struct device *pcm_dev = &pcm->streams[stream].dev; + u32 intf_type; + int ret = 0; + u16 mixer_pad; + struct media_entity *entity; + + mdev = subs->stream->chip->media_dev; + if (!mdev) + return 0; + + if (subs->media_ctl) + return 0; + + /* allocate media_ctl */ + mctl = kzalloc(sizeof(*mctl), GFP_KERNEL); + if (!mctl) + return -ENOMEM; + + mctl->media_dev = mdev; + if (stream == SNDRV_PCM_STREAM_PLAYBACK) { + intf_type = MEDIA_INTF_T_ALSA_PCM_PLAYBACK; + mctl->media_entity.function = MEDIA_ENT_F_AUDIO_PLAYBACK; + mctl->media_pad.flags = MEDIA_PAD_FL_SOURCE; + mixer_pad = 1; + } else { + intf_type = MEDIA_INTF_T_ALSA_PCM_CAPTURE; + mctl->media_entity.function = MEDIA_ENT_F_AUDIO_CAPTURE; + mctl->media_pad.flags = MEDIA_PAD_FL_SINK; + mixer_pad = 2; + } + mctl->media_entity.name = pcm->name; + media_entity_pads_init(&mctl->media_entity, 1, &mctl->media_pad); + ret = media_device_register_entity(mctl->media_dev, + &mctl->media_entity); + if (ret) + goto free_mctl; + + mctl->intf_devnode = media_devnode_create(mdev, intf_type, 0, + MAJOR(pcm_dev->devt), + MINOR(pcm_dev->devt)); + if (!mctl->intf_devnode) { + ret = -ENOMEM; + goto unregister_entity; + } + mctl->intf_link = media_create_intf_link(&mctl->media_entity, + &mctl->intf_devnode->intf, + MEDIA_LNK_FL_ENABLED); + if (!mctl->intf_link) { + ret = -ENOMEM; + goto devnode_remove; + } + + /* create link between mixer and audio */ + media_device_for_each_entity(entity, mdev) { + switch (entity->function) { + case MEDIA_ENT_F_AUDIO_MIXER: + ret = media_create_pad_link(entity, mixer_pad, + &mctl->media_entity, 0, + MEDIA_LNK_FL_ENABLED); + if (ret) + goto remove_intf_link; + break; + } + } + + subs->media_ctl = mctl; + return 0; + +remove_intf_link: + media_remove_intf_link(mctl->intf_link); +devnode_remove: + media_devnode_remove(mctl->intf_devnode); +unregister_entity: + media_device_unregister_entity(&mctl->media_entity); +free_mctl: + kfree(mctl); + return ret; +} + +void snd_media_stream_delete(struct snd_usb_substream *subs) +{ + struct media_ctl *mctl = subs->media_ctl; + + if (mctl) { + struct media_device *mdev; + + mdev = mctl->media_dev; + if (mdev && media_devnode_is_registered(mdev->devnode)) { + media_devnode_remove(mctl->intf_devnode); + media_device_unregister_entity(&mctl->media_entity); + media_entity_cleanup(&mctl->media_entity); + } + kfree(mctl); + subs->media_ctl = NULL; + } +} + +int snd_media_start_pipeline(struct snd_usb_substream *subs) +{ + struct media_ctl *mctl = subs->media_ctl; + int ret = 0; + + if (!mctl) + return 0; + + mutex_lock(&mctl->media_dev->graph_mutex); + if (mctl->media_dev->enable_source) + ret = mctl->media_dev->enable_source(&mctl->media_entity, + &mctl->media_pipe); + mutex_unlock(&mctl->media_dev->graph_mutex); + return ret; +} + +void snd_media_stop_pipeline(struct snd_usb_substream *subs) +{ + struct media_ctl *mctl = subs->media_ctl; + + if (!mctl) + return; + + mutex_lock(&mctl->media_dev->graph_mutex); + if (mctl->media_dev->disable_source) + mctl->media_dev->disable_source(&mctl->media_entity); + mutex_unlock(&mctl->media_dev->graph_mutex); +} + +static int snd_media_mixer_init(struct snd_usb_audio *chip) +{ + struct device *ctl_dev = &chip->card->ctl_dev; + struct media_intf_devnode *ctl_intf; + struct usb_mixer_interface *mixer; + struct media_device *mdev = chip->media_dev; + struct media_mixer_ctl *mctl; + u32 intf_type = MEDIA_INTF_T_ALSA_CONTROL; + int ret; + + if (!mdev) + return -ENODEV; + + ctl_intf = chip->ctl_intf_media_devnode; + if (!ctl_intf) { + ctl_intf = media_devnode_create(mdev, intf_type, 0, + MAJOR(ctl_dev->devt), + MINOR(ctl_dev->devt)); + if (!ctl_intf) + return -ENOMEM; + chip->ctl_intf_media_devnode = ctl_intf; + } + + list_for_each_entry(mixer, &chip->mixer_list, list) { + + if (mixer->media_mixer_ctl) + continue; + + /* allocate media_mixer_ctl */ + mctl = kzalloc(sizeof(*mctl), GFP_KERNEL); + if (!mctl) + return -ENOMEM; + + mctl->media_dev = mdev; + mctl->media_entity.function = MEDIA_ENT_F_AUDIO_MIXER; + mctl->media_entity.name = chip->card->mixername; + mctl->media_pad[0].flags = MEDIA_PAD_FL_SINK; + mctl->media_pad[1].flags = MEDIA_PAD_FL_SOURCE; + mctl->media_pad[2].flags = MEDIA_PAD_FL_SOURCE; + media_entity_pads_init(&mctl->media_entity, MEDIA_MIXER_PAD_MAX, + mctl->media_pad); + ret = media_device_register_entity(mctl->media_dev, + &mctl->media_entity); + if (ret) { + kfree(mctl); + return ret; + } + + mctl->intf_link = media_create_intf_link(&mctl->media_entity, + &ctl_intf->intf, + MEDIA_LNK_FL_ENABLED); + if (!mctl->intf_link) { + media_device_unregister_entity(&mctl->media_entity); + media_entity_cleanup(&mctl->media_entity); + kfree(mctl); + return -ENOMEM; + } + mctl->intf_devnode = ctl_intf; + mixer->media_mixer_ctl = mctl; + } + return 0; +} + +static void snd_media_mixer_delete(struct snd_usb_audio *chip) +{ + struct usb_mixer_interface *mixer; + struct media_device *mdev = chip->media_dev; + + if (!mdev) + return; + + list_for_each_entry(mixer, &chip->mixer_list, list) { + struct media_mixer_ctl *mctl; + + mctl = mixer->media_mixer_ctl; + if (!mixer->media_mixer_ctl) + continue; + + if (media_devnode_is_registered(mdev->devnode)) { + media_device_unregister_entity(&mctl->media_entity); + media_entity_cleanup(&mctl->media_entity); + } + kfree(mctl); + mixer->media_mixer_ctl = NULL; + } + if (media_devnode_is_registered(mdev->devnode)) + media_devnode_remove(chip->ctl_intf_media_devnode); + chip->ctl_intf_media_devnode = NULL; +} + +int snd_media_device_create(struct snd_usb_audio *chip, + struct usb_interface *iface) +{ + struct media_device *mdev; + struct usb_device *usbdev = interface_to_usbdev(iface); + int ret = 0; + + /* usb-audio driver is probed for each usb interface, and + * there are multiple interfaces per device. Avoid calling + * media_device_usb_allocate() each time usb_audio_probe() + * is called. Do it only once. + */ + if (chip->media_dev) { + mdev = chip->media_dev; + goto snd_mixer_init; + } + + mdev = media_device_usb_allocate(usbdev, KBUILD_MODNAME); + if (!mdev) + return -ENOMEM; + + /* save media device - avoid lookups */ + chip->media_dev = mdev; + +snd_mixer_init: + /* Create media entities for mixer and control dev */ + ret = snd_media_mixer_init(chip); + /* media_device might be registered, print error and continue */ + if (ret) + dev_err(&usbdev->dev, + "Couldn't create media mixer entities. Error: %d\n", + ret); + + if (!media_devnode_is_registered(mdev->devnode)) { + /* dont'register if snd_media_mixer_init() failed */ + if (ret) + goto create_fail; + + /* register media_device */ + ret = media_device_register(mdev); +create_fail: + if (ret) { + snd_media_mixer_delete(chip); + media_device_delete(mdev, KBUILD_MODNAME); + /* clear saved media_dev */ + chip->media_dev = NULL; + dev_err(&usbdev->dev, + "Couldn't register media device. Error: %d\n", + ret); + return ret; + } + } + + return ret; +} + +void snd_media_device_delete(struct snd_usb_audio *chip) +{ + struct media_device *mdev = chip->media_dev; + struct snd_usb_stream *stream; + + /* release resources */ + list_for_each_entry(stream, &chip->pcm_list, list) { + snd_media_stream_delete(&stream->substream[0]); + snd_media_stream_delete(&stream->substream[1]); + } + + snd_media_mixer_delete(chip); + + if (mdev) { + media_device_delete(mdev, KBUILD_MODNAME); + chip->media_dev = NULL; + } +} diff --git a/sound/usb/media.h b/sound/usb/media.h new file mode 100644 index 000000000000..0e8a4785f858 --- /dev/null +++ b/sound/usb/media.h @@ -0,0 +1,74 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* + * media.h - Media Controller specific ALSA driver code + * + * Copyright (c) 2018 Shuah Khan shuah@kernel.org + * + */ + +/* + * This file adds Media Controller support to the ALSA driver + * to use the Media Controller API to share the tuner with DVB + * and V4L2 drivers that control the media device. + * + * The media device is created based on the existing quirks framework. + * Using this approach, the media controller API usage can be added for + * a specific device. + */ +#ifndef __MEDIA_H + +#ifdef CONFIG_SND_USB_AUDIO_USE_MEDIA_CONTROLLER + +#include <linux/media.h> +#include <media/media-device.h> +#include <media/media-entity.h> +#include <media/media-dev-allocator.h> +#include <sound/asound.h> + +struct media_ctl { + struct media_device *media_dev; + struct media_entity media_entity; + struct media_intf_devnode *intf_devnode; + struct media_link *intf_link; + struct media_pad media_pad; + struct media_pipeline media_pipe; +}; + +/* + * One source pad each for SNDRV_PCM_STREAM_CAPTURE and + * SNDRV_PCM_STREAM_PLAYBACK. One for sink pad to link + * to AUDIO Source + */ +#define MEDIA_MIXER_PAD_MAX (SNDRV_PCM_STREAM_LAST + 2) + +struct media_mixer_ctl { + struct media_device *media_dev; + struct media_entity media_entity; + struct media_intf_devnode *intf_devnode; + struct media_link *intf_link; + struct media_pad media_pad[MEDIA_MIXER_PAD_MAX]; + struct media_pipeline media_pipe; +}; + +int snd_media_device_create(struct snd_usb_audio *chip, + struct usb_interface *iface); +void snd_media_device_delete(struct snd_usb_audio *chip); +int snd_media_stream_init(struct snd_usb_substream *subs, struct snd_pcm *pcm, + int stream); +void snd_media_stream_delete(struct snd_usb_substream *subs); +int snd_media_start_pipeline(struct snd_usb_substream *subs); +void snd_media_stop_pipeline(struct snd_usb_substream *subs); +#else +static inline int snd_media_device_create(struct snd_usb_audio *chip, + struct usb_interface *iface) + { return 0; } +static inline void snd_media_device_delete(struct snd_usb_audio *chip) { } +static inline int snd_media_stream_init(struct snd_usb_substream *subs, + struct snd_pcm *pcm, int stream) + { return 0; } +static inline void snd_media_stream_delete(struct snd_usb_substream *subs) { } +static inline int snd_media_start_pipeline(struct snd_usb_substream *subs) + { return 0; } +static inline void snd_media_stop_pipeline(struct snd_usb_substream *subs) { } +#endif +#endif /* __MEDIA_H */ diff --git a/sound/usb/mixer.h b/sound/usb/mixer.h index 3d12af8bf191..394cd9107507 100644 --- a/sound/usb/mixer.h +++ b/sound/usb/mixer.h @@ -4,6 +4,8 @@
#include <sound/info.h>
+struct media_mixer_ctl; + struct usb_mixer_interface { struct snd_usb_audio *chip; struct usb_host_interface *hostif; @@ -23,6 +25,7 @@ struct usb_mixer_interface { struct urb *rc_urb; struct usb_ctrlrequest *rc_setup_packet; u8 rc_buffer[6]; + struct media_mixer_ctl *media_mixer_ctl;
bool disconnected; }; diff --git a/sound/usb/pcm.c b/sound/usb/pcm.c index 382847154227..538ed10ab66d 100644 --- a/sound/usb/pcm.c +++ b/sound/usb/pcm.c @@ -35,6 +35,7 @@ #include "pcm.h" #include "clock.h" #include "power.h" +#include "media.h"
#define SUBSTREAM_FLAG_DATA_EP_STARTED 0 #define SUBSTREAM_FLAG_SYNC_EP_STARTED 1 @@ -776,6 +777,10 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream, struct audioformat *fmt; int ret;
+ ret = snd_media_start_pipeline(subs); + if (ret) + return ret; + if (snd_usb_use_vmalloc) ret = snd_pcm_lib_alloc_vmalloc_buffer(substream, params_buffer_bytes(hw_params)); @@ -783,7 +788,7 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream, ret = snd_pcm_lib_malloc_pages(substream, params_buffer_bytes(hw_params)); if (ret < 0) - return ret; + goto stop_pipeline;
subs->pcm_format = params_format(hw_params); subs->period_bytes = params_period_bytes(hw_params); @@ -797,12 +802,13 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream, dev_dbg(&subs->dev->dev, "cannot set format: format = %#x, rate = %d, channels = %d\n", subs->pcm_format, subs->cur_rate, subs->channels); - return -EINVAL; + ret = -EINVAL; + goto stop_pipeline; }
ret = snd_usb_lock_shutdown(subs->stream->chip); if (ret < 0) - return ret; + goto stop_pipeline;
ret = snd_usb_pcm_change_state(subs, UAC3_PD_STATE_D0); if (ret < 0) @@ -818,6 +824,12 @@ static int snd_usb_hw_params(struct snd_pcm_substream *substream,
unlock: snd_usb_unlock_shutdown(subs->stream->chip); + if (ret < 0) + goto stop_pipeline; + return ret; + + stop_pipeline: + snd_media_stop_pipeline(subs); return ret; }
@@ -830,6 +842,7 @@ static int snd_usb_hw_free(struct snd_pcm_substream *substream) { struct snd_usb_substream *subs = substream->runtime->private_data;
+ snd_media_stop_pipeline(subs); subs->cur_audiofmt = NULL; subs->cur_rate = 0; subs->period_bytes = 0; @@ -1302,6 +1315,7 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream) struct snd_usb_stream *as = snd_pcm_substream_chip(substream); struct snd_pcm_runtime *runtime = substream->runtime; struct snd_usb_substream *subs = &as->substream[direction]; + int ret;
subs->interface = -1; subs->altset_idx = 0; @@ -1315,7 +1329,13 @@ static int snd_usb_pcm_open(struct snd_pcm_substream *substream) subs->dsd_dop.channel = 0; subs->dsd_dop.marker = 1;
- return setup_hw_info(runtime, subs); + ret = setup_hw_info(runtime, subs); + if (ret == 0) { + ret = snd_media_stream_init(subs, as->pcm, direction); + if (ret) + snd_usb_autosuspend(subs->stream->chip); + } + return ret; }
static int snd_usb_pcm_close(struct snd_pcm_substream *substream) @@ -1326,6 +1346,7 @@ static int snd_usb_pcm_close(struct snd_pcm_substream *substream) int ret;
stop_endpoints(subs, true); + snd_media_stop_pipeline(subs);
if (!as->chip->keep_iface && subs->interface >= 0 && diff --git a/sound/usb/quirks-table.h b/sound/usb/quirks-table.h index b345beb447bd..6835665d311f 100644 --- a/sound/usb/quirks-table.h +++ b/sound/usb/quirks-table.h @@ -2887,6 +2887,7 @@ YAMAHA_DEVICE(0x7010, "UB99"), .product_name = pname, \ .ifnum = QUIRK_ANY_INTERFACE, \ .type = QUIRK_AUDIO_ALIGN_TRANSFER, \ + .shares_media_device = 1, \ } \ }
diff --git a/sound/usb/stream.c b/sound/usb/stream.c index d9e3de495c16..9f1623e37fb3 100644 --- a/sound/usb/stream.c +++ b/sound/usb/stream.c @@ -38,6 +38,7 @@ #include "clock.h" #include "stream.h" #include "power.h" +#include "media.h"
/* * free a substream @@ -55,6 +56,7 @@ static void free_substream(struct snd_usb_substream *subs) } kfree(subs->rate_list.list); kfree(subs->str_pd); + snd_media_stream_delete(subs); }
diff --git a/sound/usb/usbaudio.h b/sound/usb/usbaudio.h index b9faeca645fd..0968a45c8925 100644 --- a/sound/usb/usbaudio.h +++ b/sound/usb/usbaudio.h @@ -30,6 +30,9 @@ * */
+struct media_device; +struct media_intf_devnode; + struct snd_usb_audio { int index; struct usb_device *dev; @@ -66,6 +69,8 @@ struct snd_usb_audio { */
struct usb_host_interface *ctrl_intf; /* the audio control interface */ + struct media_device *media_dev; + struct media_intf_devnode *ctl_intf_media_devnode; };
#define usb_audio_err(chip, fmt, args...) \ @@ -117,6 +122,7 @@ struct snd_usb_audio_quirk { const char *profile_name; /* override the card->longname */ int16_t ifnum; uint16_t type; + bool shares_media_device; const void *data; };
Hi Shuah,
On Thu, Jan 24, 2019 at 01:32:37PM -0700, Shuah Khan wrote:
Media Device Allocator API to allows multiple drivers share a media device. This API solves a very common use-case for media devices where one physical device (an USB stick) provides both audio and video. When such media device exposes a standard USB Audio class, a proprietary Video class, two or more independent drivers will share a single physical USB bridge. In such cases, it is necessary to coordinate access to the shared resource.
Using this API, drivers can allocate a media device with the shared struct device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.
Thanks for the update. I have to apologise I haven't ended up reviewing the set for some time. After taking a look at the current version, I'm happy to see that a number of issues recognised during earlier review rounds have been addressed.
Would you happen to have a media graph (media-ctl --print-dot and media-ctl -p) from the device? That'd help understanding the device a bit better for those who are not familiar with it.
- This patch series is tested on 5.0-rc3 and addresses comments on v9 series from Hans Verkuil.
- v9 was tested on 4.20-rc6.
- Tested sharing resources with kaffeine, vlc, xawtv, tvtime, and arecord. When analog is streaming, digital and audio user-space applications detect that the tuner is busy and exit. When digital is streaming, analog and audio applications detect that the tuner is busy and exit. When arecord is owns the tuner, digital and analog detect that the tuner is busy and exit.
- Tested media device allocator API with bind/unbind testing on snd-usb-audio and au0828 drivers to make sure /dev/mediaX is released only when the last driver is unbound.
- Addressed review comments from Hans on the RFC v8 (rebased on 4.19)
- Updated change log to describe the use-case more clearly.
- No changes to 0001,0002 code since the v7 referenced below.
- 0003 is a new patch to enable ALSA defines that have been disabled for kernel between 4.9 and 4.19.
- Minor merge conflict resolution in 0004.
- Added SPDX to new files.
Changes since v9:
- Patch 1: Fix mutex assert warning from find_module() calls. This code was written before the change to find_module() that requires callers to hold module_mutex. I missed this during my testing on 4.20-rc6. Hans Verkuil reported the problem.
- Patch 4: sound/usb: Initializes all the entities it can before registering the device based on comments from Hans Verkuil
- Carried Reviewed-by tag from Takashi Iwai for the sound from v9.
- No changes to Patches 2 and 3.
References: https://lkml.org/lkml/2018/11/2/169 https://www.mail-archive.com/linux-media@vger.kernel.org/msg105854.html
Shuah Khan (4): media: Media Device Allocator API media: change au0828 to use Media Device Allocator API media: media.h: Enable ALSA MEDIA_INTF_T* interface types sound/usb: Use Media Controller API to share media resources
Documentation/media/kapi/mc-core.rst | 41 ++++ drivers/media/Makefile | 4 + drivers/media/media-dev-allocator.c | 144 +++++++++++ drivers/media/usb/au0828/au0828-core.c | 12 +- drivers/media/usb/au0828/au0828.h | 1 + include/media/media-dev-allocator.h | 53 ++++ include/uapi/linux/media.h | 25 +- sound/usb/Kconfig | 4 + sound/usb/Makefile | 2 + sound/usb/card.c | 14 ++ sound/usb/card.h | 3 + sound/usb/media.c | 327 +++++++++++++++++++++++++ sound/usb/media.h | 74 ++++++ sound/usb/mixer.h | 3 + sound/usb/pcm.c | 29 ++- sound/usb/quirks-table.h | 1 + sound/usb/stream.c | 2 + sound/usb/usbaudio.h | 6 + 18 files changed, 723 insertions(+), 22 deletions(-) create mode 100644 drivers/media/media-dev-allocator.c create mode 100644 include/media/media-dev-allocator.h create mode 100644 sound/usb/media.c create mode 100644 sound/usb/media.h
Hi Sakari,
On 1/25/19 8:28 AM, Sakari Ailus wrote:
Hi Shuah,
On Thu, Jan 24, 2019 at 01:32:37PM -0700, Shuah Khan wrote:
Media Device Allocator API to allows multiple drivers share a media device. This API solves a very common use-case for media devices where one physical device (an USB stick) provides both audio and video. When such media device exposes a standard USB Audio class, a proprietary Video class, two or more independent drivers will share a single physical USB bridge. In such cases, it is necessary to coordinate access to the shared resource.
Using this API, drivers can allocate a media device with the shared struct device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.
Thanks for the update. I have to apologise I haven't ended up reviewing the set for some time. After taking a look at the current version, I'm happy to see that a number of issues recognised during earlier review rounds have been addressed.
Would you happen to have a media graph (media-ctl --print-dot and media-ctl -p) from the device? That'd help understanding the device a bit better for those who are not familiar with it.
Please see the attached files for the below. It came in very handy for testing partial graphs as I did bind and unbind of the drivers in this mix.
media-ctl --print-dot and media-ctl --print-topology
thanks, -- Shuah
Hi Shuah,
On 1/24/19 9:32 PM, Shuah Khan wrote:
Media Device Allocator API to allows multiple drivers share a media device. This API solves a very common use-case for media devices where one physical device (an USB stick) provides both audio and video. When such media device exposes a standard USB Audio class, a proprietary Video class, two or more independent drivers will share a single physical USB bridge. In such cases, it is necessary to coordinate access to the shared resource.
Using this API, drivers can allocate a media device with the shared struct device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.
- This patch series is tested on 5.0-rc3 and addresses comments on v9 series from Hans Verkuil.
- v9 was tested on 4.20-rc6.
- Tested sharing resources with kaffeine, vlc, xawtv, tvtime, and arecord. When analog is streaming, digital and audio user-space applications detect that the tuner is busy and exit. When digital is streaming, analog and audio applications detect that the tuner is busy and exit. When arecord is owns the tuner, digital and analog detect that the tuner is busy and exit.
I've been doing some testing with my au0828, and I am confused about one thing, probably because it has been too long ago since I last looked into this in detail:
Why can't I change the tuner frequency if arecord (and only arecord) is streaming audio? If arecord is streaming, then it is recording the audio from the analog TV tuner, right? So changing the analog TV frequency should be fine.
I understand why you can't switch to DVB mode, but analog TV should be OK. Or am I missing something?
Regards,
Hans
- Tested media device allocator API with bind/unbind testing on snd-usb-audio and au0828 drivers to make sure /dev/mediaX is released only when the last driver is unbound.
- Addressed review comments from Hans on the RFC v8 (rebased on 4.19)
- Updated change log to describe the use-case more clearly.
- No changes to 0001,0002 code since the v7 referenced below.
- 0003 is a new patch to enable ALSA defines that have been disabled for kernel between 4.9 and 4.19.
- Minor merge conflict resolution in 0004.
- Added SPDX to new files.
Changes since v9:
- Patch 1: Fix mutex assert warning from find_module() calls. This code was written before the change to find_module() that requires callers to hold module_mutex. I missed this during my testing on 4.20-rc6. Hans Verkuil reported the problem.
- Patch 4: sound/usb: Initializes all the entities it can before registering the device based on comments from Hans Verkuil
- Carried Reviewed-by tag from Takashi Iwai for the sound from v9.
- No changes to Patches 2 and 3.
References: https://lkml.org/lkml/2018/11/2/169 https://www.mail-archive.com/linux-media@vger.kernel.org/msg105854.html
Shuah Khan (4): media: Media Device Allocator API media: change au0828 to use Media Device Allocator API media: media.h: Enable ALSA MEDIA_INTF_T* interface types sound/usb: Use Media Controller API to share media resources
Documentation/media/kapi/mc-core.rst | 41 ++++ drivers/media/Makefile | 4 + drivers/media/media-dev-allocator.c | 144 +++++++++++ drivers/media/usb/au0828/au0828-core.c | 12 +- drivers/media/usb/au0828/au0828.h | 1 + include/media/media-dev-allocator.h | 53 ++++ include/uapi/linux/media.h | 25 +- sound/usb/Kconfig | 4 + sound/usb/Makefile | 2 + sound/usb/card.c | 14 ++ sound/usb/card.h | 3 + sound/usb/media.c | 327 +++++++++++++++++++++++++ sound/usb/media.h | 74 ++++++ sound/usb/mixer.h | 3 + sound/usb/pcm.c | 29 ++- sound/usb/quirks-table.h | 1 + sound/usb/stream.c | 2 + sound/usb/usbaudio.h | 6 + 18 files changed, 723 insertions(+), 22 deletions(-) create mode 100644 drivers/media/media-dev-allocator.c create mode 100644 include/media/media-dev-allocator.h create mode 100644 sound/usb/media.c create mode 100644 sound/usb/media.h
Hi Hans,
On 1/28/19 5:03 AM, Hans Verkuil wrote:
Hi Shuah,
On 1/24/19 9:32 PM, Shuah Khan wrote:
Media Device Allocator API to allows multiple drivers share a media device. This API solves a very common use-case for media devices where one physical device (an USB stick) provides both audio and video. When such media device exposes a standard USB Audio class, a proprietary Video class, two or more independent drivers will share a single physical USB bridge. In such cases, it is necessary to coordinate access to the shared resource.
Using this API, drivers can allocate a media device with the shared struct device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.
- This patch series is tested on 5.0-rc3 and addresses comments on v9 series from Hans Verkuil.
- v9 was tested on 4.20-rc6.
- Tested sharing resources with kaffeine, vlc, xawtv, tvtime, and arecord. When analog is streaming, digital and audio user-space applications detect that the tuner is busy and exit. When digital is streaming, analog and audio applications detect that the tuner is busy and exit. When arecord is owns the tuner, digital and analog detect that the tuner is busy and exit.
I've been doing some testing with my au0828, and I am confused about one thing, probably because it has been too long ago since I last looked into this in detail:
Great.
Why can't I change the tuner frequency if arecord (and only arecord) is streaming audio? If arecord is streaming, then it is recording the audio from the analog TV tuner, right? So changing the analog TV frequency should be fine.
Changing analog TV frequency would be s_frequency. The way it works is any s_* calls would require holding the pipeline. In Analog TV case, it would mean holding both audio and video pipelines for any changes including TV.
As I recall, we discussed this design and the decision was to make all s_* calls interfaces to hold the tuner. A special exception is g_tuner in case of au0828. au0828 initializes the tuner from s_* interfaces and its g_tuner interfaces. Allowing s_frequency to proceed will disrupt the arecord audio stream.
Query (q_*) works just fine without holding the pipeline. I limited the analog holds to just the ones that are required. The current set is required to avoid audio stream disruptions.
I made sure v4l-ctl --all works when the pipeline is locked by any one of the 3 (audio, video, DVB).
Hope this helps.
thanks, -- Shuah
On 1/29/19 12:48 AM, shuah wrote:
Hi Hans,
On 1/28/19 5:03 AM, Hans Verkuil wrote:
Hi Shuah,
On 1/24/19 9:32 PM, Shuah Khan wrote:
Media Device Allocator API to allows multiple drivers share a media device. This API solves a very common use-case for media devices where one physical device (an USB stick) provides both audio and video. When such media device exposes a standard USB Audio class, a proprietary Video class, two or more independent drivers will share a single physical USB bridge. In such cases, it is necessary to coordinate access to the shared resource.
Using this API, drivers can allocate a media device with the shared struct device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.
- This patch series is tested on 5.0-rc3 and addresses comments on v9 series from Hans Verkuil.
- v9 was tested on 4.20-rc6.
- Tested sharing resources with kaffeine, vlc, xawtv, tvtime, and arecord. When analog is streaming, digital and audio user-space applications detect that the tuner is busy and exit. When digital is streaming, analog and audio applications detect that the tuner is busy and exit. When arecord is owns the tuner, digital and analog detect that the tuner is busy and exit.
I've been doing some testing with my au0828, and I am confused about one thing, probably because it has been too long ago since I last looked into this in detail:
Great.
Why can't I change the tuner frequency if arecord (and only arecord) is streaming audio? If arecord is streaming, then it is recording the audio from the analog TV tuner, right? So changing the analog TV frequency should be fine.
Changing analog TV frequency would be s_frequency. The way it works is any s_* calls would require holding the pipeline. In Analog TV case, it would mean holding both audio and video pipelines for any changes including TV.
As I recall, we discussed this design and the decision was to make all s_* calls interfaces to hold the tuner. A special exception is g_tuner in case of au0828. au0828 initializes the tuner from s_* interfaces and its g_tuner interfaces. Allowing s_frequency to proceed will disrupt the arecord audio stream.
Query (q_*) works just fine without holding the pipeline. I limited the analog holds to just the ones that are required. The current set is required to avoid audio stream disruptions.
So I am not sure about that ('avoid audio stream disruptions'): if I stream video AND use arecord, then I can just set the frequency while streaming. Doesn't that interrupt audio as well? And are you sure changing the tuner frequency actually disrupts audio? And if audio is disrupted, are we talking about a glitch or is audio permanently disrupted?
That's basically the inconsistent behavior I noticed: just running arecord will prevent me from changing the frequency, but if I run arecord and stream video, then it is suddenly OK to change the frequency.
BTW, I think there was also inconsistent behavior in the order of streaming audio and video: if I stream video first, then I can stream audio afterwards. But if I stream audio first, then (if I remember correctly) I can't start video streaming.
Regards,
Hans
I made sure v4l-ctl --all works when the pipeline is locked by any one of the 3 (audio, video, DVB).
Hope this helps.
thanks, -- Shuah
On 1/29/19 2:43 AM, Hans Verkuil wrote:
On 1/29/19 12:48 AM, shuah wrote:
Hi Hans,
On 1/28/19 5:03 AM, Hans Verkuil wrote:
Hi Shuah,
On 1/24/19 9:32 PM, Shuah Khan wrote:
Media Device Allocator API to allows multiple drivers share a media device. This API solves a very common use-case for media devices where one physical device (an USB stick) provides both audio and video. When such media device exposes a standard USB Audio class, a proprietary Video class, two or more independent drivers will share a single physical USB bridge. In such cases, it is necessary to coordinate access to the shared resource.
Using this API, drivers can allocate a media device with the shared struct device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.
- This patch series is tested on 5.0-rc3 and addresses comments on v9 series from Hans Verkuil.
- v9 was tested on 4.20-rc6.
- Tested sharing resources with kaffeine, vlc, xawtv, tvtime, and arecord. When analog is streaming, digital and audio user-space applications detect that the tuner is busy and exit. When digital is streaming, analog and audio applications detect that the tuner is busy and exit. When arecord is owns the tuner, digital and analog detect that the tuner is busy and exit.
I've been doing some testing with my au0828, and I am confused about one thing, probably because it has been too long ago since I last looked into this in detail:
Great.
Why can't I change the tuner frequency if arecord (and only arecord) is streaming audio? If arecord is streaming, then it is recording the audio from the analog TV tuner, right? So changing the analog TV frequency should be fine.
Changing analog TV frequency would be s_frequency. The way it works is any s_* calls would require holding the pipeline. In Analog TV case, it would mean holding both audio and video pipelines for any changes including TV.
As I recall, we discussed this design and the decision was to make all s_* calls interfaces to hold the tuner. A special exception is g_tuner in case of au0828. au0828 initializes the tuner from s_* interfaces and its g_tuner interfaces. Allowing s_frequency to proceed will disrupt the arecord audio stream.
Query (q_*) works just fine without holding the pipeline. I limited the analog holds to just the ones that are required. The current set is required to avoid audio stream disruptions.
So I am not sure about that ('avoid audio stream disruptions'): if I stream video AND use arecord, then I can just set the frequency while streaming. Doesn't that interrupt audio as well? And are you sure changing the tuner frequency actually disrupts audio? And if audio is disrupted, are we talking about a glitch or is audio permanently disrupted?
I think it is a glitch. I will run some tests and let you know.
That's basically the inconsistent behavior I noticed: just running arecord will prevent me from changing the frequency, but if I run arecord and stream video, then it is suddenly OK to change the frequency.
How are you changing frequency? I want to duplicate what you are doing.
BTW, I think there was also inconsistent behavior in the order of streaming audio and video: if I stream video first, then I can stream audio afterwards. But if I stream audio first, then (if I remember correctly) I can't start video streaming.
I will run some tests tomorrow and see what I find. Which video apps are you running for these tests?
thanks, -- Shuah
On 1/30/19 2:50 AM, shuah wrote:
On 1/29/19 2:43 AM, Hans Verkuil wrote:
On 1/29/19 12:48 AM, shuah wrote:
Hi Hans,
On 1/28/19 5:03 AM, Hans Verkuil wrote:
Hi Shuah,
On 1/24/19 9:32 PM, Shuah Khan wrote:
Media Device Allocator API to allows multiple drivers share a media device. This API solves a very common use-case for media devices where one physical device (an USB stick) provides both audio and video. When such media device exposes a standard USB Audio class, a proprietary Video class, two or more independent drivers will share a single physical USB bridge. In such cases, it is necessary to coordinate access to the shared resource.
Using this API, drivers can allocate a media device with the shared struct device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.
- This patch series is tested on 5.0-rc3 and addresses comments on v9 series from Hans Verkuil.
- v9 was tested on 4.20-rc6.
- Tested sharing resources with kaffeine, vlc, xawtv, tvtime, and arecord. When analog is streaming, digital and audio user-space applications detect that the tuner is busy and exit. When digital is streaming, analog and audio applications detect that the tuner is busy and exit. When arecord is owns the tuner, digital and analog detect that the tuner is busy and exit.
I've been doing some testing with my au0828, and I am confused about one thing, probably because it has been too long ago since I last looked into this in detail:
Great.
Why can't I change the tuner frequency if arecord (and only arecord) is streaming audio? If arecord is streaming, then it is recording the audio from the analog TV tuner, right? So changing the analog TV frequency should be fine.
Changing analog TV frequency would be s_frequency. The way it works is any s_* calls would require holding the pipeline. In Analog TV case, it would mean holding both audio and video pipelines for any changes including TV.
As I recall, we discussed this design and the decision was to make all s_* calls interfaces to hold the tuner. A special exception is g_tuner in case of au0828. au0828 initializes the tuner from s_* interfaces and its g_tuner interfaces. Allowing s_frequency to proceed will disrupt the arecord audio stream.
Query (q_*) works just fine without holding the pipeline. I limited the analog holds to just the ones that are required. The current set is required to avoid audio stream disruptions.
So I am not sure about that ('avoid audio stream disruptions'): if I stream video AND use arecord, then I can just set the frequency while streaming. Doesn't that interrupt audio as well? And are you sure changing the tuner frequency actually disrupts audio? And if audio is disrupted, are we talking about a glitch or is audio permanently disrupted?
I think it is a glitch. I will run some tests and let you know.
That's basically the inconsistent behavior I noticed: just running arecord will prevent me from changing the frequency, but if I run arecord and stream video, then it is suddenly OK to change the frequency.
How are you changing frequency? I want to duplicate what you are doing.
v4l2-ctl -f <freq>
BTW, I think there was also inconsistent behavior in the order of streaming audio and video: if I stream video first, then I can stream audio afterwards. But if I stream audio first, then (if I remember correctly) I can't start video streaming.
I will run some tests tomorrow and see what I find. Which video apps are you running for these tests?
v4l2-ctl or qv4l2.
Regards,
Hans
thanks, -- Shuah
Hi Hans,
On 1/30/19 12:42 AM, Hans Verkuil wrote:
On 1/30/19 2:50 AM, shuah wrote:
On 1/29/19 2:43 AM, Hans Verkuil wrote:
On 1/29/19 12:48 AM, shuah wrote:
Hi Hans,
On 1/28/19 5:03 AM, Hans Verkuil wrote:
Hi Shuah,
On 1/24/19 9:32 PM, Shuah Khan wrote:
Media Device Allocator API to allows multiple drivers share a media device. This API solves a very common use-case for media devices where one physical device (an USB stick) provides both audio and video. When such media device exposes a standard USB Audio class, a proprietary Video class, two or more independent drivers will share a single physical USB bridge. In such cases, it is necessary to coordinate access to the shared resource.
Using this API, drivers can allocate a media device with the shared struct device as the key. Once the media device is allocated by a driver, other drivers can get a reference to it. The media device is released when all the references are released.
- This patch series is tested on 5.0-rc3 and addresses comments on v9 series from Hans Verkuil.
- v9 was tested on 4.20-rc6.
- Tested sharing resources with kaffeine, vlc, xawtv, tvtime, and arecord. When analog is streaming, digital and audio user-space applications detect that the tuner is busy and exit. When digital is streaming, analog and audio applications detect that the tuner is busy and exit. When arecord is owns the tuner, digital and analog detect that the tuner is busy and exit.
I've been doing some testing with my au0828, and I am confused about one thing, probably because it has been too long ago since I last looked into this in detail:
Great.
Why can't I change the tuner frequency if arecord (and only arecord) is streaming audio? If arecord is streaming, then it is recording the audio from the analog TV tuner, right? So changing the analog TV frequency should be fine.
Changing analog TV frequency would be s_frequency. The way it works is any s_* calls would require holding the pipeline. In Analog TV case, it would mean holding both audio and video pipelines for any changes including TV.
As I recall, we discussed this design and the decision was to make all s_* calls interfaces to hold the tuner. A special exception is g_tuner in case of au0828. au0828 initializes the tuner from s_* interfaces and its g_tuner interfaces. Allowing s_frequency to proceed will disrupt the arecord audio stream.
Query (q_*) works just fine without holding the pipeline. I limited the analog holds to just the ones that are required. The current set is required to avoid audio stream disruptions.
So I am not sure about that ('avoid audio stream disruptions'): if I stream video AND use arecord, then I can just set the frequency while streaming. Doesn't that interrupt audio as well? And are you sure changing the tuner frequency actually disrupts audio? And if audio is disrupted, are we talking about a glitch or is audio permanently disrupted?
I think it is a glitch. I will run some tests and let you know.
That's basically the inconsistent behavior I noticed: just running arecord will prevent me from changing the frequency, but if I run arecord and stream video, then it is suddenly OK to change the frequency.
How are you changing frequency? I want to duplicate what you are doing.
v4l2-ctl -f <freq>
I am not seeing the inconsistent behavior. Here are my results.
1. Started acecord and while it is running:
arecord -M -D plughw:2,0 -c2 -f S16_LE -t wav foo.wav Recording WAVE 'foo.wav' : Signed 16 bit Little Endian, Rate 8000 Hz, Stereo
2. Ran v4l2-ctl -f as follows:
v4l2-ctl -f 700 VIDIOC_G_TUNER: failed: Device or resource busy VIDIOC_S_FREQUENCY: failed: Device or resource busy
Based on the current implementation, it failed with resource busy as expected.
3. Started v4l2-ctl as follows:
v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video0 VIDIOC_STREAMON: failed: Device or resource busy shuah@deneb:/mnt/data/lkml/v4l-utils/utils/v4l2-ctl$ v4l2-ctl -f 700 VIDIOC_G_TUNER: failed: Device or resource busy VIDIOC_S_FREQUENCY: failed: Device or resource busy
Based on the current implementation, it failed with resource busy as expected.
4. After stopping arecord:
v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video0 <<<<<<<<<<<<< 11.88 fps <<<<<<<<<<<<<<< 13.36 fps <<<<<<<<<<<<<<< 14.00 fps <<<<<<<<<<<<<<<< 14.35 fps <<<<<<<<<<<<<<< 14.57 fps <<<<<<<<<<<<<<<< 14.71 fps <<<<<<<<<
Worked as expected.
5. After stopping above video streaming:
arecord -M -D plughw:2,0 -c2 -f S16_LE -t wav foo.wav Recording WAVE 'foo.wav' : Signed 16 bit Little Endian, Rate 8000 Hz, Stereo
Worked as expected.
BTW, I think there was also inconsistent behavior in the order of streaming audio and video: if I stream video first, then I can stream audio afterwards. But if I stream audio first, then (if I remember correctly) I can't start video streaming.
Okay this is what I saw:
While v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video0
I could start arecord -M -D plughw:2,0 -c2 -f S16_LE -t wav foo.wav
I ran strace on v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video0 and I didn't see AUDIO holds. The only think of here is that
v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video0
doesn't open audio device. I didn't see this when tested with other video apps, (tvtine, xawtv, vlc). When video is streaming, I see arecord failing with device busy.
As far I can see, exclusion logic is working correctly. s_freq does fail now based on the current logic.
btw I tested all of this on 5.0-rc4
thanks, -- Shuah
On 2/1/19 1:46 AM, shuah wrote:
Hi Hans,
On 1/30/19 12:42 AM, Hans Verkuil wrote:
On 1/30/19 2:50 AM, shuah wrote:
On 1/29/19 2:43 AM, Hans Verkuil wrote:
On 1/29/19 12:48 AM, shuah wrote:
Hi Hans,
On 1/28/19 5:03 AM, Hans Verkuil wrote:
Hi Shuah,
On 1/24/19 9:32 PM, Shuah Khan wrote: > Media Device Allocator API to allows multiple drivers share a media device. > This API solves a very common use-case for media devices where one physical > device (an USB stick) provides both audio and video. When such media device > exposes a standard USB Audio class, a proprietary Video class, two or more > independent drivers will share a single physical USB bridge. In such cases, > it is necessary to coordinate access to the shared resource. > > Using this API, drivers can allocate a media device with the shared struct > device as the key. Once the media device is allocated by a driver, other > drivers can get a reference to it. The media device is released when all > the references are released. > > - This patch series is tested on 5.0-rc3 and addresses comments on > v9 series from Hans Verkuil. > - v9 was tested on 4.20-rc6. > - Tested sharing resources with kaffeine, vlc, xawtv, tvtime, and > arecord. When analog is streaming, digital and audio user-space > applications detect that the tuner is busy and exit. When digital > is streaming, analog and audio applications detect that the tuner is > busy and exit. When arecord is owns the tuner, digital and analog > detect that the tuner is busy and exit.
I've been doing some testing with my au0828, and I am confused about one thing, probably because it has been too long ago since I last looked into this in detail:
Great.
Why can't I change the tuner frequency if arecord (and only arecord) is streaming audio? If arecord is streaming, then it is recording the audio from the analog TV tuner, right? So changing the analog TV frequency should be fine.
Changing analog TV frequency would be s_frequency. The way it works is any s_* calls would require holding the pipeline. In Analog TV case, it would mean holding both audio and video pipelines for any changes including TV.
As I recall, we discussed this design and the decision was to make all s_* calls interfaces to hold the tuner. A special exception is g_tuner in case of au0828. au0828 initializes the tuner from s_* interfaces and its g_tuner interfaces. Allowing s_frequency to proceed will disrupt the arecord audio stream.
Query (q_*) works just fine without holding the pipeline. I limited the analog holds to just the ones that are required. The current set is required to avoid audio stream disruptions.
So I am not sure about that ('avoid audio stream disruptions'): if I stream video AND use arecord, then I can just set the frequency while streaming. Doesn't that interrupt audio as well? And are you sure changing the tuner frequency actually disrupts audio? And if audio is disrupted, are we talking about a glitch or is audio permanently disrupted?
I think it is a glitch. I will run some tests and let you know.
That's basically the inconsistent behavior I noticed: just running arecord will prevent me from changing the frequency, but if I run arecord and stream video, then it is suddenly OK to change the frequency.
How are you changing frequency? I want to duplicate what you are doing.
v4l2-ctl -f <freq>
I am not seeing the inconsistent behavior. Here are my results.
- Started acecord and while it is running:
arecord -M -D plughw:2,0 -c2 -f S16_LE -t wav foo.wav Recording WAVE 'foo.wav' : Signed 16 bit Little Endian, Rate 8000 Hz, Stereo
- Ran v4l2-ctl -f as follows:
v4l2-ctl -f 700 VIDIOC_G_TUNER: failed: Device or resource busy VIDIOC_S_FREQUENCY: failed: Device or resource busy
Based on the current implementation, it failed with resource busy as expected.
- Started v4l2-ctl as follows:
v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video0 VIDIOC_STREAMON: failed: Device or resource busy
Why is this? You have one analog tuner and it delivers independent audio and video streams. I should be able to start/stop audio and video independently.
And as mentioned above, if I use v4l2-ctl for video streaming, then start arecord, then that works fine. And I can change the tuner frequency while both are streaming.
But doing this the other way around (first start arecord, then v4l2-ctl) then that doesn't work.
It makes no sense.
Note that v4l2-ctl does not open audio, it solely deals with video. qv4l2 will open both audio and video, but for testing audio and video independently you need to use arecord/v4l2-ctl.
In any case, my understanding of how this should work is that both arecord and v4l2-ctl should attempt to lock the analog tuner resource, but they can share it.
If DVB is using the tuner, then both arecord and v4l2-ctl should fail with -EBUSY. Same if either of arecord/v4l2-ctl is running, then DVB should fail with -EBUSY.
BTW, I won't be able to test anything myself until Feb 9th since I'm abroad and don't have access to my au0828.
Regards,
Hans
shuah@deneb:/mnt/data/lkml/v4l-utils/utils/v4l2-ctl$ v4l2-ctl -f 700 VIDIOC_G_TUNER: failed: Device or resource busy VIDIOC_S_FREQUENCY: failed: Device or resource busy
Based on the current implementation, it failed with resource busy as expected.
- After stopping arecord:
v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video0 <<<<<<<<<<<<< 11.88 fps <<<<<<<<<<<<<<< 13.36 fps <<<<<<<<<<<<<<< 14.00 fps <<<<<<<<<<<<<<<< 14.35 fps <<<<<<<<<<<<<<< 14.57 fps <<<<<<<<<<<<<<<< 14.71 fps <<<<<<<<<
Worked as expected.
- After stopping above video streaming:
arecord -M -D plughw:2,0 -c2 -f S16_LE -t wav foo.wav Recording WAVE 'foo.wav' : Signed 16 bit Little Endian, Rate 8000 Hz, Stereo
Worked as expected.
BTW, I think there was also inconsistent behavior in the order of streaming audio and video: if I stream video first, then I can stream audio afterwards. But if I stream audio first, then (if I remember correctly) I can't start video streaming.
Okay this is what I saw:
While v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video0
I could start arecord -M -D plughw:2,0 -c2 -f S16_LE -t wav foo.wav
I ran strace on v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video0 and I didn't see AUDIO holds. The only think of here is that
v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video0
doesn't open audio device. I didn't see this when tested with other video apps, (tvtine, xawtv, vlc). When video is streaming, I see arecord failing with device busy.
As far I can see, exclusion logic is working correctly. s_freq does fail now based on the current logic.
btw I tested all of this on 5.0-rc4
thanks, -- Shuah
On 2/1/19 2:21 AM, Hans Verkuil wrote:
On 2/1/19 1:46 AM, shuah wrote:
Hi Hans,
On 1/30/19 12:42 AM, Hans Verkuil wrote:
On 1/30/19 2:50 AM, shuah wrote:
On 1/29/19 2:43 AM, Hans Verkuil wrote:
On 1/29/19 12:48 AM, shuah wrote:
Hi Hans,
On 1/28/19 5:03 AM, Hans Verkuil wrote: > Hi Shuah, > > On 1/24/19 9:32 PM, Shuah Khan wrote: >> Media Device Allocator API to allows multiple drivers share a media device. >> This API solves a very common use-case for media devices where one physical >> device (an USB stick) provides both audio and video. When such media device >> exposes a standard USB Audio class, a proprietary Video class, two or more >> independent drivers will share a single physical USB bridge. In such cases, >> it is necessary to coordinate access to the shared resource. >> >> Using this API, drivers can allocate a media device with the shared struct >> device as the key. Once the media device is allocated by a driver, other >> drivers can get a reference to it. The media device is released when all >> the references are released. >> >> - This patch series is tested on 5.0-rc3 and addresses comments on >> v9 series from Hans Verkuil. >> - v9 was tested on 4.20-rc6. >> - Tested sharing resources with kaffeine, vlc, xawtv, tvtime, and >> arecord. When analog is streaming, digital and audio user-space >> applications detect that the tuner is busy and exit. When digital >> is streaming, analog and audio applications detect that the tuner is >> busy and exit. When arecord is owns the tuner, digital and analog >> detect that the tuner is busy and exit. > > I've been doing some testing with my au0828, and I am confused about one > thing, probably because it has been too long ago since I last looked into > this in detail: >
Great.
> Why can't I change the tuner frequency if arecord (and only arecord) is > streaming audio? If arecord is streaming, then it is recording the audio > from the analog TV tuner, right? So changing the analog TV frequency > should be fine. >
Changing analog TV frequency would be s_frequency. The way it works is any s_* calls would require holding the pipeline. In Analog TV case, it would mean holding both audio and video pipelines for any changes including TV.
As I recall, we discussed this design and the decision was to make all s_* calls interfaces to hold the tuner. A special exception is g_tuner in case of au0828. au0828 initializes the tuner from s_* interfaces and its g_tuner interfaces. Allowing s_frequency to proceed will disrupt the arecord audio stream.
Query (q_*) works just fine without holding the pipeline. I limited the analog holds to just the ones that are required. The current set is required to avoid audio stream disruptions.
So I am not sure about that ('avoid audio stream disruptions'): if I stream video AND use arecord, then I can just set the frequency while streaming. Doesn't that interrupt audio as well? And are you sure changing the tuner frequency actually disrupts audio? And if audio is disrupted, are we talking about a glitch or is audio permanently disrupted?
I think it is a glitch. I will run some tests and let you know.
That's basically the inconsistent behavior I noticed: just running arecord will prevent me from changing the frequency, but if I run arecord and stream video, then it is suddenly OK to change the frequency.
How are you changing frequency? I want to duplicate what you are doing.
v4l2-ctl -f <freq>
I am not seeing the inconsistent behavior. Here are my results.
- Started acecord and while it is running:
arecord -M -D plughw:2,0 -c2 -f S16_LE -t wav foo.wav Recording WAVE 'foo.wav' : Signed 16 bit Little Endian, Rate 8000 Hz, Stereo
- Ran v4l2-ctl -f as follows:
v4l2-ctl -f 700 VIDIOC_G_TUNER: failed: Device or resource busy VIDIOC_S_FREQUENCY: failed: Device or resource busy
Based on the current implementation, it failed with resource busy as expected.
- Started v4l2-ctl as follows:
v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video0 VIDIOC_STREAMON: failed: Device or resource busy
Why is this? You have one analog tuner and it delivers independent audio and video streams. I should be able to start/stop audio and video independently.
And as mentioned above, if I use v4l2-ctl for video streaming, then start arecord, then that works fine. And I can change the tuner frequency while both are streaming.
But doing this the other way around (first start arecord, then v4l2-ctl) then that doesn't work.
It makes no sense.
Note that v4l2-ctl does not open audio, it solely deals with video. qv4l2 will open both audio and video, but for testing audio and video independently you need to use arecord/v4l2-ctl.
In any case, my understanding of how this should work is that both arecord and v4l2-ctl should attempt to lock the analog tuner resource, but they can share it.
If DVB is using the tuner, then both arecord and v4l2-ctl should fail with -EBUSY. Same if either of arecord/v4l2-ctl is running, then DVB should fail with -EBUSY.
BTW, I won't be able to test anything myself until Feb 9th since I'm abroad and don't have access to my au0828.
I am also on a business trip at the moment and won't have access to my au0828 until Feb 8th.
I think I understand your concern and the change to get the desired behavior of allowing tuner to be shared by arecord and v4l2-ctl is going to in the code that is already in the mainline which is the au0828_enable_source().
If you would like to wait to pull this series for the problem to be fixed, I can send that change in. It will be another patch added to the series.
Or if you pull this and I can send fix on top. Let me know your preference.
thanks, -- Shuah
On 2/5/19 7:10 PM, shuah wrote:
On 2/1/19 2:21 AM, Hans Verkuil wrote:
On 2/1/19 1:46 AM, shuah wrote:
Hi Hans,
On 1/30/19 12:42 AM, Hans Verkuil wrote:
On 1/30/19 2:50 AM, shuah wrote:
On 1/29/19 2:43 AM, Hans Verkuil wrote:
On 1/29/19 12:48 AM, shuah wrote: > Hi Hans, > > On 1/28/19 5:03 AM, Hans Verkuil wrote: >> Hi Shuah, >> >> On 1/24/19 9:32 PM, Shuah Khan wrote: >>> Media Device Allocator API to allows multiple drivers share a media device. >>> This API solves a very common use-case for media devices where one physical >>> device (an USB stick) provides both audio and video. When such media device >>> exposes a standard USB Audio class, a proprietary Video class, two or more >>> independent drivers will share a single physical USB bridge. In such cases, >>> it is necessary to coordinate access to the shared resource. >>> >>> Using this API, drivers can allocate a media device with the shared struct >>> device as the key. Once the media device is allocated by a driver, other >>> drivers can get a reference to it. The media device is released when all >>> the references are released. >>> >>> - This patch series is tested on 5.0-rc3 and addresses comments on >>> v9 series from Hans Verkuil. >>> - v9 was tested on 4.20-rc6. >>> - Tested sharing resources with kaffeine, vlc, xawtv, tvtime, and >>> arecord. When analog is streaming, digital and audio user-space >>> applications detect that the tuner is busy and exit. When digital >>> is streaming, analog and audio applications detect that the tuner is >>> busy and exit. When arecord is owns the tuner, digital and analog >>> detect that the tuner is busy and exit. >> >> I've been doing some testing with my au0828, and I am confused about one >> thing, probably because it has been too long ago since I last looked into >> this in detail: >> > > Great. > >> Why can't I change the tuner frequency if arecord (and only arecord) is >> streaming audio? If arecord is streaming, then it is recording the audio >> from the analog TV tuner, right? So changing the analog TV frequency >> should be fine. >> > > Changing analog TV frequency would be s_frequency. The way it works is > any s_* calls would require holding the pipeline. In Analog TV case, it > would mean holding both audio and video pipelines for any changes > including TV. > > As I recall, we discussed this design and the decision was to make all > s_* calls interfaces to hold the tuner. A special exception is g_tuner > in case of au0828. au0828 initializes the tuner from s_* interfaces and > its g_tuner interfaces. Allowing s_frequency to proceed will disrupt the > arecord audio stream. > > Query (q_*) works just fine without holding the pipeline. I limited the > analog holds to just the ones that are required. The current set is > required to avoid audio stream disruptions.
So I am not sure about that ('avoid audio stream disruptions'): if I stream video AND use arecord, then I can just set the frequency while streaming. Doesn't that interrupt audio as well? And are you sure changing the tuner frequency actually disrupts audio? And if audio is disrupted, are we talking about a glitch or is audio permanently disrupted?
I think it is a glitch. I will run some tests and let you know.
That's basically the inconsistent behavior I noticed: just running arecord will prevent me from changing the frequency, but if I run arecord and stream video, then it is suddenly OK to change the frequency.
How are you changing frequency? I want to duplicate what you are doing.
v4l2-ctl -f <freq>
I am not seeing the inconsistent behavior. Here are my results.
- Started acecord and while it is running:
arecord -M -D plughw:2,0 -c2 -f S16_LE -t wav foo.wav Recording WAVE 'foo.wav' : Signed 16 bit Little Endian, Rate 8000 Hz, Stereo
- Ran v4l2-ctl -f as follows:
v4l2-ctl -f 700 VIDIOC_G_TUNER: failed: Device or resource busy VIDIOC_S_FREQUENCY: failed: Device or resource busy
Based on the current implementation, it failed with resource busy as expected.
- Started v4l2-ctl as follows:
v4l2-ctl --stream-mmap --stream-count=100 -d /dev/video0 VIDIOC_STREAMON: failed: Device or resource busy
Why is this? You have one analog tuner and it delivers independent audio and video streams. I should be able to start/stop audio and video independently.
And as mentioned above, if I use v4l2-ctl for video streaming, then start arecord, then that works fine. And I can change the tuner frequency while both are streaming.
But doing this the other way around (first start arecord, then v4l2-ctl) then that doesn't work.
It makes no sense.
Note that v4l2-ctl does not open audio, it solely deals with video. qv4l2 will open both audio and video, but for testing audio and video independently you need to use arecord/v4l2-ctl.
In any case, my understanding of how this should work is that both arecord and v4l2-ctl should attempt to lock the analog tuner resource, but they can share it.
If DVB is using the tuner, then both arecord and v4l2-ctl should fail with -EBUSY. Same if either of arecord/v4l2-ctl is running, then DVB should fail with -EBUSY.
BTW, I won't be able to test anything myself until Feb 9th since I'm abroad and don't have access to my au0828.
I am also on a business trip at the moment and won't have access to my au0828 until Feb 8th.
I think I understand your concern and the change to get the desired behavior of allowing tuner to be shared by arecord and v4l2-ctl is going to in the code that is already in the mainline which is the au0828_enable_source().
If you would like to wait to pull this series for the problem to be fixed, I can send that change in. It will be another patch added to the series.
Or if you pull this and I can send fix on top. Let me know your preference.
I prefer to wait until I can test with the additional patch if that's OK?
Regards,
Hans
participants (4)
-
Hans Verkuil -
Sakari Ailus -
shuah -
Shuah Khan