[PATCH v2 0/6] Ancillary bus implementation and SOF multi-client support
Version 2 of this patchset is being pushed out when there are still some issues up in the air. This push, along with the changes in the change log below, is to fix the broken distribution among mailing lists and individuals of the first version submission.
At least two items still under discussion are: Final name for the bus Documentation of sysfs directories
Brief history of Ancillary Bus ============================== The ancillary bus code was originally submitted upstream as virtual bus, and was submitted through the netdev tree. This process generated up to v4. This discussion can be found here: https://lore.kernel.org/netdev/20191111192219.30259-1-jeffrey.t.kirsher@inte...
At this point, GregKH requested that we take the review and revision process to an internal mailing list and garner the buy-in of a respected kernel contributor.
The ancillary bus (then known as virtual bus) was originally submitted along with implementation code for the ice driver and irdma drive, causing the complication of also having dependencies in the rdma tree. This new submission is utilizing an ancillary bus consumer in only the sound driver tree to create the initial implementation and a single user.
Since implementation work has started on this patch set, there have been multiple inquiries about the time frame of its completion. It appears that there will be numerous consumers of this functionality.
The process of internal review and implementation using the sound drivers generated 19 internal versions. The changes, including the name change from virtual bus to ancillary bus, from these versions can be summarized as the following:
- Fixed compilation and checkpatch errors - Improved documentation to address the motivation for virtual bus. - Renamed virtual bus to ancillary bus - increased maximum device name size - Correct order in Kconfig and Makefile - removed the mid-layer adev->release layer for device unregister - pushed adev->id management to parent driver - all error paths out of ancillary_device_register return error code - all error paths out of ancillary_device_register use put_device - added adev->name element - modname in register cannot be NULL - added KBUILD_MODNAME as prefix for match_name - push adev->id responsibility to registering driver - uevent now parses adev->dev name - match_id function now parses adev->dev name - changed drivers probe function to also take an ancillary_device_id param - split ancillary_device_register into device_initialize and device_add - adjusted what is done in device_initialize and device_add - change adev to ancildev and adrv to ancildrv - change adev to ancildev in documentation
==========================
Introduces the ancillary bus implementation along with the example usage in the Sound Open Firmware(SOF) audio driver.
In some subsystems, the functionality of the core device (PCI/ACPI/other) may be too complex for a single device to be managed as a monolithic block or a part of the functionality might need to be exposed to a different subsystem. Splitting the functionality into smaller orthogonal devices makes it easier to manage data, power management and domain-specific communication with the hardware. Also, common ancillary_device functionality across primary devices can be handled by a common ancillary_device. A key requirement for such a split is that there is no dependency on a physical bus, device, register accesses or regmap support. These individual devices split from the core cannot live on the platform bus as they are not physical devices that are controlled by DT/ACPI. The same argument applies for not using MFD in this scenario as it relies on individual function devices being physical devices that are DT enumerated.
An example for this kind of requirement is the audio subsystem where a single IP handles multiple entities such as HDMI, Soundwire, local devices such as mics/speakers etc. The split for the core's functionality can be arbitrary or be defined by the DSP firmware topology and include hooks for test/debug. This allows for the audio core device to be minimal and tightly coupled with handling the hardware-specific logic and communication.
The ancillary bus is intended to be minimal, generic and avoid domain-specific assumptions. Each ancillary bus device represents a part of its parent functionality. The generic behavior can be extended and specialized as needed by encapsulating an ancillary bus device within other domain-specific structures and the use of .ops callbacks.
The SOF driver adopts the ancillary bus for implementing the multi-client support. A client in the context of the SOF driver represents a part of the core device's functionality. It is not a physical device but rather an ancillary device that needs to communicate with the DSP via IPCs. With multi-client support,the sound card can be separated into multiple orthogonal ancillary devices for local devices (mic/speakers etc), HDMI, sensing, probes, debug etc. In this series, we demonstrate the usage of the ancillary bus with the help of the IPC test client which is used for testing the serialization of IPCs when multiple clients talk to the DSP at the same time.
v2 changes: defined pr_fmt for kernel messages replaced WARN_ON calls in registration with pr_err calls adding kernel-doc function comments for device_initialize and device_add fix typo in documentation removed inaccurate line in documentation fixed formatting in drivers/bus/Makefile changed unwind path for sof_client_dev_alloc() improved comments for client list and mem freeing during client unreg removed debugfs entries in sof_ipc_test_client_drv during remove changed the signature of sof_debug_ipc_flood_test() fix a looping error in ancillary_match_id updated error value in sof_client_dev_register() mutex held while traversing client list when unregistering clients updated includes in sof-client.h
Dave Ertman (1): Add ancillary bus support
Fred Oh (1): ASoC: SOF: debug: Remove IPC flood test support in SOF core
Ranjani Sridharan (4): ASoC: SOF: Introduce descriptors for SOF client ASoC: SOF: Create client driver for IPC test ASoC: SOF: ops: Add ops for client registration ASoC: SOF: Intel: Define ops for client registration
Documentation/driver-api/ancillary_bus.rst | 229 ++++++++++++++ Documentation/driver-api/index.rst | 1 + drivers/bus/Kconfig | 3 + drivers/bus/Makefile | 3 + drivers/bus/ancillary.c | 225 ++++++++++++++ include/linux/ancillary_bus.h | 69 +++++ include/linux/mod_devicetable.h | 8 + scripts/mod/devicetable-offsets.c | 3 + scripts/mod/file2alias.c | 8 + sound/soc/sof/Kconfig | 29 +- sound/soc/sof/Makefile | 7 + sound/soc/sof/core.c | 12 + sound/soc/sof/debug.c | 230 -------------- sound/soc/sof/intel/Kconfig | 9 + sound/soc/sof/intel/Makefile | 3 + sound/soc/sof/intel/apl.c | 18 ++ sound/soc/sof/intel/bdw.c | 18 ++ sound/soc/sof/intel/byt.c | 22 ++ sound/soc/sof/intel/cnl.c | 18 ++ sound/soc/sof/intel/intel-client.c | 53 ++++ sound/soc/sof/intel/intel-client.h | 26 ++ sound/soc/sof/ops.h | 14 + sound/soc/sof/sof-client.c | 119 ++++++++ sound/soc/sof/sof-client.h | 69 +++++ sound/soc/sof/sof-ipc-test-client.c | 332 +++++++++++++++++++++ sound/soc/sof/sof-priv.h | 19 +- 26 files changed, 1308 insertions(+), 239 deletions(-) create mode 100644 Documentation/driver-api/ancillary_bus.rst create mode 100644 drivers/bus/ancillary.c create mode 100644 include/linux/ancillary_bus.h create mode 100644 sound/soc/sof/intel/intel-client.c create mode 100644 sound/soc/sof/intel/intel-client.h create mode 100644 sound/soc/sof/sof-client.c create mode 100644 sound/soc/sof/sof-client.h create mode 100644 sound/soc/sof/sof-ipc-test-client.c
Add support for the Ancillary Bus, ancillary_device and ancillary_driver. It enables drivers to create an ancillary_device and bind an ancillary_driver to it.
The bus supports probe/remove shutdown and suspend/resume callbacks. Each ancillary_device has a unique string based id; driver binds to an ancillary_device based on this id through the bus.
Co-developed-by: Kiran Patil kiran.patil@intel.com Signed-off-by: Kiran Patil kiran.patil@intel.com Co-developed-by: Ranjani Sridharan ranjani.sridharan@linux.intel.com Signed-off-by: Ranjani Sridharan ranjani.sridharan@linux.intel.com Co-developed-by: Fred Oh fred.oh@linux.intel.com Signed-off-by: Fred Oh fred.oh@linux.intel.com Reviewed-by: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Reviewed-by: Shiraz Saleem shiraz.saleem@intel.com Reviewed-by: Parav Pandit parav@mellanox.com Reviewed-by: Dan Williams dan.j.williams@intel.com Signed-off-by: Dave Ertman david.m.ertman@intel.com --- Documentation/driver-api/ancillary_bus.rst | 229 +++++++++++++++++++++ Documentation/driver-api/index.rst | 1 + drivers/bus/Kconfig | 3 + drivers/bus/Makefile | 3 + drivers/bus/ancillary.c | 225 ++++++++++++++++++++ include/linux/ancillary_bus.h | 69 +++++++ include/linux/mod_devicetable.h | 8 + scripts/mod/devicetable-offsets.c | 3 + scripts/mod/file2alias.c | 8 + 9 files changed, 549 insertions(+) create mode 100644 Documentation/driver-api/ancillary_bus.rst create mode 100644 drivers/bus/ancillary.c create mode 100644 include/linux/ancillary_bus.h
diff --git a/Documentation/driver-api/ancillary_bus.rst b/Documentation/driver-api/ancillary_bus.rst new file mode 100644 index 000000000000..66f986e8672f --- /dev/null +++ b/Documentation/driver-api/ancillary_bus.rst @@ -0,0 +1,229 @@ +.. SPDX-License-Identifier: GPL-2.0-only + +============= +Ancillary Bus +============= + +In some subsystems, the functionality of the core device (PCI/ACPI/other) is +too complex for a single device to be managed as a monolithic block or a part of +the functionality needs to be exposed to a different subsystem. Splitting the +functionality into smaller orthogonal devices would make it easier to manage +data, power management and domain-specific interaction with the hardware. A key +requirement for such a split is that there is no dependency on a physical bus, +device, register accesses or regmap support. These individual devices split from +the core cannot live on the platform bus as they are not physical devices that +are controlled by DT/ACPI. The same argument applies for not using MFD in this +scenario as MFD relies on individual function devices being physical devices. + +An example for this kind of requirement is the audio subsystem where a single +IP is handling multiple entities such as HDMI, Soundwire, local devices such as +mics/speakers etc. The split for the core's functionality can be arbitrary or +be defined by the DSP firmware topology and include hooks for test/debug. This +allows for the audio core device to be minimal and focused on hardware-specific +control and communication. + +The ancillary bus is intended to be minimal, generic and avoid domain-specific +assumptions. Each ancillary_device represents a part of its parent +functionality. The generic behavior can be extended and specialized as needed +by encapsulating an ancillary_device within other domain-specific structures and +the use of .ops callbacks. Devices on the ancillary bus do not share any +structures and the use of a communication channel with the parent is +domain-specific. + +When Should the Ancillary Bus Be Used +===================================== + +The ancillary bus is to be used when a driver and one or more kernel modules, +who share a common header file with the driver, need a mechanism to connect and +provide access to a shared object allocated by the ancillary_device's +registering driver. The registering driver for the ancillary_device(s) and the +kernel module(s) registering ancillary_drivers can be from the same subsystem, +or from multiple subsystems. + +The emphasis here is on a common generic interface that keeps subsystem +customization out of the bus infrastructure. + +One example could be a multi-port PCI network device that is rdma-capable and +needs to export this functionality and attach to an rdma driver in another +subsystem. The PCI driver will allocate and register an ancillary_device for +each physical function on the NIC. The rdma driver will register an +ancillary_driver that will be matched with and probed for each of these +ancillary_devices. This will give the rdma driver access to the shared data/ops +in the PCI drivers shared object to establish a connection with the PCI driver. + +Another use case is for the PCI device to be split out into multiple sub +functions. For each sub function an ancillary_device will be created. A PCI +sub function driver will bind to such devices that will create its own one or +more class devices. A PCI sub function ancillary device will likely be +contained in a struct with additional attributes such as user defined sub +function number and optional attributes such as resources and a link to the +parent device. These attributes could be used by systemd/udev; and hence should +be initialized before a driver binds to an ancillary_device. + +Ancillary Device +================ + +An ancillary_device is created and registered to represent a part of its parent +device's functionality. It is given a name that, combined with the registering +drivers KBUILD_MODNAME, creates a match_name that is used for driver binding, +and an id that combined with the match_name provide a unique name to register +with the bus subsystem. + +Registering an ancillary_device is a two-step process. First you must call +ancillary_device_initialize(), which will check several aspects of the +ancillary_device struct and perform a device_initialize(). After this step +completes, any error state must have a call to put_device() in its resolution +path. The second step in registering an ancillary_device is to perform a call +to ancillary_device_add(), which will set the name of the device and add the +device to the bus. + +To unregister an ancillary_device, just a call to ancillary_device_unregister() +is used. This will perform both a device_del() and a put_device(). + +.. code-block:: c + + struct ancillary_device { + struct device dev; + const char *name; + u32 id; + }; + +If two ancillary_devices both with a match_name "mod.foo" are registered onto +the bus, they must have unique id values (e.g. "x" and "y") so that the +registered devices names will be "mod.foo.x" and "mod.foo.y". If match_name + +id are not unique, then the device_add will fail and generate an error message. + +The ancillary_device.dev.type.release or ancillary_device.dev.release must be +populated with a non-NULL pointer to successfully register the ancillary_device. + +The ancillary_device.dev.parent must also be populated. + +Ancillary Device Memory Model and Lifespan +------------------------------------------ + +When a kernel driver registers an ancillary_device on the ancillary bus, we will +use the nomenclature to refer to this kernel driver as a registering driver. It +is the entity that will allocate memory for the ancillary_device and register it +on the ancillary bus. It is important to note that, as opposed to the platform +bus, the registering driver is wholly responsible for the management for the +memory used for the driver object. + +A parent object, defined in the shared header file, will contain the +ancillary_device. It will also contain a pointer to the shared object(s), which +will also be defined in the shared header. Both the parent object and the +shared object(s) will be allocated by the registering driver. This layout +allows the ancillary_driver's registering module to perform a container_of() +call to go from the pointer to the ancillary_device, that is passed during the +call to the ancillary_driver's probe function, up to the parent object, and then +have access to the shared object(s). + +The memory for the ancillary_device will be freed only in its release() +callback flow as defined by its registering driver. + +The memory for the shared object(s) must have a lifespan equal to, or greater +than, the lifespan of the memory for the ancillary_device. The ancillary_driver +should only consider that this shared object is valid as long as the +ancillary_device is still registered on the ancillary bus. It is up to the +registering driver to manage (e.g. free or keep available) the memory for the +shared object beyond the life of the ancillary_device. + +Registering driver must unregister all ancillary devices before its registering +parent device's remove() is completed. + +Ancillary Drivers +================= + +Ancillary drivers follow the standard driver model convention, where +discovery/enumeration is handled by the core, and drivers +provide probe() and remove() methods. They support power management +and shutdown notifications using the standard conventions. + +.. code-block:: c + + struct ancillary_driver { + int (*probe)(struct ancillary_device *, + const struct ancillary_device_id *id); + int (*remove)(struct ancillary_device *); + void (*shutdown)(struct ancillary_device *); + int (*suspend)(struct ancillary_device *, pm_message_t); + int (*resume)(struct ancillary_device *); + struct device_driver driver; + const struct ancillary_device_id *id_table; + }; + +Ancillary drivers register themselves with the bus by calling +ancillary_driver_register(). The id_table contains the match_names of ancillary +devices that a driver can bind with. + +Example Usage +============= + +Ancillary devices are created and registered by a subsystem-level core device +that needs to break up its functionality into smaller fragments. One way to +extend the scope of an ancillary_device would be to encapsulate it within a +domain-specific structure defined by the parent device. This structure contains +the ancillary_device and any associated shared data/callbacks needed to +establish the connection with the parent. + +An example would be: + +.. code-block:: c + + struct foo { + struct ancillary_device ancildev; + void (*connect)(struct ancillary_device *ancildev); + void (*disconnect)(struct ancillary_device *ancildev); + void *data; + }; + +The parent device would then register the ancillary_device by calling +ancillary_device_initialize(), and then ancillary_device_add(), with the pointer +to the ancildev member of the above structure. The parent would provide a name +for the ancillary_device that, combined with the parent's KBUILD_MODNAME, will +create a match_name that will be used for matching and binding with a driver. + +Whenever an ancillary_driver is registered, based on the match_name, the +ancillary_driver's probe() is invoked for the matching devices. The +ancillary_driver can also be encapsulated inside custom drivers that make the +core device's functionality extensible by adding additional domain-specific ops +as follows: + +.. code-block:: c + + struct my_ops { + void (*send)(struct ancillary_device *ancildev); + void (*receive)(struct ancillary_device *ancildev); + }; + + + struct my_driver { + struct ancillary_driver ancillary_drv; + const struct my_ops ops; + }; + +An example of this type of usage would be: + +.. code-block:: c + + const struct ancillary_device_id my_ancillary_id_table[] = { + { .name = "foo_mod.foo_dev" }, + { }, + }; + + const struct my_ops my_custom_ops = { + .send = my_tx, + .receive = my_rx, + }; + + const struct my_driver my_drv = { + .ancillary_drv = { + .driver = { + .name = "myancillarydrv", + }, + .id_table = my_ancillary_id_table, + .probe = my_probe, + .remove = my_remove, + .shutdown = my_shutdown, + }, + .ops = my_custom_ops, + }; diff --git a/Documentation/driver-api/index.rst b/Documentation/driver-api/index.rst index 5ef2cfe3a16b..9584ac2ed1f5 100644 --- a/Documentation/driver-api/index.rst +++ b/Documentation/driver-api/index.rst @@ -74,6 +74,7 @@ available subsections can be seen below. thermal/index fpga/index acpi/index + ancillary_bus backlight/lp855x-driver.rst connector console diff --git a/drivers/bus/Kconfig b/drivers/bus/Kconfig index 0c262c2aeaf2..ba82a045b847 100644 --- a/drivers/bus/Kconfig +++ b/drivers/bus/Kconfig @@ -5,6 +5,9 @@
menu "Bus devices"
+config ANCILLARY_BUS + tristate + config ARM_CCI bool
diff --git a/drivers/bus/Makefile b/drivers/bus/Makefile index 397e35392bff..7c217eb1dbb7 100644 --- a/drivers/bus/Makefile +++ b/drivers/bus/Makefile @@ -3,6 +3,9 @@ # Makefile for the bus drivers. #
+# Ancillary bus driver +obj-$(CONFIG_ANCILLARY_BUS) += ancillary.o + # Interconnect bus drivers for ARM platforms obj-$(CONFIG_ARM_CCI) += arm-cci.o obj-$(CONFIG_ARM_INTEGRATOR_LM) += arm-integrator-lm.o diff --git a/drivers/bus/ancillary.c b/drivers/bus/ancillary.c new file mode 100644 index 000000000000..93888ca36fb1 --- /dev/null +++ b/drivers/bus/ancillary.c @@ -0,0 +1,225 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Software based bus for Ancillary devices + * + * Copyright (c) 2019-2020 Intel Corporation + * + * Please see Documentation/driver-api/ancillary_bus.rst for more information. + */ + +#define pr_fmt(fmt) "%s:%s: " fmt, KBUILD_MODNAME, __func__ + +#include <linux/device.h> +#include <linux/init.h> +#include <linux/module.h> +#include <linux/pm_domain.h> +#include <linux/pm_runtime.h> +#include <linux/string.h> +#include <linux/ancillary_bus.h> + +static const struct ancillary_device_id *ancillary_match_id(const struct ancillary_device_id *id, + const struct ancillary_device *ancildev) +{ + while (id->name[0]) { + const char *p = strrchr(dev_name(&ancildev->dev), '.'); + int match_size; + + if (!p) { + id++; + continue; + } + match_size = p - dev_name(&ancildev->dev); + + /* use dev_name(&ancildev->dev) prefix before last '.' char to match to */ + if (!strncmp(dev_name(&ancildev->dev), id->name, match_size)) + return id; + id++; + } + return NULL; +} + +static int ancillary_match(struct device *dev, struct device_driver *drv) +{ + struct ancillary_device *ancildev = to_ancillary_dev(dev); + struct ancillary_driver *ancildrv = to_ancillary_drv(drv); + + return !!ancillary_match_id(ancildrv->id_table, ancildev); +} + +static int ancillary_uevent(struct device *dev, struct kobj_uevent_env *env) +{ + const char *name, *p; + + name = dev_name(dev); + p = strrchr(name, '.'); + + return add_uevent_var(env, "MODALIAS=%s%.*s", ANCILLARY_MODULE_PREFIX, (int)(p - name), + name); +} + +static const struct dev_pm_ops ancillary_dev_pm_ops = { + SET_RUNTIME_PM_OPS(pm_generic_runtime_suspend, pm_generic_runtime_resume, NULL) + SET_SYSTEM_SLEEP_PM_OPS(pm_generic_suspend, pm_generic_resume) +}; + +struct bus_type ancillary_bus_type = { + .name = "ancillary", + .match = ancillary_match, + .uevent = ancillary_uevent, + .pm = &ancillary_dev_pm_ops, +}; + +/** + * ancillary_device_initialize - check ancillary_device and initialize + * @ancildev: ancillary device struct + * + * This is the first step in the two-step process to register an ancillary_device. + * + * When this function returns an error code, then the device_initialize will *not* have + * been performed, and the caller will be responsible to free any memory allocated for the + * ancillary_device in the error path directly. + * + * It returns 0 on success. On success, the device_initialize has been performed. + * After this point any error unwinding will need to include a call to put_device(). + * In this post-initialize error scenario, a call to the device's .release callback will be + * triggered by put_device(), and all memory clean-up is expected to be handled there. + */ +int ancillary_device_initialize(struct ancillary_device *ancildev) +{ + struct device *dev = &ancildev->dev; + + dev->bus = &ancillary_bus_type; + + if (!dev->parent) { + pr_err("ancillary_device has a NULL dev->parent\n"); + return -EINVAL; + } + + if (!ancildev->name) { + pr_err("acillary_device has a NULL name\n"); + return -EINVAL; + } + + if (!(dev->type && dev->type->release) && !dev->release) { + pr_err("ancillary_device does not have a release callback defined\n"); + return -EINVAL; + } + + device_initialize(&ancildev->dev); + return 0; +} +EXPORT_SYMBOL_GPL(ancillary_device_initialize); + +/** + * __ancillary_device_add - add an ancillary bus device + * @ancildev: ancillary bus device to add to the bus + * @modname: name of the parent device's driver module + * + * This is the second step in the two-step process to register an ancillary_device. + * + * This function must be called after a successful call to ancillary_device_initialize(), which + * will perform the device_initialize. This means that if this returns an error code, then a + * put_device must be performed so that the .release callback will be triggered to free the + * memory associated with the ancillary_device. + */ +int __ancillary_device_add(struct ancillary_device *ancildev, const char *modname) +{ + struct device *dev = &ancildev->dev; + int ret; + + if (!modname) { + pr_err("ancillary device modname is NULL\n"); + return -EINVAL; + } + + ret = dev_set_name(dev, "%s.%s.%d", modname, ancildev->name, ancildev->id); + if (ret) { + pr_err("ancillary device dev_set_name failed: %d\n", ret); + return ret; + } + + ret = device_add(dev); + if (ret) + dev_err(dev, "adding ancillary device failed!: %d\n", ret); + + return ret; +} +EXPORT_SYMBOL_GPL(__ancillary_device_add); + +static int ancillary_probe_driver(struct device *dev) +{ + struct ancillary_driver *ancildrv = to_ancillary_drv(dev->driver); + struct ancillary_device *ancildev = to_ancillary_dev(dev); + int ret; + + ret = dev_pm_domain_attach(dev, true); + if (ret) { + dev_warn(dev, "Failed to attach to PM Domain : %d\n", ret); + return ret; + } + + ret = ancildrv->probe(ancildev, ancillary_match_id(ancildrv->id_table, ancildev)); + if (ret) + dev_pm_domain_detach(dev, true); + + return ret; +} + +static int ancillary_remove_driver(struct device *dev) +{ + struct ancillary_driver *ancildrv = to_ancillary_drv(dev->driver); + struct ancillary_device *ancildev = to_ancillary_dev(dev); + int ret; + + ret = ancildrv->remove(ancildev); + dev_pm_domain_detach(dev, true); + + return ret; +} + +static void ancillary_shutdown_driver(struct device *dev) +{ + struct ancillary_driver *ancildrv = to_ancillary_drv(dev->driver); + struct ancillary_device *ancildev = to_ancillary_dev(dev); + + ancildrv->shutdown(ancildev); +} + +/** + * __ancillary_driver_register - register a driver for ancillary bus devices + * @ancildrv: ancillary_driver structure + * @owner: owning module/driver + */ +int __ancillary_driver_register(struct ancillary_driver *ancildrv, struct module *owner) +{ + if (WARN_ON(!ancildrv->probe) || WARN_ON(!ancildrv->remove) || + WARN_ON(!ancildrv->shutdown) || WARN_ON(!ancildrv->id_table)) + return -EINVAL; + + ancildrv->driver.owner = owner; + ancildrv->driver.bus = &ancillary_bus_type; + ancildrv->driver.probe = ancillary_probe_driver; + ancildrv->driver.remove = ancillary_remove_driver; + ancildrv->driver.shutdown = ancillary_shutdown_driver; + + return driver_register(&ancildrv->driver); +} +EXPORT_SYMBOL_GPL(__ancillary_driver_register); + +static int __init ancillary_bus_init(void) +{ + return bus_register(&ancillary_bus_type); +} + +static void __exit ancillary_bus_exit(void) +{ + bus_unregister(&ancillary_bus_type); +} + +module_init(ancillary_bus_init); +module_exit(ancillary_bus_exit); + +MODULE_LICENSE("GPL v2"); +MODULE_DESCRIPTION("Ancillary Bus"); +MODULE_AUTHOR("David Ertman david.m.ertman@intel.com"); +MODULE_AUTHOR("Kiran Patil kiran.patil@intel.com"); diff --git a/include/linux/ancillary_bus.h b/include/linux/ancillary_bus.h new file mode 100644 index 000000000000..72169c8a5dfe --- /dev/null +++ b/include/linux/ancillary_bus.h @@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* + * Copyright (c) 2019-2020 Intel Corporation + * + * Please see Documentation/driver-api/ancillary_bus.rst for more information. + */ + +#ifndef _ANCILLARY_BUS_H_ +#define _ANCILLARY_BUS_H_ + +#include <linux/device.h> +#include <linux/mod_devicetable.h> +#include <linux/slab.h> + +struct ancillary_device { + struct device dev; + const char *name; + u32 id; +}; + +struct ancillary_driver { + int (*probe)(struct ancillary_device *ancildev, const struct ancillary_device_id *id); + int (*remove)(struct ancillary_device *ancildev); + void (*shutdown)(struct ancillary_device *ancildev); + int (*suspend)(struct ancillary_device *ancildev, pm_message_t state); + int (*resume)(struct ancillary_device *ancildev); + struct device_driver driver; + const struct ancillary_device_id *id_table; +}; + +static inline struct ancillary_device *to_ancillary_dev(struct device *dev) +{ + return container_of(dev, struct ancillary_device, dev); +} + +static inline struct ancillary_driver *to_ancillary_drv(struct device_driver *drv) +{ + return container_of(drv, struct ancillary_driver, driver); +} + +int ancillary_device_initialize(struct ancillary_device *ancildev); +int __ancillary_device_add(struct ancillary_device *ancildev, const char *modname); +#define ancillary_device_add(ancildev) __ancillary_device_add(ancildev, KBUILD_MODNAME) + +static inline void ancillary_device_unregister(struct ancillary_device *ancildev) +{ + device_unregister(&ancildev->dev); +} + +int __ancillary_driver_register(struct ancillary_driver *ancildrv, struct module *owner); +#define ancillary_driver_register(ancildrv) __ancillary_driver_register(ancildrv, THIS_MODULE) + +static inline void ancillary_driver_unregister(struct ancillary_driver *ancildrv) +{ + driver_unregister(&ancildrv->driver); +} + +/** + * module_ancillary_driver() - Helper macro for registering an ancillary driver + * @__ancillary_driver: ancillary driver struct + * + * Helper macro for ancillary drivers which do not do anything special in + * module init/exit. This eliminates a lot of boilerplate. Each module may only + * use this macro once, and calling it replaces module_init() and module_exit() + */ +#define module_ancillary_driver(__ancillary_driver) \ + module_driver(__ancillary_driver, ancillary_driver_register, ancillary_driver_unregister) + +#endif /* _ANCILLARY_BUS_H_ */ diff --git a/include/linux/mod_devicetable.h b/include/linux/mod_devicetable.h index 5b08a473cdba..7d596dc30833 100644 --- a/include/linux/mod_devicetable.h +++ b/include/linux/mod_devicetable.h @@ -838,4 +838,12 @@ struct mhi_device_id { kernel_ulong_t driver_data; };
+#define ANCILLARY_NAME_SIZE 32 +#define ANCILLARY_MODULE_PREFIX "ancillary:" + +struct ancillary_device_id { + char name[ANCILLARY_NAME_SIZE]; + kernel_ulong_t driver_data; +}; + #endif /* LINUX_MOD_DEVICETABLE_H */ diff --git a/scripts/mod/devicetable-offsets.c b/scripts/mod/devicetable-offsets.c index 27007c18e754..79e37c4c25b3 100644 --- a/scripts/mod/devicetable-offsets.c +++ b/scripts/mod/devicetable-offsets.c @@ -243,5 +243,8 @@ int main(void) DEVID(mhi_device_id); DEVID_FIELD(mhi_device_id, chan);
+ DEVID(ancillary_device_id); + DEVID_FIELD(ancillary_device_id, name); + return 0; } diff --git a/scripts/mod/file2alias.c b/scripts/mod/file2alias.c index 2417dd1dee33..99c4fcd82bf3 100644 --- a/scripts/mod/file2alias.c +++ b/scripts/mod/file2alias.c @@ -1364,6 +1364,13 @@ static int do_mhi_entry(const char *filename, void *symval, char *alias) { DEF_FIELD_ADDR(symval, mhi_device_id, chan); sprintf(alias, MHI_DEVICE_MODALIAS_FMT, *chan); + return 1; +} + +static int do_ancillary_entry(const char *filename, void *symval, char *alias) +{ + DEF_FIELD_ADDR(symval, ancillary_device_id, name); + sprintf(alias, ANCILLARY_MODULE_PREFIX "%s", *name);
return 1; } @@ -1442,6 +1449,7 @@ static const struct devtable devtable[] = { {"tee", SIZE_tee_client_device_id, do_tee_entry}, {"wmi", SIZE_wmi_device_id, do_wmi_entry}, {"mhi", SIZE_mhi_device_id, do_mhi_entry}, + {"ancillary", SIZE_ancillary_device_id, do_ancillary_entry}, };
/* Create MODULE_ALIAS() statements.
On Mon, Oct 05, 2020 at 11:24:41AM -0700, Dave Ertman wrote:
Add support for the Ancillary Bus, ancillary_device and ancillary_driver. It enables drivers to create an ancillary_device and bind an ancillary_driver to it.
I was under impression that this name is going to be changed.
The bus supports probe/remove shutdown and suspend/resume callbacks. Each ancillary_device has a unique string based id; driver binds to an ancillary_device based on this id through the bus.
Co-developed-by: Kiran Patil kiran.patil@intel.com Signed-off-by: Kiran Patil kiran.patil@intel.com Co-developed-by: Ranjani Sridharan ranjani.sridharan@linux.intel.com Signed-off-by: Ranjani Sridharan ranjani.sridharan@linux.intel.com Co-developed-by: Fred Oh fred.oh@linux.intel.com Signed-off-by: Fred Oh fred.oh@linux.intel.com Reviewed-by: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Reviewed-by: Shiraz Saleem shiraz.saleem@intel.com Reviewed-by: Parav Pandit parav@mellanox.com Reviewed-by: Dan Williams dan.j.williams@intel.com Signed-off-by: Dave Ertman david.m.ertman@intel.com
Documentation/driver-api/ancillary_bus.rst | 229 +++++++++++++++++++++ Documentation/driver-api/index.rst | 1 + drivers/bus/Kconfig | 3 + drivers/bus/Makefile | 3 + drivers/bus/ancillary.c | 225 ++++++++++++++++++++ include/linux/ancillary_bus.h | 69 +++++++ include/linux/mod_devicetable.h | 8 + scripts/mod/devicetable-offsets.c | 3 + scripts/mod/file2alias.c | 8 + 9 files changed, 549 insertions(+) create mode 100644 Documentation/driver-api/ancillary_bus.rst create mode 100644 drivers/bus/ancillary.c create mode 100644 include/linux/ancillary_bus.h
diff --git a/Documentation/driver-api/ancillary_bus.rst b/Documentation/driver-api/ancillary_bus.rst new file mode 100644 index 000000000000..66f986e8672f --- /dev/null +++ b/Documentation/driver-api/ancillary_bus.rst @@ -0,0 +1,229 @@ +.. SPDX-License-Identifier: GPL-2.0-only
+============= +Ancillary Bus +=============
+In some subsystems, the functionality of the core device (PCI/ACPI/other) is +too complex for a single device to be managed as a monolithic block or a part of +the functionality needs to be exposed to a different subsystem. Splitting the +functionality into smaller orthogonal devices would make it easier to manage +data, power management and domain-specific interaction with the hardware. A key +requirement for such a split is that there is no dependency on a physical bus, +device, register accesses or regmap support. These individual devices split from +the core cannot live on the platform bus as they are not physical devices that +are controlled by DT/ACPI. The same argument applies for not using MFD in this +scenario as MFD relies on individual function devices being physical devices.
+An example for this kind of requirement is the audio subsystem where a single +IP is handling multiple entities such as HDMI, Soundwire, local devices such as +mics/speakers etc. The split for the core's functionality can be arbitrary or +be defined by the DSP firmware topology and include hooks for test/debug. This +allows for the audio core device to be minimal and focused on hardware-specific +control and communication.
+The ancillary bus is intended to be minimal, generic and avoid domain-specific +assumptions. Each ancillary_device represents a part of its parent +functionality. The generic behavior can be extended and specialized as needed +by encapsulating an ancillary_device within other domain-specific structures and +the use of .ops callbacks. Devices on the ancillary bus do not share any +structures and the use of a communication channel with the parent is +domain-specific.
+When Should the Ancillary Bus Be Used +=====================================
+The ancillary bus is to be used when a driver and one or more kernel modules, +who share a common header file with the driver, need a mechanism to connect and +provide access to a shared object allocated by the ancillary_device's +registering driver. The registering driver for the ancillary_device(s) and the +kernel module(s) registering ancillary_drivers can be from the same subsystem, +or from multiple subsystems.
+The emphasis here is on a common generic interface that keeps subsystem +customization out of the bus infrastructure.
+One example could be a multi-port PCI network device that is rdma-capable and +needs to export this functionality and attach to an rdma driver in another +subsystem. The PCI driver will allocate and register an ancillary_device for +each physical function on the NIC. The rdma driver will register an +ancillary_driver that will be matched with and probed for each of these +ancillary_devices. This will give the rdma driver access to the shared data/ops +in the PCI drivers shared object to establish a connection with the PCI driver.
+Another use case is for the PCI device to be split out into multiple sub +functions. For each sub function an ancillary_device will be created. A PCI +sub function driver will bind to such devices that will create its own one or +more class devices. A PCI sub function ancillary device will likely be +contained in a struct with additional attributes such as user defined sub +function number and optional attributes such as resources and a link to the +parent device. These attributes could be used by systemd/udev; and hence should +be initialized before a driver binds to an ancillary_device.
+Ancillary Device +================
+An ancillary_device is created and registered to represent a part of its parent +device's functionality. It is given a name that, combined with the registering +drivers KBUILD_MODNAME, creates a match_name that is used for driver binding, +and an id that combined with the match_name provide a unique name to register +with the bus subsystem.
+Registering an ancillary_device is a two-step process. First you must call +ancillary_device_initialize(), which will check several aspects of the +ancillary_device struct and perform a device_initialize(). After this step +completes, any error state must have a call to put_device() in its resolution +path. The second step in registering an ancillary_device is to perform a call +to ancillary_device_add(), which will set the name of the device and add the +device to the bus.
+To unregister an ancillary_device, just a call to ancillary_device_unregister() +is used. This will perform both a device_del() and a put_device().
+.. code-block:: c
- struct ancillary_device {
struct device dev;const char *name;u32 id;- };
+If two ancillary_devices both with a match_name "mod.foo" are registered onto +the bus, they must have unique id values (e.g. "x" and "y") so that the +registered devices names will be "mod.foo.x" and "mod.foo.y". If match_name + +id are not unique, then the device_add will fail and generate an error message.
+The ancillary_device.dev.type.release or ancillary_device.dev.release must be +populated with a non-NULL pointer to successfully register the ancillary_device.
+The ancillary_device.dev.parent must also be populated.
+Ancillary Device Memory Model and Lifespan +------------------------------------------
+When a kernel driver registers an ancillary_device on the ancillary bus, we will +use the nomenclature to refer to this kernel driver as a registering driver. It +is the entity that will allocate memory for the ancillary_device and register it +on the ancillary bus. It is important to note that, as opposed to the platform +bus, the registering driver is wholly responsible for the management for the +memory used for the driver object.
+A parent object, defined in the shared header file, will contain the +ancillary_device. It will also contain a pointer to the shared object(s), which +will also be defined in the shared header. Both the parent object and the +shared object(s) will be allocated by the registering driver. This layout +allows the ancillary_driver's registering module to perform a container_of() +call to go from the pointer to the ancillary_device, that is passed during the +call to the ancillary_driver's probe function, up to the parent object, and then +have access to the shared object(s).
+The memory for the ancillary_device will be freed only in its release() +callback flow as defined by its registering driver.
+The memory for the shared object(s) must have a lifespan equal to, or greater +than, the lifespan of the memory for the ancillary_device. The ancillary_driver +should only consider that this shared object is valid as long as the +ancillary_device is still registered on the ancillary bus. It is up to the +registering driver to manage (e.g. free or keep available) the memory for the +shared object beyond the life of the ancillary_device.
+Registering driver must unregister all ancillary devices before its registering +parent device's remove() is completed.
+Ancillary Drivers +=================
+Ancillary drivers follow the standard driver model convention, where +discovery/enumeration is handled by the core, and drivers +provide probe() and remove() methods. They support power management +and shutdown notifications using the standard conventions.
+.. code-block:: c
- struct ancillary_driver {
int (*probe)(struct ancillary_device *,const struct ancillary_device_id *id);int (*remove)(struct ancillary_device *);void (*shutdown)(struct ancillary_device *);int (*suspend)(struct ancillary_device *, pm_message_t);int (*resume)(struct ancillary_device *);struct device_driver driver;const struct ancillary_device_id *id_table;- };
+Ancillary drivers register themselves with the bus by calling +ancillary_driver_register(). The id_table contains the match_names of ancillary +devices that a driver can bind with.
+Example Usage +=============
+Ancillary devices are created and registered by a subsystem-level core device +that needs to break up its functionality into smaller fragments. One way to +extend the scope of an ancillary_device would be to encapsulate it within a +domain-specific structure defined by the parent device. This structure contains +the ancillary_device and any associated shared data/callbacks needed to +establish the connection with the parent.
+An example would be:
+.. code-block:: c
struct foo {struct ancillary_device ancildev;void (*connect)(struct ancillary_device *ancildev);void (*disconnect)(struct ancillary_device *ancildev);void *data;};+The parent device would then register the ancillary_device by calling +ancillary_device_initialize(), and then ancillary_device_add(), with the pointer +to the ancildev member of the above structure. The parent would provide a name +for the ancillary_device that, combined with the parent's KBUILD_MODNAME, will +create a match_name that will be used for matching and binding with a driver.
+Whenever an ancillary_driver is registered, based on the match_name, the +ancillary_driver's probe() is invoked for the matching devices. The +ancillary_driver can also be encapsulated inside custom drivers that make the +core device's functionality extensible by adding additional domain-specific ops +as follows:
+.. code-block:: c
- struct my_ops {
void (*send)(struct ancillary_device *ancildev);void (*receive)(struct ancillary_device *ancildev);- };
- struct my_driver {
struct ancillary_driver ancillary_drv;const struct my_ops ops;- };
+An example of this type of usage would be:
+.. code-block:: c
- const struct ancillary_device_id my_ancillary_id_table[] = {
{ .name = "foo_mod.foo_dev" },{ },- };
- const struct my_ops my_custom_ops = {
.send = my_tx,.receive = my_rx,- };
- const struct my_driver my_drv = {
.ancillary_drv = {.driver = {.name = "myancillarydrv",
Why do we need to give control over driver name to the driver authors? It can be problematic if author puts name that already exists.
},.id_table = my_ancillary_id_table,.probe = my_probe,.remove = my_remove,.shutdown = my_shutdown,},.ops = my_custom_ops,- };
diff --git a/Documentation/driver-api/index.rst b/Documentation/driver-api/index.rst index 5ef2cfe3a16b..9584ac2ed1f5 100644 --- a/Documentation/driver-api/index.rst +++ b/Documentation/driver-api/index.rst @@ -74,6 +74,7 @@ available subsections can be seen below. thermal/index fpga/index acpi/index
- ancillary_bus backlight/lp855x-driver.rst connector console
diff --git a/drivers/bus/Kconfig b/drivers/bus/Kconfig index 0c262c2aeaf2..ba82a045b847 100644 --- a/drivers/bus/Kconfig +++ b/drivers/bus/Kconfig @@ -5,6 +5,9 @@
menu "Bus devices"
+config ANCILLARY_BUS
tristateconfig ARM_CCI bool
diff --git a/drivers/bus/Makefile b/drivers/bus/Makefile index 397e35392bff..7c217eb1dbb7 100644 --- a/drivers/bus/Makefile +++ b/drivers/bus/Makefile @@ -3,6 +3,9 @@ # Makefile for the bus drivers. #
+# Ancillary bus driver +obj-$(CONFIG_ANCILLARY_BUS) += ancillary.o
# Interconnect bus drivers for ARM platforms obj-$(CONFIG_ARM_CCI) += arm-cci.o obj-$(CONFIG_ARM_INTEGRATOR_LM) += arm-integrator-lm.o diff --git a/drivers/bus/ancillary.c b/drivers/bus/ancillary.c new file mode 100644 index 000000000000..93888ca36fb1 --- /dev/null +++ b/drivers/bus/ancillary.c @@ -0,0 +1,225 @@ +// SPDX-License-Identifier: GPL-2.0-only +/*
- Software based bus for Ancillary devices
- Copyright (c) 2019-2020 Intel Corporation
- Please see Documentation/driver-api/ancillary_bus.rst for more information.
- */
+#define pr_fmt(fmt) "%s:%s: " fmt, KBUILD_MODNAME, __func__
+#include <linux/device.h> +#include <linux/init.h> +#include <linux/module.h> +#include <linux/pm_domain.h> +#include <linux/pm_runtime.h> +#include <linux/string.h> +#include <linux/ancillary_bus.h>
+static const struct ancillary_device_id *ancillary_match_id(const struct ancillary_device_id *id,
const struct ancillary_device *ancildev)+{
- while (id->name[0]) {
const char *p = strrchr(dev_name(&ancildev->dev), '.');int match_size;if (!p) {id++;continue;}match_size = p - dev_name(&ancildev->dev);/* use dev_name(&ancildev->dev) prefix before last '.' char to match to */if (!strncmp(dev_name(&ancildev->dev), id->name, match_size))return id;id++;- }
- return NULL;
+}
+static int ancillary_match(struct device *dev, struct device_driver *drv) +{
- struct ancillary_device *ancildev = to_ancillary_dev(dev);
- struct ancillary_driver *ancildrv = to_ancillary_drv(drv);
- return !!ancillary_match_id(ancildrv->id_table, ancildev);
+}
+static int ancillary_uevent(struct device *dev, struct kobj_uevent_env *env) +{
- const char *name, *p;
- name = dev_name(dev);
- p = strrchr(name, '.');
- return add_uevent_var(env, "MODALIAS=%s%.*s", ANCILLARY_MODULE_PREFIX, (int)(p - name),
name);+}
+static const struct dev_pm_ops ancillary_dev_pm_ops = {
- SET_RUNTIME_PM_OPS(pm_generic_runtime_suspend, pm_generic_runtime_resume, NULL)
- SET_SYSTEM_SLEEP_PM_OPS(pm_generic_suspend, pm_generic_resume)
+};
+struct bus_type ancillary_bus_type = {
- .name = "ancillary",
- .match = ancillary_match,
- .uevent = ancillary_uevent,
- .pm = &ancillary_dev_pm_ops,
+};
+/**
- ancillary_device_initialize - check ancillary_device and initialize
- @ancildev: ancillary device struct
- This is the first step in the two-step process to register an ancillary_device.
- When this function returns an error code, then the device_initialize will *not* have
- been performed, and the caller will be responsible to free any memory allocated for the
- ancillary_device in the error path directly.
- It returns 0 on success. On success, the device_initialize has been performed.
- After this point any error unwinding will need to include a call to put_device().
- In this post-initialize error scenario, a call to the device's .release callback will be
- triggered by put_device(), and all memory clean-up is expected to be handled there.
- */
+int ancillary_device_initialize(struct ancillary_device *ancildev) +{
- struct device *dev = &ancildev->dev;
- dev->bus = &ancillary_bus_type;
- if (!dev->parent) {
pr_err("ancillary_device has a NULL dev->parent\n");return -EINVAL;- }
- if (!ancildev->name) {
pr_err("acillary_device has a NULL name\n");return -EINVAL;- }
- if (!(dev->type && dev->type->release) && !dev->release) {
pr_err("ancillary_device does not have a release callback defined\n");return -EINVAL;- }
- device_initialize(&ancildev->dev);
- return 0;
+} +EXPORT_SYMBOL_GPL(ancillary_device_initialize);
+/**
- __ancillary_device_add - add an ancillary bus device
- @ancildev: ancillary bus device to add to the bus
- @modname: name of the parent device's driver module
- This is the second step in the two-step process to register an ancillary_device.
- This function must be called after a successful call to ancillary_device_initialize(), which
- will perform the device_initialize. This means that if this returns an error code, then a
- put_device must be performed so that the .release callback will be triggered to free the
- memory associated with the ancillary_device.
- */
+int __ancillary_device_add(struct ancillary_device *ancildev, const char *modname) +{
- struct device *dev = &ancildev->dev;
- int ret;
- if (!modname) {
pr_err("ancillary device modname is NULL\n");return -EINVAL;- }
- ret = dev_set_name(dev, "%s.%s.%d", modname, ancildev->name, ancildev->id);
- if (ret) {
pr_err("ancillary device dev_set_name failed: %d\n", ret);return ret;- }
- ret = device_add(dev);
- if (ret)
dev_err(dev, "adding ancillary device failed!: %d\n", ret);- return ret;
+}
Sorry, but this is very strange API that requires users to put internal call to "dev" that is buried inside "struct ancillary_device".
For example in your next patch, you write this "put_device(&cdev->ancildev.dev);"
I'm pretty sure that the amount of bugs in error unwind will be astonishing, so if you are doing wrappers over core code, better do not pass complexity to the users.
+EXPORT_SYMBOL_GPL(__ancillary_device_add);
+static int ancillary_probe_driver(struct device *dev) +{
- struct ancillary_driver *ancildrv = to_ancillary_drv(dev->driver);
- struct ancillary_device *ancildev = to_ancillary_dev(dev);
- int ret;
- ret = dev_pm_domain_attach(dev, true);
- if (ret) {
dev_warn(dev, "Failed to attach to PM Domain : %d\n", ret);return ret;- }
- ret = ancildrv->probe(ancildev, ancillary_match_id(ancildrv->id_table, ancildev));
I don't think that you need to call ->probe() if ancillary_match_id() returned NULL and probably that check should be done before dev_pm_domain_attach().
- if (ret)
dev_pm_domain_detach(dev, true);- return ret;
+}
+static int ancillary_remove_driver(struct device *dev) +{
- struct ancillary_driver *ancildrv = to_ancillary_drv(dev->driver);
- struct ancillary_device *ancildev = to_ancillary_dev(dev);
- int ret;
- ret = ancildrv->remove(ancildev);
- dev_pm_domain_detach(dev, true);
- return ret;
You returned an error to user and detached from PM, what will user do with this information? Should user ignore it? retry?
+}
+static void ancillary_shutdown_driver(struct device *dev) +{
- struct ancillary_driver *ancildrv = to_ancillary_drv(dev->driver);
- struct ancillary_device *ancildev = to_ancillary_dev(dev);
- ancildrv->shutdown(ancildev);
+}
+/**
- __ancillary_driver_register - register a driver for ancillary bus devices
- @ancildrv: ancillary_driver structure
- @owner: owning module/driver
- */
+int __ancillary_driver_register(struct ancillary_driver *ancildrv, struct module *owner) +{
- if (WARN_ON(!ancildrv->probe) || WARN_ON(!ancildrv->remove) ||
WARN_ON(!ancildrv->shutdown) || WARN_ON(!ancildrv->id_table))return -EINVAL;- ancildrv->driver.owner = owner;
- ancildrv->driver.bus = &ancillary_bus_type;
- ancildrv->driver.probe = ancillary_probe_driver;
- ancildrv->driver.remove = ancillary_remove_driver;
- ancildrv->driver.shutdown = ancillary_shutdown_driver;
- return driver_register(&ancildrv->driver);
+} +EXPORT_SYMBOL_GPL(__ancillary_driver_register);
+static int __init ancillary_bus_init(void) +{
- return bus_register(&ancillary_bus_type);
+}
+static void __exit ancillary_bus_exit(void) +{
- bus_unregister(&ancillary_bus_type);
+}
+module_init(ancillary_bus_init); +module_exit(ancillary_bus_exit);
+MODULE_LICENSE("GPL v2"); +MODULE_DESCRIPTION("Ancillary Bus"); +MODULE_AUTHOR("David Ertman david.m.ertman@intel.com"); +MODULE_AUTHOR("Kiran Patil kiran.patil@intel.com"); diff --git a/include/linux/ancillary_bus.h b/include/linux/ancillary_bus.h new file mode 100644 index 000000000000..72169c8a5dfe --- /dev/null +++ b/include/linux/ancillary_bus.h @@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/*
- Copyright (c) 2019-2020 Intel Corporation
- Please see Documentation/driver-api/ancillary_bus.rst for more information.
- */
+#ifndef _ANCILLARY_BUS_H_ +#define _ANCILLARY_BUS_H_
+#include <linux/device.h> +#include <linux/mod_devicetable.h> +#include <linux/slab.h>
+struct ancillary_device {
- struct device dev;
- const char *name;
- u32 id;
+};
+struct ancillary_driver {
- int (*probe)(struct ancillary_device *ancildev, const struct ancillary_device_id *id);
- int (*remove)(struct ancillary_device *ancildev);
- void (*shutdown)(struct ancillary_device *ancildev);
- int (*suspend)(struct ancillary_device *ancildev, pm_message_t state);
- int (*resume)(struct ancillary_device *ancildev);
- struct device_driver driver;
- const struct ancillary_device_id *id_table;
+};
+static inline struct ancillary_device *to_ancillary_dev(struct device *dev) +{
- return container_of(dev, struct ancillary_device, dev);
+}
+static inline struct ancillary_driver *to_ancillary_drv(struct device_driver *drv) +{
- return container_of(drv, struct ancillary_driver, driver);
+}
+int ancillary_device_initialize(struct ancillary_device *ancildev); +int __ancillary_device_add(struct ancillary_device *ancildev, const char *modname); +#define ancillary_device_add(ancildev) __ancillary_device_add(ancildev, KBUILD_MODNAME)
+static inline void ancillary_device_unregister(struct ancillary_device *ancildev) +{
- device_unregister(&ancildev->dev);
+}
+int __ancillary_driver_register(struct ancillary_driver *ancildrv, struct module *owner); +#define ancillary_driver_register(ancildrv) __ancillary_driver_register(ancildrv, THIS_MODULE)
+static inline void ancillary_driver_unregister(struct ancillary_driver *ancildrv) +{
- driver_unregister(&ancildrv->driver);
+}
+/**
- module_ancillary_driver() - Helper macro for registering an ancillary driver
- @__ancillary_driver: ancillary driver struct
- Helper macro for ancillary drivers which do not do anything special in
- module init/exit. This eliminates a lot of boilerplate. Each module may only
- use this macro once, and calling it replaces module_init() and module_exit()
- */
+#define module_ancillary_driver(__ancillary_driver) \
- module_driver(__ancillary_driver, ancillary_driver_register, ancillary_driver_unregister)
+#endif /* _ANCILLARY_BUS_H_ */ diff --git a/include/linux/mod_devicetable.h b/include/linux/mod_devicetable.h index 5b08a473cdba..7d596dc30833 100644 --- a/include/linux/mod_devicetable.h +++ b/include/linux/mod_devicetable.h @@ -838,4 +838,12 @@ struct mhi_device_id { kernel_ulong_t driver_data; };
+#define ANCILLARY_NAME_SIZE 32 +#define ANCILLARY_MODULE_PREFIX "ancillary:"
+struct ancillary_device_id {
- char name[ANCILLARY_NAME_SIZE];
I hope that this be enough.
- kernel_ulong_t driver_data;
+};
#endif /* LINUX_MOD_DEVICETABLE_H */ diff --git a/scripts/mod/devicetable-offsets.c b/scripts/mod/devicetable-offsets.c index 27007c18e754..79e37c4c25b3 100644 --- a/scripts/mod/devicetable-offsets.c +++ b/scripts/mod/devicetable-offsets.c @@ -243,5 +243,8 @@ int main(void) DEVID(mhi_device_id); DEVID_FIELD(mhi_device_id, chan);
- DEVID(ancillary_device_id);
- DEVID_FIELD(ancillary_device_id, name);
- return 0;
} diff --git a/scripts/mod/file2alias.c b/scripts/mod/file2alias.c index 2417dd1dee33..99c4fcd82bf3 100644 --- a/scripts/mod/file2alias.c +++ b/scripts/mod/file2alias.c @@ -1364,6 +1364,13 @@ static int do_mhi_entry(const char *filename, void *symval, char *alias) { DEF_FIELD_ADDR(symval, mhi_device_id, chan); sprintf(alias, MHI_DEVICE_MODALIAS_FMT, *chan);
- return 1;
+}
+static int do_ancillary_entry(const char *filename, void *symval, char *alias) +{
DEF_FIELD_ADDR(symval, ancillary_device_id, name);
sprintf(alias, ANCILLARY_MODULE_PREFIX "%s", *name);
return 1;
} @@ -1442,6 +1449,7 @@ static const struct devtable devtable[] = { {"tee", SIZE_tee_client_device_id, do_tee_entry}, {"wmi", SIZE_wmi_device_id, do_wmi_entry}, {"mhi", SIZE_mhi_device_id, do_mhi_entry},
- {"ancillary", SIZE_ancillary_device_id, do_ancillary_entry},
};
/* Create MODULE_ALIAS() statements.
2.26.2
Thanks for the review Leon.
Add support for the Ancillary Bus, ancillary_device and ancillary_driver. It enables drivers to create an ancillary_device and bind an ancillary_driver to it.
I was under impression that this name is going to be changed.
It's part of the opens stated in the cover letter.
[...]
- const struct my_driver my_drv = {
.ancillary_drv = {.driver = {.name = "myancillarydrv",Why do we need to give control over driver name to the driver authors? It can be problematic if author puts name that already exists.
Good point. When I used the ancillary_devices for my own SoundWire test, the driver name didn't seem specifically meaningful but needed to be set to something, what mattered was the id_table. Just thinking aloud, maybe we can add prefixing with KMOD_BUILD, as we've done already to avoid collisions between device names?
[...]
+int __ancillary_device_add(struct ancillary_device *ancildev, const char *modname) +{
- struct device *dev = &ancildev->dev;
- int ret;
- if (!modname) {
pr_err("ancillary device modname is NULL\n");return -EINVAL;- }
- ret = dev_set_name(dev, "%s.%s.%d", modname, ancildev->name, ancildev->id);
- if (ret) {
pr_err("ancillary device dev_set_name failed: %d\n", ret);return ret;- }
- ret = device_add(dev);
- if (ret)
dev_err(dev, "adding ancillary device failed!: %d\n", ret);- return ret;
+}
Sorry, but this is very strange API that requires users to put internal call to "dev" that is buried inside "struct ancillary_device".
For example in your next patch, you write this "put_device(&cdev->ancildev.dev);"
I'm pretty sure that the amount of bugs in error unwind will be astonishing, so if you are doing wrappers over core code, better do not pass complexity to the users.
In initial reviews, there was pushback on adding wrappers that don't do anything except for a pointer indirection.
Others had concerns that the API wasn't balanced and blurring layers.
Both points have merits IMHO. Do we want wrappers for everything and completely hide the low-level device?
+EXPORT_SYMBOL_GPL(__ancillary_device_add);
+static int ancillary_probe_driver(struct device *dev) +{
- struct ancillary_driver *ancildrv = to_ancillary_drv(dev->driver);
- struct ancillary_device *ancildev = to_ancillary_dev(dev);
- int ret;
- ret = dev_pm_domain_attach(dev, true);
- if (ret) {
dev_warn(dev, "Failed to attach to PM Domain : %d\n", ret);return ret;- }
- ret = ancildrv->probe(ancildev, ancillary_match_id(ancildrv->id_table, ancildev));
I don't think that you need to call ->probe() if ancillary_match_id() returned NULL and probably that check should be done before dev_pm_domain_attach().
we'll look into this.
- if (ret)
dev_pm_domain_detach(dev, true);- return ret;
+}
+static int ancillary_remove_driver(struct device *dev) +{
- struct ancillary_driver *ancildrv = to_ancillary_drv(dev->driver);
- struct ancillary_device *ancildev = to_ancillary_dev(dev);
- int ret;
- ret = ancildrv->remove(ancildev);
- dev_pm_domain_detach(dev, true);
- return ret;
You returned an error to user and detached from PM, what will user do with this information? Should user ignore it? retry?
That comment was also provided in earlier reviews. In practice the error is typically ignored so there was a suggestion to move the return type to void, that could be done if this was desired by the majority.
[...]
diff --git a/include/linux/mod_devicetable.h b/include/linux/mod_devicetable.h index 5b08a473cdba..7d596dc30833 100644 --- a/include/linux/mod_devicetable.h +++ b/include/linux/mod_devicetable.h @@ -838,4 +838,12 @@ struct mhi_device_id { kernel_ulong_t driver_data; };
+#define ANCILLARY_NAME_SIZE 32 +#define ANCILLARY_MODULE_PREFIX "ancillary:"
+struct ancillary_device_id {
- char name[ANCILLARY_NAME_SIZE];
I hope that this be enough.
Are you suggesting a different value to allow for a longer string?
On Tue, Oct 06, 2020 at 10:18:07AM -0500, Pierre-Louis Bossart wrote:
Thanks for the review Leon.
Add support for the Ancillary Bus, ancillary_device and ancillary_driver. It enables drivers to create an ancillary_device and bind an ancillary_driver to it.
I was under impression that this name is going to be changed.
It's part of the opens stated in the cover letter.
ok, so what are the variants? system bus (sysbus), sbsystem bus (subbus), crossbus ?
[...]
- const struct my_driver my_drv = {
.ancillary_drv = {.driver = {.name = "myancillarydrv",Why do we need to give control over driver name to the driver authors? It can be problematic if author puts name that already exists.
Good point. When I used the ancillary_devices for my own SoundWire test, the driver name didn't seem specifically meaningful but needed to be set to something, what mattered was the id_table. Just thinking aloud, maybe we can add prefixing with KMOD_BUILD, as we've done already to avoid collisions between device names?
IMHO, it shouldn't be controlled by the drivers at all and need to have kernel module name hardwired. Users will use it later for various bind/unbind/autoprobe tricks and it will give predictability for them.
[...]
+int __ancillary_device_add(struct ancillary_device *ancildev, const char *modname) +{
- struct device *dev = &ancildev->dev;
- int ret;
- if (!modname) {
pr_err("ancillary device modname is NULL\n");return -EINVAL;- }
- ret = dev_set_name(dev, "%s.%s.%d", modname, ancildev->name, ancildev->id);
- if (ret) {
pr_err("ancillary device dev_set_name failed: %d\n", ret);return ret;- }
- ret = device_add(dev);
- if (ret)
dev_err(dev, "adding ancillary device failed!: %d\n", ret);- return ret;
+}
Sorry, but this is very strange API that requires users to put internal call to "dev" that is buried inside "struct ancillary_device".
For example in your next patch, you write this "put_device(&cdev->ancildev.dev);"
I'm pretty sure that the amount of bugs in error unwind will be astonishing, so if you are doing wrappers over core code, better do not pass complexity to the users.
In initial reviews, there was pushback on adding wrappers that don't do anything except for a pointer indirection.
Others had concerns that the API wasn't balanced and blurring layers.
Are you talking about internal review or public? If it is public, can I get a link to it?
Both points have merits IMHO. Do we want wrappers for everything and completely hide the low-level device?
This API is partially obscures low level driver-core code and needs to provide clear and proper abstractions without need to remember about put_device. There is already _add() interface why don't you do put_device() in it?
+EXPORT_SYMBOL_GPL(__ancillary_device_add);
+static int ancillary_probe_driver(struct device *dev) +{
- struct ancillary_driver *ancildrv = to_ancillary_drv(dev->driver);
- struct ancillary_device *ancildev = to_ancillary_dev(dev);
- int ret;
- ret = dev_pm_domain_attach(dev, true);
- if (ret) {
dev_warn(dev, "Failed to attach to PM Domain : %d\n", ret);return ret;- }
- ret = ancildrv->probe(ancildev, ancillary_match_id(ancildrv->id_table, ancildev));
I don't think that you need to call ->probe() if ancillary_match_id() returned NULL and probably that check should be done before dev_pm_domain_attach().
we'll look into this.
- if (ret)
dev_pm_domain_detach(dev, true);- return ret;
+}
+static int ancillary_remove_driver(struct device *dev) +{
- struct ancillary_driver *ancildrv = to_ancillary_drv(dev->driver);
- struct ancillary_device *ancildev = to_ancillary_dev(dev);
- int ret;
- ret = ancildrv->remove(ancildev);
- dev_pm_domain_detach(dev, true);
- return ret;
You returned an error to user and detached from PM, what will user do with this information? Should user ignore it? retry?
That comment was also provided in earlier reviews. In practice the error is typically ignored so there was a suggestion to move the return type to void, that could be done if this was desired by the majority.
+1 from me.
[...]
diff --git a/include/linux/mod_devicetable.h b/include/linux/mod_devicetable.h index 5b08a473cdba..7d596dc30833 100644 --- a/include/linux/mod_devicetable.h +++ b/include/linux/mod_devicetable.h @@ -838,4 +838,12 @@ struct mhi_device_id { kernel_ulong_t driver_data; };
+#define ANCILLARY_NAME_SIZE 32 +#define ANCILLARY_MODULE_PREFIX "ancillary:"
+struct ancillary_device_id {
- char name[ANCILLARY_NAME_SIZE];
I hope that this be enough.
Are you suggesting a different value to allow for a longer string?
I have no idea, but worried that there were no checks at all if name is more than 32. Maybe compiler warn about it?
Thanks
From: Leon Romanovsky leon@kernel.org Sent: Tuesday, October 6, 2020 10:33 PM
On Tue, Oct 06, 2020 at 10:18:07AM -0500, Pierre-Louis Bossart wrote:
Thanks for the review Leon.
Add support for the Ancillary Bus, ancillary_device and ancillary_driver. It enables drivers to create an ancillary_device and bind an ancillary_driver to it.
I was under impression that this name is going to be changed.
It's part of the opens stated in the cover letter.
ok, so what are the variants? system bus (sysbus), sbsystem bus (subbus), crossbus ?
Since the intended use of this bus is to (a) create sub devices that represent 'functional separation' and (b) second use case for subfunctions from a pci device,
I proposed below names in v1 of this patchset.
(a) subdev_bus (b) subfunction_bus
On Tue, Oct 06, 2020 at 05:09:09PM +0000, Parav Pandit wrote:
From: Leon Romanovsky leon@kernel.org Sent: Tuesday, October 6, 2020 10:33 PM
On Tue, Oct 06, 2020 at 10:18:07AM -0500, Pierre-Louis Bossart wrote:
Thanks for the review Leon.
Add support for the Ancillary Bus, ancillary_device and ancillary_driver. It enables drivers to create an ancillary_device and bind an ancillary_driver to it.
I was under impression that this name is going to be changed.
It's part of the opens stated in the cover letter.
ok, so what are the variants? system bus (sysbus), sbsystem bus (subbus), crossbus ?
Since the intended use of this bus is to (a) create sub devices that represent 'functional separation' and (b) second use case for subfunctions from a pci device,
I proposed below names in v1 of this patchset.
(a) subdev_bus
It sounds good, just can we avoid "_" in the name and call it subdev?
(b) subfunction_bus
Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Tue, Oct 06, 2020 at 05:09:09PM +0000, Parav Pandit wrote:
From: Leon Romanovsky leon@kernel.org Sent: Tuesday, October 6, 2020 10:33 PM
On Tue, Oct 06, 2020 at 10:18:07AM -0500, Pierre-Louis Bossart wrote:
Thanks for the review Leon.
Add support for the Ancillary Bus, ancillary_device and ancillary_driver. It enables drivers to create an ancillary_device and bind an ancillary_driver to it.
I was under impression that this name is going to be changed.
It's part of the opens stated in the cover letter.
ok, so what are the variants? system bus (sysbus), sbsystem bus (subbus), crossbus ?
Since the intended use of this bus is to (a) create sub devices that represent 'functional separation' and (b) second use case for subfunctions from a pci device,
I proposed below names in v1 of this patchset.
(a) subdev_bus
It sounds good, just can we avoid "_" in the name and call it subdev?
What is wrong with naming the bus 'ancillary bus'? I feel it's a fitting name. An ancillary software bus for ancillary devices carved off a parent device registered on a primary bus.
On Tue, Oct 06, 2020 at 05:41:00PM +0000, Saleem, Shiraz wrote:
Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Tue, Oct 06, 2020 at 05:09:09PM +0000, Parav Pandit wrote:
From: Leon Romanovsky leon@kernel.org Sent: Tuesday, October 6, 2020 10:33 PM
On Tue, Oct 06, 2020 at 10:18:07AM -0500, Pierre-Louis Bossart wrote:
Thanks for the review Leon.
> Add support for the Ancillary Bus, ancillary_device and ancillary_driver. > It enables drivers to create an ancillary_device and bind an > ancillary_driver to it.
I was under impression that this name is going to be changed.
It's part of the opens stated in the cover letter.
ok, so what are the variants? system bus (sysbus), sbsystem bus (subbus), crossbus ?
Since the intended use of this bus is to (a) create sub devices that represent 'functional separation' and (b) second use case for subfunctions from a pci device,
I proposed below names in v1 of this patchset.
(a) subdev_bus
It sounds good, just can we avoid "_" in the name and call it subdev?
What is wrong with naming the bus 'ancillary bus'? I feel it's a fitting name. An ancillary software bus for ancillary devices carved off a parent device registered on a primary bus.
Greg summarized it very well, every internal conversation about this patch with my colleagues (non-english speakers) starts with the question: "What does ancillary mean?" https://lore.kernel.org/alsa-devel/20201001071403.GC31191@kroah.com/
"For non-native english speakers this is going to be rough, given that I as a native english speaker had to go look up the word in a dictionary to fully understand what you are trying to do with that name."
Thanks
On Tue, Oct 6, 2020 at 12:21 PM Leon Romanovsky leon@kernel.org wrote:
On Tue, Oct 06, 2020 at 05:41:00PM +0000, Saleem, Shiraz wrote:
Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Tue, Oct 06, 2020 at 05:09:09PM +0000, Parav Pandit wrote:
From: Leon Romanovsky leon@kernel.org Sent: Tuesday, October 6, 2020 10:33 PM
On Tue, Oct 06, 2020 at 10:18:07AM -0500, Pierre-Louis Bossart wrote:
Thanks for the review Leon.
> > Add support for the Ancillary Bus, ancillary_device and ancillary_driver. > > It enables drivers to create an ancillary_device and bind an > > ancillary_driver to it. > > I was under impression that this name is going to be changed.
It's part of the opens stated in the cover letter.
ok, so what are the variants? system bus (sysbus), sbsystem bus (subbus), crossbus ?
Since the intended use of this bus is to (a) create sub devices that represent 'functional separation' and (b) second use case for subfunctions from a pci device,
I proposed below names in v1 of this patchset.
(a) subdev_bus
It sounds good, just can we avoid "_" in the name and call it subdev?
What is wrong with naming the bus 'ancillary bus'? I feel it's a fitting name. An ancillary software bus for ancillary devices carved off a parent device registered on a primary bus.
Greg summarized it very well, every internal conversation about this patch with my colleagues (non-english speakers) starts with the question: "What does ancillary mean?" https://lore.kernel.org/alsa-devel/20201001071403.GC31191@kroah.com/
"For non-native english speakers this is going to be rough, given that I as a native english speaker had to go look up the word in a dictionary to fully understand what you are trying to do with that name."
I suggested "auxiliary" in another splintered thread on this question. In terms of what the kernel is already using:
$ git grep auxiliary | wc -l 507 $ git grep ancillary | wc -l 153
Empirically, "auxiliary" is more common and closely matches the intended function of these devices relative to their parent device.
Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Tue, Oct 6, 2020 at 12:21 PM Leon Romanovsky leon@kernel.org wrote:
On Tue, Oct 06, 2020 at 05:41:00PM +0000, Saleem, Shiraz wrote:
Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Tue, Oct 06, 2020 at 05:09:09PM +0000, Parav Pandit wrote:
From: Leon Romanovsky leon@kernel.org Sent: Tuesday, October 6, 2020 10:33 PM
On Tue, Oct 06, 2020 at 10:18:07AM -0500, Pierre-Louis Bossart wrote: > Thanks for the review Leon. > > > > Add support for the Ancillary Bus, ancillary_device and
ancillary_driver.
> > > It enables drivers to create an ancillary_device and > > > bind an ancillary_driver to it. > > > > I was under impression that this name is going to be changed. > > It's part of the opens stated in the cover letter.
ok, so what are the variants? system bus (sysbus), sbsystem bus (subbus), crossbus ?
Since the intended use of this bus is to (a) create sub devices that represent 'functional separation' and (b) second use case for subfunctions from a pci device,
I proposed below names in v1 of this patchset.
(a) subdev_bus
It sounds good, just can we avoid "_" in the name and call it subdev?
What is wrong with naming the bus 'ancillary bus'? I feel it's a fitting name. An ancillary software bus for ancillary devices carved off a parent device
registered on a primary bus.
Greg summarized it very well, every internal conversation about this patch with my colleagues (non-english speakers) starts with the question: "What does ancillary mean?" https://lore.kernel.org/alsa-devel/20201001071403.GC31191@kroah.com/
"For non-native english speakers this is going to be rough, given that I as a native english speaker had to go look up the word in a dictionary to fully understand what you are trying to do with that name."
I suggested "auxiliary" in another splintered thread on this question. In terms of what the kernel is already using:
$ git grep auxiliary | wc -l 507 $ git grep ancillary | wc -l 153
Empirically, "auxiliary" is more common and closely matches the intended function of these devices relative to their parent device.
auxiliary bus is a befitting name as well.
On Wed, Oct 07, 2020 at 01:09:55PM +0000, Saleem, Shiraz wrote:
Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Tue, Oct 6, 2020 at 12:21 PM Leon Romanovsky leon@kernel.org wrote:
On Tue, Oct 06, 2020 at 05:41:00PM +0000, Saleem, Shiraz wrote:
Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Tue, Oct 06, 2020 at 05:09:09PM +0000, Parav Pandit wrote:
> From: Leon Romanovsky leon@kernel.org > Sent: Tuesday, October 6, 2020 10:33 PM > > On Tue, Oct 06, 2020 at 10:18:07AM -0500, Pierre-Louis Bossart wrote: > > Thanks for the review Leon. > > > > > > Add support for the Ancillary Bus, ancillary_device and
ancillary_driver.
> > > > It enables drivers to create an ancillary_device and > > > > bind an ancillary_driver to it. > > > > > > I was under impression that this name is going to be changed. > > > > It's part of the opens stated in the cover letter. > > ok, so what are the variants? > system bus (sysbus), sbsystem bus (subbus), crossbus ? Since the intended use of this bus is to (a) create sub devices that represent 'functional separation' and (b) second use case for subfunctions from a pci device,
I proposed below names in v1 of this patchset.
(a) subdev_bus
It sounds good, just can we avoid "_" in the name and call it subdev?
What is wrong with naming the bus 'ancillary bus'? I feel it's a fitting name. An ancillary software bus for ancillary devices carved off a parent device
registered on a primary bus.
Greg summarized it very well, every internal conversation about this patch with my colleagues (non-english speakers) starts with the question: "What does ancillary mean?" https://lore.kernel.org/alsa-devel/20201001071403.GC31191@kroah.com/
"For non-native english speakers this is going to be rough, given that I as a native english speaker had to go look up the word in a dictionary to fully understand what you are trying to do with that name."
I suggested "auxiliary" in another splintered thread on this question. In terms of what the kernel is already using:
$ git grep auxiliary | wc -l 507 $ git grep ancillary | wc -l 153
Empirically, "auxiliary" is more common and closely matches the intended function of these devices relative to their parent device.
auxiliary bus is a befitting name as well.
Let's share all options and decide later. I don't want to find us bikeshedding about it.
Thanks
On Wed, Oct 7, 2020 at 6:37 AM Leon Romanovsky leon@kernel.org wrote:
On Wed, Oct 07, 2020 at 01:09:55PM +0000, Saleem, Shiraz wrote:
Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Tue, Oct 6, 2020 at 12:21 PM Leon Romanovsky leon@kernel.org wrote:
On Tue, Oct 06, 2020 at 05:41:00PM +0000, Saleem, Shiraz wrote:
Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Tue, Oct 06, 2020 at 05:09:09PM +0000, Parav Pandit wrote: > > > From: Leon Romanovsky leon@kernel.org > > Sent: Tuesday, October 6, 2020 10:33 PM > > > > On Tue, Oct 06, 2020 at 10:18:07AM -0500, Pierre-Louis Bossart wrote: > > > Thanks for the review Leon. > > > > > > > > Add support for the Ancillary Bus, ancillary_device and
ancillary_driver.
> > > > > It enables drivers to create an ancillary_device and > > > > > bind an ancillary_driver to it. > > > > > > > > I was under impression that this name is going to be changed. > > > > > > It's part of the opens stated in the cover letter. > > > > ok, so what are the variants? > > system bus (sysbus), sbsystem bus (subbus), crossbus ? > Since the intended use of this bus is to > (a) create sub devices that represent 'functional separation' > and > (b) second use case for subfunctions from a pci device, > > I proposed below names in v1 of this patchset.
> (a) subdev_bus
It sounds good, just can we avoid "_" in the name and call it subdev?
What is wrong with naming the bus 'ancillary bus'? I feel it's a fitting name. An ancillary software bus for ancillary devices carved off a parent device
registered on a primary bus.
Greg summarized it very well, every internal conversation about this patch with my colleagues (non-english speakers) starts with the question: "What does ancillary mean?" https://lore.kernel.org/alsa-devel/20201001071403.GC31191@kroah.com/
"For non-native english speakers this is going to be rough, given that I as a native english speaker had to go look up the word in a dictionary to fully understand what you are trying to do with that name."
I suggested "auxiliary" in another splintered thread on this question. In terms of what the kernel is already using:
$ git grep auxiliary | wc -l 507 $ git grep ancillary | wc -l 153
Empirically, "auxiliary" is more common and closely matches the intended function of these devices relative to their parent device.
auxiliary bus is a befitting name as well.
Let's share all options and decide later. I don't want to find us bikeshedding about it.
Too late we are deep into bikeshedding at this point... it continued over here [1] for a bit, but let's try to bring the discussion back to this thread.
[1]: http://lore.kernel.org/r/10048d4d-038c-c2b7-2ed7-fd4ca87d104a@linux.intel.co...
-----Original Message----- From: Dan Williams dan.j.williams@intel.com Sent: Wednesday, October 7, 2020 11:56 AM To: Leon Romanovsky leon@kernel.org Cc: Saleem, Shiraz shiraz.saleem@intel.com; Parav Pandit parav@nvidia.com; Pierre-Louis Bossart <pierre- louis.bossart@linux.intel.com>; Ertman, David M david.m.ertman@intel.com; alsa-devel@alsa-project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux- rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; Jason Gunthorpe jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Wed, Oct 7, 2020 at 6:37 AM Leon Romanovsky leon@kernel.org wrote:
On Wed, Oct 07, 2020 at 01:09:55PM +0000, Saleem, Shiraz wrote:
Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Tue, Oct 6, 2020 at 12:21 PM Leon Romanovsky leon@kernel.org
wrote:
On Tue, Oct 06, 2020 at 05:41:00PM +0000, Saleem, Shiraz wrote:
> Subject: Re: [PATCH v2 1/6] Add ancillary bus support > > On Tue, Oct 06, 2020 at 05:09:09PM +0000, Parav Pandit wrote: > > > > > From: Leon Romanovsky leon@kernel.org > > > Sent: Tuesday, October 6, 2020 10:33 PM > > > > > > On Tue, Oct 06, 2020 at 10:18:07AM -0500, Pierre-Louis Bossart
wrote:
> > > > Thanks for the review Leon. > > > > > > > > > > Add support for the Ancillary Bus, ancillary_device and
ancillary_driver.
> > > > > > It enables drivers to create an ancillary_device and > > > > > > bind an ancillary_driver to it. > > > > > > > > > > I was under impression that this name is going to be
changed.
> > > > > > > > It's part of the opens stated in the cover letter. > > > > > > ok, so what are the variants? > > > system bus (sysbus), sbsystem bus (subbus), crossbus ? > > Since the intended use of this bus is to > > (a) create sub devices that represent 'functional separation' > > and > > (b) second use case for subfunctions from a pci device, > > > > I proposed below names in v1 of this patchset. > > > (a) subdev_bus > > It sounds good, just can we avoid "_" in the name and call it
subdev?
>
What is wrong with naming the bus 'ancillary bus'? I feel it's a fitting
name.
An ancillary software bus for ancillary devices carved off a parent
device
registered on a primary bus.
Greg summarized it very well, every internal conversation about this patch with my colleagues (non-english speakers) starts with the
question:
"What does ancillary mean?" https://lore.kernel.org/alsa-
devel/20201001071403.GC31191@kroah.com/
"For non-native english speakers this is going to be rough, given that I as a native english speaker had to go look up the word in a dictionary to fully understand what you are trying to do with that name."
I suggested "auxiliary" in another splintered thread on this question. In terms of what the kernel is already using:
$ git grep auxiliary | wc -l 507 $ git grep ancillary | wc -l 153
Empirically, "auxiliary" is more common and closely matches the
intended function
of these devices relative to their parent device.
auxiliary bus is a befitting name as well.
Let's share all options and decide later. I don't want to find us bikeshedding about it.
Too late we are deep into bikeshedding at this point... it continued over here [1] for a bit, but let's try to bring the discussion back to this thread.
fd4ca87d104a@linux.intel.com
Out of all of the suggestions put forward so far that do not have real objections to them ...
I would put my vote behind aux - short, simple, meaningful
-DaveE
On Tue, 2020-10-06 at 20:26 +0300, Leon Romanovsky wrote:
On Tue, Oct 06, 2020 at 05:09:09PM +0000, Parav Pandit wrote:
From: Leon Romanovsky leon@kernel.org Sent: Tuesday, October 6, 2020 10:33 PM
On Tue, Oct 06, 2020 at 10:18:07AM -0500, Pierre-Louis Bossart wrote:
Thanks for the review Leon.
Add support for the Ancillary Bus, ancillary_device and ancillary_driver. It enables drivers to create an ancillary_device and bind an ancillary_driver to it.
I was under impression that this name is going to be changed.
It's part of the opens stated in the cover letter.
ok, so what are the variants? system bus (sysbus), sbsystem bus (subbus), crossbus ?
Since the intended use of this bus is to (a) create sub devices that represent 'functional separation' and (b) second use case for subfunctions from a pci device,
I proposed below names in v1 of this patchset.
(a) subdev_bus
It sounds good, just can we avoid "_" in the name and call it subdev?
(b) subfunction_bus
While we're still discussing names, may I also suggest simply "software bus" instead?
Thanks,Ranjani
Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Tue, Oct 06, 2020 at 10:18:07AM -0500, Pierre-Louis Bossart wrote:
Thanks for the review Leon.
Add support for the Ancillary Bus, ancillary_device and ancillary_driver. It enables drivers to create an ancillary_device and bind an ancillary_driver to it.
I was under impression that this name is going to be changed.
It's part of the opens stated in the cover letter.
ok, so what are the variants? system bus (sysbus), sbsystem bus (subbus), crossbus ?
[...]
- const struct my_driver my_drv = {
.ancillary_drv = {.driver = {.name = "myancillarydrv",Why do we need to give control over driver name to the driver authors? It can be problematic if author puts name that already exists.
Good point. When I used the ancillary_devices for my own SoundWire test, the driver name didn't seem specifically meaningful but needed to be set to something, what mattered was the id_table. Just thinking aloud, maybe we can add prefixing with KMOD_BUILD, as we've done already to avoid collisions between device names?
IMHO, it shouldn't be controlled by the drivers at all and need to have kernel module name hardwired. Users will use it later for various bind/unbind/autoprobe tricks and it will give predictability for them.
+1. This name is not used in the match. Having the bus hardwire the modname sounds like a good idea.
Shiraz
-----Original Message----- From: Leon Romanovsky leon@kernel.org Sent: Tuesday, October 6, 2020 10:03 AM To: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Cc: Ertman, David M david.m.ertman@intel.com; alsa-devel@alsa- project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux- rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; Williams, Dan J dan.j.williams@intel.com; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Tue, Oct 06, 2020 at 10:18:07AM -0500, Pierre-Louis Bossart wrote:
Thanks for the review Leon.
Add support for the Ancillary Bus, ancillary_device and ancillary_driver. It enables drivers to create an ancillary_device and bind an ancillary_driver to it.
I was under impression that this name is going to be changed.
It's part of the opens stated in the cover letter.
ok, so what are the variants? system bus (sysbus), sbsystem bus (subbus), crossbus ?
[...]
- const struct my_driver my_drv = {
.ancillary_drv = {.driver = {.name = "myancillarydrv",Why do we need to give control over driver name to the driver authors? It can be problematic if author puts name that already exists.
Good point. When I used the ancillary_devices for my own SoundWire test,
the
driver name didn't seem specifically meaningful but needed to be set to something, what mattered was the id_table. Just thinking aloud, maybe we
can
add prefixing with KMOD_BUILD, as we've done already to avoid collisions between device names?
IMHO, it shouldn't be controlled by the drivers at all and need to have kernel module name hardwired. Users will use it later for various bind/unbind/autoprobe tricks and it will give predictability for them.
[...]
+int __ancillary_device_add(struct ancillary_device *ancildev, const
char *modname)
+{
- struct device *dev = &ancildev->dev;
- int ret;
- if (!modname) {
pr_err("ancillary device modname is NULL\n");return -EINVAL;- }
- ret = dev_set_name(dev, "%s.%s.%d", modname, ancildev->name,
ancildev->id);
- if (ret) {
pr_err("ancillary device dev_set_name failed: %d\n", ret);return ret;- }
- ret = device_add(dev);
- if (ret)
dev_err(dev, "adding ancillary device failed!: %d\n", ret);- return ret;
+}
Sorry, but this is very strange API that requires users to put internal call to "dev" that is buried inside "struct ancillary_device".
For example in your next patch, you write this "put_device(&cdev-
ancildev.dev);"
I'm pretty sure that the amount of bugs in error unwind will be astonishing, so if you are doing wrappers over core code, better do not pass complexity to the users.
In initial reviews, there was pushback on adding wrappers that don't do anything except for a pointer indirection.
Others had concerns that the API wasn't balanced and blurring layers.
Are you talking about internal review or public? If it is public, can I get a link to it?
Both points have merits IMHO. Do we want wrappers for everything and completely hide the low-level device?
This API is partially obscures low level driver-core code and needs to provide clear and proper abstractions without need to remember about put_device. There is already _add() interface why don't you do put_device() in it?
The pushback Pierre is referring to was during our mid-tier internal review. It was primarily a concern of Parav as I recall, so he can speak to his reasoning.
What we originally had was a single API call (ancillary_device_register) that started with a call to device_initialize(), and every error path out of the function performed a put_device().
Is this the model you have in mind?
-DaveE
+EXPORT_SYMBOL_GPL(__ancillary_device_add);
+static int ancillary_probe_driver(struct device *dev) +{
- struct ancillary_driver *ancildrv = to_ancillary_drv(dev->driver);
- struct ancillary_device *ancildev = to_ancillary_dev(dev);
- int ret;
- ret = dev_pm_domain_attach(dev, true);
- if (ret) {
dev_warn(dev, "Failed to attach to PM Domain : %d\n", ret);return ret;- }
- ret = ancildrv->probe(ancildev, ancillary_match_id(ancildrv-
id_table, ancildev));
I don't think that you need to call ->probe() if ancillary_match_id() returned NULL and probably that check should be done before dev_pm_domain_attach().
we'll look into this.
- if (ret)
dev_pm_domain_detach(dev, true);- return ret;
+}
+static int ancillary_remove_driver(struct device *dev) +{
- struct ancillary_driver *ancildrv = to_ancillary_drv(dev->driver);
- struct ancillary_device *ancildev = to_ancillary_dev(dev);
- int ret;
- ret = ancildrv->remove(ancildev);
- dev_pm_domain_detach(dev, true);
- return ret;
You returned an error to user and detached from PM, what will user do with this information? Should user ignore it? retry?
That comment was also provided in earlier reviews. In practice the error is typically ignored so there was a suggestion to move the return type to void, that could be done if this was desired by the majority.
+1 from me.
[...]
diff --git a/include/linux/mod_devicetable.h
b/include/linux/mod_devicetable.h
index 5b08a473cdba..7d596dc30833 100644 --- a/include/linux/mod_devicetable.h +++ b/include/linux/mod_devicetable.h @@ -838,4 +838,12 @@ struct mhi_device_id { kernel_ulong_t driver_data; };
+#define ANCILLARY_NAME_SIZE 32 +#define ANCILLARY_MODULE_PREFIX "ancillary:"
+struct ancillary_device_id {
- char name[ANCILLARY_NAME_SIZE];
I hope that this be enough.
Are you suggesting a different value to allow for a longer string?
I have no idea, but worried that there were no checks at all if name is more than 32. Maybe compiler warn about it?
Thanks
On Wed, Oct 07, 2020 at 06:06:30PM +0000, Ertman, David M wrote:
-----Original Message----- From: Leon Romanovsky leon@kernel.org Sent: Tuesday, October 6, 2020 10:03 AM To: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Cc: Ertman, David M david.m.ertman@intel.com; alsa-devel@alsa- project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux- rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; Williams, Dan J dan.j.williams@intel.com; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Tue, Oct 06, 2020 at 10:18:07AM -0500, Pierre-Louis Bossart wrote:
Thanks for the review Leon.
Add support for the Ancillary Bus, ancillary_device and ancillary_driver. It enables drivers to create an ancillary_device and bind an ancillary_driver to it.
I was under impression that this name is going to be changed.
It's part of the opens stated in the cover letter.
ok, so what are the variants? system bus (sysbus), sbsystem bus (subbus), crossbus ?
[...]
- const struct my_driver my_drv = {
.ancillary_drv = {.driver = {.name = "myancillarydrv",Why do we need to give control over driver name to the driver authors? It can be problematic if author puts name that already exists.
Good point. When I used the ancillary_devices for my own SoundWire test,
the
driver name didn't seem specifically meaningful but needed to be set to something, what mattered was the id_table. Just thinking aloud, maybe we
can
add prefixing with KMOD_BUILD, as we've done already to avoid collisions between device names?
IMHO, it shouldn't be controlled by the drivers at all and need to have kernel module name hardwired. Users will use it later for various bind/unbind/autoprobe tricks and it will give predictability for them.
[...]
+int __ancillary_device_add(struct ancillary_device *ancildev, const
char *modname)
+{
- struct device *dev = &ancildev->dev;
- int ret;
- if (!modname) {
pr_err("ancillary device modname is NULL\n");return -EINVAL;- }
- ret = dev_set_name(dev, "%s.%s.%d", modname, ancildev->name,
ancildev->id);
- if (ret) {
pr_err("ancillary device dev_set_name failed: %d\n", ret);return ret;- }
- ret = device_add(dev);
- if (ret)
dev_err(dev, "adding ancillary device failed!: %d\n", ret);- return ret;
+}
Sorry, but this is very strange API that requires users to put internal call to "dev" that is buried inside "struct ancillary_device".
For example in your next patch, you write this "put_device(&cdev-
ancildev.dev);"
I'm pretty sure that the amount of bugs in error unwind will be astonishing, so if you are doing wrappers over core code, better do not pass complexity to the users.
In initial reviews, there was pushback on adding wrappers that don't do anything except for a pointer indirection.
Others had concerns that the API wasn't balanced and blurring layers.
Are you talking about internal review or public? If it is public, can I get a link to it?
Both points have merits IMHO. Do we want wrappers for everything and completely hide the low-level device?
This API is partially obscures low level driver-core code and needs to provide clear and proper abstractions without need to remember about put_device. There is already _add() interface why don't you do put_device() in it?
The pushback Pierre is referring to was during our mid-tier internal review. It was primarily a concern of Parav as I recall, so he can speak to his reasoning.
What we originally had was a single API call (ancillary_device_register) that started with a call to device_initialize(), and every error path out of the function performed a put_device().
Is this the model you have in mind?
I don't like this flow: ancillary_device_initialize() if (ancillary_ancillary_device_add()) { put_device(....) ancillary_device_unregister() return err; }
And prefer this flow: ancillary_device_initialize() if (ancillary_device_add()) { ancillary_device_unregister() return err; }
In this way, the ancillary users won't need to do non-intuitive put_device();
Thanks
-----Original Message----- From: Alsa-devel alsa-devel-bounces@alsa-project.org On Behalf Of Leon Romanovsky Sent: Wednesday, October 7, 2020 12:26 PM To: Ertman, David M david.m.ertman@intel.com Cc: alsa-devel@alsa-project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com; fred.oh@linux.intel.com; linux-rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; Williams, Dan J dan.j.williams@intel.com; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Wed, Oct 07, 2020 at 06:06:30PM +0000, Ertman, David M wrote:
-----Original Message----- From: Leon Romanovsky leon@kernel.org Sent: Tuesday, October 6, 2020 10:03 AM To: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Cc: Ertman, David M david.m.ertman@intel.com; alsa-devel@alsa- project.org; parav@mellanox.com; tiwai@suse.de;
netdev@vger.kernel.org;
ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux- rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org;
Williams,
Dan J dan.j.williams@intel.com; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Tue, Oct 06, 2020 at 10:18:07AM -0500, Pierre-Louis Bossart wrote:
Thanks for the review Leon.
Add support for the Ancillary Bus, ancillary_device and
ancillary_driver.
It enables drivers to create an ancillary_device and bind an ancillary_driver to it.
I was under impression that this name is going to be changed.
It's part of the opens stated in the cover letter.
ok, so what are the variants? system bus (sysbus), sbsystem bus (subbus), crossbus ?
[...]
- const struct my_driver my_drv = {
.ancillary_drv = {.driver = {.name = "myancillarydrv",Why do we need to give control over driver name to the driver
authors?
It can be problematic if author puts name that already exists.
Good point. When I used the ancillary_devices for my own SoundWire
test,
the
driver name didn't seem specifically meaningful but needed to be set to something, what mattered was the id_table. Just thinking aloud, maybe
we
can
add prefixing with KMOD_BUILD, as we've done already to avoid
collisions
between device names?
IMHO, it shouldn't be controlled by the drivers at all and need to have kernel module name hardwired. Users will use it later for various bind/unbind/autoprobe tricks and it will give predictability for them.
[...]
+int __ancillary_device_add(struct ancillary_device *ancildev, const
char *modname)
+{
- struct device *dev = &ancildev->dev;
- int ret;
- if (!modname) {
pr_err("ancillary device modname is NULL\n");return -EINVAL;- }
- ret = dev_set_name(dev, "%s.%s.%d", modname, ancildev-
name,
ancildev->id);
- if (ret) {
pr_err("ancillary device dev_set_name failed: %d\n",ret);
return ret;- }
- ret = device_add(dev);
- if (ret)
dev_err(dev, "adding ancillary device failed!: %d\n",ret);
- return ret;
+}
Sorry, but this is very strange API that requires users to put internal call to "dev" that is buried inside "struct ancillary_device".
For example in your next patch, you write this "put_device(&cdev-
ancildev.dev);"
I'm pretty sure that the amount of bugs in error unwind will be astonishing, so if you are doing wrappers over core code, better do
not
pass complexity to the users.
In initial reviews, there was pushback on adding wrappers that don't do anything except for a pointer indirection.
Others had concerns that the API wasn't balanced and blurring layers.
Are you talking about internal review or public? If it is public, can I get a link to it?
Both points have merits IMHO. Do we want wrappers for everything
and
completely hide the low-level device?
This API is partially obscures low level driver-core code and needs to provide clear and proper abstractions without need to remember about put_device. There is already _add() interface why don't you do put_device() in it?
The pushback Pierre is referring to was during our mid-tier internal review.
It was
primarily a concern of Parav as I recall, so he can speak to his reasoning.
What we originally had was a single API call (ancillary_device_register) that
started
with a call to device_initialize(), and every error path out of the function
performed
a put_device().
Is this the model you have in mind?
I don't like this flow: ancillary_device_initialize() if (ancillary_ancillary_device_add()) { put_device(....) ancillary_device_unregister() return err; }
And prefer this flow: ancillary_device_initialize() if (ancillary_device_add()) { ancillary_device_unregister() return err; }
In this way, the ancillary users won't need to do non-intuitive put_device();
Isn't there a problem calling device_unregister() if device_add() fails? device_unregister() does a device_del() and if the device_add() failed there is nothing to delete?
-DaveE
Thanks
-----Original Message----- From: Ertman, David M Sent: Wednesday, October 7, 2020 12:54 PM To: 'Leon Romanovsky' leon@kernel.org Cc: alsa-devel@alsa-project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com; fred.oh@linux.intel.com; linux-rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; Williams, Dan J dan.j.williams@intel.com; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: RE: [PATCH v2 1/6] Add ancillary bus support
-----Original Message----- From: Alsa-devel alsa-devel-bounces@alsa-project.org On Behalf Of
Leon
Romanovsky Sent: Wednesday, October 7, 2020 12:26 PM To: Ertman, David M david.m.ertman@intel.com Cc: alsa-devel@alsa-project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com; fred.oh@linux.intel.com; linux-rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; Williams, Dan J dan.j.williams@intel.com; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Wed, Oct 07, 2020 at 06:06:30PM +0000, Ertman, David M wrote:
-----Original Message----- From: Leon Romanovsky leon@kernel.org Sent: Tuesday, October 6, 2020 10:03 AM To: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Cc: Ertman, David M david.m.ertman@intel.com; alsa-devel@alsa- project.org; parav@mellanox.com; tiwai@suse.de;
netdev@vger.kernel.org;
ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux- rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org;
Williams,
Dan J dan.j.williams@intel.com; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Tue, Oct 06, 2020 at 10:18:07AM -0500, Pierre-Louis Bossart wrote:
Thanks for the review Leon.
> Add support for the Ancillary Bus, ancillary_device and
ancillary_driver.
> It enables drivers to create an ancillary_device and bind an > ancillary_driver to it.
I was under impression that this name is going to be changed.
It's part of the opens stated in the cover letter.
ok, so what are the variants? system bus (sysbus), sbsystem bus (subbus), crossbus ?
[...]
> + const struct my_driver my_drv = { > + .ancillary_drv = { > + .driver = { > + .name = "myancillarydrv",
Why do we need to give control over driver name to the driver
authors?
It can be problematic if author puts name that already exists.
Good point. When I used the ancillary_devices for my own
SoundWire
test,
the
driver name didn't seem specifically meaningful but needed to be set
to
something, what mattered was the id_table. Just thinking aloud,
maybe
we
can
add prefixing with KMOD_BUILD, as we've done already to avoid
collisions
between device names?
IMHO, it shouldn't be controlled by the drivers at all and need to have kernel module name hardwired. Users will use it later for various bind/unbind/autoprobe tricks and it will give predictability for them.
[...]
> +int __ancillary_device_add(struct ancillary_device *ancildev,
const
char *modname)
> +{ > + struct device *dev = &ancildev->dev; > + int ret; > + > + if (!modname) { > + pr_err("ancillary device modname is NULL\n"); > + return -EINVAL; > + } > + > + ret = dev_set_name(dev, "%s.%s.%d", modname, ancildev-
name,
ancildev->id);
> + if (ret) { > + pr_err("ancillary device dev_set_name failed: %d\n",
ret);
> + return ret; > + } > + > + ret = device_add(dev); > + if (ret) > + dev_err(dev, "adding ancillary device failed!: %d\n",
ret);
> + > + return ret; > +}
Sorry, but this is very strange API that requires users to put internal call to "dev" that is buried inside "struct ancillary_device".
For example in your next patch, you write this "put_device(&cdev-
ancildev.dev);"
I'm pretty sure that the amount of bugs in error unwind will be astonishing, so if you are doing wrappers over core code, better do
not
pass complexity to the users.
In initial reviews, there was pushback on adding wrappers that don't
do
anything except for a pointer indirection.
Others had concerns that the API wasn't balanced and blurring layers.
Are you talking about internal review or public? If it is public, can I get a link to it?
Both points have merits IMHO. Do we want wrappers for everything
and
completely hide the low-level device?
This API is partially obscures low level driver-core code and needs to provide clear and proper abstractions without need to remember about put_device. There is already _add() interface why don't you do put_device() in it?
The pushback Pierre is referring to was during our mid-tier internal
review.
It was
primarily a concern of Parav as I recall, so he can speak to his reasoning.
What we originally had was a single API call (ancillary_device_register)
that
started
with a call to device_initialize(), and every error path out of the function
performed
a put_device().
Is this the model you have in mind?
I don't like this flow: ancillary_device_initialize() if (ancillary_ancillary_device_add()) { put_device(....) ancillary_device_unregister() return err; }
And prefer this flow: ancillary_device_initialize() if (ancillary_device_add()) { ancillary_device_unregister() return err; }
In this way, the ancillary users won't need to do non-intuitive put_device();
Isn't there a problem calling device_unregister() if device_add() fails? device_unregister() does a device_del() and if the device_add() failed there is nothing to delete?
Sorry, hit send there unintentionally.
So, would it be best to split the unregister API into two calls as well.
ancillary_device_del() ancillary_device_put() ? -DaveE
-DaveE
Thanks
From: Leon Romanovsky leon@kernel.org Sent: Thursday, October 8, 2020 12:56 AM
This API is partially obscures low level driver-core code and needs to provide clear and proper abstractions without need to remember about put_device. There is already _add() interface why don't you do put_device() in it?
The pushback Pierre is referring to was during our mid-tier internal review. It was primarily a concern of Parav as I recall, so he can speak to his
reasoning.
What we originally had was a single API call (ancillary_device_register) that started with a call to device_initialize(), and every error path out of the function performed a
put_device().
Is this the model you have in mind?
I don't like this flow: ancillary_device_initialize() if (ancillary_ancillary_device_add()) { put_device(....) ancillary_device_unregister()
Calling device_unregister() is incorrect, because add() wasn't successful. Only put_device() or a wrapper ancillary_device_put() is necessary.
return err; }
And prefer this flow: ancillary_device_initialize() if (ancillary_device_add()) { ancillary_device_unregister()
This is incorrect and a clear deviation from the current core APIs that adds the confusion.
return err; }
In this way, the ancillary users won't need to do non-intuitive put_device();
Below is most simple, intuitive and matching with core APIs for name and design pattern wise. init() { err = ancillary_device_initialize(); if (err) return ret;
err = ancillary_device_add(); if (ret) goto err_unwind;
err = some_foo(); if (err) goto err_foo; return 0;
err_foo: ancillary_device_del(adev); err_unwind: ancillary_device_put(adev->dev); return err; }
cleanup() { ancillary_device_de(adev); ancillary_device_put(adev); /* It is common to have a one wrapper for this as ancillary_device_unregister(). * This will match with core device_unregister() that has precise documentation. * but given fact that init() code need proper error unwinding, like above, * it make sense to have two APIs, and no need to export another symbol for unregister(). * This pattern is very easy to audit and code. */ }
-----Original Message----- From: Parav Pandit parav@nvidia.com Sent: Wednesday, October 7, 2020 1:17 PM To: Leon Romanovsky leon@kernel.org; Ertman, David M david.m.ertman@intel.com Cc: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com; alsa- devel@alsa-project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux-rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; Jason Gunthorpe jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; Williams, Dan J dan.j.williams@intel.com; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: RE: [PATCH v2 1/6] Add ancillary bus support
From: Leon Romanovsky leon@kernel.org Sent: Thursday, October 8, 2020 12:56 AM
This API is partially obscures low level driver-core code and needs to provide clear and proper abstractions without need to remember about put_device. There is already _add() interface why don't you do put_device() in it?
The pushback Pierre is referring to was during our mid-tier internal review. It was primarily a concern of Parav as I recall, so he can speak to
his
reasoning.
What we originally had was a single API call (ancillary_device_register) that started with a call to device_initialize(), and every error path out of the function performed a
put_device().
Is this the model you have in mind?
I don't like this flow: ancillary_device_initialize() if (ancillary_ancillary_device_add()) { put_device(....) ancillary_device_unregister()
Calling device_unregister() is incorrect, because add() wasn't successful. Only put_device() or a wrapper ancillary_device_put() is necessary.
return err; }
And prefer this flow: ancillary_device_initialize() if (ancillary_device_add()) { ancillary_device_unregister()
This is incorrect and a clear deviation from the current core APIs that adds the confusion.
return err; }
In this way, the ancillary users won't need to do non-intuitive put_device();
Below is most simple, intuitive and matching with core APIs for name and design pattern wise. init() { err = ancillary_device_initialize(); if (err) return ret;
err = ancillary_device_add(); if (ret) goto err_unwind;
err = some_foo(); if (err) goto err_foo; return 0;
err_foo: ancillary_device_del(adev); err_unwind: ancillary_device_put(adev->dev); return err; }
cleanup() { ancillary_device_de(adev); ancillary_device_put(adev); /* It is common to have a one wrapper for this as ancillary_device_unregister(). * This will match with core device_unregister() that has precise documentation. * but given fact that init() code need proper error unwinding, like above, * it make sense to have two APIs, and no need to export another symbol for unregister(). * This pattern is very easy to audit and code. */ }
I like this flow +1
But ... since the init() function is performing both device_init and device_add - it should probably be called ancillary_device_register, and we are back to a single exported API for both register and unregister.
At that point, do we need wrappers on the primitives init, add, del, and put?
-DaveE
Below is most simple, intuitive and matching with core APIs for name and design pattern wise. init() { err = ancillary_device_initialize(); if (err) return ret;
err = ancillary_device_add(); if (ret) goto err_unwind;
err = some_foo(); if (err) goto err_foo; return 0;
err_foo: ancillary_device_del(adev); err_unwind: ancillary_device_put(adev->dev); return err; }
cleanup() { ancillary_device_de(adev); ancillary_device_put(adev); /* It is common to have a one wrapper for this as ancillary_device_unregister(). * This will match with core device_unregister() that has precise documentation. * but given fact that init() code need proper error unwinding, like above, * it make sense to have two APIs, and no need to export another symbol for unregister(). * This pattern is very easy to audit and code. */ }
I like this flow +1
But ... since the init() function is performing both device_init and device_add - it should probably be called ancillary_device_register, and we are back to a single exported API for both register and unregister.
Kind reminder that we introduced the two functions to allow the caller to know if it needed to free memory when initialize() fails, and it didn't need to free memory when add() failed since put_device() takes care of it. If you have a single init() function it's impossible to know which behavior to select on error.
I also have a case with SoundWire where it's nice to first initialize, then set some data and then add.
At that point, do we need wrappers on the primitives init, add, del, and put?
-DaveE
-----Original Message----- From: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Sent: Wednesday, October 7, 2020 1:59 PM To: Ertman, David M david.m.ertman@intel.com; Parav Pandit parav@nvidia.com; Leon Romanovsky leon@kernel.org Cc: alsa-devel@alsa-project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux-rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; Jason Gunthorpe jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; Williams, Dan J dan.j.williams@intel.com; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: Re: [PATCH v2 1/6] Add ancillary bus support
Below is most simple, intuitive and matching with core APIs for name and design pattern wise. init() { err = ancillary_device_initialize(); if (err) return ret;
err = ancillary_device_add(); if (ret) goto err_unwind;
err = some_foo(); if (err) goto err_foo; return 0;
err_foo: ancillary_device_del(adev); err_unwind: ancillary_device_put(adev->dev); return err; }
cleanup() { ancillary_device_de(adev); ancillary_device_put(adev); /* It is common to have a one wrapper for this as ancillary_device_unregister(). * This will match with core device_unregister() that has precise documentation. * but given fact that init() code need proper error unwinding, like above, * it make sense to have two APIs, and no need to export another symbol for unregister(). * This pattern is very easy to audit and code. */ }
I like this flow +1
But ... since the init() function is performing both device_init and device_add - it should probably be called ancillary_device_register, and we are back to a single exported API for both register and unregister.
Kind reminder that we introduced the two functions to allow the caller to know if it needed to free memory when initialize() fails, and it didn't need to free memory when add() failed since put_device() takes care of it. If you have a single init() function it's impossible to know which behavior to select on error.
I also have a case with SoundWire where it's nice to first initialize, then set some data and then add.
The flow as outlined by Parav above does an initialize as the first step, so every error path out of the function has to do a put_device(), so you would never need to manually free the memory in the setup function. It would be freed in the release call.
-DaveE
At that point, do we need wrappers on the primitives init, add, del, and put?
-DaveE
On 10/7/20 4:22 PM, Ertman, David M wrote:
-----Original Message----- From: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Sent: Wednesday, October 7, 2020 1:59 PM To: Ertman, David M david.m.ertman@intel.com; Parav Pandit parav@nvidia.com; Leon Romanovsky leon@kernel.org Cc: alsa-devel@alsa-project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux-rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; Jason Gunthorpe jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; Williams, Dan J dan.j.williams@intel.com; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: Re: [PATCH v2 1/6] Add ancillary bus support
Below is most simple, intuitive and matching with core APIs for name and design pattern wise. init() { err = ancillary_device_initialize(); if (err) return ret;
err = ancillary_device_add(); if (ret) goto err_unwind;
err = some_foo(); if (err) goto err_foo; return 0;
err_foo: ancillary_device_del(adev); err_unwind: ancillary_device_put(adev->dev); return err; }
cleanup() { ancillary_device_de(adev); ancillary_device_put(adev); /* It is common to have a one wrapper for this as ancillary_device_unregister(). * This will match with core device_unregister() that has precise documentation. * but given fact that init() code need proper error unwinding, like above, * it make sense to have two APIs, and no need to export another symbol for unregister(). * This pattern is very easy to audit and code. */ }
I like this flow +1
But ... since the init() function is performing both device_init and device_add - it should probably be called ancillary_device_register, and we are back to a single exported API for both register and unregister.
Kind reminder that we introduced the two functions to allow the caller to know if it needed to free memory when initialize() fails, and it didn't need to free memory when add() failed since put_device() takes care of it. If you have a single init() function it's impossible to know which behavior to select on error.
I also have a case with SoundWire where it's nice to first initialize, then set some data and then add.
The flow as outlined by Parav above does an initialize as the first step, so every error path out of the function has to do a put_device(), so you would never need to manually free the memory in the setup function. It would be freed in the release call.
err = ancillary_device_initialize(); if (err) return ret;
where is the put_device() here? if the release function does any sort of kfree, then you'd need to do it manually in this case.
From: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Sent: Thursday, October 8, 2020 3:20 AM
On 10/7/20 4:22 PM, Ertman, David M wrote:
-----Original Message----- From: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Sent: Wednesday, October 7, 2020 1:59 PM To: Ertman, David M david.m.ertman@intel.com; Parav Pandit parav@nvidia.com; Leon Romanovsky leon@kernel.org Cc: alsa-devel@alsa-project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux-rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; Jason Gunthorpe jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; Williams, Dan J dan.j.williams@intel.com; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: Re: [PATCH v2 1/6] Add ancillary bus support
Below is most simple, intuitive and matching with core APIs for name and design pattern wise. init() { err = ancillary_device_initialize(); if (err) return ret;
err = ancillary_device_add(); if (ret) goto err_unwind;
err = some_foo(); if (err) goto err_foo; return 0;
err_foo: ancillary_device_del(adev); err_unwind: ancillary_device_put(adev->dev); return err; }
cleanup() { ancillary_device_de(adev); ancillary_device_put(adev); /* It is common to have a one wrapper for this as ancillary_device_unregister(). * This will match with core device_unregister() that has precise documentation. * but given fact that init() code need proper error unwinding, like above, * it make sense to have two APIs, and no need to export another symbol for unregister(). * This pattern is very easy to audit and code. */ }
I like this flow +1
But ... since the init() function is performing both device_init and device_add - it should probably be called ancillary_device_register, and we are back to a single exported API for both register and unregister.
Kind reminder that we introduced the two functions to allow the caller to know if it needed to free memory when initialize() fails, and it didn't need to free memory when add() failed since put_device() takes care of it. If you have a single init() function it's impossible to know which behavior to select on error.
I also have a case with SoundWire where it's nice to first initialize, then set some data and then add.
The flow as outlined by Parav above does an initialize as the first step, so every error path out of the function has to do a put_device(), so you would never need to manually free the memory in
the setup function.
It would be freed in the release call.
err = ancillary_device_initialize(); if (err) return ret;
where is the put_device() here? if the release function does any sort of kfree, then you'd need to do it manually in this case.
Since device_initialize() failed, put_device() cannot be done here. So yes, pseudo code should have shown, if (err) { kfree(adev); return err; }
If we just want to follow register(), unregister() pattern,
Than,
ancillar_device_register() should be,
/** * ancillar_device_register() - register an ancillary device * NOTE: __never directly free @adev after calling this function, even if it returned * an error. Always use ancillary_device_put() to give up the reference initialized by this function. * This note matches with the core and caller knows exactly what to be done. */ ancillary_device_register() { device_initialize(&adev->dev); if (!dev->parent || !adev->name) return -EINVAL; if (!dev->release && !(dev->type && dev->type->release)) { /* core is already capable and throws the warning when release callback is not set. * It is done at drivers/base/core.c:1798. * For NULL release it says, "does not have a release() function, it is broken and must be fixed" */ return -EINVAL; } err = dev_set_name(adev...); if (err) { /* kobject_release() -> kobject_cleanup() are capable to detect if name is set/ not set * and free the const if it was set. */ return err; } err = device_add(&adev->dev); If (err) return err; }
Caller code: init() { adev = kzalloc(sizeof(*foo_adev)..); if (!adev) return -ENOMEM; err = ancillary_device_register(&adev); if (err) goto err;
err: ancillary_device_put(&adev); return err; }
cleanup() { ancillary_device_unregister(&adev); }
Above pattern is fine too matching the core.
If I understand Leon correctly, he prefers simple register(), unregister() pattern. If, so it should be explicit register(), unregister() API.
However I read that Pierre mentioned that SoundWire prefers initialize(), some_data_init(), add() pattern. If SoundWire cannot do register() pattern, So, whichever first user bundled with the patchset, those APIs should be exported, because we don’t add an API without a user.
Pierre, Can you please check if SoundWire can follow register() pattern?
Assuming Leon patches and my patches for subfunction arrive after Soundwire series + ancillary bus, we can add the register() and unregister() version in our patchset later.
Greg already said that "it's not carved on stone, we can do incremental additions as the need arise". So I think we should proceed with the wrappers which follow the core convention of either (a) initialize)(), add() or (b) register(), unregister().
On Thu, Oct 08, 2020 at 04:56:01AM +0000, Parav Pandit wrote:
From: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Sent: Thursday, October 8, 2020 3:20 AM
On 10/7/20 4:22 PM, Ertman, David M wrote:
-----Original Message----- From: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Sent: Wednesday, October 7, 2020 1:59 PM To: Ertman, David M david.m.ertman@intel.com; Parav Pandit parav@nvidia.com; Leon Romanovsky leon@kernel.org Cc: alsa-devel@alsa-project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux-rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; Jason Gunthorpe jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; Williams, Dan J dan.j.williams@intel.com; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: Re: [PATCH v2 1/6] Add ancillary bus support
Below is most simple, intuitive and matching with core APIs for name and design pattern wise. init() { err = ancillary_device_initialize(); if (err) return ret;
err = ancillary_device_add(); if (ret) goto err_unwind;
err = some_foo(); if (err) goto err_foo; return 0;
err_foo: ancillary_device_del(adev); err_unwind: ancillary_device_put(adev->dev); return err; }
cleanup() { ancillary_device_de(adev); ancillary_device_put(adev); /* It is common to have a one wrapper for this as ancillary_device_unregister(). * This will match with core device_unregister() that has precise documentation. * but given fact that init() code need proper error unwinding, like above, * it make sense to have two APIs, and no need to export another symbol for unregister(). * This pattern is very easy to audit and code. */ }
I like this flow +1
But ... since the init() function is performing both device_init and device_add - it should probably be called ancillary_device_register, and we are back to a single exported API for both register and unregister.
Kind reminder that we introduced the two functions to allow the caller to know if it needed to free memory when initialize() fails, and it didn't need to free memory when add() failed since put_device() takes care of it. If you have a single init() function it's impossible to know which behavior to select on error.
I also have a case with SoundWire where it's nice to first initialize, then set some data and then add.
The flow as outlined by Parav above does an initialize as the first step, so every error path out of the function has to do a put_device(), so you would never need to manually free the memory in
the setup function.
It would be freed in the release call.
err = ancillary_device_initialize(); if (err) return ret;
where is the put_device() here? if the release function does any sort of kfree, then you'd need to do it manually in this case.
Since device_initialize() failed, put_device() cannot be done here. So yes, pseudo code should have shown, if (err) { kfree(adev); return err; }
If we just want to follow register(), unregister() pattern,
Than,
ancillar_device_register() should be,
/**
- ancillar_device_register() - register an ancillary device
- NOTE: __never directly free @adev after calling this function, even if it returned
- an error. Always use ancillary_device_put() to give up the reference initialized by this function.
- This note matches with the core and caller knows exactly what to be done.
*/ ancillary_device_register() { device_initialize(&adev->dev); if (!dev->parent || !adev->name) return -EINVAL; if (!dev->release && !(dev->type && dev->type->release)) { /* core is already capable and throws the warning when release callback is not set. * It is done at drivers/base/core.c:1798. * For NULL release it says, "does not have a release() function, it is broken and must be fixed" */ return -EINVAL; } err = dev_set_name(adev...); if (err) { /* kobject_release() -> kobject_cleanup() are capable to detect if name is set/ not set * and free the const if it was set. */ return err; } err = device_add(&adev->dev); If (err) return err; }
Caller code: init() { adev = kzalloc(sizeof(*foo_adev)..); if (!adev) return -ENOMEM; err = ancillary_device_register(&adev); if (err) goto err;
err: ancillary_device_put(&adev); return err; }
cleanup() { ancillary_device_unregister(&adev); }
Above pattern is fine too matching the core.
If I understand Leon correctly, he prefers simple register(), unregister() pattern. If, so it should be explicit register(), unregister() API.
This is my summary https://lore.kernel.org/linux-rdma/20201008052137.GA13580@unreal The API should be symmetric.
Thanks
From: Leon Romanovsky leon@kernel.org Sent: Thursday, October 8, 2020 10:56 AM
On Thu, Oct 08, 2020 at 04:56:01AM +0000, Parav Pandit wrote:
From: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Sent: Thursday, October 8, 2020 3:20 AM
On 10/7/20 4:22 PM, Ertman, David M wrote:
-----Original Message----- From: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Sent: Wednesday, October 7, 2020 1:59 PM To: Ertman, David M david.m.ertman@intel.com; Parav Pandit parav@nvidia.com; Leon Romanovsky leon@kernel.org Cc: alsa-devel@alsa-project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux-rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; Jason Gunthorpe jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; Williams, Dan J dan.j.williams@intel.com; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: Re: [PATCH v2 1/6] Add ancillary bus support
> Below is most simple, intuitive and matching with core APIs for > name and design pattern wise. > init() > { > err = ancillary_device_initialize(); > if (err) > return ret; > > err = ancillary_device_add(); > if (ret) > goto err_unwind; > > err = some_foo(); > if (err) > goto err_foo; > return 0; > > err_foo: > ancillary_device_del(adev); > err_unwind: > ancillary_device_put(adev->dev); > return err; > } > > cleanup() > { > ancillary_device_de(adev); > ancillary_device_put(adev); > /* It is common to have a one wrapper for this as > ancillary_device_unregister(). > * This will match with core device_unregister() that has > precise documentation. > * but given fact that init() code need proper error > unwinding, like above, > * it make sense to have two APIs, and no need to export > another symbol for unregister(). > * This pattern is very easy to audit and code. > */ > }
I like this flow +1
But ... since the init() function is performing both device_init and device_add - it should probably be called ancillary_device_register, and we are back to a single exported API for both register and unregister.
Kind reminder that we introduced the two functions to allow the caller to know if it needed to free memory when initialize() fails, and it didn't need to free memory when add() failed since put_device() takes care of it. If you have a single init() function it's impossible to know which behavior to select on error.
I also have a case with SoundWire where it's nice to first initialize, then set some data and then add.
The flow as outlined by Parav above does an initialize as the first step, so every error path out of the function has to do a put_device(), so you would never need to manually free the memory in
the setup function.
It would be freed in the release call.
err = ancillary_device_initialize(); if (err) return ret;
where is the put_device() here? if the release function does any sort of kfree, then you'd need to do it manually in this case.
Since device_initialize() failed, put_device() cannot be done here. So yes, pseudo code should have shown, if (err) { kfree(adev); return err; }
If we just want to follow register(), unregister() pattern,
Than,
ancillar_device_register() should be,
/**
- ancillar_device_register() - register an ancillary device
- NOTE: __never directly free @adev after calling this function, even
if it returned
- an error. Always use ancillary_device_put() to give up the reference
initialized by this function.
- This note matches with the core and caller knows exactly what to be
done.
*/ ancillary_device_register() { device_initialize(&adev->dev); if (!dev->parent || !adev->name) return -EINVAL; if (!dev->release && !(dev->type && dev->type->release)) { /* core is already capable and throws the warning when
release callback is not set.
* It is done at drivers/base/core.c:1798. * For NULL release it says, "does not have a release()function, it is broken and must be fixed"
*/ return -EINVAL;} err = dev_set_name(adev...); if (err) { /* kobject_release() -> kobject_cleanup() are capable to
detect if name is set/ not set
* and free the const if it was set. */ return err;} err = device_add(&adev->dev); If (err) return err; }
Caller code: init() { adev = kzalloc(sizeof(*foo_adev)..); if (!adev) return -ENOMEM; err = ancillary_device_register(&adev); if (err) goto err;
err: ancillary_device_put(&adev); return err; }
cleanup() { ancillary_device_unregister(&adev); }
Above pattern is fine too matching the core.
If I understand Leon correctly, he prefers simple register(), unregister()
pattern.
If, so it should be explicit register(), unregister() API.
This is my summary https://lore.kernel.org/linux-rdma/20201008052137.GA13580@unreal The API should be symmetric.
I disagree to your below point.
- You are not providing driver/core API but simplification and obfuscation
of basic primitives and structures. This is new layer. There is no room for a claim that we must to follow internal API.
If ancillary bus has ancillary_device_add(), it cannot do device_initialize() and device_add() in both.
I provided two examples and what really matters is a given patchset uses (need to use) which pattern, initialize() + add(), or register() + unregister().
As we all know that API is not added for future. It is the future patch extends it. So lets wait for Pierre to reply if soundwire can follow register(), unregister() sequence. This way same APIs can service both use-cases.
Regarding,
- You can't "ask" from users to call internal calls (put_device) over internal
fields in ancillary_device.
In that case if should be ancillary_device_put() ancillary_device_release().
Or we should follow the patten of ib_alloc_device [1], ancillary_device_alloc() -> kzalloc(adev + dev) with compile time assert check like rdma and vdpa subsystem. ->device_initialize() ancillary_device_add()
ancillar_device_de() <- balances with add ancillary_device_dealloc() <-- balances with device_alloc(), which does the put_device() + free the memory allocated in alloc().
This approach of [1] also eliminates exposing adev.dev.release = <drivers_release_method_to_free_adev> in drivers. And container_of() benefit also continues..
[1] https://elixir.bootlin.com/linux/v5.9-rc8/source/include/rdma/ib_verbs.h#L27...
On Thu, Oct 08, 2020 at 07:14:17AM +0000, Parav Pandit wrote:
From: Leon Romanovsky leon@kernel.org Sent: Thursday, October 8, 2020 10:56 AM
On Thu, Oct 08, 2020 at 04:56:01AM +0000, Parav Pandit wrote:
From: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Sent: Thursday, October 8, 2020 3:20 AM
On 10/7/20 4:22 PM, Ertman, David M wrote:
-----Original Message----- From: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Sent: Wednesday, October 7, 2020 1:59 PM To: Ertman, David M david.m.ertman@intel.com; Parav Pandit parav@nvidia.com; Leon Romanovsky leon@kernel.org Cc: alsa-devel@alsa-project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux-rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; Jason Gunthorpe jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; Williams, Dan J dan.j.williams@intel.com; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: Re: [PATCH v2 1/6] Add ancillary bus support
>> Below is most simple, intuitive and matching with core APIs for >> name and design pattern wise. >> init() >> { >> err = ancillary_device_initialize(); >> if (err) >> return ret; >> >> err = ancillary_device_add(); >> if (ret) >> goto err_unwind; >> >> err = some_foo(); >> if (err) >> goto err_foo; >> return 0; >> >> err_foo: >> ancillary_device_del(adev); >> err_unwind: >> ancillary_device_put(adev->dev); >> return err; >> } >> >> cleanup() >> { >> ancillary_device_de(adev); >> ancillary_device_put(adev); >> /* It is common to have a one wrapper for this as >> ancillary_device_unregister(). >> * This will match with core device_unregister() that has >> precise documentation. >> * but given fact that init() code need proper error >> unwinding, like above, >> * it make sense to have two APIs, and no need to export >> another symbol for unregister(). >> * This pattern is very easy to audit and code. >> */ >> } > > I like this flow +1 > > But ... since the init() function is performing both device_init > and device_add - it should probably be called > ancillary_device_register, and we are back to a single exported > API for both register and unregister.
Kind reminder that we introduced the two functions to allow the caller to know if it needed to free memory when initialize() fails, and it didn't need to free memory when add() failed since put_device() takes care of it. If you have a single init() function it's impossible to know which behavior to select on error.
I also have a case with SoundWire where it's nice to first initialize, then set some data and then add.
The flow as outlined by Parav above does an initialize as the first step, so every error path out of the function has to do a put_device(), so you would never need to manually free the memory in
the setup function.
It would be freed in the release call.
err = ancillary_device_initialize(); if (err) return ret;
where is the put_device() here? if the release function does any sort of kfree, then you'd need to do it manually in this case.
Since device_initialize() failed, put_device() cannot be done here. So yes, pseudo code should have shown, if (err) { kfree(adev); return err; }
If we just want to follow register(), unregister() pattern,
Than,
ancillar_device_register() should be,
/**
- ancillar_device_register() - register an ancillary device
- NOTE: __never directly free @adev after calling this function, even
if it returned
- an error. Always use ancillary_device_put() to give up the reference
initialized by this function.
- This note matches with the core and caller knows exactly what to be
done.
*/ ancillary_device_register() { device_initialize(&adev->dev); if (!dev->parent || !adev->name) return -EINVAL; if (!dev->release && !(dev->type && dev->type->release)) { /* core is already capable and throws the warning when
release callback is not set.
* It is done at drivers/base/core.c:1798. * For NULL release it says, "does not have a release()function, it is broken and must be fixed"
*/ return -EINVAL;} err = dev_set_name(adev...); if (err) { /* kobject_release() -> kobject_cleanup() are capable to
detect if name is set/ not set
* and free the const if it was set. */ return err;} err = device_add(&adev->dev); If (err) return err; }
Caller code: init() { adev = kzalloc(sizeof(*foo_adev)..); if (!adev) return -ENOMEM; err = ancillary_device_register(&adev); if (err) goto err;
err: ancillary_device_put(&adev); return err; }
cleanup() { ancillary_device_unregister(&adev); }
Above pattern is fine too matching the core.
If I understand Leon correctly, he prefers simple register(), unregister()
pattern.
If, so it should be explicit register(), unregister() API.
This is my summary https://lore.kernel.org/linux-rdma/20201008052137.GA13580@unreal The API should be symmetric.
I disagree to your below point.
- You are not providing driver/core API but simplification and obfuscation
of basic primitives and structures. This is new layer. There is no room for a claim that we must to follow internal API.
If ancillary bus has ancillary_device_add(), it cannot do device_initialize() and device_add() in both.
I provided two examples and what really matters is a given patchset uses (need to use) which pattern, initialize() + add(), or register() + unregister().
As we all know that API is not added for future. It is the future patch extends it. So lets wait for Pierre to reply if soundwire can follow register(), unregister() sequence. This way same APIs can service both use-cases.
Regarding,
- You can't "ask" from users to call internal calls (put_device) over internal
fields in ancillary_device.
In that case if should be ancillary_device_put() ancillary_device_release().
Or we should follow the patten of ib_alloc_device [1], ancillary_device_alloc() -> kzalloc(adev + dev) with compile time assert check like rdma and vdpa subsystem. ->device_initialize() ancillary_device_add()
ancillar_device_de() <- balances with add ancillary_device_dealloc() <-- balances with device_alloc(), which does the put_device() + free the memory allocated in alloc().
This approach of [1] also eliminates exposing adev.dev.release = <drivers_release_method_to_free_adev> in drivers. And container_of() benefit also continues..
[1] https://elixir.bootlin.com/linux/v5.9-rc8/source/include/rdma/ib_verbs.h#L27...
My code looks like this, probably yours looks the same.
247 priv->adev[i] = kzalloc(sizeof(*priv->adev[i]), GFP_KERNEL); 248 if (!priv->adev[i]) 249 goto init_err; 250 251 adev = &priv->adev[i]->adev; 252 adev->id = idx; 253 adev->name = mlx5_adev_devices[i].suffix; 254 adev->dev.parent = dev->device; 255 adev->dev.release = adev_release; 256 priv->adev[i]->mdev = dev; 257 258 ret = ancillary_device_initialize(adev); 259 if (ret) 260 goto init_err; 261 262 ret = ancillary_device_add(adev); 263 if (ret) { 264 put_device(&adev->dev); 265 goto add_err; 266 }
Thanks
From: Leon Romanovsky leon@kernel.org Sent: Thursday, October 8, 2020 1:15 PM
On Thu, Oct 08, 2020 at 07:14:17AM +0000, Parav Pandit wrote:
From: Leon Romanovsky leon@kernel.org Sent: Thursday, October 8, 2020 10:56 AM
On Thu, Oct 08, 2020 at 04:56:01AM +0000, Parav Pandit wrote:
From: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Sent: Thursday, October 8, 2020 3:20 AM
On 10/7/20 4:22 PM, Ertman, David M wrote:
> -----Original Message----- > From: Pierre-Louis Bossart > pierre-louis.bossart@linux.intel.com > Sent: Wednesday, October 7, 2020 1:59 PM > To: Ertman, David M david.m.ertman@intel.com; Parav Pandit > parav@nvidia.com; Leon Romanovsky leon@kernel.org > Cc: alsa-devel@alsa-project.org; parav@mellanox.com; > tiwai@suse.de; netdev@vger.kernel.org; > ranjani.sridharan@linux.intel.com; > fred.oh@linux.intel.com; linux-rdma@vger.kernel.org; > dledford@redhat.com; broonie@kernel.org; Jason Gunthorpe > jgg@nvidia.com; gregkh@linuxfoundation.org; > kuba@kernel.org; Williams, Dan J dan.j.williams@intel.com; > Saleem, Shiraz shiraz.saleem@intel.com; > davem@davemloft.net; Patil, Kiran kiran.patil@intel.com > Subject: Re: [PATCH v2 1/6] Add ancillary bus support > > > >>> Below is most simple, intuitive and matching with core APIs >>> for name and design pattern wise. >>> init() >>> { >>> err = ancillary_device_initialize(); >>> if (err) >>> return ret; >>> >>> err = ancillary_device_add(); >>> if (ret) >>> goto err_unwind; >>> >>> err = some_foo(); >>> if (err) >>> goto err_foo; >>> return 0; >>> >>> err_foo: >>> ancillary_device_del(adev); >>> err_unwind: >>> ancillary_device_put(adev->dev); >>> return err; >>> } >>> >>> cleanup() >>> { >>> ancillary_device_de(adev); >>> ancillary_device_put(adev); >>> /* It is common to have a one wrapper for this as >>> ancillary_device_unregister(). >>> * This will match with core device_unregister() that has >>> precise documentation. >>> * but given fact that init() code need proper error >>> unwinding, like above, >>> * it make sense to have two APIs, and no need to export >>> another symbol for unregister(). >>> * This pattern is very easy to audit and code. >>> */ >>> } >> >> I like this flow +1 >> >> But ... since the init() function is performing both >> device_init and device_add - it should probably be called >> ancillary_device_register, and we are back to a single >> exported API for both register and unregister. > > Kind reminder that we introduced the two functions to allow > the caller to know if it needed to free memory when > initialize() fails, and it didn't need to free memory when > add() failed since > put_device() takes care of it. If you have a single init() > function it's impossible to know which behavior to select on error. > > I also have a case with SoundWire where it's nice to first > initialize, then set some data and then add. >
The flow as outlined by Parav above does an initialize as the first step, so every error path out of the function has to do a put_device(), so you would never need to manually free the memory in
the setup function.
It would be freed in the release call.
err = ancillary_device_initialize(); if (err) return ret;
where is the put_device() here? if the release function does any sort of kfree, then you'd need to do it manually in this case.
Since device_initialize() failed, put_device() cannot be done here. So yes, pseudo code should have shown, if (err) { kfree(adev); return err; }
If we just want to follow register(), unregister() pattern,
Than,
ancillar_device_register() should be,
/**
- ancillar_device_register() - register an ancillary device
- NOTE: __never directly free @adev after calling this function,
even if it returned
- an error. Always use ancillary_device_put() to give up the
reference
initialized by this function.
- This note matches with the core and caller knows exactly what
to be
done.
*/ ancillary_device_register() { device_initialize(&adev->dev); if (!dev->parent || !adev->name) return -EINVAL; if (!dev->release && !(dev->type && dev->type->release)) { /* core is already capable and throws the warning when
release callback is not set.
* It is done at drivers/base/core.c:1798. * For NULL release it says, "does not have a release()function, it is broken and must be fixed"
*/ return -EINVAL;} err = dev_set_name(adev...); if (err) { /* kobject_release() -> kobject_cleanup() are capable to
detect if name is set/ not set
* and free the const if it was set. */ return err;} err = device_add(&adev->dev); If (err) return err; }
Caller code: init() { adev = kzalloc(sizeof(*foo_adev)..); if (!adev) return -ENOMEM; err = ancillary_device_register(&adev); if (err) goto err;
err: ancillary_device_put(&adev); return err; }
cleanup() { ancillary_device_unregister(&adev); }
Above pattern is fine too matching the core.
If I understand Leon correctly, he prefers simple register(), unregister()
pattern.
If, so it should be explicit register(), unregister() API.
This is my summary https://lore.kernel.org/linux-rdma/20201008052137.GA13580@unreal The API should be symmetric.
I disagree to your below point.
- You are not providing driver/core API but simplification and
obfuscation of basic primitives and structures. This is new layer. There is no room for a claim that we must to follow internal API.
If ancillary bus has ancillary_device_add(), it cannot do device_initialize() and device_add() in
both.
I provided two examples and what really matters is a given patchset uses (need to use) which pattern, initialize() + add(), or register() + unregister().
As we all know that API is not added for future. It is the future patch
extends it.
So lets wait for Pierre to reply if soundwire can follow register(),
unregister() sequence.
This way same APIs can service both use-cases.
Regarding,
- You can't "ask" from users to call internal calls (put_device)
over internal fields in ancillary_device.
In that case if should be ancillary_device_put() ancillary_device_release().
Or we should follow the patten of ib_alloc_device [1], ancillary_device_alloc() -> kzalloc(adev + dev) with compile time assert check like rdma and vdpa
subsystem.
->device_initialize()ancillary_device_add()
ancillar_device_de() <- balances with add ancillary_device_dealloc() <-- balances with device_alloc(), which does the
put_device() + free the memory allocated in alloc().
This approach of [1] also eliminates exposing adev.dev.release =
<drivers_release_method_to_free_adev> in drivers.
And container_of() benefit also continues..
[1] https://elixir.bootlin.com/linux/v5.9-rc8/source/include/rdma/ib_verbs .h#L2791
My code looks like this, probably yours looks the same.
247 priv->adev[i] = kzalloc(sizeof(*priv->adev[i]), GFP_KERNEL); 248 if (!priv->adev[i]) 249 goto init_err; 250 251 adev = &priv->adev[i]->adev; 252 adev->id = idx; 253 adev->name = mlx5_adev_devices[i].suffix; 254 adev->dev.parent = dev->device; 255 adev->dev.release = adev_release; 256 priv->adev[i]->mdev = dev; 257 258 ret = ancillary_device_initialize(adev); 259 if (ret) 260 goto init_err; 261 262 ret = ancillary_device_add(adev); 263 if (ret) { 264 put_device(&adev->dev); 265 goto add_err; 266 }
Yes, subfunction code is also very similar. You expressed concerned that you didn't like put_device() at [1]. But in above code is touching adev->dev.{parent, release} is ok?
254 adev->dev.parent = dev->device; 255 adev->dev.release = adev_release;
If not,
We can make it elegant by doing,
the patten of ib_alloc_device [1], ancillary_device_alloc() -> kzalloc(adev + dev) with compile time assert check like rdma and vdpa subsystem. ->device_initialize() ancillary_device_add()
ancillar_device_de() <- balances with add ancillary_device_dealloc() <-- balances with device_alloc(), which does the put_device() + free the memory allocated in alloc().
This approach of [2] also eliminates exposing adev.dev.release = <drivers_release_method_to_free_adev> in drivers. And container_of() benefit also continues..
[1] https://lore.kernel.org/linux-rdma/20201007192610.GD3964015@unreal/ [2] https://elixir.bootlin.com/linux/v5.9-rc8/source/include/rdma/ib_verbs.h#L27...
On Thu, Oct 08, 2020 at 09:45:29AM +0000, Parav Pandit wrote:
From: Leon Romanovsky leon@kernel.org Sent: Thursday, October 8, 2020 1:15 PM
On Thu, Oct 08, 2020 at 07:14:17AM +0000, Parav Pandit wrote:
From: Leon Romanovsky leon@kernel.org Sent: Thursday, October 8, 2020 10:56 AM
On Thu, Oct 08, 2020 at 04:56:01AM +0000, Parav Pandit wrote:
From: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Sent: Thursday, October 8, 2020 3:20 AM
On 10/7/20 4:22 PM, Ertman, David M wrote: >> -----Original Message----- >> From: Pierre-Louis Bossart >> pierre-louis.bossart@linux.intel.com >> Sent: Wednesday, October 7, 2020 1:59 PM >> To: Ertman, David M david.m.ertman@intel.com; Parav Pandit >> parav@nvidia.com; Leon Romanovsky leon@kernel.org >> Cc: alsa-devel@alsa-project.org; parav@mellanox.com; >> tiwai@suse.de; netdev@vger.kernel.org; >> ranjani.sridharan@linux.intel.com; >> fred.oh@linux.intel.com; linux-rdma@vger.kernel.org; >> dledford@redhat.com; broonie@kernel.org; Jason Gunthorpe >> jgg@nvidia.com; gregkh@linuxfoundation.org; >> kuba@kernel.org; Williams, Dan J dan.j.williams@intel.com; >> Saleem, Shiraz shiraz.saleem@intel.com; >> davem@davemloft.net; Patil, Kiran kiran.patil@intel.com >> Subject: Re: [PATCH v2 1/6] Add ancillary bus support >> >> >> >>>> Below is most simple, intuitive and matching with core APIs >>>> for name and design pattern wise. >>>> init() >>>> { >>>> err = ancillary_device_initialize(); >>>> if (err) >>>> return ret; >>>> >>>> err = ancillary_device_add(); >>>> if (ret) >>>> goto err_unwind; >>>> >>>> err = some_foo(); >>>> if (err) >>>> goto err_foo; >>>> return 0; >>>> >>>> err_foo: >>>> ancillary_device_del(adev); >>>> err_unwind: >>>> ancillary_device_put(adev->dev); >>>> return err; >>>> } >>>> >>>> cleanup() >>>> { >>>> ancillary_device_de(adev); >>>> ancillary_device_put(adev); >>>> /* It is common to have a one wrapper for this as >>>> ancillary_device_unregister(). >>>> * This will match with core device_unregister() that has >>>> precise documentation. >>>> * but given fact that init() code need proper error >>>> unwinding, like above, >>>> * it make sense to have two APIs, and no need to export >>>> another symbol for unregister(). >>>> * This pattern is very easy to audit and code. >>>> */ >>>> } >>> >>> I like this flow +1 >>> >>> But ... since the init() function is performing both >>> device_init and device_add - it should probably be called >>> ancillary_device_register, and we are back to a single >>> exported API for both register and unregister. >> >> Kind reminder that we introduced the two functions to allow >> the caller to know if it needed to free memory when >> initialize() fails, and it didn't need to free memory when >> add() failed since >> put_device() takes care of it. If you have a single init() >> function it's impossible to know which behavior to select on error. >> >> I also have a case with SoundWire where it's nice to first >> initialize, then set some data and then add. >> > > The flow as outlined by Parav above does an initialize as the > first step, so every error path out of the function has to do > a put_device(), so you would never need to manually free the > memory in the setup function. > It would be freed in the release call.
err = ancillary_device_initialize(); if (err) return ret;
where is the put_device() here? if the release function does any sort of kfree, then you'd need to do it manually in this case.
Since device_initialize() failed, put_device() cannot be done here. So yes, pseudo code should have shown, if (err) { kfree(adev); return err; }
If we just want to follow register(), unregister() pattern,
Than,
ancillar_device_register() should be,
/**
- ancillar_device_register() - register an ancillary device
- NOTE: __never directly free @adev after calling this function,
even if it returned
- an error. Always use ancillary_device_put() to give up the
reference
initialized by this function.
- This note matches with the core and caller knows exactly what
to be
done.
*/ ancillary_device_register() { device_initialize(&adev->dev); if (!dev->parent || !adev->name) return -EINVAL; if (!dev->release && !(dev->type && dev->type->release)) { /* core is already capable and throws the warning when
release callback is not set.
* It is done at drivers/base/core.c:1798. * For NULL release it says, "does not have a release()function, it is broken and must be fixed"
*/ return -EINVAL;} err = dev_set_name(adev...); if (err) { /* kobject_release() -> kobject_cleanup() are capable to
detect if name is set/ not set
* and free the const if it was set. */ return err;} err = device_add(&adev->dev); If (err) return err; }
Caller code: init() { adev = kzalloc(sizeof(*foo_adev)..); if (!adev) return -ENOMEM; err = ancillary_device_register(&adev); if (err) goto err;
err: ancillary_device_put(&adev); return err; }
cleanup() { ancillary_device_unregister(&adev); }
Above pattern is fine too matching the core.
If I understand Leon correctly, he prefers simple register(), unregister()
pattern.
If, so it should be explicit register(), unregister() API.
This is my summary https://lore.kernel.org/linux-rdma/20201008052137.GA13580@unreal The API should be symmetric.
I disagree to your below point.
- You are not providing driver/core API but simplification and
obfuscation of basic primitives and structures. This is new layer. There is no room for a claim that we must to follow internal API.
If ancillary bus has ancillary_device_add(), it cannot do device_initialize() and device_add() in
both.
I provided two examples and what really matters is a given patchset uses (need to use) which pattern, initialize() + add(), or register() + unregister().
As we all know that API is not added for future. It is the future patch
extends it.
So lets wait for Pierre to reply if soundwire can follow register(),
unregister() sequence.
This way same APIs can service both use-cases.
Regarding,
- You can't "ask" from users to call internal calls (put_device)
over internal fields in ancillary_device.
In that case if should be ancillary_device_put() ancillary_device_release().
Or we should follow the patten of ib_alloc_device [1], ancillary_device_alloc() -> kzalloc(adev + dev) with compile time assert check like rdma and vdpa
subsystem.
->device_initialize()ancillary_device_add()
ancillar_device_de() <- balances with add ancillary_device_dealloc() <-- balances with device_alloc(), which does the
put_device() + free the memory allocated in alloc().
This approach of [1] also eliminates exposing adev.dev.release =
<drivers_release_method_to_free_adev> in drivers.
And container_of() benefit also continues..
[1] https://elixir.bootlin.com/linux/v5.9-rc8/source/include/rdma/ib_verbs .h#L2791
My code looks like this, probably yours looks the same.
247 priv->adev[i] = kzalloc(sizeof(*priv->adev[i]), GFP_KERNEL); 248 if (!priv->adev[i]) 249 goto init_err; 250 251 adev = &priv->adev[i]->adev; 252 adev->id = idx; 253 adev->name = mlx5_adev_devices[i].suffix; 254 adev->dev.parent = dev->device; 255 adev->dev.release = adev_release; 256 priv->adev[i]->mdev = dev; 257 258 ret = ancillary_device_initialize(adev); 259 if (ret) 260 goto init_err; 261 262 ret = ancillary_device_add(adev); 263 if (ret) { 264 put_device(&adev->dev); 265 goto add_err; 266 }
Yes, subfunction code is also very similar. You expressed concerned that you didn't like put_device() at [1]. But in above code is touching adev->dev.{parent, release} is ok?
Yes, "adev->dev.{parent, release}" is not ok, but at least it doesn't complicate error unwinding. This is why I didn't say anything about it.
254 adev->dev.parent = dev->device; 255 adev->dev.release = adev_release;
If not,
We can make it elegant by doing,
I like your idea, IMHO it is more clear and less error prone.
Thanks
the patten of ib_alloc_device [1], ancillary_device_alloc() -> kzalloc(adev + dev) with compile time assert check like rdma and vdpa subsystem. ->device_initialize() ancillary_device_add()
ancillar_device_de() <- balances with add ancillary_device_dealloc() <-- balances with device_alloc(), which does the put_device() + free the memory allocated in alloc().
This approach of [2] also eliminates exposing adev.dev.release = <drivers_release_method_to_free_adev> in drivers. And container_of() benefit also continues..
[1] https://lore.kernel.org/linux-rdma/20201007192610.GD3964015@unreal/ [2] https://elixir.bootlin.com/linux/v5.9-rc8/source/include/rdma/ib_verbs.h#L27...
But ... since the init() function is performing both device_init and device_add - it should probably be called ancillary_device_register, and we are back to a single exported API for both register and unregister.
Kind reminder that we introduced the two functions to allow the caller to know if it needed to free memory when initialize() fails, and it didn't need to free memory when add() failed since put_device() takes care of it. If you have a single init() function it's impossible to know which behavior to select on error.
I also have a case with SoundWire where it's nice to first initialize, then set some data and then add.
The flow as outlined by Parav above does an initialize as the first step, so every error path out of the function has to do a put_device(), so you would never need to manually free the memory in
the setup function.
It would be freed in the release call.
err = ancillary_device_initialize(); if (err) return ret;
where is the put_device() here? if the release function does any sort of kfree, then you'd need to do it manually in this case.
Since device_initialize() failed, put_device() cannot be done here. So yes, pseudo code should have shown, if (err) { kfree(adev); return err; }
This doesn't work if the adev is part of a larger structure allocated by the parent, which is pretty much the intent to extent the basic bus and pass additional information which can be accessed with container_of().
Only the parent can do the kfree() explicitly in that case. If the parent relies on devm_kzalloc, this also can make the .release callback with no memory free required at all.
See e.g. the code I cooked for the transition of SoundWire away from platform devices at
https://github.com/thesofproject/linux/pull/2484/commits/d0540ae3744f3a748d4...
The allocation is done on an 'ldev' which contains 'adev'.
I really don't seen how an ancillary_device_register() could model the different ways to allocate memory, for maximum flexibility across different domains it seems more relevant to keep the initialize() and add() APIs separate. I will accept the argument that this puts more responsibility on the parent, but it also provides more flexibility to the parent.
If we go with the suggested solution above, that already prevents SoundWire from using this bus. Not so good.
On Thu, Oct 08, 2020 at 08:29:00AM -0500, Pierre-Louis Bossart wrote:
But ... since the init() function is performing both device_init and device_add - it should probably be called ancillary_device_register, and we are back to a single exported API for both register and unregister.
Kind reminder that we introduced the two functions to allow the caller to know if it needed to free memory when initialize() fails, and it didn't need to free memory when add() failed since put_device() takes care of it. If you have a single init() function it's impossible to know which behavior to select on error.
I also have a case with SoundWire where it's nice to first initialize, then set some data and then add.
The flow as outlined by Parav above does an initialize as the first step, so every error path out of the function has to do a put_device(), so you would never need to manually free the memory in
the setup function.
It would be freed in the release call.
err = ancillary_device_initialize(); if (err) return ret;
where is the put_device() here? if the release function does any sort of kfree, then you'd need to do it manually in this case.
Since device_initialize() failed, put_device() cannot be done here. So yes, pseudo code should have shown, if (err) { kfree(adev); return err; }
This doesn't work if the adev is part of a larger structure allocated by the parent, which is pretty much the intent to extent the basic bus and pass additional information which can be accessed with container_of().
Please take a look how ib_alloc_device() is implemented. It does all that you wrote above in very similar manner to netdev_alloc.
In a nutshell, ib_alloc_device receives needed size from the user and requires from the users to extend their structures below "general" one.
Thanks
-----Original Message----- From: Parav Pandit parav@nvidia.com Sent: Wednesday, October 7, 2020 9:56 PM To: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com; Ertman, David M david.m.ertman@intel.com; Leon Romanovsky leon@kernel.org Cc: alsa-devel@alsa-project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux-rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; Jason Gunthorpe jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; Williams, Dan J dan.j.williams@intel.com; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: RE: [PATCH v2 1/6] Add ancillary bus support
From: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Sent: Thursday, October 8, 2020 3:20 AM
On 10/7/20 4:22 PM, Ertman, David M wrote:
-----Original Message----- From: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Sent: Wednesday, October 7, 2020 1:59 PM To: Ertman, David M david.m.ertman@intel.com; Parav Pandit parav@nvidia.com; Leon Romanovsky leon@kernel.org Cc: alsa-devel@alsa-project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux-rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; Jason Gunthorpe jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; Williams, Dan J dan.j.williams@intel.com; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: Re: [PATCH v2 1/6] Add ancillary bus support
Below is most simple, intuitive and matching with core APIs for name and design pattern wise. init() { err = ancillary_device_initialize(); if (err) return ret;
err = ancillary_device_add(); if (ret) goto err_unwind;
err = some_foo(); if (err) goto err_foo; return 0;
err_foo: ancillary_device_del(adev); err_unwind: ancillary_device_put(adev->dev); return err; }
cleanup() { ancillary_device_de(adev); ancillary_device_put(adev); /* It is common to have a one wrapper for this as ancillary_device_unregister(). * This will match with core device_unregister() that has precise documentation. * but given fact that init() code need proper error unwinding, like above, * it make sense to have two APIs, and no need to export another symbol for unregister(). * This pattern is very easy to audit and code. */ }
I like this flow +1
But ... since the init() function is performing both device_init and device_add - it should probably be called ancillary_device_register, and we are back to a single exported API for both register and unregister.
Kind reminder that we introduced the two functions to allow the caller to know if it needed to free memory when initialize() fails, and it didn't need to free memory when add() failed since put_device() takes care of it. If you have a single init() function it's impossible to know which behavior to select on error.
I also have a case with SoundWire where it's nice to first initialize, then set some data and then add.
The flow as outlined by Parav above does an initialize as the first step, so every error path out of the function has to do a put_device(), so you would never need to manually free the memory in
the setup function.
It would be freed in the release call.
err = ancillary_device_initialize(); if (err) return ret;
where is the put_device() here? if the release function does any sort of kfree, then you'd need to do it manually in this case.
Since device_initialize() failed, put_device() cannot be done here. So yes, pseudo code should have shown, if (err) { kfree(adev); return err; }
If we just want to follow register(), unregister() pattern,
Than,
ancillar_device_register() should be,
/**
- ancillar_device_register() - register an ancillary device
- NOTE: __never directly free @adev after calling this function, even if it
returned
- an error. Always use ancillary_device_put() to give up the reference
initialized by this function.
- This note matches with the core and caller knows exactly what to be done.
*/ ancillary_device_register() { device_initialize(&adev->dev); if (!dev->parent || !adev->name) return -EINVAL; if (!dev->release && !(dev->type && dev->type->release)) { /* core is already capable and throws the warning when release callback is not set. * It is done at drivers/base/core.c:1798. * For NULL release it says, "does not have a release() function, it is broken and must be fixed" */ return -EINVAL; }
That code is in device_release(). Because of this check we will never hit that code.
We either need to leave the error message here, or if we are going to rely on the core to find this condition at the end of the process, then we need to completely remove this check from the registration flow.
-DaveE
From: Ertman, David M david.m.ertman@intel.com Sent: Thursday, October 8, 2020 10:24 PM
From: Parav Pandit parav@nvidia.com Sent: Wednesday, October 7, 2020 9:56 PM
/**
- ancillar_device_register() - register an ancillary device
- NOTE: __never directly free @adev after calling this function, even
if it returned
- an error. Always use ancillary_device_put() to give up the
reference initialized by this function.
- This note matches with the core and caller knows exactly what to be
done.
*/ ancillary_device_register() { device_initialize(&adev->dev); if (!dev->parent || !adev->name) return -EINVAL; if (!dev->release && !(dev->type && dev->type->release)) { /* core is already capable and throws the warning when
release
callback is not set. * It is done at drivers/base/core.c:1798. * For NULL release it says, "does not have a release()
function, it
is broken and must be fixed" */ return -EINVAL; }
That code is in device_release(). Because of this check we will never hit that code.
We either need to leave the error message here, or if we are going to rely on the core to find this condition at the end of the process, then we need to completely remove this check from the registration flow.
Yes. Since the core is checking it, ancillary bus doesn't need to check here and release callback check can be removed.
-DaveE
-----Original Message----- From: Parav Pandit parav@nvidia.com Sent: Thursday, October 8, 2020 10:35 AM To: Ertman, David M david.m.ertman@intel.com; Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com; Leon Romanovsky leon@kernel.org Cc: alsa-devel@alsa-project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux-rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; Jason Gunthorpe jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; Williams, Dan J dan.j.williams@intel.com; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: RE: [PATCH v2 1/6] Add ancillary bus support
From: Ertman, David M david.m.ertman@intel.com Sent: Thursday, October 8, 2020 10:24 PM
From: Parav Pandit parav@nvidia.com Sent: Wednesday, October 7, 2020 9:56 PM
/**
- ancillar_device_register() - register an ancillary device
- NOTE: __never directly free @adev after calling this function, even
if it returned
- an error. Always use ancillary_device_put() to give up the
reference initialized by this function.
- This note matches with the core and caller knows exactly what to be
done.
*/ ancillary_device_register() { device_initialize(&adev->dev); if (!dev->parent || !adev->name) return -EINVAL; if (!dev->release && !(dev->type && dev->type->release)) { /* core is already capable and throws the warning when
release
callback is not set. * It is done at drivers/base/core.c:1798. * For NULL release it says, "does not have a release()
function, it
is broken and must be fixed" */ return -EINVAL; }
That code is in device_release(). Because of this check we will never hit
that
code.
We either need to leave the error message here, or if we are going to rely
on
the core to find this condition at the end of the process, then we need to completely remove this check from the registration flow.
Yes. Since the core is checking it, ancillary bus doesn't need to check here and release callback check can be removed.
Will do
-DaveE
On Wed, Oct 07, 2020 at 08:46:45PM +0000, Ertman, David M wrote:
-----Original Message----- From: Parav Pandit parav@nvidia.com Sent: Wednesday, October 7, 2020 1:17 PM To: Leon Romanovsky leon@kernel.org; Ertman, David M david.m.ertman@intel.com Cc: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com; alsa- devel@alsa-project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux-rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; Jason Gunthorpe jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; Williams, Dan J dan.j.williams@intel.com; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: RE: [PATCH v2 1/6] Add ancillary bus support
From: Leon Romanovsky leon@kernel.org Sent: Thursday, October 8, 2020 12:56 AM
This API is partially obscures low level driver-core code and needs to provide clear and proper abstractions without need to remember about put_device. There is already _add() interface why don't you do put_device() in it?
The pushback Pierre is referring to was during our mid-tier internal review. It was primarily a concern of Parav as I recall, so he can speak to
his
reasoning.
What we originally had was a single API call (ancillary_device_register) that started with a call to device_initialize(), and every error path out of the function performed a
put_device().
Is this the model you have in mind?
I don't like this flow: ancillary_device_initialize() if (ancillary_ancillary_device_add()) { put_device(....) ancillary_device_unregister()
Calling device_unregister() is incorrect, because add() wasn't successful. Only put_device() or a wrapper ancillary_device_put() is necessary.
return err; }
And prefer this flow: ancillary_device_initialize() if (ancillary_device_add()) { ancillary_device_unregister()
This is incorrect and a clear deviation from the current core APIs that adds the confusion.
return err; }
In this way, the ancillary users won't need to do non-intuitive put_device();
Below is most simple, intuitive and matching with core APIs for name and design pattern wise. init() { err = ancillary_device_initialize(); if (err) return ret;
err = ancillary_device_add(); if (ret) goto err_unwind;
err = some_foo(); if (err) goto err_foo; return 0;
err_foo: ancillary_device_del(adev); err_unwind: ancillary_device_put(adev->dev); return err; }
cleanup() { ancillary_device_de(adev); ancillary_device_put(adev); /* It is common to have a one wrapper for this as ancillary_device_unregister(). * This will match with core device_unregister() that has precise documentation. * but given fact that init() code need proper error unwinding, like above, * it make sense to have two APIs, and no need to export another symbol for unregister(). * This pattern is very easy to audit and code. */ }
I like this flow +1
But ... since the init() function is performing both device_init and device_add - it should probably be called ancillary_device_register, and we are back to a single exported API for both register and unregister.
At that point, do we need wrappers on the primitives init, add, del, and put?
Let me summarize. 1. You are not providing driver/core API but simplification and obfuscation of basic primitives and structures. This is new layer. There is no room for a claim that we must to follow internal API. 2. API should be symmetric. If you call to _register()/_add(), you will need to call to _unregister()/_del(). Please don't add obscure _put(). 3. You can't "ask" from users to call internal calls (put_device) over internal fields in ancillary_device. 4. This API should be clear to drivers authors, "device_add()" call (and semantic) is not used by the drivers (git grep " device_add(" drivers/).
Thanks
-DaveE
On Wed, Oct 7, 2020 at 10:21 PM Leon Romanovsky leon@kernel.org wrote:
On Wed, Oct 07, 2020 at 08:46:45PM +0000, Ertman, David M wrote:
-----Original Message----- From: Parav Pandit parav@nvidia.com Sent: Wednesday, October 7, 2020 1:17 PM To: Leon Romanovsky leon@kernel.org; Ertman, David M david.m.ertman@intel.com Cc: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com; alsa- devel@alsa-project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux-rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; Jason Gunthorpe jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; Williams, Dan J dan.j.williams@intel.com; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: RE: [PATCH v2 1/6] Add ancillary bus support
From: Leon Romanovsky leon@kernel.org Sent: Thursday, October 8, 2020 12:56 AM
This API is partially obscures low level driver-core code and needs to provide clear and proper abstractions without need to remember about put_device. There is already _add() interface why don't you do put_device() in it?
The pushback Pierre is referring to was during our mid-tier internal review. It was primarily a concern of Parav as I recall, so he can speak to
his
reasoning.
What we originally had was a single API call (ancillary_device_register) that started with a call to device_initialize(), and every error path out of the function performed a
put_device().
Is this the model you have in mind?
I don't like this flow: ancillary_device_initialize() if (ancillary_ancillary_device_add()) { put_device(....) ancillary_device_unregister()
Calling device_unregister() is incorrect, because add() wasn't successful. Only put_device() or a wrapper ancillary_device_put() is necessary.
return err; }
And prefer this flow: ancillary_device_initialize() if (ancillary_device_add()) { ancillary_device_unregister()
This is incorrect and a clear deviation from the current core APIs that adds the confusion.
return err; }
In this way, the ancillary users won't need to do non-intuitive put_device();
Below is most simple, intuitive and matching with core APIs for name and design pattern wise. init() { err = ancillary_device_initialize(); if (err) return ret;
err = ancillary_device_add(); if (ret) goto err_unwind; err = some_foo(); if (err) goto err_foo; return 0;err_foo: ancillary_device_del(adev); err_unwind: ancillary_device_put(adev->dev); return err; }
cleanup() { ancillary_device_de(adev); ancillary_device_put(adev); /* It is common to have a one wrapper for this as ancillary_device_unregister(). * This will match with core device_unregister() that has precise documentation. * but given fact that init() code need proper error unwinding, like above, * it make sense to have two APIs, and no need to export another symbol for unregister(). * This pattern is very easy to audit and code. */ }
I like this flow +1
But ... since the init() function is performing both device_init and device_add - it should probably be called ancillary_device_register, and we are back to a single exported API for both register and unregister.
At that point, do we need wrappers on the primitives init, add, del, and put?
Let me summarize.
- You are not providing driver/core API but simplification and obfuscation
of basic primitives and structures. This is new layer. There is no room for a claim that we must to follow internal API.
Yes, this a driver core api, Greg even questioned why it was in drivers/bus instead of drivers/base which I think makes sense.
- API should be symmetric. If you call to _register()/_add(), you will need
to call to _unregister()/_del(). Please don't add obscure _put().
It's not obscure it's a long standing semantic for how to properly handle device_add() failures. Especially in this case where there is no way to have something like a common auxiliary_device_alloc() that will work for everyone the only other option is require all device destruction to go through the provided release method (put_device()) after a device_add() failure.
- You can't "ask" from users to call internal calls (put_device) over internal
fields in ancillary_device.
Sure it can. platform_device_add() requires a put_device() on failure, but also note how platform_device_add() *requires* platform_device_alloc() be used to create the device. That inflexibility is something this auxiliary bus is trying to avoid.
- This API should be clear to drivers authors, "device_add()" call (and
semantic) is not used by the drivers (git grep " device_add(" drivers/).
This shows 141 instances for me, so I'm not sure what you're getting at?
Look, this api is meant to be a replacement for places where platform devices were being abused. The device_initialize() + customize device + device_add() organization has the flexibility needed to let users customize naming and other parts of device creation in a way that a device_register() flow, or platform_device_{register,add} in particular, did not.
If the concern is that you'd like to have an auxiliary_device_put() for symmetry that would need to come with the same warning as commented on platform_device_put(), i.e. that's it's really only vanity symmetry to be used in error paths. The semantics of device_add() and device_put() on failure are long established, don't invent new behavior for auxiliary_device_add() and auxiliary_device_put() / put_device().
On Wed, Oct 07, 2020 at 11:32:11PM -0700, Dan Williams wrote:
On Wed, Oct 7, 2020 at 10:21 PM Leon Romanovsky leon@kernel.org wrote:
On Wed, Oct 07, 2020 at 08:46:45PM +0000, Ertman, David M wrote:
-----Original Message----- From: Parav Pandit parav@nvidia.com Sent: Wednesday, October 7, 2020 1:17 PM To: Leon Romanovsky leon@kernel.org; Ertman, David M david.m.ertman@intel.com Cc: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com; alsa- devel@alsa-project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux-rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; Jason Gunthorpe jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; Williams, Dan J dan.j.williams@intel.com; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: RE: [PATCH v2 1/6] Add ancillary bus support
From: Leon Romanovsky leon@kernel.org Sent: Thursday, October 8, 2020 12:56 AM
> This API is partially obscures low level driver-core code and needs > to provide clear and proper abstractions without need to remember > about put_device. There is already _add() interface why don't you do > put_device() in it? >
The pushback Pierre is referring to was during our mid-tier internal review. It was primarily a concern of Parav as I recall, so he can speak to
his
reasoning.
What we originally had was a single API call (ancillary_device_register) that started with a call to device_initialize(), and every error path out of the function performed a
put_device().
Is this the model you have in mind?
I don't like this flow: ancillary_device_initialize() if (ancillary_ancillary_device_add()) { put_device(....) ancillary_device_unregister()
Calling device_unregister() is incorrect, because add() wasn't successful. Only put_device() or a wrapper ancillary_device_put() is necessary.
return err; }
And prefer this flow: ancillary_device_initialize() if (ancillary_device_add()) { ancillary_device_unregister()
This is incorrect and a clear deviation from the current core APIs that adds the confusion.
return err; }
In this way, the ancillary users won't need to do non-intuitive put_device();
Below is most simple, intuitive and matching with core APIs for name and design pattern wise. init() { err = ancillary_device_initialize(); if (err) return ret;
err = ancillary_device_add(); if (ret) goto err_unwind; err = some_foo(); if (err) goto err_foo; return 0;err_foo: ancillary_device_del(adev); err_unwind: ancillary_device_put(adev->dev); return err; }
cleanup() { ancillary_device_de(adev); ancillary_device_put(adev); /* It is common to have a one wrapper for this as ancillary_device_unregister(). * This will match with core device_unregister() that has precise documentation. * but given fact that init() code need proper error unwinding, like above, * it make sense to have two APIs, and no need to export another symbol for unregister(). * This pattern is very easy to audit and code. */ }
I like this flow +1
But ... since the init() function is performing both device_init and device_add - it should probably be called ancillary_device_register, and we are back to a single exported API for both register and unregister.
At that point, do we need wrappers on the primitives init, add, del, and put?
Let me summarize.
- You are not providing driver/core API but simplification and obfuscation
of basic primitives and structures. This is new layer. There is no room for a claim that we must to follow internal API.
Yes, this a driver core api, Greg even questioned why it was in drivers/bus instead of drivers/base which I think makes sense.
We can argue till death, but at the end, this is a bus.
- API should be symmetric. If you call to _register()/_add(), you will need
to call to _unregister()/_del(). Please don't add obscure _put().
It's not obscure it's a long standing semantic for how to properly handle device_add() failures. Especially in this case where there is no way to have something like a common auxiliary_device_alloc() that will work for everyone the only other option is require all device destruction to go through the provided release method (put_device()) after a device_add() failure.
And this is my main concern, this is not device_add() failure but ancillary_device_add() which hides driver_* logic.
We won't expect to see inside ancillary drivers direct calls to device_*(), why will it be different here with put_device?
- You can't "ask" from users to call internal calls (put_device) over internal
fields in ancillary_device.
Sure it can. platform_device_add() requires a put_device() on failure, but also note how platform_device_add() *requires* platform_device_alloc() be used to create the device. That inflexibility is something this auxiliary bus is trying to avoid.
I'm writing below, the rationale behind this bus is RDMA, netdev and other cross-subsystem devices.
- This API should be clear to drivers authors, "device_add()" call (and
semantic) is not used by the drivers (git grep " device_add(" drivers/).
This shows 141 instances for me, so I'm not sure what you're getting at?
Did you look at them? I did, most if not all of the calls are in bus/core/generic logic, drivers are not calling to it or at least not supposed to.
Look, this api is meant to be a replacement for places where platform devices were being abused. The device_initialize() + customize device
- device_add() organization has the flexibility needed to let users
customize naming and other parts of device creation in a way that a device_register() flow, or platform_device_{register,add} in particular, did not.
It is hard me to say if the goal it to replace platform devices or not, but this ancillary_device bus adventure started after request to stop reinvent PCI logic for every new RDMA (RoCE) drivers. This is there full power of this virtbus solution comes into full power by deleting tons of complex code.
If the concern is that you'd like to have an auxiliary_device_put() for symmetry that would need to come with the same warning as commented on platform_device_put(), i.e. that's it's really only vanity symmetry to be used in error paths. The semantics of device_add() and device_put() on failure are long established, don't invent new behavior for auxiliary_device_add() and auxiliary_device_put() / put_device().
All stated above is my opinion, it can be different from yours.
Thanks
On Thu, Oct 8, 2020 at 12:01 AM Leon Romanovsky leon@kernel.org wrote: [..]
All stated above is my opinion, it can be different from yours.
Yes, but we need to converge to move this forward. Jason was involved in the current organization for registration, Greg was angling for this to be core functionality. I have use cases outside of RDMA and netdev. Parav was ok with the current organization. The SOF folks already have a proposed incorporation of it. The argument I am hearing is that "this registration api seems hard for driver writers" when we have several driver writers who have already taken a look and can make it work. If you want to follow on with a simpler wrappers for your use case, great, but I do not yet see anyone concurring with your opinion that the current organization is irretrievably broken or too obscure to use.
On Thu, Oct 08, 2020 at 12:38:00AM -0700, Dan Williams wrote:
On Thu, Oct 8, 2020 at 12:01 AM Leon Romanovsky leon@kernel.org wrote: [..]
All stated above is my opinion, it can be different from yours.
Yes, but we need to converge to move this forward. Jason was involved in the current organization for registration, Greg was angling for this to be core functionality. I have use cases outside of RDMA and netdev. Parav was ok with the current organization. The SOF folks already have a proposed incorporation of it. The argument I am hearing is that "this registration api seems hard for driver writers" when we have several driver writers who have already taken a look and can make it work. If you want to follow on with a simpler wrappers for your use case, great, but I do not yet see anyone concurring with your opinion that the current organization is irretrievably broken or too obscure to use.
That's kind of because I tuned out of this thread a long time ago :)
I do agree with Leon that I think the current patch is not the correct way to do this the easiest, but don't have a competing proposal to show what I mean.
Yet.
Let's see what happens after 5.10-rc1 is out, it's too late now for any of this for this next merge window so we can not worry about it for a few weeks.
thanks,
greg k-h
From: gregkh@linuxfoundation.org gregkh@linuxfoundation.org Sent: Thursday, October 8, 2020 1:21 PM
On Thu, Oct 08, 2020 at 12:38:00AM -0700, Dan Williams wrote:
On Thu, Oct 8, 2020 at 12:01 AM Leon Romanovsky leon@kernel.org
wrote:
[..]
All stated above is my opinion, it can be different from yours.
Yes, but we need to converge to move this forward. Jason was involved in the current organization for registration, Greg was angling for this to be core functionality. I have use cases outside of RDMA and netdev. Parav was ok with the current organization. The SOF folks already have a proposed incorporation of it. The argument I am hearing is that "this registration api seems hard for driver writers" when we have several driver writers who have already taken a look and can make it work. If you want to follow on with a simpler wrappers for your use case, great, but I do not yet see anyone concurring with your opinion that the current organization is irretrievably broken or too obscure to use.
That's kind of because I tuned out of this thread a long time ago :)
I do agree with Leon that I think the current patch is not the correct way to do this the easiest, but don't have a competing proposal to show what I mean.
Yet.
Please consider the approach of ib_alloc_device(), ib_dealloc_device() and ib_register_register()/unregister(). (a) It avoids driver calling put_device() on error unwinding path. (b) still achieves container_of().
Let's see what happens after 5.10-rc1 is out, it's too late now for any of this for this next merge window so we can not worry about it for a few weeks.
Ok. INHO giving direction to Dave and others to either refine current APIs or follow ib_alloc_device() approach will be a helpful input.
ancillary bus can do better APIs than the newly (march 2020 !) introduced vdpa bus [1] and its drivers which follows put_device() pattern in [2] and [3] in error unwinding path.
[1] https://elixir.bootlin.com/linux/v5.9-rc8/source/drivers/vdpa/vdpa.c [2] https://elixir.bootlin.com/linux/v5.9-rc8/source/drivers/vdpa/ifcvf/ifcvf_ma... [3] https://elixir.bootlin.com/linux/v5.9-rc8/source/drivers/vdpa/mlx5/net/mlx5_...
thanks,
greg k-h
-----Original Message----- From: Parav Pandit parav@nvidia.com Sent: Thursday, October 8, 2020 4:10 AM To: gregkh@linuxfoundation.org; Williams, Dan J dan.j.williams@intel.com Cc: Leon Romanovsky leon@kernel.org; Ertman, David M david.m.ertman@intel.com; Pierre-Louis Bossart <pierre- louis.bossart@linux.intel.com>; alsa-devel@alsa-project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux- rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; Jason Gunthorpe jgg@nvidia.com; kuba@kernel.org; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: RE: [PATCH v2 1/6] Add ancillary bus support
From: gregkh@linuxfoundation.org gregkh@linuxfoundation.org Sent: Thursday, October 8, 2020 1:21 PM
On Thu, Oct 08, 2020 at 12:38:00AM -0700, Dan Williams wrote:
On Thu, Oct 8, 2020 at 12:01 AM Leon Romanovsky leon@kernel.org
wrote:
[..]
All stated above is my opinion, it can be different from yours.
Yes, but we need to converge to move this forward. Jason was involved in the current organization for registration, Greg was angling for this to be core functionality. I have use cases outside of RDMA and netdev. Parav was ok with the current organization. The SOF folks already have a proposed incorporation of it. The argument I am hearing is that "this registration api seems hard for driver writers" when we have several driver writers who have already taken a look and can make it work. If you want to follow on with a simpler wrappers for your use case, great, but I do not yet see anyone concurring with your opinion that the current organization is irretrievably broken or too obscure to use.
That's kind of because I tuned out of this thread a long time ago :)
I do agree with Leon that I think the current patch is not the correct way to do this the easiest, but don't have a competing proposal to show what I mean.
Yet.
Please consider the approach of ib_alloc_device(), ib_dealloc_device() and ib_register_register()/unregister(). (a) It avoids driver calling put_device() on error unwinding path. (b) still achieves container_of().
Let's see what happens after 5.10-rc1 is out, it's too late now for any of this for this next merge window so we can not worry about it for a few weeks.
Ok. INHO giving direction to Dave and others to either refine current APIs or follow ib_alloc_device() approach will be a helpful input.
ancillary bus can do better APIs than the newly (march 2020 !) introduced vdpa bus [1] and its drivers which follows put_device() pattern in [2] and [3] in error unwinding path.
[1] https://elixir.bootlin.com/linux/v5.9-rc8/source/drivers/vdpa/vdpa.c [2] https://elixir.bootlin.com/linux/v5.9- rc8/source/drivers/vdpa/ifcvf/ifcvf_main.c#L475 [3] https://elixir.bootlin.com/linux/v5.9- rc8/source/drivers/vdpa/mlx5/net/mlx5_vnet.c#L1967
thanks,
greg k-h
IMHO we need to stay with the two step registration process that we currently have (initialize then add) so that the driver writer knows if they need to explicitly free the memory allocated for auxillary_device. Sound folks have indicated that this really helps their flow also. Greg asked to have these two functions fully commented with kernel-doc headers, which has been done.
Without enforcing an "auxillary_object" that contains just an auxillary_device and a void pointer, we cannot do the allocation of memory in the bus infrastructure without breaking the container_of functionality.
-DaveE
On Thu, Oct 08, 2020 at 12:38:00AM -0700, Dan Williams wrote:
On Thu, Oct 8, 2020 at 12:01 AM Leon Romanovsky leon@kernel.org wrote: [..]
All stated above is my opinion, it can be different from yours.
Yes, but we need to converge to move this forward. Jason was involved in the current organization for registration, Greg was angling for this to be core functionality. I have use cases outside of RDMA and netdev. Parav was ok with the current organization. The SOF folks already have a proposed incorporation of it. The argument I am hearing is that "this registration api seems hard for driver writers" when we have several driver writers who have already taken a look and can make it work. If you want to follow on with a simpler wrappers for your use case, great, but I do not yet see anyone concurring with your opinion that the current organization is irretrievably broken or too obscure to use.
Can it be that I'm first one to use this bus for very large driver (>120K LOC) that has 5 different ->probe() flows?
For example, this https://lore.kernel.org/linux-rdma/20201006172317.GN1874917@unreal/ hints to me that this bus wasn't used with anything complex as it was initially intended.
And regarding registration, I said many times that init()/add() scheme is ok, the inability to call to uninit() after add() failure is not ok from my point of view.
Thanks
On Thu, Oct 8, 2020 at 1:00 AM Leon Romanovsky leon@kernel.org wrote:
On Thu, Oct 08, 2020 at 12:38:00AM -0700, Dan Williams wrote:
On Thu, Oct 8, 2020 at 12:01 AM Leon Romanovsky leon@kernel.org wrote: [..]
All stated above is my opinion, it can be different from yours.
Yes, but we need to converge to move this forward. Jason was involved in the current organization for registration, Greg was angling for this to be core functionality. I have use cases outside of RDMA and netdev. Parav was ok with the current organization. The SOF folks already have a proposed incorporation of it. The argument I am hearing is that "this registration api seems hard for driver writers" when we have several driver writers who have already taken a look and can make it work. If you want to follow on with a simpler wrappers for your use case, great, but I do not yet see anyone concurring with your opinion that the current organization is irretrievably broken or too obscure to use.
Can it be that I'm first one to use this bus for very large driver (>120K LOC) that has 5 different ->probe() flows?
For example, this https://lore.kernel.org/linux-rdma/20201006172317.GN1874917@unreal/ hints to me that this bus wasn't used with anything complex as it was initially intended.
I missed that. Yes, I agree that's broken.
And regarding registration, I said many times that init()/add() scheme is ok, the inability to call to uninit() after add() failure is not ok from my point of view.
Ok, I got to the wrong conclusion about your position.
-----Original Message----- From: Leon Romanovsky leon@kernel.org Sent: Thursday, October 8, 2020 1:00 AM To: Williams, Dan J dan.j.williams@intel.com Cc: Ertman, David M david.m.ertman@intel.com; Parav Pandit parav@nvidia.com; Pierre-Louis Bossart <pierre- louis.bossart@linux.intel.com>; alsa-devel@alsa-project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux- rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; Jason Gunthorpe jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Thu, Oct 08, 2020 at 12:38:00AM -0700, Dan Williams wrote:
On Thu, Oct 8, 2020 at 12:01 AM Leon Romanovsky leon@kernel.org
wrote:
[..]
All stated above is my opinion, it can be different from yours.
Yes, but we need to converge to move this forward. Jason was involved in the current organization for registration, Greg was angling for this to be core functionality. I have use cases outside of RDMA and netdev. Parav was ok with the current organization. The SOF folks already have a proposed incorporation of it. The argument I am hearing is that "this registration api seems hard for driver writers" when we have several driver writers who have already taken a look and can make it work. If you want to follow on with a simpler wrappers for your use case, great, but I do not yet see anyone concurring with your opinion that the current organization is irretrievably broken or too obscure to use.
Can it be that I'm first one to use this bus for very large driver (>120K LOC) that has 5 different ->probe() flows?
For example, this https://lore.kernel.org/linux- rdma/20201006172317.GN1874917@unreal/ hints to me that this bus wasn't used with anything complex as it was initially intended.
And regarding registration, I said many times that init()/add() scheme is ok, the inability to call to uninit() after add() failure is not ok from my point of view.
So, to address your concern of not being able to call an uninit after a add failure I can break the unregister flow into two steps also. An uninit and a delete to mirror the registration process's init and add.
Would this make the registration and un-registration flow acceptable?
-DaveE
Thanks
On Thu, Oct 08, 2020 at 04:42:48PM +0000, Ertman, David M wrote:
-----Original Message----- From: Leon Romanovsky leon@kernel.org Sent: Thursday, October 8, 2020 1:00 AM To: Williams, Dan J dan.j.williams@intel.com Cc: Ertman, David M david.m.ertman@intel.com; Parav Pandit parav@nvidia.com; Pierre-Louis Bossart <pierre- louis.bossart@linux.intel.com>; alsa-devel@alsa-project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux- rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; Jason Gunthorpe jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Thu, Oct 08, 2020 at 12:38:00AM -0700, Dan Williams wrote:
On Thu, Oct 8, 2020 at 12:01 AM Leon Romanovsky leon@kernel.org
wrote:
[..]
All stated above is my opinion, it can be different from yours.
Yes, but we need to converge to move this forward. Jason was involved in the current organization for registration, Greg was angling for this to be core functionality. I have use cases outside of RDMA and netdev. Parav was ok with the current organization. The SOF folks already have a proposed incorporation of it. The argument I am hearing is that "this registration api seems hard for driver writers" when we have several driver writers who have already taken a look and can make it work. If you want to follow on with a simpler wrappers for your use case, great, but I do not yet see anyone concurring with your opinion that the current organization is irretrievably broken or too obscure to use.
Can it be that I'm first one to use this bus for very large driver (>120K LOC) that has 5 different ->probe() flows?
For example, this https://lore.kernel.org/linux- rdma/20201006172317.GN1874917@unreal/ hints to me that this bus wasn't used with anything complex as it was initially intended.
And regarding registration, I said many times that init()/add() scheme is ok, the inability to call to uninit() after add() failure is not ok from my point of view.
So, to address your concern of not being able to call an uninit after a add failure I can break the unregister flow into two steps also. An uninit and a delete to mirror the registration process's init and add.
Would this make the registration and un-registration flow acceptable?
Yes, sure.
-DaveE
Thanks
-----Original Message----- From: Dan Williams dan.j.williams@intel.com Sent: Wednesday, October 7, 2020 11:32 PM To: Leon Romanovsky leon@kernel.org Cc: Ertman, David M david.m.ertman@intel.com; Parav Pandit parav@nvidia.com; Pierre-Louis Bossart <pierre- louis.bossart@linux.intel.com>; alsa-devel@alsa-project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux- rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; Jason Gunthorpe jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Wed, Oct 7, 2020 at 10:21 PM Leon Romanovsky leon@kernel.org wrote:
On Wed, Oct 07, 2020 at 08:46:45PM +0000, Ertman, David M wrote:
-----Original Message----- From: Parav Pandit parav@nvidia.com Sent: Wednesday, October 7, 2020 1:17 PM To: Leon Romanovsky leon@kernel.org; Ertman, David M david.m.ertman@intel.com Cc: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com; alsa- devel@alsa-project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux-rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; Jason Gunthorpe jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org;
Williams,
Dan J dan.j.williams@intel.com; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: RE: [PATCH v2 1/6] Add ancillary bus support
From: Leon Romanovsky leon@kernel.org Sent: Thursday, October 8, 2020 12:56 AM
> This API is partially obscures low level driver-core code and needs > to provide clear and proper abstractions without need to
remember
> about put_device. There is already _add() interface why don't you
do
> put_device() in it? >
The pushback Pierre is referring to was during our mid-tier internal review. It was primarily a concern of Parav as I recall, so he can
speak to
his
reasoning.
What we originally had was a single API call (ancillary_device_register) that started with a call to device_initialize(), and every error path out of the function
performed a
put_device().
Is this the model you have in mind?
I don't like this flow: ancillary_device_initialize() if (ancillary_ancillary_device_add()) { put_device(....) ancillary_device_unregister()
Calling device_unregister() is incorrect, because add() wasn't
successful.
Only put_device() or a wrapper ancillary_device_put() is necessary.
return err; }
And prefer this flow: ancillary_device_initialize() if (ancillary_device_add()) { ancillary_device_unregister()
This is incorrect and a clear deviation from the current core APIs that
adds the
confusion.
return err; }
In this way, the ancillary users won't need to do non-intuitive
put_device();
Below is most simple, intuitive and matching with core APIs for name
and
design pattern wise. init() { err = ancillary_device_initialize(); if (err) return ret;
err = ancillary_device_add(); if (ret) goto err_unwind; err = some_foo(); if (err) goto err_foo; return 0;err_foo: ancillary_device_del(adev); err_unwind: ancillary_device_put(adev->dev); return err; }
cleanup() { ancillary_device_de(adev); ancillary_device_put(adev); /* It is common to have a one wrapper for this as ancillary_device_unregister(). * This will match with core device_unregister() that has precise documentation. * but given fact that init() code need proper error unwinding, like above, * it make sense to have two APIs, and no need to export another symbol for unregister(). * This pattern is very easy to audit and code. */ }
I like this flow +1
But ... since the init() function is performing both device_init and device_add - it should probably be called ancillary_device_register, and we are back to a single exported API for both register and unregister.
At that point, do we need wrappers on the primitives init, add, del, and put?
Let me summarize.
- You are not providing driver/core API but simplification and obfuscation
of basic primitives and structures. This is new layer. There is no room for a claim that we must to follow internal API.
Yes, this a driver core api, Greg even questioned why it was in drivers/bus instead of drivers/base which I think makes sense.
Will move to drivers/base with next patch set.
-DaveE
-----Original Message----- From: Leon Romanovsky leon@kernel.org Sent: Tuesday, October 6, 2020 10:03 AM To: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Cc: Ertman, David M david.m.ertman@intel.com; alsa-devel@alsa- project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux- rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; Williams, Dan J dan.j.williams@intel.com; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Tue, Oct 06, 2020 at 10:18:07AM -0500, Pierre-Louis Bossart wrote:
Thanks for the review Leon.
[...]
+EXPORT_SYMBOL_GPL(__ancillary_device_add);
+static int ancillary_probe_driver(struct device *dev) +{
- struct ancillary_driver *ancildrv = to_ancillary_drv(dev->driver);
- struct ancillary_device *ancildev = to_ancillary_dev(dev);
- int ret;
- ret = dev_pm_domain_attach(dev, true);
- if (ret) {
dev_warn(dev, "Failed to attach to PM Domain : %d\n", ret);return ret;- }
- ret = ancildrv->probe(ancildev, ancillary_match_id(ancildrv-
id_table, ancildev));
I don't think that you need to call ->probe() if ancillary_match_id() returned NULL and probably that check should be done before dev_pm_domain_attach().
we'll look into this.
AKAIK, this callback is only accessed from the bus subsystem after a successful return from ancillary_match().
-DaveE
-----Original Message----- From: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Sent: Tuesday, October 6, 2020 8:18 AM To: Leon Romanovsky leon@kernel.org; Ertman, David M david.m.ertman@intel.com Cc: alsa-devel@alsa-project.org; parav@mellanox.com; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux-rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; Williams, Dan J dan.j.williams@intel.com; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: Re: [PATCH v2 1/6] Add ancillary bus support
Thanks for the review Leon.
Add support for the Ancillary Bus, ancillary_device and ancillary_driver. It enables drivers to create an ancillary_device and bind an ancillary_driver to it.
I was under impression that this name is going to be changed.
It's part of the opens stated in the cover letter.
[...]
- const struct my_driver my_drv = {
.ancillary_drv = {.driver = {.name = "myancillarydrv",Why do we need to give control over driver name to the driver authors? It can be problematic if author puts name that already exists.
Good point. When I used the ancillary_devices for my own SoundWire test, the driver name didn't seem specifically meaningful but needed to be set to something, what mattered was the id_table. Just thinking aloud, maybe we can add prefixing with KMOD_BUILD, as we've done already to avoid collisions between device names?
[...]
Since we have eliminated all IDA type things out of the bus infrastructure, I like the idea of prefixing the driver name with KBUILD_MODNAME through a macro front. Since a parent driver can register more than one ancillary driver, this allow the parent to have an internally meaningful name while still ensuring its uniqueness.
-DaveE
On Mon, Oct 05, 2020 at 11:24:41AM -0700, Dave Ertman wrote:
Add support for the Ancillary Bus, ancillary_device and ancillary_driver. It enables drivers to create an ancillary_device and bind an ancillary_driver to it.
The bus supports probe/remove shutdown and suspend/resume callbacks. Each ancillary_device has a unique string based id; driver binds to an ancillary_device based on this id through the bus.
Co-developed-by: Kiran Patil kiran.patil@intel.com Signed-off-by: Kiran Patil kiran.patil@intel.com Co-developed-by: Ranjani Sridharan ranjani.sridharan@linux.intel.com Signed-off-by: Ranjani Sridharan ranjani.sridharan@linux.intel.com Co-developed-by: Fred Oh fred.oh@linux.intel.com Signed-off-by: Fred Oh fred.oh@linux.intel.com Reviewed-by: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Reviewed-by: Shiraz Saleem shiraz.saleem@intel.com Reviewed-by: Parav Pandit parav@mellanox.com Reviewed-by: Dan Williams dan.j.williams@intel.com Signed-off-by: Dave Ertman david.m.ertman@intel.com
<...>
+/**
- __ancillary_driver_register - register a driver for ancillary bus devices
- @ancildrv: ancillary_driver structure
- @owner: owning module/driver
- */
+int __ancillary_driver_register(struct ancillary_driver *ancildrv, struct module *owner) +{
- if (WARN_ON(!ancildrv->probe) || WARN_ON(!ancildrv->remove) ||
WARN_ON(!ancildrv->shutdown) || WARN_ON(!ancildrv->id_table))return -EINVAL;
In our driver ->shutdown is empty, it will be best if ancillary bus will do "if (->remove) ..->remove()" pattern.
- ancildrv->driver.owner = owner;
- ancildrv->driver.bus = &ancillary_bus_type;
- ancildrv->driver.probe = ancillary_probe_driver;
- ancildrv->driver.remove = ancillary_remove_driver;
- ancildrv->driver.shutdown = ancillary_shutdown_driver;
I think that this part is wrong, probe/remove/shutdown functions should come from ancillary_bus_type. You are overwriting private device_driver callbacks that makes impossible to make container_of of ancillary_driver to chain operations.
- return driver_register(&ancildrv->driver);
+} +EXPORT_SYMBOL_GPL(__ancillary_driver_register);
Thanks
Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Mon, Oct 05, 2020 at 11:24:41AM -0700, Dave Ertman wrote:
Add support for the Ancillary Bus, ancillary_device and ancillary_driver. It enables drivers to create an ancillary_device and bind an ancillary_driver to it.
The bus supports probe/remove shutdown and suspend/resume callbacks. Each ancillary_device has a unique string based id; driver binds to an ancillary_device based on this id through the bus.
Co-developed-by: Kiran Patil kiran.patil@intel.com Signed-off-by: Kiran Patil kiran.patil@intel.com Co-developed-by: Ranjani Sridharan ranjani.sridharan@linux.intel.com Signed-off-by: Ranjani Sridharan ranjani.sridharan@linux.intel.com Co-developed-by: Fred Oh fred.oh@linux.intel.com Signed-off-by: Fred Oh fred.oh@linux.intel.com Reviewed-by: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Reviewed-by: Shiraz Saleem shiraz.saleem@intel.com Reviewed-by: Parav Pandit parav@mellanox.com Reviewed-by: Dan Williams dan.j.williams@intel.com Signed-off-by: Dave Ertman david.m.ertman@intel.com
<...>
+/**
- __ancillary_driver_register - register a driver for ancillary bus
+devices
- @ancildrv: ancillary_driver structure
- @owner: owning module/driver
- */
+int __ancillary_driver_register(struct ancillary_driver *ancildrv, +struct module *owner) {
- if (WARN_ON(!ancildrv->probe) || WARN_ON(!ancildrv->remove) ||
WARN_ON(!ancildrv->shutdown) || WARN_ON(!ancildrv->id_table))return -EINVAL;In our driver ->shutdown is empty, it will be best if ancillary bus will do "if (-
remove) ..->remove()" pattern.
I prefer that too if its possible. We will look into it.
-----Original Message----- From: Leon Romanovsky leon@kernel.org Sent: Tuesday, October 6, 2020 10:23 AM To: Ertman, David M david.m.ertman@intel.com Cc: alsa-devel@alsa-project.org; tiwai@suse.de; broonie@kernel.org; linux- rdma@vger.kernel.org; jgg@nvidia.com; dledford@redhat.com; netdev@vger.kernel.org; davem@davemloft.net; kuba@kernel.org; gregkh@linuxfoundation.org; ranjani.sridharan@linux.intel.com; pierre- louis.bossart@linux.intel.com; fred.oh@linux.intel.com; parav@mellanox.com; Saleem, Shiraz shiraz.saleem@intel.com; Williams, Dan J dan.j.williams@intel.com; Patil, Kiran kiran.patil@intel.com Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Mon, Oct 05, 2020 at 11:24:41AM -0700, Dave Ertman wrote:
Add support for the Ancillary Bus, ancillary_device and ancillary_driver. It enables drivers to create an ancillary_device and bind an ancillary_driver to it.
The bus supports probe/remove shutdown and suspend/resume callbacks. Each ancillary_device has a unique string based id; driver binds to an ancillary_device based on this id through the bus.
Co-developed-by: Kiran Patil kiran.patil@intel.com Signed-off-by: Kiran Patil kiran.patil@intel.com Co-developed-by: Ranjani Sridharan ranjani.sridharan@linux.intel.com Signed-off-by: Ranjani Sridharan ranjani.sridharan@linux.intel.com Co-developed-by: Fred Oh fred.oh@linux.intel.com Signed-off-by: Fred Oh fred.oh@linux.intel.com Reviewed-by: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Reviewed-by: Shiraz Saleem shiraz.saleem@intel.com Reviewed-by: Parav Pandit parav@mellanox.com Reviewed-by: Dan Williams dan.j.williams@intel.com Signed-off-by: Dave Ertman david.m.ertman@intel.com
<...>
+/**
- __ancillary_driver_register - register a driver for ancillary bus devices
- @ancildrv: ancillary_driver structure
- @owner: owning module/driver
- */
+int __ancillary_driver_register(struct ancillary_driver *ancildrv, struct
module *owner)
+{
- if (WARN_ON(!ancildrv->probe) || WARN_ON(!ancildrv->remove)
||
WARN_ON(!ancildrv->shutdown) || WARN_ON(!ancildrv-id_table))
return -EINVAL;In our driver ->shutdown is empty, it will be best if ancillary bus will do "if (->remove) ..->remove()" pattern.
Yes, looking it over, only the probe needs to mandatory. I will change the others to the conditional model, and adjust the WARN_ONs.
- ancildrv->driver.owner = owner;
- ancildrv->driver.bus = &ancillary_bus_type;
- ancildrv->driver.probe = ancillary_probe_driver;
- ancildrv->driver.remove = ancillary_remove_driver;
- ancildrv->driver.shutdown = ancillary_shutdown_driver;
I think that this part is wrong, probe/remove/shutdown functions should come from ancillary_bus_type.
From checking other usage cases, this is the model that is used for probe, remove,
and shutdown in drivers. Here is the example from Greybus.
int greybus_register_driver(struct greybus_driver *driver, struct module *owner, const char *mod_name) { int retval;
if (greybus_disabled()) return -ENODEV;
driver->driver.bus = &greybus_bus_type; driver->driver.name = driver->name; driver->driver.probe = greybus_probe; driver->driver.remove = greybus_remove; driver->driver.owner = owner; driver->driver.mod_name = mod_name;
You are overwriting private device_driver callbacks that makes impossible to make container_of of ancillary_driver to chain operations.
I am sorry, you lost me here. you cannot perform container_of on the callbacks because they are pointers, but if you are referring to going from device_driver to the auxiliary_driver, that is what happens in auxiliary_probe_driver in the very beginning.
static int auxiliary_probe_driver(struct device *dev) 145 { 146 struct auxiliary_driver *auxdrv = to_auxiliary_drv(dev->driver); 147 struct auxiliary_device *auxdev = to_auxiliary_dev(dev);
Did I miss your meaning?
-DaveE
- return driver_register(&ancildrv->driver);
+} +EXPORT_SYMBOL_GPL(__ancillary_driver_register);
Thanks
On Thu, Oct 8, 2020 at 3:04 PM Ertman, David M david.m.ertman@intel.com wrote:
-----Original Message----- From: Leon Romanovsky leon@kernel.org Sent: Tuesday, October 6, 2020 10:23 AM To: Ertman, David M david.m.ertman@intel.com Cc: alsa-devel@alsa-project.org; tiwai@suse.de; broonie@kernel.org; linux- rdma@vger.kernel.org; jgg@nvidia.com; dledford@redhat.com; netdev@vger.kernel.org; davem@davemloft.net; kuba@kernel.org; gregkh@linuxfoundation.org; ranjani.sridharan@linux.intel.com; pierre- louis.bossart@linux.intel.com; fred.oh@linux.intel.com; parav@mellanox.com; Saleem, Shiraz shiraz.saleem@intel.com; Williams, Dan J dan.j.williams@intel.com; Patil, Kiran kiran.patil@intel.com Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Mon, Oct 05, 2020 at 11:24:41AM -0700, Dave Ertman wrote:
Add support for the Ancillary Bus, ancillary_device and ancillary_driver. It enables drivers to create an ancillary_device and bind an ancillary_driver to it.
The bus supports probe/remove shutdown and suspend/resume callbacks. Each ancillary_device has a unique string based id; driver binds to an ancillary_device based on this id through the bus.
Co-developed-by: Kiran Patil kiran.patil@intel.com Signed-off-by: Kiran Patil kiran.patil@intel.com Co-developed-by: Ranjani Sridharan ranjani.sridharan@linux.intel.com Signed-off-by: Ranjani Sridharan ranjani.sridharan@linux.intel.com Co-developed-by: Fred Oh fred.oh@linux.intel.com Signed-off-by: Fred Oh fred.oh@linux.intel.com Reviewed-by: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Reviewed-by: Shiraz Saleem shiraz.saleem@intel.com Reviewed-by: Parav Pandit parav@mellanox.com Reviewed-by: Dan Williams dan.j.williams@intel.com Signed-off-by: Dave Ertman david.m.ertman@intel.com
<...>
+/**
- __ancillary_driver_register - register a driver for ancillary bus devices
- @ancildrv: ancillary_driver structure
- @owner: owning module/driver
- */
+int __ancillary_driver_register(struct ancillary_driver *ancildrv, struct
module *owner)
+{
- if (WARN_ON(!ancildrv->probe) || WARN_ON(!ancildrv->remove)
||
WARN_ON(!ancildrv->shutdown) || WARN_ON(!ancildrv-id_table))
return -EINVAL;In our driver ->shutdown is empty, it will be best if ancillary bus will do "if (->remove) ..->remove()" pattern.
Yes, looking it over, only the probe needs to mandatory. I will change the others to the conditional model, and adjust the WARN_ONs.
- ancildrv->driver.owner = owner;
- ancildrv->driver.bus = &ancillary_bus_type;
- ancildrv->driver.probe = ancillary_probe_driver;
- ancildrv->driver.remove = ancillary_remove_driver;
- ancildrv->driver.shutdown = ancillary_shutdown_driver;
I think that this part is wrong, probe/remove/shutdown functions should come from ancillary_bus_type.
From checking other usage cases, this is the model that is used for probe, remove, and shutdown in drivers. Here is the example from Greybus.
int greybus_register_driver(struct greybus_driver *driver, struct module *owner, const char *mod_name) { int retval;
if (greybus_disabled()) return -ENODEV; driver->driver.bus = &greybus_bus_type; driver->driver.name = driver->name; driver->driver.probe = greybus_probe; driver->driver.remove = greybus_remove; driver->driver.owner = owner; driver->driver.mod_name = mod_name;You are overwriting private device_driver callbacks that makes impossible to make container_of of ancillary_driver to chain operations.
I am sorry, you lost me here. you cannot perform container_of on the callbacks because they are pointers, but if you are referring to going from device_driver to the auxiliary_driver, that is what happens in auxiliary_probe_driver in the very beginning.
static int auxiliary_probe_driver(struct device *dev) 145 { 146 struct auxiliary_driver *auxdrv = to_auxiliary_drv(dev->driver); 147 struct auxiliary_device *auxdev = to_auxiliary_dev(dev);
Did I miss your meaning?
I think you're misunderstanding the cases when the bus_type.{probe,remove} is used vs the driver.{probe,remove} callbacks. The bus_type callbacks are to implement a pattern where the 'probe' and 'remove' method are typed to the bus device type. For example 'struct pci_dev *' instead of raw 'struct device *'. See this conversion of dax bus as an example of going from raw 'struct device *' typed probe/remove to dax-device typed probe/remove:
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?...
- ancildrv->driver.owner = owner;
- ancildrv->driver.bus = &ancillary_bus_type;
- ancildrv->driver.probe = ancillary_probe_driver;
- ancildrv->driver.remove = ancillary_remove_driver;
- ancildrv->driver.shutdown = ancillary_shutdown_driver;
I think that this part is wrong, probe/remove/shutdown functions should come from ancillary_bus_type.
From checking other usage cases, this is the model that is used for probe, remove, and shutdown in drivers. Here is the example from Greybus.
int greybus_register_driver(struct greybus_driver *driver, struct module *owner, const char *mod_name) { int retval;
if (greybus_disabled()) return -ENODEV; driver->driver.bus = &greybus_bus_type; driver->driver.name = driver->name; driver->driver.probe = greybus_probe; driver->driver.remove = greybus_remove; driver->driver.owner = owner; driver->driver.mod_name = mod_name;You are overwriting private device_driver callbacks that makes impossible to make container_of of ancillary_driver to chain operations.
I am sorry, you lost me here. you cannot perform container_of on the callbacks because they are pointers, but if you are referring to going from device_driver to the auxiliary_driver, that is what happens in auxiliary_probe_driver in the very beginning.
static int auxiliary_probe_driver(struct device *dev) 145 { 146 struct auxiliary_driver *auxdrv = to_auxiliary_drv(dev->driver); 147 struct auxiliary_device *auxdev = to_auxiliary_dev(dev);
Did I miss your meaning?
I think you're misunderstanding the cases when the bus_type.{probe,remove} is used vs the driver.{probe,remove} callbacks. The bus_type callbacks are to implement a pattern where the 'probe' and 'remove' method are typed to the bus device type. For example 'struct pci_dev *' instead of raw 'struct device *'. See this conversion of dax bus as an example of going from raw 'struct device *' typed probe/remove to dax-device typed probe/remove:
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?...
Thanks Dan for the reference, very useful. This doesn't look like a a big change to implement, just wondering about the benefits and drawbacks, if any? I am a bit confused here.
First, was the initial pattern wrong as Leon asserted it? Such code exists in multiple examples in the kernel and there's nothing preventing the use of container_of that I can think of. Put differently, if this code was wrong then there are other existing buses that need to be updated.
Second, what additional functionality does this move from driver to bus_type provide? The commit reference just states 'In preparation for introducing seed devices the dax-bus core needs to be able to intercept ->probe() and ->remove() operations", but that doesn't really help me figure out what 'intercept' means. Would you mind elaborating?
And last, the existing probe function does calls dev_pm_domain_attach():
static int ancillary_probe_driver(struct device *dev) { struct ancillary_driver *ancildrv = to_ancillary_drv(dev->driver); struct ancillary_device *ancildev = to_ancillary_dev(dev); int ret;
ret = dev_pm_domain_attach(dev, true);
So the need to access the raw device still exists. Is this still legit if the probe() is moved to the bus_type structure?
I have no objection to this change if it preserves the same functionality and possibly extends it, just wanted to better understand the reasons for the change and in which cases the bus probe() makes more sense than a driver probe().
Thanks for enlightening the rest of us!
On Fri, Oct 9, 2020 at 7:27 AM Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com wrote:
- ancildrv->driver.owner = owner;
- ancildrv->driver.bus = &ancillary_bus_type;
- ancildrv->driver.probe = ancillary_probe_driver;
- ancildrv->driver.remove = ancillary_remove_driver;
- ancildrv->driver.shutdown = ancillary_shutdown_driver;
I think that this part is wrong, probe/remove/shutdown functions should come from ancillary_bus_type.
From checking other usage cases, this is the model that is used for probe, remove, and shutdown in drivers. Here is the example from Greybus.
int greybus_register_driver(struct greybus_driver *driver, struct module *owner, const char *mod_name) { int retval;
if (greybus_disabled()) return -ENODEV; driver->driver.bus = &greybus_bus_type; driver->driver.name = driver->name; driver->driver.probe = greybus_probe; driver->driver.remove = greybus_remove; driver->driver.owner = owner; driver->driver.mod_name = mod_name;You are overwriting private device_driver callbacks that makes impossible to make container_of of ancillary_driver to chain operations.
I am sorry, you lost me here. you cannot perform container_of on the callbacks because they are pointers, but if you are referring to going from device_driver to the auxiliary_driver, that is what happens in auxiliary_probe_driver in the very beginning.
static int auxiliary_probe_driver(struct device *dev) 145 { 146 struct auxiliary_driver *auxdrv = to_auxiliary_drv(dev->driver); 147 struct auxiliary_device *auxdev = to_auxiliary_dev(dev);
Did I miss your meaning?
I think you're misunderstanding the cases when the bus_type.{probe,remove} is used vs the driver.{probe,remove} callbacks. The bus_type callbacks are to implement a pattern where the 'probe' and 'remove' method are typed to the bus device type. For example 'struct pci_dev *' instead of raw 'struct device *'. See this conversion of dax bus as an example of going from raw 'struct device *' typed probe/remove to dax-device typed probe/remove:
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?...
Thanks Dan for the reference, very useful. This doesn't look like a a big change to implement, just wondering about the benefits and drawbacks, if any? I am a bit confused here.
First, was the initial pattern wrong as Leon asserted it? Such code exists in multiple examples in the kernel and there's nothing preventing the use of container_of that I can think of. Put differently, if this code was wrong then there are other existing buses that need to be updated.
Second, what additional functionality does this move from driver to bus_type provide? The commit reference just states 'In preparation for introducing seed devices the dax-bus core needs to be able to intercept ->probe() and ->remove() operations", but that doesn't really help me figure out what 'intercept' means. Would you mind elaborating?
And last, the existing probe function does calls dev_pm_domain_attach():
static int ancillary_probe_driver(struct device *dev) { struct ancillary_driver *ancildrv = to_ancillary_drv(dev->driver); struct ancillary_device *ancildev = to_ancillary_dev(dev); int ret;
ret = dev_pm_domain_attach(dev, true);So the need to access the raw device still exists. Is this still legit if the probe() is moved to the bus_type structure?
Sure, of course.
I have no objection to this change if it preserves the same functionality and possibly extends it, just wanted to better understand the reasons for the change and in which cases the bus probe() makes more sense than a driver probe().
Thanks for enlightening the rest of us!
tl;dr: The ops set by the device driver should never be overwritten by the bus, the bus can only wrap them in its own ops.
The reason to use the bus_type is because the bus type is the only agent that knows both how to convert a raw 'struct device *' to the bus's native type, and how to convert a raw 'struct device_driver *' to the bus's native driver type. The driver core does:
if (dev->bus->probe) { ret = dev->bus->probe(dev); } else if (drv->probe) { ret = drv->probe(dev); }
...so that the bus has the first priority for probing a device / wrapping the native driver ops. The bus ->probe, in addition to optionally performing some bus specific pre-work, lets the bus upcast the device to bus-native type.
The bus also knows the types of drivers that will be registered to it, so the bus can upcast the dev->driver to the native type.
So with bus_type based driver ops driver authors can do:
struct auxiliary_device_driver auxdrv { .probe = fn(struct auxiliary_device *, <any aux bus custom probe arguments>) };
auxiliary_driver_register(&auxdrv); <-- the core code can hide bus details
Without bus_type the driver author would need to do:
struct auxiliary_device_driver auxdrv { .drv = { .probe = fn(struct device *), <-- no opportunity for bus specific probe args .bus = &auxilary_bus_type, <-- unnecessary export to device drivers }, };
driver_register(&auxdrv.drv)
- ancildrv->driver.owner = owner;
- ancildrv->driver.bus = &ancillary_bus_type;
- ancildrv->driver.probe = ancillary_probe_driver;
- ancildrv->driver.remove = ancillary_remove_driver;
- ancildrv->driver.shutdown = ancillary_shutdown_driver;
I think that this part is wrong, probe/remove/shutdown functions should come from ancillary_bus_type.
From checking other usage cases, this is the model that is used for probe, remove, and shutdown in drivers. Here is the example from Greybus.
int greybus_register_driver(struct greybus_driver *driver, struct module *owner, const char *mod_name) { int retval;
if (greybus_disabled()) return -ENODEV; driver->driver.bus = &greybus_bus_type; driver->driver.name = driver->name; driver->driver.probe = greybus_probe; driver->driver.remove = greybus_remove; driver->driver.owner = owner; driver->driver.mod_name = mod_name;You are overwriting private device_driver callbacks that makes impossible to make container_of of ancillary_driver to chain operations.
I am sorry, you lost me here. you cannot perform container_of on the callbacks because they are pointers, but if you are referring to going from device_driver to the auxiliary_driver, that is what happens in auxiliary_probe_driver in the very beginning.
static int auxiliary_probe_driver(struct device *dev) 145 { 146 struct auxiliary_driver *auxdrv = to_auxiliary_drv(dev->driver); 147 struct auxiliary_device *auxdev = to_auxiliary_dev(dev);
Did I miss your meaning?
I think you're misunderstanding the cases when the bus_type.{probe,remove} is used vs the driver.{probe,remove} callbacks. The bus_type callbacks are to implement a pattern where the 'probe' and 'remove' method are typed to the bus device type. For example 'struct pci_dev *' instead of raw 'struct device *'. See this conversion of dax bus as an example of going from raw 'struct device *' typed probe/remove to dax-device typed probe/remove:
https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?...
Thanks Dan for the reference, very useful. This doesn't look like a a big change to implement, just wondering about the benefits and drawbacks, if any? I am a bit confused here.
First, was the initial pattern wrong as Leon asserted it? Such code exists in multiple examples in the kernel and there's nothing preventing the use of container_of that I can think of. Put differently, if this code was wrong then there are other existing buses that need to be updated.
Second, what additional functionality does this move from driver to bus_type provide? The commit reference just states 'In preparation for introducing seed devices the dax-bus core needs to be able to intercept ->probe() and ->remove() operations", but that doesn't really help me figure out what 'intercept' means. Would you mind elaborating?
And last, the existing probe function does calls dev_pm_domain_attach():
static int ancillary_probe_driver(struct device *dev) { struct ancillary_driver *ancildrv = to_ancillary_drv(dev->driver); struct ancillary_device *ancildev = to_ancillary_dev(dev); int ret;
ret = dev_pm_domain_attach(dev, true);So the need to access the raw device still exists. Is this still legit if the probe() is moved to the bus_type structure?
Sure, of course.
I have no objection to this change if it preserves the same functionality and possibly extends it, just wanted to better understand the reasons for the change and in which cases the bus probe() makes more sense than a driver probe().
Thanks for enlightening the rest of us!
tl;dr: The ops set by the device driver should never be overwritten by the bus, the bus can only wrap them in its own ops.
The reason to use the bus_type is because the bus type is the only agent that knows both how to convert a raw 'struct device *' to the bus's native type, and how to convert a raw 'struct device_driver *' to the bus's native driver type. The driver core does:
if (dev->bus->probe) { ret = dev->bus->probe(dev); } else if (drv->probe) { ret = drv->probe(dev); }...so that the bus has the first priority for probing a device / wrapping the native driver ops. The bus ->probe, in addition to optionally performing some bus specific pre-work, lets the bus upcast the device to bus-native type.
The bus also knows the types of drivers that will be registered to it, so the bus can upcast the dev->driver to the native type.
So with bus_type based driver ops driver authors can do:
struct auxiliary_device_driver auxdrv { .probe = fn(struct auxiliary_device *, <any aux bus custom probe arguments>) };
auxiliary_driver_register(&auxdrv); <-- the core code can hide bus details
Without bus_type the driver author would need to do:
struct auxiliary_device_driver auxdrv { .drv = { .probe = fn(struct device *), <-- no opportunity for bus specific probe args .bus = &auxilary_bus_type, <-- unnecessary export to device drivers }, };
driver_register(&auxdrv.drv)
Thanks Dan, I appreciate the explanation.
I guess the misunderstanding on my side was that in practice the drivers only declare a probe at the auxiliary level:
struct auxiliary_device_driver auxdrv { .drv = { .name = "my driver" <<< .probe not set here. } .probe = fn(struct auxiliary_device *, int id), }
It looks indeed cleaner with your suggestion. DaveE and I were talking about this moments ago and made the change, will be testing later today.
Again thanks for the write-up and have a nice week-end.
-----Original Message----- From: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Sent: Friday, October 9, 2020 12:39 PM To: Williams, Dan J dan.j.williams@intel.com Cc: Ertman, David M david.m.ertman@intel.com; alsa-devel@alsa- project.org; parav@mellanox.com; Leon Romanovsky leon@kernel.org; tiwai@suse.de; netdev@vger.kernel.org; ranjani.sridharan@linux.intel.com; fred.oh@linux.intel.com; linux-rdma@vger.kernel.org; dledford@redhat.com; broonie@kernel.org; jgg@nvidia.com; gregkh@linuxfoundation.org; kuba@kernel.org; Saleem, Shiraz shiraz.saleem@intel.com; davem@davemloft.net; Patil, Kiran kiran.patil@intel.com Subject: Re: [PATCH v2 1/6] Add ancillary bus support
> + > + ancildrv->driver.owner = owner; > + ancildrv->driver.bus = &ancillary_bus_type; > + ancildrv->driver.probe = ancillary_probe_driver; > + ancildrv->driver.remove = ancillary_remove_driver; > + ancildrv->driver.shutdown = ancillary_shutdown_driver; > +
I think that this part is wrong, probe/remove/shutdown functions
should
come from ancillary_bus_type.
From checking other usage cases, this is the model that is used for
probe, remove,
and shutdown in drivers. Here is the example from Greybus.
int greybus_register_driver(struct greybus_driver *driver, struct
module *owner,
const char *mod_name){ int retval;
if (greybus_disabled()) return -ENODEV; driver->driver.bus = &greybus_bus_type; driver->driver.name = driver->name; driver->driver.probe = greybus_probe; driver->driver.remove = greybus_remove; driver->driver.owner = owner; driver->driver.mod_name = mod_name;You are overwriting private device_driver callbacks that makes impossible to make container_of of
ancillary_driver
to chain operations.
I am sorry, you lost me here. you cannot perform container_of on the
callbacks
because they are pointers, but if you are referring to going from
device_driver
to the auxiliary_driver, that is what happens in auxiliary_probe_driver
in the
very beginning.
static int auxiliary_probe_driver(struct device *dev) 145 { 146 struct auxiliary_driver *auxdrv = to_auxiliary_drv(dev->driver); 147 struct auxiliary_device *auxdev = to_auxiliary_dev(dev);
Did I miss your meaning?
I think you're misunderstanding the cases when the bus_type.{probe,remove} is used vs the driver.{probe,remove} callbacks. The bus_type callbacks are to implement a pattern where the 'probe' and 'remove' method are typed to the bus device type. For example 'struct pci_dev *' instead of raw 'struct device *'. See this conversion of dax bus as an example of going from raw 'struct device *' typed probe/remove to dax-device typed probe/remove:
next.git/commit/?id=75797273189d
Thanks Dan for the reference, very useful. This doesn't look like a a big change to implement, just wondering about the benefits and drawbacks, if any? I am a bit confused here.
First, was the initial pattern wrong as Leon asserted it? Such code exists in multiple examples in the kernel and there's nothing preventing the use of container_of that I can think of. Put differently, if this code was wrong then there are other existing buses that need to be
updated.
Second, what additional functionality does this move from driver to bus_type provide? The commit reference just states 'In preparation for introducing seed devices the dax-bus core needs to be able to intercept ->probe() and ->remove() operations", but that doesn't really help me figure out what 'intercept' means. Would you mind elaborating?
And last, the existing probe function does calls dev_pm_domain_attach():
static int ancillary_probe_driver(struct device *dev) { struct ancillary_driver *ancildrv = to_ancillary_drv(dev->driver); struct ancillary_device *ancildev = to_ancillary_dev(dev); int ret;
ret = dev_pm_domain_attach(dev, true);So the need to access the raw device still exists. Is this still legit if the probe() is moved to the bus_type structure?
Sure, of course.
I have no objection to this change if it preserves the same functionality and possibly extends it, just wanted to better understand the reasons for the change and in which cases the bus probe() makes
more
sense than a driver probe().
Thanks for enlightening the rest of us!
tl;dr: The ops set by the device driver should never be overwritten by the bus, the bus can only wrap them in its own ops.
The reason to use the bus_type is because the bus type is the only agent that knows both how to convert a raw 'struct device *' to the bus's native type, and how to convert a raw 'struct device_driver *' to the bus's native driver type. The driver core does:
if (dev->bus->probe) { ret = dev->bus->probe(dev); } else if (drv->probe) { ret = drv->probe(dev); }...so that the bus has the first priority for probing a device / wrapping the native driver ops. The bus ->probe, in addition to optionally performing some bus specific pre-work, lets the bus upcast the device to bus-native type.
The bus also knows the types of drivers that will be registered to it, so the bus can upcast the dev->driver to the native type.
So with bus_type based driver ops driver authors can do:
struct auxiliary_device_driver auxdrv { .probe = fn(struct auxiliary_device *, <any aux bus custom probe
arguments>)
};
auxiliary_driver_register(&auxdrv); <-- the core code can hide bus details
Without bus_type the driver author would need to do:
struct auxiliary_device_driver auxdrv { .drv = { .probe = fn(struct device *), <-- no opportunity for bus specific probe args .bus = &auxilary_bus_type, <-- unnecessary export to device drivers }, };
driver_register(&auxdrv.drv)
Thanks Dan, I appreciate the explanation.
I guess the misunderstanding on my side was that in practice the drivers only declare a probe at the auxiliary level:
struct auxiliary_device_driver auxdrv { .drv = { .name = "my driver" <<< .probe not set here. } .probe = fn(struct auxiliary_device *, int id), }
It looks indeed cleaner with your suggestion. DaveE and I were talking about this moments ago and made the change, will be testing later today.
Again thanks for the write-up and have a nice week-end.
Like Pierre said, I have already changed the probe, remove, and shutdown callbacks into the bus_type.
But it should be noted that you are not supposed to have these callbacks in both the auxdrv->drv->* and in the bus->*.
in drivers/base/driver.c line 158 it checks for this:
if ((drv->bus->probe && drv->probe) || (drv->bus->remove && drv->remove) || (drv->bus->shutdown && drv->shutdown)) pr_warn("Driver '%s' needs updating - please use " "bus_type methods\n", drv->name);
So, changing to the bus_type for these is the right thing to do, but driver writers need to make sure that auxdrv->drv->[probe|remove|shutdown] are NULL.
-DaveE
On Mon, Oct 05, 2020 at 11:24:41AM -0700, Dave Ertman wrote:
Add support for the Ancillary Bus, ancillary_device and ancillary_driver. It enables drivers to create an ancillary_device and bind an ancillary_driver to it.
The bus supports probe/remove shutdown and suspend/resume callbacks. Each ancillary_device has a unique string based id; driver binds to an ancillary_device based on this id through the bus.
Co-developed-by: Kiran Patil kiran.patil@intel.com Signed-off-by: Kiran Patil kiran.patil@intel.com Co-developed-by: Ranjani Sridharan ranjani.sridharan@linux.intel.com Signed-off-by: Ranjani Sridharan ranjani.sridharan@linux.intel.com Co-developed-by: Fred Oh fred.oh@linux.intel.com Signed-off-by: Fred Oh fred.oh@linux.intel.com Reviewed-by: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Reviewed-by: Shiraz Saleem shiraz.saleem@intel.com Reviewed-by: Parav Pandit parav@mellanox.com Reviewed-by: Dan Williams dan.j.williams@intel.com Signed-off-by: Dave Ertman david.m.ertman@intel.com
<...>
+static const struct ancillary_device_id *ancillary_match_id(const struct ancillary_device_id *id,
const struct ancillary_device *ancildev)+{
- while (id->name[0]) {
const char *p = strrchr(dev_name(&ancildev->dev), '.');int match_size;if (!p) {id++;continue;}match_size = p - dev_name(&ancildev->dev);/* use dev_name(&ancildev->dev) prefix before last '.' char to match to */if (!strncmp(dev_name(&ancildev->dev), id->name, match_size))
This check is wrong, it causes to wrong matching if strlen(id->name) > match_size In my case, the trigger was: [ 5.175848] ancillary:ancillary_match_id: dev mlx5_core.ib.0, id mlx5_core.ib_rep
From cf8f10af72f9e0d57c7ec077d59238cc12b0650f Mon Sep 17 00:00:00 2001
From: Leon Romanovsky leonro@nvidia.com Date: Thu, 8 Oct 2020 19:40:03 +0300 Subject: [PATCH] fixup! Fixes to ancillary bus
Signed-off-by: Leon Romanovsky leonro@nvidia.com --- drivers/bus/ancillary.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/bus/ancillary.c b/drivers/bus/ancillary.c index 54858f744ef5..615ce40ef8e4 100644 --- a/drivers/bus/ancillary.c +++ b/drivers/bus/ancillary.c @@ -31,8 +31,10 @@ static const struct ancillary_device_id *ancillary_match_id(const struct ancilla match_size = p - dev_name(&ancildev->dev);
/* use dev_name(&ancildev->dev) prefix before last '.' char to match to */ - if (!strncmp(dev_name(&ancildev->dev), id->name, match_size)) + if (match_size == strlen(id->name) && !strncmp(dev_name(&ancildev->dev), id->name, match_size)) { return id; + } + id++; } return NULL; -- 2.26.2
return id;id++;- }
- return NULL;
+}
-----Original Message----- From: Leon Romanovsky leon@kernel.org Sent: Thursday, October 8, 2020 10:20 AM To: Ertman, David M david.m.ertman@intel.com Cc: alsa-devel@alsa-project.org; tiwai@suse.de; broonie@kernel.org; linux- rdma@vger.kernel.org; jgg@nvidia.com; dledford@redhat.com; netdev@vger.kernel.org; davem@davemloft.net; kuba@kernel.org; gregkh@linuxfoundation.org; ranjani.sridharan@linux.intel.com; pierre- louis.bossart@linux.intel.com; fred.oh@linux.intel.com; parav@mellanox.com; Saleem, Shiraz shiraz.saleem@intel.com; Williams, Dan J dan.j.williams@intel.com; Patil, Kiran kiran.patil@intel.com Subject: Re: [PATCH v2 1/6] Add ancillary bus support
On Mon, Oct 05, 2020 at 11:24:41AM -0700, Dave Ertman wrote:
Add support for the Ancillary Bus, ancillary_device and ancillary_driver. It enables drivers to create an ancillary_device and bind an ancillary_driver to it.
The bus supports probe/remove shutdown and suspend/resume callbacks. Each ancillary_device has a unique string based id; driver binds to an ancillary_device based on this id through the bus.
Co-developed-by: Kiran Patil kiran.patil@intel.com Signed-off-by: Kiran Patil kiran.patil@intel.com Co-developed-by: Ranjani Sridharan ranjani.sridharan@linux.intel.com Signed-off-by: Ranjani Sridharan ranjani.sridharan@linux.intel.com Co-developed-by: Fred Oh fred.oh@linux.intel.com Signed-off-by: Fred Oh fred.oh@linux.intel.com Reviewed-by: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Reviewed-by: Shiraz Saleem shiraz.saleem@intel.com Reviewed-by: Parav Pandit parav@mellanox.com Reviewed-by: Dan Williams dan.j.williams@intel.com Signed-off-by: Dave Ertman david.m.ertman@intel.com
<...>
+static const struct ancillary_device_id *ancillary_match_id(const struct
ancillary_device_id *id,
const structancillary_device *ancildev)
+{
- while (id->name[0]) {
const char *p = strrchr(dev_name(&ancildev->dev), '.');int match_size;if (!p) {id++;continue;}match_size = p - dev_name(&ancildev->dev);/* use dev_name(&ancildev->dev) prefix before last '.' charto match to */
if (!strncmp(dev_name(&ancildev->dev), id->name,match_size))
This check is wrong, it causes to wrong matching if strlen(id->name) > match_size In my case, the trigger was: [ 5.175848] ancillary:ancillary_match_id: dev mlx5_core.ib.0, id mlx5_core.ib_rep
Nice catch , I will look into this.
-DaveE
From cf8f10af72f9e0d57c7ec077d59238cc12b0650f Mon Sep 17 00:00:00 2001 From: Leon Romanovsky leonro@nvidia.com Date: Thu, 8 Oct 2020 19:40:03 +0300 Subject: [PATCH] fixup! Fixes to ancillary bus
Signed-off-by: Leon Romanovsky leonro@nvidia.com
drivers/bus/ancillary.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/bus/ancillary.c b/drivers/bus/ancillary.c index 54858f744ef5..615ce40ef8e4 100644 --- a/drivers/bus/ancillary.c +++ b/drivers/bus/ancillary.c @@ -31,8 +31,10 @@ static const struct ancillary_device_id *ancillary_match_id(const struct ancilla match_size = p - dev_name(&ancildev->dev);
/* use dev_name(&ancildev->dev) prefix before last '.' charto match to */
if (!strncmp(dev_name(&ancildev->dev), id->name,match_size))
if (match_size == strlen(id->name) &&!strncmp(dev_name(&ancildev->dev), id->name, match_size)) { return id;
}- id++; } return NULL;
-- 2.26.2
return id;id++;- }
- return NULL;
+}
From: Ranjani Sridharan ranjani.sridharan@linux.intel.com
A client in the SOF (Sound Open Firmware) context is a device that needs to communicate with the DSP via IPC messages. The SOF core is responsible for serializing the IPC messages to the DSP from the different clients. One example of an SOF client would be an IPC test client that floods the DSP with test IPC messages to validate if the serialization works as expected. Multi-client support will also add the ability to split the existing audio cards into multiple ones, so as to e.g. to deal with HDMI with a dedicated client instead of adding HDMI to all cards.
This patch introduces descriptors for SOF client driver and SOF client device along with APIs for registering and unregistering a SOF client driver, sending IPCs from a client device and accessing the SOF core debugfs root entry.
Along with this, add a couple of new members to struct snd_sof_dev that will be used for maintaining the list of clients.
Reviewed-by: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Signed-off-by: Ranjani Sridharan ranjani.sridharan@linux.intel.com Co-developed-by: Fred Oh fred.oh@linux.intel.com Signed-off-by: Fred Oh fred.oh@linux.intel.com Signed-off-by: Dave Ertman david.m.ertman@intel.com --- sound/soc/sof/Kconfig | 19 ++++++ sound/soc/sof/Makefile | 3 + sound/soc/sof/core.c | 2 + sound/soc/sof/sof-client.c | 119 +++++++++++++++++++++++++++++++++++++ sound/soc/sof/sof-client.h | 69 +++++++++++++++++++++ sound/soc/sof/sof-priv.h | 9 +++ 6 files changed, 221 insertions(+) create mode 100644 sound/soc/sof/sof-client.c create mode 100644 sound/soc/sof/sof-client.h
diff --git a/sound/soc/sof/Kconfig b/sound/soc/sof/Kconfig index 4dda4b62509f..cea7efedafef 100644 --- a/sound/soc/sof/Kconfig +++ b/sound/soc/sof/Kconfig @@ -50,6 +50,24 @@ config SND_SOC_SOF_DEBUG_PROBES Say Y if you want to enable probes. If unsure, select "N".
+config SND_SOC_SOF_CLIENT + tristate + select ANCILLARY_BUS + help + This option is not user-selectable but automagically handled by + 'select' statements at a higher level + +config SND_SOC_SOF_CLIENT_SUPPORT + bool "SOF enable clients" + depends on SND_SOC_SOF + help + This adds support for ancillary client devices to separate out the debug + functionality for IPC tests, probes etc. into separate devices. This + option would also allow adding client devices based on DSP FW + capabilities and ACPI/OF device information. + Say Y if you want to enable clients with SOF. + If unsure select "N". + config SND_SOC_SOF_DEVELOPER_SUPPORT bool "SOF developer options support" depends on EXPERT @@ -186,6 +204,7 @@ endif ## SND_SOC_SOF_DEVELOPER_SUPPORT
config SND_SOC_SOF tristate + select SND_SOC_SOF_CLIENT if SND_SOC_SOF_CLIENT_SUPPORT select SND_SOC_TOPOLOGY select SND_SOC_SOF_NOCODEC if SND_SOC_SOF_NOCODEC_SUPPORT help diff --git a/sound/soc/sof/Makefile b/sound/soc/sof/Makefile index 05718dfe6cd2..5e46f25a3851 100644 --- a/sound/soc/sof/Makefile +++ b/sound/soc/sof/Makefile @@ -2,6 +2,7 @@
snd-sof-objs := core.o ops.o loader.o ipc.o pcm.o pm.o debug.o topology.o\ control.o trace.o utils.o sof-audio.o +snd-sof-client-objs := sof-client.o snd-sof-$(CONFIG_SND_SOC_SOF_DEBUG_PROBES) += probe.o compress.o
snd-sof-pci-objs := sof-pci-dev.o @@ -18,6 +19,8 @@ obj-$(CONFIG_SND_SOC_SOF_ACPI) += snd-sof-acpi.o obj-$(CONFIG_SND_SOC_SOF_OF) += snd-sof-of.o obj-$(CONFIG_SND_SOC_SOF_PCI) += snd-sof-pci.o
+obj-$(CONFIG_SND_SOC_SOF_CLIENT) += snd-sof-client.o + obj-$(CONFIG_SND_SOC_SOF_INTEL_TOPLEVEL) += intel/ obj-$(CONFIG_SND_SOC_SOF_IMX_TOPLEVEL) += imx/ obj-$(CONFIG_SND_SOC_SOF_XTENSA) += xtensa/ diff --git a/sound/soc/sof/core.c b/sound/soc/sof/core.c index adc7c37145d6..72a97219395f 100644 --- a/sound/soc/sof/core.c +++ b/sound/soc/sof/core.c @@ -314,8 +314,10 @@ int snd_sof_device_probe(struct device *dev, struct snd_sof_pdata *plat_data) INIT_LIST_HEAD(&sdev->widget_list); INIT_LIST_HEAD(&sdev->dai_list); INIT_LIST_HEAD(&sdev->route_list); + INIT_LIST_HEAD(&sdev->client_list); spin_lock_init(&sdev->ipc_lock); spin_lock_init(&sdev->hw_lock); + mutex_init(&sdev->client_mutex);
if (IS_ENABLED(CONFIG_SND_SOC_SOF_PROBE_WORK_QUEUE)) INIT_WORK(&sdev->probe_work, sof_probe_work); diff --git a/sound/soc/sof/sof-client.c b/sound/soc/sof/sof-client.c new file mode 100644 index 000000000000..8d861686bdc2 --- /dev/null +++ b/sound/soc/sof/sof-client.c @@ -0,0 +1,119 @@ +// SPDX-License-Identifier: GPL-2.0-only +// +// Copyright(c) 2020 Intel Corporation. All rights reserved. +// +// Author: Ranjani Sridharan ranjani.sridharan@linux.intel.com +// + +#include <linux/debugfs.h> +#include <linux/errno.h> +#include <linux/list.h> +#include <linux/module.h> +#include <linux/mutex.h> +#include <linux/slab.h> +#include "sof-client.h" +#include "sof-priv.h" + +static void sof_client_ancildev_release(struct device *dev) +{ + struct ancillary_device *ancildev = to_ancillary_dev(dev); + struct sof_client_dev *cdev = ancillary_dev_to_sof_client_dev(ancildev); + + ida_simple_remove(cdev->client_ida, ancildev->id); + kfree(cdev); +} + +static struct sof_client_dev *sof_client_dev_alloc(struct snd_sof_dev *sdev, const char *name, + struct ida *client_ida) +{ + struct sof_client_dev *cdev; + struct ancillary_device *ancildev; + int ret; + + cdev = kzalloc(sizeof(*cdev), GFP_KERNEL); + if (!cdev) + return ERR_PTR(-ENOMEM); + + cdev->sdev = sdev; + cdev->client_ida = client_ida; + ancildev = &cdev->ancildev; + ancildev->name = name; + ancildev->dev.parent = sdev->dev; + ancildev->dev.release = sof_client_ancildev_release; + + ancildev->id = ida_alloc(client_ida, GFP_KERNEL); + if (ancildev->id < 0) { + dev_err(sdev->dev, "error: get IDA idx for ancillary device %s failed\n", name); + ret = ancildev->id; + goto err_free; + } + + ret = ancillary_device_initialize(ancildev); + if (ret < 0) { + dev_err(sdev->dev, "error: failed to initialize client dev %s\n", name); + goto ida_free; + } + + return cdev; +ida_free: + ida_simple_remove(client_ida, ancildev->id); +err_free: + kfree(cdev); + return NULL; +} + +int sof_client_dev_register(struct snd_sof_dev *sdev, const char *name, + struct ida *client_ida) +{ + struct sof_client_dev *cdev; + int ret; + + cdev = sof_client_dev_alloc(sdev, name, client_ida); + if (IS_ERR_OR_NULL(cdev)) + return PTR_ERR(cdev); + + ret = ancillary_device_add(&cdev->ancildev); + if (ret < 0) { + dev_err(sdev->dev, "error: failed to add client dev %s\n", name); + put_device(&cdev->ancildev.dev); + return ret; + } + + /* add to list of SOF client devices */ + mutex_lock(&sdev->client_mutex); + list_add(&cdev->list, &sdev->client_list); + mutex_unlock(&sdev->client_mutex); + + return 0; +} +EXPORT_SYMBOL_NS_GPL(sof_client_dev_register, SND_SOC_SOF_CLIENT); + +void sof_client_dev_unregister(struct sof_client_dev *cdev) +{ + struct snd_sof_dev *sdev = cdev->sdev; + + lockdep_assert_held(&sdev->client_mutex); + + /* remove from list of SOF client devices */ + list_del(&cdev->list); + + /* cdev will be freed when the release callback for the ancillary device is invoked */ + ancillary_device_unregister(&cdev->ancildev); +} +EXPORT_SYMBOL_NS_GPL(sof_client_dev_unregister, SND_SOC_SOF_CLIENT); + +int sof_client_ipc_tx_message(struct sof_client_dev *cdev, u32 header, void *msg_data, + size_t msg_bytes, void *reply_data, size_t reply_bytes) +{ + return sof_ipc_tx_message(cdev->sdev->ipc, header, msg_data, msg_bytes, + reply_data, reply_bytes); +} +EXPORT_SYMBOL_NS_GPL(sof_client_ipc_tx_message, SND_SOC_SOF_CLIENT); + +struct dentry *sof_client_get_debugfs_root(struct sof_client_dev *cdev) +{ + return cdev->sdev->debugfs_root; +} +EXPORT_SYMBOL_NS_GPL(sof_client_get_debugfs_root, SND_SOC_SOF_CLIENT); + +MODULE_LICENSE("GPL"); diff --git a/sound/soc/sof/sof-client.h b/sound/soc/sof/sof-client.h new file mode 100644 index 000000000000..0a67f0211074 --- /dev/null +++ b/sound/soc/sof/sof-client.h @@ -0,0 +1,69 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#ifndef __SOUND_SOC_SOF_CLIENT_H +#define __SOUND_SOC_SOF_CLIENT_H + +#include <linux/ancillary_bus.h> +#include <linux/device.h> +#include <linux/idr.h> +#include <linux/list.h> + +#define SOF_CLIENT_PROBE_TIMEOUT_MS 2000 + +struct snd_sof_dev; + +/* SOF client device */ +struct sof_client_dev { + struct ancillary_device ancildev; + struct snd_sof_dev *sdev; + struct list_head list; /* item in SOF core client dev list */ + struct ida *client_ida; + void *data; +}; + +/* client-specific ops, all optional */ +struct sof_client_ops { + int (*client_ipc_rx)(struct sof_client_dev *cdev, u32 msg_cmd); +}; + +struct sof_client_drv { + const char *name; + const struct sof_client_ops ops; + struct ancillary_driver ancillary_drv; +}; + +#define ancillary_dev_to_sof_client_dev(ancillary_dev) \ + container_of(ancillary_dev, struct sof_client_dev, ancildev) + +static inline int sof_client_drv_register(struct sof_client_drv *drv) +{ + return ancillary_driver_register(&drv->ancillary_drv); +} + +static inline void sof_client_drv_unregister(struct sof_client_drv *drv) +{ + ancillary_driver_unregister(&drv->ancillary_drv); +} + +int sof_client_dev_register(struct snd_sof_dev *sdev, const char *name, + struct ida *client_ida); +void sof_client_dev_unregister(struct sof_client_dev *cdev); + +int sof_client_ipc_tx_message(struct sof_client_dev *cdev, u32 header, void *msg_data, + size_t msg_bytes, void *reply_data, size_t reply_bytes); + +struct dentry *sof_client_get_debugfs_root(struct sof_client_dev *cdev); + +/** + * module_sof_client_driver() - Helper macro for registering an SOF Client + * driver + * @__sof_client_driver: SOF client driver struct + * + * Helper macro for SOF client drivers which do not do anything special in + * module init/exit. This eliminates a lot of boilerplate. Each module may only + * use this macro once, and calling it replaces module_init() and module_exit() + */ +#define module_sof_client_driver(__sof_client_driver) \ + module_driver(__sof_client_driver, sof_client_drv_register, sof_client_drv_unregister) + +#endif diff --git a/sound/soc/sof/sof-priv.h b/sound/soc/sof/sof-priv.h index 64f28e082049..8603924e56e3 100644 --- a/sound/soc/sof/sof-priv.h +++ b/sound/soc/sof/sof-priv.h @@ -438,6 +438,15 @@ struct snd_sof_dev {
bool msi_enabled;
+ /* + * Used to keep track of registered client devices so that they can be removed when the + * parent SOF module is removed. + */ + struct list_head client_list; + + /* mutex to protect client list */ + struct mutex client_mutex; + void *private; /* core does not touch this */ };
On 10/5/20 11:24 AM, Dave Ertman wrote:
diff --git a/sound/soc/sof/Kconfig b/sound/soc/sof/Kconfig index 4dda4b62509f..cea7efedafef 100644 --- a/sound/soc/sof/Kconfig +++ b/sound/soc/sof/Kconfig @@ -50,6 +50,24 @@ config SND_SOC_SOF_DEBUG_PROBES Say Y if you want to enable probes. If unsure, select "N".
+config SND_SOC_SOF_CLIENT
- tristate
- select ANCILLARY_BUS
- help
This option is not user-selectable but automagically handled by'select' statements at a higher level+config SND_SOC_SOF_CLIENT_SUPPORT
- bool "SOF enable clients"
Tell users what "SOF" means.
- depends on SND_SOC_SOF
- help
This adds support for ancillary client devices to separate out the debugfunctionality for IPC tests, probes etc. into separate devices. Thisoption would also allow adding client devices based on DSP FW
spell out firmware
capabilities and ACPI/OF device information.Say Y if you want to enable clients with SOF.If unsure select "N".
+config SND_SOC_SOF_CLIENT
- tristate
- select ANCILLARY_BUS
- help
This option is not user-selectable but automagically handled by'select' statements at a higher level+config SND_SOC_SOF_CLIENT_SUPPORT
- bool "SOF enable clients"
Tell users what "SOF" means.
This option can only be reached if the user already selected the topic-level option. From there on the SOF acronym is used. Is this not enough?
config SND_SOC_SOF_TOPLEVEL bool "Sound Open Firmware Support" help This adds support for Sound Open Firmware (SOF). SOF is a free and generic open source audio DSP firmware for multiple devices. Say Y if you have such a device that is supported by SOF.
- depends on SND_SOC_SOF
- help
This adds support for ancillary client devices to separate out the debugfunctionality for IPC tests, probes etc. into separate devices. Thisoption would also allow adding client devices based on DSP FWspell out firmware
agree on this one.
capabilities and ACPI/OF device information.Say Y if you want to enable clients with SOF.If unsure select "N".
On 10/12/20 6:31 PM, Pierre-Louis Bossart wrote:
+config SND_SOC_SOF_CLIENT + tristate + select ANCILLARY_BUS + help + This option is not user-selectable but automagically handled by + 'select' statements at a higher level
+config SND_SOC_SOF_CLIENT_SUPPORT + bool "SOF enable clients"
Tell users what "SOF" means.
This option can only be reached if the user already selected the topic-level option. From there on the SOF acronym is used. Is this not enough?
Yes, that's enough. I didn't see it. Sorry about that.
config SND_SOC_SOF_TOPLEVEL bool "Sound Open Firmware Support" help This adds support for Sound Open Firmware (SOF). SOF is a free and generic open source audio DSP firmware for multiple devices. Say Y if you have such a device that is supported by SOF.
+ depends on SND_SOC_SOF + help + This adds support for ancillary client devices to separate out the debug + functionality for IPC tests, probes etc. into separate devices. This + option would also allow adding client devices based on DSP FW
spell out firmware
agree on this one.
+ capabilities and ACPI/OF device information. + Say Y if you want to enable clients with SOF. + If unsure select "N".
thanks.
On 10/12/20 6:55 PM, Randy Dunlap wrote:
On 10/12/20 6:31 PM, Pierre-Louis Bossart wrote:
+config SND_SOC_SOF_CLIENT + tristate + select ANCILLARY_BUS + help + This option is not user-selectable but automagically handled by + 'select' statements at a higher level
+config SND_SOC_SOF_CLIENT_SUPPORT + bool "SOF enable clients"
Tell users what "SOF" means.
This option can only be reached if the user already selected the topic-level option. From there on the SOF acronym is used. Is this not enough?
Yes, that's enough. I didn't see it. Sorry about that.
Huh. I still don't see that Kconfig option. Which patch is it in?
I only saw patches 1,2,3 on LKML.
config SND_SOC_SOF_TOPLEVEL bool "Sound Open Firmware Support" help This adds support for Sound Open Firmware (SOF). SOF is a free and generic open source audio DSP firmware for multiple devices. Say Y if you have such a device that is supported by SOF.
+ depends on SND_SOC_SOF + help + This adds support for ancillary client devices to separate out the debug + functionality for IPC tests, probes etc. into separate devices. This + option would also allow adding client devices based on DSP FW
spell out firmware
agree on this one.
+ capabilities and ACPI/OF device information. + Say Y if you want to enable clients with SOF. + If unsure select "N".
thanks.
+config SND_SOC_SOF_CLIENT + tristate + select ANCILLARY_BUS + help + This option is not user-selectable but automagically handled by + 'select' statements at a higher level
+config SND_SOC_SOF_CLIENT_SUPPORT + bool "SOF enable clients"
Tell users what "SOF" means.
This option can only be reached if the user already selected the topic-level option. From there on the SOF acronym is used. Is this not enough?
Yes, that's enough. I didn't see it. Sorry about that.
Huh. I still don't see that Kconfig option. Which patch is it in?
I only saw patches 1,2,3 on LKML.
The Sound Open Firmware (SOF) driver is upstream since 2019, see https://elixir.bootlin.com/linux/latest/source/sound/soc/sof/Kconfig
What was shared in these patches is just an evolution to make the driver more modular to handle of 'subfunctions' with the auxiliary bus.
we'd love to hear your feedback if you think the help text can be improved. Thanks!
On 10/13/20 8:08 AM, Pierre-Louis Bossart wrote:
+config SND_SOC_SOF_CLIENT + tristate + select ANCILLARY_BUS + help + This option is not user-selectable but automagically handled by + 'select' statements at a higher level
+config SND_SOC_SOF_CLIENT_SUPPORT + bool "SOF enable clients"
Tell users what "SOF" means.
This option can only be reached if the user already selected the topic-level option. From there on the SOF acronym is used. Is this not enough?
Yes, that's enough. I didn't see it. Sorry about that.
Huh. I still don't see that Kconfig option. Which patch is it in?
I only saw patches 1,2,3 on LKML.
The Sound Open Firmware (SOF) driver is upstream since 2019, see https://elixir.bootlin.com/linux/latest/source/sound/soc/sof/Kconfig
What was shared in these patches is just an evolution to make the driver more modular to handle of 'subfunctions' with the auxiliary bus.
we'd love to hear your feedback if you think the help text can be improved. Thanks!
OK, I looked at the SOF Kconfig files. They are mostly OK except for missing '.' at the end of lots of sentences and a few other typos.
Do you want patches?
> +config SND_SOC_SOF_CLIENT_SUPPORT > + bool "SOF enable clients"
Tell users what "SOF" means.
This option can only be reached if the user already selected the topic-level option. From there on the SOF acronym is used. Is this not enough?
Yes, that's enough. I didn't see it. Sorry about that.
Huh. I still don't see that Kconfig option. Which patch is it in?
I only saw patches 1,2,3 on LKML.
The Sound Open Firmware (SOF) driver is upstream since 2019, see https://elixir.bootlin.com/linux/latest/source/sound/soc/sof/Kconfig
What was shared in these patches is just an evolution to make the driver more modular to handle of 'subfunctions' with the auxiliary bus.
we'd love to hear your feedback if you think the help text can be improved. Thanks!
OK, I looked at the SOF Kconfig files. They are mostly OK except for missing '.' at the end of lots of sentences and a few other typos.
Do you want patches?
Sure! Thanks in advance.
From: Ranjani Sridharan ranjani.sridharan@linux.intel.com
Create an SOF client driver for IPC flood test. This driver is used to set up the debugfs entries and the read/write ops for initiating the IPC flood test that would be used to measure the min/max/avg response times for sending IPCs to the DSP.
Reviewed-by: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Signed-off-by: Ranjani Sridharan ranjani.sridharan@linux.intel.com Co-developed-by: Fred Oh fred.oh@linux.intel.com Signed-off-by: Fred Oh fred.oh@linux.intel.com Signed-off-by: Dave Ertman david.m.ertman@intel.com --- sound/soc/sof/Kconfig | 10 + sound/soc/sof/Makefile | 4 + sound/soc/sof/sof-ipc-test-client.c | 332 ++++++++++++++++++++++++++++ 3 files changed, 346 insertions(+) create mode 100644 sound/soc/sof/sof-ipc-test-client.c
diff --git a/sound/soc/sof/Kconfig b/sound/soc/sof/Kconfig index cea7efedafef..55a2a20c3ec9 100644 --- a/sound/soc/sof/Kconfig +++ b/sound/soc/sof/Kconfig @@ -190,6 +190,16 @@ config SND_SOC_SOF_DEBUG_IPC_FLOOD_TEST Say Y if you want to enable IPC flood test. If unsure, select "N".
+config SND_SOC_SOF_DEBUG_IPC_FLOOD_TEST_CLIENT + tristate "SOF enable IPC flood test client" + depends on SND_SOC_SOF_CLIENT + help + This option enables a separate client device for IPC flood test + which can be used to flood the DSP with test IPCs and gather stats + about response times. + Say Y if you want to enable IPC flood test. + If unsure, select "N". + config SND_SOC_SOF_DEBUG_RETAIN_DSP_CONTEXT bool "SOF retain DSP context on any FW exceptions" help diff --git a/sound/soc/sof/Makefile b/sound/soc/sof/Makefile index 5e46f25a3851..baa93fe2cc9a 100644 --- a/sound/soc/sof/Makefile +++ b/sound/soc/sof/Makefile @@ -9,6 +9,8 @@ snd-sof-pci-objs := sof-pci-dev.o snd-sof-acpi-objs := sof-acpi-dev.o snd-sof-of-objs := sof-of-dev.o
+snd-sof-ipc-test-objs := sof-ipc-test-client.o + snd-sof-nocodec-objs := nocodec.o
obj-$(CONFIG_SND_SOC_SOF) += snd-sof.o @@ -21,6 +23,8 @@ obj-$(CONFIG_SND_SOC_SOF_PCI) += snd-sof-pci.o
obj-$(CONFIG_SND_SOC_SOF_CLIENT) += snd-sof-client.o
+obj-$(CONFIG_SND_SOC_SOF_DEBUG_IPC_FLOOD_TEST_CLIENT) += snd-sof-ipc-test.o + obj-$(CONFIG_SND_SOC_SOF_INTEL_TOPLEVEL) += intel/ obj-$(CONFIG_SND_SOC_SOF_IMX_TOPLEVEL) += imx/ obj-$(CONFIG_SND_SOC_SOF_XTENSA) += xtensa/ diff --git a/sound/soc/sof/sof-ipc-test-client.c b/sound/soc/sof/sof-ipc-test-client.c new file mode 100644 index 000000000000..e541cace06f4 --- /dev/null +++ b/sound/soc/sof/sof-ipc-test-client.c @@ -0,0 +1,332 @@ +// SPDX-License-Identifier: GPL-2.0-only +// +// Copyright(c) 2020 Intel Corporation. All rights reserved. +// +// Author: Ranjani Sridharan ranjani.sridharan@linux.intel.com +// + +#include <linux/completion.h> +#include <linux/debugfs.h> +#include <linux/ktime.h> +#include <linux/mod_devicetable.h> +#include <linux/module.h> +#include <linux/pm_runtime.h> +#include <linux/slab.h> +#include <linux/uaccess.h> +#include <linux/ancillary_bus.h> +#include <sound/sof/header.h> +#include "sof-client.h" + +#define MAX_IPC_FLOOD_DURATION_MS 1000 +#define MAX_IPC_FLOOD_COUNT 10000 +#define IPC_FLOOD_TEST_RESULT_LEN 512 +#define SOF_IPC_CLIENT_SUSPEND_DELAY_MS 3000 + +struct sof_ipc_client_data { + struct dentry *dfs_root; + char *buf; +}; + +/* + * helper function to perform the flood test. Only one of the two params, ipc_duration_ms + * or ipc_count, will be non-zero and will determine the type of test + */ +static int sof_debug_ipc_flood_test(struct sof_client_dev *cdev, unsigned long ipc_duration_ms, + unsigned long ipc_count) +{ + struct sof_ipc_client_data *ipc_client_data = cdev->data; + struct device *dev = &cdev->ancildev.dev; + struct sof_ipc_cmd_hdr hdr; + struct sof_ipc_reply reply; + u64 min_response_time = U64_MAX; + u64 avg_response_time = 0; + u64 max_response_time = 0; + ktime_t cur = ktime_get(); + ktime_t test_end; + int i = 0; + int ret = 0; + bool end_test = false; + + /* configure test IPC */ + hdr.cmd = SOF_IPC_GLB_TEST_MSG | SOF_IPC_TEST_IPC_FLOOD; + hdr.size = sizeof(hdr); + + /* set test end time for duration flood test */ + test_end = ktime_get_ns() + ipc_duration_ms * NSEC_PER_MSEC; + + /* send test IPC's */ + do { + ktime_t start; + u64 ipc_response_time; + + start = ktime_get(); + ret = sof_client_ipc_tx_message(cdev, hdr.cmd, &hdr, hdr.size, &reply, + sizeof(reply)); + if (ret < 0) + break; + cur = ktime_get(); + + i++; + + /* compute min and max response times */ + ipc_response_time = ktime_to_ns(ktime_sub(cur, start)); + min_response_time = min(min_response_time, ipc_response_time); + max_response_time = max(max_response_time, ipc_response_time); + + /* sum up response times */ + avg_response_time += ipc_response_time; + + /* end test? */ + if (ipc_count && i == ipc_count) + end_test = true; + else if (ipc_duration_ms && (ktime_to_ns(cur) >= test_end)) + end_test = true; + + } while (!end_test); + + if (ret < 0) + return ret; + + /* return if the first IPC fails */ + if (!i) + return ret; + + /* compute average response time */ + DIV_ROUND_CLOSEST(avg_response_time, i); + + /* clear previous test output */ + memset(ipc_client_data->buf, 0, IPC_FLOOD_TEST_RESULT_LEN); + + if (!ipc_count) { + dev_dbg(dev, "IPC Flood test duration: %lums\n", ipc_duration_ms); + snprintf(ipc_client_data->buf, IPC_FLOOD_TEST_RESULT_LEN, + "IPC Flood test duration: %lums\n", ipc_duration_ms); + } + + dev_dbg(dev, + "IPC Flood count: %d, Avg response time: %lluns\n", i, avg_response_time); + dev_dbg(dev, "Max response time: %lluns\n", max_response_time); + dev_dbg(dev, "Min response time: %lluns\n", min_response_time); + + /* format output string and save test results */ + snprintf(ipc_client_data->buf + strlen(ipc_client_data->buf), + IPC_FLOOD_TEST_RESULT_LEN - strlen(ipc_client_data->buf), + "IPC Flood count: %d\nAvg response time: %lluns\n", i, avg_response_time); + + snprintf(ipc_client_data->buf + strlen(ipc_client_data->buf), + IPC_FLOOD_TEST_RESULT_LEN - strlen(ipc_client_data->buf), + "Max response time: %lluns\nMin response time: %lluns\n", + max_response_time, min_response_time); + + return ret; +} + +/* + * Writing to the debugfs entry initiates the IPC flood test based on + * the IPC count or the duration specified by the user. + */ +static ssize_t sof_ipc_dfsentry_write(struct file *file, const char __user *buffer, + size_t count, loff_t *ppos) +{ + struct dentry *dentry = file->f_path.dentry; + struct sof_client_dev *cdev = file->private_data; + struct device *dev = &cdev->ancildev.dev; + unsigned long ipc_duration_ms = 0; + bool flood_duration_test; + unsigned long ipc_count = 0; + char *string; + size_t size; + int err; + int ret; + + string = kzalloc(count, GFP_KERNEL); + if (!string) + return -ENOMEM; + + size = simple_write_to_buffer(string, count, ppos, buffer, count); + + flood_duration_test = !strcmp(dentry->d_name.name, "ipc_flood_duration_ms"); + + /* limit max duration/ipc count for flood test */ + if (flood_duration_test) { + ret = kstrtoul(string, 0, &ipc_duration_ms); + if (ret < 0) + goto out; + + if (!ipc_duration_ms) { + ret = size; + goto out; + } + + ipc_duration_ms = min_t(unsigned long, ipc_duration_ms, MAX_IPC_FLOOD_DURATION_MS); + } else { + ret = kstrtoul(string, 0, &ipc_count); + if (ret < 0) + goto out; + + if (!ipc_count) { + ret = size; + goto out; + } + + ipc_count = min_t(unsigned long, ipc_count, MAX_IPC_FLOOD_COUNT); + } + + ret = pm_runtime_get_sync(dev); + if (ret < 0 && ret != -EACCES) { + dev_err_ratelimited(dev, "error: debugfs write failed to resume %d\n", ret); + pm_runtime_put_noidle(dev); + goto out; + } + + ret = sof_debug_ipc_flood_test(cdev, ipc_duration_ms, ipc_count); + + pm_runtime_mark_last_busy(dev); + err = pm_runtime_put_autosuspend(dev); + if (err < 0) { + ret = err; + goto out; + } + + /* return size if test is successful */ + if (ret >= 0) + ret = size; +out: + kfree(string); + return ret; +} + +/* return the result of the last IPC flood test */ +static ssize_t sof_ipc_dfsentry_read(struct file *file, char __user *buffer, + size_t count, loff_t *ppos) +{ + struct sof_client_dev *cdev = file->private_data; + struct sof_ipc_client_data *ipc_client_data = cdev->data; + size_t size_ret; + + if (*ppos) + return 0; + + /* return results of the last IPC test */ + count = min_t(size_t, count, strlen(ipc_client_data->buf)); + size_ret = copy_to_user(buffer, ipc_client_data->buf, count); + if (size_ret) + return -EFAULT; + + *ppos += count; + return count; +} + +static const struct file_operations sof_ipc_dfs_fops = { + .open = simple_open, + .read = sof_ipc_dfsentry_read, + .llseek = default_llseek, + .write = sof_ipc_dfsentry_write, +}; + +/* + * The IPC test client creates a couple of debugfs entries that will be used + * flood tests. Users can write to these entries to execute the IPC flood test + * by specifying either the number of IPCs to flood the DSP with or the duration + * (in ms) for which the DSP should be flooded with test IPCs. At the + * end of each test, the average, min and max response times are reported back. + * The results of the last flood test can be accessed by reading the debugfs + * entries. + */ +static int sof_ipc_test_probe(struct ancillary_device *ancildev, + const struct ancillary_device_id *id) +{ + struct sof_client_dev *cdev = ancillary_dev_to_sof_client_dev(ancildev); + struct sof_ipc_client_data *ipc_client_data; + + /* + * The ancillary device has a usage count of 0 even before runtime PM + * is enabled. So, increment the usage count to let the device + * suspend after probe is complete. + */ + pm_runtime_get_noresume(&ancildev->dev); + + /* allocate memory for client data */ + ipc_client_data = devm_kzalloc(&ancildev->dev, sizeof(*ipc_client_data), GFP_KERNEL); + if (!ipc_client_data) + return -ENOMEM; + + ipc_client_data->buf = devm_kzalloc(&ancildev->dev, IPC_FLOOD_TEST_RESULT_LEN, GFP_KERNEL); + if (!ipc_client_data->buf) + return -ENOMEM; + + cdev->data = ipc_client_data; + + /* create debugfs root folder with device name under parent SOF dir */ + ipc_client_data->dfs_root = debugfs_create_dir(dev_name(&ancildev->dev), + sof_client_get_debugfs_root(cdev)); + + /* create read-write ipc_flood_count debugfs entry */ + debugfs_create_file("ipc_flood_count", 0644, ipc_client_data->dfs_root, + cdev, &sof_ipc_dfs_fops); + + /* create read-write ipc_flood_duration_ms debugfs entry */ + debugfs_create_file("ipc_flood_duration_ms", 0644, ipc_client_data->dfs_root, + cdev, &sof_ipc_dfs_fops); + + /* enable runtime PM */ + pm_runtime_set_autosuspend_delay(&ancildev->dev, SOF_IPC_CLIENT_SUSPEND_DELAY_MS); + pm_runtime_use_autosuspend(&ancildev->dev); + pm_runtime_set_active(&ancildev->dev); + pm_runtime_enable(&ancildev->dev); + pm_runtime_mark_last_busy(&ancildev->dev); + pm_runtime_put_autosuspend(&ancildev->dev); + + return 0; +} + +static int sof_ipc_test_cleanup(struct ancillary_device *ancildev) +{ + struct sof_client_dev *cdev = ancillary_dev_to_sof_client_dev(ancildev); + struct sof_ipc_client_data *ipc_client_data = cdev->data; + + pm_runtime_disable(&ancildev->dev); + + debugfs_remove_recursive(ipc_client_data->dfs_root); + + return 0; +} + +static int sof_ipc_test_remove(struct ancillary_device *ancildev) +{ + return sof_ipc_test_cleanup(ancildev); +} + +static void sof_ipc_test_shutdown(struct ancillary_device *ancildev) +{ + sof_ipc_test_cleanup(ancildev); +} + +static const struct ancillary_device_id sof_ipc_ancilbus_id_table[] = { + { .name = "snd_sof_client.ipc_test" }, + {}, +}; +MODULE_DEVICE_TABLE(ancillary, sof_ipc_ancilbus_id_table); + +/* + * No need for driver pm_ops as the generic pm callbacks in the ancillary bus type are enough to + * ensure that the parent SOF device resumes to bring the DSP back to D0. + */ +static struct sof_client_drv sof_ipc_test_client_drv = { + .name = "sof-ipc-test-client-drv", + .ancillary_drv = { + .driver = { + .name = "sof-ipc-test-ancilbus-drv", + }, + .id_table = sof_ipc_ancilbus_id_table, + .probe = sof_ipc_test_probe, + .remove = sof_ipc_test_remove, + .shutdown = sof_ipc_test_shutdown, + }, +}; + +module_sof_client_driver(sof_ipc_test_client_drv); + +MODULE_DESCRIPTION("SOF IPC Test Client Driver"); +MODULE_LICENSE("GPL"); +MODULE_IMPORT_NS(SND_SOC_SOF_CLIENT);
From: Ranjani Sridharan ranjani.sridharan@linux.intel.com
Add new ops for registering/unregistering clients based on DSP capabilities and/or DT information.
Reviewed-by: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Signed-off-by: Ranjani Sridharan ranjani.sridharan@linux.intel.com Signed-off-by: Dave Ertman david.m.ertman@intel.com --- sound/soc/sof/core.c | 10 ++++++++++ sound/soc/sof/ops.h | 14 ++++++++++++++ sound/soc/sof/sof-priv.h | 4 ++++ 3 files changed, 28 insertions(+)
diff --git a/sound/soc/sof/core.c b/sound/soc/sof/core.c index 72a97219395f..ddb9a12d5aac 100644 --- a/sound/soc/sof/core.c +++ b/sound/soc/sof/core.c @@ -246,8 +246,17 @@ static int sof_probe_continue(struct snd_sof_dev *sdev) if (plat_data->sof_probe_complete) plat_data->sof_probe_complete(sdev->dev);
+ /* If registering certain clients fails, unregister the previously registered clients. */ + ret = snd_sof_register_clients(sdev); + if (ret < 0) { + dev_err(sdev->dev, "error: failed to register clients %d\n", ret); + goto client_reg_err; + } + return 0;
+client_reg_err: + snd_sof_unregister_clients(sdev); fw_trace_err: snd_sof_free_trace(sdev); fw_run_err: @@ -356,6 +365,7 @@ int snd_sof_device_remove(struct device *dev) dev_warn(dev, "error: %d failed to prepare DSP for device removal", ret);
+ snd_sof_unregister_clients(sdev); snd_sof_fw_unload(sdev); snd_sof_ipc_free(sdev); snd_sof_free_debug(sdev); diff --git a/sound/soc/sof/ops.h b/sound/soc/sof/ops.h index b21632f5511a..00370f8bcd75 100644 --- a/sound/soc/sof/ops.h +++ b/sound/soc/sof/ops.h @@ -470,6 +470,20 @@ snd_sof_set_mach_params(const struct snd_soc_acpi_mach *mach, sof_ops(sdev)->set_mach_params(mach, dev); }
+static inline int snd_sof_register_clients(struct snd_sof_dev *sdev) +{ + if (sof_ops(sdev) && sof_ops(sdev)->register_clients) + return sof_ops(sdev)->register_clients(sdev); + + return 0; +} + +static inline void snd_sof_unregister_clients(struct snd_sof_dev *sdev) +{ + if (sof_ops(sdev) && sof_ops(sdev)->unregister_clients) + sof_ops(sdev)->unregister_clients(sdev); +} + static inline const struct snd_sof_dsp_ops *sof_get_ops(const struct sof_dev_desc *d, const struct sof_ops_table mach_ops[], int asize) diff --git a/sound/soc/sof/sof-priv.h b/sound/soc/sof/sof-priv.h index 8603924e56e3..1c29199132c5 100644 --- a/sound/soc/sof/sof-priv.h +++ b/sound/soc/sof/sof-priv.h @@ -249,6 +249,10 @@ struct snd_sof_dsp_ops { void (*set_mach_params)(const struct snd_soc_acpi_mach *mach, struct device *dev); /* optional */
+ /* client ops */ + int (*register_clients)(struct snd_sof_dev *sdev); /* optional */ + void (*unregister_clients)(struct snd_sof_dev *sdev); /* optional */ + /* DAI ops */ struct snd_soc_dai_driver *drv; int num_drv;
From: Ranjani Sridharan ranjani.sridharan@linux.intel.com
Define client ops for Intel platforms. For now, we only add 2 IPC test clients that will be used for run tandem IPC flood tests for.
For ACPI platforms, change the Kconfig to select SND_SOC_SOF_PROBE_WORK_QUEUE to allow the ancillary driver to probe when the client is registered.
Reviewed-by: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Signed-off-by: Ranjani Sridharan ranjani.sridharan@linux.intel.com Co-developed-by: Fred Oh fred.oh@linux.intel.com Signed-off-by: Fred Oh fred.oh@linux.intel.com Signed-off-by: Dave Ertman david.m.ertman@intel.com --- sound/soc/sof/intel/Kconfig | 9 +++++ sound/soc/sof/intel/Makefile | 3 ++ sound/soc/sof/intel/apl.c | 18 ++++++++++ sound/soc/sof/intel/bdw.c | 18 ++++++++++ sound/soc/sof/intel/byt.c | 22 +++++++++++++ sound/soc/sof/intel/cnl.c | 18 ++++++++++ sound/soc/sof/intel/intel-client.c | 53 ++++++++++++++++++++++++++++++ sound/soc/sof/intel/intel-client.h | 26 +++++++++++++++ 8 files changed, 167 insertions(+) create mode 100644 sound/soc/sof/intel/intel-client.c create mode 100644 sound/soc/sof/intel/intel-client.h
diff --git a/sound/soc/sof/intel/Kconfig b/sound/soc/sof/intel/Kconfig index 3aaf25e4f766..28aba42f4658 100644 --- a/sound/soc/sof/intel/Kconfig +++ b/sound/soc/sof/intel/Kconfig @@ -13,6 +13,8 @@ config SND_SOC_SOF_INTEL_ACPI def_tristate SND_SOC_SOF_ACPI select SND_SOC_SOF_BAYTRAIL if SND_SOC_SOF_BAYTRAIL_SUPPORT select SND_SOC_SOF_BROADWELL if SND_SOC_SOF_BROADWELL_SUPPORT + select SND_SOC_SOF_PROBE_WORK_QUEUE if SND_SOC_SOF_CLIENT + select SND_SOC_SOF_INTEL_CLIENT if SND_SOC_SOF_CLIENT help This option is not user-selectable but automagically handled by 'select' statements at a higher level @@ -29,6 +31,7 @@ config SND_SOC_SOF_INTEL_PCI select SND_SOC_SOF_TIGERLAKE if SND_SOC_SOF_TIGERLAKE_SUPPORT select SND_SOC_SOF_ELKHARTLAKE if SND_SOC_SOF_ELKHARTLAKE_SUPPORT select SND_SOC_SOF_JASPERLAKE if SND_SOC_SOF_JASPERLAKE_SUPPORT + select SND_SOC_SOF_INTEL_CLIENT if SND_SOC_SOF_CLIENT help This option is not user-selectable but automagically handled by 'select' statements at a higher level @@ -57,6 +60,12 @@ config SND_SOC_SOF_INTEL_COMMON This option is not user-selectable but automagically handled by 'select' statements at a higher level
+config SND_SOC_SOF_INTEL_CLIENT + tristate + help + This option is not user-selectable but automagically handled by + 'select' statements at a higher level + if SND_SOC_SOF_INTEL_ACPI
config SND_SOC_SOF_BAYTRAIL_SUPPORT diff --git a/sound/soc/sof/intel/Makefile b/sound/soc/sof/intel/Makefile index f7e9358f1f06..50e40caaa787 100644 --- a/sound/soc/sof/intel/Makefile +++ b/sound/soc/sof/intel/Makefile @@ -5,6 +5,8 @@ snd-sof-intel-bdw-objs := bdw.o
snd-sof-intel-ipc-objs := intel-ipc.o
+snd-sof-intel-client-objs := intel-client.o + snd-sof-intel-hda-common-objs := hda.o hda-loader.o hda-stream.o hda-trace.o \ hda-dsp.o hda-ipc.o hda-ctrl.o hda-pcm.o \ hda-dai.o hda-bus.o \ @@ -18,3 +20,4 @@ obj-$(CONFIG_SND_SOC_SOF_BROADWELL) += snd-sof-intel-bdw.o obj-$(CONFIG_SND_SOC_SOF_INTEL_HIFI_EP_IPC) += snd-sof-intel-ipc.o obj-$(CONFIG_SND_SOC_SOF_HDA_COMMON) += snd-sof-intel-hda-common.o obj-$(CONFIG_SND_SOC_SOF_HDA) += snd-sof-intel-hda.o +obj-$(CONFIG_SND_SOC_SOF_INTEL_CLIENT) += snd-sof-intel-client.o diff --git a/sound/soc/sof/intel/apl.c b/sound/soc/sof/intel/apl.c index 9e29d4fd393a..b31353b1a3ea 100644 --- a/sound/soc/sof/intel/apl.c +++ b/sound/soc/sof/intel/apl.c @@ -15,9 +15,12 @@ * Hardware interface for audio DSP on Apollolake and GeminiLake */
+#include <linux/list.h> #include "../sof-priv.h" #include "hda.h" #include "../sof-audio.h" +#include "../sof-client.h" +#include "intel-client.h"
static const struct snd_sof_debugfs_map apl_dsp_debugfs[] = { {"hda", HDA_DSP_HDA_BAR, 0, 0x4000, SOF_DEBUGFS_ACCESS_ALWAYS}, @@ -25,6 +28,16 @@ static const struct snd_sof_debugfs_map apl_dsp_debugfs[] = { {"dsp", HDA_DSP_BAR, 0, 0x10000, SOF_DEBUGFS_ACCESS_ALWAYS}, };
+static int apl_register_clients(struct snd_sof_dev *sdev) +{ + return intel_register_ipc_test_clients(sdev); +} + +static void apl_unregister_clients(struct snd_sof_dev *sdev) +{ + intel_unregister_ipc_test_clients(sdev); +} + /* apollolake ops */ const struct snd_sof_dsp_ops sof_apl_ops = { /* probe and remove */ @@ -101,6 +114,10 @@ const struct snd_sof_dsp_ops sof_apl_ops = { .trace_release = hda_dsp_trace_release, .trace_trigger = hda_dsp_trace_trigger,
+ /* client ops */ + .register_clients = apl_register_clients, + .unregister_clients = apl_unregister_clients, + /* DAI drivers */ .drv = skl_dai, .num_drv = SOF_SKL_NUM_DAIS, @@ -140,3 +157,4 @@ const struct sof_intel_dsp_desc apl_chip_info = { .ssp_base_offset = APL_SSP_BASE_OFFSET, }; EXPORT_SYMBOL_NS(apl_chip_info, SND_SOC_SOF_INTEL_HDA_COMMON); +MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_CLIENT); diff --git a/sound/soc/sof/intel/bdw.c b/sound/soc/sof/intel/bdw.c index 99fd0bd7276e..b14026c5fa97 100644 --- a/sound/soc/sof/intel/bdw.c +++ b/sound/soc/sof/intel/bdw.c @@ -12,12 +12,15 @@ * Hardware interface for audio DSP on Broadwell */
+#include <linux/list.h> #include <linux/module.h> #include <sound/sof.h> #include <sound/sof/xtensa.h> #include "../ops.h" #include "shim.h" #include "../sof-audio.h" +#include "../sof-client.h" +#include "intel-client.h"
/* BARs */ #define BDW_DSP_BAR 0 @@ -563,6 +566,16 @@ static void bdw_set_mach_params(const struct snd_soc_acpi_mach *mach, mach_params->platform = dev_name(dev); }
+static int bdw_register_clients(struct snd_sof_dev *sdev) +{ + return intel_register_ipc_test_clients(sdev); +} + +static void bdw_unregister_clients(struct snd_sof_dev *sdev) +{ + intel_unregister_ipc_test_clients(sdev); +} + /* Broadwell DAIs */ static struct snd_soc_dai_driver bdw_dai[] = { { @@ -638,6 +651,10 @@ const struct snd_sof_dsp_ops sof_bdw_ops = { /*Firmware loading */ .load_firmware = snd_sof_load_firmware_memcpy,
+ /* client ops */ + .register_clients = bdw_register_clients, + .unregister_clients = bdw_unregister_clients, + /* DAI drivers */ .drv = bdw_dai, .num_drv = ARRAY_SIZE(bdw_dai), @@ -662,3 +679,4 @@ EXPORT_SYMBOL_NS(bdw_chip_info, SND_SOC_SOF_BROADWELL); MODULE_LICENSE("Dual BSD/GPL"); MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_HIFI_EP_IPC); MODULE_IMPORT_NS(SND_SOC_SOF_XTENSA); +MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_CLIENT); diff --git a/sound/soc/sof/intel/byt.c b/sound/soc/sof/intel/byt.c index 49f67f1b94e0..8951f756d078 100644 --- a/sound/soc/sof/intel/byt.c +++ b/sound/soc/sof/intel/byt.c @@ -12,13 +12,16 @@ * Hardware interface for audio DSP on Baytrail, Braswell and Cherrytrail. */
+#include <linux/list.h> #include <linux/module.h> #include <sound/sof.h> #include <sound/sof/xtensa.h> #include "../ops.h" #include "shim.h" #include "../sof-audio.h" +#include "../sof-client.h" #include "../../intel/common/soc-intel-quirks.h" +#include "intel-client.h"
/* DSP memories */ #define IRAM_OFFSET 0x0C0000 @@ -821,6 +824,16 @@ static int byt_acpi_probe(struct snd_sof_dev *sdev) return ret; }
+static int byt_register_clients(struct snd_sof_dev *sdev) +{ + return intel_register_ipc_test_clients(sdev); +} + +static void byt_unregister_clients(struct snd_sof_dev *sdev) +{ + intel_unregister_ipc_test_clients(sdev); +} + /* baytrail ops */ const struct snd_sof_dsp_ops sof_byt_ops = { /* device init */ @@ -879,6 +892,10 @@ const struct snd_sof_dsp_ops sof_byt_ops = { .suspend = byt_suspend, .resume = byt_resume,
+ /* client ops */ + .register_clients = byt_register_clients, + .unregister_clients = byt_unregister_clients, + /* DAI drivers */ .drv = byt_dai, .num_drv = 3, /* we have only 3 SSPs on byt*/ @@ -958,6 +975,10 @@ const struct snd_sof_dsp_ops sof_cht_ops = { .suspend = byt_suspend, .resume = byt_resume,
+ /* client ops */ + .register_clients = byt_register_clients, + .unregister_clients = byt_unregister_clients, + /* DAI drivers */ .drv = byt_dai, /* all 6 SSPs may be available for cherrytrail */ @@ -985,3 +1006,4 @@ EXPORT_SYMBOL_NS(cht_chip_info, SND_SOC_SOF_BAYTRAIL); MODULE_LICENSE("Dual BSD/GPL"); MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_HIFI_EP_IPC); MODULE_IMPORT_NS(SND_SOC_SOF_XTENSA); +MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_CLIENT); diff --git a/sound/soc/sof/intel/cnl.c b/sound/soc/sof/intel/cnl.c index 16db0f50d139..5d7c2a667798 100644 --- a/sound/soc/sof/intel/cnl.c +++ b/sound/soc/sof/intel/cnl.c @@ -15,10 +15,13 @@ * Hardware interface for audio DSP on Cannonlake. */
+#include <linux/list.h> #include "../ops.h" #include "hda.h" #include "hda-ipc.h" #include "../sof-audio.h" +#include "../sof-client.h" +#include "intel-client.h"
static const struct snd_sof_debugfs_map cnl_dsp_debugfs[] = { {"hda", HDA_DSP_HDA_BAR, 0, 0x4000, SOF_DEBUGFS_ACCESS_ALWAYS}, @@ -231,6 +234,16 @@ static void cnl_ipc_dump(struct snd_sof_dev *sdev) hipcida, hipctdr, hipcctl); }
+static int cnl_register_clients(struct snd_sof_dev *sdev) +{ + return intel_register_ipc_test_clients(sdev); +} + +static void cnl_unregister_clients(struct snd_sof_dev *sdev) +{ + intel_unregister_ipc_test_clients(sdev); +} + /* cannonlake ops */ const struct snd_sof_dsp_ops sof_cnl_ops = { /* probe and remove */ @@ -307,6 +320,10 @@ const struct snd_sof_dsp_ops sof_cnl_ops = { .trace_release = hda_dsp_trace_release, .trace_trigger = hda_dsp_trace_trigger,
+ /* client ops */ + .register_clients = cnl_register_clients, + .unregister_clients = cnl_unregister_clients, + /* DAI drivers */ .drv = skl_dai, .num_drv = SOF_SKL_NUM_DAIS, @@ -417,3 +434,4 @@ const struct sof_intel_dsp_desc jsl_chip_info = { .ssp_base_offset = CNL_SSP_BASE_OFFSET, }; EXPORT_SYMBOL_NS(jsl_chip_info, SND_SOC_SOF_INTEL_HDA_COMMON); +MODULE_IMPORT_NS(SND_SOC_SOF_INTEL_CLIENT); diff --git a/sound/soc/sof/intel/intel-client.c b/sound/soc/sof/intel/intel-client.c new file mode 100644 index 000000000000..eecab96f1089 --- /dev/null +++ b/sound/soc/sof/intel/intel-client.c @@ -0,0 +1,53 @@ +// SPDX-License-Identifier: GPL-2.0-only +// +// Copyright(c) 2020 Intel Corporation. All rights reserved. +// +// Author: Ranjani Sridharan ranjani.sridharan@linux.intel.com +// + +#include <linux/module.h> +#include "../sof-priv.h" +#include "../sof-client.h" +#include "intel-client.h" + +#if IS_ENABLED(CONFIG_SND_SOC_SOF_DEBUG_IPC_FLOOD_TEST_CLIENT) +DEFINE_IDA(sof_ipc_test_client_ida); + +int intel_register_ipc_test_clients(struct snd_sof_dev *sdev) +{ + int ret; + + /* + * Register 2 IPC clients to facilitate tandem flood test. The device name below is + * appended with the device ID assigned automatically when the ancillary device is + * registered making them unique. + */ + ret = sof_client_dev_register(sdev, "ipc_test", &sof_ipc_test_client_ida); + if (ret < 0) + return ret; + + return sof_client_dev_register(sdev, "ipc_test", &sof_ipc_test_client_ida); +} +EXPORT_SYMBOL_NS_GPL(intel_register_ipc_test_clients, SND_SOC_SOF_INTEL_CLIENT); + +void intel_unregister_ipc_test_clients(struct snd_sof_dev *sdev) +{ + struct sof_client_dev *cdev, *_cdev; + + mutex_lock(&sdev->client_mutex); + + /* unregister ipc_test clients */ + list_for_each_entry_safe(cdev, _cdev, &sdev->client_list, list) { + if (!strcmp(cdev->ancildev.name, "ipc_test")) + sof_client_dev_unregister(cdev); + } + + mutex_unlock(&sdev->client_mutex); + + ida_destroy(&sof_ipc_test_client_ida); +} +EXPORT_SYMBOL_NS_GPL(intel_unregister_ipc_test_clients, SND_SOC_SOF_INTEL_CLIENT); +#endif + +MODULE_LICENSE("GPL"); +MODULE_IMPORT_NS(SND_SOC_SOF_CLIENT); diff --git a/sound/soc/sof/intel/intel-client.h b/sound/soc/sof/intel/intel-client.h new file mode 100644 index 000000000000..49b2c6c0dcc4 --- /dev/null +++ b/sound/soc/sof/intel/intel-client.h @@ -0,0 +1,26 @@ +/* SPDX-License-Identifier: (GPL-2.0-only OR BSD-3-Clause) */ +/* + * This file is provided under a dual BSD/GPLv2 license. When using or + * redistributing this file, you may do so under either license. + * + * Copyright(c) 2020 Intel Corporation. All rights reserved. + * + * Author: Ranjani Sridharan ranjani.sridharan@linux.intel.com + */ + +#ifndef __INTEL_CLIENT_H +#define __INTEL_CLIENT_H + +#if IS_ENABLED(CONFIG_SND_SOC_SOF_DEBUG_IPC_FLOOD_TEST_CLIENT) +int intel_register_ipc_test_clients(struct snd_sof_dev *sdev); +void intel_unregister_ipc_test_clients(struct snd_sof_dev *sdev); +#else +static inline int intel_register_ipc_test_clients(struct snd_sof_dev *sdev) +{ + return 0; +} + +static void intel_unregister_ipc_test_clients(struct snd_sof_dev *sdev) {} +#endif + +#endif
From: Fred Oh fred.oh@linux.intel.com
Remove the IPC flood test support in the SOF core as it is now added in the IPC flood test client.
Reviewed-by: Pierre-Louis Bossart pierre-louis.bossart@linux.intel.com Signed-off-by: Fred Oh fred.oh@linux.intel.com Signed-off-by: Ranjani Sridharan ranjani.sridharan@linux.intel.com Signed-off-by: Dave Ertman david.m.ertman@intel.com --- sound/soc/sof/Kconfig | 8 -- sound/soc/sof/debug.c | 230 --------------------------------------- sound/soc/sof/sof-priv.h | 6 +- 3 files changed, 1 insertion(+), 243 deletions(-)
diff --git a/sound/soc/sof/Kconfig b/sound/soc/sof/Kconfig index 55a2a20c3ec9..4046e96eed92 100644 --- a/sound/soc/sof/Kconfig +++ b/sound/soc/sof/Kconfig @@ -182,14 +182,6 @@ config SND_SOC_SOF_DEBUG_ENABLE_FIRMWARE_TRACE module parameter (similar to dynamic debug) If unsure, select "N".
-config SND_SOC_SOF_DEBUG_IPC_FLOOD_TEST - bool "SOF enable IPC flood test" - help - This option enables the IPC flood test which can be used to flood - the DSP with test IPCs and gather stats about response times. - Say Y if you want to enable IPC flood test. - If unsure, select "N". - config SND_SOC_SOF_DEBUG_IPC_FLOOD_TEST_CLIENT tristate "SOF enable IPC flood test client" depends on SND_SOC_SOF_CLIENT diff --git a/sound/soc/sof/debug.c b/sound/soc/sof/debug.c index 8e15f105d1d5..d224641768da 100644 --- a/sound/soc/sof/debug.c +++ b/sound/soc/sof/debug.c @@ -232,120 +232,10 @@ static int snd_sof_debugfs_probe_item(struct snd_sof_dev *sdev, } #endif
-#if IS_ENABLED(CONFIG_SND_SOC_SOF_DEBUG_IPC_FLOOD_TEST) -#define MAX_IPC_FLOOD_DURATION_MS 1000 -#define MAX_IPC_FLOOD_COUNT 10000 -#define IPC_FLOOD_TEST_RESULT_LEN 512 - -static int sof_debug_ipc_flood_test(struct snd_sof_dev *sdev, - struct snd_sof_dfsentry *dfse, - bool flood_duration_test, - unsigned long ipc_duration_ms, - unsigned long ipc_count) -{ - struct sof_ipc_cmd_hdr hdr; - struct sof_ipc_reply reply; - u64 min_response_time = U64_MAX; - ktime_t start, end, test_end; - u64 avg_response_time = 0; - u64 max_response_time = 0; - u64 ipc_response_time; - int i = 0; - int ret; - - /* configure test IPC */ - hdr.cmd = SOF_IPC_GLB_TEST_MSG | SOF_IPC_TEST_IPC_FLOOD; - hdr.size = sizeof(hdr); - - /* set test end time for duration flood test */ - if (flood_duration_test) - test_end = ktime_get_ns() + ipc_duration_ms * NSEC_PER_MSEC; - - /* send test IPC's */ - while (1) { - start = ktime_get(); - ret = sof_ipc_tx_message(sdev->ipc, hdr.cmd, &hdr, hdr.size, - &reply, sizeof(reply)); - end = ktime_get(); - - if (ret < 0) - break; - - /* compute min and max response times */ - ipc_response_time = ktime_to_ns(ktime_sub(end, start)); - min_response_time = min(min_response_time, ipc_response_time); - max_response_time = max(max_response_time, ipc_response_time); - - /* sum up response times */ - avg_response_time += ipc_response_time; - i++; - - /* test complete? */ - if (flood_duration_test) { - if (ktime_to_ns(end) >= test_end) - break; - } else { - if (i == ipc_count) - break; - } - } - - if (ret < 0) - dev_err(sdev->dev, - "error: ipc flood test failed at %d iterations\n", i); - - /* return if the first IPC fails */ - if (!i) - return ret; - - /* compute average response time */ - do_div(avg_response_time, i); - - /* clear previous test output */ - memset(dfse->cache_buf, 0, IPC_FLOOD_TEST_RESULT_LEN); - - if (flood_duration_test) { - dev_dbg(sdev->dev, "IPC Flood test duration: %lums\n", - ipc_duration_ms); - snprintf(dfse->cache_buf, IPC_FLOOD_TEST_RESULT_LEN, - "IPC Flood test duration: %lums\n", ipc_duration_ms); - } - - dev_dbg(sdev->dev, - "IPC Flood count: %d, Avg response time: %lluns\n", - i, avg_response_time); - dev_dbg(sdev->dev, "Max response time: %lluns\n", - max_response_time); - dev_dbg(sdev->dev, "Min response time: %lluns\n", - min_response_time); - - /* format output string */ - snprintf(dfse->cache_buf + strlen(dfse->cache_buf), - IPC_FLOOD_TEST_RESULT_LEN - strlen(dfse->cache_buf), - "IPC Flood count: %d\nAvg response time: %lluns\n", - i, avg_response_time); - - snprintf(dfse->cache_buf + strlen(dfse->cache_buf), - IPC_FLOOD_TEST_RESULT_LEN - strlen(dfse->cache_buf), - "Max response time: %lluns\nMin response time: %lluns\n", - max_response_time, min_response_time); - - return ret; -} -#endif
static ssize_t sof_dfsentry_write(struct file *file, const char __user *buffer, size_t count, loff_t *ppos) { -#if IS_ENABLED(CONFIG_SND_SOC_SOF_DEBUG_IPC_FLOOD_TEST) - struct snd_sof_dfsentry *dfse = file->private_data; - struct snd_sof_dev *sdev = dfse->sdev; - unsigned long ipc_duration_ms = 0; - bool flood_duration_test = false; - unsigned long ipc_count = 0; - struct dentry *dentry; - int err; -#endif size_t size; char *string; int ret; @@ -357,78 +247,6 @@ static ssize_t sof_dfsentry_write(struct file *file, const char __user *buffer, size = simple_write_to_buffer(string, count, ppos, buffer, count); ret = size;
-#if IS_ENABLED(CONFIG_SND_SOC_SOF_DEBUG_IPC_FLOOD_TEST) - /* - * write op is only supported for ipc_flood_count or - * ipc_flood_duration_ms debugfs entries atm. - * ipc_flood_count floods the DSP with the number of IPC's specified. - * ipc_duration_ms test floods the DSP for the time specified - * in the debugfs entry. - */ - dentry = file->f_path.dentry; - if (strcmp(dentry->d_name.name, "ipc_flood_count") && - strcmp(dentry->d_name.name, "ipc_flood_duration_ms")) { - ret = -EINVAL; - goto out; - } - - if (!strcmp(dentry->d_name.name, "ipc_flood_duration_ms")) - flood_duration_test = true; - - /* test completion criterion */ - if (flood_duration_test) - ret = kstrtoul(string, 0, &ipc_duration_ms); - else - ret = kstrtoul(string, 0, &ipc_count); - if (ret < 0) - goto out; - - /* limit max duration/ipc count for flood test */ - if (flood_duration_test) { - if (!ipc_duration_ms) { - ret = size; - goto out; - } - - /* find the minimum. min() is not used to avoid warnings */ - if (ipc_duration_ms > MAX_IPC_FLOOD_DURATION_MS) - ipc_duration_ms = MAX_IPC_FLOOD_DURATION_MS; - } else { - if (!ipc_count) { - ret = size; - goto out; - } - - /* find the minimum. min() is not used to avoid warnings */ - if (ipc_count > MAX_IPC_FLOOD_COUNT) - ipc_count = MAX_IPC_FLOOD_COUNT; - } - - ret = pm_runtime_get_sync(sdev->dev); - if (ret < 0) { - dev_err_ratelimited(sdev->dev, - "error: debugfs write failed to resume %d\n", - ret); - pm_runtime_put_noidle(sdev->dev); - goto out; - } - - /* flood test */ - ret = sof_debug_ipc_flood_test(sdev, dfse, flood_duration_test, - ipc_duration_ms, ipc_count); - - pm_runtime_mark_last_busy(sdev->dev); - err = pm_runtime_put_autosuspend(sdev->dev); - if (err < 0) - dev_err_ratelimited(sdev->dev, - "error: debugfs write failed to idle %d\n", - err); - - /* return size if test is successful */ - if (ret >= 0) - ret = size; -out: -#endif kfree(string); return ret; } @@ -444,25 +262,6 @@ static ssize_t sof_dfsentry_read(struct file *file, char __user *buffer, int size; u8 *buf;
-#if IS_ENABLED(CONFIG_SND_SOC_SOF_DEBUG_IPC_FLOOD_TEST) - struct dentry *dentry; - - dentry = file->f_path.dentry; - if ((!strcmp(dentry->d_name.name, "ipc_flood_count") || - !strcmp(dentry->d_name.name, "ipc_flood_duration_ms")) && - dfse->cache_buf) { - if (*ppos) - return 0; - - count = strlen(dfse->cache_buf); - size_ret = copy_to_user(buffer, dfse->cache_buf, count); - if (size_ret) - return -EFAULT; - - *ppos += count; - return count; - } -#endif size = dfse->size;
/* validate position & count */ @@ -606,17 +405,6 @@ int snd_sof_debugfs_buf_item(struct snd_sof_dev *sdev, dfse->size = size; dfse->sdev = sdev;
-#if IS_ENABLED(CONFIG_SND_SOC_SOF_DEBUG_IPC_FLOOD_TEST) - /* - * cache_buf is unused for SOF_DFSENTRY_TYPE_BUF debugfs entries. - * So, use it to save the results of the last IPC flood test. - */ - dfse->cache_buf = devm_kzalloc(sdev->dev, IPC_FLOOD_TEST_RESULT_LEN, - GFP_KERNEL); - if (!dfse->cache_buf) - return -ENOMEM; -#endif - debugfs_create_file(name, mode, sdev->debugfs_root, dfse, &sof_dfs_fops); /* add to dfsentry list */ @@ -662,24 +450,6 @@ int snd_sof_dbg_init(struct snd_sof_dev *sdev) return err; #endif
-#if IS_ENABLED(CONFIG_SND_SOC_SOF_DEBUG_IPC_FLOOD_TEST) - /* create read-write ipc_flood_count debugfs entry */ - err = snd_sof_debugfs_buf_item(sdev, NULL, 0, - "ipc_flood_count", 0666); - - /* errors are only due to memory allocation, not debugfs */ - if (err < 0) - return err; - - /* create read-write ipc_flood_duration_ms debugfs entry */ - err = snd_sof_debugfs_buf_item(sdev, NULL, 0, - "ipc_flood_duration_ms", 0666); - - /* errors are only due to memory allocation, not debugfs */ - if (err < 0) - return err; -#endif - return 0; } EXPORT_SYMBOL_GPL(snd_sof_dbg_init); diff --git a/sound/soc/sof/sof-priv.h b/sound/soc/sof/sof-priv.h index 1c29199132c5..a20870900a76 100644 --- a/sound/soc/sof/sof-priv.h +++ b/sound/soc/sof/sof-priv.h @@ -50,10 +50,6 @@ extern int sof_core_debug; #define SOF_FORMATS (SNDRV_PCM_FMTBIT_S16_LE | SNDRV_PCM_FMTBIT_S24_LE | \ SNDRV_PCM_FMTBIT_S32_LE | SNDRV_PCM_FMTBIT_FLOAT)
-#define ENABLE_DEBUGFS_CACHEBUF \ - (IS_ENABLED(CONFIG_SND_SOC_SOF_DEBUG_ENABLE_DEBUGFS_CACHE) || \ - IS_ENABLED(CONFIG_SND_SOC_SOF_DEBUG_IPC_FLOOD_TEST)) - /* DSP power state */ enum sof_dsp_power_states { SOF_DSP_PM_D0, @@ -298,7 +294,7 @@ struct snd_sof_dfsentry { * or if it is accessible only when the DSP is in D0. */ enum sof_debugfs_access_type access_type; -#if ENABLE_DEBUGFS_CACHEBUF +#if IS_ENABLED(CONFIG_SND_SOC_SOF_DEBUG_ENABLE_DEBUGFS_CACHE) char *cache_buf; /* buffer to cache the contents of debugfs memory */ #endif struct snd_sof_dev *sdev;
participants (10)
-
Dan Williams -
Dave Ertman -
Ertman, David M
-
gregkh＠linuxfoundation.org -
Leon Romanovsky -
Parav Pandit -
Pierre-Louis Bossart -
Randy Dunlap -
Ranjani Sridharan -
Saleem, Shiraz