[alsa-devel] Reflections on implementing Web MIDI on top of ALSA seq

[Takashi Iwai]

At Fri, 29 May 2015 17:56:20 -0400,
Adam Goode wrote:
> 
> Hi,
> 
> Chrome 43 is now in beta, with Web MIDI support.
> (http://blog.chromium.org/2015/04/chrome-43-beta-web-midi-and-upgrading.html)
> 
> The Linux (and Chrome OS) version of Chrome uses ALSA seq to implement
> Web MIDI. I wanted to reflect a bit on some of the issues I discovered
> while implementing this new API on top of seq.
> 
> This is a little bit long, so the tl;dr is:
> * ALSA seq worked pretty well for implementing Web MIDI, but I need a
> way to get the card number for a given client, before my assumptions
> about the kernel break.
> 
> 
> 
> Here is a quick overview of Web MIDI (http://www.w3.org/TR/webmidi/):
>  - provides hotplug events for port connect/disconnect
>  - incoming events are timestamped
>  - outgoing events can be timestamped for future delivery by the system
>  - timestamps are high precision, and use a monotonic clock
>  - the port abstraction contains "name", "manufacturer", and "version" fields
>  - open ports auto-reconnect if the port disappears and reappears, for
> both send and receive
>  - input and output ports are distinct: there are no duplex ports
>  - uses MIDI bytestreams (not an event structure), but requires full
> MIDI messages to be sent
> 
> The initial prototype of Chrome's Web MIDI support was built on top of
> rawmidi. I rewrote it to use seq instead (to get multiple client and
> user client access).
> 
> Here are my observations:
>  - Getting the "manufacturer" is very hard from seq. I needed to join
> against udev data, which relies on the current implementation of ALSA
> in the kernel. These udev attributes are also used to match for
> transparent reconnect.
>  - I could not use the kernel seq timestamp, either for send or
> receive, since I could not get it to just use a CLOCK_MONOTONIC
> offset. Instead, timestamping is done in userspace.
>  - Hotplug was tricky, since I needed to listen to both seq events and
> udev events. I would prefer just to use udev, since I have to anyway.
> 
> I definitely want to fix the kernel so that I can stop relying on a
> few implementation details. Here are the current assumptions I am
> making:
>  - Card based clients start at 16. (I need to know which clients
> actually have cards)
>  - Clients are in the same order as cards.
>  - There is a 1:1 correspondence between clients and OPL3+rawmidi devices.
> 
> One way to fix these assumptions would be to fully expose all seq
> events through sysfs. Right now, there is a seq device in sysfs but it
> has no information.
> 
> Here are some ways that the kernel might be extended to simplify the
> code and/or fix bad assumptions:
>  1. Expose all seq clients+ports as kobjects, with a symlink to the
> card if it's a hardware device
>  2. Expose all subscriptions through sysfs
>  3. Add a simplified timestamp mechanism that just uses a configurable
> system clock, with at least CLOCK_MONOTONIC available (something like
> what sound/core/timer.c does with tread?)
> 
> The first change would allow me to eliminate all the hardcoded
> assumptions about which cards correspond to which clients. I believe
> it would also allow me to drive the hotplug logic directly from udev,
> instead of both udev and seq.
> The second change is not strictly necessary, but would remove more
> need to react to seq events for some uses.
> The third change would allow me to use the kernel timestamp reliably,
> since right now I timestamp events with CLOCK_MONOTONIC in userspace.

I think both points can be relatively easily implemented as
extension without breaking the backward compatibility.  For example,
the former can be embedded in the new fields (replacing reserved
bytes) in snd_seq_client_info, including the corresponding card#,
rawmidi dev#, etc.  For the latter, we may extend the event flag bit
to indicate that the timestamp is monotonic.

Exposing more information in sysfs is also feasible, of course.  It
doesn't conflict with extensions above, either.

> If I had to start today and create a new MIDI kernel interface for
> ALSA, it would look like this: not-quite-raw-midi:
>  - hotplug events completely driven by sysfs

This should be easy to add even with the current code; we just need to
issue a uevent.

>  - ability to interoperate with existing seq clients (for kernel and
> user clients)
>  - non-exclusive access
>  - reads and writes use a simple structured format, with timestamp and
> a midi message as bytes
>  - no or minimal alsa-lib needed
>  - one device file per port (no subdevices)

This isn't always good, depending on the situation.

>  - some mechanism for creating userspace clients, preferably with some
> number of key-value attributes (so I can report "manufacturer" and
> other things if the user client sets it)

This should be possible.  Maybe we need a new ioctl set, but the
implementation itself must be easy.  The only question is the design
of ioctl struct (and hands to write the code :)


Takashi