[Sound-open-firmware] mixer and formats question
Hi, we have a rather odd issue [1] with mixer/formats, where somehow an error is thrown by firmware when we play an S24/32_LE PCM stream through a mixer and into a DAI configured in S16_LE (because of codec restrictions)
I could use feedback on the following points:
a) was it really a design decision to assume that all sources to the mixer share the same format?
b) if yes, how is the format for a mixer defined? We typically convert using volume elements, but somehow a mixer would need to be set-up with a predefined format (i.e. not propagated by hw_params()) if we want the volume elements to know when/what to convert. I see in my experiments that the format for the mixer sources and sinks seems to be changed dynamically, which doesn't seem quite right to me.
c) if no, same question "how is the format of the mixer defined"? we would need to have the ability to set a format for the mixer so that it can convert on the fly, but we don't seem to set formats anywhere except for PCM streams and DAIs.
Thanks, -Pierre
On Tue, 2020-08-25 at 17:57 -0500, Pierre-Louis Bossart wrote:
Hi, we have a rather odd issue [1] with mixer/formats, where somehow an error is thrown by firmware when we play an S24/32_LE PCM stream through a mixer and into a DAI configured in S16_LE (because of codec restrictions)
I could use feedback on the following points:
a) was it really a design decision to assume that all sources to the mixer share the same format?
Yes. Mixer only mixes same format today, however the PCM converter can be invoked here on the pipeline (but I dont think the mixer does this today).
b) if yes, how is the format for a mixer defined? We typically convert using volume elements, but somehow a mixer would need to be set-up with a predefined format (i.e. not propagated by hw_params()) if we want the volume elements to know when/what to convert. I see in my experiments that the format for the mixer sources and sinks seems to be changed dynamically, which doesn't seem quite right to me.
It could be the PCM converter being invoked by host or dai component depending on which PCM is opened in which order etc ?.
c) if no, same question "how is the format of the mixer defined"? we would need to have the ability to set a format for the mixer so that it can convert on the fly, but we don't seem to set formats anywhere except for PCM streams and DAIs.
Liam
Thanks, -Pierre
[1] https://github.com/thesofproject/sof/pull/3341 _______________________________________________ Sound-open-firmware mailing list Sound-open-firmware@alsa-project.org https://mailman.alsa-project.org/mailman/listinfo/sound-open-firmware
Yes. Mixer only mixes same format today, however the PCM converter can be invoked here on the pipeline (but I dont think the mixer does this today).
b) if yes, how is the format for a mixer defined? We typically convert using volume elements, but somehow a mixer would need to be set-up with a predefined format (i.e. not propagated by hw_params()) if we want the volume elements to know when/what to convert. I see in my experiments that the format for the mixer sources and sinks seems to be changed dynamically, which doesn't seem quite right to me.
It could be the PCM converter being invoked by host or dai component depending on which PCM is opened in which order etc ?.
Yes, the volume can convert, but the question is to what?
take a topology like this
PCM1 (S32_LE) -> volume1----v --- mixer (S??) -- volume -- DAI (S16_LE) PCM2 (S16_LE) -> volume2 -- ^
currently we only set the formats for the PCM devices and the DAIs. It's my understanding that the format used by the mixer would be propagated in the hw_params stage.
That's really bad! This would mean the mixer format would be different depending on which of PCM1 or PCM2 is used first.
I think if we want a consistent result we *have* to define the format used by the mixer with e.g. a token passed to firmware with the config IPC.
A mux is probably similar btw.
On Wed, 2020-08-26 at 10:38 -0500, Pierre-Louis Bossart wrote:
Yes. Mixer only mixes same format today, however the PCM converter can be invoked here on the pipeline (but I dont think the mixer does this today).
b) if yes, how is the format for a mixer defined? We typically convert using volume elements, but somehow a mixer would need to be set- up with a predefined format (i.e. not propagated by hw_params()) if we want the volume elements to know when/what to convert. I see in my experiments that the format for the mixer sources and sinks seems to be changed dynamically, which doesn't seem quite right to me.
It could be the PCM converter being invoked by host or dai component depending on which PCM is opened in which order etc ?.
Yes, the volume can convert, but the question is to what?
I think this is no longer true. Volume used to convert but I think this was moved to pcm_converter.c
take a topology like this
PCM1 (S32_LE) -> volume1----v --- mixer (S??) -- volume -- DAI (S16_LE) PCM2 (S16_LE) -> volume2 -- ^
currently we only set the formats for the PCM devices and the DAIs. It's my understanding that the format used by the mixer would be propagated in the hw_params stage.
That's really bad! This would mean the mixer format would be different depending on which of PCM1 or PCM2 is used first.
Yes, today first user sets the format.
I think if we want a consistent result we *have* to define the format used by the mixer with e.g. a token passed to firmware with the config IPC.
A mux is probably similar btw.
Yes, that's also the intention of topology 2. Each state of the pipeline can have a predefined immutable format/rate/etc.
Liam
On Wed, Aug 26, 2020 at 09:18:57PM +0100, Liam Girdwood wrote:
On Wed, 2020-08-26 at 10:38 -0500, Pierre-Louis Bossart wrote:
Yes. Mixer only mixes same format today, however the PCM converter can be invoked here on the pipeline (but I dont think the mixer does this today).
b) if yes, how is the format for a mixer defined? We typically convert using volume elements, but somehow a mixer would need to be set- up with a predefined format (i.e. not propagated by hw_params()) if we want the volume elements to know when/what to convert. I see in my experiments that the format for the mixer sources and sinks seems to be changed dynamically, which doesn't seem quite right to me.
It could be the PCM converter being invoked by host or dai component depending on which PCM is opened in which order etc ?.
Yes, the volume can convert, but the question is to what?
I think this is no longer true. Volume used to convert but I think this was moved to pcm_converter.c
take a topology like this
PCM1 (S32_LE) -> volume1----v --- mixer (S??) -- volume -- DAI (S16_LE) PCM2 (S16_LE) -> volume2 -- ^
currently we only set the formats for the PCM devices and the DAIs. It's my understanding that the format used by the mixer would be propagated in the hw_params stage.
That's really bad! This would mean the mixer format would be different depending on which of PCM1 or PCM2 is used first.
Yes, today first user sets the format.
déjà vu :-)
This was one of the reasons to develop the media driver subsystem with the user-space subdevice API VIDEO_V4L2_SUBDEV_API and since many years audio wanted to move over to using the media framework too. This might be a long shot, but maybe we should consider that at some point, maybe even sooner than later.
Thanks Guennadi
I think if we want a consistent result we *have* to define the format used by the mixer with e.g. a token passed to firmware with the config IPC.
A mux is probably similar btw.
Yes, that's also the intention of topology 2. Each state of the pipeline can have a predefined immutable format/rate/etc.
Liam
Sound-open-firmware mailing list Sound-open-firmware@alsa-project.org https://mailman.alsa-project.org/mailman/listinfo/sound-open-firmware
participants (3)
-
Guennadi Liakhovetski -
Liam Girdwood -
Pierre-Louis Bossart