[Sound-open-firmware] [PATCH] topology: speed up topology build
Speed up topology with batch processing of m4 files Signed-off-by: Ranjani Sridharan <ranjani.sridharan@linux.intel.com> --- topology/test/tplg-build.sh | 37 ++++++++++++++++++------------------- 1 file changed, 18 insertions(+), 19 deletions(-) diff --git a/topology/test/tplg-build.sh b/topology/test/tplg-build.sh index 7c55fba..c85b5b5 100755 --- a/topology/test/tplg-build.sh +++ b/topology/test/tplg-build.sh @@ -8,14 +8,14 @@ set -e # M4 preprocessor flags -M4_FLAGS="-I ../ -I ../m4 -I ../common" +export M4PATH="../:../m4:../common" # Simple component test cases # can be used on components with 1 sink and 1 source. SIMPLE_TESTS=(test-ssp test-capture-ssp test-playback-ssp) TONE_TEST=test-tone-playback-ssp - - +TEST_STRINGS="" +M4_STRINGS="" # process m4 simple tests - # simple_test(name, pipe_name, be_name, format, dai_id, dai_format, dai_phy_bits, dai_data_bits dai_bclk) # 1) name - test filename suffix @@ -35,21 +35,11 @@ function simple_test { for i in ${SIMPLE_TESTS[@]} do TFILE="$i$5-${11}-$2-$4-$6-48k-$((${10} / 1000))k-$1" - echo "M4 pre-processing test $i -> ${TFILE}" - m4 ${M4_FLAGS} \ - -DTEST_PIPE_NAME="$2" \ - -DTEST_DAI_LINK_NAME="$3" \ - -DTEST_SSP_PORT=$5 \ - -DTEST_SSP_FORMAT=$6 \ - -DTEST_PIPE_FORMAT=$4 \ - -DTEST_SSP_BCLK=$9 \ - -DTEST_SSP_MCLK=${10} \ - -DTEST_SSP_PHY_BITS=$7 \ - -DTEST_SSP_DATA_BITS=$8 \ - -DTEST_SSP_MODE=${11} \ - $i.m4 > ${TFILE}.conf - echo "Compiling test $i -> ${TFILE}.tplg" - alsatplg -v 1 -c ${TFILE}.conf -o ${TFILE}.tplg + #create input string for batch m4 processing + M4_STRINGS+="-DTEST_PIPE_NAME=$2,-DTEST_DAI_LINK_NAME=$3,-DTEST_SSP_PORT=$5,-DTEST_SSP_FORMAT=$6,-DTEST_PIPE_FORMAT=$4,-DTEST_SSP_BCLK=$9,-DTEST_SSP_MCLK=${10},-DTEST_SSP_PHY_BITS=$7,-DTEST_SSP_DATA_BITS=$8,-DTEST_SSP_MODE=${11},$i.m4,${TFILE}," + #create input string for batch processing of conf files + TEST_STRINGS+=${TFILE}"," + done } @@ -69,7 +59,7 @@ function simple_test { function tone_test { TFILE="$TONE_TEST$5-$2-$4-$6-48k-$1" echo "M4 pre-processing test $TONE_TEST -> ${TFILE}" - m4 ${M4_FLAGS} \ + m4 \ -DTEST_PIPE_NAME="$2" \ -DTEST_DAI_LINK_NAME="$3" \ -DTEST_SSP_PORT=$5 \ @@ -84,6 +74,8 @@ function tone_test { alsatplg -v 1 -c ${TFILE}.conf -o ${TFILE}.tplg } +echo "Preparing topology build input..." + # Pre-process the simple tests simple_test nocodec passthrough "NoCodec" s16le 2 s16le 20 16 1920000 19200000 I2S simple_test nocodec passthrough "NoCodec" s24le 2 s24le 25 24 2400000 19200000 I2S @@ -166,4 +158,11 @@ simple_test nocodec src "NoCodec" s24le 4 s24le 25 24 2400000 24000000 I2S # Tone test: Tone component only supports s32le currently tone_test codec tone "SSP2-Codec" s32le 2 s16le 20 16 1920000 19200000 +echo "Batch processing m4 files..." +#m4 processing +echo $M4_STRINGS | tr '\n' '\0' | xargs -d ',' -n12 -P10 bash -c 'm4 "${@:1:${#}-1}" > ${12}.conf' m4 + +#execute alsatplg to create topology binary +TEST_STRINGS=${TEST_STRINGS%?} +echo $TEST_STRINGS | tr '\n' '\0' | xargs -d ',' -n1 -P10 -I string alsatplg -v 1 -c string".conf" -o string".tplg" -- 2.14.1
On Wed, 2018-04-25 at 15:46 -0700, Ranjani Sridharan wrote:
Speed up topology with batch processing of m4 files
I was thinking xargs too :) Just one minor comment. Btw, what's the speed up ?
Signed-off-by: Ranjani Sridharan <ranjani.sridharan@linux.intel.com> --- topology/test/tplg-build.sh | 37 ++++++++++++++++++------------------- 1 file changed, 18 insertions(+), 19 deletions(-)
diff --git a/topology/test/tplg-build.sh b/topology/test/tplg-build.sh index 7c55fba..c85b5b5 100755 --- a/topology/test/tplg-build.sh +++ b/topology/test/tplg-build.sh @@ -8,14 +8,14 @@ set -e
# M4 preprocessor flags -M4_FLAGS="-I ../ -I ../m4 -I ../common" +export M4PATH="../:../m4:../common"
# Simple component test cases # can be used on components with 1 sink and 1 source. SIMPLE_TESTS=(test-ssp test-capture-ssp test-playback-ssp) TONE_TEST=test-tone-playback-ssp - - +TEST_STRINGS="" +M4_STRINGS="" # process m4 simple tests - # simple_test(name, pipe_name, be_name, format, dai_id, dai_format, dai_phy_bits, dai_data_bits dai_bclk) # 1) name - test filename suffix @@ -35,21 +35,11 @@ function simple_test { for i in ${SIMPLE_TESTS[@]} do TFILE="$i$5-${11}-$2-$4-$6-48k-$((${10} / 1000))k-$1" - echo "M4 pre-processing test $i -> ${TFILE}" - m4 ${M4_FLAGS} \ - -DTEST_PIPE_NAME="$2" \ - -DTEST_DAI_LINK_NAME="$3" \ - -DTEST_SSP_PORT=$5 \ - -DTEST_SSP_FORMAT=$6 \ - -DTEST_PIPE_FORMAT=$4 \ - -DTEST_SSP_BCLK=$9 \ - -DTEST_SSP_MCLK=${10} \ - -DTEST_SSP_PHY_BITS=$7 \ - -DTEST_SSP_DATA_BITS=$8 \ - -DTEST_SSP_MODE=${11} \ - $i.m4 > ${TFILE}.conf - echo "Compiling test $i -> ${TFILE}.tplg" - alsatplg -v 1 -c ${TFILE}.conf -o ${TFILE}.tplg + #create input string for batch m4 processing + M4_STRINGS+="-DTEST_PIPE_NAME=$2,-DTEST_DAI_LINK_NAME=$3,- DTEST_SSP_PORT=$5,-DTEST_SSP_FORMAT=$6,-DTEST_PIPE_FORMAT=$4,- DTEST_SSP_BCLK=$9,-DTEST_SSP_MCLK=${10},-DTEST_SSP_PHY_BITS=$7,- DTEST_SSP_DATA_BITS=$8,-DTEST_SSP_MODE=${11},$i.m4,${TFILE},"
Can this be spread over multiple lines using \ to make it more readable.
+ #create input string for batch processing of conf files + TEST_STRINGS+=${TFILE}"," + done }
@@ -69,7 +59,7 @@ function simple_test { function tone_test { TFILE="$TONE_TEST$5-$2-$4-$6-48k-$1" echo "M4 pre-processing test $TONE_TEST -> ${TFILE}" - m4 ${M4_FLAGS} \ + m4 \ -DTEST_PIPE_NAME="$2" \ -DTEST_DAI_LINK_NAME="$3" \ -DTEST_SSP_PORT=$5 \ @@ -84,6 +74,8 @@ function tone_test { alsatplg -v 1 -c ${TFILE}.conf -o ${TFILE}.tplg }
+echo "Preparing topology build input..." + # Pre-process the simple tests simple_test nocodec passthrough "NoCodec" s16le 2 s16le 20 16 1920000 19200000 I2S simple_test nocodec passthrough "NoCodec" s24le 2 s24le 25 24 2400000 19200000 I2S @@ -166,4 +158,11 @@ simple_test nocodec src "NoCodec" s24le 4 s24le 25 24 2400000 24000000 I2S # Tone test: Tone component only supports s32le currently tone_test codec tone "SSP2-Codec" s32le 2 s16le 20 16 1920000 19200000
+echo "Batch processing m4 files..." +#m4 processing +echo $M4_STRINGS | tr '\n' '\0' | xargs -d ',' -n12 -P10 bash -c 'm4 "${@:1:${#}-1}" > ${12}.conf' m4 + +#execute alsatplg to create topology binary +TEST_STRINGS=${TEST_STRINGS%?} +echo $TEST_STRINGS | tr '\n' '\0' | xargs -d ',' -n1 -P10 -I string alsatplg -v 1 -c string".conf" -o string".tplg"
On Mon, 2018-04-30 at 16:33 +0100, Liam Girdwood wrote:
I was thinking xargs too :) Just one minor comment.
Btw, what's the speed up ?
Forgot to add, we should also check if xargs is available in configure.ac too (it will be on mostly everything but you never know). Oh I think the xargs -P should be number of CPU cores. There are some ways to read this back via bash (just Google). Thanks Liam
On Mon, 2018-04-30 at 16:40 +0100, Liam Girdwood wrote:
On Mon, 2018-04-30 at 16:33 +0100, Liam Girdwood wrote:
I was thinking xargs too :) Just one minor comment.
Btw, what's the speed up ?
Forgot to add, we should also check if xargs is available in configure.ac too (it will be on mostly everything but you never know).
Oh I think the xargs -P should be number of CPU cores. There are some ways to read this back via bash (just Google).
The speed was almost 2X. On my host, the build time dropped from 18s to ~9s. I'll make the check in configure and fix the number of cores too. Thanks!
Thanks
Liam
_______________________________________________ Sound-open-firmware mailing list Sound-open-firmware@alsa-project.org http://mailman.alsa-project.org/mailman/listinfo/sound-open-firmware
participants (2)
-
Liam Girdwood -
Ranjani Sridharan