On Tue, Sep 12, 2023 at 03:23:34PM -0400, Nícolas F. R. A. Prado wrote:
On Mon, Sep 11, 2023 at 03:29:00PM +0100, Mark Brown wrote:
So, I would interpret the overall result for the suite as being "No errors were found in any of the cards discovered" if there is no configuration file specified which enumerates the set of cards that are expected (if there is a config file that's a different matter, we know what we're expecting). I'm not sure that the different behaviour for 0 cards is super useful.
Right... So what we want to be doing is adding a config file for every platform defining the card(s) and PCMs expected, so that when they're missing a test failure will be triggered which is even more helpful. Although I've noticed that only missing PCMs are detected currently, but I imagine it should be possible to to extend the code to detect missing cards as well.
It seems like a reasonable approach for systems that do want to have this confirmation.
I take it the intention is to expand the conf.d directory with configs for all platforms currently being tested then? There's only one example file there so I wasn't sure.
I think it's a question for people working on individual systems if they want that coverage, for example I don't really care for the things in my CI because I have higher level stuff which will notice any newly missing tests so I don't need the test to do anything here.