[PATCH v2 4/4] soundwire: cadence: further simplify low-level xfer_msg_defer() callback