And, what I meant in the previous thread was that the check in the given patch wasn't "enough", i.e. it needs more careful checks considering the boundary crossing. That is, you can't simply compare appl_ptr vs old_appl_ptr as a single condition for the backward move.
Indeed, that's why I tried to avoid any checks on pointers :-)
For example, check snd_pcm_playback_avail() and co. That contains a couple of more condition checks and corrections due to the possible boundary crossing. (Here, runtime->boundary may differ depending on 32 or 64bit context.)
The actual implementation of the backward move check would be slightly different from those, but I hope you get my idea.
I think I do but not sure how to precisely deal with the boundary wrap-around.
The only suggestion I have at this point would be to compare the 'avail' space before and after the appl_ptr changes in pcm_lib_apply_appl_ptr(). If the 'avail' space grows as a result of user-space changes, that indicates a rewind (both for capture and playback), doesn't it?
A tentative solution is shared here: https://github.com/thesofproject/linux/pull/3207