[alsa-devel] [PATCH] ALSA: core: fix unsigned int pages overflow when comapred

[Takashi Iwai]

On Thu, 19 Jul 2018 08:42:14 +0200,
Takashi Iwai wrote:
> 
> On Thu, 19 Jul 2018 08:08:06 +0200,
> Zhang, Jun wrote:
> > 
> > Hello, Takashi
> > 
> > I think use our patch, it's NOT possible that the returned size is over sgbuf->tblsize.
> > 
> > In function snd_malloc_sgbuf_pages, 
> > 
> > Pages is align page,
> > sgbuf->tblsize is align 32*page,
> > chunk is align 2^n*page,
> > 
> > in our panic case, pages = 123, tlbsize = 128,  
> > 1st loop trunk = 32
> > 2nd loop trunk = 32
> > 3rd loop trunk = 32
> > 4th loop trunk = 16
> > 5th loop trunk = 16
> > So in 5th loop pages-trunk = -5, which make dead loop. 
> 
> Looking at the code again, yeah, you are right, that won't happen.
> 
> And now it becomes clear: the fundamental problem is that
> snd_dma_alloc_pages_fallback() returns a larger size than requested.
> It would be acceptable if the internal allocator aligns a larger size,
> but it shouldn't appear in the returned size outside.  I believe this
> was just a misunderstanding of get_order() usage there.
> (BTW, it's interesting that the allocation with a larger block worked
>  while allocation with a smaller chunk failed; it must be a rare case
>  and that's one of reasons this bug didn't hit frequently.)
> 
> That being said, what we should fix is rather the function
> snd_dma_alloc_pages_fallback() to behave as expected, and it'll be
> like the patch below.

And we can reduce even more lines.  A proper patch is below.


thanks,

Takashi

-- 8< --
From: Takashi Iwai <tiwai at suse.de>
Subject: [PATCH] ALSA: memalloc: Don't exceed over the requested size

snd_dma_alloc_pages_fallback() tries to allocate pages again when the
allocation fails with reduced size.  But the first try actually
*increases* the size to power-of-two, which may give back a larger
chunk than the requested size.  This confuses the callers, e.g. sgbuf
assumes that the size is equal or less, and it may result in a bad
loop due to the underflow and eventually lead to Oops.

The code of this function seems incorrectly assuming the usage of
get_order().  We need to decrease at first, then align to
power-of-two.

Reported-by: he, bo <bo.he at intel.com>
Reported-by: zhang jun <jun.zhang at intel.com>
Cc: <stable at vger.kernel.org>
Signed-off-by: Takashi Iwai <tiwai at suse.de>
---
 sound/core/memalloc.c | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/sound/core/memalloc.c b/sound/core/memalloc.c
index 7f89d3c79a4b..753d5fc4b284 100644
--- a/sound/core/memalloc.c
+++ b/sound/core/memalloc.c
@@ -242,16 +242,12 @@ int snd_dma_alloc_pages_fallback(int type, struct device *device, size_t size,
 	int err;
 
 	while ((err = snd_dma_alloc_pages(type, device, size, dmab)) < 0) {
-		size_t aligned_size;
 		if (err != -ENOMEM)
 			return err;
 		if (size <= PAGE_SIZE)
 			return -ENOMEM;
-		aligned_size = PAGE_SIZE << get_order(size);
-		if (size != aligned_size)
-			size = aligned_size;
-		else
-			size >>= 1;
+		size >>= 1;
+		size = PAGE_SIZE << get_order(size);
 	}
 	if (! dmab->area)
 		return -ENOMEM;
-- 
2.18.0