[alsa-devel] sound: use-after-free in snd_timer_stop
Hello,
I've hit the following use-after-free while running syzkaller fuzzer. It is followed by a splat of other reports and finally kernel death. I wasn't able to reproduce it with a standalone C program (there is probably some global state involved). But it reproduces by replaying fuzzer logs in a loop (you will need Go toolchain):
$ go get github.com/google/syzkaller $ cd $GOPATH/src/github.com/google/syzkaller $ make executor execprog $ scp bin/syz-executor bin/syz-execprog your@machine $ scp snd_timer_stop your@machine # the attached file on test machine: $ ./syz-execprog -executor ./syz-executor -cover=0 -repeat=0 -procs=16 snd_timer_stop
================================================================== BUG: KASAN: use-after-free in _snd_timer_stop+0x394/0x450 at addr ffff8800356d0b70 Read of size 4 by task syz-executor/6313 ============================================================================= BUG kmalloc-256 (Not tainted): kasan: bad access detected -----------------------------------------------------------------------------
INFO: Allocated in snd_timer_instance_new+0x52/0x3a0 age=20 cpu=1 pid=6312 [< none >] ___slab_alloc+0x486/0x4e0 mm/slub.c:2468 [< none >] __slab_alloc+0x66/0xc0 mm/slub.c:2497 [< inline >] slab_alloc_node mm/slub.c:2560 [< inline >] slab_alloc mm/slub.c:2602 [< none >] kmem_cache_alloc_trace+0x284/0x310 mm/slub.c:2619 [< inline >] kmalloc include/linux/slab.h:458 [< inline >] kzalloc include/linux/slab.h:602 [< none >] snd_timer_instance_new+0x52/0x3a0 sound/core/timer.c:105 [< none >] snd_timer_open+0x4ff/0xc50 sound/core/timer.c:286 [< none >] snd_seq_timer_open+0x223/0x540 sound/core/seq/seq_timer.c:279 [< none >] snd_seq_queue_use+0x147/0x230 sound/core/seq/seq_queue.c:526 [< none >] snd_seq_queue_alloc+0x36a/0x4d0 sound/core/seq/seq_queue.c:197 [< none >] snd_seq_ioctl_create_queue+0xdb/0x2b0 sound/core/seq/seq_clientmgr.c:1536 [< none >] snd_seq_do_ioctl+0x19a/0x1c0 sound/core/seq/seq_clientmgr.c:2209 [< none >] snd_seq_ioctl+0x5d/0x80 sound/core/seq/seq_clientmgr.c:2224 [< inline >] vfs_ioctl fs/ioctl.c:43 [< none >] do_vfs_ioctl+0x681/0xe40 fs/ioctl.c:607 [< inline >] SYSC_ioctl fs/ioctl.c:622 [< none >] SyS_ioctl+0x8f/0xc0 fs/ioctl.c:613 [< none >] entry_SYSCALL_64_fastpath+0x16/0x7a arch/x86/entry/entry_64.S:185
INFO: Freed in snd_timer_close+0x351/0x5e0 age=10 cpu=3 pid=6345 [< none >] __slab_free+0x1fc/0x320 mm/slub.c:2678 [< inline >] slab_free mm/slub.c:2833 [< none >] kfree+0x2a8/0x2d0 mm/slub.c:3662 [< none >] snd_timer_close+0x351/0x5e0 sound/core/timer.c:364 [< none >] snd_seq_timer_close+0x9e/0x100 sound/core/seq/seq_timer.c:312 [< none >] snd_seq_queue_timer_close+0x28/0x50 sound/core/seq/seq_queue.c:473 [< none >] snd_seq_ioctl_set_queue_timer+0x159/0x300 sound/core/seq/seq_clientmgr.c:1809 [< none >] snd_seq_do_ioctl+0x19a/0x1c0 sound/core/seq/seq_clientmgr.c:2209 [< none >] snd_seq_ioctl+0x5d/0x80 sound/core/seq/seq_clientmgr.c:2224 [< inline >] vfs_ioctl fs/ioctl.c:43 [< none >] do_vfs_ioctl+0x681/0xe40 fs/ioctl.c:607 [< inline >] SYSC_ioctl fs/ioctl.c:622 [< none >] SyS_ioctl+0x8f/0xc0 fs/ioctl.c:613 [< none >] entry_SYSCALL_64_fastpath+0x16/0x7a arch/x86/entry/entry_64.S:185
INFO: Slab 0xffffea0000d5b400 objects=22 used=8 fp=0xffff8800356d0b60 flags=0x1fffc0000004080 INFO: Object 0xffff8800356d0b60 @offset=2912 fp=0xffff8800356d27d0 CPU: 0 PID: 6313 Comm: syz-executor Tainted: G B 4.4.0+ #222 Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011 00000000ffffffff ffff880036757748 ffffffff82904add ffff88003e807000 ffff8800356d0b60 ffff8800356d0000 ffff880036757778 ffffffff8172af94 ffff88003e807000 ffffea0000d5b400 ffff8800356d0b60 ffff8800356d0b60
Call Trace: [<ffffffff817344ae>] __asan_report_load4_noabort+0x3e/0x40 mm/kasan/report.c:294 [<ffffffff84aefda4>] _snd_timer_stop+0x394/0x450 sound/core/timer.c:488 [<ffffffff84aefe84>] snd_timer_stop+0x24/0x140 sound/core/timer.c:535 [<ffffffff84b6947e>] snd_seq_timer_close+0x7e/0x100 sound/core/seq/seq_timer.c:311 [<ffffffff84b6235b>] queue_delete+0x3b/0x90 sound/core/seq/seq_queue.c:146 [<ffffffff84b63f86>] snd_seq_queue_client_leave+0x36/0x130 sound/core/seq/seq_queue.c:593 [<ffffffff84b558a8>] seq_free_client1+0x58/0x290 sound/core/seq/seq_clientmgr.c:273 [<ffffffff84b55b45>] seq_free_client+0x65/0x160 sound/core/seq/seq_clientmgr.c:299 [<ffffffff84b5882d>] snd_seq_release+0x4d/0xb0 sound/core/seq/seq_clientmgr.c:380 [<ffffffff8177b453>] __fput+0x233/0x780 fs/file_table.c:208 [<ffffffff8177ba25>] ____fput+0x15/0x20 fs/file_table.c:244 [<ffffffff8139a8fb>] task_work_run+0x16b/0x200 kernel/task_work.c:115 [< inline >] exit_task_work include/linux/task_work.h:21 [<ffffffff81347b4b>] do_exit+0x8bb/0x2b20 kernel/exit.c:750 [<ffffffff81349f28>] do_group_exit+0x108/0x320 kernel/exit.c:880 [<ffffffff8136d124>] get_signal+0x5e4/0x1500 kernel/signal.c:2307 [<ffffffff81192d83>] do_signal+0x83/0x1c90 arch/x86/kernel/signal.c:712 [<ffffffff81006685>] exit_to_usermode_loop+0x1a5/0x210 arch/x86/entry/common.c:247 [< inline >] prepare_exit_to_usermode arch/x86/entry/common.c:282 [<ffffffff8100851a>] syscall_return_slowpath+0x2ba/0x340 arch/x86/entry/common.c:344 [<ffffffff85e74a62>] int_ret_from_sys_call+0x25/0x9f arch/x86/entry/entry_64.S:281 ==================================================================
On commit afd2ff9b7e1b367172f18ba7f693dfb62bdcb2dc (Jan 10).
On Tue, 12 Jan 2016 11:22:09 +0100, Dmitry Vyukov wrote:
Hello,
I've hit the following use-after-free while running syzkaller fuzzer. It is followed by a splat of other reports and finally kernel death. I wasn't able to reproduce it with a standalone C program (there is probably some global state involved). But it reproduces by replaying fuzzer logs in a loop (you will need Go toolchain):
$ go get github.com/google/syzkaller $ cd $GOPATH/src/github.com/google/syzkaller $ make executor execprog
Just a note: I had to run "mkdir bin" beforehand. Also for my distro, I had to remove -static.
$ scp bin/syz-executor bin/syz-execprog your@machine $ scp snd_timer_stop your@machine # the attached file on test machine: $ ./syz-execprog -executor ./syz-executor -cover=0 -repeat=0 -procs=16 snd_timer_stop
I tried this but couldn't reproduce the issue on my machine, unfortunately. (Instead I hit another kernel panic regarding apparmor :)
But looking through your log, it seems like a missing race protection. Does the patch below work for you?
thanks,
Takashi
--- diff --git a/sound/core/seq/seq_queue.c b/sound/core/seq/seq_queue.c index 7dfd0f429410..8ea17d0a4299 100644 --- a/sound/core/seq/seq_queue.c +++ b/sound/core/seq/seq_queue.c @@ -142,8 +142,10 @@ static struct snd_seq_queue *queue_new(int owner, int locked) static void queue_delete(struct snd_seq_queue *q) { /* stop and release the timer */ + mutex_lock(&q->timer_mutex); snd_seq_timer_stop(q->timer); snd_seq_timer_close(q); + mutex_unlock(&q->timer_mutex); /* wait until access free */ snd_use_lock_sync(&q->use_lock); /* release resources... */ @@ -470,7 +472,9 @@ int snd_seq_queue_timer_close(int queueid) queue = queueptr(queueid); if (queue == NULL) return -EINVAL; + mutex_lock(&queue->timer_mutex); snd_seq_timer_close(queue); + mutex_unlock(&queue->timer_mutex); queuefree(queue); return result; }
On Tue, 12 Jan 2016 15:05:24 +0100, Takashi Iwai wrote:
On Tue, 12 Jan 2016 11:22:09 +0100, Dmitry Vyukov wrote:
Hello,
I've hit the following use-after-free while running syzkaller fuzzer. It is followed by a splat of other reports and finally kernel death. I wasn't able to reproduce it with a standalone C program (there is probably some global state involved). But it reproduces by replaying fuzzer logs in a loop (you will need Go toolchain):
$ go get github.com/google/syzkaller $ cd $GOPATH/src/github.com/google/syzkaller $ make executor execprog
Just a note: I had to run "mkdir bin" beforehand. Also for my distro, I had to remove -static.
$ scp bin/syz-executor bin/syz-execprog your@machine $ scp snd_timer_stop your@machine # the attached file on test machine: $ ./syz-execprog -executor ./syz-executor -cover=0 -repeat=0 -procs=16 snd_timer_stop
I tried this but couldn't reproduce the issue on my machine, unfortunately. (Instead I hit another kernel panic regarding apparmor :)
But looking through your log, it seems like a missing race protection. Does the patch below work for you?
Gah, scratch this. The second hunk causes a deadlock. The revised patch is below (just containing the first hunk).
Takashi
--- diff --git a/sound/core/seq/seq_queue.c b/sound/core/seq/seq_queue.c index 7dfd0f429410..0bec02e89d51 100644 --- a/sound/core/seq/seq_queue.c +++ b/sound/core/seq/seq_queue.c @@ -142,8 +142,10 @@ static struct snd_seq_queue *queue_new(int owner, int locked) static void queue_delete(struct snd_seq_queue *q) { /* stop and release the timer */ + mutex_lock(&q->timer_mutex); snd_seq_timer_stop(q->timer); snd_seq_timer_close(q); + mutex_unlock(&q->timer_mutex); /* wait until access free */ snd_use_lock_sync(&q->use_lock); /* release resources... */
On Tue, Jan 12, 2016 at 3:16 PM, Takashi Iwai tiwai@suse.de wrote:
On Tue, 12 Jan 2016 15:05:24 +0100, Takashi Iwai wrote:
On Tue, 12 Jan 2016 11:22:09 +0100, Dmitry Vyukov wrote:
Hello,
I've hit the following use-after-free while running syzkaller fuzzer. It is followed by a splat of other reports and finally kernel death. I wasn't able to reproduce it with a standalone C program (there is probably some global state involved). But it reproduces by replaying fuzzer logs in a loop (you will need Go toolchain):
$ go get github.com/google/syzkaller $ cd $GOPATH/src/github.com/google/syzkaller $ make executor execprog
Just a note: I had to run "mkdir bin" beforehand. Also for my distro, I had to remove -static.
$ scp bin/syz-executor bin/syz-execprog your@machine $ scp snd_timer_stop your@machine # the attached file on test machine: $ ./syz-execprog -executor ./syz-executor -cover=0 -repeat=0 -procs=16 snd_timer_stop
I tried this but couldn't reproduce the issue on my machine, unfortunately. (Instead I hit another kernel panic regarding apparmor :)
But looking through your log, it seems like a missing race protection. Does the patch below work for you?
Gah, scratch this. The second hunk causes a deadlock. The revised patch is below (just containing the first hunk).
Yes, this patch fixes the crashes for me. Reproduction may require running syz-execprog for several minutes.
Tested-by: Dmitry Vyukov dvyukov@google.com
diff --git a/sound/core/seq/seq_queue.c b/sound/core/seq/seq_queue.c index 7dfd0f429410..0bec02e89d51 100644 --- a/sound/core/seq/seq_queue.c +++ b/sound/core/seq/seq_queue.c @@ -142,8 +142,10 @@ static struct snd_seq_queue *queue_new(int owner, int locked) static void queue_delete(struct snd_seq_queue *q) { /* stop and release the timer */
mutex_lock(&q->timer_mutex); snd_seq_timer_stop(q->timer); snd_seq_timer_close(q);mutex_unlock(&q->timer_mutex); /* wait until access free */ snd_use_lock_sync(&q->use_lock); /* release resources... */
On Tue, 12 Jan 2016 17:46:15 +0100, Dmitry Vyukov wrote:
On Tue, Jan 12, 2016 at 3:16 PM, Takashi Iwai tiwai@suse.de wrote:
On Tue, 12 Jan 2016 15:05:24 +0100, Takashi Iwai wrote:
On Tue, 12 Jan 2016 11:22:09 +0100, Dmitry Vyukov wrote:
Hello,
I've hit the following use-after-free while running syzkaller fuzzer. It is followed by a splat of other reports and finally kernel death. I wasn't able to reproduce it with a standalone C program (there is probably some global state involved). But it reproduces by replaying fuzzer logs in a loop (you will need Go toolchain):
$ go get github.com/google/syzkaller $ cd $GOPATH/src/github.com/google/syzkaller $ make executor execprog
Just a note: I had to run "mkdir bin" beforehand. Also for my distro, I had to remove -static.
$ scp bin/syz-executor bin/syz-execprog your@machine $ scp snd_timer_stop your@machine # the attached file on test machine: $ ./syz-execprog -executor ./syz-executor -cover=0 -repeat=0 -procs=16 snd_timer_stop
I tried this but couldn't reproduce the issue on my machine, unfortunately. (Instead I hit another kernel panic regarding apparmor :)
But looking through your log, it seems like a missing race protection. Does the patch below work for you?
Gah, scratch this. The second hunk causes a deadlock. The revised patch is below (just containing the first hunk).
Yes, this patch fixes the crashes for me. Reproduction may require running syz-execprog for several minutes.
Tested-by: Dmitry Vyukov dvyukov@google.com
Good to hear. FWIW, below is the final patch I'm going to queue. Thanks!
Takashi
-- 8< -- From: Takashi Iwai tiwai@suse.de Subject: [PATCH] ALSA: seq: Fix race at timer setup and close
ALSA sequencer code has an open race between the timer setup ioctl and the close of the client. This was triggered by syzkaller fuzzer, and a use-after-free was caught there as a result.
This patch papers over it by adding a proper queue->timer_mutex lock around the timer-related calls in the relevant code path.
Reported-by: Dmitry Vyukov dvyukov@google.com Tested-by: Dmitry Vyukov dvyukov@google.com Cc: stable@vger.kernel.org Signed-off-by: Takashi Iwai tiwai@suse.de --- sound/core/seq/seq_queue.c | 2 ++ 1 file changed, 2 insertions(+)
diff --git a/sound/core/seq/seq_queue.c b/sound/core/seq/seq_queue.c index 7dfd0f429410..0bec02e89d51 100644 --- a/sound/core/seq/seq_queue.c +++ b/sound/core/seq/seq_queue.c @@ -142,8 +142,10 @@ static struct snd_seq_queue *queue_new(int owner, int locked) static void queue_delete(struct snd_seq_queue *q) { /* stop and release the timer */ + mutex_lock(&q->timer_mutex); snd_seq_timer_stop(q->timer); snd_seq_timer_close(q); + mutex_unlock(&q->timer_mutex); /* wait until access free */ snd_use_lock_sync(&q->use_lock); /* release resources... */
participants (2)
-
Dmitry Vyukov -
Takashi Iwai