[alsa-devel] USB transfer_buffer allocations on 64bit systems

[Robert Hancock]

On 04/07/2010 08:59 AM, Alan Stern wrote:
> On Wed, 7 Apr 2010, Daniel Mack wrote:
>
>> Hi,
>>
>> I was pointed to https://bugzilla.kernel.org/show_bug.cgi?id=15580 by
>> Pedro Ribeiro, and unfortunately I'm pretty late in the game. I wasn't
>> aware of that thread until yesterday.
>>
>> While the report is quite confusing, the reason seams pretty clear to me
>> as I've been thru quite some time-demanding debugging of a very similar
>> issue on Mac OS X. But I'm not totally sure whether we really hit the
>> same issue here, so I'd like to have your opinions first.
>>
>> The problem is appearantly the way the transfer buffer is allocated in
>> the drivers. In the snd-usb-caiaq driver, I used kzalloc() to get memory
>> which works fine on 32bit systems. On x86_64, however, it seems that
>> kzalloc() hands out memory beyond the 32bit addressable boundary, which
>> the DMA controller of the 32bit PCI-connected EHCI controller is unable
>> to write to or read from. Am I correct on this conclusion?
>
> That seems like the right answer.  You are correct that an EHCI
> controller capable only of 32-bit memory accesses would not be able to
> use a buffer above the 4 GB line.
>
>> Depending on the condition of the memory management, things might work
>> or not, and especially right after a reboot, there's a better chance to
>> get lower memory.
>>
>> The fix is to use usb_buffer_alloc() for that purpose which ensures
>> memory that is suitable for DMA. And on x86_64, this also means that the
>> upper 32 bits of the address returned are all 0's.
>
> That is not a good fix.  usb_buffer_alloc() provides coherent memory,
> which is not what we want.  I believe the correct fix is to specify the
> GFP_DMA32 flag in the kzalloc() call.
>
> Of course, some EHCI hardware _is_ capable of using 64-bit addresses.
> But not all, and other controller types aren't.  In principle we could
> create a new allocation routine, which would take a pointer to the USB
> bus as an additional argument and use it to decide whether the memory
> needs to lie below 4 GB.  I'm not sure adding this extra complexity
> would be worthwhile.

AFAIK, the driver shouldn't have to worry about this at all. When the 
buffer gets DMA-mapped for the controller, the DMA mapping code should 
see that the device has a 32-bit DMA mask and either bounce or IOMMU-map 
the memory so that it appears below 4GB.

Not to say that something might not be sabotaging this somehow, but this 
complexity really shouldn't be needed.

>
>> If what I've stated is true, there are quite some more drivers affected
>> by this issue.
>
> Practically every USB driver, I should think.  And maybe a lot of
> non-USB drivers too.
>
>> I collected a list of places where similar fixes are
>> needed, and I can send patches if I get a thumbs-up.
>>
>> Pedro is currently testing a patch I sent out yesterday.
>
> Good work -- it certainly would have taken me a long time to figure
> this out.
>
> Alan Stern
>