Ytdyklly

I have a really strange situation here. My PC works fine, at least in most cases, but there's one thing that I can't deal with. When I try to copy a file from my pendrive, everything is ok -- I got 16-19M/s , it works pretty well. But when I try to copy something to the same pendrive, my PC freezes. The mouse pointer stops moving for a sec or two, then it moves a little bit and it stops again. When something is playing, for example, in Amarok, the sound acts like a machine gun. The speed jumps from 500K/s to 15M/s, average 8M/s. This occurs only when I'm copying something to a pendrive. When the process of copying is done, everything backs to normal.

I tried everything -- other pendrive, a different USB port on front panel or those ports from back, I even changed the USB pins on motherboard (front panel), but no matter where I put my USB stick, it's always the same. I tried different filesystem -- fat32, ext4. I have no problem with the device on Windows, on my laptop. It has to be my PC or something in my system. I have no idea what to look for. I'm using Debian testing with standalone Openbox. My PC is kind of old -- Pentium D 3GHz, 1GiB of RAM, 1,5TB WD Green disk. If you have something that would help me to solve this issue, I'd be glad to hear that.

I don't know what else info I should provide, but if you need something, just ask, I'll update this post as soon as possible.

I tried to reproduce this problem on ubuntu 13.04 live cd. I mounted my encrypted partition + encrypted swap and connected my pendrive to a usb port. Next I tried to start some apps, and now I have ~820MiB in RAM and about 400MiB in SWAP. There's no problem with copying, no freezing at all, everything is as it should be. So, it looks like it's a fault of the system, but where exactly? What would cause such a weird behavior?

Are you using a 64-bit version of Linux with a lot of memory? In that case the problem could be that Linux can lock for minutes on big writes on slow devices like for
example SD cards or USB sticks. It's a known bug that should be fixed in newer kernels.

See http://lwn.net/Articles/572911/

Workaround: as root issue:

echo $((16*1024*1024)) > /proc/sys/vm/dirty_background_bytes

echo $((48*1024*1024)) > /proc/sys/vm/dirty_bytes

I have added it to my /etc/rc.local file in my 64bit machines.

TANSTAAFL; this change can (and probably will) reduce your throughput to these devices --- it's a compromise between latency and speed. To get back to the previous behavior you can

echo 0 > /proc/sys/vm/dirty_background_bytes

echo 0 > /proc/sys/vm/dirty_bytes

...which are the default values, meaning that the writeback behavior will be controlled by the parameters dirty_ratio and dirty_background_ratio.

Note for the not-so-expert-with-linux people: the files in /proc are pseudofiles --- just communication channels between the kernel and user space. Never use an editor to change or look at them; get instead a shell prompt --- for example, with sudo -i (Ubuntu flavors) or su root and use echo and cat).

Update 2016/04/18 it seems that, after all, the problem is still here. You can look at it at LWN.net, in this article about writeback queues.

The reason can be write amplification, as the system tries to write in chunks that are smaller, than erase block (doing read/mod/write) + block misalignment.

To check yours current setting do:

cat /sys/block/sd**X**/device/max_sectors

You can tune hall rules for those devices:

Change value of USB "max_sectors" for an entire family of devices

In this case I had replaced max_sectors for all devices, that used default of 240 (USB storage) to 32K sectors or 2K sectors.

On my system (Mageia 4, 3.14.24 core i7) I had to do this due terribly slow write speeds (2MB/sec) on Kingston DT101 G2 16GB:

vi /usr/lib/udev/rules.d/81-udisks_maxsect.rules

and add:

SUBSYSTEMS=="scsi", ATTR{max_sectors}=="240", ATTR{max_sectors}="32678"

And the dd write speed went up 3x times. mc cp probably 10-20x up (after I had started first partition @8192'th sector and reformatted with 64k aligned clusters):

fdisk -u /dev/sdh # make DOS compat off if on

mkfs.vfat /dev/sdh1 -n KINGSTON16G -s 128 **-R 4592*** and use *fsck.vfat -v /dev/sdh1

to check alignment (check [data start sector] should be multiple of 128 (cluster size)). Adjust number of the reserved sectors (-R) if needed.

Default max_sectors (240) seem to cause high write amplification on some of the cheap new drives. But be very careful with such high setting, the similar effect is achieved at 2048 sectors (probably 1M erase blocks:

SUBSYSTEMS=="scsi", ATTR{max_sectors}=="240", ATTR{max_sectors}="2048"

Test all yours old USB devices, that they still work well. Use vendor/model attributes in the rules files to be more specific.

hardware vs. software

I've run into strange problem similar to this with USB thumbdrives, and in my research it's almost always either a driver issue or the specific hardware within the PC/Motherboard.

I know this because I've got several systems that are identical hardware, and on one, I can do this operation without issue, while on another the problem shows up.

What to do?

You options are really limited here. About the only things you can do are make sure you have the latest BIOS/firmware installed on your system, and make sure you have the latest versions of your disto's packages.

Beyond that all I can suggest is making sure that you avoid this situation by not attempting to copy files while another copy is in progress.

If you have the type of personality where things like this irk you, you could try another live distro of Linux and repeat the steps that lead to your problem. This would just eliminate whether it's a distro specific issue or a hardware issue as I've described above. It would be a small consolation, but I always like to know things rather than bury my head in the sand, and not.

Anything else?

If you're truly obsessive you could try running the application that you're doing the copy with through strace in the hopes of catching the system in whatever system call is freezing. You should be able to do this from the command line as well.

Example

$ strace -o cp1.log cp -r /path/to/dir1 /path/to/usb/. 

Then while that's running start another one.

$ strace -o cp2.log cp -r /path/to/dir2 /path/to/usb/. 

The system will hopefully freeze during this operation and maybe you'll get lucky and find some smoke in either of those log files.