Wipe a USB Stick Before Formatting

Preparing 8G of files for a project made me check my spare USB sticks. One was big enough, 32 GB, but was previously formatted NTFS for file transfers to Windows systems. Preferring EXT3 for the upcoming files, just repartition and format. Finding some leftover data from two reformats ago, it’s time to wipe it.

Plugging a USBv2 stick into a USBv3 socket runs faster than plugging it into a USBv2 socket, although not as fast as using a USBv3 stick. Don’t want to wait too long. Wipe it with dd(1):

# time dd if=/dev/zero of=/dev/sdc status=progress
32713724928 bytes (33 GB, 30 GiB) copied, 8594 s, 3.8 MB/s
dd: writing to '/dev/sdc': No space left on device 
63901441+0 records in 
63901441+0 records out
32717537280 bytes (33 GB, 30 GiB) copied, 8709.25 s, 3.8 MB/s

real    145m9.250s 
user    0m36.508s 
sys     6m4.407s

That was surprisingly long. Progress reporting started out averaging over 6 MB/s, but after some time dropped to the final rate. Could it run faster?

Defaulting the block size in a simple dd command line uses 512 bytes. That’s the size of a typical disk sector.

Defaulting the block count goes until dd gets to the device end. No need to figure anything out. Of course, if you must know the count, fdisk(8) or sfdisk(8) tells the sector size and sector count. Use fdisk -l or sfdisk -l.

It isn’t etched in stone. Block size doesn’t have to match the sector size. The device driver handles sector size. Thinking of each block write as one operation cycle suggests an increased block size could use fewer cycles, less time. Either dd or the device driver may break up a block too large into a more appropriate size.

To see whether writing multiple sectors per block runs faster until that break up happens, try timing the doubling of block sizes until greater efficiency shows.

Here’s the result of using twice the default block size:

# time dd if=/dev/zero of=/dev/sdc bs=1024 status=progress 
32714421248 bytes (33 GB, 30 GiB) copied, 8479 s, 3.9 MB/s 
dd: writing to '/dev/sdc': No space left on device 
31950721+0 records in 
31950720+0 records out 
32717537280 bytes (33 GB, 30 GiB) copied, 8619.76 s, 3.8 MB/s 

real    143m39.758s 
user    0m20.696s 
sys     5m20.572s

No significant change. What about four times the default size?

# time dd if=/dev/zero of=/dev/sdc bs=2048 status=progress 
32717305856 bytes (33 GB, 30 GiB) copied, 8594.41 s, 3.8 MB/s 
dd: writing to '/dev/sdc': No space left on device 
15975361+0 records in 
15975360+0 records out 
32717537280 bytes (33 GB, 30 GiB) copied, 8764.31 s, 3.7 MB/s 

real    146m4.316s 
user    0m19.636s 
sys     5m43.889s

No improvement here, either. Eight times?

# time dd if=/dev/zero of=/dev/sdc bs=4096 status=progress 
32711454720 bytes (33 GB, 30 GiB) copied, 3190 s, 10.3 MB/s 
dd: writing to '/dev/sdc': No space left on device 
7987681+0 records in 
7987680+0 records out 
32717537280 bytes (33 GB, 30 GiB) copied, 3326.72 s, 9.8 MB/s 

real    55m26.729s 
user    0m1.993s 
sys     0m33.134s

Now, we’ve got something!

This ran somewhat over 1/3 the time of the default buffer size. Any improvement at 16 times?

# time dd if=/dev/zero of=/dev/sdc bs=8192 status=progress 
32707854336 bytes (33 GB, 30 GiB) copied, 3194.01 s, 10.2 MB/s 
dd: writing to '/dev/sdc': No space left on device 
3993841+0 records in 
3993840+0 records out 
32717537280 bytes (33 GB, 30 GiB) copied, 3351.33 s, 9.8 MB/s 

real    55m51.332s 
user    0m1.525s 
sys     0m31.905s

Clearly no significant change. A 4K block size is a reasonable choice for the top end of the range. Does the top end break lower, between 2K and 4K?

# time dd if=/dev/zero of=/dev/sdc bs=3072 status=progress 
32712655872 bytes (33 GB, 30 GiB) copied, 8433.05 s, 3.9 MB/s 
dd: writing to '/dev/sdc': No space left on device 
10650241+0 records in 
10650240+0 records out 
32717537280 bytes (33 GB, 30 GiB) copied, 8567.51 s, 3.8 MB/s 

real    142m47.518s 
user    0m13.897s 
sys     4m43.755s

So, 3K is similarly slow as earlier sizes less than 4K. One more try at 3.5K, which is 7 default block sizes.

# time dd if=/dev/zero of=/dev/sdc bs=3584 status=progress 
32715820032 bytes (33 GB, 30 GiB) copied, 8440 s, 3.9 MB/s 
dd: writing to '/dev/sdc': No space left on device 
9128778+0 records in 
9128777+0 records out 
32717537280 bytes (33 GB, 30 GiB) copied, 8624.19 s, 3.8 MB/s 

real    143m44.197s 
user    0m14.013s 
sys     4m45.804s

No consequential difference.

Using a 4K buffer size is dd‘s most efficient for this USB stick. Going higher changes little. Using 32K, even 64K block size runs in about the same time with throughput of just under 10 MB/s.

Leave a Comment