Linux Affected by Decade Old Bug in Software Raid Around O_direct Usage

3 weeks ago 2

LINUX STORAGE

A Phoronix reader pointed out a bug report from 2015 now getting renewed interest... Linux software RAID via MD RAID, DRBD, LVM RAID, and similar software-based solutions can be broken from user-space around O_DIRECT usage. The issue is that these RAID arrays can be put silently into an inconsistent state across disks.

The 2015 bug report provided a sample script for showing how MD RAID and similar can be put into an inconsistent state across disks from a user-space program poorly abusing O_DIRECT. As Wolfgang Bumiller later summed up in the thread with the heart of the issue:

"This is not at all about the contents of the data. It is expected that garbage is written to the disks, but each disk making up the raid will contain different garbage, which means the disks are out of sync, iow. the raid is "broken". This in turn means the user space can "break" the raid.

The problem is that with O_DIRECT the the user space pointer is passed to the block drivers for the underlying layers making up the raid, and they all read from it independently. Any user who can run a program where they can use O_DIRECT on a file on a raid can break the raid.

It is expected that garbage is written to the disk, but the whole point of a raid is that each disk should contain the *same* garbage. Keep the garbage consistent... or something."

That was ten years ago and this bug report started by Stanislav German-Evtushenko remains open.

RAID array

In the years since, discussion around the bug has come up occasionally, most recently in the context if this bug could surface in the context of VM live migration.

10 year RAID bug