what's needed is a application barrier() call

Posted Sep 9, 2009 21:49 UTC (Wed) by njs (subscriber, #40338)
In reply to: what's needed is a application barrier() call by dlang
Parent article: POSIX v. reality: A position on O_PONIES

Yes, that sort of ordering is what she's talking about. (The Featherstitch papers are worth checking out.)

IIUC, the trickiness in implementing it is that you need to keep track of which writes depend on which other writes, and which intermediate states are allowed, but then you have to keep non-dependent writes as disentangled as possible, to avoid the slowdowns and soft-updates craziness that she describes in the original article -- but you can't be *too* clever about it, or your accounting overhead will become a bottleneck.

Featherstitch's solution involves careful optimizations and giant graphs in kernel memory.

to post comments

what's needed is a application barrier() call

Posted Sep 9, 2009 23:44 UTC (Wed) by dlang (guest, #313) [Link] (4 responses)

my gut feeling is that they are trying too hard.

if you have a way to prevent memory blocks that have been submitted for I/O from being changed before the I/O is completed this is just a matter of prohibiting reordering in the device stack.

this would be overkill for what is being asked for (the user cares about ordering the changes to their file, this would order changes to the entire device), but it would do the job without having to worry about tracing dependancies

if the device stack were to mark all buffers it has pending as COW when it gets a barrier() call, this doesn't seem that hard to do.

now, it does open up the possibility of running out of memory and having problems writing something to swap (until the currently pending writes complete), but if you do the barrier on a per-partition basis this shouldn't be that bad (this assumes that the device stack can easily tell what partition pending writes would go to)

what's needed is a application barrier() call

Posted Sep 10, 2009 6:51 UTC (Thu) by njs (subscriber, #40338) [Link] (3 responses)

I dunno, I'm not an expert, though "prohibiting reordering in the device stack" raises a big red flag for me -- reordering is pretty much the entire way you get speed out of spinning-disk hard drives!

But whether my analysis is accurately pin-pointing the problem or not, with all respect, I think I'll trust Val's word over yours that there *is* a problem :-).

what's needed is a application barrier() call

Posted Sep 10, 2009 7:01 UTC (Thu) by dlang (guest, #313) [Link] (2 responses)

it's only prohibiting reordering around the barrier, not all reordering.

Val is definantly right about the difficulty in doing it the best possible way, as I noted my approach would run some possibility of the COW causing out of memory problems, but I suspect that it's a 90% solution.

what's needed is a application barrier() call

Posted Sep 10, 2009 7:35 UTC (Thu) by njs (subscriber, #40338) [Link] (1 responses)

But now you're synchronizing all writes by all processes everywhere on the disk, not just the two blocks that actually got written to in this one file somewhere. If apps used that sort of barrier() too often then I think you'd end up with disk throughput that looks like you mounted with -o sync -- very little reordering would ever be allowed.

I guess this is complicated by the question of when dirty blocks get flushed, and in how large batches; maybe it's solvable. But I don't think memory is the main concern, at least.

what's needed is a application barrier() call

Posted Sep 10, 2009 7:48 UTC (Thu) by dlang (guest, #313) [Link]

you are absolutely correct, that's why I say it's only a 90% solution, but it may be something that gives you most of the benefit for a fraction of the effort.