Vectorized Read?

Posted Mar 22, 2007 18:35 UTC (Thu) by vmole (guest, #111)
In reply to: Vectorized Read? by ldo
Parent article: The 2007 Linux Storage and File Systems Workshop

Because fsck needs block addressed access, and aio_read() is based on (fd, offset, count). Otherwise, you don't even need AIO; you'd just use readv(2). OTOH, it seems like readv() would be sufficient given the RAW_IO to the partition...so maybe someone else should answer this :-)

to post comments

Vectorized Read?

Posted Mar 23, 2007 4:14 UTC (Fri) by ldo (guest, #40946) [Link] (6 responses)

Because fsck needs block addressed access, and aio_read() is based on (fd, offset, count).

Hint:

fd = open("/dev/sda", ...)

Vectorized Read?

Posted Mar 23, 2007 16:55 UTC (Fri) by vmole (guest, #111) [Link] (5 responses)

Yes, that's what I meant by "RAW_IO on the partition" :-) But, again, if that would work, then readv(2) would seem to provide the required vectorized API, so why are the smart people saying they need a new API?

Vectorized Read?

Posted Mar 23, 2007 20:05 UTC (Fri) by zlynx (guest, #2285) [Link] (4 responses)

readv and writev both use a vector of memory buffers, but they are not for writing a vector of disk blocks.

We probably need readiov/writeiov and readviov/writeviov or something like that.

I also had a crazy idea just now. What if they used device mapper to create a dm device with a linear view of every block fsck needed to read? Let readahead run on that.

How about readahead(2) or fadvise or posix_fadvise?

Vectorized Read?

Posted Mar 23, 2007 20:22 UTC (Fri) by vmole (guest, #111) [Link] (3 responses)

Please explain further (I'm not doubting you, just trying to understand). If you open a block device (e.g. /dev/hda1), and you can read at specific offsets (which is what readv(2) gives you), isn't that being able to read specific blocks? I mean, I understand that it's not the underlying physical disk blocks, but isn't the mapping between physical disk blocks and the logical blocks that the filesystem sees all handled in hardware?

A very quick glance at the e2fsprogs source indeed seems to use open("/dev/hda1") and read/write.

Where's Mr. Tso when we need him?

Vectorized Read?

Posted Mar 23, 2007 21:28 UTC (Fri) by zlynx (guest, #2285) [Link] (1 responses)

I am going on the man page for readv.

ssize_t readv(int fd, const struct iovec *vector, int count);
struct iovec {
void *iov_base; /* Starting address */
size_t iov_len; /* Number of bytes */
};

iov_base and iov_len apply to RAM addresses, not disk block or character addresses. The descriptor "fd" is read linearly for "count" iovec structures.

So, readv does *not* give you the ability to read at specific offsets. As I understand it from reading the documentation.

Vectorized Read?

Posted Mar 23, 2007 21:36 UTC (Fri) by vmole (guest, #111) [Link]

<p>Oh, duh, right. I knew that, at one time in the distant past. Sorry for the noise.

Vectorized Read?

Posted Mar 23, 2007 21:34 UTC (Fri) by giraffedata (guest, #1954) [Link]

... and you can read at specific offsets (which is what readv(2) gives you), isn't that being able to read specific blocks?

You need to be able to read not just specific offsets, but specific discontiguous offsets. readv() does not give you that. You have to read a contiguous area of the file (block device). You cannot make it read from 4K to 8K and 12K to 16K.

What readv() adds to read() is that you can read that contiguous file region into discontiguous memory, whereas with read() it has to go into a single contiguous range of memory addresses.