Short Read/Write and Buffered File
Recently, I was bitten by a bug in my code, where the buffer is too small and reads return prematurely without error. This has led me to a deep dive into the
madness story of what I later know as “short read” on Linux.
Yes, this article is mostly me rambling. If you are only interested in short read/write, please read this seriously written article.
First, you probably already know that blocking syscalls like
p?(read|write)v? already may short read or short write. This is nothing fancy.
Because of this correspondence, the same syscalls in io_uring may also short r/w. (As it turns out, in earlier kernel, it won’t!)
This does not make much sense to me, since io_uring allows linking requests together, and you can’t link read/write in any request chain whatsoever because the kernel might short r/w! Pain.
We first examine
fread in musl libc.
FILE*is locked to a thread (
FLOCK) using the Linux
futexsyscall. That’s why the functions are thread-safe!
_IO_FILEhas a virtual table in it ! See
file->(read|write|seek). Because of this,
- Every file has a fixed-size buffer. See
#define BUFSIZ 1024. Because of this,
- You can provide your own buffer when opening a file using
- The actual reading is done with
readv, where two buffer is given, one from the user, and one is the internal fixed-size buffer. See
- speculative execution… SPECULATIVE EXECUTION?!
As it turns out, buffering and using the same file handle for read and write need special consideration.
__toread handles clearing out the write buffer before trying to read in anything.
Next, we turn our eyes to
fwrite. Rather than the oddly specific logic where
'\n' determines when to actually send the syscall, we also find
__towrite, within documented the author’s experience with summoning nasal demons.
Scary, the whole experience is.