Built-in compression does inefficient I/O
Bug #1095249 reported by
Alexey Kopytov
This bug affects 2 people
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Percona XtraBackup moved to https://jira.percona.com/projects/PXB |
Fix Released
|
High
|
Alexey Kopytov | ||
2.0 |
Fix Released
|
High
|
Alexey Kopytov | ||
2.1 |
Fix Released
|
High
|
Alexey Kopytov |
Bug Description
--compress does 4 unbuffered I/O operations per every 64K block of uncompressed data:
- it first writes an 8-byte compressed block header
- then it writes an 8-byte file offset
- then it writes a 4-byte checksum
- finally the compressed contents of the original 64K block is written
Each write is followed by a posix_fadvise(
The fix is to buffer the output compressed stream.
Related branches
lp://staging/~akopytov/percona-xtrabackup/bug1095249-2.0
- Laurynas Biveinis (community): Approve
- George Ormond Lorch III (community): Approve (g2)
-
Diff: 378 lines (+289/-8)6 files modifiedsrc/Makefile (+2/-1)
src/buffer.c (+197/-0)
src/buffer.h (+31/-0)
src/compress.c (+22/-4)
src/datasink.h (+5/-3)
test/t/ib_compress_basic.sh (+32/-0)
lp://staging/~akopytov/percona-xtrabackup/bug1095249-2.1
- Laurynas Biveinis (community): Approve
- George Ormond Lorch III (community): Approve (g2)
-
Diff: 395 lines (+272/-6)8 files modifiedsrc/Makefile (+2/-1)
src/datasink.c (+4/-0)
src/datasink.h (+2/-1)
src/ds_buffer.c (+186/-0)
src/ds_buffer.h (+31/-0)
src/ds_compress.c (+0/-2)
src/xtrabackup.c (+15/-2)
test/t/ib_compress_basic.sh (+32/-0)
To post a comment you must log in.
There is also one here:
stream_ one_file( File file, xb_wstream_file_t *xbfile) BUFFER_ SIZE];
{
uchar buf[XBSTREAM_
size_t bytes;
posix_ fadvise( file, 0, 0, POSIX_FADV_ SEQUENTIAL) ;
while ((bytes = my_read(file, buf, XBSTREAM_ BUFFER_ SIZE, write_data( xbfile, buf, bytes)) { write_data( ) failed.\n",
my_progname) ; fadvise( file, 0, 0, POSIX_FADV_ DONTNEED) ;
MYF(MY_WME))) > 0) {
if (xb_stream_
msg("%s: xb_stream_
return 1;
}
posix_
}
......
.....
here, it is doing fadvise POSIX_FADV_SEQ which doubles the
readahead but in the loop it discards it everytime. It should
either do POSIX_FADV_DONTNEED at the end (outside the loop) or
use ranges with POSIX_FADV_DONTNEED in the loop.