Failed to set O_DIRECT on xb_doublewrite when running MTR test cases

Bug #1669414 reported by Sveta Smirnova
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Percona Server moved to https://jira.percona.com/projects/PS
Status tracked in 5.7
5.5
Invalid
Undecided
Unassigned
5.6
Invalid
Undecided
Unassigned
5.7
Fix Released
High
Laurynas Biveinis

Bug Description

All MTR test fail with error:

==============================================================================

TEST RESULT TIME (ms) or COMMENT
--------------------------------------------------------------------------

worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 13000..13009
main.alias [ fail ] Found warnings/errors in server log file!
        Test ended at 2017-03-02 15:49:41
line
2017-03-02T12:49:33.406293Z 0 [Warning] InnoDB: Failed to set O_DIRECT on file/home/sveta/build/ps-5.7/mysql-test/var/mysqld.1/data/xb_doublewrite;create: Invalid argument, continuing anyway. O_DIRECT is known to result in 'Invalid argument' on Linux on tmpfs, see MySQL Bug#26662.

FS: ecryptfs

Removing var directory before running test does not help.

How to repeat:

sveta@Thinkie:~/build/ps-5.7/mysql-test$ mount
...
/home/.ecryptfs/sveta/.Private on /home/sveta type ecryptfs (rw,nosuid,nodev,relatime,ecryptfs_fnek_sig=1fe38ac67150d26d,ecryptfs_sig=3da24c1114964df7,ecryptfs_cipher=aes,ecryptfs_key_bytes=16,ecryptfs_unlink_sigs)
gvfsd-fuse on /run/user/1000/gvfs type fuse.gvfsd-fuse (rw,nosuid,nodev,relatime,user_id=1000,group_id=1000)
sveta@Thinkie:~/build/ps-5.7/mysql-test$ rm -rf var
sveta@Thinkie:~/build/ps-5.7/mysql-test$ ls -l var
ls: cannot access 'var': No such file or directory
sveta@Thinkie:~/build/ps-5.7/mysql-test$ ./mtr alias
Logging: ./mtr alias
MySQL Version 5.7.17
Checking supported features...
 - SSL connections supported
 - binaries are debug compiled
Collecting tests...
Removing old var directory...
Creating var directory '/home/sveta/build/ps-5.7/mysql-test/var'...
Installing system database...
Using parallel: 1

==============================================================================

TEST RESULT TIME (ms) or COMMENT
--------------------------------------------------------------------------

worker[1] Using MTR_BUILD_THREAD 300, with reserved ports 13000..13009
main.alias [ fail ] Found warnings/errors in server log file!
        Test ended at 2017-03-02 15:55:01
line
2017-03-02T12:54:53.171057Z 0 [Warning] InnoDB: Failed to set O_DIRECT on file/home/sveta/build/ps-5.7/mysql-test/var/mysqld.1/data/xb_doublewrite;create: Invalid argument, continuing anyway. O_DIRECT is known to result in 'Invalid argument' on Linux on tmpfs, see MySQL Bug#26662.
^ Found warnings in /home/sveta/build/ps-5.7/mysql-test/var/log/mysqld.1.err
ok

 - the logfile can be found in '/home/sveta/build/ps-5.7/mysql-test/var/log/main.alias/alias.log'
--------------------------------------------------------------------------
The servers were restarted 0 times
Spent 0.000 of 32 seconds executing testcases
mysql-test-run: WARNING: Got errors/warnings while running tests, please examine '/home/sveta/build/ps-5.7/mysql-test/var/log/warnings' for details.

Completed: Failed 1/1 tests, 0.00% were successful.

Failing test(s): main.alias

The log files in var/log may give you some hint of what went wrong.

If you want to report this error, please read first the documentation
at http://dev.mysql.com/doc/mysql/en/mysql-test-suite.html

mysql-test-run: *** ERROR: there were failing test cases

Revision history for this message
Sveta Smirnova (svetasmirnova) wrote :

Not repeatable with version 5.6

tags: added: parallel-doublewrite regression
tags: added: xtradb
Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

Besides the warning message, this might result in reduced durability if innodb_flush_method=ALL_O_DIRECT, similar to bug 801989.

Parallel doublewrite O_DIRECT setting is unconditional and not dependent on innodb_flush_method. The fix would be:
- only for the doublewrite buffer, downgrade the message to Note severity;
- add check for os_file_set_nocache failure in parallel doublewrite create, save it in a flag, check it before flushing in buf_dblwr_flush_buffered_writes
- fix the incorrect comment added by bug 801989 fix in os_file_create_func re. fsync on parallel doublewrite (as it uses the simple os0file API)

summary: - Failed to set O_DIRECT on file ...mysql-
- test/var/mysqld.1/data/xb_doublewrite when run MTR test cases
+ Failed to set O_DIRECT on xb_doublewrite when runring MTR test cases
summary: - Failed to set O_DIRECT on xb_doublewrite when runring MTR test cases
+ Failed to set O_DIRECT on xb_doublewrite when running MTR test cases
Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

Sveta, would you be able to test whether https://github.com/laurynas-biveinis/percona-server/tree/bug1669414 fixes the issue in your environment?

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

Sorry, the branch is not ready yet - will add a comment when it is

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

The branch is ready now

Revision history for this message
Sveta Smirnova (svetasmirnova) wrote :

I still see the warning

Revision history for this message
Sveta Smirnova (svetasmirnova) wrote :

Re-tested and confirm: bug is fixed.

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :
Revision history for this message
Shahriyar Rzayev (rzayev-sehriyar) wrote :

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-1068

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.