Include innodb_log_group_home_dir and innodb_undo_directory as special variables supported by sst_special_dirs
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Percona XtraBackup moved to https://jira.percona.com/projects/PXB |
Fix Released
|
Medium
|
Sergei Glushchenko | ||
2.2 |
Fix Released
|
Medium
|
Sergei Glushchenko | ||
2.3 |
Fix Released
|
Medium
|
Sergei Glushchenko |
Bug Description
Currently, xtrabackup-v2 fails when innodb_
Tested on PXC 5.6.20/CentOS 6:
Joiner my.cnf:
[mysqld]
user=mysql
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
user=mysql
# Path to Galera library
wsrep_provider=
# Cluster connection URL contains the IPs of node#1, node#2 and node#3
wsrep_cluster_
# In order for Galera to work correctly binlog format should be ROW
binlog_format=ROW
# MyISAM storage engine has only experimental support
default_
# This changes how InnoDB autoincrement locks are managed and is a requirement for Galera
innodb_
# Node #2 address
wsrep_node_
# SST method
wsrep_sst_
# Cluster name
wsrep_cluster_
# Authentication for SST method
wsrep_sst_
datadir=
innodb_
innodb_
#innodb_
innodb_
[mysqld_safe]
log-error=
pid-file=
[sst]
sst_special_dirs=1
Donor my.cnf:
[mysqld]
user=mysql
# Disabling symbolic-links is recommended to prevent assorted security risks
symbolic-links=0
user=mysql
# Path to Galera library
wsrep_provider=
# Cluster connection URL contains the IPs of node#1, node#2 and node#3
wsrep_cluster_
# In order for Galera to work correctly binlog format should be ROW
binlog_format=ROW
# MyISAM storage engine has only experimental support
default_
# This changes how InnoDB autoincrement locks are managed and is a requirement for Galera
innodb_
# Node #3 address
wsrep_node_
# SST method
wsrep_sst_
# Cluster name
wsrep_cluster_
# Authentication for SST method
wsrep_sst_
datadir=
innodb_
innodb_
#innodb_
innodb_
[sst]
sst_special_dirs=1
[mysqld_safe]
log-error=
pid-file=
Test:
[root@pxc2 mysql]# service mysql restart
Shutting down MySQL (Percona XtraDB Cluster) ERROR! MySQL (Percona XtraDB Cluster) PID file could not be found!
ERROR! MySQL (Percona XtraDB Cluster) is not running, but lock file (/var/lock/
Starting MySQL (Percona XtraDB Cluster)......State transfer in progress, setting sleep higher
......... ERROR! The server quit without updating PID file (/var/lib/
ERROR! MySQL (Percona XtraDB Cluster) server startup failed!
ERROR! Failed to restart server.
Logs:
cat /var/lib/
InnoDB Backup Utility v1.5.1-xtrabackup; Copyright 2003, 2009 Innobase Oy
and Percona LLC and/or its affiliates 2009-2013. All Rights Reserved.
This software is published under
the GNU GENERAL PUBLIC LICENSE Version 2, June 1991.
Get the latest version of Percona XtraBackup, documentation, and help resources:
http://
IMPORTANT: Please check that the move-back run completes successfully.
At the end of a successful move-back run innobackupex
prints "completed OK!".
innobackupex: Error: Original InnoDB log directory '/var/lib/
Error log:
WSREP_SST: [INFO] Cleaning the existing datadir and innodb-data/log directories (20141016 12:31:53.054)
removed directory: `/var/lib/
removed `/var/lib/
WSREP_SST: [INFO] Evaluating socat -u TCP-LISTEN:
2014-10-16 12:32:36 16914 [Note] WSREP: 0.0 (pxc3): State transfer to 1.0 (pxc2) complete.
2014-10-16 12:32:36 16914 [Note] WSREP: Member 0.0 (pxc3) synced with group.
WSREP_SST: [INFO] Preparing the backup at /var/lib/
WSREP_SST: [INFO] Evaluating innobackupex --no-version-check --apply-log $rebuildcmd ${DATA} &>${DATA}
rm: cannot remove `/var/lib/
rm: cannot remove `/var/lib/
WSREP_SST: [INFO] Moving the backup to /var/lib/mysql/ (20141016 12:33:03.580)
WSREP_SST: [INFO] Evaluating innobackupex --defaults-
WSREP_SST: [ERROR] Cleanup after exit with status:1 (20141016 12:33:05.472)
WSREP_SST: [INFO] Removing the sst_in_progress file (20141016 12:33:05.475)
2014-10-16 12:33:05 16914 [ERROR] WSREP: Process completed with error: wsrep_sst_
2014-10-16 12:33:05 16914 [ERROR] WSREP: Failed to read uuid:seqno from joiner script.
2014-10-16 12:33:05 16914 [ERROR] WSREP: SST failed: 1 (Operation not permitted)
2014-10-16 12:33:05 16914 [ERROR] Aborting
I confirm this problem for PXC 5.6.20. When the innodb_ log_group_ home_dir points to a directory inside innodb_ data_home_ dir - xtrabackup-v2 SST method fails.
percona1 mysql> select @@version, @@version_ comment; ------- ---+--- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- -----+ ------- ---+--- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- -----+ ------- ---+--- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- ------- -----+
+------
| @@version | @@version_comment |
+------
| 5.6.20-68.0-56 | Percona XtraDB Cluster (GPL), Release rel68.0, Revision 888, WSREP version 25.7, wsrep_25.7.r4126 |
+------
1 row in set (0.00 sec)
percona1 mysql> show variables like 'innodb_%dir%'; ------- ------- ------- -----+- ------- ------- ----+ ------- ------- ------- -----+- ------- ------- ----+ data_home_ dir | /data/innodb | log_group_ home_dir | /data/innodb/logs | max_dirty_ pages_pct | 75 | max_dirty_ pages_pct_ lwm | 0 | undo_directory | /data/innodb | ------- ------- ------- -----+- ------- ------- ----+
+------
| Variable_name | Value |
+------
| innodb_
| innodb_log_arch_dir | /data/innodb/logs |
| innodb_
| innodb_
| innodb_
| innodb_
+------
6 rows in set (0.00 sec)
in my.cnf:
[sst]
streamfmt=xbstream
sst_special_dirs=1
Forced SST to percona1 node:
WSREP_SST: [INFO] Proceeding with SST (20141018 10:58:14.247) innodb/ ibdata1' innodb/ logs/ib_ logfile1' innodb/ logs/ib_ logfile0' innodb/ undo002' innodb/ undo001' 4444,reuseaddr stdio | xbstream -x; RC=( ${PIPESTATUS[@]} ) (20141018 10:58:14.295) /innobackup. prepare. log (20141018 10:58:29.666) file=/etc/ my.cnf --no-version-check --move-back --force- non-empty- directories ${DATA} &>${DATA} /innobackup. move.log (20141018 10:58:34.927)
WSREP_SST: [INFO] Cleaning the existing datadir and innodb-data/log directories (20141018 10:58:14.250)
removed `/data/
removed `/data/
removed `/data/
removed directory: `/data/innodb/logs'
removed `/data/
removed `/data/
(...)
WSREP_SST: [INFO] Evaluating socat -u TCP-LISTEN:
2014-10-18 10:58:29 18105 [Note] WSREP: 1.0 (percona2): State transfer to 2.0 (percona1) complete.
2014-10-18 10:58:29 18105 [Note] WSREP: Member 1.0 (percona2) synced with group.
WSREP_SST: [INFO] Compressed qpress files found (20141018 10:58:29.062)
WSREP_SST: [INFO] Decompression with 2 threads (20141018 10:58:29.067)
WSREP_SST: [INFO] Evaluating find /data/myisam//.sst -type f -name '*.qp' -printf '%p\n%h\n' | xargs -n 2 qpress -T2d (20141018 10:58:29.071)
WSREP_SST: [INFO] Removing qpress files after decompression (20141018 10:58:29.646)
WSREP_SST: [INFO] Preparing the backup at /data/myisam//.sst (20141018 10:58:29.663)
WSREP_SST: [INFO] Evaluating innobackupex --no-version-check --apply-log $rebuildcmd ${DATA} &>${DATA}
WSREP_SST: [INFO] Moving the backup to /data/myisam/ (20141018 10:58:34.923)
WSREP_SST: [INFO] Evaluating innobackupex --defaults-
WSREP_SST: [ERROR] Cleanup after exit with status:1 (20141018 10:58:35.524)
WSREP...