pt-table-checksum fails if a database is dropped while the tool is running

Bug #1658811 reported by Agustín
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Percona Toolkit moved to https://jira.percona.com/projects/PT
Fix Released
Medium
Carlos Salguero

Bug Description

If a database is dropped while pt-table-checksum is running, the tool exists with a fatal error:

$ pt-table-checksum --no-check-binlog-format h=127.0.0.1,u=root,p=msandbox,P=9870
            TS ERRORS DIFFS ROWS CHUNKS SKIPPED TIME TABLE
01-23T16:40:10 0 0 0 1 0 0.012 mysql.columns_priv
<...output trimmed>

DBD::mysql::db selectall_arrayref failed: Unknown database 'pttctest98' [for Statement "SHOW /*!50002 FULL*/ TABLES FROM `pttctest98`"] at /usr/bin/pt-table-checksum line 7535.

$ echo $?
255

How to reproduce:

In one session in the master server run the following:

for i in `seq 1000`; do
mysql -e "CREATE SCHEMA pttctest$i";
mysql -e "DROP SCHEMA pttctest$(($i-1))";
done;

In another one (after the previous command is run), run the pt-table-checksum. The tool will exit with an error like seen above.

Suggested fix:

Check if the database exists before running the SHOW FULL TABLES FROM ... command.

Tags: i164863 pt105
Revision history for this message
Agustín (agustin-gallego) wrote :

Sorry, I forgot to mention versions.

MySQL 5.7.17, and pt-table-checksum 2.2.19

Revision history for this message
markus_albe (markus-albe) wrote :

The technique of observing whether a db exists before using SHOW FULL TABLES FROM... is bound to same type of race conditions. Am not sure it's possible to provide a solution until DDL and schemas become transactional.

Revision history for this message
Agustín (agustin-gallego) wrote :

Agreed, but two comments on this:

1- Even if it's bound to some race conditions, we should at least check.

2- The tool should not error out. Instead (as being done with tables that don't exist), it should show a warning, set the error number bitmask to 'non-fatal error', and continue with other schemas.

Changed in percona-toolkit:
assignee: nobody → Carlos Salguero (carlos-salguero)
Changed in percona-toolkit:
status: New → Confirmed
tags: added: pt105
Changed in percona-toolkit:
importance: Undecided → Medium
status: Confirmed → In Progress
milestone: none → 3.0.2
Revision history for this message
Carlos Salguero (carlos-salguero) wrote :

Since the change goes to SchemaIterator, all these programs also get the fix:
 modified: bin/pt-duplicate-key-checker
 modified: bin/pt-index-usage
 modified: bin/pt-table-checksum
 modified: bin/pt-table-sync

Revision history for this message
Carlos Salguero (carlos-salguero) wrote :
Changed in percona-toolkit:
status: In Progress → Fix Committed
Changed in percona-toolkit:
status: Fix Committed → Fix Released
Revision history for this message
Shahriyar Rzayev (rzayev-sehriyar) wrote :

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PT-727

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.