python-pymysql fails to build on xenial and yakkety

Bug #1592664 reported by Matthias Klose
14
This bug affects 1 person
Affects Status Importance Assigned to Milestone
python-pymysql (Ubuntu)
Fix Released
High
Lars Tangvald
Xenial
Fix Released
High
Lars Tangvald

Bug Description

Python-pymysql sets up MySQL in a temporary location for running its tests. It runs mysql_install_db and mysqld using options that have been removed in 5.7, causing d/run_tests.sh to fail.
Additionally, there's a test that fails in 5.7; It expects a deprecation warning that 5.7 gives, but only for python >= 3.2. The test failure has been fixed upstream: https://github.com/PyMySQL/PyMySQL/issues/478

[Impact]
Test run fails, causing build to fail

[Test Case]
Build python-pymysql from source

[Regression Potential]
The altered database init will not work with MySQL earlier than 5.7 (even the xenial patch that still uses mysql_install_db)

[Original description]
seen with a test rebuild on xenial:
http://people.ubuntuwire.org/~wgrant/rebuild-ftbfs-test/test-rebuild-20160614-xenial.html

   debian/rules override_dh_auto_test
make[1]: Entering directory '/«PKGBUILDDIR»'
pyversions: missing X(S)-Python-Version in control file, fall back to debian/pyversions
pyversions: missing debian/pyversions file, fall back to supported versions
py3versions: no X-Python3-Version in control file, using supported versions
chmod +x debian/run_tests.sh
./debian/run_tests.sh
+ mktemp -d
+ MYTEMP_DIR=/tmp/tmp.i5ZAMugsdp
+ whoami
+ ME=buildd
+ mysql_install_db --no-defaults --datadir=/tmp/tmp.i5ZAMugsdp --force --skip-name-resolve --user=buildd
mysql_install_db: [ERROR] unknown option '--force'
2016-06-03 11:03:28 [ERROR] Unrecognized options
debian/rules:25: recipe for target 'override_dh_auto_test' failed
make[1]: *** [override_dh_auto_test] Error 1
make[1]: Leaving directory '/«PKGBUILDDIR»'
debian/rules:10: recipe for target 'build' failed
make: *** [build] Error 2
dpkg-buildpackage: error: debian/rules build gave error exit status 2

Matthias Klose (doko)
Changed in python-pymysql (Ubuntu):
assignee: nobody → Ubuntu Server Team (ubuntu-server)
Revision history for this message
Robie Basak (racb) wrote :

Lars, could you advise please? I suspect the mysql_install_db invocation needs replacing with the new method?

Revision history for this message
Lars Tangvald (lars-tangvald) wrote :

Yes.
Ideally change to use mysqld --initialize:
https://dev.mysql.com/doc/refman/5.7/en/data-directory-initialization-mysqld.html

It should also be fixable by simply removing --force and --skip-name-resolve and adding --basedir=/usr to the mysql_install_db command

Robie Basak (racb)
tags: added: mysql-5.7-transition
Revision history for this message
Lars Tangvald (lars-tangvald) wrote :

One addition: If the system relies on the database being created with passwordless root access, --insecure also needs to be added (otherwise a one-time random password will be generated and written to the error log)

Robie Basak (racb)
Changed in python-pymysql (Ubuntu):
assignee: Ubuntu Server Team (ubuntu-server) → Lars Tangvald (lars-tangvald)
Changed in python-pymysql (Ubuntu Xenial):
assignee: nobody → Lars Tangvald (lars-tangvald)
importance: Undecided → High
status: New → Triaged
Changed in python-pymysql (Ubuntu):
status: Confirmed → Triaged
Mathew Hodson (mhodson)
Changed in python-pymysql (Ubuntu):
milestone: xenial-updates → none
Changed in python-pymysql (Ubuntu Xenial):
milestone: none → xenial-updates
Revision history for this message
Lars Tangvald (lars-tangvald) wrote :

Adding --insecure and --basedir=/usr and removing --force and --skip-name-resolve, plus removing --skip-grant from the following mysqld command, seems to fix it, but I'm getting a test failure:
test_issue_363, which I'm looking into

Revision history for this message
Lars Tangvald (lars-tangvald) wrote :

The patch fixes the use of mysql_install_db and mysqld in d/run_tests.sh and adds d/patches/mysql-5.7.patch to fix the failing test for issue_363

Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

The attachment "Patch for test script and test failures" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

tags: added: patch
Revision history for this message
Robie Basak (racb) wrote :

Any chance of a complete debdiff for this please Lars, with a debian/changelog entry? Then I can easily attribute you fully for the upload, which will help with getting you direct upload access in the future.

Revision history for this message
Lars Tangvald (lars-tangvald) wrote :

Yeah, will add that. Need to remove the comment about --force, too :)
Was testing out using mysqld --initialize instead of mysql_install_db, but it creates some extra complexity with the permissions for datadir, so at least for Xenial I think we'll stick to just changing the options used.

Revision history for this message
Lars Tangvald (lars-tangvald) wrote :

Suggested debdiff for yakkety

Since --skip-grant-tables is likely to be removed in the future, we instead specify the root user when connecting to the temporary database.

Revision history for this message
Lars Tangvald (lars-tangvald) wrote :

Suggested debdiff for xenial

Revision history for this message
Lars Tangvald (lars-tangvald) wrote :

The Xenial patch is a bit smaller since it changes --skip-grant to --skip-grant-tables instead of removing it (which requires specifying the root user later in the script).

Revision history for this message
Lars Tangvald (lars-tangvald) wrote :

Yakkety diff needs rechecking since the test failure has been fixed upstream

description: updated
Revision history for this message
Matthias Klose (doko) wrote :

any update on this? the fix didn't land yet.

Revision history for this message
Robie Basak (racb) wrote :

Subscribing ~ubuntu-sponsors.

Revision history for this message
Marc Deslauriers (mdeslaur) wrote :

ACK on the debdiff in comment #9. I removed the patch since it's no longer needed with 0.7.6. Uploaded to yakkety. Thanks!

Revision history for this message
Marc Deslauriers (mdeslaur) wrote :

The debdiff in comment #10 says it's for xenial, but it's for 0.7.5 which isn't in xenial at all.

Changed in python-pymysql (Ubuntu Xenial):
status: Triaged → Incomplete
Revision history for this message
Lars Tangvald (lars-tangvald) wrote :

Must have applied it to the wrong version. Will redo and upload another diff.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python-pymysql - 0.7.6-1ubuntu1

---------------
python-pymysql (0.7.6-1ubuntu1) yakkety; urgency=medium

  * Fix ftfs with MySQL 5.7
    - d/run_tests.sh was using old MySQL initialization. Updated for 5.7:
      - Switched mysql_install_db with mysqld --initialize
      - Removed deprecated option --skip-grant from mysqld
      - Added -uroot to client commands
    (LP: #1592664)

 -- Lars Tangvald <email address hidden> Tue, 05 Jul 2016 10:37:47 +0200

Changed in python-pymysql (Ubuntu):
status: Triaged → Fix Released
Revision history for this message
Lars Tangvald (lars-tangvald) wrote :

New patch for Xenial, for the correct version (0.7.2)

Changed in python-pymysql (Ubuntu Xenial):
status: Incomplete → In Progress
Revision history for this message
Marc Deslauriers (mdeslaur) wrote :

Debdiff in comment #19 looks good. Uploaded for processing by the SRU team.

Thanks!

Revision history for this message
Martin Pitt (pitti) wrote : Please test proposed package

Hello Matthias, or anyone else affected,

Accepted python-pymysql into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/python-pymysql/0.7.2-1ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in python-pymysql (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed
Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote : [python-pymysql/xenial] verification still needed

The fix for this bug has been awaiting testing feedback in the -proposed repository for xenial for more than 90 days. Please test this fix and update the bug appropriately with the results. In the event that the fix for this bug is still not verified 15 days from now, the package will be removed from the -proposed repository.

tags: added: removal-candidate
Revision history for this message
Lars Tangvald (lars-tangvald) wrote :

New package 0.7.2-1ubuntu1 verified to build successfully on Xenial, and all tests pass (this is what failed previously, see [Test Case].
Also verified that the previous version still fails as described in the bug description.

(Sorry, I missed the comment asking for verification)

tags: removed: verification-needed
Revision history for this message
Lars Tangvald (lars-tangvald) wrote :

Setting it to verified since it was a build failure issue, and it builds in proposed now

tags: added: verification-done
Matthias Klose (doko)
tags: removed: removal-candidate
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python-pymysql - 0.7.2-1ubuntu1

---------------
python-pymysql (0.7.2-1ubuntu1) xenial; urgency=medium

  * Fix ftbfs for MySQL 5.7
   - Update database initialization in d/run_tests.sh for 5.7:
     - Switch mysql_install_db with mysqld --initialize
     - Replace --skip-grant with --skip-grant-tables (deprecated)
   - Added patch to fix test failure with 5.7
   (LP: #1592664)

 -- Lars Tangvald <email address hidden> Fri, 16 Sep 2016 07:14:50 +0200

Changed in python-pymysql (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of the Stable Release Update for python-pymysql has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.