Logs.var.log.mysql.error.log.txt contains usernames and passwords

Bug #1574458 reported by BCB
262
This bug affects 1 person
Affects Status Importance Assigned to Milestone
mariadb-10.0 (Ubuntu)
Invalid
Undecided
Unassigned
Xenial
Confirmed
Undecided
Unassigned
mariadb-10.1 (Ubuntu)
Confirmed
Undecided
Unassigned
mariadb-5.5 (Ubuntu)
Invalid
Undecided
Unassigned
Trusty
Confirmed
Undecided
Unassigned
mysql-5.5 (Ubuntu)
Invalid
Undecided
Unassigned
Trusty
Confirmed
Undecided
Unassigned
mysql-5.6 (Ubuntu)
Invalid
Undecided
Unassigned
Trusty
Invalid
Undecided
Unassigned
mysql-5.7 (Ubuntu)
Fix Released
High
Lars Tangvald
Xenial
Fix Released
High
Robie Basak

Bug Description

MySQL has some logic for ensuring passwords aren't written to the logs, detailed at https://dev.mysql.com/doc/refman/5.7/en/password-logging.html (passwords are rewritten before they are logged). However, a failed grant statement is written unaltered to the error log, bypassing the password rewriting logic.

[Impact]
Ubuntu's bug reporting system will suggest uploading the error log to a bug report. This can lead to user credentials written in plain text in public bug reports.

[Test case]
(note/todo: I had a simpler test for this, but can't find the exact syntax for it)
* Add the following to the server config:
plugin-load=validate_password.so
validate-password=FORCE_PLUS_PERMANENT
and restart the server
* Log in and run GRANT ALL ON *.* TO 'user'@'localhost' IDENTIFIED BY '123';
* Observe statement failing because it doesn't follow password validation rules
* Run "ubuntu-bug mysql-server"
* Choose "View Report"
* Search for "123"

Expected behavior:
Password is scrambled or otherwise not written to the apport report

Actual behavior:
The entire failed grant statement is written to the apport report

[Regression Potential]
The fix replaces all lines in the log that contain any of the terms mentioned on the password-logging site, so it will rewrite more lines than strictly necessary, potentially making debugging harder.

[Original description]
Your automated bug reports are posting Logs.var.log.mysql.error.log.txt in clear text. These logs may contain PII as well as user credentials.

information type: Private Security → Public Security
Revision history for this message
Seth Arnold (seth-arnold) wrote :

Thanks for the report; I found two instances in our bugs with the following messages:

[Warning] Did not write failed 'GRANT ALL PRIVILEGES ON `phpmyadmin`.* TO `phpmyadmin`@'localhost' IDENTIFIED BY 'password'' into binary log while granting/revoking privileges in databases.
[Warning] Did not write failed 'grant all privileges on wordpress.* to wordpressuser@localhost identified by "password"' into binary log while granting/revoking privileges in databases.

(I've replaced the passwords with "password".)

Are there other instances of passwords or usernames that go into this log?

Thanks

Revision history for this message
Lars Tangvald (lars-tangvald) wrote :

With MySQL 5.7, if you run mysqld --initialize yourself, for a new database, it will generate a random root password and put it in the log (users will be required to change this password on first login), but the normal package installation will either set unix socket authentication or a password chosen during configuration.

Revision history for this message
Lars Tangvald (lars-tangvald) wrote :

Verified on MySQL 5.7; Password logging should follow the rules specified on https://dev.mysql.com/doc/refman/5.7/en/password-logging.html, but it seems it's not caught correctly when the grant statement fails. I'll send this upstream.

The error log will contain usernames for failed logins, but I can't think of much else in the way of PII it would contain.

Revision history for this message
BCB (fdajkddcek) wrote : Re: [Bug 1574458] Re: Logs.var.log.mysql.error.log.txt contains usernames and passwords

I just checked the aprox 39 mysql-5.7 bug reports with xxx.error.log.txt
attached and did not see anymore.

I locked my report as my user name and a few random inserts were appeared
in the log.
"ssunderlin"
"Stephen Sunderlin"

If you can remove/replace/ or just delete that error log you can make my
report public again.

https://i255940206.restricted.launchpadlibrarian.net/255940206/Logs.var.log.mysql.error.log.txt?token=vjZn2FHWMBn7TDH8QfsM7fBf9Xccw9sv

Thank you.

On Tue, Apr 26, 2016 at 4:28 AM, Lars Tangvald <email address hidden>
wrote:

> Verified on MySQL 5.7; Password logging should follow the rules
> specified on https://dev.mysql.com/doc/refman/5.7/en/password-
> logging.html, but it seems it's not caught correctly when the grant
> statement fails. I'll send this upstream.
>
> The error log will contain usernames for failed logins, but I can't
> think of much else in the way of PII it would contain.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1574458
>
> Title:
> Logs.var.log.mysql.error.log.txt contains usernames and passwords
>
> Status in mariadb-10.0 package in Ubuntu:
> New
> Status in mariadb-5.5 package in Ubuntu:
> New
> Status in mysql-5.5 package in Ubuntu:
> New
> Status in mysql-5.6 package in Ubuntu:
> New
> Status in mysql-5.7 package in Ubuntu:
> New
>
> Bug description:
> Your automated bug reports are posting
> Logs.var.log.mysql.error.log.txt in clear text. These logs may
> contain PII as well as user credentials.
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/mariadb-10.0/+bug/1574458/+subscriptions
>

Revision history for this message
BCB (fdajkddcek) wrote :

I just checked the aprox 39 mysql-5.7 bug reports with xxx.error.log.txt
attached and did not see anymore.

I locked my report as my user name and a few random inserts were appeared
in the log.
"ssunderlin"
"Stephen Sunderlin"

If you can remove/replace/ or just delete that error log you can make my
report public again.

https://i255940206.restricted.launchpadlibrarian.net/255940206/Logs.var.log.mysql.error.log.txt?token=vjZn2FHWMBn7TDH8QfsM7fBf9Xccw9sv

Thank you.

On Mon, Apr 25, 2016 at 11:08 PM, Seth Arnold <email address hidden>
wrote:

> Thanks for the report; I found two instances in our bugs with the
> following messages:
>
> [Warning] Did not write failed 'GRANT ALL PRIVILEGES ON `phpmyadmin`.* TO
> `phpmyadmin`@'localhost' IDENTIFIED BY 'password'' into binary log while
> granting/revoking privileges in databases.
> [Warning] Did not write failed 'grant all privileges on wordpress.* to
> wordpressuser@localhost identified by "password"' into binary log while
> granting/revoking privileges in databases.
>
> (I've replaced the passwords with "password".)
>
> Are there other instances of passwords or usernames that go into this
> log?
>
> Thanks
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1574458
>
> Title:
> Logs.var.log.mysql.error.log.txt contains usernames and passwords
>
> Status in mariadb-10.0 package in Ubuntu:
> New
> Status in mariadb-5.5 package in Ubuntu:
> New
> Status in mysql-5.5 package in Ubuntu:
> New
> Status in mysql-5.6 package in Ubuntu:
> New
> Status in mysql-5.7 package in Ubuntu:
> New
>
> Bug description:
> Your automated bug reports are posting
> Logs.var.log.mysql.error.log.txt in clear text. These logs may
> contain PII as well as user credentials.
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/mariadb-10.0/+bug/1574458/+subscriptions
>

Revision history for this message
Seth Arnold (seth-arnold) wrote :

BCB, it's a bit ironic; because the bug is private, I can't get to it to delete the attachment; and because the url is now visible, it's easily downloaded even without privileges.

The right column of the launchpad interface, near the bottom of the column, is a box with all the attachments; hit 'edit' next to Logs.var... and the page that shows up will have a 'delete' option of some sort.

Thanks

Revision history for this message
BCB (fdajkddcek) wrote :

well that was easy. Thanks.

On Wed, Apr 27, 2016 at 6:00 PM, Seth Arnold <email address hidden>
wrote:

> BCB, it's a bit ironic; because the bug is private, I can't get to it to
> delete the attachment; and because the url is now visible, it's easily
> downloaded even without privileges.
>
> The right column of the launchpad interface, near the bottom of the
> column, is a box with all the attachments; hit 'edit' next to
> Logs.var... and the page that shows up will have a 'delete' option of
> some sort.
>
> Thanks
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1574458
>
> Title:
> Logs.var.log.mysql.error.log.txt contains usernames and passwords
>
> Status in mariadb-10.0 package in Ubuntu:
> New
> Status in mariadb-5.5 package in Ubuntu:
> New
> Status in mysql-5.5 package in Ubuntu:
> New
> Status in mysql-5.6 package in Ubuntu:
> New
> Status in mysql-5.7 package in Ubuntu:
> New
>
> Bug description:
> Your automated bug reports are posting
> Logs.var.log.mysql.error.log.txt in clear text. These logs may
> contain PII as well as user credentials.
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/mariadb-10.0/+bug/1574458/+subscriptions
>

Robie Basak (racb)
Changed in mysql-5.7 (Ubuntu):
assignee: nobody → Lars Tangvald (lars-tangvald)
importance: Undecided → High
status: New → Triaged
Robie Basak (racb)
Changed in mysql-5.7 (Ubuntu):
milestone: none → ubuntu-16.05
Revision history for this message
Lars Tangvald (lars-tangvald) wrote :

Until this is fixed upstream, as a mitigation effort we'll change the apport hook script to filter out (with a notice that it's been done) any lines in the error log that might contain private info.

Changed in mysql-5.7 (Ubuntu):
status: Triaged → In Progress
Revision history for this message
Robie Basak (racb) wrote :

Password logging aside, there's an open question about whether logging usernames is acceptable. I'd say it is, since usernames are generally actually webapps and thus are useful debugging information. Users are already prompted to see the report before sending it, and actively choose to send it. Dropping usernames would go in the direction of crippling useful reporting. Since users already choose to send reports, those bothered by this could just not send them.

Comments appreciated, but I'd look to the Ubuntu security team to make a final decision.

Revision history for this message
Seth Arnold (seth-arnold) wrote :

Since the usernames I've seen so far were all from applications I never really thought about it being a breach of privacy to include those in the logs. Filtering usernames seems like a good idea to me but I'm bothered way less about it than e.g. passwords.

Thanks

Revision history for this message
Robie Basak (racb) wrote :

The apport hook is no longer sending passwords as of 5.7.13-0ubuntu1 in Yakkety. Adding a task for Xenial.

I'm preparing an SRU for Xenial that will include this fix.

Seth, alternatively do you want it in the security pocket? Though that would mess with my planned SRU a little. We can arrange something if needed.

Changed in mysql-5.7 (Ubuntu):
status: In Progress → Fix Released
Changed in mysql-5.7 (Ubuntu Xenial):
assignee: nobody → Robie Basak (racb)
status: New → In Progress
Robie Basak (racb)
Changed in mysql-5.7 (Ubuntu Xenial):
importance: Undecided → High
Robie Basak (racb)
Changed in mysql-5.7 (Ubuntu Xenial):
milestone: none → xenial-updates
milestone: xenial-updates → ubuntu-16.04.1
description: updated
Revision history for this message
Robie Basak (racb) wrote :

Uploaded mysql-5.7 to Xenial. SRU team: you may find it easier to review if you verify my uploaded delta is the same as https://git.launchpad.net/~racb/ubuntu/+source/mysql-5.7/log/?h=mysql-5.7/ubuntu/xenial and then use that.

Revision history for this message
Adam Conrad (adconrad) wrote : Please test proposed package

Hello BCB, or anyone else affected,

Accepted mysql-5.7 into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/mysql-5.7/5.7.12-0ubuntu1.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in mysql-5.7 (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Verified that my testcase works on Xenial container, upgraded to proposed and checked again.
Log now contains:
--- Line containing protected term grant stripped from log by apport hook. Ref. Launchpad bug #1574458

So, verified

tags: added: verification-done
removed: verification-needed
Revision history for this message
Robie Basak (racb) wrote :

For mysql-5.7, I verified actual behaviour in xenial, and then upgraded to xenial-proposed. Now the apport report instead says "--- Line containing protected term grant stripped from log by apport hook. Ref. Launchpad bug #1574458" instead of revealing the password as expected.

description: updated
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package mysql-5.7 - 5.7.13-0ubuntu0.16.04.2

---------------
mysql-5.7 (5.7.13-0ubuntu0.16.04.2) xenial-security; urgency=medium

  * SECURITY UPDATE: Update to 5.7.13 to fix security issues (LP: #1604796)
    - http://www.oracle.com/technetwork/security-advisory/cpujul2016-2881720.html
    - CVE-2016-3424
    - CVE-2016-3459
    - CVE-2016-3477
    - CVE-2016-3486
    - CVE-2016-3501
    - CVE-2016-3518
    - CVE-2016-3521
    - CVE-2016-3588
    - CVE-2016-3614
    - CVE-2016-3615
    - CVE-2016-5436
    - CVE-2016-5437
    - CVE-2016-5439
    - CVE-2016-5440
    - CVE-2016-5441
    - CVE-2016-5442
    - CVE-2016-5443
  * debian/patches/mysql-export-scramble.patch: removed, upstream.

 -- Marc Deslauriers <email address hidden> Wed, 20 Jul 2016 08:44:25 -0400

Changed in mysql-5.7 (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
Nish Aravamudan (nacc) wrote :

mysql-5.5, mysql-5.6, mariadb-5.5 are all not in 16.04.

no longer affects: mysql-5.6 (Ubuntu Xenial)
no longer affects: mysql-5.5 (Ubuntu Xenial)
no longer affects: mariadb-5.5 (Ubuntu Xenial)
Revision history for this message
Nish Aravamudan (nacc) wrote :

@racb, @lars-tangvald:

I think I have set the various tasks correctly now. But I'm not sure if anyone has verified if the issue is present in Trusty (older versions of the MySQL packages) or in MariaDB's packaging.

no longer affects: mysql-5.7 (Ubuntu Trusty)
no longer affects: mariadb-10.0 (Ubuntu Trusty)
Changed in mysql-5.6 (Ubuntu):
status: New → Invalid
Changed in mysql-5.5 (Ubuntu):
status: New → Invalid
Changed in mariadb-5.5 (Ubuntu):
status: New → Invalid
Nish Aravamudan (nacc)
Changed in mysql-5.6 (Ubuntu Trusty):
status: New → Invalid
Changed in mariadb-10.0 (Ubuntu):
status: New → Invalid
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Checked 5.6 in Trusty it does not have the related apport hook that adds error.log
5.5 in Trusty has that hook so it generally applies there.

Nish Aravamudan (nacc)
Changed in mariadb-10.0 (Ubuntu Xenial):
status: New → Confirmed
Changed in mariadb-10.1 (Ubuntu):
status: New → Confirmed
Changed in mysql-5.5 (Ubuntu Trusty):
status: New → Confirmed
Changed in mariadb-5.5 (Ubuntu Trusty):
status: New → Confirmed
Revision history for this message
Nish Aravamudan (nacc) wrote :

I believe the 4 remaining tasks are now confirmed (my check was looking at the source package to see if the apport hook existed, and if it does whether it had the same fix as src:mysql-5.7 to filter the logs (they all filter the conf files already)).

Revision history for this message
Andreas Hasenack (ahasenack) wrote :

The mysql-5.5 trusty task is still open. Lars, can you pick that one up perpahs?

Mariadb is in universe and probably won't get an update.

Revision history for this message
Otto Kekäläinen (otto) wrote :

MariaDB hasn't been using passwords by default for a very long time, and most likely this bug never affected the MariaDB versions in Debian nor Ubuntu.

To post a comment you must log in.
This report contains Public Security information  
Everyone can see this security related information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.