[bionic] mysql is not up sometimes in AIO setup

Bug #1916021 reported by Andrey Pavlov
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Percona Cluster Charm
Triaged
Medium
Unassigned

Bug Description

ubuntu 18.04, All-In-One setup

2021-02-18 01:53:49 DEBUG juju-log shared-db:37: Opening db connection for root@None
2021-02-18 01:53:49 WARNING shared-db-relation-changed Traceback (most recent call last):
2021-02-18 01:53:49 WARNING shared-db-relation-changed File "/var/lib/juju/agents/unit-mysql-0/charm/hooks/shared-db-relation-changed", line 1155, in <module>
2021-02-18 01:53:49 WARNING shared-db-relation-changed main()
2021-02-18 01:53:49 WARNING shared-db-relation-changed File "/var/lib/juju/agents/unit-mysql-0/charm/hooks/shared-db-relation-changed", line 1145, in main
2021-02-18 01:53:49 WARNING shared-db-relation-changed hooks.execute(sys.argv)
2021-02-18 01:53:49 WARNING shared-db-relation-changed File "/var/lib/juju/agents/unit-mysql-0/charm/charmhelpers/core/hookenv.py", line 945, in execute
2021-02-18 01:53:49 WARNING shared-db-relation-changed self._hooks[hook_name]()
2021-02-18 01:53:49 WARNING shared-db-relation-changed File "/var/lib/juju/agents/unit-mysql-0/charm/hooks/shared-db-relation-changed", line 853, in shared_db_changed
2021-02-18 01:53:49 WARNING shared-db-relation-changed db_helper)
2021-02-18 01:53:49 WARNING shared-db-relation-changed File "/var/lib/juju/agents/unit-mysql-0/charm/hooks/shared-db-relation-changed", line 805, in configure_db_for_hosts
2021-02-18 01:53:49 WARNING shared-db-relation-changed password = db_helper.configure_db(host, database, username)
2021-02-18 01:53:49 WARNING shared-db-relation-changed File "/var/lib/juju/agents/unit-mysql-0/charm/charmhelpers/contrib/database/mysql.py", line 464, in configure_db
2021-02-18 01:53:49 WARNING shared-db-relation-changed self.connect(password=self.get_mysql_root_password())
2021-02-18 01:53:49 WARNING shared-db-relation-changed File "/var/lib/juju/agents/unit-mysql-0/charm/charmhelpers/contrib/database/mysql.py", line 100, in connect
2021-02-18 01:53:49 WARNING shared-db-relation-changed self.connection = MySQLdb.connect(**_connection_info)
2021-02-18 01:53:49 WARNING shared-db-relation-changed File "/usr/lib/python3/dist-packages/MySQLdb/__init__.py", line 86, in Connect
2021-02-18 01:53:49 WARNING shared-db-relation-changed return Connection(*args, **kwargs)
2021-02-18 01:53:49 WARNING shared-db-relation-changed File "/usr/lib/python3/dist-packages/MySQLdb/connections.py", line 204, in __init__
2021-02-18 01:53:49 WARNING shared-db-relation-changed super(Connection, self).__init__(*args, **kwargs2)
2021-02-18 01:53:49 WARNING shared-db-relation-changed _mysql_exceptions.OperationalError: (2002, "Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)")
2021-02-18 01:53:50 ERROR juju.worker.uniter.operation runhook.go:136 hook "shared-db-relation-changed" (via explicit, bespoke hook script) failed: exit status 1

/var/log/syslog

Feb 18 01:19:42 juju-7f4505-0-lxd-3 mysql[11797]: * Stopping MySQL (Percona XtraDB Cluster) mysqld
Feb 18 01:19:57 juju-7f4505-0-lxd-3 /etc/init.d/mysql[11897]: MySQL PID not found, pid_file detected/guessed: /var/run/mysqld/mysqld.pid
Feb 18 01:19:57 juju-7f4505-0-lxd-3 /etc/init.d/mysql[11901]: MySQL PID not found, pid_file detected/guessed: /var/run/mysqld/mysqld.pid
Feb 18 01:19:57 juju-7f4505-0-lxd-3 mysql[11797]: ...done.
Feb 18 01:19:57 juju-7f4505-0-lxd-3 systemd[1]: Stopped Percona XtraDB Cluster daemon.
Feb 18 01:19:57 juju-7f4505-0-lxd-3 systemd[1]: system-mysql.slice: Failed to reset devices.list: Operation not permitted
Feb 18 01:19:57 juju-7f4505-0-lxd-3 systemd[1]: Created slice system-mysql.slice.
Feb 18 01:19:57 juju-7f4505-0-lxd-3 systemd[1]: Starting Percona XtraDB Cluster daemon with config /etc/default/mysql.bootstrap...
Feb 18 01:20:00 juju-7f4505-0-lxd-3 mysql[11974]: WSREP: Recovered position 51893779-7187-11eb-95b0-1ab480f92572:2
Feb 18 01:20:01 juju-7f4505-0-lxd-3 systemd[1]: Failed to attach 11978 to compat systemd cgroup /<email address hidden>: Permission denied
Feb 18 01:20:01 juju-7f4505-0-lxd-3 mysql[11978]: * Starting MySQL (Percona XtraDB Cluster) database server mysqld
Feb 18 01:20:01 juju-7f4505-0-lxd-3 /etc/init.d/mysql[12015]: MySQL PID not found, pid_file detected/guessed: /var/run/mysqld/mysqld.pid
Feb 18 01:20:05 juju-7f4505-0-lxd-3 mysql[11978]: ...done.
Feb 18 01:20:05 juju-7f4505-0-lxd-3 systemd[1]: Started Percona XtraDB Cluster daemon with config /etc/default/mysql.bootstrap.
Feb 18 01:24:17 juju-7f4505-0-lxd-3 mysql[16242]: * Stopping MySQL (Percona XtraDB Cluster) mysqld
Feb 18 01:24:17 juju-7f4505-0-lxd-3 /etc/init.d/mysql[16269]: MySQL PID not found, pid_file detected/guessed: /var/run/mysqld/mysqld.pid
Feb 18 01:24:17 juju-7f4505-0-lxd-3 /etc/init.d/mysql[16273]: MySQL PID not found, pid_file detected/guessed: /var/run/mysqld/mysqld.pid
Feb 18 01:24:17 juju-7f4505-0-lxd-3 mysql[16242]: ...done.

Revision history for this message
Andrey Pavlov (apavlov-e) wrote :
summary: - mysql is not up sometimes in AIO setup
+ [bionic] mysql is not up sometimes in AIO setup
Revision history for this message
Alex Kavanagh (ajkavanagh) wrote :

Setting to medium as AIO is a test target, and wouldn't be used in production. Still be good to resolve why it is happening.

Changed in charm-percona-cluster:
status: New → Triaged
importance: Undecided → Medium
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Bug attachments

Remote bug watches

Bug watches keep track of this bug in other bug trackers.