Strange issue, nodes join the cluster and then suddenly abort
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Galera |
New
|
Undecided
|
Unassigned |
Bug Description
Please advice me as per below logs
I am unable to find any reason
Distributor ID: Ubuntu
Description: Ubuntu 20.04.5 LTS
Release: 20.04
Codename: focal
Galera setup :
Total of 4 nodes and all three are connected except the fourth one.
the fourth one is running on aarch hardware.
Below logs are for the fourth node which connects and later gets kicked out for no reason,
(systemctl status mariadb)
● mariadb.service - MariaDB 10.3.37 database server
Loaded: loaded (/lib/systemd/
Active: failed (Result: exit-code) since Tue 2023-01-24 04:42:10 UTC; 1min 19s ago
Docs: man:mysqld(8)
https:/
Process: 838546 ExecStartPre=
Process: 838547 ExecStartPre=
Process: 838549 ExecStartPre=
Process: 838670 ExecStart=
Main PID: 838670 (code=exited, status=1/FAILURE)
Status: "MariaDB server is down"
Jan 24 04:41:37 instance-
Jan 24 04:41:39 instance-
Jan 24 04:41:39 instance-
Jan 24 04:42:10 instance-
Jan 24 04:42:10 instance-
Jan 24 04:42:10 instance-
Linux instance-
mysql Ver 15.1 Distrib 10.3.37-MariaDB, for debian-linux-gnu (aarch64) using readline 5.2
(tail -f /var/log/
2023-01-22 6:26:52 0 [Note] WSREP: Read nil XID from storage engines, skipping position init
2023-01-22 6:26:52 0 [Note] WSREP: wsrep_load(): loading provider library '/usr/lib/
2023-01-22 6:26:52 0 [Note] WSREP: wsrep_load(): Galera 3.29(ra60e019) by Codership Oy <email address hidden> loaded successfully.
2023-01-22 6:26:52 0 [Note] WSREP: CRC-32C: using "slicing-by-8" algorithm.
2023-01-22 6:26:52 0 [Note] WSREP: Found saved state: f1e1e4a1-
2023-01-22 6:26:52 0 [Note] WSREP: Passing config to GCS: base_dir = /var/lib/mysql/; base_host = 10.0.1.169; base_port = 4567; cert.log_conflicts = no; cert.optimistic_pa = yes; debug = no; evs.auto_evict = 0; evs.delay_margin = PT1S; evs.delayed_
2023-01-22 6:26:52 0 [Note] WSREP: Assign initial position for certification: 0, protocol version: -1
2023-01-22 6:26:52 0 [Note] WSREP: wsrep_sst_grab()
2023-01-22 6:26:52 0 [Note] WSREP: Start replication
2023-01-22 6:26:52 0 [Note] WSREP: Setting initial position to f1e1e4a1-
2023-01-22 6:26:52 0 [Note] WSREP: protonet asio version 0
2023-01-22 6:26:52 0 [Note] WSREP: Using CRC-32C for message checksums.
2023-01-22 6:26:52 0 [Note] WSREP: backend: asio
2023-01-22 6:26:52 0 [Note] WSREP: gcomm thread scheduling priority set to other:0
2023-01-22 6:26:52 0 [Warning] WSREP: access file(/var/
2023-01-22 6:26:52 0 [Note] WSREP: restore pc from disk failed
2023-01-22 6:26:52 0 [Note] WSREP: GMCast version 0
2023-01-22 6:26:52 0 [Note] WSREP: (c273183b, 'tcp://
2023-01-22 6:26:52 0 [Note] WSREP: (c273183b, 'tcp://
2023-01-22 6:26:52 0 [Note] WSREP: EVS version 0
2023-01-22 6:26:52 0 [Note] WSREP: gcomm: connecting to group 'MariaDB Galera Cluster', peer '35.212.
2023-01-22 6:26:52 0 [Note] WSREP: (c273183b, 'tcp://
2023-01-22 6:26:52 0 [Note] WSREP: (c273183b, 'tcp://
2023-01-22 6:26:52 0 [Note] WSREP: (c273183b, 'tcp://
2023-01-22 6:26:52 0 [Note] WSREP: (c273183b, 'tcp://
2023-01-22 6:26:53 0 [Note] WSREP: (c273183b, 'tcp://
2023-01-22 6:26:53 0 [Note] WSREP: (c273183b, 'tcp://
2023-01-22 6:26:53 0 [Note] WSREP: (c273183b, 'tcp://
2023-01-22 6:26:54 0 [Note] WSREP: declaring 5149ecf7 at tcp://35.
2023-01-22 6:26:54 0 [Note] WSREP: declaring 6ab548fb at tcp://85.
2023-01-22 6:26:54 0 [Note] WSREP: declaring 7191d038 at tcp://192.
2023-01-22 6:26:55 0 [Note] WSREP: view(view_
5149ecf7,0
6ab548fb,0
7191d038,0
c273183b,0
} joined {
} left {
} partitioned {
07d1e878,0
77a052d6,0
9e708b0d,0
a1d8493f,0
a79d17a2,0
b2984c45,0
b60181c9,0
bad3a406,0
c401108d,0
d392217d,0
dc8cc78b,0
})
2023-01-22 6:26:56 0 [Note] WSREP: (c273183b, 'tcp://
2023-01-22 6:27:23 0 [ERROR] WSREP: failed to open gcomm backend connection: 110: failed to reach primary view: 110 (Connection timed out)
at gcomm/src/
2023-01-22 6:27:23 0 [ERROR] WSREP: gcs/src/
2023-01-22 6:27:23 0 [ERROR] WSREP: gcs/src/
2023-01-22 6:27:23 0 [ERROR] WSREP: gcs connect failed: Connection timed out
2023-01-22 6:27:23 0 [ERROR] WSREP: wsrep::
2023-01-22 6:27:23 0 [ERROR] Aborting
description: | updated |