Corosync init script doesn't shut down properly, causing split brain

On 11.01.2010 17:21, halfgaar wrote:

> The init script makes a mention of that bug and it indeed seems that
> their 'fix' is included, yet I still get a split brain whenever I stop
> and start corosync, or when I reboot the machine.

Unfortunately, that fix doesn't solve the issue. As a workaround, I put
my node offline before stoping corosync.

IIRC, I've included that workaround into init script. I'll check if that
isn't the case.

          Status: Triaged

This is what the init script says now:

do_stop()
{
        # Return
        # 0 if daemon has been stopped
        # 1 if daemon was already stopped
        # 2 if daemon could not be stopped
        # other if a failure occurred
        # Workaround for a shutdown bug in pacemaker
        # (https://bugzilla.redhat.com/show_bug.cgi?id=525589)
        if [ -r /usr/sbin/crm ]; then
                crm node standby
                start-stop-daemon --stop --quiet --retry=QUIT/5/QUIT/15 --pidfile $PIDFILE
                RETVAL="$?"
        else
                start-stop-daemon --stop --quiet --signal=QUIT --retry=5 --pidfile $PIDFILE
                RETVAL="$?"
        fi
        [ "$RETVAL" = 2 ] && return 2
        # Many daemons don't delete their pidfiles when they exit.
        rm -f $PIDFILE
        return "$RETVAL"
}

It does put the node on standby, but that isn't enough, apparently. Does this have to do with it being a background operation?

Also, when nodes are put in standby like that, they don't automatically start when corosync starts. So, the nodes are left offline when the machine boots.

Plus, it'd be better to check for -x, as opposed to -r.

What might be useful into to add, is that I'm testing a cluster setup on two old machines. One 400 MHz, 192MB ram, slow 2GB disk and one AMD 1600+, 512 MB ram, slow old 4 GB disk. Perhaps the problem only shows on my machines because they are so slow. It's still a bug of course, but it gives some additional insight.

I'll also try to patch the init script to work around the pacemaker problem a bit.