The test fails since slaves are constantly going out-of-sync and cannot rejoin
the cluster. After three failovers, the cluster has three standalone server and
the monitor gets confused which server is the master. For now, limit the test to
two failovers.
The tests can now wait for a number of monitor intervals. This removes the
need to have hard-coded sleeps in the code and makes monitor tests more
robust under heavier load.
When auto_failover has been disabled due to failure to perform automatic
failover, the test should reset the number of failures. Resetting it
before the check for the log message allows the test to still fail if the
message is missing.
- Start 4 threads where each thread sits in a loop and performs
20% updates and 80% selects. Each thread has a table of its own.
- The main thread executes the following in a loop.
- Take down the current master and wait a while (failover assumed
to happen).
- Put up the old master node and wait a while.
Keep on doing that for 1.5 minutes.
At the end check that:
- There is one 'Master'.
- The other nodes are either
- 'Slave' or
- 'Running' in which case it is checked it is because the node could
not be rejoined.
