123 Commits

Author SHA1 Message Date
Esa Korhonen
39df829384 Don't kick out super-users when performing rejoin
When rejoining a server, preventing writes is not as essential as in switchover.
2019-06-19 17:14:39 +03:00
Esa Korhonen
d128c9a09b MXS-2504 Kick out super-users from master server during switchover
The monitor queries for logged in users with super-privileges and kicks them out to
prevent writes to master. Normal users can stay since their writes are prevented by
read_only. Also, the master-status is removed from the master manually to signal to
routers that no more writes should go to master.
2019-05-27 18:11:46 +03:00
Esa Korhonen
f8a22d0ac0 MXS-2344 Add setting for enabling SSL for replication
If the monitor setting "replication_master_ssl" is set to on, any CHANGE MASTER TO-command
will have MASTER_SSL=1. If set to off or unset, MASTER_SSL is left unchanged to match existing
behaviour.
2019-04-15 19:15:45 +03:00
Esa Korhonen
040562f718 MXS-2342 Run MariaDBMonitor diagnostics concurrent with the monitor loop
This fixes some situations where MaxAdmin/MaxCtrl would block and wait
until a monitor operation or tick is complete. This also fixes a deadlock
caused by calling monitor diagnostics inside a monitor script.

Concurrency is enabled by adding one mutex per server object to protect
array-like fields from concurrent reading/writing.
2019-03-12 10:50:16 +02:00
Esa Korhonen
50f588db3e MXS-2370 Clarify query timeout warning message
The message now more clearly states if the failure was due to timeout or
a different Connector-C error.
2019-03-11 13:20:50 +02:00
Esa Korhonen
4fd4b726a1 MXS-2325 Only enable events that were enabled on the master
The monitor now continuously updates a list of enabled server events. When
promoting a new master in failover/switchover, only events that were enabled
on the previous master are enabled on the new. This avoids enabling events
that may have been disabled on the master yet stayed in the SLAVESIDE_DISABLED-
state on the slave.

In the case of reset-replication command, events on the new master are only
enabled if the monitor had a master when the command was launched. Otherwise
all events remain disabled.
2019-03-04 16:00:07 +02:00
Esa Korhonen
b47d4a4105 Use max_statement_time
Used only with supporting server versions. Using the time limit ensures that
the server interrupts the query at the same point Connector-C would cut the
connection. This prevents lingering queries.

Also, cleans up some associated error messages.
2018-11-23 15:15:44 +02:00
Esa Korhonen
0a5a3309e0 Add missing quotes when printing server names
Some of the log messages didn't have the quotes.
2018-11-23 14:02:09 +02:00
Esa Korhonen
fb52e565fe Store capabilities of monitored server
Checking the version number in various places in the code gets confusing.
It's better to check the version in one place and record the relevant data.
2018-11-21 17:36:52 +02:00
Esa Korhonen
01628dd0de Cleanup server version updating 2018-11-21 17:36:52 +02:00
Esa Korhonen
1a046bd453 MXS-2168 Add 'assume_unique_hostnames'-setting to MariaDBMonitor
Adds the setting and takes it into use during replication graph creation
and the most important checks.
2018-11-21 10:30:11 +02:00
Esa Korhonen
6a1cfddb43 MXS-2158 Clean up gtid updating during rejoin
Error messages from update_gtids() are now printed. can_replicate_from()
no longer updates gtid:s.
2018-11-16 12:56:24 +02:00
Esa Korhonen
14e38e4e08 MXS-2158 Return true if update_gtids() succeeds, even if no data is returned
Previously, if the server had no gtid:s, the method would fail leading to
a confusing error message. This could even totally stop the monitor from working
if a recent server version (10.X) did not have any gtid events.
2018-11-14 10:56:42 +02:00
Esa Korhonen
fb3ccc94d6 MXS-1944 Cleanup function parameters
The naming and ordering is now a bit more consistent between promote() and demote().
2018-11-07 12:55:59 +02:00
Esa Korhonen
e4e2235297 MXS-1944 Use time limited methods in rejoin
Uses switchover time limit, since the typical rejoin of a standalone server
is somewhat similar to a switchover.
2018-11-07 12:55:59 +02:00
Esa Korhonen
a4ce4e4613 MariaDBServer no longer uses ClusterOperation
The functions in the server class now only use the general parameters object.
2018-11-07 12:55:59 +02:00
Esa Korhonen
8877e7180b Continue separation of ClusterOperation elements 2018-11-07 12:55:59 +02:00
Esa Korhonen
90e6ff078a Divide ClusterOperation to two types
The main class was getting unwieldly and too general. Dividing the fields
helps adding support for other operation types.

This commit leaves most data duplicated, later commits clean up the affected code.
2018-11-07 12:55:59 +02:00
Esa Korhonen
c5a54d2fe9 Disallow switchover promotion of a server low on disk space
This protects the user and also prevents a neverending series of
automatic switchovers in the case when all servers are low.
2018-11-06 11:44:50 +02:00
Markus Mäkelä
906d8cee5b
Format all files
Formatted all files with uncrustify.
2018-10-31 09:46:02 +02:00
Esa Korhonen
36b666898c Fix connection merging
The conditional was inverted.
2018-10-16 16:09:38 +03:00
Esa Korhonen
2d61b78439 Fix low disk space maintenance
The setting didn't work because the code updated a status flag which
would be overwritten before being read. Also, promotion code now checks
that the server is not in maintenance.
2018-10-16 16:09:38 +03:00
Esa Korhonen
0c203fa02d Don't redirect duplicate connections
The redirection method checks if a slave connection to the redirection
target already exists. If so, the connection is not modified. Also, failover
better detects duplicate connections during promotion.
2018-10-16 16:09:38 +03:00
Esa Korhonen
e930270b9c Use copy when checking removed connections
The function modifies the reference parameter contents indirectly.
2018-10-16 16:09:38 +03:00
Esa Korhonen
f554ef770b Allow switchover for arbitrary topologies
The demoted server no longer needs to be the master.
2018-10-16 16:09:38 +03:00
Esa Korhonen
0cf8ea43f7 Redirect slaves of promotion target
This affects situations where the promoted server is a relay or multimaster
group member.
2018-10-16 16:09:38 +03:00
Esa Korhonen
d0444ff054 SlaveStatus::to_short_string() uses member field
The owner server name is now stored in a field.
2018-10-10 17:26:48 +03:00
Esa Korhonen
2f1512a22d Cleanup slave connection removal during promotion/demotion
The removing and slave status updating is now separated to a function.
As the MariaDBServer object now contains the updated slave connections,
keeping track of removed connections is no longer required.
2018-10-09 14:29:49 +03:00
Esa Korhonen
c10fab977d Cleanup slave connection copy & merge
The two cases are now separated. In switchover, the promotion and
demotion targets can swap connections between each other without worry.
In failover, the two connection lists must be merged semi-intelligently.

The slave connections of the two servers are now saved to the operation
descriptor object at the start of the operation. This allows slave status
updating during the operation.
2018-10-09 14:29:49 +03:00
Esa Korhonen
4d6f961695 Cleanup mariadbserver.hh 2018-10-09 14:29:49 +03:00
Esa Korhonen
68d65682b5 Reorganize MariaDBServer code
The server-class keeps growing, so the additional classes are moved out of
the main class file.
2018-10-09 14:29:49 +03:00
Esa Korhonen
a398da58a4 Add sleep to execute_cmd_time_limit
If the query fails instantly, the retries end up busy-looping. Now
each try is at least one second.
2018-10-04 20:19:57 +03:00
Esa Korhonen
30eb21914f MXS-1845 Switchover cleanup
Several small changes:
Binlog is flushed at the end of old master demotion.
Only new master is required to catch up to old master.
Use the same replication check method as failover.
2018-10-04 11:45:33 +03:00
Esa Korhonen
49e85d9a28 MXS-1845 Add demotion code
The master demotion in switchover now uses query retrying with
the switchover time limit.
2018-10-04 11:45:33 +03:00
Esa Korhonen
d14b9bfe43 MXS-1845 Cluster stabilization rewrite
No longer writes events to the master, as this creates problems if the
promoted server was not the overall master. Instead, the slave status
output is inspected.
2018-10-02 11:09:16 +03:00
Esa Korhonen
1ca5d02abb MXS-1845 Add redirection code
Should work with multimaster replication.
2018-10-02 11:09:16 +03:00
Esa Korhonen
6b8443aba6 MXS-1845 Complete server promotion code
Now copies slave connections from the previous master. Promotion
code taken into use.
2018-10-01 18:06:39 +03:00
Niclas Antti
c65edd1298 Enhance StopWatch
Clean up, comments and enhancements. StopWatch lap() didn't mean lap-time, but elapsed time. Changed meaning to lap-time and added split() for split-time.
2018-10-01 09:30:24 +03:00
Esa Korhonen
fe81b399b2 Use maxbase time and clock classes instead of std::chrono 2018-09-27 17:04:59 +03:00
Esa Korhonen
05d18e81ae Use string instead of stringstream
Most of the monitor was already using string for formatted printing.
2018-09-27 16:46:59 +03:00
Esa Korhonen
dd9ff27743 MXS-1845 Rewrite server promotion code
In progress, does not yet overwrite existing code.

The new promotion mechanism automatically retries queries which timed out. It also
handles multimaster situations correctly.
2018-09-26 13:20:29 +03:00
Esa Korhonen
c58041d4fb Format MariaDBMonitor source
Some parts were manually edited for better results. No functional changes.
2018-09-26 12:49:14 +03:00
Esa Korhonen
c20a17238b MXS-1944 Store failover parameters in an object
Several of the parameters are passed on from function to function. Having them all
in an object cleans things up and makes adding more data easier.
2018-09-26 12:26:35 +03:00
Esa Korhonen
dfc10b109d MXS-1937 Add failover/switchover with server events test
The test adds a scheduled server event, the does failover, rejoin and
switchover and checks that event is manipulated correctly. Also includes
a change to the monitor to fix an invalid ALTER EVENT query when the event
definer has wildcard host.
2018-09-24 10:43:17 +03:00
Esa Korhonen
a3adcea524 MXS-1712 Cleanup reset-replication
Now logs messages explaining what has been done. Scheduled events are
disabled/enabled during the operation. Redirection of slaves is done at
the end similar to failover/switchover.
2018-09-21 10:54:36 +03:00
Markus Mäkelä
71ffef5708
Partially revert 4ba011266843857bbd3201e5b925a47e88e1808f
Add back leading operator enforcement.
2018-09-20 15:57:30 +03:00
Esa Korhonen
02ac394e38 Cleanup slave status handling
Further reduce direct indexing slave status array to improve compatibility with
multimaster replication.
2018-09-19 13:37:24 +03:00
Esa Korhonen
56c84541df MXS-1712 Add reset replication to MariaDB Monitor
The 'reset_replication' module command deletes all slave connections and binlogs,
sets gtid to sequence 0 and restarts replication from the given master. Should be
only used if gtid:s are incompatible but the actual data is known to be in sync.
2018-09-14 17:15:05 +03:00
Esa Korhonen
cb54880b99 MXS-1937 Cleanup event handling
Event handling is now enabled by default. If the monitor cannot query the EVENTS-
table (most likely because of missing credentials), print an error suggesting to
turn the feature off.

When disabling events on a rejoining standalone server (likely a former master),
disable binlog event recording for the session. This prevents the ALTER EVENT
queries from generating binlog events.

Also added documentation and combined similar parts in the code.
2018-09-14 16:54:24 +03:00
Markus Mäkelä
108638b0cf
Format with Uncrustify 0.67 2018-09-10 13:31:39 +03:00