hcq/MaxScale
hcq/MaxScale
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch '2.3' of github.com:mariadb-corporation/MaxScale into 2.3

Browse Source
This commit is contained in:
Timofey Turenko 2019-04-18 09:29:20 +03:00
commit 802a19879b
33 changed files with 258 additions and 186 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
BUILD/mdbci/copy_repos.sh
View File

@ -8,7 +8,10 @@ if [ "$box_type" == "RPM" ] ; then
# Build can be executed to check if it is possible to build
# and to run install and upgrade tests
# with thre real RHEL, but we use CentOS packages for production
if [ "$platform" != "rhel" ] ; then
if [[ "$platform" != "rhel" || ( "$platform" == "rhel" && "$platform_version" == "8" ) ]]; then
if [[ "$platform" == "rhel" && "$platform_version" == "8" ]]; then
export platform="centos"
fi
export arch=`ssh $sshopt "arch"`
. ${script_dir}/generate_build_info_path.sh

1
Documentation/Changelog.md
View File

@ -31,6 +31,7 @@
For more details, please refer to:
* [MariaDB MaxScale 2.3.6 Release Notes](Release-Notes/MaxScale-2.3.6-Release-Notes.md)
* [MariaDB MaxScale 2.3.5 Release Notes](Release-Notes/MaxScale-2.3.5-Release-Notes.md)
* [MariaDB MaxScale 2.3.4 Release Notes](Release-Notes/MaxScale-2.3.4-Release-Notes.md)
* [MariaDB MaxScale 2.3.3 Release Notes](Release-Notes/MaxScale-2.3.3-Release-Notes.md)

8
Documentation/Getting-Started/Configuration-Guide.md
View File

@ -679,7 +679,9 @@ _qc_sqlite_.
#### `query_classifier_cache_size`
Specifies the maximum size of the query classifier cache. The default limit is
40% of total system memory.
15% of total system memory starting with MaxScale 2.3.7. In older versions the
default limit was 40% of total system memory. This feature was added in MaxScale
2.3.0.
When the query classifier cache has been enabled, MaxScale will, after a
statement has been parsed, store the classification result using the
@ -855,7 +857,7 @@ than `0`, this configuration setting will not have an effect.
#### `writeq_high_water`
High water mark for network write buffer. Controls when network traffic
throtting is started. The parameter accepts size type values.
throtting is started. The parameter accepts [size type values](#sizes).
More specifically, if the client side write queue is above this value, it will
block traffic coming from backend servers. If the backend side write queue is
@ -869,7 +871,7 @@ throtting is enabled. By default, traffic throttling is disabled.
Low water mark for network write buffer. Once the traffic throttling is enabled,
it will only be disabled when the write queue is below `writeq_low_water`. The
parameter accepts size type values. The minimum allowed size is 512
parameter accepts [size type values](#sizes). The minimum allowed size is 512
bytes. `writeq_high_water` must always be greater than `writeq_low_water`.
#### `load_persisted_configs`

15
Documentation/Monitors/MariaDB-Monitor.md
View File

@ -651,7 +651,7 @@ executed.
Both `replication_user` and `replication_password` parameters must be defined if
a custom replication user is used. If neither of the parameters is defined, the
`CHANGE MASTER TO` command will use the monitor credentials for the replication
`CHANGE MASTER TO`-command will use the monitor credentials for the replication
user.
The credentials used for replication must have the `REPLICATION SLAVE`
@ -661,6 +661,19 @@ privilege.
parameters. If password encryption is in use, `replication_password` must be
encrypted with the same key to avoid erroneous decryption.
#### `replication_master_ssl`
Type: bool Default: off
If set to ON, any `CHANGE MASTER TO`-command generated will set `MASTER_SSL=1` to enable
encryption for the replication stream. This setting should only be enabled if the backend
servers are configured for ssl. This typically means setting *ssl_ca*, *ssl_cert* and
*ssl_key* in the server configuration file. Additionally, credentials for the replication
user should require an encrypted connection (`e.g. ALTER USER repl@'%' REQUIRE SSL;`).
If the setting is left OFF, `MASTER_SSL` is not set at all, which will preserve existing
settings when redirecting a slave connection.
#### `failover_timeout` and `switchover_timeout`
Time limit for failover and switchover operations, in seconds. The default

5
Documentation/Release-Notes/MaxScale-2.3.6-Release-Notes.md
View File

@ -11,6 +11,7 @@ report on [our Jira](https://jira.mariadb.org/projects/MXS).
## New Features
* [MXS-2417](https://jira.mariadb.org/browse/MXS-2417) MaxScale main config should take precedence over runtime config on restart
* [MXS-2344](https://jira.mariadb.org/browse/MXS-2344) Support MASTER_SSL in mariadbmon for encrypting replication traffic
### REST API & MaxCtrl: Hard maintenance mode
@ -24,8 +25,12 @@ the `set` endpoint.
## Bug fixes
* [MXS-2423](https://jira.mariadb.org/browse/MXS-2423) retain_last_statements not in maxctrl show maxscale
* [MXS-2419](https://jira.mariadb.org/browse/MXS-2419) Hangs on query during multiple transaction replays
* [MXS-2418](https://jira.mariadb.org/browse/MXS-2418) Crash on transaction replay if log_info is on and session starts with no master
* [MXS-2416](https://jira.mariadb.org/browse/MXS-2416) Possible memory leak
* [MXS-2324](https://jira.mariadb.org/browse/MXS-2324) Maxscale disconnect after commit without begin from Icinga2
* [MXS-2259](https://jira.mariadb.org/browse/MXS-2259) Maxscale consumes large amounts of memory even with buffer limits set.
## Known Issues and Limitations

1
include/maxscale/protocol/rwbackend.hh
View File

@ -156,6 +156,7 @@ private:
bool m_local_infile_requested; /**< Whether a LOCAL INFILE was requested */
ResponseStat m_response_stat;
uint64_t m_num_coldefs = 0;
bool m_skip_next = false;
inline bool is_opening_cursor() const
{

22
maxctrl/lib/alter.js
View File

@ -135,6 +135,28 @@ exports.builder = function(yargs) {
return updateValue(host, 'maxscale', 'data.attributes.parameters.' + argv.key, argv.value)
})
})
.command('user <name> <password>', 'Alter admin user passwords', function(yargs) {
return yargs.epilog('Changes the password for a user. To change the user type, destroy the user and then create it again.')
.usage('Usage: alter user <name> <password>')
}, function(argv) {
maxctrl(argv, function(host) {
var user = {
'data': {
'id': argv.name,
'type': 'inet',
'attributes': {
'password': argv.password
}
}
}
return getJson(host, 'users/inet/' + argv.name)
.then((res) => user.data.attributes.account = res.data.attributes.account)
.then(() => doRequest(host, 'users/inet/' + argv.name, null, {method: 'DELETE'}))
.then(() => doRequest(host, 'users/inet', null, {method: 'POST', body: user}))
})
})
.usage('Usage: alter <command>')
.help()
.command('*', 'the default command', {}, function(argv) {

11
maxctrl/lib/create.js
View File

@ -159,12 +159,10 @@ exports.builder = function(yargs) {
}
}
if (argv.params) {
var err = validateParams(argv, argv.params)
if (err) {
return Promise.reject(err)
}
var err = false;
if (argv.params) {
err = validateParams(argv, argv.params)
monitor.data.attributes.parameters = argv.params.reduce(to_obj, {})
}
@ -182,6 +180,9 @@ exports.builder = function(yargs) {
}
maxctrl(argv, function(host) {
if (err) {
return Promise.reject(err)
}
return doRequest(host, 'monitors', null, {method: 'POST', body: monitor})
})
})

12
maxctrl/test/alter.js
View File

@ -108,5 +108,17 @@ describe("Alter Commands", function() {
.should.be.rejected
})
it('creates user', function() {
return verifyCommand('create user testuser test', 'users/inet/testuser')
})
it('alters the password of a user', function() {
return verifyCommand('alter user testuser test2', 'users/inet/testuser')
})
it('destroys the altered user', function() {
return doCommand('destroy user testuser')
})
after(stopMaxScale)
});

17
maxctrl/test/createdestroy.js
View File

@ -18,6 +18,18 @@ describe("Create/Destroy Commands", function() {
.should.be.rejected
})
it('monitor without parameters fails due to missing user parameter', function() {
return verifyCommand('create monitor my-monitor mysqlmon', 'monitors/my-monitor')
.should.be.rejected
})
it('destroy monitor created without parameters', function() {
return doCommand('destroy monitor my-monitor')
.should.be.fulfilled
.then(() => doCommand('show monitor my-monitor'))
.should.be.rejected
})
it('will not destroy the same monitor again', function() {
return doCommand('destroy monitor my-monitor')
.should.be.rejected
@ -38,6 +50,11 @@ describe("Create/Destroy Commands", function() {
.should.be.rejected
})
it('will not create monitor with malformed parameters', function() {
return doCommand('create monitor my-monitor mariadbmon not-a-param')
.should.be.rejected
})
it('create monitor with options', function() {
return doCommand('unlink monitor MariaDB-Monitor server4')
.then(() => verifyCommand('create monitor my-monitor mysqlmon --servers server4 --monitor-user maxuser --monitor-password maxpwd',

5
maxctrl/test/drain.js
View File

@ -15,5 +15,10 @@ describe("Draining servers", function() {
.should.eventually.have.string("Maintenance")
})
it('does not drain non-existent server', function() {
return doCommand('drain server not-a-server')
.should.be.rejected
})
after(stopMaxScale)
});

3
maxctrl/test/unknown.js
View File

@ -19,7 +19,8 @@ describe("Unknown Commands", function() {
'alter',
'rotate',
'call',
'cluster'
'cluster',
'drain'
]
endpoints.forEach(function (i) {

2
maxscale-system-test/maxadmin_operations.h
View File

@ -18,8 +18,6 @@
#include <errno.h>
#include <getopt.h>
// #include <version.h>
/**
* @brief Connect to the MaxScale server

8
maxscale-system-test/mdbci/cnf/galera_server1.cnf
View File

@ -7,7 +7,7 @@ wsrep_on=ON
# Row binary log format is required by Galera
binlog_format=ROW
log-bin
log-bin=mar-bin
# InnoDB is currently the only storage engine supported by Galera
default-storage-engine=innodb
@ -16,9 +16,6 @@ innodb_file_per_table
# To avoid issues with 'bulk mode inserts' using autoincrement fields
innodb_autoinc_lock_mode=2
# Required to prevent deadlocks on parallel transaction execution
innodb_locks_unsafe_for_binlog=1
# Query Cache is not supported by Galera wsrep replication
query_cache_size=0
query_cache_type=0
@ -87,9 +84,6 @@ wsrep_auto_increment_control=1
# Retry autoinc insert, when the insert failed for "duplicate key error"
wsrep_drupal_282555_workaround=0
# Enable "strictly synchronous" semantics for read operations
wsrep_causal_reads=1
# Command to call when node status or cluster membership changes.
# Will be passed all or some of the following options:
# --status - new status of this node

8
maxscale-system-test/mdbci/cnf/galera_server2.cnf
View File

@ -7,7 +7,7 @@ wsrep_on=ON
# Row binary log format is required by Galera
binlog_format=ROW
log-bin
log-bin=mar-bin
# InnoDB is currently the only storage engine supported by Galera
default-storage-engine=innodb
@ -16,9 +16,6 @@ innodb_file_per_table
# To avoid issues with 'bulk mode inserts' using autoincrement fields
innodb_autoinc_lock_mode=2
# Required to prevent deadlocks on parallel transaction execution
innodb_locks_unsafe_for_binlog=1
# Query Cache is not supported by Galera wsrep replication
query_cache_size=0
query_cache_type=0
@ -87,9 +84,6 @@ wsrep_auto_increment_control=1
# Retry autoinc insert, when the insert failed for "duplicate key error"
wsrep_drupal_282555_workaround=0
# Enable "strictly synchronous" semantics for read operations
wsrep_causal_reads=1
# Command to call when node status or cluster membership changes.
# Will be passed all or some of the following options:
# --status - new status of this node

8
maxscale-system-test/mdbci/cnf/galera_server3.cnf
View File

@ -7,7 +7,7 @@ wsrep_on=ON
# Row binary log format is required by Galera
binlog_format=ROW
log-bin
log-bin=mar-bin
# InnoDB is currently the only storage engine supported by Galera
default-storage-engine=innodb
@ -16,9 +16,6 @@ innodb_file_per_table
# To avoid issues with 'bulk mode inserts' using autoincrement fields
innodb_autoinc_lock_mode=2
# Required to prevent deadlocks on parallel transaction execution
innodb_locks_unsafe_for_binlog=1
# Query Cache is not supported by Galera wsrep replication
query_cache_size=0
query_cache_type=0
@ -87,9 +84,6 @@ wsrep_auto_increment_control=1
# Retry autoinc insert, when the insert failed for "duplicate key error"
wsrep_drupal_282555_workaround=0
# Enable "strictly synchronous" semantics for read operations
wsrep_causal_reads=1
# Command to call when node status or cluster membership changes.
# Will be passed all or some of the following options:
# --status - new status of this node

8
maxscale-system-test/mdbci/cnf/galera_server4.cnf
View File

@ -7,7 +7,7 @@ wsrep_on=ON
# Row binary log format is required by Galera
binlog_format=ROW
log-bin
log-bin=mar-bin
# InnoDB is currently the only storage engine supported by Galera
default-storage-engine=innodb
@ -16,9 +16,6 @@ innodb_file_per_table
# To avoid issues with 'bulk mode inserts' using autoincrement fields
innodb_autoinc_lock_mode=2
# Required to prevent deadlocks on parallel transaction execution
innodb_locks_unsafe_for_binlog=1
# Query Cache is not supported by Galera wsrep replication
query_cache_size=0
query_cache_type=0
@ -87,9 +84,6 @@ wsrep_auto_increment_control=1
# Retry autoinc insert, when the insert failed for "duplicate key error"
wsrep_drupal_282555_workaround=0
# Enable "strictly synchronous" semantics for read operations
wsrep_causal_reads=1
# Command to call when node status or cluster membership changes.
# Will be passed all or some of the following options:
# --status - new status of this node

1
maxscale-system-test/mxs559_block_master.cpp
View File

@ -114,6 +114,7 @@ int main(int argc, char* argv[])
test.try_query(test.maxscales->conn_rwsplit[0], "DROP TABLE IF EXISTS t1");
test.maxscales->close_maxscale_connections(0);
test.maxscales->wait_for_monitor();
test.check_maxscale_alive(0);
test.log_excludes(0, "due to authentication failure");
test.log_excludes(0, "due to handshake failure");

2
maxscale-system-test/mxs585.py
View File

@ -6,7 +6,7 @@
import maxpython
test1 = maxpython.MaxScaleTest("mxs585.py")
test1 = maxpython.MaxScaleTest("mxs585.py1")
for i in range(0,100):
if i % 10 == 0:

4
maxscale-system-test/nodes.cpp
View File

@ -50,9 +50,7 @@ void Nodes::generate_ssh_cmd(char* cmd, int node, const char* ssh, bool sudo)
}
else
{
sprintf(cmd,
"%s",
ssh);
sprintf(cmd, "%s", ssh);
}
}
else

46
maxscale-system-test/testconnections.cpp
View File

@ -413,6 +413,24 @@ TestConnections::TestConnections(int argc, char* argv[])
}
}
if ((maxscale::restart_galera) && (galera))
{
galera->stop_nodes();
galera->start_replication();
}
if (maxscale::check_nodes)
{
if (repl && !repl->fix_replication())
{
exit(BROKEN_VM_FAUILT);
}
if (galera && !galera->fix_replication())
{
exit(BROKEN_VM_FAUILT);
}
}
if (repl && maxscale::required_repl_version.length())
{
int ver_repl_required = get_int_version(maxscale::required_repl_version);
@ -443,24 +461,6 @@ TestConnections::TestConnections(int argc, char* argv[])
}
}
if ((maxscale::restart_galera) && (galera))
{
galera->stop_nodes();
galera->start_replication();
}
if (maxscale::check_nodes)
{
if (repl && !repl->fix_replication())
{
exit(BROKEN_VM_FAUILT);
}
if (galera && !galera->fix_replication())
{
exit(BROKEN_VM_FAUILT);
}
}
if (maxscale_init)
{
init_maxscales();
@ -605,7 +605,8 @@ void TestConnections::read_mdbci_info()
target = readenv("target", "develop");
mdbci_config_name = readenv("mdbci_config_name", "local");
vm_path = std::string(mdbci_vm_path) + std::string(mdbci_config_name);
vm_path = std::string(mdbci_vm_path) + "/" + std::string(mdbci_config_name);
if (mdbci_config_name != NULL)
{
std::ifstream nc_file;
@ -2191,6 +2192,13 @@ int TestConnections::call_mdbci(const char * options)
tprintf("MDBCI failed to bring up virtual machines");
return 1;
}
std::string team_keys = readenv("team_keys", "~/.ssh/id_rsa.pub");
system((std::string("mdbci public_keys --key ") +
team_keys +
std::string(" ") +
std::string(mdbci_config_name)).c_str() );
read_env();
if (repl)
{

2
server/core/config.cc
View File

@ -2850,7 +2850,7 @@ void config_set_global_defaults()
gateway.peer_password[0] = '\0';
gateway.log_target = MXB_LOG_TARGET_DEFAULT;
gateway.qc_cache_properties.max_size = get_total_memory() * 0.4;
gateway.qc_cache_properties.max_size = get_total_memory() * 0.15;
if (gateway.qc_cache_properties.max_size == 0)
{

7
server/core/query_classifier.cc
View File

@ -176,10 +176,15 @@ public:
mxb_assert(peek(canonical_stmt) == nullptr);
mxb_assert(this_unit.classifier);
// 0xffffff is the maximum packet size, 4 is for packet header and 1 is for command byte. These are
// MariaDB/MySQL protocol specific values that are also defined in <maxscale/protocol/mysql.h> but
// should not be exposed to the core.
constexpr int64_t max_entry_size = 0xffffff - 5;
int64_t cache_max_size = this_unit.cache_max_size() / config_get_global_options()->n_threads;
int64_t size = canonical_stmt.size();
if (size <= cache_max_size)
if (size < max_entry_size && size <= cache_max_size)
{
int64_t required_space = (m_stats.size + size) - cache_max_size;

21
server/modules/monitor/csmon/csmon.cc
View File

@ -56,17 +56,22 @@ static std::string do_query(MXS_MONITORED_SERVER* srv, const char* query)
// Returns a numeric version similar to mysql_get_server_version
int get_cs_version(MXS_MONITORED_SERVER* srv)
{
// GCC 4.8 appears to have a broken std::regex_constants::ECMAScript that doesn't support brackets
std::regex re("Columnstore \\([0-9]*\\)[.]\\([0-9]*\\)[.]\\([0-9]*\\)-[0-9]*",
std::regex_constants::basic);
std::string result = do_query(srv, "SELECT @@version_comment");
std::smatch match;
int rval = 0;
std::string prefix = "Columnstore ";
std::string result = do_query(srv, "SELECT @@version_comment");
auto pos = result.find(prefix);
if (std::regex_match(result, match, re) && match.size() == 4)
if (pos != std::string::npos)
{
rval = atoi(match[1].str().c_str()) * 10000 + atoi(match[2].str().c_str()) * 100
+ atoi(match[3].str().c_str());
std::istringstream os(result.substr(pos + prefix.length()));
int major = 0, minor = 0, patch = 0;
char dot;
os >> major;
os >> dot;
os >> minor;
os >> dot;
os >> patch;
rval = major * 10000 + minor * 100 + patch;
}
return rval;

105
server/modules/monitor/mariadbmon/cluster_manipulation.cc
View File

@ -27,13 +27,11 @@ using maxscale::string_printf;
using maxbase::StopWatch;
using maxbase::Duration;
static const char RE_ENABLE_FMT[] = "To re-enable automatic %s, manually set '%s' to 'true' "
"for monitor '%s' via MaxAdmin or the REST API, or restart MaxScale.";
const char NO_SERVER[] = "Server '%s' is not monitored by '%s'.";
const char FAILOVER_OK[] = "Failover '%s' -> '%s' performed.";
const char FAILOVER_FAIL[] = "Failover '%s' -> '%s' failed.";
const char SWITCHOVER_OK[] = "Switchover '%s' -> '%s' performed.";
const char SWITCHOVER_FAIL[] = "Switchover %s -> %s failed";
const char SWITCHOVER_FAIL[] = "Switchover %s -> %s failed.";
/**
* Run a manual switchover, promoting a new master server and demoting the existing master.
@ -64,14 +62,8 @@ bool MariaDBMonitor::manual_switchover(SERVER* promotion_server, SERVER* demotio
{
string msg = string_printf(SWITCHOVER_FAIL,
op->demotion.target->name(), op->promotion.target->name());
bool failover_setting = config_get_bool(m_monitor->parameters, CN_AUTO_FAILOVER);
if (failover_setting)
{
disable_setting(CN_AUTO_FAILOVER);
msg += ", automatic failover has been disabled";
}
msg += ".";
PRINT_MXS_JSON_ERROR(error_out, "%s", msg.c_str());
delay_auto_cluster_ops();
}
}
else
@ -411,6 +403,10 @@ string MariaDBMonitor::generate_change_master_cmd(const string& master_host, int
change_cmd << "CHANGE MASTER TO MASTER_HOST = '" << master_host << "', ";
change_cmd << "MASTER_PORT = " << master_port << ", ";
change_cmd << "MASTER_USE_GTID = current_pos, ";
if (m_replication_ssl)
{
change_cmd << "MASTER_SSL = 1, ";
}
change_cmd << "MASTER_USER = '" << m_replication_user << "', ";
const char MASTER_PW[] = "MASTER_PASSWORD = '";
const char END[] = "';";
@ -621,6 +617,7 @@ uint32_t MariaDBMonitor::do_rejoin(const ServerArray& joinable_servers, json_t**
SERVER* master_server = m_master->m_server_base->server;
const char* master_name = master_server->name;
uint32_t servers_joined = 0;
bool rejoin_error = false;
if (!joinable_servers.empty())
{
for (MariaDBServer* joinable : joinable_servers)
@ -630,7 +627,8 @@ uint32_t MariaDBMonitor::do_rejoin(const ServerArray& joinable_servers, json_t**
// Rejoin doesn't have its own time limit setting. Use switchover time limit for now since
// the first phase of standalone rejoin is similar to switchover.
maxbase::Duration time_limit((double)m_switchover_timeout);
GeneralOpData general(m_replication_user, m_replication_password, output, time_limit);
GeneralOpData general(m_replication_user, m_replication_password, m_replication_ssl,
output, time_limit);
if (joinable->m_slave_status.empty())
{
@ -651,7 +649,8 @@ uint32_t MariaDBMonitor::do_rejoin(const ServerArray& joinable_servers, json_t**
else
{
PRINT_MXS_JSON_ERROR(output,
"Failed to prepare (demote) standalone server '%s' for rejoin.", name);
"Failed to prepare (demote) standalone server '%s' for rejoin.",
name);
}
}
else
@ -670,8 +669,17 @@ uint32_t MariaDBMonitor::do_rejoin(const ServerArray& joinable_servers, json_t**
servers_joined++;
m_cluster_modified = true;
}
else
{
rejoin_error = true;
}
}
}
if (rejoin_error)
{
delay_auto_cluster_ops();
}
return servers_joined;
}
@ -1410,7 +1418,8 @@ unique_ptr<MariaDBMonitor::FailoverParams> MariaDBMonitor::failover_prepare(Log
ServerOperation promotion(promotion_target, promoting_to_master,
m_handle_event_scheduler, m_promote_sql_file,
demotion_target->m_slave_status, demotion_target->m_enabled_events);
GeneralOpData general(m_replication_user, m_replication_password, error_out, time_limit);
GeneralOpData general(m_replication_user, m_replication_password, m_replication_ssl,
error_out, time_limit);
rval.reset(new FailoverParams(promotion, demotion_target, general));
}
}
@ -1468,7 +1477,7 @@ void MariaDBMonitor::handle_auto_failover()
else
{
MXS_ERROR(FAILOVER_FAIL, op->demotion_target->name(), op->promotion.target->name());
report_and_disable("failover", CN_AUTO_FAILOVER, &m_auto_failover);
delay_auto_cluster_ops();
}
}
else
@ -1529,26 +1538,10 @@ void MariaDBMonitor::check_cluster_operations_support()
{
const char PROBLEMS[] =
"The backend cluster does not support failover/switchover due to the following reason(s):\n"
"%s\n"
"Automatic failover/switchover has been disabled. They should only be enabled "
"after the above issues have been resolved.";
string p1 = string_printf(PROBLEMS, all_reasons.c_str());
string p2 = string_printf(RE_ENABLE_FMT, "failover", CN_AUTO_FAILOVER, m_monitor->name);
string p3 = string_printf(RE_ENABLE_FMT, "switchover", CN_SWITCHOVER_ON_LOW_DISK_SPACE,
m_monitor->name);
string total_msg = p1 + " " + p2 + " " + p3;
MXS_ERROR("%s", total_msg.c_str());
if (m_auto_failover)
{
m_auto_failover = false;
disable_setting(CN_AUTO_FAILOVER);
}
if (m_switchover_on_low_disk_space)
{
m_switchover_on_low_disk_space = false;
disable_setting(CN_SWITCHOVER_ON_LOW_DISK_SPACE);
}
"%s\n";
string msg = string_printf(PROBLEMS, all_reasons.c_str());
MXS_ERROR("%s", msg.c_str());
delay_auto_cluster_ops();
}
}
@ -1703,7 +1696,8 @@ MariaDBMonitor::switchover_prepare(SERVER* promotion_server, SERVER* demotion_se
ServerOperation demotion(demotion_target, master_swap, m_handle_event_scheduler,
m_demote_sql_file, promotion_target->m_slave_status,
EventNameSet() /* unused */);
GeneralOpData general(m_replication_user, m_replication_password, error_out, time_limit);
GeneralOpData general(m_replication_user, m_replication_password, m_replication_ssl,
error_out, time_limit);
rval.reset(new SwitchoverParams(promotion, demotion, general));
}
return rval;
@ -1712,6 +1706,7 @@ MariaDBMonitor::switchover_prepare(SERVER* promotion_server, SERVER* demotion_se
void MariaDBMonitor::enforce_read_only_on_slaves()
{
const char QUERY[] = "SET GLOBAL read_only=1;";
bool error = false;
for (MariaDBServer* server : m_servers)
{
if (server->is_slave() && !server->is_read_only()
@ -1725,9 +1720,15 @@ void MariaDBMonitor::enforce_read_only_on_slaves()
else
{
MXS_ERROR("Setting read_only on '%s' failed: '%s'.", server->name(), mysql_error(conn));
error = true;
}
}
}
if (error)
{
delay_auto_cluster_ops();
}
}
void MariaDBMonitor::handle_low_disk_space_master()
@ -1755,8 +1756,7 @@ void MariaDBMonitor::handle_low_disk_space_master()
else
{
MXS_ERROR(SWITCHOVER_FAIL, op->demotion.target->name(), op->promotion.target->name());
report_and_disable("switchover", CN_SWITCHOVER_ON_LOW_DISK_SPACE,
&m_switchover_on_low_disk_space);
delay_auto_cluster_ops();
}
}
else
@ -1791,19 +1791,6 @@ void MariaDBMonitor::handle_auto_rejoin()
// get_joinable_servers prints an error if master is unresponsive
}
void MariaDBMonitor::report_and_disable(const string& operation, const string& setting_name,
bool* setting_var)
{
string p1 = string_printf("Automatic %s failed, disabling automatic %s.",
operation.c_str(),
operation.c_str());
string p2 = string_printf(RE_ENABLE_FMT, operation.c_str(), setting_name.c_str(), m_monitor->name);
string error_msg = p1 + " " + p2;
MXS_ERROR("%s", error_msg.c_str());
*setting_var = false;
disable_setting(setting_name.c_str());
}
/**
* Check that the slaves to demotion target are using gtid replication and that the gtid domain of the
* cluster is defined. Only the slave connections to the demotion target are checked.
@ -1871,6 +1858,24 @@ ServerArray MariaDBMonitor::get_redirectables(const MariaDBServer* old_master,
return redirectable_slaves;
}
void MariaDBMonitor::delay_auto_cluster_ops()
{
if (m_auto_failover || m_auto_rejoin || m_enforce_read_only_slaves || m_switchover_on_low_disk_space)
{
const char DISABLING_AUTO_OPS[] = "Disabling automatic cluster operations for %i monitor ticks.";
MXS_NOTICE(DISABLING_AUTO_OPS, m_failcount);
}
// + 1 because the start of next tick subtracts 1.
cluster_operation_disable_timer = m_failcount + 1;
}
bool MariaDBMonitor::can_perform_cluster_ops()
{
return (!config_get_global_options()->passive && cluster_operation_disable_timer <= 0 &&
!m_cluster_modified);
}
MariaDBMonitor::SwitchoverParams::SwitchoverParams(const ServerOperation& promotion,
const ServerOperation& demotion,
const GeneralOpData& general)

46
server/modules/monitor/mariadbmon/mariadbmon.cc
View File

@ -26,8 +26,6 @@
#include <maxscale/routingworker.h>
#include <maxscale/secrets.h>
#include <maxscale/utils.hh>
// TODO: For monitor_add_parameters
#include "../../../core/internal/monitor.h"
using std::string;
using maxscale::string_printf;
@ -54,9 +52,7 @@ static const char CN_MASTER_FAILURE_TIMEOUT[] = "master_failure_timeout";
// Replication credentials parameters for failover/switchover/join
static const char CN_REPLICATION_USER[] = "replication_user";
static const char CN_REPLICATION_PASSWORD[] = "replication_password";
static const char DIAG_ERROR[] = "Internal error, could not print diagnostics. "
"Check log for more information.";
static const char CN_REPLICATION_MASTER_SSL[] = "replication_master_ssl";
MariaDBMonitor::MariaDBMonitor(MXS_MONITOR* monitor)
: maxscale::MonitorInstance(monitor)
@ -227,6 +223,7 @@ bool MariaDBMonitor::configure(const MXS_CONFIG_PARAMETER* params)
m_switchover_on_low_disk_space = config_get_bool(params, CN_SWITCHOVER_ON_LOW_DISK_SPACE);
m_maintenance_on_low_disk_space = config_get_bool(params, CN_MAINTENANCE_ON_LOW_DISK_SPACE);
m_handle_event_scheduler = config_get_bool(params, CN_HANDLE_EVENTS);
m_replication_ssl = config_get_bool(params, CN_REPLICATION_MASTER_SSL);
/* Reset all monitored state info. The server dependent values must be reset as servers could have been
* added, removed and modified. */
@ -441,6 +438,11 @@ void MariaDBMonitor::tick()
mon_srv->mon_prev_status = status;
}
if (cluster_operation_disable_timer > 0)
{
cluster_operation_disable_timer--;
}
// Query all servers for their status.
for (MariaDBServer* server : m_servers)
{
@ -458,7 +460,7 @@ void MariaDBMonitor::tick()
update_topology();
m_cluster_topology_changed = false;
// If cluster operations are enabled, check topology support and disable if needed.
if (m_auto_failover || m_switchover_on_low_disk_space)
if (m_auto_failover || m_switchover_on_low_disk_space || m_auto_rejoin)
{
check_cluster_operations_support();
}
@ -531,16 +533,16 @@ void MariaDBMonitor::process_state_changes()
}
}
if (!config_get_global_options()->passive)
if (can_perform_cluster_ops())
{
if (m_auto_failover && !m_cluster_modified)
if (m_auto_failover)
{
handle_auto_failover();
}
// Do not auto-join servers on this monitor loop if a failover (or any other cluster modification)
// has been performed, as server states have not been updated yet. It will happen next iteration.
if (m_auto_rejoin && !m_cluster_modified && cluster_can_be_joined())
if (m_auto_rejoin && cluster_can_be_joined() && can_perform_cluster_ops())
{
// Check if any servers should be autojoined to the cluster and try to join them.
handle_auto_rejoin();
@ -549,13 +551,13 @@ void MariaDBMonitor::process_state_changes()
/* Check if any slave servers have read-only off and turn it on if user so wishes. Again, do not
* perform this if cluster has been modified this loop since it may not be clear which server
* should be a slave. */
if (m_enforce_read_only_slaves && !m_cluster_modified)
if (m_enforce_read_only_slaves && can_perform_cluster_ops())
{
enforce_read_only_on_slaves();
}
/* Check if the master server is on low disk space and act on it. */
if (m_switchover_on_low_disk_space && !m_cluster_modified)
if (m_switchover_on_low_disk_space && can_perform_cluster_ops())
{
handle_low_disk_space_master();
}
@ -684,25 +686,6 @@ void MariaDBMonitor::assign_new_master(MariaDBServer* new_master)
m_warn_have_better_master = true;
}
/**
* Set a monitor config parameter to "false". The effect persists over stopMonitor/startMonitor but not
* MaxScale restart. Only use on boolean config settings.
*
* @param setting_name Setting to disable
*/
void MariaDBMonitor::disable_setting(const std::string& setting)
{
Worker* worker = static_cast<Worker*>(mxs_rworker_get(MXS_RWORKER_MAIN));
worker->execute([=]() {
MXS_CONFIG_PARAMETER p = {};
p.name = const_cast<char*>(setting.c_str());
p.value = const_cast<char*>("false");
monitor_add_parameters(m_monitor, &p);
},
EXECUTE_AUTO);
}
/**
* Check sql text file parameters. A parameter should either be empty or a valid file which can be opened.
*
@ -1067,6 +1050,9 @@ extern "C" MXS_MODULE* MXS_CREATE_MODULE()
{
CN_REPLICATION_PASSWORD, MXS_MODULE_PARAM_STRING
},
{
CN_REPLICATION_MASTER_SSL, MXS_MODULE_PARAM_BOOL, "false"
},
{
CN_VERIFY_MASTER_FAILURE, MXS_MODULE_PARAM_BOOL, "true"
},

12
server/modules/monitor/mariadbmon/mariadbmon.hh
View File

@ -177,6 +177,10 @@ private:
* Causes a topology rebuild on the current tick. */
bool m_cluster_modified = false; /* Has a cluster operation been performed this loop? Prevents
* other operations during this tick. */
/* Counter for temporary automatic cluster operation disabling. */
int cluster_operation_disable_timer = 0;
CycleMap m_cycles; /* Map from cycle number to cycle member servers */
CycleInfo m_master_cycle_status; /* Info about master server cycle from previous round */
@ -212,6 +216,7 @@ private:
// Cluster operations additional settings
std::string m_replication_user; /* Replication user for CHANGE MASTER TO-commands */
std::string m_replication_password; /* Replication password for CHANGE MASTER TO-commands */
bool m_replication_ssl = false; /* Set MASTER_SSL = 1 in CHANGE MASTER TO-commands */
bool m_handle_event_scheduler = true;/* Should failover/switchover enable/disable any scheduled
* events on the servers during promote/demote? */
uint32_t m_failover_timeout = 10; /* Time limit in seconds for failover */
@ -295,6 +300,9 @@ private:
bool switchover_perform(SwitchoverParams& operation);
bool failover_perform(FailoverParams& op);
void delay_auto_cluster_ops();
bool can_perform_cluster_ops();
// Methods used by failover/switchover/rejoin
MariaDBServer* select_promotion_target(MariaDBServer* current_master, OperationType op,
Log log_mode, json_t** error_out);
@ -318,8 +326,6 @@ private:
std::string generate_change_master_cmd(const std::string& master_host, int master_port);
void wait_cluster_stabilization(GeneralOpData& op, const ServerArray& slaves,
const MariaDBServer* new_master);
void report_and_disable(const std::string& operation, const std::string& setting_name,
bool* setting_var);
// Rejoin methods
bool cluster_can_be_joined();
@ -327,8 +333,6 @@ private:
bool server_is_rejoin_suspect(MariaDBServer* rejoin_cand, json_t** output);
uint32_t do_rejoin(const ServerArray& joinable_servers, json_t** output);
// Other methods
void disable_setting(const std::string& setting);
bool check_sql_files();
void enforce_read_only_on_slaves();
void log_master_changes();

4
server/modules/monitor/mariadbmon/mariadbserver.cc
View File

@ -2064,6 +2064,10 @@ string MariaDBServer::generate_change_master_cmd(GeneralOpData& op, const SlaveS
slave_conn.name.c_str(),
slave_conn.master_host.c_str(), slave_conn.master_port);
change_cmd += "MASTER_USE_GTID = current_pos, ";
if (op.replication_ssl)
{
change_cmd += "MASTER_SSL = 1, ";
}
change_cmd += string_printf("MASTER_USER = '%s', ", op.replication_user.c_str());
const char MASTER_PW[] = "MASTER_PASSWORD = '%s';";
#if defined (SS_DEBUG)

5
server/modules/monitor/mariadbmon/server_utils.cc
View File

@ -174,10 +174,11 @@ ServerOperation::ServerOperation(MariaDBServer* target, bool was_is_master, bool
{
}
GeneralOpData::GeneralOpData(const string& replication_user, const string& replication_password,
json_t** error, maxbase::Duration time_remaining)
GeneralOpData::GeneralOpData(const std::string& replication_user, const std::string& replication_password,
bool replication_ssl, json_t** error, maxbase::Duration time_remaining)
: replication_user(replication_user)
, replication_password(replication_password)
, replication_ssl(replication_ssl)
, error_out(error)
, time_remaining(time_remaining)
{

3
server/modules/monitor/mariadbmon/server_utils.hh
View File

@ -224,11 +224,12 @@ class GeneralOpData
public:
const std::string replication_user; // User for CHANGE MASTER TO ...
const std::string replication_password; // Password for CHANGE MASTER TO ...
const bool replication_ssl; // MASTER_SSL=1 in CHANGE MASTER TO ...
json_t** const error_out; // Json error output
maxbase::Duration time_remaining; // How much time remains to complete the operation
GeneralOpData(const std::string& replication_user, const std::string& replication_password,
json_t** error, maxbase::Duration time_remaining);
bool replication_ssl, json_t** error, maxbase::Duration time_remaining);
};
// Operation data which concerns a single server

11
server/modules/protocol/MySQL/rwbackend.cc
View File

@ -296,6 +296,17 @@ void RWBackend::process_packets(GWBUF* result)
auto end = std::next(it, len);
uint8_t cmd = *it;
// Ignore the tail end of a large packet large packet. Only resultsets can generate packets this large
// and we don't care what the contents are and thus it is safe to ignore it.
bool skip_next = m_skip_next;
m_skip_next = len == GW_MYSQL_MAX_PACKET_LEN;
if (skip_next)
{
it = end;
continue;
}
switch (m_reply_state)
{
case REPLY_STATE_START:

29
server/modules/routing/avrorouter/avro_client.cc
View File

@ -34,6 +34,7 @@
#include <maxscale/alloc.h>
#include <maxscale/buffer.hh>
#include <maxscale/utils.hh>
#include <maxscale/routingworker.hh>
std::pair<std::string, std::string> get_avrofile_and_gtid(std::string file);
@ -239,22 +240,14 @@ bool file_in_dir(const char* dir, const char* file)
}
/**
* @brief The client callback for sending data
*
* @param dcb Client DCB
* @param reason Why the callback was called
* @param userdata Data provided when the callback was added
* @return Always 0
* Queue the client callback for execution
*/
int avro_client_callback(DCB* dcb, DCB_REASON reason, void* userdata)
void AvroSession::queue_client_callback()
{
if (reason == DCB_REASON_DRAINED)
{
AvroSession* client = static_cast<AvroSession*>(userdata);
client->client_callback();
}
return 0;
auto worker = mxs::RoutingWorker::get(mxs::RoutingWorker::MAIN);
worker->execute([this]() {
client_callback();
}, mxs::RoutingWorker::EXECUTE_QUEUED);
}
/**
@ -338,11 +331,7 @@ void AvroSession::process_command(GWBUF* queue)
if (file_in_dir(router->avrodir.c_str(), avro_binfile.c_str()))
{
/* set callback routine for data sending */
dcb_add_callback(dcb, DCB_REASON_DRAINED, avro_client_callback, this);
/* Add fake event that will call the avro_client_callback() routine */
poll_fake_write_event(dcb);
queue_client_callback();
}
else
{
@ -734,7 +723,7 @@ void AvroSession::client_callback()
if (next_file || read_more)
{
poll_fake_write_event(dcb);
queue_client_callback();
}
}

7
server/modules/routing/avrorouter/avrorouter.hh
View File

@ -170,11 +170,6 @@ public:
*/
int routeQuery(GWBUF* buffer);
/**
* Handler for the EPOLLOUT event
*/
void client_callback();
private:
AvroSession(Avro* instance, MXS_SESSION* session);
@ -187,6 +182,8 @@ private:
bool seek_to_gtid();
bool stream_data();
void rotate_avro_file(std::string fullname);
void client_callback();
void queue_client_callback();
};
void read_table_info(uint8_t* ptr,