Merge branch '2.3' into 2.4

Documentation/Filters/CCRFilter.md

@@ -8,9 +8,10 @@ The Consistent Critical Read (CCR) filter allows consistent critical reads to be
 done through MaxScale while still allowing scaleout of non-critical reads.
 
 When the filter detects a statement that would modify the database, it attaches
-a routing hint to all following statements. This routing hint guides the routing
+a routing hint to all following statements done by that connection. This routing
-module to route the statement to the master server where data is guaranteed to
+hint guides the routing module to route the statement to the master server where
-be in an up-to-date state.
+data is guaranteed to be in an up-to-date state. Writes from one session do not,
+by default, propagate to other sessions.
 
 ### Controlling the Filter with SQL Comments
 

Documentation/Monitors/MariaDB-Monitor.md

@@ -1,46 +1,6 @@
 # MariaDB Monitor
 
-Table of Contents
+[TOC]
-=================
-
-  * [Overview](#overview)
-  * [Master selection](#master-selection)
-  * [Configuration](#configuration)
-  * [Common Monitor Parameters](#common-monitor-parameters)
-  * [MariaDB Monitor optional parameters](#mariadb-monitor-optional-parameters)
-     * [assume_unique_hostnames](#assume_unique_hostnames)
-     * [detect_replication_lag](#detect_replication_lag)
-     * [detect_stale_master](#detect_stale_master)
-     * [detect_stale_slave](#detect_stale_slave)
-     * [mysql51_replication](#mysql51_replication)
-     * [multimaster](#multimaster)
-     * [ignore_external_masters](#ignore_external_masters)
-     * [detect_standalone_master](#detect_standalone_master)
-     * [failcount](#failcount)
-     * [allow_cluster_recovery](#allow_cluster_recovery)
-     * [enforce_read_only_slaves](#enforce_read_only_slaves)
-     * [maintenance_on_low_disk_space](#maintenance_on_low_disk_space)
-  * [Cluster manipulation operations](#cluster-manipulation-operations)
-     * [Operation details](#operation-details)
-     * [Manual activation](#manual-activation)
-     * [Automatic activation](#automatic-activation)
-     * [Limitations and requirements](#limitations-and-requirements)
-     * [External master support](#external-master-support)
-     * [Configuration parameters](#configuration-parameters)
-        * [auto_failover](#auto_failover)
-        * [auto_rejoin](#auto_rejoin)
-        * [switchover_on_low_disk_space](#switchover_on_low_disk_space)
-        * [enforce_simple_topology](#enforce_simple_topology)
-        * [replication_user and replication_password](#replication_user-and-replication_password)
-        * [failover_timeout and switchover_timeout](#failover_timeout-and-switchover_timeout)
-        * [verify_master_failure and master_failure_timeout](#verify_master_failure-and-master_failure_timeout)
-        * [servers_no_promotion](#servers_no_promotion)
-        * [promotion_sql_file and demotion_sql_file](#promotion_sql_file-and-demotion_sql_file)
-        * [handle_events](#handle_events)
-  * [Troubleshooting](#troubleshooting)
-     * [Failover/switchover fails](#failoverswitchover-fails)
-     * [Slave detection shows external masters](#slave-detection-shows-external-masters)
-  * [Using the MariaDB Monitor With Binlogrouter](#using-the-mariadb-monitor-with-binlogrouter)
 
 ## Overview
 
@@ -261,6 +221,14 @@ multiple valid master servers in the cluster.
 
 The default value is 5 failures.
 
+The worst-case delay between the master failure and the start of the failover
+can be calculated by summing up the timeout values and `monitor_interval` and
+multiplying that by `failcount`:
+
+```
+(monitor_interval + backend_connect_timeout) * failcount
+```
+
 ### `allow_cluster_recovery`
 
 Deprecated and unused as of MaxScale 2.3. Can be defined but is ignored.

Documentation/Monitors/Monitor-Common.md

@@ -252,6 +252,13 @@ the message will be logged on the notice level. Whitespace before, after or
 between the keyword and the colon is ignored and the matching is
 case-insensitive.
 
+Currently, the script must not execute any of the following MaxCtrl and/or
+MaxAdmin calls as they cause a deadlock:
+
+* `alter monitor` to the monitor executing the script
+* `stop monitor` to the monitor executing the script
+* `call command` to a MariaDB-Monitor that is executing the script
+
 ### `script_timeout`
 
 The timeout for the executed script. The interval is specified as documented

Documentation/Reference/MaxAdmin.md

@@ -4,24 +4,7 @@
 
 # The Maxscale Administrative & Monitoring Client Application
 
- - [Overview](#overview)
+[TOC]
- - [Configuring MariaDB MaxScale for MaxAdmin](#configuring-mariadb-maxscale-for-maxadmin)
- - [Running MaxAdmin](#running-maxadmin)
- - [Working With Administration Interface Users](#working-with-administration-interface-users)
- - [Getting Help](#getting-help)
- - [Working with Services](#working-with-services)
- - [Working with Servers](#working-with-servers)
- - [Working with Sessions](#working-with-sessions)
- - [Descriptor Control Blocks](#descriptor-control-blocks)
- - [Working with Filters](#working-with-filters)
- - [Working with Monitors](#working-with-monitors)
- - [MariaDB MaxScale Status Commands](#maxscale-status-commands)
- - [Administration Commands](#administration-commands)
- - [Runtime Configuration Changes](#runtime-configuration-changes)
-   - [Servers](#servers)
-   - [Listeners](#listeners)
-   - [Monitors](#monitors)
- - [Tuning MariaDB MaxScale](#tuning-mariadb-maxscale)
 
 # Overview
 

include/maxscale/monitor.hh

@@ -139,7 +139,8 @@ public:
         NEWCONN_OK,    /* No existing connection or no ping reply. New connection created
                         * successfully. */
         REFUSED,       /* No existing connection or no ping reply. Server refused new connection. */
-        TIMEOUT        /* No existing connection or no ping reply. Timeout on new connection. */
+        TIMEOUT,       /* No existing connection or no ping reply. Timeout on new connection. */
+        ACCESS_DENIED  /* Server refused new connection due to authentication failure */
     };
 
     /**

server/core/monitor.cc

@@ -110,7 +110,7 @@ public:
         if (iter != m_server_owners.end())
         {
             // Server is already claimed by a monitor.
-            *existing_owner = iter->second;
+            * existing_owner = iter->second;
         }
         else
         {
@@ -718,25 +718,20 @@ bool Monitor::test_permissions(const string& query)
 
     for (MonitorServer* mondb : m_servers)
     {
-        if (!connection_is_ok(mondb->ping_or_connect(m_settings.conn_settings)))
+        auto result = mondb->ping_or_connect(m_settings.conn_settings);
+
+        if (!connection_is_ok(result))
         {
             MXS_ERROR("[%s] Failed to connect to server '%s' ([%s]:%d) when"
-                      " checking monitor user credentials and permissions: %s",
+                      " checking monitor user credentials and permissions.",
                       name(),
                       mondb->server->name(),
                       mondb->server->address,
-                      mondb->server->port,
+                      mondb->server->port);
-                      mysql_error(mondb->con));
-            switch (mysql_errno(mondb->con))
-            {
-            case ER_ACCESS_DENIED_ERROR:
-            case ER_DBACCESS_DENIED_ERROR:
-            case ER_ACCESS_DENIED_NO_PASSWORD_ERROR:
-                break;
 
-            default:
+            if (result != ConnectResult::ACCESS_DENIED)
+            {
                 rval = true;
-                break;
             }
         }
         else if (mxs_mysql_query(mondb->con, query.c_str()) != 0)
@@ -1180,8 +1175,14 @@ Monitor::ping_or_connect_to_db(const MonitorServer::ConnectionSettings& sett, SE
             conn_result = ConnectResult::TIMEOUT;
         }
 
+        auto err = mysql_errno(pConn);
         mysql_close(pConn);
         pConn = nullptr;
+
+        if (err == ER_ACCESS_DENIED_ERROR || err == ER_ACCESS_DENIED_NO_PASSWORD_ERROR)
+        {
+            conn_result = ConnectResult::ACCESS_DENIED;
+        }
     }
 
     MXS_FREE(dpwd);
@@ -2104,7 +2105,7 @@ void MonitorWorkerSimple::tick()
                  */
                 pMs->clear_pending_status(MonitorServer::SERVER_DOWN_CLEAR_BITS);
 
-                if (mysql_errno(pMs->con) == ER_ACCESS_DENIED_ERROR)
+                if (rval == ConnectResult::ACCESS_DENIED)
                 {
                     pMs->set_pending_status(SERVER_AUTH_ERROR);
                 }

server/modules/monitor/mariadbmon/mariadbmon.cc

@@ -95,7 +95,7 @@ void MariaDBMonitor::reset_server_info()
         m_servers.push_back(new MariaDBServer(mon_server, m_servers.size(), m_settings.shared));
     }
 
-    m_resolver = DNSResolver(); // Erases result cache.
+    m_resolver = DNSResolver();     // Erases result cache.
 }
 
 void MariaDBMonitor::reset_node_index_info()
@@ -144,7 +144,7 @@ MariaDBServer* MariaDBMonitor::get_server(const EndPoint& search_ep)
                     }
                 }
             }
-            breakout:;
+breakout:   ;
         }
     }
     return found;
@@ -251,7 +251,8 @@ bool MariaDBMonitor::configure(const MXS_CONFIG_PARAMETER* params)
     m_settings.enforce_read_only_slaves = params->get_bool(CN_ENFORCE_READONLY);
     m_settings.enforce_simple_topology = params->get_bool(CN_ENFORCE_SIMPLE_TOPOLOGY);
     m_settings.verify_master_failure = params->get_bool(CN_VERIFY_MASTER_FAILURE);
-    m_settings.master_failure_timeout = params->get_duration<std::chrono::seconds>(CN_MASTER_FAILURE_TIMEOUT).count();
+    m_settings.master_failure_timeout =
+        params->get_duration<std::chrono::seconds>(CN_MASTER_FAILURE_TIMEOUT).count();
     m_settings.shared.promotion_sql_file = params->get_string(CN_PROMOTION_SQL_FILE);
     m_settings.shared.demotion_sql_file = params->get_string(CN_DEMOTION_SQL_FILE);
     m_settings.switchover_on_low_disk_space = params->get_bool(CN_SWITCHOVER_ON_LOW_DISK_SPACE);
@@ -294,13 +295,13 @@ bool MariaDBMonitor::configure(const MXS_CONFIG_PARAMETER* params)
         // This is a "mega-setting" which turns on several other features regardless of their individual
         // settings.
         auto warn_and_enable = [](bool* setting, const char* setting_name) {
-            const char setting_activated[] = "%s enables %s, overriding any existing setting or default.";
+                const char setting_activated[] = "%s enables %s, overriding any existing setting or default.";
-            if (*setting == false)
+                if (*setting == false)
-            {
+                {
-                *setting = true;
+                    *setting = true;
-                MXB_WARNING(setting_activated, CN_ENFORCE_SIMPLE_TOPOLOGY, setting_name);
+                    MXB_WARNING(setting_activated, CN_ENFORCE_SIMPLE_TOPOLOGY, setting_name);
-            }
+                }
-        };
+            };
 
         warn_and_enable(&m_settings.assume_unique_hostnames, CN_ASSUME_UNIQUE_HOSTNAMES);
         warn_and_enable(&m_settings.auto_failover, CN_AUTO_FAILOVER);
@@ -346,8 +347,8 @@ string MariaDBMonitor::diagnostics_to_string() const
     rval.reserve(1000);     // Enough for basic output.
 
     auto bool_to_zstr = [](bool val) -> const char* {
-        return val ? "Enabled" : "Disabled";
+            return val ? "Enabled" : "Disabled";
-    };
+        };
     rval += string_printf("Automatic failover:      %s\n", bool_to_zstr(m_settings.auto_failover));
     rval += string_printf("Failcount:               %i\n", m_settings.failcount);
     rval += string_printf("Failover timeout:        %u\n", m_settings.failover_timeout);
@@ -982,33 +983,33 @@ extern "C" MXS_MODULE* MXS_CREATE_MODULE()
         NULL,                                       /* Thread finish. */
         {
             {
-                "detect_replication_lag",            MXS_MODULE_PARAM_BOOL,      "false",
+                "detect_replication_lag",            MXS_MODULE_PARAM_BOOL,                          "false",
                 MXS_MODULE_OPT_DEPRECATED
             },
             {
-                "detect_stale_master",               MXS_MODULE_PARAM_BOOL,      "true"
+                "detect_stale_master",               MXS_MODULE_PARAM_BOOL,                          "true"
             },
             {
-                "detect_stale_slave",                MXS_MODULE_PARAM_BOOL,      "true"
+                "detect_stale_slave",                MXS_MODULE_PARAM_BOOL,                          "true"
             },
             {
-                CN_DETECT_STANDALONE_MASTER,         MXS_MODULE_PARAM_BOOL,      "true"
+                CN_DETECT_STANDALONE_MASTER,         MXS_MODULE_PARAM_BOOL,                          "true"
             },
             {
-                CN_FAILCOUNT,                        MXS_MODULE_PARAM_COUNT,     "5"
+                CN_FAILCOUNT,                        MXS_MODULE_PARAM_COUNT,                         "5"
             },
             {
-                "ignore_external_masters",           MXS_MODULE_PARAM_BOOL,      "false"
+                "ignore_external_masters",           MXS_MODULE_PARAM_BOOL,                          "false"
             },
             {
-                CN_AUTO_FAILOVER,                    MXS_MODULE_PARAM_BOOL,      "false"
+                CN_AUTO_FAILOVER,                    MXS_MODULE_PARAM_BOOL,                          "false"
             },
             {
-                CN_FAILOVER_TIMEOUT,                 MXS_MODULE_PARAM_DURATION,  "90s",
+                CN_FAILOVER_TIMEOUT,                 MXS_MODULE_PARAM_DURATION,                      "90s",
                 MXS_MODULE_OPT_DURATION_S
             },
             {
-                CN_SWITCHOVER_TIMEOUT,               MXS_MODULE_PARAM_DURATION,  "90s",
+                CN_SWITCHOVER_TIMEOUT,               MXS_MODULE_PARAM_DURATION,                      "90s",
                 MXS_MODULE_OPT_DURATION_S
             },
             {
@@ -1018,20 +1019,20 @@ extern "C" MXS_MODULE* MXS_CREATE_MODULE()
                 CN_REPLICATION_PASSWORD,             MXS_MODULE_PARAM_PASSWORD
             },
             {
-                CN_REPLICATION_MASTER_SSL,           MXS_MODULE_PARAM_BOOL,      "false"
+                CN_REPLICATION_MASTER_SSL,           MXS_MODULE_PARAM_BOOL,                          "false"
             },
             {
-                CN_VERIFY_MASTER_FAILURE,            MXS_MODULE_PARAM_BOOL,      "true"
+                CN_VERIFY_MASTER_FAILURE,            MXS_MODULE_PARAM_BOOL,                          "true"
             },
             {
-                CN_MASTER_FAILURE_TIMEOUT,           MXS_MODULE_PARAM_DURATION,  "10s",
+                CN_MASTER_FAILURE_TIMEOUT,           MXS_MODULE_PARAM_DURATION,                      "10s",
                 MXS_MODULE_OPT_DURATION_S
             },
             {
-                CN_AUTO_REJOIN,                      MXS_MODULE_PARAM_BOOL,      "false"
+                CN_AUTO_REJOIN,                      MXS_MODULE_PARAM_BOOL,                          "false"
             },
             {
-                CN_ENFORCE_READONLY,                 MXS_MODULE_PARAM_BOOL,      "false"
+                CN_ENFORCE_READONLY,                 MXS_MODULE_PARAM_BOOL,                          "false"
             },
             {
                 CN_NO_PROMOTE_SERVERS,               MXS_MODULE_PARAM_SERVERLIST
@@ -1043,19 +1044,19 @@ extern "C" MXS_MODULE* MXS_CREATE_MODULE()
                 CN_DEMOTION_SQL_FILE,                MXS_MODULE_PARAM_PATH
             },
             {
-                CN_SWITCHOVER_ON_LOW_DISK_SPACE,     MXS_MODULE_PARAM_BOOL,      "false"
+                CN_SWITCHOVER_ON_LOW_DISK_SPACE,     MXS_MODULE_PARAM_BOOL,                          "false"
             },
             {
-                CN_MAINTENANCE_ON_LOW_DISK_SPACE,    MXS_MODULE_PARAM_BOOL,      "true"
+                CN_MAINTENANCE_ON_LOW_DISK_SPACE,    MXS_MODULE_PARAM_BOOL,                          "true"
             },
             {
-                CN_HANDLE_EVENTS,                    MXS_MODULE_PARAM_BOOL,      "true"
+                CN_HANDLE_EVENTS,                    MXS_MODULE_PARAM_BOOL,                          "true"
             },
             {
-                CN_ASSUME_UNIQUE_HOSTNAMES,          MXS_MODULE_PARAM_BOOL,      "true"
+                CN_ASSUME_UNIQUE_HOSTNAMES,          MXS_MODULE_PARAM_BOOL,                          "true"
             },
             {
-                CN_ENFORCE_SIMPLE_TOPOLOGY,          MXS_MODULE_PARAM_BOOL,      "false"
+                CN_ENFORCE_SIMPLE_TOPOLOGY,          MXS_MODULE_PARAM_BOOL,                          "false"
             },
             {MXS_END_MODULE_PARAMS}
         }

server/modules/monitor/mariadbmon/mariadbserver.cc

@@ -1212,28 +1212,27 @@ const SlaveStatus* MariaDBServer::slave_connection_status_host_port(const MariaD
 bool MariaDBServer::enable_events(BinlogMode binlog_mode, const EventNameSet& event_names, json_t** error_out)
 {
     EventStatusMapper mapper = [&event_names](const EventInfo& event) {
-        string rval;
+            string rval;
-        if (event_names.count(event.name) > 0
+            if (event_names.count(event.name) > 0
-            && (event.status == "SLAVESIDE_DISABLED" || event.status == "DISABLED"))
+                && (event.status == "SLAVESIDE_DISABLED" || event.status == "DISABLED"))
-        {
+            {
-            rval = "ENABLE";
+                rval = "ENABLE";
-        }
+            }
-        return rval;
+            return rval;
-    };
+        };
     return alter_events(binlog_mode, mapper, error_out);
-
 }
 
 bool MariaDBServer::disable_events(BinlogMode binlog_mode, json_t** error_out)
 {
     EventStatusMapper mapper = [](const EventInfo& event) {
-        string rval;
+            string rval;
-        if (event.status == "ENABLED")
+            if (event.status == "ENABLED")
-        {
+            {
-            rval = "DISABLE ON SLAVE";
+                rval = "DISABLE ON SLAVE";
-        }
+            }
-        return rval;
+            return rval;
-    };
+        };
     return alter_events(binlog_mode, mapper, error_out);
 }
 
@@ -1267,17 +1266,17 @@ MariaDBServer::alter_events(BinlogMode binlog_mode, const EventStatusMapper& map
     int events_altered = 0;
     // Helper function which alters an event depending on the mapper-function.
     EventManipulator alterer = [this, &target_events, &events_altered, &mapper](const EventInfo& event,
-                                                                               json_t** error_out) {
+                                                                                json_t** error_out) {
-        string target_state = mapper(event);
+            string target_state = mapper(event);
-        if (!target_state.empty())
+            if (!target_state.empty())
-        {
-            target_events++;
-            if (alter_event(event, target_state, error_out))
             {
-                events_altered++;
+                target_events++;
+                if (alter_event(event, target_state, error_out))
+                {
+                    events_altered++;
+                }
             }
-        }
+        };
-    };
 
     bool rval = false;
     // TODO: For better error handling, this function should try to re-enable any disabled events if a later
@@ -1907,9 +1906,9 @@ bool MariaDBServer::merge_slave_conns(GeneralOpData& op, const SlaveStatusArray&
                         accepted = false;
                         const auto& endpoint = slave_conn.settings.master_endpoint;
                         ignore_reason = string_printf(
-                                "its Master_Host (%s) and Master_Port (%i) match an existing "
+                            "its Master_Host (%s) and Master_Port (%i) match an existing "
-                                "slave connection on %s.",
+                            "slave connection on %s.",
-                                endpoint.host().c_str(), endpoint.port(), name());
+                            endpoint.host().c_str(), endpoint.port(), name());
                     }
                 }
             }
@@ -2234,8 +2233,8 @@ void MariaDBServer::update_server(bool time_to_update_disk_space,
         /* The current server is not running. Clear some of the bits. User-set bits and some long-term bits
          * can stay. */
         server->clear_status(MonitorServer::SERVER_DOWN_CLEAR_BITS);
-        auto conn_errno = mysql_errno(conn);
+
-        if (conn_errno == ER_ACCESS_DENIED_ERROR || conn_errno == ER_ACCESS_DENIED_NO_PASSWORD_ERROR)
+        if (conn_status == ConnectResult::ACCESS_DENIED)
         {
             server->set_status(SERVER_AUTH_ERROR);
         }
@@ -2262,13 +2261,13 @@ bool MariaDBServer::kick_out_super_users(GeneralOpData& op)
     auto error_out = op.error_out;
     // Only select unique rows...
     string get_ids_query = "SELECT DISTINCT * FROM ("
-                           // select conn id and username from live connections ...
+        // select conn id and username from live connections ...
                            "SELECT P.id,P.user FROM information_schema.PROCESSLIST as P "
-                           // match with user information ...
+        // match with user information ...
                            "INNER JOIN mysql.user as U ON (U.user = P.user) WHERE "
-                           // where the user has super-privileges, is not replicating ...
+        // where the user has super-privileges, is not replicating ...
                            "(U.Super_priv = 'Y' AND P.COMMAND != 'Binlog Dump' "
-                           // and is not the current user.
+        // and is not the current user.
                            "AND P.id != (SELECT CONNECTION_ID()))) as I;";
 
     string error_msg;