diff --git a/Documentation/Documentation-Contents.md b/Documentation/Documentation-Contents.md index c2e3e6154..3090c5205 100644 --- a/Documentation/Documentation-Contents.md +++ b/Documentation/Documentation-Contents.md @@ -41,6 +41,11 @@ - [Tee Filter](filters/Tee-Filter.md) - [Top N Filter](filters/Top-N-Filter.md) - [Firewall Filter](filters/Firewall-Filter.md) + - [RabbitMQ Filter](filters/RabbitMQ-Filter.md) + +## Utilities + + - [RabbitMQ Consumer Client](filters/RabbitMQ-Consumer-Client.md) ## Design Documents @@ -55,3 +60,4 @@ - [MaxScale 1.0 Release Notes](Release-Notes/MaxScale-1.0-Release-Notes.md) - [MaxScale 1.0.1 Release Notes](Release-Notes/MaxScale-1.0.1-Release-Notes.md) - [MaxScale 1.0.3 Release Notes](Release-Notes/MaxScale-1.0.3-Release-Notes.md) + diff --git a/Documentation/Getting-Started/Configuration-Guide.md b/Documentation/Getting-Started/Configuration-Guide.md index 8741a760b..943f8f0d6 100644 --- a/Documentation/Getting-Started/Configuration-Guide.md +++ b/Documentation/Getting-Started/Configuration-Guide.md @@ -1,236 +1,60 @@ MaxScale - Configuration & Usage Scenarios -Mark Riddoch - -Last Updated: 29th November 2014 - -# Contents - -[[TOC]] - -# Document History - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
DateChangeWho
21st July 2013Initial versionMark Riddoch
23rd July 2013Addition of default user and password for a monitor and discussion of monitor user requirements -New monitor documented for Galera clusters -Addition of example Galera cluster configurationMark Riddoch
13th November 2013state for Galera Monitor is "synced"Massimiliano Pinto
2nd December 2013Updated the description of the command line arguments to match the code updates. -Improved descriptions and general documentation. -Enhanced example configurationsMark Riddoch
6th February 2014Added “enable_root_user” as a service parameterMassimiliano Pinto
7th February 2014Addition of bind address information -Clarification of user configuration required for monitoring users and the user needed to fetch the user dataMark Riddoch
3rd March 2014MySQL authentication with hostnamesMassimiliano Pinto
3rd March 2014Addition of section that describes authentication requirements and the rules for creating user credentialsMark Riddoch
28th March 2014Unix socket supportMassimiliano Pinto
8th May 2014Added “version_string” parameter in serviceMassimiliano Pinto
29th May 2014Added troubleshooting sectionMassimiliano Pinto
2nd June 2014Correction of some typos, clarification of the meaning of session modification statements and the default user for the CLI. -Addition of debugcli configuration option for developer and user modes.Mark Riddoch
4th June 2014Addition of “monitor_interval” for monitorsMassimiliano Pinto
6th June 2014Addition of filters sectionsMark Riddoch
27th June 2014Addition of server weighting, the configuration for the maxadmin clientMark Riddoch
2nd July 2014Addition of new readwritesplit router options with description and examples.Vilho Raatikka
31st July 2014Addition of NDB monitor for MySQL ClusterMassimiliano Pinto
28th August 2014Addition of “detect_stale_master” option for MySQL monitorMassimiliano Pinto
26th September 2014Addition of ‘localhost_match_wildcard_host’ service optionMassimiliano Pinto
24th October 2014Addition of “disable_master_failback” option for Galera monitorMassimiliano Pinto
4th November 2014Addition of timeouts for all monitorsMassimiliano Pinto
11th November 2014Addition of missing top filterMark Riddoch
- - # Introduction The purpose of this document is to describe how to configure MaxScale and to discuss some possible usage scenarios for MaxScale. MaxScale is designed with flexibility in mind, and consists of an event processing core with various support functions and plugin modules that tailor the behaviour of the MaxScale itself. ## Terms - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
TermDescription
serviceA service represents a set of databases with a specific access mechanism that is offered to clients of MaxScale. The access mechanism defines the algorithm that MaxScale will use to direct particular requests to the individual databases.
serverA server represents an individual database server to which a client can be connected via MaxScale.
routerA router is a module within MaxScale that will route client requests to the various database servers which MaxScale provides a service interface to.
connection routingConnection routing is a method of handling requests in which MaxScale will accept connections from a client and route data on that connection to a single database using a single connection. Connection based routing will not examine individual requests on a connection and it will not move that connection once it is established.
statement routingStatement routing is a method of handling requests in which each request within a connection will be handled individually. Requests may be sent to one or more servers and connections may be dynamically added or removed from the session.
protocolA protocol is a module of software that is used to communicate with another software entity within the system. MaxScale supports the dynamic loading of protocol modules to allow for increased flexibility.
moduleA module is a separate code entity that may be loaded dynamically into MaxScale to increase the available functionality. Modules are implemented as run-time loadable shared objects.
monitorA monitor is a module that can be executed within MaxScale to monitor the state of a set of database. The use of an internal monitor is optional, monitoring may be performed externally to MaxScale.
listenerA listener is the network endpoint that is used to listen for connections to MaxScale from the client applications. A listener is associated to a single service, however a service may have many listeners.
connection failoverWhen a connection currently being used between MaxScale and the database server fails a replacement will be automatically created to another server by MaxScale without client intervention
backend databaseA term used to refer to a database that sits behind MaxScale and is accessed by applications via MaxScale.
filterA module that can be placed between the client and the MaxScale router module. All client data passes through the filter module and may be examined or modified by the filter modules. -Filters may be chained together to form processing pipelines.
+ Term | Description +-------------------|------------------ + service | A service represents a set of databases with a specific access mechanism that is offered to clients of MaxScale. The access mechanism defines the algorithm that MaxScale will use to direct particular requests to the individual databases. + server | A server represents an individual database server to which a client can be connected via MaxScale. + router | A router is a module within MaxScale that will route client requests to the various database servers which MaxScale provides a service interface to. +connection routing | Connection routing is a method of handling requests in which MaxScale will accept connections from a client and route data on that connection to a single database using a single connection. Connection based routing will not examine individual requests on a connection and it will not move that connection once it is established. +statement routing | Statement routing is a method of handling requests in which each request within a connection will be handled individually. Requests may be sent to one or more servers and connections may be dynamically added or removed from the session. + protocol | A protocol is a module of software that is used to communicate with another software entity within the system. MaxScale supports the dynamic loading of protocol modules to allow for increased flexibility. + module | A module is a separate code entity that may be loaded dynamically into MaxScale to increase the available functionality. Modules are implemented as run-time loadable shared objects. + monitor | A monitor is a module that can be executed within MaxScale to monitor the state of a set of database. The use of an internal monitor is optional, monitoring may be performed externally to MaxScale. + listener | A listener is the network endpoint that is used to listen for connections to MaxScale from the client applications. A listener is associated to a single service, however a service may have many listeners. +connection failover| When a connection currently being used between MaxScale and the database server fails a replacement will be automatically created to another server by MaxScale without client intervention + backend database | A term used to refer to a database that sits behind MaxScale and is accessed by applications via MaxScale. + filter | A module that can be placed between the client and the MaxScale router module. All client data passes through the filter module and may be examined or modified by the filter modules. Filters may be chained together to form processing pipelines. # Configuration The MaxScale configuration is read from a file which can be located in a number of placing, MaxScale will search for the configuration file in a number of locations. -1. If the environment variable MAXSCALE_HOME is set then MaxScale will look for a configuration file called MaxScale.cnf in the directory $MAXSCALE_HOME/etc +1. If the environment variable `MAXSCALE_HOME` is set then MaxScale will look for a configuration file called `MaxScale.cnf` in the directory `$MAXSCALE_HOME/etc`. -2. If MAXSCALE_HOME is not set or the configuration file is not in the location above MaxScale will look for a file in /etc/MaxScale.cnf +2. If `MAXSCALE_HOME` is not set or the configuration file is not in the location above MaxScale will look for a file in `/etc/MaxScale.cnf`. -Alternatively MaxScale can be started with the -c flag and the path of the MaxScale home directory tree. +Alternatively MaxScale can be started with the `-c` flag and the path of the MaxScale home directory tree. -An explicit path to a configuration file can be passed by using the -f option to MaxScale. +An explicit path to a configuration file can be passed by using the `-f` option to MaxScale. -The configuration file itself is based on the "ini" file format and consists of various sections that are used to build the configuration, these sections define services, servers, listeners, monitors and global settings. +The configuration file itself is based on the ".ini" file format and consists of various sections that are used to build the configuration, these sections define services, servers, listeners, monitors and global settings. ## Global Settings -The global settings, in a section named [MaxScale], allow various parameters that affect MaxScale as a whole to be tuned. Currently the only setting that is supported is the number of threads to use to handle the network traffic. MaxScale will also accept the section name of [gateway] for global settings. This is for backward compatibility with versions prior to the naming of MaxScale. +The global settings, in a section named `[MaxScale]`, allow various parameters that affect MaxScale as a whole to be tuned. Currently the only setting that is supported is the number of threads to use to handle the network traffic. MaxScale will also accept the section name of `[gateway]` for global settings. This is for backward compatibility with versions prior to the naming of MaxScale. -### Threads +### `threads` To control the number of threads that poll for network traffic set the parameter threads to a number. It is recommended that you start with a single thread and add more as you find the performance is not satisfactory. MaxScale is implemented to be very thread efficient, so a small number of threads is usually adequate to support reasonably heavy workloads. Adding more threads may not improve performance and can consume resources needlessly. +``` # Valid options are: - # threads= [MaxScale] - threads=1 +``` -It should be noted that additional threads will be created to execute other internal services within MaxScale, this setting is merely used to configure the number of threads that will be used to manage the user connections. +It should be noted that additional threads will be created to execute other internal services within MaxScale. This setting is used to configure the number of threads that will be used to manage the user connections. ## Service @@ -242,49 +66,57 @@ Several different services may be defined using the same set of backend servers. A service is identified by a service name, which is the name of the configuration file section and a type parameter of service +``` [Test Service] - type=service +``` In order for MaxScale to forward any requests it must have at least one service defined within the configuration file. The definition of a service alone is not enough to allow MaxScale to forward requests however, the service is merely present to link together the other configuration elements. -### Router +### `router` The router parameter of a service defines the name of the router module that will be used to implement the routing algorithm between the client of MaxScale and the backend databases. Additionally routers may also be passed a comma separated list of options that are used to control the behaviour of the routing algorithm. The two parameters that control the routing choice are router and router_options. The router options are specific to a particular router and are used to modify the behaviour of the router. The read connection router can be passed options of master, slave or synced, an example of configuring a service to use this router and limiting the choice of servers to those in slave state would be as follows. +``` router=readconnroute - router_options=slave +``` To change the router to connect on to servers in the master state as well as slave servers, the router options can be modified to include the master state. +``` router=readconnroute - router_options=master,slave +``` A more complete description of router options and what is available for a given router is included with the documentation of the router itself. -### Filters +### `filters` The filters option allow a set of filters to be defined for a service; requests from the client are passed through these filters before being sent to the router for dispatch to the backend server. The filters parameter takes one or more filter names, as defined within the filter definition section of the configuration file. Multiple filters are separated using the | character. +``` filters=counter | QLA +``` The requests pass through the filters from left to right in the order defined in the configuration parameter. -### Servers +### `servers` The servers parameter in a service definition provides a comma separated list of the backend servers that comprise the service. The server names are those used in the name section of a block with a type parameter of server (see below). +``` servers=server1,server2,server3 +``` -### User +### `user` The user parameter, along with the passwd parameter are used to define the credentials used to connect to the backend servers to extract the list of database users from the backend database that is used for the client authentication. +``` user=maxscale - passwd=Mhu87p2D +``` Authentication of incoming connections is performed by MaxScale itself rather than by the database server to which the client is connected. The client will authenticate itself with MaxScale, using the username, hostname and password information that MaxScale has extracted from the backend database servers. For a detailed discussion of how this impacts the authentication process please see the "Authentication" section below. @@ -296,140 +128,126 @@ In order for MaxScale to obtain all the data it must be given a username it can The account used must be able to select from the mysql.user table, the following is an example showing how to create this user. +``` MariaDB [mysql]> create user 'maxscale'@'maxscalehost' identified by 'Mhu87p2D'; - Query OK, 0 rows affected (0.01 sec) MariaDB [mysql]> grant SELECT on mysql.user to 'maxscale'@'maxscalehost'; - Query OK, 0 rows affected (0.00 sec) +``` -Additionally, GRANT SELECT on the mysql.db table and SHOW DATABASES privileges are required in order to load databases name and grants suitable for database name authorization. +Additionally, `GRANT SELECT` on the `mysql.db` table and `SHOW DATABASES` privileges are required in order to load databases name and grants suitable for database name authorization. +``` MariaDB [(none)]> GRANT SELECT ON mysql.db TO 'username'@'maxscalehost'; - Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> GRANT SHOW DATABASES ON *.* TO 'username'@'maxscalehost'; - Query OK, 0 rows affected (0.00 sec) +``` -### Passwd +### `passwd` -The passwd parameter provides the password information for the above user and may be either a plain text password or it may be an encrypted password. See the section on encrypting passwords for use in the MaxScale.cnf file. This user must be capable of connecting to the backend database and executing the SQL statement "SELECT user, host, password,Select_priv FROM mysql.user" +The passwd parameter provides the password information for the above user and may be either a plain text password or it may be an encrypted password. See the section on encrypting passwords for use in the MaxScale.cnf file. This user must be capable of connecting to the backend database and executing these SQL statements to load database names and grants from the backends: -and additionally these SQL statements loading database names and database grants. +* `SELECT user, host, password,Select_priv FROM mysql.user`. +* `SELECT user, host, db FROM mysql.db` +* `SELECT * FROM INFORMATION_SCHEMA.SCHEMATA` +* `SELECT GRANTEE,PRIVILEGE_TYPE FROM INFORMATION_SCHEMA.USER_PRIVILEGES` -* "SELECT user, host, db FROM mysql.db" - -* "SELECT * FROM INFORMATION_SCHEMA.SCHEMATA" - -* "SELECT GRANTEE,PRIVILEGE_TYPE FROM INFORMATION_SCHEMA.USER_PRIVILEGES" - -**enable_root_user** +### `enable_root_user` This parameter controls the ability of the root user to connect to MaxScale and hence onwards to the backend servers via MaxScale. -The default value is 0, disabling the ability of the root user to connect to MaxScale. +The default value is `0`, disabling the ability of the root user to connect to MaxScale. Example for enabling root user: +``` enable_root_user=1 +``` -Values of "on" or “true” may also be given to enable the root user and “off” or “false” may be given to disable the use of the root user. +Values of `on` or `true` may also be given to enable the root user and `off` or `false` may be given to disable the use of the root user. +``` enable_root_user=true +``` -**localhost_match_wildcard_host** +### `localhost_match_wildcard_host` -This parameter enables matching of ‘127.0.0.1’ (localhost) against ‘%’ wildcard host for MySQL protocol authentication. The default value is 0, therefore in order to authenticate a connection from the same machine as the one which MaxScale is running on an explicit user@lcoalhost entry will be required in the MySQL user table. +This parameter enables matching of "127.0.0.1" (localhost) against "%" wildcard host for MySQL protocol authentication. The default value is `0`, so in order to authenticate a connection from the same machine as the one on which MaxScale is running, an explicit user@lcoalhost entry will be required in the MySQL user table. -**version_string** +### `version_string` This parameter sets a custom version string that is sent in the MySQL Handshake from MaxScale to clients. Example: +``` version_string=5.5.37-MariaDB-RWsplit +``` If not set, the default value is the server version of the embedded MySQL/MariaDB library. Example: 5.5.35-MariaDB -### weightby +### `weightby` The weightby parameter is used in conjunction with server parameters in order to control the load balancing applied in the router in use by the service. This allows varying weights to be applied to each server to create a non-uniform distribution of the load amongst the servers. An example of this might be to define a parameter for each server that represents the amount of resource available on the server, we could call this serversize. Every server should then have a serversize parameter set for the server. +``` serversize=10 +``` The service would then have the parameter weightby set. If there are 4 servers defined in the service, serverA, serverB, serverC and serverD, with the serversize set as shown in the table below, the connections would balanced using the percentages in this table. - - - - - - - - - - - - - - - - - - - - - - - - - - -
Serverserversize% connections
serverA1018%
serverB1527%
serverC1018%
serverD2036%
+ Server |serversize|% connections +---------|----------|------------- +serverA | 10 | 18% +serverB | 15 | 27% +serverC | 10 | 18% +serverD | 20 | 36% ## Server Server sections are used to define the backend database servers that can be formed into a service. A server may be a member of one or more services within MaxScale. Servers are identified by a server name which is the section name in the configuration file. Servers have a type parameter of server, plus address port and protocol parameters. +``` [server1] - type=server - address=127.0.0.1 - port=3000 - protocol=MySQLBackend +``` -### Address +### `address` The IP address or hostname of the machine running the database server that is being defined. MaxScale will use this address to connect to the backend database server. -### Port +### `port` The port on which the database listens for incoming connections. MaxScale will use this port to connect to the database server. -### Protocol +### `protocol` The name for the protocol module to use to connect MaxScale to the database. Currently only one backend protocol is supported, the MySQLBackend module. -### Monitoruser +### `monitoruser` The monitor has a username and password that is used to connect to all servers for monitoring purposes, this may be overridden by supplying a monitoruser statement for each individual server +``` monitoruser=mymonitoruser +``` -### MonitorPw +### `monitorpw` The monitor has a username and password that is used to connect to all servers for monitoring purposes, this may be overridden by supplying a monpasswd statement for the individual servers +``` monitorpw=mymonitorpasswd +``` The monpasswd parameter may be either a plain text password or it may be an encrypted password. See the section on encrypting passwords for use in the MaxScale.cnf file. @@ -437,39 +255,35 @@ The monpasswd parameter may be either a plain text password or it may be an encr The listener defines a port and protocol pair that is used to listen for connections to a service. A service may have multiple listeners associated with it, either to support multiple protocols or multiple ports. As with other elements of the configuration the section name is the listener name and it can be selected freely. A type parameter is used to identify the section as a listener definition. Address is optional and it allows the user to limit connections to certain interface only. Socket is also optional and used for Unix socket connections. +``` [] - type=listener - -service=] - -protocol=[MySQLClient|HTTPD] - -address=[IP|hostname] - -port= - +service=] +protocol=[MySQLClient|HTTPD] +address=[IP|hostname] +port= socket= +``` -### Service +### `service` The service to which the listener is associated. This is the name of a service that is defined elsewhere in the configuration file. -### Protocol +### `protocol` The name of the protocol module that is used for the communication between the client and MaxScale itself. -### Address +### `address` -The address option sets the address that will be used to bind the listening socket. The address may be specified as an IP address in ‘dot notation’ or as a hostname. If the address option is not included in the listener definition the listener will bind to all network interfaces. +The address option sets the address that will be used to bind the listening socket. The address may be specified as an IP address in 'dot notation' or as a hostname. If the address option is not included in the listener definition the listener will bind to all network interfaces. -### Port +### `port` The port to use to listen for incoming connections to MaxScale from the clients. If the port is omitted from the configuration a default port for the protocol will be used. -**Socket** +### `socket` -The socket option may be included in a listener definition, this configures the listener to use Unix domain sockets to listen for incoming connections. The parameter value given is the name of the socket to use. +The `socket` option may be included in a listener definition, this configures the listener to use Unix domain sockets to listen for incoming connections. The parameter value given is the name of the socket to use. If a socket option and an address option is given then the listener will listen on both the specific IP address and the Unix socket. @@ -477,57 +291,49 @@ If a socket option and an address option is given then the listener will listen Filters provide a means to manipulate or process requests as they pass through MaxScale between the client side protocol and the query router. A filter should be defined in a section with a type of filter. +``` [QLA] - type=filter - module=qlafilter - options=/tmp/QueryLog +``` The section name may then be used in one or more services by using the filters= parameter in the service section. In order to use the above filter for a service called "QLA Service", an entry of the following form would exist for that service. +``` [QLA Service] - type=service - router=readconnroute - router_options=slave - servers=server1,server2,server3,server4 - user=massi - passwd=6628C50E07CCE1F0392EDEEB9D1203F3 - filters=QLA +``` ![image alt text](images/image_10.png) See the Services section for more details on how to configure the various options of a service. Note that some filters require parsing of the statement which makes them compatible with statement-based routers only, such as Read/Write Split router. -### Module +### `module` The module parameter defines the name of the loadable module that implements the filter. -### Options +### `options` The options parameter is used to pass options to the filter to control the actions the filter will perform. The values that can be passed differ between filter implementation, the inclusion of an options parameter is optional. ### Other Parameters -Any other parameters present in the filters section will be passed to the filter to be interpreted by the filter. An example of this is the regexfilter that requires the two parameters match and replace +Any other parameters present in the filters section will be passed to the filter to be interpreted by the filter. An example of this is the regexfilter that requires the two parameters `match` and `replace`: +``` [regex] - type=filter - module=regexfilter - match=form - replace=from +``` ## Monitor @@ -535,85 +341,66 @@ In order for the various router modules to function correctly they require infor Monitors are defined in much the same way as other elements in the configuration file, with the section name being the name of the monitor instance and the type being set to monitor. +``` [MySQL Monitor] - type=monitor - module=mysqlmon - servers=server1,server2,server3 - user=dbmonitoruser - passwd=dbmonitorpwd - monitor_interval=8000 - backend_connect_timeout=3 - backend_read_timeout=1 - backend_write_timeout=2 # mysqlmon specific options - detect_replication_lag=0 - detect_stale_master=0 [Galera Monitor] - type=monitor - module=galeramon - servers=server1,server2,server3 - user=dbmonitoruser - passwd=dbmonitorpwd - monitor_interval=8000 - backend_connect_timeout=3 - backend_read_timeout=1 - backend_write_timeout=2 # galeramon specific options - disable_master_failback=0 +``` -### Module +### `module` The module parameter defines the name of the loadable module that implements the monitor. This module is loaded and executed on a separate thread within MaxScale. -### Servers +### `servers` The servers parameter is a comma separated list of server names to monitor, these are the names defined elsewhere in the configuration file. The set of servers monitored by a single monitor need not be the same as the set of servers used within any particular server, a single monitor instance may monitor servers in multiple servers. -### User +### `user` The user parameter defines the username that the monitor will use to connect to the monitored databases. Depending on the monitoring module used this user will require specific privileges in order to determine the state of the nodes, details of those privileges can be found in the sections on each of the monitor modules. Individual servers may define override values for the user and password the monitor uses by setting the monuser and monpasswd parameters in the server section. -### Passwd +### `passwd` -The password parameter may be either a plain text password or it may be an encrypted password. See the section on encrypting passwords for use in the MaxScale.cnf file. +The password parameter may be either a plain text password or it may be an encrypted password. See the section on encrypting passwords for use in the `MaxScale.cnf` file. -**Monitor_interval** +### `monitor_interval` The monitor_interval parameter sets the sampling interval in milliseconds for each monitor, the default value is 10000 milliseconds. -**Detect_replication_lag** +### `detect_replication_lag` -This options if set to 1 will allow MySQL monitor to collect the replication lag among all configured slaves by checking the content of maxscale_schema.replication_heartbeat table. The master server writes in and slaves fetch a UNIX timestamp from that there. +This options if set to 1 will allow MySQL monitor to collect the replication lag among all configured slaves by checking the content of `maxscale_schema.replication_heartbeat` table. The master server writes in and slaves fetch a UNIX timestamp from that there. -This timestamp is updated in each node server struct and it’s used to calculate the replication lag. +This timestamp is updated in each node server struct and it's used to calculate the replication lag. -That value is also used by the Read / Write split module via max_slave_replication_lag and LEAST_BEHIND_MASTER options. +That value is also used by the Read / Write split module via `max_slave_replication_lag` and `LEAST_BEHIND_MASTER` options. Replication lag is measured by writing to a table, replication_heartbeat in the maxscale_schema, updates to this table will be observed on the slave in order to determine the lag between the slave and the master on which it was written. If the slave is many minutes behind the master and MaxScale is then started the information in the slave table is not available and that slave may be excluded from the routing decision. @@ -621,49 +408,47 @@ A specific grant for the monitor user might be required in order to create schem This monitor option is not enabled by default. -**Detect_stale_master** +### `detect_stale_master` This options if set to 1 will allow MySQL monitor to select the previous selected Master for next operations even if no slaves at all are found by the monitor polling. -This is such a case when the replication on all slave has been stopped via STOP SLAVE or the current configuration was removed by RESET SLAVE ALL. +This is such a case when the replication on all slave has been stopped via `STOP SLAVE` or the current configuration was removed by `RESET SLAVE ALL`. -As there are no slaves the replication topology cannot be computed and MaxScale can only check if the current monitored server was the master before: if that’s the case +As there are no slaves the replication topology cannot be computed and MaxScale can only check if the current monitored server was the master before: if that's the case -MySQL monitor adds to the server status field the SERVER_STALE_STATUS bit and a log entry appears in the Message Log file. +MySQL monitor adds to the server status field the `SERVER_STALE_STATUS` bit and a log entry appears in the Message Log file. If MaxScale or monitor is restarted and the Replication is still not configured or started there will not be any master server available even with this option enabled. This option is not enabled by default and should be used at the administrator risk. -**Disable_master_failback** +### `disable_master_failback` -This option if set to 1 will allow Galera monitor to keep the existing selected master even if another node,after joining back the cluster may be selected as candidate master. +This option if set to 1 will allow Galera monitor to keep the existing selected master even if another node, after joining back the cluster may be selected as candidate master. -The master role assignment currently follows one rule: take the server with lowest wsrep_local_index value. +The master role assignment currently follows one rule: take the server with lowest `wsrep_local_index` value. By default, if a node takes a lower index than the current master one the monitor will set the master role to that node: this monitor option, if set, prevents the master change. -The server status field may have the SERVER_MASTER_STICKINESS bit, meaning the current master selection is not based on the available rules but it’s the one previously selected and then kept, accordingly to option value equal 1. +The server status field may have the `SERVER_MASTER_STICKINESS` bit, meaning the current master selection is not based on the available rules but it's the one previously selected and then kept, accordingly to option value equal 1. - +Anyway, a new master will be selected in case of current master failure, regardless the option value. -Anyway, a new master will be selected in case of current master failure, regardless the option value. +### `backend_connect_timeout` -**Backend_connect_timeout** +This option, with default value of `3` sets the monitor connect timeout to backends. -This option, with default value of 3 sets the monitor connect timeout to backends. +### `backend_read_timeout` -**Backend_read_timeout** +Default value is `1`. Read Timeout is the timeout in seconds for each attempt to read from the server. There are retries if necessary, so the total effective timeout value is three times the option value. That's for `mysql_real_connect` C API. -Default value is 1. Read Timeout is the timeout in seconds for each attempt to read from the server. There are retries if necessary, so the total effective timeout value is three times the option value. That’s for mysql_real_connect C API. +### `backend_write_timeout` -**Backend_write_timeout** - -Default value is 2. Write Timeout is the timeout in seconds for each attempt to write to the server. There is a retry if necessary, so the total effective timeout value is two times the option value. That’s for mysql_real_connect C API. +Default value is `2`. Write Timeout is the timeout in seconds for each attempt to write to the server. There is a retry if necessary, so the total effective timeout value is two times the option value. That's for `mysql_real_connect` C API. # Protocol Modules -The protocols supported by MaxScale are implemented as external modules that are loaded dynamically into the MaxScale core. These modules reside in the directory $MAXSCALE_HOME/modules, if the environment variable $MAXSCALE_HOME is not set it defaults to /usr/local/skysql/MaxScale. It may also be set by passing the -c option on the MaxScale command line. +The protocols supported by MaxScale are implemented as external modules that are loaded dynamically into the MaxScale core. These modules reside in the directory `$MAXSCALE_HOME/modules`, if the environment variable `$MAXSCALE_HOME` is not set it defaults to `/usr/local/skysql/MaxScale`. It may also be set by passing the `-c` option on the MaxScale command line. ## MySQLClient @@ -673,7 +458,7 @@ This is the implementation of the MySQL protocol that is used by clients of MaxS The MySQLBackend protocol module is the implementation of the protocol that MaxScale uses to connect to the backend MySQL, MariaDB and Percona Server databases. This implementation is tailored for the MaxScale to MySQL Database traffic and is not a general purpose implementation of the MySQL protocol. -## Telnetd +## telnetd The telnetd protocol module is used for connections to MaxScale itself for the purposes of creating interactive user sessions with the MaxScale instance itself. Currently this is used in conjunction with a special router implementation, the debugcli. @@ -711,449 +496,298 @@ This is a connection based query router that was originally targeted at environm Whenever a new connection is received the router will examine the state of all the servers that form part of the service and route the connection to the server with least connections currently that matches the filter constraints given in the router options. This results in a balancing of the active connections, however different connections may have different lifetimes and the connections may become unbalanced when later viewed. -The read connection router can be configured to balance the connections from the clients across all the backend servers that are running, just those backend servers that are currently replication slaves or those that are replication masters when routing to a master slave replication environment. When a Galera cluster environment is in use the servers can be filtered to just the set that are part of the cluster and in the ‘synced’ state. These options are configurable via the router_options that can be set within a service. The router_option strings supported are "master", “slave” and “synced”. +The read connection router can be configured to balance the connections from the clients across all the backend servers that are running, just those backend servers that are currently replication slaves or those that are replication masters when routing to a master slave replication environment. When a Galera cluster environment is in use the servers can be filtered to just the set that are part of the cluster and in the _Synced_ state. These options are configurable via the router_options that can be set within a service. The `router_option` values supported are `master`, `slave` and `synced`. #### Master/Slave Replication Setup -To setup MaxScale to route connections evenly between all the current slave servers in a replication cluster, a service entry of the form shown below is required. +To set up MaxScale to route connections evenly between all the current slave servers in a replication cluster, a service entry of the form shown below is required: +``` [Read Service] - type=service - router=readconnroute - router_options=slave - servers=server1,server2,server3,server4 - user=maxscale - passwd=thepasswd +``` -With the addition of a listener for this service, which defines the port and protocol that MaxScale uses +And then add a listener for this service, which defines the port and protocol that MaxScale uses: +``` [Read Listener] - type=listener - service=Read Service - protocol=MySQLClient - port=4006 +``` -the client can now connect to port 4006 on the host which is running MaxScale. Statements sent using this connection will then be routed to one of the slaves in the server set defined in the Read Service. Exactly which is selected will be determined by balancing the number of connections to each of those whose current state is "slave". +The client can now connect to port 4006 on the host where MaxScale runs. Statements sent using this connection will then be routed to one of the slaves in the server set defined in the Read Service. Exactly which is selected will be determined by balancing the number of connections to each of those whose current state is *Slave*. -Altering the router options to be slave, master would result in the connections being balanced between all the servers within the cluster. +Altering `router_options` to be `slave,master` would result in the connections being balanced between all the servers within the cluster. -It is assumed that the client will have a separate connection to the master server, however this can be routed via MaxScale, allowing MaxScale to manage the determination of which server is master. To do this you would add a second service and listener definition for the master server. +It is assumed that the client will have a separate connection to the master server, however this can be routed via MaxScale, allowing MaxScale to designate which server is master. To do this you would add a second service and listener definition for the master server. +``` [Write Service] - type=service - router=readconnroute - router_options=master - servers=server1,server2,server3,server4 - user=maxscale - passwd=thepasswd [Write Listener] - type=listener - service=Write Service - protocol=MySQLClient - port=4007 +``` This allows the clients to direct write requests to port 4007 and read requests to port 4006 of the MaxScale host without the clients needing to understand the configuration of the Master/Slave replication cluster. Connections to port 4007 would automatically be directed to the server that is the master for replication at the time connection is opened. Whilst this is a simple mapping to a single server it does give the advantage that the clients have no requirement to track which server is currently the master, devolving responsibility for managing the failover to MaxScale. -In order for MaxScale to be able to determine the state of these servers the mysqlmon monitor module should be run against the set of servers that comprise the service. +In order for MaxScale to be able to determine the state of these servers the **mysqlmon** monitor module should be run against the set of servers that comprise the service. #### Galera Cluster Configuration for Read Connection router -Although not primarily designed for a multi-master replication setup, it is possible to use the readconnroute in this situation. The readconnroute connection router can be used to balance the connections across a Galera cluster. A special monitor is available that detects if nodes are joined to a Galera Cluster, with the addition of a router option to only route connections to nodes marked as synced. MaxScale can ensure that users are never connected to a node that is not a full cluster member. +Although not primarily designed for a multi-master replication setup, it is possible to use **readconnroute** in this situation. The **readconnroute** connection router can be used to balance the connections across a Galera cluster. A special monitor is available that detects if nodes are joined to a Galera Cluster, with the addition of a router option to only route connections to nodes marked as synced. MaxScale can ensure that users are never connected to a node that is not a full cluster member. +``` [Galera Service] - type=service - router=readconnroute - router_options=synced - servers=server1,server2,server3,server4 - user=maxscale - passwd=thepasswd [Galera Listener] - type=listener - service=Galera Service - protocol=MySQLClient - port=3336 [Galera Monitor] - type=monitor - module=galeramon - servers=server1,server2,server3,server4 - user=galeramon - passwd=galeramon +``` -The specialized Galera monitor can also select one of the node in the cluster as master, the others will be marked as slave. These roles are only assigned to synced nodes. +The specialized Galera monitor can also select one of the node in the cluster as _Master_, the others will be marked as _Slave_. These roles are only assigned to _Synced_ nodes. -It then possible to have services/listeners with router_options=master or slave accessing a subset of all galera nodes. The "synced" simply means: access all nodes. Examples of different readconn router configurations for Galera: +It then possible to have services/listeners with `router_options=master` or `slave` accessing a subset of all galera nodes. The _Synced_ state simply means: access all nodes. Examples of different **readconn** router configurations for Galera: +``` [Galera Master Service] - type=service - router=readconnroute - router_options=master [Galera Slave Service] - type=service - router=readconnroute - router_options=slave +``` #### MySQL Cluster Configuration for Read Connection router -The readconnroute connection router can be used to balance the connections across a MySQL cluster SQL nodes. A special monitor is available that detects if SQL nodes are connected to data nodes, with the addition of a router option to only route connections to nodes marked as NDB. +The **readconnroute** connection router can be used to balance the connections across a MySQL cluster SQL nodes. A special monitor is available that detects if SQL nodes are connected to data nodes, with the addition of a router option to only route connections to nodes marked as NDB. MaxScale can ensure that users are never connected to a node that is not a full cluster member. +``` [NDB Cluster Monitor] - type=monitor - module=ndbclustermon - servers=server1,server2 - user=monitor - passwd=monitor -[MySQL Cluster Service] - +[MySQL Cluster Service] type=service - router=readconnroute - router_options=ndb - servers=server1,server2 [Cluster Listener] - type=listener - service=MySQL Cluster Service - protocol=MySQLClient - port=4906 +``` -The “ndb” router option simply means: access all SQL nodes marked with NDB status, i.e. they are members of the cluster. +The `ndb` router option simply means: access all SQL nodes marked with NDB status, i.e. they are members of the cluster. -### Read/Write Split router +### Read/Write Split Router -The Read/Write Split router is implemented in readwritesplit module. It is a statement-based router that has been designed for use within Master/Slave replication environments. It examines and optionally parses every statement to find out whether the statement can be routed to slave instead of master. +The Read/Write Split Router is implemented in readwritesplit module. It is a statement-based router that has been designed for use within Master/Slave replication environments. It examines and optionally parses every statement to find out whether the statement can be routed to slave instead of master. #### Starting a readwritesplit router session When client connects to readwritesplit service for the first time, client is authenticated against user data loaded from backend database. After successful authentication connection for client queries is created and followed by that, a readwritesplit router session is initialized. -Router session processes its specific configuration parameters and establishes connections to master and slaves. The number of slaves in each session depends on the value of max_slave_connections parameter (default is 1) and the availability of slaves. Most suitable number of slaves varies as it depends on the number of clients, and the backend servers and the type of load. In Figure below Server 1 is the master and Servers 2-7 are the available slaves. In this example max_slave_connections=3. +Router session processes its specific configuration parameters and establishes connections to master and slaves. The number of slaves in each session depends on the value of `max_slave_connections` parameter (default is `1`) and the availability of slaves. Most suitable number of slaves varies as it depends on the number of clients, and the backend servers and the type of load. In Figure below Server 1 is the master and Servers 2-7 are the available slaves. In this example `max_slave_connections=3`. ![image alt text](images/image_11.png) -#### Routing to master +#### Routing to *Master* Routing to master is important for data consistency and because majority of writes are written to binlog and thus become replicated to slaves. The following operations are routed to master: * write statements, - * all statements within an open transaction, - * stored procedure calls, and - * user-defined function calls. - -* DDL statements (DROP|CREATE|ALTER TABLE … etc.) - -* EXECUTE (prepared) statements - +* DDL statements (`DROP`|`CREATE`|`ALTER TABLE` … etc.) +* `EXECUTE` (prepared) statements * all statements using temporary tables -In addition to these, if readwritesplit service is configured with max_slave_replication_lag parameter, and if all slaves suffer from too long replication lag, then statements will be routed to master. (There might be other similar configuration parameters in the future which limit the number of statements that will be routed to slaves.) +In addition to these, if the **readwritesplit** service is configured with the `max_slave_replication_lag` parameter, and if all slaves suffer from too much replication lag, then statements will be routed to the _Master_. (There might be other similar configuration parameters in the future which limit the number of statements that will be routed to slaves.) -#### Routing to slaves +#### Routing to *Slave*s -Ability to route some statements to slaves is important because it also decreases the load targeted to master. Moreover, it is possible to have multiple slaves to share the load in contrast to single master. +The ability to route some statements to *Slave*s is important because it also decreases the load targeted to master. Moreover, it is possible to have multiple slaves to share the load in contrast to single master. Queries which can be routed to slaves must be auto committed and belong to one of the following group: * read-only database queries, - * read-only queries to system, or user-defined variables, - -* SHOW statements, and - +* `SHOW` statements, and * system function calls. #### Routing to every session backend -Third class of statements includes those, which modify session data, such as session system variables, user-defined variables, the database being used etc. We call them session commands, and they must be replicated as they affect the future results of read and write operations, so they must be executed on all servers that could execute statements on behalf of this client. +A third class of statements includes those which modify session data, such as session system variables, user-defined variables, the default database, etc. We call them session commands, and they must be replicated as they affect the future results of read and write operations, so they must be executed on all servers that could execute statements on behalf of this client. Session commands include for example: -* SET statements +* `SET` statements +* `USE `*``* +* system/user-defined variable assignments embedded in read-only statements, such as `SELECT (@myvar := 5)` +* `PREPARE` statements +* `QUIT`, `PING`, `STMT RESET`, `CHANGE USER`, etc. commands -* USE +**NOTE: if variable assignment is embedded in a write statement it is routed to _Master_ only. For example, `INSERT INTO t1 values(@myvar:=5, 7)` would be routed to _Master_ only.** -* embedded system/user-defined variable assignments (SELECT (@myvar := 5)) in read-only statements +#### Configuring the Read/Write Split router -* PREPARE statements - -* QUIT, PING, STMT RESET, CHANGE USER, etc. commands - -Note: if variable assignment is embedded in write statement it is routed to master only. For example, INSERT INTO t1 values(@myvar:=5, 7) would be routed to master only. - -#### Configuring Read/Write Split router - -Read/Write Split router-specific settings are specified in the configuration file of MaxScale in its specific section. The section can be freely named but the name is used latter as a reference from listener section. +Read/Write Split router-specific settings are specified in the configuration file of MaxScale in its specific section. The section can be freely named but the name is used later as a reference from listener section. The configuration consists of mandatory and optional parameters. ##### Mandatory parameters -**type** specifies the type of service. For readwritesplit module the type is: +`type` specifies the type of service. For **readwritesplit** module the type is `router`: -type=router + type=router -**service **specifies the router module to be used. For readwritesplit the value is: +`service` specifies the router module to be used. For **readwritesplit** the value is `readwritesplit`: - service=readwritesplit + service=readwritesplit -**servers **provide a list of servers, which must include one master and available slaves. Syntax for servers is: +`servers` provides a list of servers, which must include one master and available slaves: - servers= + servers=server1,server2,server3 -*Note that each server on the list must have its own section in the configuration file where it is defined.* +**NOTE: Each server on the list must have its own section in the configuration file where it is defined.** -**user **is assigned with the username which router session uses for accessing backends for loading the content of mysql.user table (and mysql.db and database names as well) and optionally for creating, and using maxscale_schema.replication_heartbeat table. +`user` is the username the router session uses for accessing backends in order to load the content of the `mysql.user` table (and `mysql.db` and database names as well) and optionally for creating, and using `maxscale_schema.replication_heartbeat` table. -**passwd **specifies corresponding password for the user. Syntax for user and passwd is: +`passwd` specifies corresponding password for the user. Syntax for user and passwd is: +``` user= - passwd= +``` ##### Optional parameters -**max_slave_connections **sets the maximum number of slaves a router session uses at any moment. Default value is 1. Syntax for max_slave_connections is: +`max_slave_connections` sets the maximum number of slaves a router session uses at any moment. Default value is `1`. max_slave_connections= -**max_slave_replication_lag **specifies how many seconds a slave is allowed to be behind the master. If the lag is bigger than configured value a slave can’t be used for routing. Syntax for max_slave_replication_lag is: +`max_slave_replication_lag` specifies how many seconds a slave is allowed to be behind the master. If the lag is bigger than configured value a slave can't be used for routing. max_slave_replication_lag= -This applies to Master/Slave replication with MySQL monitor and detect_replication_lag=1 options set +This applies to Master/Slave replication with MySQL monitor and `detect_replication_lag=1` options set -**router_options **may include multiple readwritesplit-specific options. Values are either singular or parameter-value pairs. Currently available is a single option which specifies the criteria used in slave selection both in initialization of router session and per each query. *Note, that due to the current monitor implementations, the value specified here should be twice the monitor interval + 1**.* +`router_options` may include multiple **readwritesplit**-specific options. Values are either singular or parameter-value pairs. Currently available is a single option which specifies the criteria used in slave selection both in initialization of router session and per each query. Note that due to the current monitor implementation, the value specified here should be ** + 1. options=slave_selection_criteria= -where is one of the following: +where ** is one of the following: - /** slave with least connections in total */ +* `LEAST_GLOBAL_CONNECTIONS`, the slave with least connections in total +* `LEAST_ROUTER_CONNECTIONS`, the slave with least connections from this router +* `LEAST_BEHIND_MASTER`, the slave with smallest replication lag +* `LEAST_CURRENT_OPERATIONS` (default), the slave with least active operations - LEAST_GLOBAL_CONNECTIONS +`use_sql_variables_in` specifies where should queries, which read session variable, be routed. The syntax for `use_sql_variable_in` is: - /** slave with least connections from this router */ - - LEAST_ROUTER_CONNECTIONS - - /** slave with smallest replication lag */ - - LEAST_BEHIND_MASTER - - /** slave with least active operations */ - - LEAST_CURRENT_OPERATIONS** **(default) - -**use_sql_variables_in** specifies where should queries, which read session variable, be routed. The syntax for use_sql_variable_in is: - - use_sql_variables_in=[master|all] + use_sql_variables_in=[master|all] When value all is used, queries reading session variables can be routed to any available slave (depending on selection criteria). Note, that queries modifying session variables are routed to all backend servers by default, excluding write queries with embedded session variable modifications, such as: - INSERT INTO test.t1 VALUES (@myid:=@myid+1) + INSERT INTO test.t1 VALUES (@myid:=@myid+1) -In above-mentioned case the user-defined variable would only be updated in the master where query would be routed due to INSERT statement. +In above-mentioned case the user-defined variable would only be updated in the master where query would be routed due to `INSERT` statement. An example of Read/Write Split router configuration : +``` [RWSplit Service] - type=service - router=readwritesplit - router_options=slave_selection_criteria=LEAST_BEHIND_MASTER - max_slave_connections=50% - max_slave_replication_lag=61 - servers=server1,server2,server3,server4 - user=myuser - passwd=mypass - filters=qla|fetch|from +``` In addition to this, readwritesplit needs configuration for a listener, for all servers listed, and for each filter. Listener, server - and filter configurations are described in their own sections in this document. Below is a listener example for the "RWSplit Service" defined above: +``` [RWSplit Listener] - type=listener - service=RWSplit Service - protocol=MySQLClient - port=4044 +``` The client would merely connect to port 4044 on the MaxScale host and statements would be directed to the master, slave or all backends as appropriate. Determination of the master or slave status may be done via a monitor module within MaxScale or externally. In this latter case the server flags would need to be set via the MaxScale debug interface, in future versions an API will be available for this purpose. #### Galera Cluster Configuration for Read/Write Split router -Galera monitor assigns Master and Slave roles to appropriate sync’ed Galera nodes. Using readwritesplit with Galera is seamless; only change needed to the configuration above is replacing the list of MySQL replication servers with list of Galera nodes. With the same example as above: +Galera monitor assigns Master and Slave roles to appropriate sync'ed Galera nodes. Using **readwritesplit** with Galera is seamless; the only change needed to the configuration above is replacing the list of MySQL replication servers with list of Galera nodes. With the same example as above: -Simply configure a Split Service with galera nodes: +Simply configure a RWSplit Service with Galera nodes: +``` [RWSplit Service] - type=service - router=readwritesplit - max_slave_connections=50% - servers=galera_node1,galera_node2,galera_node3 - user=myuser - passwd=mypass - filters=qla|fetch|from - -### Debugcli - -The debugcli is a special case of a statement based router. Rather than direct the statements at an external data source they are handled internally. These statements are simple text commands and the results are the output of debug commands within MaxScale. The service and listener definitions for a debug cli service only differ from other services in that they require no backend server definitions. - -#### Debug CLI Configuration - -The definition of the debug cli service is illustrated below - -[Debug Service] - -type=service - -router=debugcli - -[Debug Listener] - -type=listener - -service=Debug Service - -protocol=telnetd - -port=4442 - -Connections using the telnet protocol to port 4442 of the MaxScale host will result in a new debug CLI session. A default username and password are used for this module, new users may be created using the add user command. As soon as any users are explicitly created the default username will no longer continue to work. The default username is admin with a password of skysql. - -The debugcli supports two modes of operation, developer mode and user mode. The mode is set via the router_options parameter of the debugcli. The user mode is more suited to end-users and administrators, whilst the develop mode is explicitly targeted to software developing adding or maintaining the MaxScale code base. Details of the differences between the modes can be found in the debugging guide for MaxScale. The default mode for the debugcli is user mode. The following service definition would enable a developer version of the debugcli. - -[Debug Service] - -type=service - -router=debugcli - -router_options=developer - -It should be noted that both a user and a developer version of the debugcli may be defined within the same instance of MaxScale, however they must be defined as two distinct services, each with a distinct listener. - -[Debug Service] - -type=service - -router=debugcli - -router_options=developer - -[Debug Listener] - -type=listener - -service=Debug Service - -protocol=telnetd - -port=4442 - -[Admin Service] - -type=service - -router=debugcli - -[Admin Listener] - -type=listener - -service=Debug Service - -protocol=telnetd - -port=4242 +``` ### CLI -The command line interface as used by maxadmin. This is a variant of the debugcli that is built slightly differently so that it may be accessed by the client application maxadmin. The CLI requires the use of the maxscaled protocol. +The command line interface as used by `maxadmin`. This is a variant of the debugcli that is built slightly differently so that it may be accessed by the client application `maxadmin`. The CLI requires the use of the `maxscaled` protocol. #### CLI Configuration @@ -1161,128 +795,159 @@ There are two components to the definition required in order to run the command The default entries required are shown below. -[CLI\] - +``` +[CLI] type=service - router=cli [CLI Listener] - type=listener - service=CLI - protocol=maxscaled - address=localhost - port=6603 +``` Note that this uses the default port of 6603 and confines the connections to localhost connections only. Remove the address= entry to allow connections from any machine on your network. Changing the port from 6603 will mean that you must allows pass a -p option to the MaxAdmin command. +### Debug CLI + +The **debugcli** router is a special kind of statement based router. Rather than direct the statements at an external data source they are handled internally. These statements are simple text commands and the results are the output of debug commands within MaxScale. The service and listener definitions for a debug cli service only differ from other services in that they require no backend server definitions. + +#### Debug CLI Configuration + +The definition of the debug cli service is illustrated below + +``` +[Debug Service] +type=service +router=debugcli + +[Debug Listener] +type=listener +service=Debug Service +protocol=telnetd +port=4442 +``` + +Connections using the telnet protocol to port 4442 of the MaxScale host will result in a new debug CLI session. A default username and password are used for this module, new users may be created using the add user command. As soon as any users are explicitly created the default username will no longer continue to work. The default username is admin with a password of skysql. + +The debugcli supports two modes of operation, `developer` and `user`. The mode is set via the `router_options` parameter. The user mode is more suited to end-users and administrators, whilst the develop mode is explicitly targeted to software developing adding or maintaining the MaxScale code base. Details of the differences between the modes can be found in the debugging guide for MaxScale. The default is `user` mode. The following service definition would enable a developer version of the debugcli. + +``` +[Debug Service] +type=service +router=debugcli +router_options=developer +``` + +It should be noted that both `user` and `developer` instances of debugcli may be defined within the same instance of MaxScale, however they must be defined as two distinct services, each with a distinct listener. + +``` +[Debug Service] +type=service +router=debugcli +router_options=developer + +[Debug Listener] +type=listener +service=Debug Service +protocol=telnetd +port=4442 + +[Admin Service] +type=service +router=debugcli + +[Admin Listener] +type=listener +service=Debug Service +protocol=telnetd +port=4242 +``` + # Monitor Modules Monitor modules are used by MaxScale to internally monitor the state of the backend databases in order to set the server flags for each of those servers. The router modules then use these flags to determine if the particular server is a suitable destination for routing connections for particular query classifications. The monitors are run within separate threads of MaxScale and do not affect the MaxScale performance. The use of monitors is optional, it is possible to run MaxScale with external monitoring, in which case arrangements must be made for an external entity to set the status of each of the servers that MaxScale can route to. -# Parameters that apply to all monitors are: +Parameters that apply to all monitors are: -* monitor_interval - -* backend_connect_timeout - -* backend_read_timeout - -* backend_write_timeout +* `monitor_interval` +* `backend_connect_timeout` +* `backend_read_timeout` +* `backend_write_timeout` Other parameters are monitor specific. -## Mysqlmon +## mysqlmon The MySQLMon monitor is a simple monitor designed for use with MySQL Master/Slave replication cluster. To execute the mysqlmon monitor an entry as shown below should be added to the MaxScale configuration file. +``` [MySQL Monitor] - type=monitor - module=mysqlmon - servers=server1,server2,server3,server4 +``` -This will monitor the 4 servers; server1, server2, server3 and server4. It will set the status of running or failed and master or slave for each of the servers. +This will monitor the 4 servers; `server1`, `server2`, `server3` and `server4`. It will set the status of running or failed and master or slave for each of the servers. -The monitor uses the username given in the monitor section or the server specific user that is given in the server section to connect to the server. This user must have sufficient permissions on the database to determine the state of replication. The roles that must be granted to this user are REPLICATION SLAVE and REPLICATION CLIENT. +The monitor uses the username given in the monitor section or the server specific user that is given in the server section to connect to the server. This user must have sufficient permissions on the database to determine the state of replication. The roles that must be granted to this user are `REPLICATION SLAVE` and `REPLICATION CLIENT`. -To create a user that can be used to monitor the state of the cluster, the following commands could be used, assuming that MaxScale is running on the host ‘maxscalehost’ +To create a user that can be used to monitor the state of the cluster, the following commands could be used, assuming that MaxScale is running on the host 'maxscalehost' +``` MariaDB [mysql]> create user 'maxscalemon'@'maxscalehost' identified by 'Ha79hjds'; - Query OK, 0 rows affected (0.01 sec) MariaDB [mysql]> grant REPLICATION SLAVE on *.* to 'maxscalemon'@'maxscalehost'; - Query OK, 0 rows affected (0.00 sec) MariaDB [mysql]> grant REPLICATION CLIENT on *.* to 'maxscalemon'@'maxscalehost'; - Query OK, 0 rows affected (0.00 sec) +``` -MariaDB [mysql]> +MySQL monitor fetches the `@@server_id` variable and other informations from `SHOW SLAVE STATUS` in order to compute the replication topology tree that may include intermediate master servers, called relay servers. -MySQL monitor fetches the @@server_id variable and other informations from SHOW SLAVE STATUS in order to compute the replication topology tree that may include intermediate master servers, called relay servers. +The *Master* server used by router modules is the so called "root master": a server that has the `SERVER_MASTER` status bit set and it's at the lowest level of the replication depth. -The Master server used by router modules is the so called "root master": a server that has the SERVER_MASTER status bit set and it’s at the lowest level of the replication depth. +MySQL monitor may optionally (`detect_replication_lag=1`) detect the replication lag among servers by using the `maxscale_schema.replication_heartbeat` table: the monitor user must have rights to create it and write into. -MySQL monitor may optionally (detect_replication_lag=1) detect the replication lag among servers by using the maxscale_schema.replication_heartbeat table: the monitor user must have rights to create it and write into. - -Another option (detect_stale_master=1) may also allow to set a Stale Master when the replication has been stopped or the configuration doesn’t allow to have both IO and SQL replication threads running on all slaves: the previous detected working Master will be selected for read and write operations. +Another option (`detect_stale_master=1`) may also allow to set a Stale Master when the replication has been stopped or the configuration doesn't allow to have both IO and SQL replication threads running on all slaves: the previous detected working Master will be selected for read and write operations. Please note, those two options are not enabled by default. -## Galeramon +## galeramon The Galeramon monitor is a simple router designed for use with MySQL Galera cluster. To execute the galeramon monitor an entry as shown below should be added to the MaxScale configuration file. +``` [Galera Monitor] - type=monitor - module=galeramon - servers=galera_node1,galera_node2,galera_node3 +``` -This will monitor the 4 servers; server1, server2, server3 and server4. It will set the status of running or failed and joined for those servers that reported the Galera JOINED status. - -The user that is configured for use with the Galera monitor must have sufficient privileges to select from the information_schema database and GLOBAL_STATUS table within that database. +This will monitor the 4 servers; server1, server2, server3 and server4. It will set the status of *Running* or *Failed* and *Joined* for those servers that reported the Galera JOINED status. To create a user that can be used to monitor the state of the cluster, the following commands could be used, assuming that MaxScale is running on the host maxscalehost. +``` MariaDB [mysql]> create user 'maxscalemon'@'maxscalehost' identified by 'Ha79hjds'; - Query OK, 0 rows affected (0.01 sec) +``` -MariaDB [mysql]> +The Galera monitor also assigns *Master* and *Slave* roles to the configured nodes. Among the set of synced servers, the one with the lowest value of `wsrep_local_index` is selected as the current *Master* while the others are given the role of *Slave*; that's the only available master selection rule right now. -The Galera monitor can also assign Master and Slave roles to the configured nodes: +In this way it is possible to configure the node access based not only on *Synced* state but even on *Master* and *Slave* role enabling the use of the Read/Write Split router on a Galera cluster and avoiding any possible write conflict. -among the set of synced servers, the one with the lowest value of ‘wsrep_local_index’ is selected as the current master while the others are slaves: that’s the only available master selection rule right now. +It may happen that after a node failure or reboot or node joining back the cluster, the node's `wsrep_local_index` in the cluster nodes changes. This might result in monitor assigning the *Master* role to another server. In order to avoid such situation, the `disable_master_failback` configuration option helps keep the current master regardless of the value of `wsrep_local_index`. This option is not enabled by default. -This way is possible to configure the node access based not only on ‘synced’ state but even on Master and Slave role enabling the use of Read Write split operation on a Galera cluster and avoiding any possible write conflict. - -It may happen that after a node failure or reboot or node joining back the cluster, the - -‘wsrep_local_index’ in the cluster nodes changes. - -This might result in monitor assigning the Master role to another server. - -In order to avoid such situation the disable_master_failback=1 configuration option helps keeping the current master regardless ‘wsrep_local_index’ value. - -The option it’s not enabled by default. - -Example status for a Galera server node is: +This is an example status for a Galera server node: +``` Server 0x261fe50 (server2) Server: 192.168.1.101 @@ -1303,30 +968,26 @@ Server 0x2d1b3c0 (server4) Port: 3306 Server Version: 5.5.40-MariaDB-wsrep-log Node Id: 1 +``` +## ndbclustermon -## NDBclustermon - -The NDBclustermon monitor is a simple router designed for use with MySQL Cluster. To execute the ndclustermon monitor an entry as shown below should be added to the MaxScale configuration file. +The NDB Cluster Monitor (ndbclustermon) is a simple router designed for use with MySQL Cluster. To execute the ndclustermon monitor an entry as shown below should be added to the MaxScale configuration file. Example for monitor section: +``` [NDB Cluster Monitor] - type=monitor - module=ndbclustermon - servers=server1,server2 +``` -This will monitor the two SQL Node server1, server2 and will set the status of running or failed and NDB for those servers that reported the value of status variable Ndb_number_of_ready_data_nodes is greater than 0 - i.e. the monitored SQL node is able to contact one or more data nodes. - -The user that is configured for use with the NDBcluster monitor must have sufficient privileges to select from the information_schema database and GLOBAL_STATUS table within that database.. - +This will monitor the two SQL nodes `server1` and `server2` and will set the status of *NDB* and *Running* or *Failed* for those servers with the value of status variable `Ndb_number_of_ready_data_nodes` greater than 0, i.e. the monitored SQL node is able to contact one or more data nodes. Example of a monitored server: - +``` Server 0x3873a40 (server2) Server: 192.168.90.81 Status: NDB, Running @@ -1334,37 +995,34 @@ Example of a monitored server: Port: 3306 Server Version: 5.5.38-ndb-7.2.17-cluster-gpl Node Id: 13 - -Server 0x3873a40 (server2) +``` The MySQL Cluster variables fetched by the monitor are: +``` mysql> SHOW STATUS LIKE 'Ndb_number_of_ready_data_nodes'; - - +--------------------------------+-------+ - + - | Variable_name | Value | - +--------------------------------+-------+ - | Ndb_number_of_ready_data_nodes | 2 | - +--------------------------------+-------+ - - 1 row in set (0.00 sec) - ++--------------------------------+-------+ ++ +| Variable_name | Value | ++--------------------------------+-------+ +| Ndb_number_of_ready_data_nodes | 2 | ++--------------------------------+-------+ +1 row in set (0.00 sec) +``` The result is greater than 0 so the NBD status is added to status +``` mysql> SHOW STATUS LIKE 'Ndb_cluster_node_id'; ++---------------------+-------+ +| Variable_name | Value | ++---------------------+-------+ +| Ndb_cluster_node_id | 13 | ++---------------------+-------+ +1 row in set (0.00 sec) +``` - +---------------------+-------+ - | Variable_name | Value | - +---------------------+-------+ - | Ndb_cluster_node_id | 13 | - +---------------------+-------+ - - 1 row in set (0.00 sec) - - -The value is stored in node_id server field. +The value is stored in `node_id` server field. # Filter Modules @@ -1406,45 +1064,43 @@ The statement counting filter is implemented in the module names testfilter and In order to add this filter to an existing service create a filter section to name the filter as follows +``` [counter] - type=filter - module=testfilter +``` Then add the filter to your service by including the filters= parameter in the service section. -filters=counter + filters=counter -## Query Log All Filter +## Query Log All (QLA) Filter -The QLA filter simply writes all SQL statements to a log file along with a timestamp for the statement. An example of the file produced by the QLA filter is shown below +The Query Log All Filter (qlafilter) simply writes all SQL statements to a log file along with a timestamp for the statement. An example of the file produced by the QLA filter is shown below +``` 00:36:04.922 5/06/2014, select @@version_comment limit 1 - 00:36:12.663 5/06/2014, SELECT DATABASE() - 00:36:12.664 5/06/2014, show databases - 00:36:12.665 5/06/2014, show tables +``` A new file is created for each client connection, the name of the logfile can be controlled by the use of the router options. No parameters are used by the QLA filter. The filter is implemented by the loadable module qlafilter. To add the QLA filter to a service you must create a filter section to name the filter, associated the loadable module and define the filename option. +``` [QLA] - type=filter - module=qlafilter - options=/tmp/QueryLog +``` Then add the filters= parameter into the service that you wish to log by adding this parameter to the service section -filters=QLA + filters=QLA -A log file will be created for each client connection, the name of that log file will be /tmp/QueryLog. +A log file will be created for each client connection, the name of that log file will be `/tmp/QueryLog.`*``* ## Regular Expression Filter @@ -1452,55 +1108,50 @@ The regular expression filter is a simple text based query rewriting filter. It To add the filter to your service you must first create a filter section to name the filter and give the match and replacement strings. Here we define a filter that will convert to MariaDB 10 command show all slaves status to the older form of show slave status for MariaDB 5.5. +``` [slavestatus] - type=filter - module=regexfilter - match=show *all *slaves - replace=show slave +``` You must then add this filter to your service by adding the filters= option -filters=slavestatus + filters=slavestatus -Another example would be a filter to convert from the MySQL 5.1 create table syntax that used the TYPE keyword to the newer ENGINE keyword. +Another example would be a filter to convert from the MySQL 5.1 create table syntax that used the `TYPE` keyword to the newer `ENGINE` keyword. +``` [EnginerFilter] - type=filter - module=regexfilter - match=TYPE - replace=ENGINE +``` This would then change the SQL sent by a client application written to work with MySQL 5.1 into SQL that was compliant with MySQL 5.5. The statement -create table supplier(id integer, name varchar(80)) type=innodb + create table supplier(id integer, name varchar(80)) TYPE=innodb would be replaced with -create table supplier(id integer, name varchar(80)) ENGINE=innodb + create table supplier(id integer, name varchar(80)) ENGINE=innodb -before being sent to the server. Note that the text in the match string is case independent. +before being sent to the server. Note that the text in the match string is case-insensitive. ## Tee Filter -The tee filter is a filter module for MaxScale is a "plumbing" fitting in the MaxScale filter toolkit. It can be used in a filter pipeline of a service to make a copy of requests from the client and dispatch a copy of the request to another service within MaxScale. +The **tee** filter is a filter module for MaxScale that acts as a "plumbing" fitting in the MaxScale filter toolkit. It can be used in a filter pipeline of a service to make a copy of requests from the client and dispatch a copy of the request to another service within MaxScale. -The configuration block for the TEE filter requires the minimal filter parameters in it’s section within the MaxScale.cnf file that defines the filter to load and the service to send the duplicates to. +The configuration block for the **tee** filter requires the minimal filter parameters in its section within the `MaxScale.cnf` file that defines the filter to load and the service to send the duplicates to. +``` [ArchiveFilter] - type=filter - module=tee - service=Archive +``` In addition parameters may be added to define patterns to match against to either include or exclude particular SQL statements to be duplicated. You may also define that the filter is only active for connections from a particular source or when a particular user is connected. @@ -1508,17 +1159,15 @@ In addition parameters may be added to define patterns to match against to eithe The top filter is a filter module for MaxScale that monitors every SQL statement that passes through the filter. It measures the duration of that statement, the time between the statement being sent and the first result being returned. The top N times are kept, along with the SQL text itself and a list sorted on the execution times of the query is written to a file upon closure of the client session. -The configuration block for the TOP filter requires the minimal filter options in it’s section within the MaxScale.cnf file, stored in $MAXSCALE_HOME/etc/MaxScale.cnf. +The configuration block for the **top** filter requires the minimal filter options in its section within the `MaxScale.cnf` file, stored in `$MAXSCALE_HOME/etc/MaxScale.cnf`. +``` [MyLogFilter] - type=filter - module=topfilter - filebase=/var/log/Top10Queries - count=10 +``` In addition parameters may be added to define patterns to match against to either include or exclude particular SQL statements to be duplicated. You may also define that the filter is only active for connections from a particular source or when a particular user is connected. @@ -1526,35 +1175,31 @@ In addition parameters may be added to define patterns to match against to eithe Passwords stored in the MaxScale.cnf file may optionally be encrypted for added security. This is done by creation of an encryption key on installation of MaxScale. Encryption keys may be created manually by executing the maxkeys utility with the argument of the filename to store the key. -maxkeys $MAXSCALE_HOME/etc/.secrets + maxkeys $MAXSCALE_HOME/etc/.secrets Changing the encryption key for MaxScale will invalidate any currently encrypted keys stored in the MaxScale.cnf file. ## Creating Encrypted Passwords -Encrypted passwords are created by executing the maxpasswd command with the password you require to encrypt as an argument. The environment variable MAXSCALE_HOME must be set, or MaxScale must be installed in the default location before maxpasswd can be executed. +Encrypted passwords are created by executing the maxpasswd command with the password you require to encrypt as an argument. The environment variable `MAXSCALE_HOME` must be set, or MaxScale must be installed in the default location before maxpasswd can be executed. -maxpasswd MaxScalePw001 - -61DD955512C39A4A8BC4BB1E5F116705 + maxpasswd MaxScalePw001 + 61DD955512C39A4A8BC4BB1E5F116705 The output of the maxpasswd command is a hexadecimal string, this should be inserted into the MaxScale.cnf file in place of the ordinary, plain text, password. MaxScale will determine this as an encrypted password and automatically decrypt it before sending it the database server. +``` [Split Service] - type=service - router=readwritesplit - servers=server1,server2,server3,server4 - user=maxscale - password=61DD955512C39A4A8BC4BB1E5F116705 +``` -# Configuration Updates +# Reloading Configuration -The current MaxScale configuration may be updating by editing the configuration file and then forcing MaxScale to reread the configuration file. To force MaxScale to reread the configuration file a SIGTERM signal is sent to the MaxScale process. +The current MaxScale configuration may be updated by editing the configuration file and then forcing MaxScale to reread the configuration file. To force MaxScale to reread the configuration file, send a SIGHUP signal to the MaxScale process or execute `reload config` in the `maxadmin` client. Some changes in configuration can not be dynamically applied and require a complete restart of MaxScale, whilst others will take some time to be applied. @@ -1572,35 +1217,18 @@ MySQL uses username, passwords and the client host in order to authenticate a us It is important to understand, however, that when MaxScale itself makes connections to the backend servers the backend server will see all connections as originating from the host that runs MaxScale and not the original host from which the client connected to MaxScale. Therefore the backend servers should be configured to allow connections from the MaxScale host for every user that can connect from any host. Since there is only a single password within the database server for a given host, this limits the configuration such that a given user name must have the same password for every host from which they can connect. -To clarify, if a user X is defined as using password *pass1* from host a and *pass2* from host b then there must be an entry in the user table for user X form the MaxScale host, say *pass1*. +To clarify, if a user *X* is defined as using password *pass1* from host *a* and *pass2* from host *b* then there must be an entry in the `user` table for user *X* from the MaxScale host, say *pass1*. This would result in rows in the user table as follows - - - - - - - - - - - - - - - - - - - - - -
UsernamePasswordClient Host
Xpass1a
Xpass2b
Xpass1MaxScale
+Username|Password|Client Host +--------|--------|----------- + X | pass1 | a + X | pass2 | b + X | pass1 | MaxScale -In this case the user X would be able to connect to MaxScale from host a giving the password of *pass1*. In addition MaxScale would be able to create connections for this user to the backend servers using the username X and password *pass1*, since the MaxScale host is also defined to have password *pass1*. User X would not however be able to connect from host b since they would need to provide the password *pass2* in order to connect to MaxScale, but then MaxScale would not be able to connect to the backends as it would also use the password *pass2* for these connections. +In this case the user *X* would be able to connect to MaxScale from host a giving the password of *pass1*. In addition MaxScale would be able to create connections for this user to the backend servers using the username *X* and password *pass1*, since the MaxScale host is also defined to have password *pass1*. User *X* would not however be able to connect from host *b* since they would need to provide the password *pass2* in order to connect to MaxScale, but then MaxScale would not be able to connect to the backends as it would also use the password *pass2* for these connections. ## Wildcard Hosts @@ -1616,9 +1244,10 @@ If MaxScale is running on the same host as one or more of the database nodes to In all these cases the issue may be solved by adding an explicit entry for the localhost address that has the same password as the wildcard entry. This may be done using a statement as below for each of the databases that are required: -MariaDB [mysql]> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP ON employee.* 'user1'@'localhost' IDENTIFIED BY ‘xxx’; - +``` +MariaDB [mysql]> GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP ON employee.* 'user1'@'localhost' IDENTIFIED BY 'xxx'; Query OK, 0 rows affected (0.00 sec) +``` ## Limitations @@ -1627,24 +1256,20 @@ At the time of writing the authentication mechanism within MaxScale does not sup Wildcard address supported in the current version of MaxScale are: 192.168.3.% - 192.168.%.% - 192.%.%.% and short notations 192.% - 192.%.% - 192.168.% # Error Reporting -MaxScale is designed to be executed as a service, therefore all error reports, including configuration errors, are written to the MaxScale error log file. MaxScale will log to a set of files in the directory $MAXSCALE_HOME/log, the only exception to this is if the log directory is not writable, in which case a message is sent to the standard error descriptor. +MaxScale is designed to be executed as a service, therefore all error reports, including configuration errors, are written to the MaxScale error log file. MaxScale will log to a set of files in the directory `$MAXSCALE_HOME/log`, the only exception to this is if the log directory is not writable, in which case a message is sent to the standard error descriptor. -Troubleshooting +## Troubleshooting MaxScale binds on TCP ports and UNIX sockets as well. @@ -1654,19 +1279,15 @@ If the firewall is a network facility among all the involved servers, a configur Example: +``` [Galera Listener] - type=listener - - address=192.1681.3.33 - - port=4408 - - socket=/servers/maxscale/galera.sock - - +address=192.1681.3.33 +port=4408 +socket=/servers/maxscale/galera.sock +``` TCP/IP Traffic must be permitted to 192.1681.3.33 port 4408 -For Unix socket, the socket file path (example: /servers/maxscale/galera.sock) must be writable by the Unix user MaxScale runs as. +For Unix socket, the socket file path (example: `/servers/maxscale/galera.sock`) must be writable by the Unix user MaxScale runs as. diff --git a/Documentation/Tutorials/Replication-Proxy-Binlog-Router-Tutorial.md b/Documentation/Tutorials/Replication-Proxy-Binlog-Router-Tutorial.md index 84616a721..9d062d861 100644 --- a/Documentation/Tutorials/Replication-Proxy-Binlog-Router-Tutorial.md +++ b/Documentation/Tutorials/Replication-Proxy-Binlog-Router-Tutorial.md @@ -59,11 +59,16 @@ As with uuid, MaxScale must have a unique server-id for the connection it makes ### user -This is the user name that MaxScale uses when it connects to the master. This user name must have the rights required for replication as with any other user that a slave uses for replication purposes. +This is the user name that MaxScale uses when it connects to the master. This user name must have the rights required for replication as with any other user that a slave uses for replication purposes. If the user parameter is not given in the router options then the same user as is used to retrieve the credential information will be used for the replication connection, i.e. the user in the service entry. + +The user that is used for replication, either defined using the user= option in the router options or using the username and password defined of the service must be granted replication privileges on the database server. + + MariaDB> CREATE USER 'repl'@'maxscalehost' IDENTIFIED by 'password'; + MariaDB> GRANT REPLICATION SLAVE ON *.* TO 'repl'@'maxscalehost'; ### password -The password of the above user. +The password of the above user. If the password is not explicitly given then the password in the service entry will be used. For compatibility with other username and password definitions within the MaxScale configuration file it is also possible to use the parameter passwd=. ### master-id @@ -75,7 +80,7 @@ This parameter is used to provide the stem of the file names that are used to st ### initialfile -This optional parameter allows for the administrator to define the number of the first binlog file to download. In normal circumstances MaxScale will use any existing binlog file to determine what to request from the master. If there are no files it will then ask for the binlog file with the index number defined in the initialfile parameter. If this parameter is not set then MaxScale will ask the master for binlog events from file 1. +This optional parameter allows for the administrator to define the number of the first binlog file to download. If MaxScale has previously received binlogs it will use those existing binlog files to determine what to request from the master. If no files have been downloaded MaxScale will then ask for the binlog file with the index number defined in the initialfile parameter. If this parameter is not set then MaxScale will ask the master for binlog events from file 1. ### binlogdir @@ -87,19 +92,21 @@ This defines the value of the heartbeat interval in seconds for the connection t ### burstsize -This parameter is used to define the maximum amount of data that will be sent to a slave by MaxScale when that slave is lagging behind the master. In this situation the slave is said to be in "catchup mode", this parameter is designed to both prevent flooding of that slave and also to prevent threads within MaxScale spending disproportionate amounts of time with slaves that are lagging behind the master. The burst size can be defined in Kb, Mb or Gb by adding the qualifier K, M or G to the number given. +This parameter is used to define the maximum amount of data that will be sent to a slave by MaxScale when that slave is lagging behind the master. In this situation the slave is said to be in "catchup mode", this parameter is designed to both prevent flooding of that slave and also to prevent threads within MaxScale spending disproportionate amounts of time with slaves that are lagging behind the master. The burst size can be defined in Kb, Mb or Gb by adding the qualifier K, M or G to the number given. The default value of burstsize is 1Mb and will be used if burstsize is not given in the router options. A complete example of a service entry for a binlog router service would be as follows. [Replication] type=service router=binlogrouter - servers=maserdb + servers=masterdb version_string=5.6.17-log router_options=uuid=f12fcb7f-b97b-11e3-bc5e-0401152c4c22,server-id=3,user=repl,password=slavepass,master-id=1,filestem=mybin,heartbeat=30,binlogdir=/home/mriddoch/binlogs user=maxscale passwd=Mhu87p2D +The minimum set of router options that must be given in the configuration are are server-id and aster-id, default values may be used for all other options. + ## Listener Section As per any service in MaxScale a listener section is required to define the address, port and protocol that is used to listen for incoming connections. In this case those incoming connections will originate from the slave servers. diff --git a/Documentation/filters/RabbitMQ-Consumer-Client.md b/Documentation/filters/RabbitMQ-Consumer-Client.md new file mode 100644 index 000000000..9e8bf5edc --- /dev/null +++ b/Documentation/filters/RabbitMQ-Consumer-Client.md @@ -0,0 +1,38 @@ +#RabbitMQ Consumer Client + +## Overview + +This utility tool is used to read messages from a RabbitMQ broker sent by the [RabbitMQ Filter](RabbitMQ-Filter.md) and forward these messages into an SQL database as queries. + +## Command Line Arguments + +The **RabbitMQ Consumer Client** only has one command line argument. + +| Command | Argument | +|---------|-------------------------------------------------| +| -c | Path to the folder containing the configuration file | + +## Installation + +To install the RabbitMQ Consumer Client you ca either use the provided packages or you can compile it from source code. The source code is included as a part of the MaxScale source code and can be found in the `rabbtmq_consumer` folder. Please refer to the [README](../../rabbitmq_consumer/README) in the folder for more detailed instructions about installation and configuration. + +## Configuration + +The consumer client requires that the `consumer.cnf` configuration file is either be present in the `etc` folder of the installation directory or in the folder specified by the `-c` argument. + +The source broker, the destination database and the message log file can be configured into the separate `consumer.cnf` file. + +| Option | Desctiption | +|-----------|---------------------------------------------| +| hostname | Hostname of the RabbitMQ server | +| port | Port of the RabbitMQ server | +| vhost | Virtual host location of the RabbitMQ server | +| user | Username for the RabbitMQ server | +| passwd | Password for the RabbitMQ server | +| queue | Queue to consume from | +| dbserver | Hostname of the SQL server | +| dbport | Port of the SQL server | +| dbname | Name of the SQL database to use | +| dbuser | Database username | +| dbpasswd | Database passwork | +| logfile | Message log filename | diff --git a/Documentation/filters/RabbitMQ-Filter.md b/Documentation/filters/RabbitMQ-Filter.md new file mode 100644 index 000000000..885b620f6 --- /dev/null +++ b/Documentation/filters/RabbitMQ-Filter.md @@ -0,0 +1,61 @@ +#RabbitMQ Filter + +## Overview +This filter is designed to extract queries and transform them into a canonical form e.g. `INSERT INTO dabata.table VALUES ("John Doe", "Downtown",100,50.0);` turns into `INSERT INTO dabata.table VALUES ("?", "?",?,?);`. The filter pushes these canonized queries and their replies in to a RabbitMQ broker where they can later be retrieved. The retrieval can be done with your own application or the [RabbitMQ Consumer Client](RabbitMQ-Consumer-Client.md) utility tool, which reads the messages from the broker and sends the contents of those messages as SQL queries to a database. + +## Configuration + +The configuration block for the **mqfilter** filter requires the minimal filter options in it’s section within the MaxScale.cnf file, stored in $MAXSCALE_HOME/etc/MaxScale.cnf. Although the filter will start, it will use the default values which only work with a freshly installed RabbitMQ server and use its default values. This setup is mostly intednded for testing the filter. + +The following is an example of a mqfilter configuration in the MaxScale.cnf file used for actual logging of queries to a RabbitMQ broker on a different host. + +``` +[RabbitMQ] +type=filter +module=mqfilter +hostname=192.168.122.100 +port=4000 +username=messageuser +password=msgpwd +exchange=msg-ex-1 +key=MaxScale +logging_trigger=object,schema,source +logging_strict=false +logging_log_all=false +logging_object=my1 +logging_schema=test +logging_source_user=maxtest +``` + +### Filter Options + +The mqfilter filter does not support any filter options. + +### Filter Parameters + +The RabbitMQ filter has parameters to control which queries are logged based on either the attributes of the user or the query itself. These can be combined to to only log queries targeting a certain table in a certain database from a certain user from a certain network address. + + + Option | Description | Accepted Values | Default | +--------|-------------|-----------------|------------- + logging_trigger | Set the logging level | `all, source, schema, object` | `all` | + logging_strict | Sets whether to trigger when any of the parameters match or only if all parameters match | `true, false` | `false` | + logging_log_all | Log only SELECT, UPDATE, DELETE and INSERT or all possible queries | `true, false` | `true` | + logging_source_user | Comma-separated list of usernames to log | | | + logging_source_host | Comma-separated list of hostnames to log | | | + logging_schema | Comma-separated list of databases | | | + logging_object | Comma-separated list of database objects | + hostname | The server hostname where the messages are sent | | `localhost` | + port | Port to send the messages to | | `5672` | + username | Server login username | | `guest` | + password | Server login password | | `guest` | + vhost | The virtual host location on the server, where the messages are sent | | `/` | + exchange | The name of the exchange | | `default_exchange` | + exchange_type | The type of the exchange | `direct, fanout, topic, headers` | `direct` | + key | The routing key used when sending messages to the exchange | | `key` | + queue | The queue that will be bound to the used exchange | | | + ssl_CA_cert | Path to the CA certificate in PEM format | | | + ssl_client_cert | Path to the client cerificate in PEM format | | | + ssl_client_key | Path to the client public key in PEM format | | | + + diff --git a/client/maxadmin.c b/client/maxadmin.c index 65b7ed4a6..e78fcf307 100644 --- a/client/maxadmin.c +++ b/client/maxadmin.c @@ -614,18 +614,27 @@ char line[400]; continue; name = strtok_r(line, "=", &brkt); value = strtok_r(NULL, "=", &brkt); - if (strcmp(name, "hostname") == 0) - *hostname = strdup(value); - else if (strcmp(name, "port") == 0) - *port = strdup(value); - else if (strcmp(name, "user") == 0) - *user = strdup(value); - else if (strcmp(name, "passwd") == 0) - *passwd = strdup(value); + if (name && value) + { + if (strcmp(name, "hostname") == 0) + *hostname = strdup(value); + else if (strcmp(name, "port") == 0) + *port = strdup(value); + else if (strcmp(name, "user") == 0) + *user = strdup(value); + else if (strcmp(name, "passwd") == 0) + *passwd = strdup(value); + else + { + fprintf(stderr, "WARNING: Unrecognised " + "parameter '%s' in .maxadmin file\n", name); + } + } else { - fprintf(stderr, "WARNING: Unrecognised " - "parameter '%s' in .maxadmin file\n", name); + fprintf(stderr, "WARNING: Expected name=value " + "parameters in .maxadmin file but found " + "'%s'.\n", line); } } fclose(fp); diff --git a/macros.cmake b/macros.cmake index fc097d241..0bac0be23 100644 --- a/macros.cmake +++ b/macros.cmake @@ -95,6 +95,7 @@ macro(check_dirs) if(DEFINED MYSQL_DIR) debugmsg("Searching for MySQL headers at: ${MYSQL_DIR}") + list(APPEND CMAKE_INCLUDE_PATH ${MYSQL_DIR}) find_path(MYSQL_DIR_LOC mysql.h PATHS ${MYSQL_DIR} PATH_SUFFIXES mysql mariadb NO_DEFAULT_PATH) else() find_path(MYSQL_DIR_LOC mysql.h PATH_SUFFIXES mysql mariadb) diff --git a/server/modules/include/blr.h b/server/modules/include/blr.h index f8b8d3139..82e4f886f 100644 --- a/server/modules/include/blr.h +++ b/server/modules/include/blr.h @@ -250,6 +250,8 @@ typedef struct router_instance { char *user; /*< User name to use with master */ char *password; /*< Password to use with master */ char *fileroot; /*< Root of binlog filename */ + bool master_chksum;/*< Does the master provide checksums */ + char *master_uuid; /*< UUID of the master */ DCB *master; /*< DCB for master connection */ DCB *client; /*< DCB for dummy client */ SESSION *session; /*< Fake session for master connection */ diff --git a/server/modules/routing/binlog/blr.c b/server/modules/routing/binlog/blr.c index a89307794..965bbe31c 100644 --- a/server/modules/routing/binlog/blr.c +++ b/server/modules/routing/binlog/blr.c @@ -181,6 +181,8 @@ unsigned char *defuuid; spinlock_init(&inst->binlog_lock); inst->binlog_fd = -1; + inst->master_chksum = true; + inst->master_uuid = NULL; inst->low_water = DEF_LOW_WATER; inst->high_water = DEF_HIGH_WATER; @@ -192,10 +194,12 @@ unsigned char *defuuid; inst->binlogdir = NULL; inst->heartbeat = 300; // Default is every 5 minutes + inst->user = strdup(service->credentials.name); + inst->password = strdup(service->credentials.authdata); + my_uuid_init((ulong)rand()*12345,12345); if ((defuuid = (char *)malloc(20)) != NULL) { - int i; my_uuid(defuuid); if ((inst->uuid = (char *)malloc(38)) != NULL) sprintf(inst->uuid, "%02x%02x%02x%02x-%02x%02x-%02x%02x-%02x%02x-%02x%02x%02x%02x%02x%02x", @@ -268,6 +272,10 @@ unsigned char *defuuid; { inst->password = strdup(value); } + else if (strcmp(options[i], "passwd") == 0) + { + inst->password = strdup(value); + } else if (strcmp(options[i], "master-id") == 0) { inst->masterid = atoi(value); @@ -342,10 +350,16 @@ unsigned char *defuuid; } } } - if (inst->fileroot == NULL) - inst->fileroot = strdup(BINLOG_NAME_ROOT); + } + else + { + LOGIF(LE, (skygw_log_write( + LOGFILE_ERROR, "%s: No router options supplied for binlogrouter", + service->name))); } + if (inst->fileroot == NULL) + inst->fileroot = strdup(BINLOG_NAME_ROOT); inst->active_logs = 0; inst->reconnect_pending = 0; inst->handling_threads = 0; @@ -969,6 +983,24 @@ ROUTER_INSTANCE *router = (ROUTER_INSTANCE *)instance; int error, len; char msg[85], *errmsg; + if (action == ERRACT_RESET) + { + backend_dcb->dcb_errhandle_called = false; + return; + } + + /** Don't handle same error twice on same DCB */ + if (backend_dcb->dcb_errhandle_called) + { + /** we optimistically assume that previous call succeed */ + *succp = true; + return; + } + else + { + backend_dcb->dcb_errhandle_called = true; + } + len = sizeof(error); if (router->master && getsockopt(router->master->fd, SOL_SOCKET, SO_ERROR, &error, &len) == 0 && error != 0) { diff --git a/server/modules/routing/binlog/blr_file.c b/server/modules/routing/binlog/blr_file.c index 1042fb356..9ed587638 100644 --- a/server/modules/routing/binlog/blr_file.c +++ b/server/modules/routing/binlog/blr_file.c @@ -354,6 +354,10 @@ int n; unsigned long filelen = 0; struct stat statb; + if (!file) + { + return NULL; + } if (fstat(file->fd, &statb) == 0) filelen = statb.st_size; if (pos >= filelen) diff --git a/server/modules/routing/binlog/blr_master.c b/server/modules/routing/binlog/blr_master.c index ac216c665..9d8cdbbff 100644 --- a/server/modules/routing/binlog/blr_master.c +++ b/server/modules/routing/binlog/blr_master.c @@ -79,6 +79,8 @@ void blr_extract_header(uint8_t *pkt, REP_HEADER *hdr); inline uint32_t extract_field(uint8_t *src, int bits); static void blr_log_packet(logfile_id_t file, char *msg, uint8_t *ptr, int len); static void blr_master_close(ROUTER_INSTANCE *); +static char *blr_extract_column(GWBUF *buf, int col); + static int keepalive = 1; /** @@ -314,6 +316,7 @@ char query[128]; "Invalid master state machine state (%d) for binlog router.", router->master_state))); gwbuf_consume(buf, gwbuf_length(buf)); + spinlock_acquire(&router->lock); if (router->reconnect_pending) { @@ -335,7 +338,20 @@ char query[128]; return; } - if (router->master_state != BLRM_BINLOGDUMP && MYSQL_RESPONSE_ERR(buf)) + if (router->master_state == BLRM_GTIDMODE && MYSQL_RESPONSE_ERR(buf)) + { + /* + * If we get an error response to the GTID Mode then we + * asusme the server does not support GTID modes and + * continue. The error is saved and replayed to slaves if + * they also request the GTID mode. + */ + LOGIF(LE, (skygw_log_write( + LOGFILE_ERROR, + "%s: Master server does not support GTID Mode.", + router->service->name))); + } + else if (router->master_state != BLRM_BINLOGDUMP && MYSQL_RESPONSE_ERR(buf)) { LOGIF(LE, (skygw_log_write( LOGFILE_ERROR, @@ -370,6 +386,9 @@ char query[128]; router->retry_backoff = 1; break; case BLRM_SERVERID: + { + char *val = blr_extract_column(buf, 1); + // Response to fetch of master's server-id if (router->saved_master.server_id) GWBUF_CONSUME_ALL(router->saved_master.server_id); @@ -384,6 +403,7 @@ char query[128]; router->master_state = BLRM_HBPERIOD; router->master->func.write(router->master, buf); break; + } case BLRM_HBPERIOD: // Response to set the heartbeat period if (router->saved_master.heartbeat) @@ -405,6 +425,15 @@ char query[128]; router->master->func.write(router->master, buf); break; case BLRM_CHKSUM2: + { + char *val = blr_extract_column(buf, 1); + + if (val && strncasecmp(val, "NONE", 4) == 0) + { + router->master_chksum = false; + } + if (val) + free(val); // Response to the master_binlog_checksum, should be stored if (router->saved_master.chksum2) GWBUF_CONSUME_ALL(router->saved_master.chksum2); @@ -414,6 +443,7 @@ char query[128]; router->master_state = BLRM_GTIDMODE; router->master->func.write(router->master, buf); break; + } case BLRM_GTIDMODE: // Response to the GTID_MODE, should be stored if (router->saved_master.gtid_mode) @@ -425,6 +455,10 @@ char query[128]; router->master->func.write(router->master, buf); break; case BLRM_MUUID: + { + char *val = blr_extract_column(buf, 1); + router->master_uuid = val; + // Response to the SERVER_UUID, should be stored if (router->saved_master.uuid) GWBUF_CONSUME_ALL(router->saved_master.uuid); @@ -435,6 +469,7 @@ char query[128]; router->master_state = BLRM_SUUID; router->master->func.write(router->master, buf); break; + } case BLRM_SUUID: // Response to the SET @server_uuid, should be stored if (router->saved_master.setslaveuuid) @@ -860,30 +895,33 @@ static REP_HEADER phdr; * First check that the checksum we calculate matches the * checksum in the packet we received. */ - uint32_t chksum, pktsum; - - chksum = crc32(0L, NULL, 0); - chksum = crc32(chksum, ptr + 5, hdr.event_size - 4); - pktsum = EXTRACT32(ptr + hdr.event_size + 1); - if (pktsum != chksum) + if (router->master_chksum) { - router->stats.n_badcrc++; - if (msg) + uint32_t chksum, pktsum; + + chksum = crc32(0L, NULL, 0); + chksum = crc32(chksum, ptr + 5, hdr.event_size - 4); + pktsum = EXTRACT32(ptr + hdr.event_size + 1); + if (pktsum != chksum) { - free(msg); - msg = NULL; + router->stats.n_badcrc++; + if (msg) + { + free(msg); + msg = NULL; + } + LOGIF(LE,(skygw_log_write(LOGFILE_ERROR, + "%s: Checksum error in event " + "from master, " + "binlog %s @ %d. " + "Closing master connection.", + router->service->name, + router->binlog_name, + router->binlog_position))); + blr_master_close(router); + blr_master_delayed_connect(router); + return; } - LOGIF(LE,(skygw_log_write(LOGFILE_ERROR, - "%s: Checksum error in event " - "from master, " - "binlog %s @ %d. " - "Closing master connection.", - router->service->name, - router->binlog_name, - router->binlog_position))); - blr_master_close(router); - blr_master_delayed_connect(router); - return; } router->stats.n_binlogs++; router->lastEventReceived = hdr.event_type; @@ -1384,3 +1422,59 @@ blr_master_connected(ROUTER_INSTANCE *router) { return router->master_state == BLRM_BINLOGDUMP; } + +/** + * Extract a result value from the set of messages that make up a + * MySQL response packet. + * + * @param buf The GWBUF containing the response + * @param col The column number to return + * @return The result form the column or NULL. The caller must free the result + */ +static char * +blr_extract_column(GWBUF *buf, int col) +{ +uint8_t *ptr; +int len, ncol, collen; +char *rval; + + ptr = (uint8_t *)GWBUF_DATA(buf); + /* First packet should be the column count */ + len = EXTRACT24(ptr); + ptr += 3; + if (*ptr != 1) // Check sequence number is 1 + return NULL; + ptr++; + ncol = *ptr++; + if (ncol < col) // Not that many column in result + return NULL; + // Now ptr points at the column definition + while (ncol-- > 0) + { + len = EXTRACT24(ptr); + ptr += 4; // Skip to payload + ptr += len; // Skip over payload + } + // Now we should have an EOF packet + len = EXTRACT24(ptr); + ptr += 4; // Skip to payload + if (*ptr != 0xfe) + return NULL; + ptr += len; + + // Finally we have reached the row + len = EXTRACT24(ptr); + ptr += 4; + while (--col > 0) + { + collen = *ptr++; + ptr += collen; + } + collen = *ptr++; + if ((rval = malloc(collen + 1)) == NULL) + return NULL; + memcpy(rval, ptr, collen); + rval[collen] = 0; // NULL terminate + + return rval; +} diff --git a/server/modules/routing/binlog/blr_slave.c b/server/modules/routing/binlog/blr_slave.c index dba0f7cde..bad4a196b 100644 --- a/server/modules/routing/binlog/blr_slave.c +++ b/server/modules/routing/binlog/blr_slave.c @@ -70,6 +70,7 @@ static void blr_slave_send_fde(ROUTER_INSTANCE *router, ROUTER_SLAVE *slave); static int blr_slave_send_maxscale_version(ROUTER_INSTANCE *router, ROUTER_SLAVE *slave); static int blr_slave_send_maxscale_variables(ROUTER_INSTANCE *router, ROUTER_SLAVE *slave); static int blr_slave_send_master_status(ROUTER_INSTANCE *router, ROUTER_SLAVE *slave); +static int blr_slave_send_slave_status(ROUTER_INSTANCE *router, ROUTER_SLAVE *slave); static int blr_slave_send_slave_hosts(ROUTER_INSTANCE *router, ROUTER_SLAVE *slave); static int blr_slave_send_fieldcount(ROUTER_INSTANCE *router, ROUTER_SLAVE *slave, int count); static int blr_slave_send_columndef(ROUTER_INSTANCE *router, ROUTER_SLAVE *slave, char *name, int type, int len, uint8_t seqno); @@ -330,7 +331,7 @@ int query_len; else if (strcasecmp(word, "STATUS") == 0) { free(query_text); - return blr_slave_send_master_status(router, slave); + return blr_slave_send_slave_status(router, slave); } else if (strcasecmp(word, "HOSTS") == 0) { @@ -634,6 +635,284 @@ int len, file_len; return blr_slave_send_eof(router, slave, 9); } +/* + * Columns to send for a "SHOW SLAVE STATUS" command + */ +static char *slave_status_columns[] = { + "Slave_IO_State", "Master_Host", "Master_User", "Master_Port", "Connect_Retry", + "Master_Log_File", "Read_Master_Log_Pos", "Relay_Log_File", "Relay_Log_Pos", + "Relay_Master_Log_File", "Slave_IO_Running", "Slave_SQL_Running", "Replicate_Do_DB", + "Replicate_Ignore_DB", "Replicate_Do_Table", + "Replicate_Ignore_Table", "Replicate_Wild_Do_Table", "Replicate_Wild_Ignore_Table", + "Last_Errno", "Last_Error", "Skip_Counter", "Exec_Master_Log_Pos", "Relay_Log_Space", + "Until_Condition", "Until_Log_File", "Until_Log_Pos", "Master_SSL_Allowed", + "Master_SSL_CA_File", "Master_SSL_CA_Path", "Master_SSL_Cert", "Master_SSL_Cipher", + "Master_SSL_Key", + "Seconds_Behind_Master", "Last_IO_Errno", "Last_IO_Error", "Last_SQL_Errno", + "Last_SQL_Error", "Replicate_Ignore_Server_Ids", "Master_Server_Id", "Master_UUID", + "Master_Info_File", "SQL_Delay", "SQL_Remaining_Delay", "Slave_SQL_Running_State", + "Master_Retry_Count", "Master_Bind", "Last_IO_Error_TimeStamp", + "Last_SQL_Error_Timestamp", "Master_SSL_Crl", "Master_SSL_Crlpath", + "Retrieved_Gtid_Set", "Executed_Gtid_Set", "Auto_Position", NULL +}; + +/** + * Send the response to the SQL command "SHOW SLAVE STATUS" + * + * @param router The binlog router instance + * @param slave The slave server to which we are sending the response + * @return Non-zero if data was sent + */ +static int +blr_slave_send_slave_status(ROUTER_INSTANCE *router, ROUTER_SLAVE *slave) +{ +GWBUF *pkt; +char column[42]; +uint8_t *ptr; +int len, actual_len, col_len, seqno, ncols, i; + + /* Count the columns */ + for (ncols = 0; slave_status_columns[ncols]; ncols++); + + blr_slave_send_fieldcount(router, slave, ncols); + seqno = 2; + for (i = 0; slave_status_columns[i]; i++) + blr_slave_send_columndef(router, slave, slave_status_columns[i], 0xf, 40, seqno++); + blr_slave_send_eof(router, slave, seqno++); + + len = 5 + (ncols * 41); // Max length + if ((pkt = gwbuf_alloc(len)) == NULL) + return 0; + ptr = GWBUF_DATA(pkt); + encode_value(ptr, len - 4, 24); // Add length of data packet + ptr += 3; + *ptr++ = seqno++; // Sequence number in response + + sprintf(column, "%s", blrm_states[router->master_state]); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + sprintf(column, "%s", router->master->remote ? router->master->remote : ""); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + sprintf(column, "%s", router->user ? router->user : ""); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + sprintf(column, "%d", router->service->dbref->server->port); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + sprintf(column, "%d", 60); // Connect retry + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + sprintf(column, "%s", router->binlog_name); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + sprintf(column, "%ld", router->binlog_position); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + /* We have no relay log, we relay the binlog, so we will send the same data */ + sprintf(column, "%s", router->binlog_name); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + sprintf(column, "%ld", router->binlog_position); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + /* We have no relay log, we relay the binlog, so we will send the same data */ + sprintf(column, "%s", router->binlog_name); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + strcpy(column, "Yes"); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + strcpy(column, "Yes"); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + *ptr++ = 0; // Send 6 empty values + *ptr++ = 0; + *ptr++ = 0; + *ptr++ = 0; + *ptr++ = 0; + *ptr++ = 0; + + /* Last error information */ + sprintf(column, "%d", 0); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + *ptr++ = 0; + + /* Skip_Counter */ + sprintf(column, "%d", 0); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + sprintf(column, "%ld", router->binlog_position); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + sprintf(column, "%ld", router->binlog_position); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + strcpy(column, "None"); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + *ptr++ = 0; + + /* Until_Log_Pos */ + sprintf(column, "%d", 0); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + /* Master_SSL_Allowed */ + strcpy(column, "No"); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + *ptr++ = 0; // Empty SSL columns + *ptr++ = 0; + *ptr++ = 0; + *ptr++ = 0; + *ptr++ = 0; + + /* Seconds_Behind_Master */ + sprintf(column, "%d", 0); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + /* Master_SSL_Verify_Server_Cert */ + strcpy(column, "No"); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + /* Last_IO_Error */ + sprintf(column, "%d", 0); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + *ptr++ = 0; + + /* Last_SQL_Error */ + sprintf(column, "%d", 0); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + *ptr++ = 0; + + *ptr++ = 0; + + sprintf(column, "%s", router->master_uuid ? + router->master_uuid : router->uuid); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + *ptr++ = 0; + + /* SQL_Delay*/ + sprintf(column, "%d", 0); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + *ptr++ = 0xfb; // NULL value + + /* Slave_Running_State */ + strcpy(column, "Slave running"); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + /* Master_Retry_Count */ + sprintf(column, "%d", 1000); + col_len = strlen(column); + *ptr++ = col_len; // Length of result string + strncpy((char *)ptr, column, col_len); // Result string + ptr += col_len; + + *ptr++ = 0; // Send 5 empty values + *ptr++ = 0; + *ptr++ = 0; + *ptr++ = 0; + *ptr++ = 0; + + // No GTID support send empty values + *ptr++ = 0; + *ptr++ = 0; + *ptr++ = 0; + *ptr++ = 0; + + actual_len = ptr - (uint8_t *)GWBUF_DATA(pkt); + ptr = GWBUF_DATA(pkt); + encode_value(ptr, actual_len - 4, 24); // Add length of data packet + + pkt = gwbuf_rtrim(pkt, len - actual_len); // Trim the buffer to the actual size + + slave->dcb->func.write(slave->dcb, pkt); + return blr_slave_send_eof(router, slave, seqno++); +} + /** * Send the response to the SQL command "SHOW SLAVE HOSTS" * @@ -673,7 +952,7 @@ ROUTER_SLAVE *sptr; sprintf(host, "%s", sptr->hostname ? sptr->hostname : ""); sprintf(port, "%d", sptr->port); sprintf(master_id, "%d", router->serverid); - sprintf(slave_uuid, "%s", sptr->uuid); + sprintf(slave_uuid, "%s", sptr->uuid ? sptr->uuid : ""); len = 5 + strlen(server_id) + strlen(host) + strlen(port) + strlen(master_id) + strlen(slave_uuid) + 5; if ((pkt = gwbuf_alloc(len)) == NULL) @@ -800,6 +1079,15 @@ uint32_t chksum; ptr = GWBUF_DATA(queue); len = extract_field(ptr, 24); binlognamelen = len - 11; + if (binlognamelen > BINLOG_FNAMELEN) + { + LOGIF(LE, (skygw_log_write( + LOGFILE_ERROR, + "blr_slave_binlog_dump truncating binlog filename " + "from %d to %d", + binlognamelen, BINLOG_FNAMELEN))); + binlognamelen = BINLOG_FNAMELEN; + } ptr += 4; // Skip length and sequence number if (*ptr++ != COM_BINLOG_DUMP) { @@ -819,6 +1107,13 @@ uint32_t chksum; strncpy(slave->binlogfile, (char *)ptr, binlognamelen); slave->binlogfile[binlognamelen] = 0; + LOGIF(LD, (skygw_log_write( + LOGFILE_DEBUG, + "%s: COM_BINLOG_DUMP: binlog name '%s', length %d, " + "from position %d.", router->service->name, + slave->binlogfile, binlognamelen, + slave->binlog_pos))); + slave->seqno = 1;