diff --git a/Documentation/About/Limitations.md b/Documentation/About/Limitations.md index c7ad8f75a..60b1734a3 100644 --- a/Documentation/About/Limitations.md +++ b/Documentation/About/Limitations.md @@ -21,6 +21,35 @@ collect columns, functions and tables used in the `SELECT` defining the Consequently, the database firewall will **not** block `WITH` statements where the `SELECT` of the `WITH` clause refers to forbidden columns. +## Query Classification + +Follow the [MXS-1350](https://jira.mariadb.org/browse/MXS-1350) Jira issue +to track the progress on this limitation. + +XA transactions are not detected as transactions by MaxScale. This means +that all XA commands will be treated as unknown commands and will be +treated as operations that potentially modify the database (in the case of +readwritesplit, the statements are routed to the master). + +MaxScale **will not** track the XA transaction state which means that any +SELECT queries done inside an XA transaction can be routed to servers that +are not part of the XA transaction. + +This limitation can be avoided on the client side by disabling autocommit +before any XA transactions are done. The following example shows how a +simple XA transaction is done via MaxScale by disabling autocommit for the +duration of the XA transaction. + +``` +SET autocommit=0; +XA START 'MyXA'; +INSERT INTO test.t1 VALUES(1); +XA END 'MyXA'; +XA PREPARE 'MyXA'; +XA COMMIT 'MyXA'; +SET autocommit=1; +``` + ## Prepared Statements For its proper functioning, MaxScale needs in general to be aware of the @@ -112,6 +141,14 @@ The avrorouter does not support the following data types and conversions: The avrorouter does not do any crash recovery. This means that the avro files need to be truncated to valid block lengths before starting the avrorouter. +#### Binlog Checksums + +The avrorouter does not support binlog checksums. They must must not be used in +any of the binlogs that the avrorouter will process. + +Follow [MXS-1341](https://jira.mariadb.org/browse/MXS-1341) for progress +on this issue. + ### Limitations in the connection router (readconnroute) If Master changes (ie. new Master promotion) during current connection, the @@ -164,12 +201,6 @@ this option for better performance. For more information, read the [ReadWriteSplit](../Routers/ReadWriteSplit.md) router documentation. -#### Parsing limitations - -Galera Cluster variables, such as `@@wsrep_node_name`, are not resolved by the -embedded MariaDB parser. This usually means that the query will be routed to the -master. - #### Limitations in client session handling Some of the queries that a client sends are routed to all backends instead of diff --git a/Documentation/Changelog.md b/Documentation/Changelog.md index 84f55e6bc..7a33aeb63 100644 --- a/Documentation/Changelog.md +++ b/Documentation/Changelog.md @@ -21,6 +21,7 @@ * MaxScale now supports IPv6 For more details, please refer to: +* [MariaDB MaxScale 2.1.6 Release Notes](Release-Notes/MaxScale-2.1.6-Release-Notes.md) * [MariaDB MaxScale 2.1.5 Release Notes](Release-Notes/MaxScale-2.1.5-Release-Notes.md) * [MariaDB MaxScale 2.1.4 Release Notes](Release-Notes/MaxScale-2.1.4-Release-Notes.md) * [MariaDB MaxScale 2.1.3 Release Notes](Release-Notes/MaxScale-2.1.3-Release-Notes.md) diff --git a/Documentation/Release-Notes/MaxScale-2.1.6-Release-Notes.md b/Documentation/Release-Notes/MaxScale-2.1.6-Release-Notes.md new file mode 100644 index 000000000..225d6d5fd --- /dev/null +++ b/Documentation/Release-Notes/MaxScale-2.1.6-Release-Notes.md @@ -0,0 +1,52 @@ +# MariaDB MaxScale 2.1.6 Release Notes -- 2017-08-14 + +Release 2.1.6 is a GA release. + +This document describes the changes in release 2.1.6, when compared to +release [2.1.5](MaxScale-2.1.5-Release-Notes.md). + +If you are upgrading from release 2.0, please also read the following +release notes: +[2.1.5](./MaxScale-2.1.5-Release-Notes.md) +[2.1.4](./MaxScale-2.1.4-Release-Notes.md) +[2.1.3](./MaxScale-2.1.3-Release-Notes.md) +[2.1.2](./MaxScale-2.1.2-Release-Notes.md) +[2.1.1](./MaxScale-2.1.1-Release-Notes.md) +[2.1.0](./MaxScale-2.1.0-Release-Notes.md) + +For any problems you encounter, please consider submitting a bug +report at [Jira](https://jira.mariadb.org). + +## Bug fixes + +[Here is a list of bugs fixed in MaxScale 2.1.6.](https://jira.mariadb.org/issues/?jql=project%20%3D%20MXS%20AND%20issuetype%20%3D%20Bug%20AND%20status%20%3D%20Closed%20AND%20fixVersion%20%3D%202.1.6) + +* [MXS-1352](https://jira.mariadb.org/browse/MXS-1352) Not all query failures in monitors are reported +* [MXS-1351](https://jira.mariadb.org/browse/MXS-1351) Partially authenticated connections are put into the connection pool +* (MXS-1343)[https://jira.mariadb.org/browse/MXS-1343] MaxScale's binlogrouter does not send hostname to its master +* [MXS-1338](https://jira.mariadb.org/browse/MXS-1338) Buffer objects are bound to indiviudual buffers + +## New Features + +* It is now possible to configure the binlog router to identify itself +to the master using a custom hostname: [MXS-1343](https://jira.mariadb.org/browse/MXS-1343) + +## Known Issues and Limitations + +There are some limitations and known issues within this version of MaxScale. +For more information, please refer to the [Limitations](../About/Limitations.md) document. + +## Packaging + +RPM and Debian packages are provided for the Linux distributions supported +by MariaDB Enterprise. + +Packages can be downloaded [here](https://mariadb.com/resources/downloads). + +## Source Code + +The source code of MaxScale is tagged at GitHub with a tag, which is identical +with the version of MaxScale. For instance, the tag of version X.Y.Z of MaxScale +is maxscale-X.Y.Z. + +The source code is available [here](https://github.com/mariadb-corporation/MaxScale). diff --git a/Documentation/Routers/Binlogrouter.md b/Documentation/Routers/Binlogrouter.md index fb46969e6..358af4e4d 100644 --- a/Documentation/Routers/Binlogrouter.md +++ b/Documentation/Routers/Binlogrouter.md @@ -93,6 +93,14 @@ version of the real master. This option allows the router to use a custom versio By default, the router will identify itself to the slaves using the hostname of the real master. This option allows the router to use a custom hostname. +### `slave_hostname` + +Since MaxScale 2.1.6 the router can optionally identify itself +to the master using a custom hostname. +The specified hostname can be seen in the master via +`SHOW SLAVE HOSTS` command. +The default is not to send any hostname string during registration. + ### `user` This is the user name that MariaDB MaxScale uses when it connects to the @@ -271,7 +279,25 @@ follows. version_string=5.6.17-log user=maxscale passwd=Mhu87p2D - router_options=uuid=f12fcb7f-b97b-11e3-bc5e-0401152c4c22,server_id=3,user=repl,password=slavepass,master_id=32,heartbeat=30,binlogdir=/var/binlogs,transaction_safety=1,master_version=5.6.19-common,master_hostname=common_server,master_uuid=xxx-fff-cccc-common,mariadb10-compatibility=1,send_slave_heartbeat=1,ssl_cert_verification_depth=9,semisync=1,encrypt_binlog=1,encryption_algorithm=aes_ctr,encryption_key_file=/var/binlogs/enc_key.txt + router_options=uuid=f12fcb7f-b97b-11e3-bc5e-0401152c4c22, + server_id=3, + user=repl, + password=slavepass, + master_id=32, + heartbeat=30, + binlogdir=/var/binlogs, + transaction_safety=1, + master_version=5.6.19-common, + master_hostname=common_server, + master_uuid=xxx-fff-cccc-common, + mariadb10-compatibility=1, + send_slave_heartbeat=1, + ssl_cert_verification_depth=9, + semisync=1, + encrypt_binlog=1, + encryption_algorithm=aes_ctr, + encryption_key_file=/var/binlogs/enc_key.txt, + slave_hostname=maxscale-blr-1 ``` The minimum set of router options that must be given in the configuration are diff --git a/Documentation/Tutorials/Replication-Proxy-Binlog-Router-Tutorial.md b/Documentation/Tutorials/Replication-Proxy-Binlog-Router-Tutorial.md index 29fd496a3..f5df5ed6c 100644 --- a/Documentation/Tutorials/Replication-Proxy-Binlog-Router-Tutorial.md +++ b/Documentation/Tutorials/Replication-Proxy-Binlog-Router-Tutorial.md @@ -32,134 +32,8 @@ Using MariaDB MaxScale as a Binlog Server is much the same as using MariaDB MaxS As with any MariaDB MaxScale configuration a good starting point is with the service definition with the *maxscale.cnf* file. The service requires a name which is the section name in the ini file, a type parameter with a value of service and the name of the router plugin that should be loaded. In the case of replication proxies this router name is *binlogrouter*. -``` -[Replication] -type=service -router=binlogrouter -``` -Other standard service parameters need to be given in the configuration section that are used to retrieve the set of users from the backend (master) database, also a version string can be given such that the MariaDB MaxScale instance will report this version string to the slave servers that connect to MariaDB MaxScale. - -``` -[Replication] -type=service -router=binlogrouter -version_string=5.6.17-log -user=maxscale -passwd=Mhu87p2D -``` - -The *user* and *passwd* entries in the above example are used in order for MariaDB MaxScale to populate the credential information that is required to allow the slaves to connect to MariaDB MaxScale. This user should be configured in exactly the same way a for any other MariaDB MaxScale service, i.e. the user needs access to the *mysql.user* table and the *mysql.db* table as well as having the ability to perform a *SHOW DATABASES* command. - -This user is the only one available for MySQL connection to MaxScale Binlog Server for administration when master connection is not done yet. - -The master server details are currently provided by a **master.ini** file located in binlog directory and could be changed via *CHANGE MASTER TO* command issued via MySQL connection to MariaDB MaxScale; refer to the Master setup section below for further details. - -In the current implementation of the router only a single server can be used. - -The final configuration requirement is the router specific options. The binlog router requires a set of parameters to be passed, these are passed, as a comma separated list of name value pairs, in the *router_options* parameter of the service definition.. - -### binlogdir - -This parameter allows the location that MariaDB MaxScale uses to store binlog files to be set. If this parameter is not set to a directory name then MariaDB MaxScale will store the binlog files in the directory */var/cache/maxscale/*. -In the binlog dir there is also the 'cache' directory that contains data retrieved from the master during registration phase and the *master.ini* file which contains the configuration of current configured master. - -### uuid - -This is used to set the unique uuid that the binlog router uses when it connects to the master server. -If no explicit value is given for the uuid in the configuration file then a uuid will be generated. - -### server-id - -As with uuid, MariaDB MaxScale must have a unique server-id for the connection it makes to the master, this parameter provides the value of server-id that MariaDB MaxScale will use when connecting to the master. - -### master-id - -The server-id value that MariaDB MaxScale should use to report to the slaves that connect to MariaDB MaxScale. -This may either be the same as the server-id of the real master or can be chosen to be different if the slaves need to be aware of the proxy layer. -The real master server-id will be used if the option is not set. - -### master_uuid - -It is a requirement of replication that each slave have a unique UUID value. The MariaDB MaxScale router will identify itself to the slaves using the uuid of the real master if this option is not set. - -### master_version - -The MariaDB MaxScale router will identify itself to the slaves using the server version of the real master if this option is not set. - -### master_hostname - -The MariaDB MaxScale router will identify itself to the slaves using the server hostname of the real master if this option is not set. - -### user - -This is the user name that MariaDB MaxScale uses when it connects to the master. This user name must have the rights required for replication as with any other user that a slave uses for replication purposes. If the user parameter is not given in the router options then the same user as is used to retrieve the credential information will be used for the replication connection, i.e. the user in the service entry. - -This user is also the only one available for Binlog Server administration when the connection with master is not ready yet: the 'master.ini' file doesn't exists and no other users are available for authentication. - -The user that is used for replication, either defined using the *user=* option in the router options or using the username and password defined of the service must be granted replication privileges on the database server. - -``` - MariaDB> CREATE USER 'repl'@'maxscalehost' IDENTIFIED by 'password'; - MariaDB> GRANT REPLICATION SLAVE ON *.* TO 'repl'@'maxscalehost'; -``` - -### password - -The password of the above user. If the password is not explicitly given then the password in the service entry will be used. For compatibility with other username and password definitions within the MariaDB MaxScale configuration file it is also possible to use the parameter *passwd=*. - -### heartbeat - -This defines the value of the heartbeat interval in seconds for the connection to the master. MariaDB MaxScale requests the master to ensure that a binlog event is sent at least every heartbeat period. If there are no real binlog events to send the master will sent a special heartbeat event. The default value for the heartbeat period is every 5 minutes. The current interval value is reported in the diagnostic output. - -### send_slave_heartbeat - -This defines whether (on | off) MariaDB MaxScale sends to the slave the heartbeat packet when there are no real binlog events to send. The default value if 'off', no heartbeat event is sent to slave server. If value is 'on' the interval value (requested by the slave during registration) is reported in the diagnostic output and the packet is send after the time interval without any event to send. - -### burstsize - -This parameter is used to define the maximum amount of data that will be sent to a slave by MariaDB MaxScale when that slave is lagging behind the master. In this situation the slave is said to be in "catchup mode", this parameter is designed to both prevent flooding of that slave and also to prevent threads within MariaDB MaxScale spending disproportionate amounts of time with slaves that are lagging behind the master. The burst size can be defined in Kb, Mb or Gb by adding the qualifier K, M or G to the number given. The default value of burstsize is 1Mb and will be used if burstsize is not given in the router options. - -### transaction_safety - -This parameter is used to enable/disable incomplete transactions detection in binlog router. -When MariaDB MaxScale starts an error message may appear if current binlog file is corrupted or an incomplete transaction is found. -During normal operations binlog events are not distributed to the slaves until a *COMMIT* is seen. -The default value is off, set *transaction_safety=on* to enable the incomplete transactions detection. - -### semisync - -This parameter controls whether binlog server could ask Master server to start the Semi-Synchronous replication. -In order to get semi-sync working the Master server must have the *rpl_semi_sync_master* plugin installed. -The availability of the plugin and the value of the GLOBAL VARIABLE *rpl_semi_sync_master_enabled* are checked in the Master registration phase: if the plugin is installed in the Master database the binlog server subsequently requests the semi-sync option. - -Note: - - the network replication stream from Master has two additional bytes before each binlog event. - - the Semi-Sync protocol requires an acknowledge packet to be sent back to Master only when requested: the semi-sync flag will have value of 1. - This flag is set only if *rpl_semi_sync_master_enabled=1* in the Master, otherwise it will always have value of 0 and no ack packet is sent back. - -Please note that semi-sync replication is only related to binlog server to Master communication. - - -### ssl_cert_verification_depth - -This parameter sets the maximum length of the certificate authority chain that will be accepted. Legal values are positive integers. -This applies to SSL connection to master server that could be acivated either by writing options in master.ini or later via CHANGE MASTER TO. -This parameter cannot be modified at runtime, default is 9. - -### `encrypt_binlog` -Whether to encrypt binlog files: the default is Off - -When set to On the binlog files will be encrypted using specified AES algorithm and the KEY in the specified key file. - -### `encryption_algorithm` -aes_ctr or aes_cbc - -The default is 'aes_cbc' - -### `encryption_key_file` -The specified key file must have this format: -a line with `1;HEX(KEY)` +The minimum set of router options that must be given in the configuration are are *server-id* and *master-id*, default values may be used for all other options. Additional information about the encryption of the Binlog files can be found here: [Binlogrouter - The replication protocol proxy module for MariaDB MaxScale](../Routers/Binlogrouter.md). diff --git a/Documentation/Upgrading/Upgrading-To-MaxScale-2.1.md b/Documentation/Upgrading/Upgrading-To-MaxScale-2.1.md index a07949329..a55cefa03 100644 --- a/Documentation/Upgrading/Upgrading-To-MaxScale-2.1.md +++ b/Documentation/Upgrading/Upgrading-To-MaxScale-2.1.md @@ -7,6 +7,7 @@ For more information about MariaDB MaxScale 2.1, please refer to the [ChangeLog](../Changelog.md). For a complete list of changes in MaxScale 2.1, refer to the +[MaxScale 2.1.6 Release Notes](../Release-Notes/MaxScale-2.1.6-Release-Notes.md). [MaxScale 2.1.5 Release Notes](../Release-Notes/MaxScale-2.1.5-Release-Notes.md). [MaxScale 2.1.4 Release Notes](../Release-Notes/MaxScale-2.1.4-Release-Notes.md). [MaxScale 2.1.3 Release Notes](../Release-Notes/MaxScale-2.1.3-Release-Notes.md). diff --git a/Documentation/list_fixed_bugs.sh b/Documentation/list_fixed_bugs.sh new file mode 100755 index 000000000..bc7ab2b60 --- /dev/null +++ b/Documentation/list_fixed_bugs.sh @@ -0,0 +1,11 @@ +#!/bin/bash + + +if [ $# -ne 1 ] +then + echo "USAGE: $0 VERSION" + exit 1 +fi + +version=$1 +curl -s "https://jira.mariadb.org/sr/jira.issueviews:searchrequest-csv-current-fields/temp/SearchRequest.csv?jqlQuery=project+%3D+MXS+AND+status+%3D+Closed+AND+fixVersion+%3D+$version"|cut -f 1,2 -d ,|tail -n+2|sed 's/\(.*\),\(.*\)/* (\2)[https:\/\/jira.mariadb.org\/browse\/\2] \1/' diff --git a/VERSION21.cmake b/VERSION21.cmake index 56646f85c..c507fa2aa 100644 --- a/VERSION21.cmake +++ b/VERSION21.cmake @@ -5,7 +5,7 @@ set(MAXSCALE_VERSION_MAJOR "2" CACHE STRING "Major version") set(MAXSCALE_VERSION_MINOR "1" CACHE STRING "Minor version") -set(MAXSCALE_VERSION_PATCH "5" CACHE STRING "Patch version") +set(MAXSCALE_VERSION_PATCH "6" CACHE STRING "Patch version") # This should only be incremented if a package is rebuilt set(MAXSCALE_BUILD_NUMBER 1 CACHE STRING "Release number") diff --git a/server/modules/routing/binlogrouter/blr.c b/server/modules/routing/binlogrouter/blr.c index 1f19f0850..1e8702d25 100644 --- a/server/modules/routing/binlogrouter/blr.c +++ b/server/modules/routing/binlogrouter/blr.c @@ -211,6 +211,7 @@ MXS_MODULE* MXS_CREATE_MODULE() {"master_uuid", MXS_MODULE_PARAM_STRING}, {"master_version", MXS_MODULE_PARAM_STRING}, {"master_hostname", MXS_MODULE_PARAM_STRING}, + {"slave_hostname", MXS_MODULE_PARAM_STRING}, {"mariadb10-compatibility", MXS_MODULE_PARAM_BOOL, "false"}, {"maxwell-compatibility", MXS_MODULE_PARAM_BOOL, "false"}, {"filestem", MXS_MODULE_PARAM_STRING, BINLOG_NAME_ROOT}, @@ -372,6 +373,7 @@ createInstance(SERVICE *service, char **options) inst->trx_safe = config_get_bool(params, "transaction_safety"); inst->set_master_version = config_copy_string(params, "master_version"); inst->set_master_hostname = config_copy_string(params, "master_hostname"); + inst->set_slave_hostname = config_copy_string(params, "slave_hostname"); inst->fileroot = config_copy_string(params, "filestem"); inst->serverid = config_get_integer(params, "server_id"); @@ -382,8 +384,6 @@ createInstance(SERVICE *service, char **options) inst->master_uuid = config_copy_string(params, "master_uuid"); inst->set_master_uuid = inst->master_uuid != NULL; - inst->set_master_version = NULL; - inst->set_master_hostname = NULL; inst->send_slave_heartbeat = config_get_bool(params, "send_slave_heartbeat"); /* Semi-Sync support */ @@ -538,6 +538,11 @@ createInstance(SERVICE *service, char **options) MXS_FREE(inst->set_master_hostname); inst->set_master_hostname = MXS_STRDUP_A(value); } + else if (strcmp(options[i], "slave_hostname") == 0) + { + MXS_FREE(inst->set_slave_hostname); + inst->set_slave_hostname = MXS_STRDUP_A(value); + } else if (strcmp(options[i], "mariadb10-compatibility") == 0) { inst->mariadb10_compat = config_truth_value(value); @@ -1154,6 +1159,7 @@ free_instance(ROUTER_INSTANCE *instance) MXS_FREE(instance->password); MXS_FREE(instance->set_master_version); MXS_FREE(instance->set_master_hostname); + MXS_FREE(instance->set_slave_hostname); MXS_FREE(instance->fileroot); MXS_FREE(instance->binlogdir); /* SSL options */ diff --git a/server/modules/routing/binlogrouter/blr.h b/server/modules/routing/binlogrouter/blr.h index babaa31bd..6d33af7f0 100644 --- a/server/modules/routing/binlogrouter/blr.h +++ b/server/modules/routing/binlogrouter/blr.h @@ -731,6 +731,7 @@ typedef struct router_instance uint32_t mariadb10_gtid_domain;/*< MariaDB 10 GTID Domain ID */ sqlite3 *gtid_maps; /*< MariaDB 10 GTID storage */ enum binlog_storage_type storage_type;/*< Enables hierachical binlog file storage */ + char *set_slave_hostname; /*< Send custom Hostname to Master */ struct router_instance *next; } ROUTER_INSTANCE; diff --git a/server/modules/routing/binlogrouter/blr_master.c b/server/modules/routing/binlogrouter/blr_master.c index 6d895bb0a..b9816219e 100644 --- a/server/modules/routing/binlogrouter/blr_master.c +++ b/server/modules/routing/binlogrouter/blr_master.c @@ -616,28 +616,55 @@ blr_make_registration(ROUTER_INSTANCE *router) { GWBUF *buf; unsigned char *data; - int len = 18; + int len = 18; // Min size of COM_REGISTER_SLAVE payload int port = 3306; + int hostname_len = 0; + + // Send router->set_slave_hostname + if (router->set_slave_hostname && router->set_slave_hostname[0]) + { + hostname_len = strlen(router->set_slave_hostname); + } + + // Add hostname len + len += hostname_len; if ((buf = gwbuf_alloc(len + MYSQL_HEADER_LEN)) == NULL) { return NULL; } + data = GWBUF_DATA(buf); encode_value(&data[0], len, 24); // Payload length data[3] = 0; // Sequence ID data[4] = COM_REGISTER_SLAVE; // Command encode_value(&data[5], router->serverid, 32); // Slave Server ID - data[9] = 0; // Slave hostname length - data[10] = 0; // Slave username length - data[11] = 0; // Slave password length + + // Point to hostname len offset + data += 9; + + *data++ = hostname_len; // Slave hostname length + + // Copy hostname + if (hostname_len) + { + memcpy(data, router->set_slave_hostname, hostname_len); + } + + // Point to user + data += hostname_len; + // Set empty user + *data++ = 0; // Slave username length + // Set empty password + *data++ = 0; // Slave password length + // Add port if (router->service->ports) { port = router->service->ports->port; } - encode_value(&data[12], port, 16); // Slave master port - encode_value(&data[14], 0, 32); // Replication rank - encode_value(&data[18], router->masterid, 32); // Master server-id + encode_value(&data[0], port, 16); // Slave master port, 2 bytes + encode_value(&data[2], 0, 32); // Replication rank, 4 bytes + encode_value(&data[6], router->masterid, 32); // Master server-id, 4 bytes // This is hack to get the result set processing in order for binlogrouter MySQLProtocol *proto = (MySQLProtocol*)router->master->protocol; @@ -1891,10 +1918,15 @@ static void blr_log_identity(ROUTER_INSTANCE *router) /* Seen by the master */ MXS_NOTICE("%s: identity seen by the master: " - "server_id: %d, uuid: %s", + "Server_id: %d, Slave_UUID: %s, Host: %s", router->service->name, router->serverid, - (router->uuid == NULL ? "not available" : router->uuid)); + router->uuid == NULL ? + "not available" : + router->uuid, + (router->set_slave_hostname && router->set_slave_hostname[0]) ? + router->set_slave_hostname : + "not set"); /* Seen by the slaves */