Merge branch '2.1' into develop

This commit is contained in:
Johan Wikman 2017-08-14 10:36:34 +03:00
commit 4798810f76
11 changed files with 181 additions and 146 deletions

View File

@ -21,6 +21,35 @@ collect columns, functions and tables used in the `SELECT` defining the
Consequently, the database firewall will **not** block `WITH` statements
where the `SELECT` of the `WITH` clause refers to forbidden columns.
## Query Classification
Follow the [MXS-1350](https://jira.mariadb.org/browse/MXS-1350) Jira issue
to track the progress on this limitation.
XA transactions are not detected as transactions by MaxScale. This means
that all XA commands will be treated as unknown commands and will be
treated as operations that potentially modify the database (in the case of
readwritesplit, the statements are routed to the master).
MaxScale **will not** track the XA transaction state which means that any
SELECT queries done inside an XA transaction can be routed to servers that
are not part of the XA transaction.
This limitation can be avoided on the client side by disabling autocommit
before any XA transactions are done. The following example shows how a
simple XA transaction is done via MaxScale by disabling autocommit for the
duration of the XA transaction.
```
SET autocommit=0;
XA START 'MyXA';
INSERT INTO test.t1 VALUES(1);
XA END 'MyXA';
XA PREPARE 'MyXA';
XA COMMIT 'MyXA';
SET autocommit=1;
```
## Prepared Statements
For its proper functioning, MaxScale needs in general to be aware of the
@ -112,6 +141,14 @@ The avrorouter does not support the following data types and conversions:
The avrorouter does not do any crash recovery. This means that the avro files
need to be truncated to valid block lengths before starting the avrorouter.
#### Binlog Checksums
The avrorouter does not support binlog checksums. They must must not be used in
any of the binlogs that the avrorouter will process.
Follow [MXS-1341](https://jira.mariadb.org/browse/MXS-1341) for progress
on this issue.
### Limitations in the connection router (readconnroute)
If Master changes (ie. new Master promotion) during current connection, the
@ -164,12 +201,6 @@ this option for better performance.
For more information, read the
[ReadWriteSplit](../Routers/ReadWriteSplit.md) router documentation.
#### Parsing limitations
Galera Cluster variables, such as `@@wsrep_node_name`, are not resolved by the
embedded MariaDB parser. This usually means that the query will be routed to the
master.
#### Limitations in client session handling
Some of the queries that a client sends are routed to all backends instead of

View File

@ -21,6 +21,7 @@
* MaxScale now supports IPv6
For more details, please refer to:
* [MariaDB MaxScale 2.1.6 Release Notes](Release-Notes/MaxScale-2.1.6-Release-Notes.md)
* [MariaDB MaxScale 2.1.5 Release Notes](Release-Notes/MaxScale-2.1.5-Release-Notes.md)
* [MariaDB MaxScale 2.1.4 Release Notes](Release-Notes/MaxScale-2.1.4-Release-Notes.md)
* [MariaDB MaxScale 2.1.3 Release Notes](Release-Notes/MaxScale-2.1.3-Release-Notes.md)

View File

@ -0,0 +1,52 @@
# MariaDB MaxScale 2.1.6 Release Notes -- 2017-08-14
Release 2.1.6 is a GA release.
This document describes the changes in release 2.1.6, when compared to
release [2.1.5](MaxScale-2.1.5-Release-Notes.md).
If you are upgrading from release 2.0, please also read the following
release notes:
[2.1.5](./MaxScale-2.1.5-Release-Notes.md)
[2.1.4](./MaxScale-2.1.4-Release-Notes.md)
[2.1.3](./MaxScale-2.1.3-Release-Notes.md)
[2.1.2](./MaxScale-2.1.2-Release-Notes.md)
[2.1.1](./MaxScale-2.1.1-Release-Notes.md)
[2.1.0](./MaxScale-2.1.0-Release-Notes.md)
For any problems you encounter, please consider submitting a bug
report at [Jira](https://jira.mariadb.org).
## Bug fixes
[Here is a list of bugs fixed in MaxScale 2.1.6.](https://jira.mariadb.org/issues/?jql=project%20%3D%20MXS%20AND%20issuetype%20%3D%20Bug%20AND%20status%20%3D%20Closed%20AND%20fixVersion%20%3D%202.1.6)
* [MXS-1352](https://jira.mariadb.org/browse/MXS-1352) Not all query failures in monitors are reported
* [MXS-1351](https://jira.mariadb.org/browse/MXS-1351) Partially authenticated connections are put into the connection pool
* (MXS-1343)[https://jira.mariadb.org/browse/MXS-1343] MaxScale's binlogrouter does not send hostname to its master
* [MXS-1338](https://jira.mariadb.org/browse/MXS-1338) Buffer objects are bound to indiviudual buffers
## New Features
* It is now possible to configure the binlog router to identify itself
to the master using a custom hostname: [MXS-1343](https://jira.mariadb.org/browse/MXS-1343)
## Known Issues and Limitations
There are some limitations and known issues within this version of MaxScale.
For more information, please refer to the [Limitations](../About/Limitations.md) document.
## Packaging
RPM and Debian packages are provided for the Linux distributions supported
by MariaDB Enterprise.
Packages can be downloaded [here](https://mariadb.com/resources/downloads).
## Source Code
The source code of MaxScale is tagged at GitHub with a tag, which is identical
with the version of MaxScale. For instance, the tag of version X.Y.Z of MaxScale
is maxscale-X.Y.Z.
The source code is available [here](https://github.com/mariadb-corporation/MaxScale).

View File

@ -93,6 +93,14 @@ version of the real master. This option allows the router to use a custom versio
By default, the router will identify itself to the slaves using the
hostname of the real master. This option allows the router to use a custom hostname.
### `slave_hostname`
Since MaxScale 2.1.6 the router can optionally identify itself
to the master using a custom hostname.
The specified hostname can be seen in the master via
`SHOW SLAVE HOSTS` command.
The default is not to send any hostname string during registration.
### `user`
This is the user name that MariaDB MaxScale uses when it connects to the
@ -271,7 +279,25 @@ follows.
version_string=5.6.17-log
user=maxscale
passwd=Mhu87p2D
router_options=uuid=f12fcb7f-b97b-11e3-bc5e-0401152c4c22,server_id=3,user=repl,password=slavepass,master_id=32,heartbeat=30,binlogdir=/var/binlogs,transaction_safety=1,master_version=5.6.19-common,master_hostname=common_server,master_uuid=xxx-fff-cccc-common,mariadb10-compatibility=1,send_slave_heartbeat=1,ssl_cert_verification_depth=9,semisync=1,encrypt_binlog=1,encryption_algorithm=aes_ctr,encryption_key_file=/var/binlogs/enc_key.txt
router_options=uuid=f12fcb7f-b97b-11e3-bc5e-0401152c4c22,
server_id=3,
user=repl,
password=slavepass,
master_id=32,
heartbeat=30,
binlogdir=/var/binlogs,
transaction_safety=1,
master_version=5.6.19-common,
master_hostname=common_server,
master_uuid=xxx-fff-cccc-common,
mariadb10-compatibility=1,
send_slave_heartbeat=1,
ssl_cert_verification_depth=9,
semisync=1,
encrypt_binlog=1,
encryption_algorithm=aes_ctr,
encryption_key_file=/var/binlogs/enc_key.txt,
slave_hostname=maxscale-blr-1
```
The minimum set of router options that must be given in the configuration are

View File

@ -32,134 +32,8 @@ Using MariaDB MaxScale as a Binlog Server is much the same as using MariaDB MaxS
As with any MariaDB MaxScale configuration a good starting point is with the service definition with the *maxscale.cnf* file. The service requires a name which is the section name in the ini file, a type parameter with a value of service and the name of the router plugin that should be loaded. In the case of replication proxies this router name is *binlogrouter*.
```
[Replication]
type=service
router=binlogrouter
```
Other standard service parameters need to be given in the configuration section that are used to retrieve the set of users from the backend (master) database, also a version string can be given such that the MariaDB MaxScale instance will report this version string to the slave servers that connect to MariaDB MaxScale.
```
[Replication]
type=service
router=binlogrouter
version_string=5.6.17-log
user=maxscale
passwd=Mhu87p2D
```
The *user* and *passwd* entries in the above example are used in order for MariaDB MaxScale to populate the credential information that is required to allow the slaves to connect to MariaDB MaxScale. This user should be configured in exactly the same way a for any other MariaDB MaxScale service, i.e. the user needs access to the *mysql.user* table and the *mysql.db* table as well as having the ability to perform a *SHOW DATABASES* command.
This user is the only one available for MySQL connection to MaxScale Binlog Server for administration when master connection is not done yet.
The master server details are currently provided by a **master.ini** file located in binlog directory and could be changed via *CHANGE MASTER TO* command issued via MySQL connection to MariaDB MaxScale; refer to the Master setup section below for further details.
In the current implementation of the router only a single server can be used.
The final configuration requirement is the router specific options. The binlog router requires a set of parameters to be passed, these are passed, as a comma separated list of name value pairs, in the *router_options* parameter of the service definition..
### binlogdir
This parameter allows the location that MariaDB MaxScale uses to store binlog files to be set. If this parameter is not set to a directory name then MariaDB MaxScale will store the binlog files in the directory */var/cache/maxscale/<Service Name>*.
In the binlog dir there is also the 'cache' directory that contains data retrieved from the master during registration phase and the *master.ini* file which contains the configuration of current configured master.
### uuid
This is used to set the unique uuid that the binlog router uses when it connects to the master server.
If no explicit value is given for the uuid in the configuration file then a uuid will be generated.
### server-id
As with uuid, MariaDB MaxScale must have a unique server-id for the connection it makes to the master, this parameter provides the value of server-id that MariaDB MaxScale will use when connecting to the master.
### master-id
The server-id value that MariaDB MaxScale should use to report to the slaves that connect to MariaDB MaxScale.
This may either be the same as the server-id of the real master or can be chosen to be different if the slaves need to be aware of the proxy layer.
The real master server-id will be used if the option is not set.
### master_uuid
It is a requirement of replication that each slave have a unique UUID value. The MariaDB MaxScale router will identify itself to the slaves using the uuid of the real master if this option is not set.
### master_version
The MariaDB MaxScale router will identify itself to the slaves using the server version of the real master if this option is not set.
### master_hostname
The MariaDB MaxScale router will identify itself to the slaves using the server hostname of the real master if this option is not set.
### user
This is the user name that MariaDB MaxScale uses when it connects to the master. This user name must have the rights required for replication as with any other user that a slave uses for replication purposes. If the user parameter is not given in the router options then the same user as is used to retrieve the credential information will be used for the replication connection, i.e. the user in the service entry.
This user is also the only one available for Binlog Server administration when the connection with master is not ready yet: the 'master.ini' file doesn't exists and no other users are available for authentication.
The user that is used for replication, either defined using the *user=* option in the router options or using the username and password defined of the service must be granted replication privileges on the database server.
```
MariaDB> CREATE USER 'repl'@'maxscalehost' IDENTIFIED by 'password';
MariaDB> GRANT REPLICATION SLAVE ON *.* TO 'repl'@'maxscalehost';
```
### password
The password of the above user. If the password is not explicitly given then the password in the service entry will be used. For compatibility with other username and password definitions within the MariaDB MaxScale configuration file it is also possible to use the parameter *passwd=*.
### heartbeat
This defines the value of the heartbeat interval in seconds for the connection to the master. MariaDB MaxScale requests the master to ensure that a binlog event is sent at least every heartbeat period. If there are no real binlog events to send the master will sent a special heartbeat event. The default value for the heartbeat period is every 5 minutes. The current interval value is reported in the diagnostic output.
### send_slave_heartbeat
This defines whether (on | off) MariaDB MaxScale sends to the slave the heartbeat packet when there are no real binlog events to send. The default value if 'off', no heartbeat event is sent to slave server. If value is 'on' the interval value (requested by the slave during registration) is reported in the diagnostic output and the packet is send after the time interval without any event to send.
### burstsize
This parameter is used to define the maximum amount of data that will be sent to a slave by MariaDB MaxScale when that slave is lagging behind the master. In this situation the slave is said to be in "catchup mode", this parameter is designed to both prevent flooding of that slave and also to prevent threads within MariaDB MaxScale spending disproportionate amounts of time with slaves that are lagging behind the master. The burst size can be defined in Kb, Mb or Gb by adding the qualifier K, M or G to the number given. The default value of burstsize is 1Mb and will be used if burstsize is not given in the router options.
### transaction_safety
This parameter is used to enable/disable incomplete transactions detection in binlog router.
When MariaDB MaxScale starts an error message may appear if current binlog file is corrupted or an incomplete transaction is found.
During normal operations binlog events are not distributed to the slaves until a *COMMIT* is seen.
The default value is off, set *transaction_safety=on* to enable the incomplete transactions detection.
### semisync
This parameter controls whether binlog server could ask Master server to start the Semi-Synchronous replication.
In order to get semi-sync working the Master server must have the *rpl_semi_sync_master* plugin installed.
The availability of the plugin and the value of the GLOBAL VARIABLE *rpl_semi_sync_master_enabled* are checked in the Master registration phase: if the plugin is installed in the Master database the binlog server subsequently requests the semi-sync option.
Note:
- the network replication stream from Master has two additional bytes before each binlog event.
- the Semi-Sync protocol requires an acknowledge packet to be sent back to Master only when requested: the semi-sync flag will have value of 1.
This flag is set only if *rpl_semi_sync_master_enabled=1* in the Master, otherwise it will always have value of 0 and no ack packet is sent back.
Please note that semi-sync replication is only related to binlog server to Master communication.
### ssl_cert_verification_depth
This parameter sets the maximum length of the certificate authority chain that will be accepted. Legal values are positive integers.
This applies to SSL connection to master server that could be acivated either by writing options in master.ini or later via CHANGE MASTER TO.
This parameter cannot be modified at runtime, default is 9.
### `encrypt_binlog`
Whether to encrypt binlog files: the default is Off
When set to On the binlog files will be encrypted using specified AES algorithm and the KEY in the specified key file.
### `encryption_algorithm`
aes_ctr or aes_cbc
The default is 'aes_cbc'
### `encryption_key_file`
The specified key file must have this format:
a line with `1;HEX(KEY)`
The minimum set of router options that must be given in the configuration are are *server-id* and *master-id*, default values may be used for all other options.
Additional information about the encryption of the Binlog files can be found here:
[Binlogrouter - The replication protocol proxy module for MariaDB MaxScale](../Routers/Binlogrouter.md).

View File

@ -7,6 +7,7 @@ For more information about MariaDB MaxScale 2.1, please refer to the
[ChangeLog](../Changelog.md).
For a complete list of changes in MaxScale 2.1, refer to the
[MaxScale 2.1.6 Release Notes](../Release-Notes/MaxScale-2.1.6-Release-Notes.md).
[MaxScale 2.1.5 Release Notes](../Release-Notes/MaxScale-2.1.5-Release-Notes.md).
[MaxScale 2.1.4 Release Notes](../Release-Notes/MaxScale-2.1.4-Release-Notes.md).
[MaxScale 2.1.3 Release Notes](../Release-Notes/MaxScale-2.1.3-Release-Notes.md).

View File

@ -0,0 +1,11 @@
#!/bin/bash
if [ $# -ne 1 ]
then
echo "USAGE: $0 VERSION"
exit 1
fi
version=$1
curl -s "https://jira.mariadb.org/sr/jira.issueviews:searchrequest-csv-current-fields/temp/SearchRequest.csv?jqlQuery=project+%3D+MXS+AND+status+%3D+Closed+AND+fixVersion+%3D+$version"|cut -f 1,2 -d ,|tail -n+2|sed 's/\(.*\),\(.*\)/* (\2)[https:\/\/jira.mariadb.org\/browse\/\2] \1/'

View File

@ -5,7 +5,7 @@
set(MAXSCALE_VERSION_MAJOR "2" CACHE STRING "Major version")
set(MAXSCALE_VERSION_MINOR "1" CACHE STRING "Minor version")
set(MAXSCALE_VERSION_PATCH "5" CACHE STRING "Patch version")
set(MAXSCALE_VERSION_PATCH "6" CACHE STRING "Patch version")
# This should only be incremented if a package is rebuilt
set(MAXSCALE_BUILD_NUMBER 1 CACHE STRING "Release number")

View File

@ -211,6 +211,7 @@ MXS_MODULE* MXS_CREATE_MODULE()
{"master_uuid", MXS_MODULE_PARAM_STRING},
{"master_version", MXS_MODULE_PARAM_STRING},
{"master_hostname", MXS_MODULE_PARAM_STRING},
{"slave_hostname", MXS_MODULE_PARAM_STRING},
{"mariadb10-compatibility", MXS_MODULE_PARAM_BOOL, "false"},
{"maxwell-compatibility", MXS_MODULE_PARAM_BOOL, "false"},
{"filestem", MXS_MODULE_PARAM_STRING, BINLOG_NAME_ROOT},
@ -372,6 +373,7 @@ createInstance(SERVICE *service, char **options)
inst->trx_safe = config_get_bool(params, "transaction_safety");
inst->set_master_version = config_copy_string(params, "master_version");
inst->set_master_hostname = config_copy_string(params, "master_hostname");
inst->set_slave_hostname = config_copy_string(params, "slave_hostname");
inst->fileroot = config_copy_string(params, "filestem");
inst->serverid = config_get_integer(params, "server_id");
@ -382,8 +384,6 @@ createInstance(SERVICE *service, char **options)
inst->master_uuid = config_copy_string(params, "master_uuid");
inst->set_master_uuid = inst->master_uuid != NULL;
inst->set_master_version = NULL;
inst->set_master_hostname = NULL;
inst->send_slave_heartbeat = config_get_bool(params, "send_slave_heartbeat");
/* Semi-Sync support */
@ -538,6 +538,11 @@ createInstance(SERVICE *service, char **options)
MXS_FREE(inst->set_master_hostname);
inst->set_master_hostname = MXS_STRDUP_A(value);
}
else if (strcmp(options[i], "slave_hostname") == 0)
{
MXS_FREE(inst->set_slave_hostname);
inst->set_slave_hostname = MXS_STRDUP_A(value);
}
else if (strcmp(options[i], "mariadb10-compatibility") == 0)
{
inst->mariadb10_compat = config_truth_value(value);
@ -1154,6 +1159,7 @@ free_instance(ROUTER_INSTANCE *instance)
MXS_FREE(instance->password);
MXS_FREE(instance->set_master_version);
MXS_FREE(instance->set_master_hostname);
MXS_FREE(instance->set_slave_hostname);
MXS_FREE(instance->fileroot);
MXS_FREE(instance->binlogdir);
/* SSL options */

View File

@ -731,6 +731,7 @@ typedef struct router_instance
uint32_t mariadb10_gtid_domain;/*< MariaDB 10 GTID Domain ID */
sqlite3 *gtid_maps; /*< MariaDB 10 GTID storage */
enum binlog_storage_type storage_type;/*< Enables hierachical binlog file storage */
char *set_slave_hostname; /*< Send custom Hostname to Master */
struct router_instance *next;
} ROUTER_INSTANCE;

View File

@ -616,28 +616,55 @@ blr_make_registration(ROUTER_INSTANCE *router)
{
GWBUF *buf;
unsigned char *data;
int len = 18;
int len = 18; // Min size of COM_REGISTER_SLAVE payload
int port = 3306;
int hostname_len = 0;
// Send router->set_slave_hostname
if (router->set_slave_hostname && router->set_slave_hostname[0])
{
hostname_len = strlen(router->set_slave_hostname);
}
// Add hostname len
len += hostname_len;
if ((buf = gwbuf_alloc(len + MYSQL_HEADER_LEN)) == NULL)
{
return NULL;
}
data = GWBUF_DATA(buf);
encode_value(&data[0], len, 24); // Payload length
data[3] = 0; // Sequence ID
data[4] = COM_REGISTER_SLAVE; // Command
encode_value(&data[5], router->serverid, 32); // Slave Server ID
data[9] = 0; // Slave hostname length
data[10] = 0; // Slave username length
data[11] = 0; // Slave password length
// Point to hostname len offset
data += 9;
*data++ = hostname_len; // Slave hostname length
// Copy hostname
if (hostname_len)
{
memcpy(data, router->set_slave_hostname, hostname_len);
}
// Point to user
data += hostname_len;
// Set empty user
*data++ = 0; // Slave username length
// Set empty password
*data++ = 0; // Slave password length
// Add port
if (router->service->ports)
{
port = router->service->ports->port;
}
encode_value(&data[12], port, 16); // Slave master port
encode_value(&data[14], 0, 32); // Replication rank
encode_value(&data[18], router->masterid, 32); // Master server-id
encode_value(&data[0], port, 16); // Slave master port, 2 bytes
encode_value(&data[2], 0, 32); // Replication rank, 4 bytes
encode_value(&data[6], router->masterid, 32); // Master server-id, 4 bytes
// This is hack to get the result set processing in order for binlogrouter
MySQLProtocol *proto = (MySQLProtocol*)router->master->protocol;
@ -1891,10 +1918,15 @@ static void blr_log_identity(ROUTER_INSTANCE *router)
/* Seen by the master */
MXS_NOTICE("%s: identity seen by the master: "
"server_id: %d, uuid: %s",
"Server_id: %d, Slave_UUID: %s, Host: %s",
router->service->name,
router->serverid,
(router->uuid == NULL ? "not available" : router->uuid));
router->uuid == NULL ?
"not available" :
router->uuid,
(router->set_slave_hostname && router->set_slave_hostname[0]) ?
router->set_slave_hostname :
"not set");
/* Seen by the slaves */