MXS-1484: set binlog storage to TREE mode

When mariadb10_master_gtid is on the storage of binlog file is
automatically set to TREE mode.
This commit is contained in:
MassimilianoPinto
2017-10-23 11:25:19 +02:00
parent 37c804e0d3
commit 65dc9e0d30
3 changed files with 14 additions and 80 deletions

View File

@ -336,22 +336,6 @@ in the binlog files with ignorable events.
- It's not possible to specify the GTID _domain_id: the master one - It's not possible to specify the GTID _domain_id: the master one
is being used for all operations. All slave servers must use the same replication domain as the master server. is being used for all operations. All slave servers must use the same replication domain as the master server.
### `binlog_structure`
This option controls the way binlog file are saved in the _binlogdir_:
there are two possible values, `flat | tree`
The `tree` mode can only be set with `mariadb10_master_gtid=On`
- `flat` is the default value, files are saved as usual.
- `tree` enables the saving of files using this hierarchy model:
_binlogdir_/_domain_id_/_server_id_/_filename_
The _tree_ structure easily allows the changing of the master server
without caring about binlog filename and sequence:
just change _host_ and _port_, the replication will
resume from last GTID MaxScale has seen.
### `master_retry_count` ### `master_retry_count`
This option sets the maximum number of connection retries when the master server is disconnected or not reachable. This option sets the maximum number of connection retries when the master server is disconnected or not reachable.
@ -392,7 +376,6 @@ follows.
encryption_key_file=/var/binlogs/enc_key.txt, encryption_key_file=/var/binlogs/enc_key.txt,
mariadb10_slave_gtid=On, mariadb10_slave_gtid=On,
mariadb10_master_gtid=Off, mariadb10_master_gtid=Off,
binlog_structure=flat,
slave_hostname=maxscale-blr-1, slave_hostname=maxscale-blr-1,
master_retry_count=1000, master_retry_count=1000,
connect_retry=60 connect_retry=60

View File

@ -68,7 +68,6 @@ A **complete example** of a service entry for a binlog router service would be a
encryption_key_file=/var/binlogs/enc_key.txt, encryption_key_file=/var/binlogs/enc_key.txt,
mariadb10_slave_gtid=On, mariadb10_slave_gtid=On,
mariadb10_master_gtid=Off, mariadb10_master_gtid=Off,
binlog_structure=flat,
slave_hostname=maxscale-blr-1, slave_hostname=maxscale-blr-1,
master_retry_count=1000, master_retry_count=1000,
connect_retry=60 connect_retry=60
@ -285,7 +284,7 @@ The default is that a slave connection must not include any GTID
feature: `MASTER_USE_GTID=no` feature: `MASTER_USE_GTID=no`
Starting from MaxScale 2.2 it's also possible to register to MariaDB 10.X master using Starting from MaxScale 2.2 it's also possible to register to MariaDB 10.X master using
**GTID** using the two new options *mariadb10_master_gtid* and *binlog_structure*. **GTID** using the new option *mariadb10_master_gtid*.
Current GTID implementation limitations: Current GTID implementation limitations:
@ -458,19 +457,12 @@ error logs and in *SHOW SLAVE STATUS*,
##### MariaDB 10 GTID ##### MariaDB 10 GTID
If _mariadb10_master_gtid_ is On changing the master doesn't require the setting of a If _mariadb10_master_gtid_ is On changing the master doesn't require the setting of a
new _file_ and _pos_, just specify new host and port with CHANGE MASTER; depending on the _binlog_structure_ values some additional steps might be required. new _file_ and _pos_, just specify new host and port with CHANGE MASTER.
If _binlog_structure=flat_, in order to keep previous binlog files untouched in MaxScale _binlogdir_ (no overwriting), As the binlog files will be automatically saved using a hierarchy model
the next in sequence file must exist in the Master server, as per above scenario _file and pos_ (2).
It might also happen that each server in the replication setup has its own binlog file name
convention (server1_bin, server2_bin etc) or the user doesn't want to care at all about
name and sequence. The _binlog_structure_ option set to _tree_ value simplifies the change
master process: as the binlog files are saved using a hierarchy model
(_binlogdir/domain_id/server_id/_filename_), MaxScale can work with any filename and any (_binlogdir/domain_id/server_id/_filename_), MaxScale can work with any filename and any
sequence and no binlog file will be overwritten by accident. sequence and no binlog file will be overwritten by accident.
**Scenario** example: **Scenario** example:
Let's start saying it's a good practice to issue in the new Master `FLUSH TABLES` which Let's start saying it's a good practice to issue in the new Master `FLUSH TABLES` which
@ -508,38 +500,17 @@ MariaDB> SELECT @@global.gtid_current_pos;
``` ```
Starting the replication in MaxScale, `START SLAVE`, Starting the replication in MaxScale, `START SLAVE`,
will result in new events being downloaded and stored. will result in new events being downloaded and stored in the new file
`0/10333/mysql-bin.000001` (which should be the current file in the new master)
If _binlog_structure=flat_ (default), the binlog events are saved in the new file
`mysql-bin.000061`, which should have been created in the Master before starting
replication from MaxScale, see above scenario (2)
If _binlog_structure=tree_, the binlog events are saved in the new file
`0/10333/mysql-bin.000001` (which is the current file in the new master)
The latter example clearly shows that the binlog file has a different sequence number
(1 instead of 61) and possibly a new name.
As usual, check for any error in log files and with As usual, check for any error in log files and with
MariaDB> SHOW SLAVE STATUS; MariaDB> SHOW SLAVE STATUS;
Issuing the admin command `SHOW BINARY LOGS` it's possible to see the list Issuing the admin command `SHOW BINARY LOGS` it's possible to see the list
of log files which have been downloaded: of log files which have been downloaded and to follow the _master change_
history: the displayed log file names have a prefix with
``` replication domain_id and server_id.
MariaDB> SHOW BINARY LOGS;
+------------------+-----------+
| Log_name | File_size |
+------------------+-----------+
| mysql-bin.000113 | 2214 |
...
| mysql-bin.000117 | 535 |
+------------------+-----------+
```
It's possible to follow the _master change_ history if option `binlog_structure=tree`:
the displayed log file names have a prefix with replication domain_id and server_id.
``` ```
MariaDB> SHOW BINARY LOGS; MariaDB> SHOW BINARY LOGS;

View File

@ -379,10 +379,8 @@ createInstance(SERVICE *service, char **options)
/* Set router uuid */ /* Set router uuid */
inst->uuid = config_copy_string(params, "uuid"); inst->uuid = config_copy_string(params, "uuid");
/* Enable Flat or Tree storage of binlog files */ /* Set Flat storage of binlog files as default */
inst->storage_type = config_get_enum(params, inst->storage_type = BLR_BINLOG_STORAGE_FLAT;
"binlog_structure",
binlog_storage_values);
if (inst->uuid == NULL) if (inst->uuid == NULL)
{ {
@ -549,13 +547,6 @@ createInstance(SERVICE *service, char **options)
{ {
inst->mariadb10_master_gtid = config_truth_value(value); inst->mariadb10_master_gtid = config_truth_value(value);
} }
else if (strcmp(options[i], "binlog_structure") == 0)
{
/* Enable Flat or Tree storage of binlog files */
inst->storage_type = strcasecmp(value, "tree") == 0 ?
BLR_BINLOG_STORAGE_TREE :
BLR_BINLOG_STORAGE_FLAT;
}
else if (strcmp(options[i], "encryption_algorithm") == 0) else if (strcmp(options[i], "encryption_algorithm") == 0)
{ {
int ret = blr_check_encryption_algorithm(value); int ret = blr_check_encryption_algorithm(value);
@ -780,24 +771,12 @@ createInstance(SERVICE *service, char **options)
inst->mariadb10_compat = true; inst->mariadb10_compat = true;
} }
/**
* Force GTID slave request handling if GTID Master registration is On
*/
if (inst->mariadb10_master_gtid) if (inst->mariadb10_master_gtid)
{ {
/* Force GTID slave request handling */
inst->mariadb10_gtid = true; inst->mariadb10_gtid = true;
} /* Force binlog storage as tree */
inst->storage_type = BLR_BINLOG_STORAGE_TREE;
if (!inst->mariadb10_master_gtid &&
inst->storage_type == BLR_BINLOG_STORAGE_TREE)
{
MXS_ERROR("%s: binlog_structure 'tree' mode can be enabled only"
" with MariaDB Master GTID registration feature."
" Please enable it with option"
" 'mariadb10_master_gtid = on'",
service->name);
free_instance(inst);
return NULL;
} }
/* Log binlog structure storage mode */ /* Log binlog structure storage mode */
@ -806,6 +785,7 @@ createInstance(SERVICE *service, char **options)
inst->storage_type == BLR_BINLOG_STORAGE_FLAT ? inst->storage_type == BLR_BINLOG_STORAGE_FLAT ?
"'flat' mode" : "'flat' mode" :
"'tree' mode using GTID domain_id and server_id"); "'tree' mode using GTID domain_id and server_id");
/* Enable MariaDB the GTID maps store */ /* Enable MariaDB the GTID maps store */
if (inst->mariadb10_compat && if (inst->mariadb10_compat &&
inst->mariadb10_gtid) inst->mariadb10_gtid)