Add table of contents to router documentation
The TOC was added only to the long documentation files to make them easier to navigate. Also modified the headings for Avro and Binlog routers and did some minor cleanup.
This commit is contained in:
Esa Korhonen
parent
e287e29ad0
commit
ff22624d3c
@ -4,11 +4,12 @@ The avrorouter is a MariaDB 10.0 binary log to Avro file converter. It consumes
|
||||
binary logs from a local directory and transforms them into a set of Avro files.
|
||||
These files can then be queried by clients for various purposes.
|
||||
|
||||
This router is intended to be used in tandem with the [Binlog Server](Binlogrouter.md).
|
||||
The Binlog Server can connect to a master server and request binlog records. These
|
||||
records can then consumed by the avrorouter directly from the binlog cache of
|
||||
the Binlog Server. This allows MariaDB MaxScale to automatically transform binlog events
|
||||
on the master to local Avro format files.
|
||||
This router is intended to be used in tandem with the
|
||||
[Binlog Server](Binlogrouter.md).
|
||||
The Binlog Server can connect to a master server and request binlog records.
|
||||
These records can then consumed by the avrorouter directly from the binlog cache
|
||||
of the Binlog Server. This allows MariaDB MaxScale to automatically transform
|
||||
binlog events on the master to local Avro format files.
|
||||
|
||||
|
||||
|
||||
@ -21,14 +22,46 @@ should be used to communicate with the avrorouter and currently it is the only
|
||||
supported protocol. The clients can request either Avro or JSON format data
|
||||
streams from a database table.
|
||||
|
||||
# Configuration
|
||||
Table of Contents
|
||||
=================
|
||||
|
||||
* [Configuration](#configuration)
|
||||
* [Router Parameters](#router-parameters)
|
||||
* [source](#source)
|
||||
* [codec](#codec)
|
||||
* [match](#match)
|
||||
* [exclude](#exclude)
|
||||
* [Router Options](#router-options)
|
||||
* [General Options](#general-options)
|
||||
* [binlogdir](#binlogdir)
|
||||
* [avrodir](#avrodir)
|
||||
* [filestem](#filestem)
|
||||
* [start_index](#start_index)
|
||||
* [Avro file options](#avro-file-options)
|
||||
* [group_trx](#group_trx)
|
||||
* [group_rows](#group_rows)
|
||||
* [block_size](#block_size)
|
||||
* [Module commands](#module-commands)
|
||||
* [avrorouter::convert SERVICE {start | stop}](#avrorouterconvert-service-start--stop)
|
||||
* [avrorouter::purge SERVICE](#avrorouterpurge-service)
|
||||
* [Files Created by the Avrorouter](#files-created-by-the-avrorouter)
|
||||
* [Resetting the Conversion Process](#resetting-the-conversion-process)
|
||||
* [Stopping the Avrorouter](#stopping-the-avrorouter)
|
||||
* [Example Client](#example-client)
|
||||
* [Avro Schema Generator](#avro-schema-generator)
|
||||
* [Python Schema Generator](#python-schema-generator)
|
||||
* [Go Schema Generator](#go-schema-generator)
|
||||
* [Examples](#examples)
|
||||
* [Building Avrorouter](#building-avrorouter)
|
||||
|
||||
## Configuration
|
||||
|
||||
For information about common service parameters, refer to the
|
||||
[Configuration Guide](../Getting-Started/Configuration-Guide.md).
|
||||
|
||||
## Router Parameters
|
||||
### Router Parameters
|
||||
|
||||
### `source`
|
||||
#### `source`
|
||||
|
||||
The source for the binary logs. This is an optional parameter.
|
||||
|
||||
@ -57,7 +90,7 @@ router=avrorouter
|
||||
source=replication-router
|
||||
```
|
||||
|
||||
### `codec`
|
||||
#### `codec`
|
||||
|
||||
The compression codec to use. By default, the avrorouter does not use compression.
|
||||
|
||||
@ -66,7 +99,7 @@ _deflate_. These are the mandatory compression algorithms required by the
|
||||
Avro specification. For more information about the compression types,
|
||||
refer to the [Avro specification](https://avro.apache.org/docs/current/spec.html#Required+Codecs).
|
||||
|
||||
### `match`
|
||||
#### `match`
|
||||
|
||||
Only process events for tables that match this PCRE2 regular expression. See
|
||||
[Regular Expressions](../Getting-Started/Configuration-Guide.md#regular-expressions)
|
||||
@ -74,7 +107,7 @@ for more information about regular expressions.
|
||||
|
||||
This parameter was added in MaxScale 2.2.14.
|
||||
|
||||
### `exclude`
|
||||
#### `exclude`
|
||||
|
||||
Ignore events for tables that match this PCRE2 regular expression. This can be
|
||||
combined with the `match` parameter to implement table event filtering.
|
||||
@ -100,7 +133,7 @@ filestem=binlog
|
||||
avrodir=/var/lib/maxscale
|
||||
```
|
||||
|
||||
## Router Options
|
||||
### Router Options
|
||||
|
||||
The avrorouter is configured with a comma-separated list of key-value pairs.
|
||||
Currently the router has one mandatory parameter, `binlogdir`. If no `source`
|
||||
@ -108,11 +141,11 @@ parameter is defined, binlogdir needs to be manually defined in the router
|
||||
options. The following options should be given as a value to the
|
||||
`router_options` parameter.
|
||||
|
||||
### General Options
|
||||
#### General Options
|
||||
|
||||
These options control various file locations and names.
|
||||
|
||||
#### `binlogdir`
|
||||
##### `binlogdir`
|
||||
|
||||
The location of the binary log files. This is the first mandatory parameter
|
||||
and it defines where the module will read binlog files from. Read access to
|
||||
@ -122,7 +155,7 @@ If used in conjunction with the Binlog Server, the value of this option should
|
||||
be the same for both the Binlog Server and the avrorouter if the `source` parameter
|
||||
is not used.
|
||||
|
||||
#### `avrodir`
|
||||
##### `avrodir`
|
||||
|
||||
The location where the Avro files are stored. This is the second mandatory
|
||||
parameter and it governs where the converted files are stored. This directory
|
||||
@ -135,7 +168,7 @@ files. These files are named _avro.index_ and _avro-conversion.ini_. By default,
|
||||
the default data directory, _/var/lib/maxscale/_, is used. Before version 2.1 of
|
||||
MaxScale, the value of _binlogdir_ was used as the default value for _avrodir_.
|
||||
|
||||
#### `filestem`
|
||||
##### `filestem`
|
||||
|
||||
The base name of the binlog files. The default value is "mysql-bin". The binlog
|
||||
files are assumed to follow the naming schema _<filestem>.<N>_ where _<N>_ is
|
||||
@ -149,7 +182,7 @@ filestem=mybin,binlogdir=/var/lib/mysql/binlogs/
|
||||
|
||||
The first binlog file the avrorouter would look for is `/var/lib/mysql/binlogs/mybin.000001`.
|
||||
|
||||
#### `start_index`
|
||||
##### `start_index`
|
||||
|
||||
The starting index number of the binlog file. The default value is 1.
|
||||
For the binlog _mysql-bin.000001_ the index would be 1, for _mysql-bin.000005_
|
||||
@ -159,17 +192,18 @@ If you need to start from a binlog file other than 1, you need to set the value
|
||||
of this option to the correct index. The avrorouter will always start from the
|
||||
beginning of the binary log file.
|
||||
|
||||
**Note**: MaxScale version 2.2 introduces MariaDB GTID support
|
||||
in Binlog Server: currently, if used with Avrorouter, the option `mariadb10_master_gtid`
|
||||
must be set to off in the Binlog Server configuration in order to correclty
|
||||
read the binlog files.
|
||||
**Note**: MaxScale version 2.2 introduces MariaDB GTID support in Binlog Server:
|
||||
currently, if used with Avrorouter, the option `mariadb10_master_gtid` must be
|
||||
set to off in the Binlog Server configuration in order to correclty read the
|
||||
binlog files.
|
||||
|
||||
### Avro file options
|
||||
#### Avro file options
|
||||
|
||||
These options control how large the Avro file data blocks can get.
|
||||
Increasing or lowering the block size could have a positive effect
|
||||
depending on your use case. For more information about the Avro file
|
||||
format and how it organizes data, refer to the [Avro documentation](https://avro.apache.org/docs/current/).
|
||||
format and how it organizes data, refer to the
|
||||
[Avro documentation](https://avro.apache.org/docs/current/).
|
||||
|
||||
The avrorouter will flush a block and start a new one when either `group_trx`
|
||||
transactions or `group_rows` row events have been processed. Changing these
|
||||
@ -180,29 +214,29 @@ It is highly recommended to keep the block sizes relatively large to allow
|
||||
larger chunks of memory to be flushed to disk at one time. This will make
|
||||
the conversion process noticeably faster.
|
||||
|
||||
#### `group_trx`
|
||||
##### `group_trx`
|
||||
|
||||
Controls the number of transactions that are grouped into a single Avro
|
||||
data block. The default value is 1 transaction.
|
||||
|
||||
|
||||
#### `group_rows`
|
||||
##### `group_rows`
|
||||
|
||||
Controls the number of row events that are grouped into a single Avro
|
||||
data block. The default value is 1000 row events.
|
||||
|
||||
#### `block_size`
|
||||
##### `block_size`
|
||||
|
||||
The Avro data block size in bytes. The default is 16 kilobytes. Increase this
|
||||
value if individual events in the binary logs are very large. The value is a
|
||||
size type parameter which means that it can also be defined with an SI
|
||||
suffix. Refer to the
|
||||
[Configuration Guide](../Getting-Started/Configuration-Guide.md) for more
|
||||
details about size type parameters and how to use them.
|
||||
size type parameter which means that it can also be defined with an SI suffix.
|
||||
Refer to the [Configuration Guide](../Getting-Started/Configuration-Guide.md)
|
||||
for more details about size type parameters and how to use them.
|
||||
|
||||
## Module commands
|
||||
|
||||
Read [Module Commands](../Reference/Module-Commands.md) documentation for details about module commands.
|
||||
Read [Module Commands](../Reference/Module-Commands.md) documentation for
|
||||
details about module commands.
|
||||
|
||||
The avrorouter supports the following module commands.
|
||||
|
||||
@ -224,7 +258,7 @@ the conversion process. Issuing a `convert start` command **will not work**.
|
||||
**WARNING:** You will lose any and all converted data when this command is
|
||||
executed.
|
||||
|
||||
# Files Created by the Avrorouter
|
||||
## Files Created by the Avrorouter
|
||||
|
||||
The avrorouter creates two files in the location pointed by _avrodir_:
|
||||
_avro.index_ and _avro-conversion.ini_. The _avro.index_ file is used to store
|
||||
@ -232,7 +266,7 @@ the locations of the GTIDs in the .avro files. The _avro-conversion.ini_ contain
|
||||
the last converted position and GTID in the binlogs. If you need to reset the
|
||||
conversion process, delete these two files and restart MaxScale.
|
||||
|
||||
# Resetting the Conversion Process
|
||||
## Resetting the Conversion Process
|
||||
|
||||
To reset the binlog conversion process, issue the `purge` module command by
|
||||
executing it via MaxAdmin and stop MaxScale. If manually created schema files
|
||||
@ -240,7 +274,7 @@ were used, they need to be recreated once MaxScale is stopped. After stopping
|
||||
MaxScale and optionally creating the schema files, the conversion process can be
|
||||
started by starting MaxScale.
|
||||
|
||||
# Stopping the Avrorouter
|
||||
## Stopping the Avrorouter
|
||||
|
||||
The safest way to stop the avrorouter when used with the binlogrouter is to
|
||||
follow the following steps:
|
||||
@ -252,7 +286,7 @@ follow the following steps:
|
||||
This guarantees that the conversion process halts at a known good position in
|
||||
the latest binlog file.
|
||||
|
||||
# Example Client
|
||||
## Example Client
|
||||
|
||||
The avrorouter comes with an example client program, _cdc.py_, written in Python 3.
|
||||
This client can connect to a MaxScale configured with the CDC protocol and the
|
||||
@ -266,13 +300,13 @@ and [CDC Users](../Protocols/CDC_users.md) documentation.
|
||||
Read the output of `cdc.py --help` for a full list of supported options
|
||||
and a short usage description of the client program.
|
||||
|
||||
# Avro Schema Generator
|
||||
## Avro Schema Generator
|
||||
|
||||
If the CREATE TABLE statements for the tables aren't present in the current
|
||||
binary logs, the schema files must be generated with a schema file
|
||||
generator. There are currently two methods to generate the .avsc schema files.
|
||||
|
||||
## Python Schema Generator
|
||||
### Python Schema Generator
|
||||
|
||||
```
|
||||
usage: cdc_schema.py [--help] [-h HOST] [-P PORT] [-u USER] [-p PASSWORD] DATABASE
|
||||
@ -285,7 +319,7 @@ The script will generate the .avsc schema files into the current directory. Run
|
||||
the script for all required databases copy the generated .avsc files to the
|
||||
directory where the avrorouter stores the .avro files (the value of `avrodir`).
|
||||
|
||||
## Go Schema Generator
|
||||
### Go Schema Generator
|
||||
|
||||
The _cdc_schema.go_ example Go program is provided with MaxScale. This file
|
||||
can be used to create Avro schemas for the avrorouter by connecting to a
|
||||
@ -302,15 +336,15 @@ to be used by the router itself.
|
||||
Read the output of `go run cdc_schema.go -help` for more information on how
|
||||
to run the program.
|
||||
|
||||
# Examples
|
||||
## Examples
|
||||
|
||||
The [Avrorouter Tutorial](../Tutorials/Avrorouter-Tutorial.md) shows you how
|
||||
the Avrorouter works with the Binlog Server to convert binlogs from a master server
|
||||
into easy to process Avro data.
|
||||
|
||||
Here is a simple configuration example which reads binary logs locally from `/var/lib/mysql/`
|
||||
and stores them as Avro files in `/var/lib/maxscale/avro/`. The service has one listener
|
||||
listening on port 4001 for CDC protocol clients.
|
||||
Here is a simple configuration example which reads binary logs locally from
|
||||
`/var/lib/mysql/` and stores them as Avro files in `/var/lib/maxscale/avro/`.
|
||||
The service has one listener listening on port 4001 for CDC protocol clients.
|
||||
|
||||
```
|
||||
[avro-converter]
|
||||
@ -327,33 +361,40 @@ protocol=CDC
|
||||
port=4001
|
||||
```
|
||||
|
||||
Here is an example how you can query for data in JSON format using the _cdc.py_ Python script.
|
||||
It queries the table _test.mytable_ for all change records.
|
||||
Here is an example how you can query for data in JSON format using the _cdc.py_
|
||||
Python script. It queries the table _test.mytable_ for all change records.
|
||||
|
||||
```
|
||||
cdc.py --user=myuser --password=mypasswd --host=127.0.0.1 --port=4001 test.mytable
|
||||
```
|
||||
|
||||
You can then combine it with the _cdc_kafka_producer.py_ to publish these change records to a Kafka broker.
|
||||
You can then combine it with the _cdc_kafka_producer.py_ to publish these change
|
||||
records to a Kafka broker.
|
||||
|
||||
```
|
||||
cdc.py --user=myuser --password=mypasswd --host=127.0.0.1 --port=4001 test.mytable | cdc_kafka_producer.py --kafka-broker 127.0.0.1:9092 --kafka-topic test.mytable
|
||||
cdc.py --user=myuser --password=mypasswd --host=127.0.0.1 --port=4001 test.mytable |
|
||||
cdc_kafka_producer.py --kafka-broker 127.0.0.1:9092 --kafka-topic test.mytable
|
||||
```
|
||||
|
||||
For more information on how to use these scripts, see the output of `cdc.py -h` and `cdc_kafka_producer.py -h`.
|
||||
For more information on how to use these scripts, see the output of `cdc.py -h`
|
||||
and `cdc_kafka_producer.py -h`.
|
||||
|
||||
# Building Avrorouter
|
||||
## Building Avrorouter
|
||||
|
||||
To build the avrorouter from source, you will need the [Avro C](https://avro.apache.org/docs/current/api/c/)
|
||||
library, liblzma, [the Jansson library](http://www.digip.org/jansson/) and sqlite3 development headers. When
|
||||
configuring MaxScale with CMake, you will need to add `-DBUILD_CDC=Y` to build the CDC module set.
|
||||
To build the avrorouter from source, you will need the
|
||||
[Avro C](https://avro.apache.org/docs/current/api/c/) library, liblzma,
|
||||
[the Jansson library](http://www.digip.org/jansson/) and sqlite3 development
|
||||
headers. When configuring MaxScale with CMake, you will need to add
|
||||
`-DBUILD_CDC=Y` to build the CDC module set.
|
||||
|
||||
The Avro C library needs to be build with position independent code enabled. You can do this by
|
||||
adding the following flags to the CMake invocation when configuring the Avro C library.
|
||||
The Avro C library needs to be build with position independent code enabled. You
|
||||
can do this by adding the following flags to the CMake invocation when
|
||||
configuring the Avro C library.
|
||||
|
||||
```
|
||||
-DCMAKE_C_FLAGS=-fPIC -DCMAKE_CXX_FLAGS=-fPIC
|
||||
```
|
||||
|
||||
For more details about building MaxScale from source, please refer to the
|
||||
[Building MaxScale from Source Code](../Getting-Started/Building-MaxScale-from-Source-Code.md) document.
|
||||
[Building MaxScale from Source Code](../Getting-Started/Building-MaxScale-from-Source-Code.md)
|
||||
document.
|
||||
|
||||
@ -9,17 +9,50 @@ logs. You can switch to a new master server without the slaves noticing that the
|
||||
actual master server has changed. This allows for a more highly available
|
||||
replication setup where replication is high-priority.
|
||||
|
||||
# Configuration
|
||||
Table of Contents
|
||||
=================
|
||||
|
||||
## Mandatory Router Parameters
|
||||
* [Configuration](#configuration)
|
||||
* [Mandatory Router Parameters](#mandatory-router-parameters)
|
||||
* [Router Parameters](#router-parameters)
|
||||
* [binlogdir](#binlogdir)
|
||||
* [server_id](#server_id)
|
||||
* [master_id](#master_id)
|
||||
* [uuid](#uuid)
|
||||
* [master_uuid](#master_uuid)
|
||||
* [master_version](#master_version)
|
||||
* [master_hostname](#master_hostname)
|
||||
* [slave_hostname](#slave_hostname)
|
||||
* [user](#user)
|
||||
* [password](#password)
|
||||
* [heartbeat](#heartbeat)
|
||||
* [burstsize](#burstsize)
|
||||
* [mariadb10-compatibility](#mariadb10-compatibility)
|
||||
* [transaction_safety](#transaction_safety)
|
||||
* [send_slave_heartbeat](#send_slave_heartbeat)
|
||||
* [semisync](#semisync)
|
||||
* [ssl_cert_verification_depth](#ssl_cert_verification_depth)
|
||||
* [encrypt_binlog](#encrypt_binlog)
|
||||
* [encryption_algorithm](#encryption_algorithm)
|
||||
* [encryption_key_file](#encryption_key_file)
|
||||
* [mariadb10_master_gtid](#mariadb10_master_gtid)
|
||||
* [master_retry_count](#master_retry_count)
|
||||
* [connect_retry](#connect_retry)
|
||||
* [Using secondary masters](#using-secondary-masters)
|
||||
* [Examples](#examples)
|
||||
|
||||
## Configuration
|
||||
|
||||
### Mandatory Router Parameters
|
||||
|
||||
The binlogrouter requires the `user` and `password` parameters. These should be
|
||||
configured according to the
|
||||
[Configuration Guide](../Getting-Started/Configuration-Guide.md#service).
|
||||
|
||||
In addition to these two parameters, the `server_id` and `binlogdir` parameters needs to be defined.
|
||||
In addition to these two parameters, the `server_id` and `binlogdir` parameters
|
||||
needs to be defined.
|
||||
|
||||
## Router Parameters
|
||||
### Router Parameters
|
||||
|
||||
The binlogrouter accepts the following parameters.
|
||||
|
||||
@ -30,7 +63,7 @@ parameters. The values defined in `router_options` will have priority over the
|
||||
parameters to support legacy configurations. The use of `router_options` is
|
||||
deprecated.
|
||||
|
||||
### `binlogdir`
|
||||
#### `binlogdir`
|
||||
|
||||
This parameter controls the location where MariaDB MaxScale stores the binary log
|
||||
files. This is a mandatory parameter.
|
||||
@ -50,7 +83,7 @@ Read the [MySQL Authenticator](../Authenticators/MySQL-Authenticator.md)
|
||||
documentation for instructions on how to define a custom location for the user
|
||||
cache.
|
||||
|
||||
### `server_id`
|
||||
#### `server_id`
|
||||
|
||||
MariaDB MaxScale must have a unique _server_id_. This parameter configures
|
||||
the value of the _server_id_ that MariaDB MaxScale will use when
|
||||
@ -59,7 +92,7 @@ connecting to the master. This is a mandatory parameter.
|
||||
Older versions of MaxScale allowed the ID to be specified using `server-id`.
|
||||
This has been deprecated and will be removed in a future release of MariaDB MaxScale.
|
||||
|
||||
### `master_id`
|
||||
#### `master_id`
|
||||
|
||||
The _server_id_ value that MariaDB MaxScale should use to report to the slaves
|
||||
that connect to MariaDB MaxScale.
|
||||
@ -71,36 +104,36 @@ layer. The real master server ID will be used if the option is not set.
|
||||
Older versions of MaxScale allowed the ID to be specified using `master-id`.
|
||||
This has been deprecated and will be removed in a future release of MariaDB MaxScale.
|
||||
|
||||
### `uuid`
|
||||
#### `uuid`
|
||||
|
||||
This is used to set the unique UUID that the binlog router uses when it connects
|
||||
to the master server. By default the UUID will be generated.
|
||||
|
||||
### `master_uuid`
|
||||
#### `master_uuid`
|
||||
|
||||
It is a requirement of replication that each server has a unique UUID value. If
|
||||
this option is not set, binlogrouter will identify itself to the slaves using
|
||||
the UUID of the real master.
|
||||
|
||||
### `master_version`
|
||||
#### `master_version`
|
||||
|
||||
By default, the router will identify itself to the slaves using the server
|
||||
version of the real master. This option allows the router to use a custom
|
||||
version string.
|
||||
|
||||
### `master_hostname`
|
||||
#### `master_hostname`
|
||||
|
||||
By default, the router will identify itself to the slaves using the hostname of
|
||||
the real master. This option allows the router to use a custom hostname.
|
||||
|
||||
### `slave_hostname`
|
||||
#### `slave_hostname`
|
||||
|
||||
Since MaxScale 2.1.6 the router can optionally identify itself to the master
|
||||
using a custom hostname. The specified hostname can be seen in the master via
|
||||
`SHOW SLAVE HOSTS` command. The default is not to send any hostname string
|
||||
during registration.
|
||||
|
||||
### `user`
|
||||
#### `user`
|
||||
|
||||
*Note:* This is option can only be given to the `router_options` parameter. Use
|
||||
the `user` parameter of the service instead.
|
||||
@ -128,7 +161,7 @@ CREATE USER 'repl'@'maxscalehost' IDENTIFIED by 'password';
|
||||
GRANT REPLICATION SLAVE ON *.* TO 'repl'@'maxscalehost';
|
||||
```
|
||||
|
||||
### `password`
|
||||
#### `password`
|
||||
|
||||
*Note:* This is option can only be given to the `router_options` parameter. Use
|
||||
the `password` parameter of the service instead.
|
||||
@ -138,7 +171,7 @@ password in the service entry will be used. For compatibility with other
|
||||
username and password definitions within the MariaDB MaxScale configuration file
|
||||
it is also possible to use the parameter `passwd`.
|
||||
|
||||
### `heartbeat`
|
||||
#### `heartbeat`
|
||||
|
||||
This defines the value of the heartbeat interval in seconds for the connection
|
||||
to the master. The default value for the heartbeat period is every 5 minutes.
|
||||
@ -148,7 +181,7 @@ least every heartbeat period. If there are no real binlog events to send the
|
||||
master will sent a special heartbeat event. The current interval value is
|
||||
reported in the diagnostic output.
|
||||
|
||||
### `burstsize`
|
||||
#### `burstsize`
|
||||
|
||||
This parameter is used to define the maximum amount of data that will be sent to
|
||||
a slave by MariaDB MaxScale when that slave is lagging behind the master. The
|
||||
@ -164,7 +197,7 @@ designed to both prevent flooding of that slave and also to prevent threads
|
||||
within MariaDB MaxScale spending disproportionate amounts of time with slaves
|
||||
that are lagging behind the master.
|
||||
|
||||
### `mariadb10-compatibility`
|
||||
#### `mariadb10-compatibility`
|
||||
|
||||
This parameter allows binlogrouter to replicate from a MariaDB 10.0 master
|
||||
server: this parameter is enabled by default since MaxScale 2.2.0.
|
||||
@ -200,7 +233,7 @@ When a slave server connects with a GTID request a lookup is made for
|
||||
the value match and following binlog events will be sent.
|
||||
|
||||
|
||||
### `transaction_safety`
|
||||
#### `transaction_safety`
|
||||
|
||||
This parameter is used to enable/disable incomplete transactions detection in
|
||||
binlog router. The default value is _off_.
|
||||
@ -210,7 +243,7 @@ is corrupted or an incomplete transaction is found. During normal operations
|
||||
binlog events are not distributed to the slaves until a COMMIT is seen. Set
|
||||
transaction_safety=on to enable detection of incomplete transactions.
|
||||
|
||||
### `send_slave_heartbeat`
|
||||
#### `send_slave_heartbeat`
|
||||
|
||||
This defines whether MariaDB MaxScale sends the heartbeat packet to the slave
|
||||
when there are no real binlog events to send. This parameter takes a boolean
|
||||
@ -221,7 +254,7 @@ If value is set to true the interval value (requested by the slave during
|
||||
registration) is reported in the diagnostic output and the packet is send after
|
||||
the time interval without any event to send.
|
||||
|
||||
### `semisync`
|
||||
#### `semisync`
|
||||
|
||||
This parameter controls whether binlog server could ask Master server to start
|
||||
the Semi-Synchronous replication. This parameter takes a boolean value and the
|
||||
@ -233,7 +266,7 @@ value of the GLOBAL VARIABLE *rpl_semi_sync_master_enabled* are checked in the
|
||||
Master registration phase: if the plugin is installed in the Master database,
|
||||
the binlog server subsequently requests the semi-sync option.
|
||||
|
||||
Note:
|
||||
**Note:**
|
||||
- the network replication stream from Master has two additional bytes before
|
||||
each binlog event.
|
||||
- the Semi-Sync protocol requires an acknowledge packet to be sent back to
|
||||
@ -245,7 +278,7 @@ Note:
|
||||
Please note that semi-sync replication is only related to binlog server to
|
||||
Master communication.
|
||||
|
||||
### `ssl_cert_verification_depth`
|
||||
#### `ssl_cert_verification_depth`
|
||||
|
||||
This parameter sets the maximum length of the certificate authority chain that
|
||||
will be accepted. Legal values are positive integers. This applies to SSL
|
||||
@ -253,7 +286,7 @@ connection to master server that could be acivated either by writing options in
|
||||
master.ini or later via a _CHANGE MASTER TO_ command. This parameter cannot be
|
||||
modified at runtime. The default verification depth is 9.
|
||||
|
||||
### `encrypt_binlog`
|
||||
#### `encrypt_binlog`
|
||||
|
||||
Whether to encrypt binlog files: the default is _off_.
|
||||
|
||||
@ -275,11 +308,11 @@ MaxScale binlog server adds its own StartEncryption to binlog files consequently
|
||||
the binlog events positions in binlog file are the same as in the master binlog
|
||||
file and there is no position mismatch.
|
||||
|
||||
### `encryption_algorithm`
|
||||
#### `encryption_algorithm`
|
||||
|
||||
The encryption algorithm, either 'aes_ctr' or 'aes_cbc'. The default is 'aes_cbc'
|
||||
|
||||
### `encryption_key_file`
|
||||
#### `encryption_key_file`
|
||||
|
||||
The specified key file must contains lines with following format:
|
||||
|
||||
@ -308,7 +341,7 @@ Example key file with multiple keys:
|
||||
3;bbbbbbbbbaaaaaaabbbbbccccceeeddddd3333333ddddaaaaffffffeeeeecccd
|
||||
```
|
||||
|
||||
### `mariadb10_master_gtid`
|
||||
#### `mariadb10_master_gtid`
|
||||
|
||||
This option allows MaxScale binlog router to register with MariaDB 10.X master
|
||||
using GTID instead of _binlog_file_ name and _position_ in CHANGE MASTER TO
|
||||
@ -347,15 +380,18 @@ by the master server before any event.
|
||||
In order to avoid holes in the binlog files, MaxScale will fill all gaps
|
||||
in the binlog files with ignorable events.
|
||||
- It's not possible to specify the GTID _domain_id: the master one
|
||||
is being used for all operations. All slave servers must use the same replication domain as the master server.
|
||||
is being used for all operations. All slave servers must use the same replication
|
||||
domain as the master server.
|
||||
|
||||
### `master_retry_count`
|
||||
#### `master_retry_count`
|
||||
|
||||
This option sets the maximum number of connection retries when the master server is disconnected or not reachable.
|
||||
This option sets the maximum number of connection retries when the master server is
|
||||
disconnected or not reachable.
|
||||
Default value is 1000.
|
||||
|
||||
### `connect_retry`
|
||||
The option sets the time interval for a new connection retry to master server, default value is 60 seconds.
|
||||
#### `connect_retry`
|
||||
The option sets the time interval for a new connection retry to master server,
|
||||
default value is 60 seconds.
|
||||
|
||||
|
||||
**A complete example** of a service entry for a binlog router service would be as
|
||||
@ -375,7 +411,7 @@ encryption_algorithm=aes_ctr
|
||||
encryption_key_file=/var/binlogs/enc_key.txt
|
||||
```
|
||||
|
||||
## Using secondary masters
|
||||
### Using secondary masters
|
||||
|
||||
From MaxScale 2.3 onwards it is possible to specify secondary masters that
|
||||
the binlog router can use in case the connection to the default master fails.
|
||||
@ -454,8 +490,8 @@ master_host=192.168.121.76
|
||||
|
||||
## Examples
|
||||
|
||||
The [Replication Proxy](../Tutorials/Replication-Proxy-Binlog-Router-Tutorial.md) tutorial will
|
||||
show you how to configure and administrate a binlogrouter installation.
|
||||
The [Replication Proxy](../Tutorials/Replication-Proxy-Binlog-Router-Tutorial.md)
|
||||
tutorial will show you how to configure and administrate a binlogrouter installation.
|
||||
|
||||
Tutorial also includes SSL communication setup to the master server and SSL
|
||||
client connections setup to MaxScale Binlog Server.
|
||||
|
||||
@ -5,6 +5,44 @@ and its intended use case scenarios. It also displays all router configuration
|
||||
parameters with their descriptions. A list of current limitations of the module
|
||||
is included and use examples are provided.
|
||||
|
||||
Table of Contents
|
||||
=================
|
||||
|
||||
|
||||
* [Overview](#overview)
|
||||
* [Configuration](#configuration)
|
||||
* [Parameters](#parameters)
|
||||
* [max_slave_connections](#max_slave_connections)
|
||||
* [max_slave_replication_lag](#max_slave_replication_lag)
|
||||
* [use_sql_variables_in](#use_sql_variables_in)
|
||||
* [connection_keepalive](#connection_keepalive)
|
||||
* [master_reconnection](#master_reconnection)
|
||||
* [slave_selection_criteria](#slave_selection_criteria)
|
||||
* [Server Weights and slave_selection_criteria](#server-weights-and-slave_selection_criteria)
|
||||
* [Interaction Between slave_selection_criteria and max_slave_connections](#interaction-between-slave_selection_criteria-and-max_slave_connections)
|
||||
* [max_sescmd_history](#max_sescmd_history)
|
||||
* [disable_sescmd_history](#disable_sescmd_history)
|
||||
* [master_accept_reads](#master_accept_reads)
|
||||
* [strict_multi_stmt](#strict_multi_stmt)
|
||||
* [strict_sp_calls](#strict_sp_calls)
|
||||
* [master_failure_mode](#master_failure_mode)
|
||||
* [retry_failed_reads](#retry_failed_reads)
|
||||
* [delayed_retry](#delayed_retry)
|
||||
* [delayed_retry_timeout](#delayed_retry_timeout)
|
||||
* [transaction_replay](#transaction_replay)
|
||||
* [transaction_replay_max_size](#transaction_replay_max_size)
|
||||
* [optimistic_trx](#optimistic_trx)
|
||||
* [causal_reads](#causal_reads)
|
||||
* [causal_reads_timeout](#causal_reads_timeout)
|
||||
* [Routing hints](#routing-hints)
|
||||
* [Limitations](#limitations)
|
||||
* [Legacy Configuration](#legacy-configuration)
|
||||
* [Examples](#examples)
|
||||
* [Readwritesplit routing decisions](#readwritesplit-routing-decisions)
|
||||
* [Routing to Master](#routing-to-master)
|
||||
* [Routing to Slaves](#routing-to-slaves)
|
||||
* [Routing to every session backend](#routing-to-every-session-backend)
|
||||
|
||||
## Overview
|
||||
|
||||
The **readwritesplit** router is designed to increase the read-only processing
|
||||
|
||||
@ -1,16 +1,34 @@
|
||||
# SchemaRouter Router
|
||||
# SchemaRouter
|
||||
|
||||
The SchemaRouter router provides an easy and manageable sharding SOLUTION by
|
||||
The SchemaRouter provides an easy and manageable sharding solution by
|
||||
building a single logical database server from multiple separate ones. Each
|
||||
database is shown to the client and queries targeting unique databases are
|
||||
routed to their respective servers. In addition to providing simple
|
||||
database-based sharding, the schemarouter router also enables cross-node
|
||||
database-based sharding, the schemarouter also enables cross-node
|
||||
session variable usage by routing all queries that modify the session to all
|
||||
nodes.
|
||||
|
||||
From 2.3.0 onwards, the SchemaRouter is capable of table family sharding,
|
||||
in addition to being capable of sharding databases.
|
||||
|
||||
Table of Contents
|
||||
=================
|
||||
|
||||
* [Routing Logic](#routing-logic)
|
||||
* [Configuration](#configuration)
|
||||
* [Router Parameters](#router-parameters)
|
||||
* [ignore_databases](#ignore_databases)
|
||||
* [ignore_databases_regex](#ignore_databases_regex)
|
||||
* [preferred_server](#preferred_server)
|
||||
* [Table Family Sharding](#table-family-sharding)
|
||||
* [Router Options](#router-options)
|
||||
* [max_sescmd_history](#max_sescmd_history)
|
||||
* [disable_sescmd_history](#disable_sescmd_history)
|
||||
* [refresh_databases](#refresh_databases)
|
||||
* [refresh_interval](#refresh_interval)
|
||||
* [Limitations](#limitations)
|
||||
* [Examples](#examples)
|
||||
|
||||
## Routing Logic
|
||||
|
||||
If a command line client is used, i.e. `mysql`, and a direct connection to
|
||||
@ -34,7 +52,7 @@ configuration in the next chapter.
|
||||
|
||||
## Configuration
|
||||
|
||||
Here is an example configuration of the schemarouter router:
|
||||
Here is an example configuration of the schemarouter:
|
||||
|
||||
```
|
||||
[Shard Router]
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user