hcq/MaxScale
hcq/MaxScale
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Massimiliano pinto doc update 2.1 (#125)

Browse Source 
* Update Nagios-Plugins.md

* Update MM-Monitor.md

* Update Nagios-Plugins.md

* Update Nagios-Plugins.md

* Update Galera-Cluster-Connection-Routing-Tutorial.md

* Update Galera-Cluster-Connection-Routing-Tutorial.md

* Update RabbitMQ-Setup-And-MaxScale-Integration.md

* Update MaxScale-Tutorial.md

* Update Cache.md

* Update Transaction-Performance-Monitoring-Filter.md

* Update RabbitMQ-Consumer-Client.md

* Update Building-MaxScale-from-Source-Code.md

* Update MariaDB-MaxScale-Installation-Guide.md

* Update Install-MariaDB-MaxScale-Using-a-Tarball.md

* Update RabbitMQ-Setup-And-MaxScale-Integration.md

* Update RabbitMQ-Consumer-Client.md
This commit is contained in:
MassimilianoPinto 2017-04-20 13:36:13 +02:00 committed by GitHub
parent 33b809eec1
commit 695ba19965
11 changed files with 459 additions and 154 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
Documentation/Filters/Cache.md
View File

@ -654,7 +654,8 @@ created, under which the actual instance specific cache directories are created.
Specifies whether RocksDB should collect statistics that later can be queried
using `maxadmin`. It should be noted, though, that collecting RocksDB statistics
is not without a cost. From the [RocksDB Documentation](https://github.com/facebook/rocksdb/wiki/Statistics)
is not without a cost.
From the [RocksDB Documentation](https://github.com/facebook/rocksdb/wiki/Statistics)
_The overhead of statistics is usually small but non-negligible. We usually
observe an overhead of 5%-10%._

17
Documentation/Filters/RabbitMQ-Consumer-Client.md
View File

@ -1,8 +1,10 @@
#RabbitMQ Consumer Client
# RabbitMQ Consumer Client
## Overview
This utility tool is used to read messages from a RabbitMQ broker sent by the [RabbitMQ Filter](RabbitMQ-Filter.md) and forward these messages into an SQL database as queries.
This utility tool is used to read messages from a RabbitMQ broker sent by the
[RabbitMQ Filter](RabbitMQ-Filter.md) and forward these messages into an
SQL database as queries.
## Command Line Arguments
@ -14,7 +16,9 @@ The **RabbitMQ Consumer Client** only has one command line argument.
## Installation
To install the RabbitMQ Consumer Client you ca either use the provided packages or you can compile it from source code. The source code is included as a part of the MariaDB MaxScale source code and can be found in the `rabbitmq_consumer` folder.
To install the RabbitMQ Consumer Client you can either use the provided packages
or you can compile it from source code. The source code is included as a part of the
MariaDB MaxScale source code and can be found in the `rabbitmq_consumer` folder.
## Building from source
@ -48,9 +52,12 @@ include and library directories 'in buildvars.inc'
## Configuration
The consumer client requires that the `consumer.cnf` configuration file is either be present in the `etc` folder of the installation directory or in the folder specified by the `-c` argument.
The consumer client requires that the `consumer.cnf` configuration file is either
be present in the `etc` folder of the installation directory or in the folder
specified by the `-c` argument.
The source broker, the destination database and the message log file can be configured into the separate `consumer.cnf` file.
The source broker, the destination database and the message log file can be
configured into the separate `consumer.cnf` file.
| Option | Description |
|-----------|---------------------------------------------|

44
Documentation/Filters/Transaction-Performance-Monitoring-Filter.md
View File

@ -2,11 +2,17 @@
## Overview
The Transaction Performance Monitoring (TPM) filter is a filter module for MaxScale that monitors every SQL statement that passes through the filter. The filter groups a series of SQL statements into a transaction by detecting 'commit' or 'rollback' statements. It logs all committed transactions with necessary information, such as timestamp, client, SQL statements, latency, etc., which can be used later for transaction performance analysis.
The Transaction Performance Monitoring (TPM) filter is a filter module for MaxScale
that monitors every SQL statement that passes through the filter.
The filter groups a series of SQL statements into a transaction by detecting
'commit' or 'rollback' statements. It logs all committed transactions with necessary
information, such as timestamp, client, SQL statements, latency, etc., which
can be used later for transaction performance analysis.
## Configuration
The configuration block for the TPM filter requires the minimal filter options in it's section within the maxscale.cnf file, stored in /etc/maxscale.cnf.
The configuration block for the TPM filter requires the minimal filter
options in it's section within the maxscale.cnf file, stored in /etc/maxscale.cnf.
```
[MyLogFilter]
@ -32,7 +38,8 @@ The TPM filter accepts a number of optional parameters.
### Filename
The name of the output file created for performance logging. The default filename is **tpm.log**.
The name of the output file created for performance logging.
The default filename is **tpm.log**.
```
filename=/tmp/SqlQueryLog
@ -40,7 +47,10 @@ filename=/tmp/SqlQueryLog
### Source
The optional `source` parameter defines an address that is used to match against the address from which the client connection to MaxScale originates. Only sessions that originate from this address will be logged.
The optional `source` parameter defines an address that is used
to match against the address from which the client connection
to MaxScale originates. Only sessions that originate from this
address will be logged.
```
source=127.0.0.1
@ -48,7 +58,10 @@ source=127.0.0.1
### User
The optional `user` parameter defines a user name that is used to match against the user from which the client connection to MaxScale originates. Only sessions that are connected using this username are logged.
The optional `user` parameter defines a user name that is used
to match against the user from which the client connection to
MaxScale originates. Only sessions that are connected using
this username are logged.
```
user=john
@ -56,7 +69,8 @@ user=john
### Delimiter
The optional `delimiter` parameter defines a delimiter that is used to distinguish columns in the log. The default delimiter is **`:::`**.
The optional `delimiter` parameter defines a delimiter that is used to
distinguish columns in the log. The default delimiter is **`:::`**.
```
delimiter=:::
@ -64,7 +78,9 @@ delimiter=:::
### Query_delimiter
The optional `query_delimiter` defines a delimiter that is used to distinguish different SQL statements in a transaction. The default query delimiter is **`@@@`**.
The optional `query_delimiter` defines a delimiter that is used to
distinguish different SQL statements in a transaction.
The default query delimiter is **`@@@`**.
```
query_delimiter=@@@
@ -72,7 +88,11 @@ query_delimiter=@@@
### Named_pipe
**`named_pipe`** is the path to a named pipe, which TPM filter uses to communicate with 3rd-party applications (e.g., [DBSeer](http://dbseer.org)). Logging is enabled when the router receives the character '1' and logging is disabled when the router receives the character '0' from this named pipe. The default named pipe is **`/tmp/tpmfilter`** and logging is **disabled** by default.
**`named_pipe`** is the path to a named pipe, which TPM filter uses to
communicate with 3rd-party applications (e.g., [DBSeer](http://dbseer.org)).
Logging is enabled when the router receives the character '1' and logging is
disabled when the router receives the character '0' from this named pipe.
The default named pipe is **`/tmp/tpmfilter`** and logging is **disabled** by default.
named_pipe=/tmp/tpmfilter
@ -89,7 +109,8 @@ Similarly, the following command disables the logging:
### Example 1 - Log Transactions for Performance Analysis
You want to log every transaction with its SQL statements and latency for future transaction performance analysis.
You want to log every transaction with its SQL statements and latency
for future transaction performance analysis.
Add a filter with the following definition:
@ -111,7 +132,8 @@ passwd=mypasswd
filters=PerformanceLogger
```
After the filter reads the character '1' from its named pipe, the following is an example log that is generated from the above TPM filter with the above configuration:
After the filter reads the character '1' from its named pipe, the following
is an example log that is generated from the above TPM filter with the above configuration:
```
@ -120,4 +142,4 @@ After the filter reads the character '1' from its named pipe, the following is a
...
```
Note that 3 and 5 are latencies of each transaction in milliseconds.
Note that 3 and 5 are latencies of each transaction in milliseconds.

3
Documentation/Getting-Started/Building-MaxScale-from-Source-Code.md
View File

@ -106,7 +106,8 @@ make test
sudo make install
```
Other useful targets for Make are `documentation`, which generates the Doxygen documentation, and `uninstall` which uninstall MariaDB MaxScale binaries after an install.
Other useful targets for Make are `documentation`, which generates the Doxygen documentation,
and `uninstall` which uninstall MariaDB MaxScale binaries after an install.
**Note**: If you configure CMake multiple times, it's possible that you will run
into problems when building MaxScale. Most of the time this manifests as a

50
Documentation/Getting-Started/Install-MariaDB-MaxScale-Using-a-Tarball.md
View File

@ -1,6 +1,8 @@
# Installing MariaDB MaxScale using a tarball
MariaDB MaxScale is also made available as a tarball, which is named like `maxscale-x.y.z.OS.tar.gz` where `x.y.z` is the same as the corresponding version and `OS` identifies the operating system, e.g. `maxscale-2.0.1.centos.7.tar.gz`.
MariaDB MaxScale is also made available as a tarball, which is named like
`maxscale-x.y.z.OS.tar.gz` where `x.y.z` is the same as the corresponding version and `OS`
identifies the operating system, e.g. `maxscale-2.0.1.centos.7.tar.gz`.
In order to use the tarball, the following libraries are required:
@ -8,11 +10,14 @@ In order to use the tarball, the following libraries are required:
- libaio
- OpenSSL
The tarball has been built with the assumption that it will be installed in `/usr/local`. However, it is possible to install it in any directory, but in that case MariaDB MaxScale must be invoked with a flag.
The tarball has been built with the assumption that it will be installed in `/usr/local`.
However, it is possible to install it in any directory, but in that case MariaDB MaxScale
must be invoked with a flag.
## Installing as root in `/usr/local`
If you have root access to the system you probably want to install MariaDB MaxScale under the user and group `maxscale`.
If you have root access to the system you probably want to install MariaDB MaxScale under
the user and group `maxscale`.
The required steps are as follows:
@ -24,11 +29,15 @@ The required steps are as follows:
$ cd maxscale
$ sudo chown -R maxscale var
Creating the symbolic link is necessary, since MariaDB MaxScale has been built with with the assumption that the plugin directory is `/usr/local/maxscale/lib/maxscale`.
Creating the symbolic link is necessary, since MariaDB MaxScale has been built
with the assumption that the plugin directory is `/usr/local/maxscale/lib/maxscale`.
The symbolic link also makes it easy to switch between different versions of MariaDB MaxScale that have been installed side by side in `/usr/local`; just make the symbolic link point to another installation.
The symbolic link also makes it easy to switch between different versions of
MariaDB MaxScale that have been installed side by side in `/usr/local`;
just make the symbolic link point to another installation.
In addition, the first time you install MariaDB MaxScale from a tarball you need to create the following directories:
In addition, the first time you install MariaDB MaxScale from a tarball
you need to create the following directories:
$ sudo mkdir /var/log/maxscale
$ sudo mkdir /var/lib/maxscale
@ -42,21 +51,28 @@ and make `maxscale` the owner of them:
$ sudo chown maxscale /var/run/maxscale
$ sudo chown maxscale /var/cache/maxscale
The following step is to create the MariaDB MaxScale configuration file `/etc/maxscale.cnf`. The file `etc/maxscale.cnf.template` can be used as a base. Please refer to [Configuration Guide](Configuration-Guide.md) for details.
The following step is to create the MariaDB MaxScale configuration file `/etc/maxscale.cnf`.
The file `etc/maxscale.cnf.template` can be used as a base.
Please refer to [Configuration Guide](Configuration-Guide.md) for details.
When the configuration file has been created, MariaDB MaxScale can be started.
$ sudo bin/maxscale --user=maxscale -d
The `-d` flag causes maxscale _not_ to turn itself into a daemon, which is adviseable the first time MariaDB MaxScale is started, as it makes it easier to spot problems.
The `-d` flag causes maxscale _not_ to turn itself into a daemon,
which is adviseable the first time MariaDB MaxScale is started, as it makes it easier to spot problems.
If you want to place the configuration file somewhere else but in `/etc` you can invoke MariaDB MaxScale with the `--config` flag, for instance, `--config=/usr/local/maxscale/etc/maxscale.cnf`.
If you want to place the configuration file somewhere else but in `/etc`
you can invoke MariaDB MaxScale with the `--config` flag,
for instance, `--config=/usr/local/maxscale/etc/maxscale.cnf`.
Note also that if you want to keep _everything_ under `/usr/local/maxscale` you can invoke MariaDB MaxScale using the flag `--basedir`.
Note also that if you want to keep _everything_ under `/usr/local/maxscale`
you can invoke MariaDB MaxScale using the flag `--basedir`.
$ sudo bin/maxscale --user=maxscale --basedir=/usr/local/maxscale -d
That will cause MariaDB MaxScale to look for its configuration file in `/usr/local/maxscale/etc` and to store all runtime files under `/usr/local/maxscale/var`.
That will cause MariaDB MaxScale to look for its configuration file in
`/usr/local/maxscale/etc` and to store all runtime files under `/usr/local/maxscale/var`.
## Installing in any Directory
@ -64,16 +80,22 @@ Enter a directory where you have the right to create a subdirectory. Then do as
$ tar -xzvf maxscale-x.y.z.OS.tar.gz
The next step is to create the MaxScale configuration file `maxscale-x.y.z/etc/maxscale.cnf`. The file `maxscale-x.y.z/etc/maxscale.cnf.template` can be used as a base. Please refer to [Configuration Guide](Configuration-Guide.md) for details.
The next step is to create the MaxScale configuration file `maxscale-x.y.z/etc/maxscale.cnf`.
The file `maxscale-x.y.z/etc/maxscale.cnf.template` can be used as a base.
Please refer to [Configuration Guide](Configuration-Guide.md) for details.
When the configuration file has been created, MariaDB MaxScale can be started.
$ cd maxscale-x.y.z.OS
$ bin/maxscale -d --basedir=.
With the flag `--basedir`, MariaDB MaxScale is told where the `lib`, `etc` and `var` directories are found. Unless it is specified, MariaDB MaxScale assumes the `lib` directory is found in `/usr/local/maxscale`, and the `var` and `etc` directories in `/`.
With the flag `--basedir`, MariaDB MaxScale is told where the `lib`, `etc` and `var`
directories are found. Unless it is specified, MariaDB MaxScale assumes
the `lib` directory is found in `/usr/local/maxscale`,
and the `var` and `etc` directories in `/`.
It is also possible to specify the directories and the location of the configuration file individually. Invoke MaxScale like
It is also possible to specify the directories and the location of
the configuration file individually. Invoke MaxScale like
$ bin/maxscale --help

142
Documentation/Getting-Started/MariaDB-MaxScale-Installation-Guide.md
View File

@ -2,11 +2,21 @@
## First Steps With MariaDB MaxScale
In this introduction to MariaDB MaxScale the aim is to take the reader from the point of installation to making the decision as to which of the various setups that are possible with MariaDB MaxScale should be the initial configuration to use. One of the problems that new users to MariaDB MaxScale suffer is deciding exactly what they should consider as a base configuration to start exploring what MariaDB MaxScale is capable of. MariaDB MaxScale is highly configurable, with new plugins expanding the capabilities of MariaDB MaxScale, whilst this makes it a very adaptable tool it does lead to an initial hurdle in configuring MariaDB MaxScale.
In this introduction to MariaDB MaxScale the aim is to take the reader
from the point of installation to making the decision as to which of
the various setups that are possible with MariaDB MaxScale should be
the initial configuration to use. One of the problems that new users to
MariaDB MaxScale suffer is deciding exactly what they should consider
as a base configuration to start exploring what MariaDB MaxScale
is capable of. MariaDB MaxScale is highly configurable,
with new plugins expanding the capabilities of MariaDB MaxScale,
whilst this makes it a very adaptable tool it does lead to an initial
hurdle in configuring MariaDB MaxScale.
## Installation
MariaDB MaxScale can be installed either using the MariaDB Enterprise Repository or directly from a downloaded package.
MariaDB MaxScale can be installed either using the MariaDB Enterprise Repository
or directly from a downloaded package.
### Using the MariaDB Enterprise Repository
@ -40,7 +50,8 @@ $ sudo apt-get install -f
### Starting MariaDB MaxScale
Before starting MariaDB MaxScale, you need to create a configuration file for it; please see further [down](#configuring-mariadb-maxscale).
Before starting MariaDB MaxScale, you need to create a configuration file for it;
please see further [down](#configuring-mariadb-maxscale).
Once a configuration file has been created you can start MariaDB MaxScale:
@ -48,7 +59,8 @@ Once a configuration file has been created you can start MariaDB MaxScale:
systemctl start maxscale.service
```
If your system does not support systemd you can start MariaDB MaxScale using the installed init.d script.
If your system does not support systemd you can start MariaDB MaxScale using the
installed init.d script.
```
service maxscale start
@ -56,56 +68,130 @@ service maxscale start
Starting with version 2.0.3, MaxScale also supports Upstart.
An example configuration file is installed into the `/etc/` folder. This file should be changed according to your needs.
An example configuration file is installed into the `/etc/` folder.
This file should be changed according to your needs.
## Install MariaDB MaxScale Using a Tarball
MaxScale can also be installed using a tarball. That may be required if you are using a Linux distribution for which there exist no installation package or if you want to install many different MaxScale versions side by side. For instructions on how to do that, please refer to [Install MariaDB MaxScale using a Tarball](Install-MariaDB-MaxScale-Using-a-Tarball.md).
MaxScale can also be installed using a tarball.
That may be required if you are using a Linux distribution for which there
exist no installation package or if you want to install many different
MaxScale versions side by side. For instructions on how to do that, please refer to
[Install MariaDB MaxScale using a Tarball](Install-MariaDB-MaxScale-Using-a-Tarball.md).
## Building MariaDB MaxScale From Source Code
Alternatively you may download the MariaDB MaxScale source and build your own binaries. To do this, refer to the separate document [Building MariaDB MaxScale from Source Code](Building-MaxScale-from-Source-Code.md)
Alternatively you may download the MariaDB MaxScale source and build your own binaries.
To do this, refer to the separate document
[Building MariaDB MaxScale from Source Code](Building-MaxScale-from-Source-Code.md)
## Configuring MariaDB MaxScale
The first step in configuring your MariaDB MaxScale is to determine what it is you want to achieve with your MariaDB MaxScale and what environment it will run in. The later is probably the easiest starting point for choosing which configuration route you wish to take. There are two distinct database environments which the first GA release of MariaDB MaxScale supports; MySQL Master/Slave Replication clusters and Galera Cluster.
The first step in configuring your MariaDB MaxScale is to determine
what it is you want to achieve with your MariaDB MaxScale and what environment
it will run in. The later is probably the easiest starting point for choosing
which configuration route you wish to take.
There are two distinct database environments which the first GA release
of MariaDB MaxScale supports; MySQL Master/Slave Replication clusters and Galera Cluster.
For more details, refer to the [Configuration Guide](Configuration-Guide.md).
### Master/Slave Replication Clusters
There are two major configuration options available to use MariaDB MaxScale with a MySQL Replication cluster; connection routing with separate read and write connections, or read/write splitting with a single connection. A separate tutorial is available for each of these configurations that describes how to build the configuration file for MariaDB MaxScale that will work with your environment.
There are two major configuration options available to use MariaDB MaxScale
with a MySQL Replication cluster; connection routing with separate read and
write connections, or read/write splitting with a single connection.
A separate tutorial is available for each of these configurations that
describes how to build the configuration file for MariaDB MaxScale that
will work with your environment.
Using a MySQL Master/Slave Replication cluster will provide one node server within the cluster that is the master server and the remainder of the servers will be slaves. The slaves are read replicas of the master. In a replication cluster like this all write operations must be performed on the master. This can provide not just added security of your data, but also read scalability. In an application environment with a substantial proportions of read operations, directing those read operations to the slave servers can increase the total load which the system can handle by offloading the master server from the burden of these read operations.
Using a MySQL Master/Slave Replication cluster will provide one node server
within the cluster that is the master server and the remainder of the servers
will be slaves. The slaves are read replicas of the master.
In a replication cluster like this all write operations must be performed
on the master.
This can provide not just added security of your data, but also read scalability.
In an application environment with a substantial proportions of read operations,
directing those read operations to the slave servers can increase
the total load which the system can handle by offloading the master server
from the burden of these read operations.
Making the choice between these two setups is relatively simple, if you have an application that understands that there are some database servers that it can only read from and one it must send all of the writes to, then the connection routing option can be used. Applications that are not written to separate read and write statements must use a service within MariaDB MaxScale that will split the incoming stream of SQL statements into operations that can be executed on the master and those that can be set to the slave. These applications should use the statement based routing provided by the Read/Write Splitter router.
Making the choice between these two setups is relatively simple,
if you have an application that understands that there are some database servers
that it can only read from and one it must send all of the writes to,
then the connection routing option can be used.
Applications that are not written to separate read and write statements must use
a service within MariaDB MaxScale that will split the incoming stream of SQL statements
into operations that can be executed on the master and those that can be set to the slave.
These applications should use the statement based routing provided by
the Read/Write Splitter router.
### Galera Cluster
A Galera Cluster provides a true multi-master cluster option for MariaDB and MySQL database environments. In such a setup any node that is part of the cluster can be used to both execute read and write operations. MariaDB MaxScale again offers two different configurations that can be used with Galera; a connection balancing configuration or a statement splitting mechanism that can be used to isolate write operations to a single node within the cluster. Again there is a tutorial guide available for both of these major configurations.
A Galera Cluster provides a true multi-master cluster option for MariaDB and MySQL
database environments. In such a setup any node that is part of the cluster
can be used to both execute read and write operations.
MariaDB MaxScale again offers two different configurations that can be used with Galera;
a connection balancing configuration or a statement splitting mechanism that can be used
to isolate write operations to a single node within the cluster.
Again there is a tutorial guide available for both of these major configurations.
The connection based load balancing configuration is used in an environment in which you have a cluster that you want to be available to an application without the application needing to be aware of the cluster configuration or state of the database nodes. MariaDB MaxScale will monitor the nodes within the database cluster and will route connections from the application to database nodes that are active members of the cluster. MariaDB MaxScale will also keep track of the number of connections to each database node keep equal numbers of connections to each node, at the time the connection is established.
The connection based load balancing configuration is used in an environment in which
you have a cluster that you want to be available to an application without
the application needing to be aware of the cluster configuration or state of
the database nodes.
MariaDB MaxScale will monitor the nodes within the database cluster and will
route connections from the application to database nodes that
are active members of the cluster.
MariaDB MaxScale will also keep track of the number of connections to each
database node keep equal numbers of connections to each node,
at the time the connection is established.
It is also possible to use the Read/Write Splitter with Galera. Although it is not necessary to segregate the write operations to a single node, there are advantages in doing this if you have an application where the write load is not too great to be handled by a single node in the cluster. Galera Cluster uses an optimistic locking strategy that will allow transactions to progress independently on each node within the cluster. It is only when the transaction commits that the transaction is checked for conflicts with other transactions that are committing on the other nodes. At this stage the commit can fail with a deadlock detection error. This can be inconvenient for applications and, some older applications, that are not aware that the transaction can fail at this stage may not check for this failure. Using the Read/Write Splitter will allow this to be avoided since it will isolate the write to one node and no deadlock detection will occur. MariaDB MaxScale provides a monitoring module that will maintain pseudo states of master and slave for the Galera cluster that allows for this type of configuration.
It is also possible to use the Read/Write Splitter with Galera.
Although it is not necessary to segregate the write operations to a single node,
there are advantages in doing this if you have an application where the write load
is not too great to be handled by a single node in the cluster.
Galera Cluster uses an optimistic locking strategy that will allow transactions
to progress independently on each node within the cluster.
It is only when the transaction commits that the transaction is checked for conflicts
with other transactions that are committing on the other nodes.
At this stage the commit can fail with a deadlock detection error.
This can be inconvenient for applications and, some older applications,
that are not aware that the transaction can fail at this stage
may not check for this failure.
Using the Read/Write Splitter will allow this to be avoided since
it will isolate the write to one node and no deadlock detection will occur.
MariaDB MaxScale provides a monitoring module that will maintain pseudo states
of master and slave for the Galera cluster that allows for this type of configuration.
### Other MariaDB MaxScale Configuration
As well as the four major configuration choices outlined above there are also other configurations sub-options that may be mixed with those to provide a variety of different configuration and functionality. The MariaDB MaxScale filter concept allows the basic configurations to be built upon in a large variety of ways. A separate filter tutorial is available that discusses the concept and gives some examples of ways to use filters.
As well as the four major configuration choices outlined above there are also other
configurations sub-options that may be mixed with those to provide a variety of different
configuration and functionality. The MariaDB MaxScale filter concept allows the basic configurations
to be built upon in a large variety of ways. A separate filter tutorial is available
that discusses the concept and gives some examples of ways to use filters.
## Encrypting Passwords
Passwords stored in the maxscale.cnf file may optionally be encrypted for added security. This is done by creation of an encryption key on installation of MariaDB MaxScale. Encryption keys may be created manually by executing the maxkeys utility with the argument of the filename to store the key. The default location MariaDB MaxScale stores the keys is `/var/lib/maxscale`.
Passwords stored in the maxscale.cnf file may optionally be encrypted for added security.
This is done by creation of an encryption key on installation of MariaDB MaxScale.
Encryption keys may be created manually by executing the maxkeys utility with the argument
of the filename to store the key. The default location MariaDB MaxScale stores
the keys is `/var/lib/maxscale`.
```
# Usage: maxkeys [PATH]
maxkeys /var/lib/maxscale/
```
Changing the encryption key for MariaDB MaxScale will invalidate any currently encrypted keys stored in the maxscale.cnf file.
Changing the encryption key for MariaDB MaxScale will invalidate any currently
encrypted keys stored in the maxscale.cnf file.
### Creating Encrypted Passwords
Encrypted passwords are created by executing the maxpasswd command with the location of the .secrets file and the password you require to encrypt as an argument.
Encrypted passwords are created by executing the maxpasswd command with the location
of the .secrets file and the password you require to encrypt as an argument.
```
# Usage: maxpasswd PATH PASSWORD
@ -113,7 +199,10 @@ maxpasswd /var/lib/maxscale/ MaxScalePw001
61DD955512C39A4A8BC4BB1E5F116705
```
The output of the maxpasswd command is a hexadecimal string, this should be inserted into the maxscale.cnf file in place of the ordinary, plain text, password. MariaDB MaxScale will determine this as an encrypted password and automatically decrypt it before sending it the database server.
The output of the maxpasswd command is a hexadecimal string, this should be inserted
into the maxscale.cnf file in place of the ordinary, plain text, password.
MariaDB MaxScale will determine this as an encrypted password and automatically decrypt
it before sending it the database server.
```
[Split Service]
@ -135,8 +224,19 @@ modules it will search using a configurable search path. The priority of these p
2. Look in the directory defined with libdir=PATH in the configuration file under the [maxscale] section
3. Look in default directory in /usr/lib64/maxscale
Configuration is read by default from the file /etc/maxscale.cnf. An example file is included in in the installation and can be found in the /usr/share/maxscale folder within the MariaDB MaxScale installation. The -f flag can be used on the command line to set the name and the location of the configuration file. The -C flag can be used to set the directory where the configuration file is searched for. Without the -f or -C flags, the file is read from the /etc directory.
Configuration is read by default from the file /etc/maxscale.cnf. An example file is
included in in the installation and can be found in the /usr/share/maxscale folder within
the MariaDB MaxScale installation. The -f flag can be used on the command line to set
the name and the location of the configuration file. The -C flag can be used to set
the directory where the configuration file is searched for. Without the -f or -C flags,
the file is read from the /etc directory.
## Administration Of MariaDB MaxScale
There are various administration tasks that may be done with MariaDB MaxScale, a client command, maxadmin, is available that will interact with a running MariaDB MaxScale and allow the status of MariaDB MaxScale to be monitored and give some control of the MariaDB MaxScale functionality. There is [a separate reference guide](../Reference/MaxAdmin.md) for the maxadmin utility and also [a short administration tutorial](../Tutorials/Administration-Tutorial.md) that covers the common administration tasks that need to be done with MariaDB MaxScale.
There are various administration tasks that may be done with MariaDB MaxScale,
a client command, maxadmin, is available that will interact with a running
MariaDB MaxScale and allow the status of MariaDB MaxScale to be monitored and
give some control of the MariaDB MaxScale functionality.
There is [a separate reference guide](../Reference/MaxAdmin.md) for the maxadmin utility
and also [a short administration tutorial](../Tutorials/Administration-Tutorial.md)
that covers the common administration tasks that need to be done with MariaDB MaxScale.

18
Documentation/Monitors/MM-Monitor.md
View File

@ -2,11 +2,15 @@
## Overview
The Multi-Master Monitor is a monitoring module for MaxScale that monitors Master-Master replication. It assigns master and slave roles inside MaxScale based on whether the read_only parameter on a server is set to off or on.
The Multi-Master Monitor is a monitoring module for MaxScale that monitors Master-Master replication.
It assigns master and slave roles inside MaxScale based on whether the read_only parameter on a server
is set to off or on.
## Configuration
A minimal configuration for a monitor requires a set of servers for monitoring and a username and a password to connect to these servers. The user requires the REPLICATION CLIENT privilege to successfully monitor the state of the servers.
A minimal configuration for a monitor requires a set of servers for monitoring and an username
and a password to connect to these servers. The user requires the REPLICATION CLIENT privilege
to successfully monitor the state of the servers.
```
[Multi-Master Monitor]
@ -20,7 +24,8 @@ passwd=mypwd
## Common Monitor Parameters
For a list of optional parameters that all monitors support, read the [Monitor Common](Monitor-Common.md) document.
For a list of optional parameters that all monitors support, read
the [Monitor Common](Monitor-Common.md) document.
## Multi-Master Monitor optional parameters
@ -28,9 +33,12 @@ These are optional parameters specific to the Multi-Master Monitor.
### `detect_stale_master`
Allow previous master to be available even in case of stopped or misconfigured replication. This allows services that depend on master and slave roles to continue functioning as long as the master server is available.
Allow previous master to be available even in case of stopped or misconfigured replication.
This allows services that depend on master and slave roles to continue functioning as long as
the master server is available.
This is a situation which can happen if all slave servers are unreachable or the replication breaks for some reason.
This is a situation which can happen if all slave servers are unreachable or the
replication breaks for some reason.
```
detect_stale_master=true

116
Documentation/Tutorials/Galera-Cluster-Connection-Routing-Tutorial.md
View File

@ -2,33 +2,53 @@
# Environment & Solution Space
The object of this tutorial is to have a system that has two ports available, one for write connections to the database cluster and the other for read connections to the database.
The object of this tutorial is to have a system that has two ports available, one for
write connections to the database cluster and the other for read connections to the database.
## Setting up MariaDB MaxScale
The first part of this tutorial is covered in [MariaDB MaxScale Tutorial](MaxScale-Tutorial.md). Please read it and follow the instructions for setting up MariaDB MaxScale with the type of cluster you want to use.
The first part of this tutorial is covered in [MariaDB MaxScale Tutorial](MaxScale-Tutorial.md).
Please read it and follow the instructions for setting up MariaDB MaxScale with the
type of cluster you want to use.
Once you have MariaDB MaxScale installed and the database users created, we can create the configuration file for MariaDB MaxScale.
Once you have MariaDB MaxScale installed and the database users created, we can create
the configuration file for MariaDB MaxScale.
## Creating Your MariaDB MaxScale Configuration
MariaDB MaxScale configuration is held in an ini file that is located in the file maxscale.cnf in the directory /etc, if you have installed in the default location then this file is available in /etc/maxscale.cnf. This is not created as part of the installation process and must be manually created. A template file does exist within the /usr/share/maxscale directory that may be use as a basis for your configuration.
MariaDB MaxScale configuration is held in an ini file that is located in the file maxscale.cnf
in the directory /etc, if you have installed in the default location then this file
is available in /etc/maxscale.cnf. This is not created as part of the installation process
and must be manually created. A template file does exist within the /usr/share/maxscale directory
that may be use as a basis for your configuration.
A global, maxscale, section is included within every MariaDB MaxScale configuration file; this is used to set the values of various MariaDB MaxScale wide parameters, perhaps the most important of these is the number of threads that MariaDB MaxScale will use to execute the code that forwards requests and handles responses for clients.
A global, maxscale, section is included within every MariaDB MaxScale configuration file;
this is used to set the values of various MariaDB MaxScale wide parameters, perhaps
the most important of these is the number of threads that MariaDB MaxScale will use
to execute the code that forwards requests and handles responses for clients.
```
[maxscale]
threads=4
```
Since we are using Galera Cluster and connection routing we want a single to which the client application can connect; MariaDB MaxScale will then route connections to this port onwards to the various nodes within the Galera Cluster. To achieve this within MariaDB MaxScale we need to define a service in the ini file. Create a section for each in your MariaDB MaxScale configuration file and set the type to service, the section name is the names of the service and should be meaningful to the administrator. Names may contain whitespace.
Since we are using Galera Cluster and connection routing we want a single to which
the client application can connect; MariaDB MaxScale will then route connections to
this port onwards to the various nodes within the Galera Cluster. To achieve this
within MariaDB MaxScale we need to define a service in the ini file. Create a section
for each in your MariaDB MaxScale configuration file and set the type to service,
the section name is the names of the service and should be meaningful to
the administrator. Names may contain whitespace.
```
[Galera Service]
type=service
```
The router for this section the readconnroute module, also the service should be provided with the list of servers that will be part of the cluster. The server names given here are actually the names of server sections in the configuration file and not the physical hostnames or addresses of the servers.
The router for this section the readconnroute module, also the service should be
provided with the list of servers that will be part of the cluster. The server names
given here are actually the names of server sections in the configuration file and not
the physical hostnames or addresses of the servers.
```
[Galera Service]
@ -37,7 +57,12 @@ router=readconnroute
servers=dbserv1, dbserv2, dbserv3
```
In order to instruct the router to which servers it should route we must add router options to the service. The router options are compared to the status that the monitor collects from the servers and used to restrict the eligible set of servers to which that service may route. In our case we use the option that restricts us to servers that are fully functional members of the Galera cluster which are able to support SQL operations on the cluster. To achieve this we use the router option synced.
In order to instruct the router to which servers it should route we must add router
options to the service. The router options are compared to the status that the monitor
collects from the servers and used to restrict the eligible set of servers to which that
service may route. In our case we use the option that restricts us to servers that are
fully functional members of the Galera cluster which are able to support SQL operations
on the cluster. To achieve this we use the router option synced.
```
[Galera Service]
@ -47,14 +72,20 @@ router_options=synced
servers=dbserv1, dbserv2, dbserv3
```
The final step in the service section is to add the username and password that will be used to populate the user data from the database cluster. There are two options for representing the password, either plain text or encrypted passwords may be used. In order to use encrypted passwords a set of keys must be generated that will be used by the encryption and decryption process. To generate the keys use the maxkeys command and pass the name of the secrets file in which the keys are stored.
The final step in the service section is to add the username and password that will be
used to populate the user data from the database cluster. There are two options for
representing the password, either plain text or encrypted passwords may be used.
In order to use encrypted passwords a set of keys must be generated that will be used
by the encryption and decryption process. To generate the keys use the maxkeys command
and pass the name of the secrets file in which the keys are stored.
```
% maxkeys /var/lib/maxscale/.secrets
%
```
Once the keys have been created the maxpasswd command can be used to generate the encrypted password.
Once the keys have been created the maxpasswd command can be used to generate
the encrypted password.
```
% maxpasswd plainpassword
@ -62,7 +93,8 @@ Once the keys have been created the maxpasswd command can be used to generate th
%
```
The username and password, either encrypted or plain text, are stored in the service section using the user and passwd parameters.
The username and password, either encrypted or plain text, are stored in the service
section using the user and passwd parameters.
```
[Galera Service]
@ -74,7 +106,12 @@ user=maxscale
passwd=96F99AA1315BDC3604B006F427DD9484
```
This completes the definitions required by the service, however listening ports must be associated with a service in order to allow network connections. This is done by creating a series of listener sections. These sections again are named for the convenience of the administrator and should be of type listener with an entry labeled service which contains the name of the service to associate the listener with. Each service may have multiple listeners.
This completes the definitions required by the service, however listening ports must
be associated with a service in order to allow network connections. This is done by
creating a series of listener sections. These sections again are named for the convenience
of the administrator and should be of type listener with an entry labeled service which
contains the name of the service to associate the listener with.
Each service may have multiple listeners.
```
[Galera Listener]
@ -82,7 +119,10 @@ type=listener
service=Galera Service
```
A listener must also define the protocol module it will use for the incoming network protocol, currently this should be the MySQLClient protocol for all database listeners. The listener may then supply a network port to listen on and/or a socket within the file system.
A listener must also define the protocol module it will use for the incoming
network protocol, currently this should be the MySQLClient protocol for all
database listeners. The listener may then supply a network port to listen on
and/or a socket within the file system.
```
[Galera Listener]
@ -93,9 +133,15 @@ port=4306
socket=/tmp/DB.Cluster
```
An address parameter may be given if the listener is required to bind to a particular network address when using hosts with multiple network addresses. The default behavior is to listen on all network interfaces.
An address parameter may be given if the listener is required to bind to a particular
network address when using hosts with multiple network addresses.
The default behavior is to listen on all network interfaces.
The next stage is the configuration is to define the server information. This defines how to connect to each of the servers within the cluster, again a section is created for each server, with the type set to server, the network address and port to connect to and the protocol to use to connect to the server. Currently the protocol for all database connections in MySQLBackend.
The next stage is the configuration is to define the server information.
This defines how to connect to each of the servers within the cluster, again a section
is created for each server, with the type set to server, the network address and port to
connect to and the protocol to use to connect to the server. Currently the protocol for
all database connections in MySQLBackend.
```
[dbserv1]
@ -117,7 +163,12 @@ port=3306
protocol=MySQLBackend
```
In order for MariaDB MaxScale to monitor the servers using the correct monitoring mechanisms a section should be provided that defines the monitor to use and the servers to monitor. Once again a section is created with a symbolic name for the monitor, with the type set to monitor. Parameters are added for the module to use, the list of servers to monitor and the username and password to use when connecting to the the servers with the monitor.
In order for MariaDB MaxScale to monitor the servers using the correct monitoring
mechanisms a section should be provided that defines the monitor to use and
the servers to monitor. Once again a section is created with a symbolic name for
the monitor, with the type set to monitor. Parameters are added for the module to use,
the list of servers to monitor and the username and password to use when connecting to
the servers with the monitor.
```
[Galera Monitor]
@ -128,9 +179,12 @@ user=maxscale
passwd=96F99AA1315BDC3604B006F427DD9484
```
As with the password definition in the server either plain text or encrypted passwords may be used.
As with the password definition in the server either plain text or encrypted
passwords may be used.
The final stage in the configuration is to add the option service which is used by the maxadmin command to connect to MariaDB MaxScale for monitoring and administration purposes. This creates a service section and a listener section.
The final stage in the configuration is to add the option service which is used by
the maxadmin command to connect to MariaDB MaxScale for monitoring and
administration purposes. This creates a service section and a listener section.
```
[CLI]
@ -145,7 +199,9 @@ socket=default
## Starting MariaDB MaxScale
Upon completion of the configuration process MariaDB MaxScale is ready to be started for the first time. This may either be done manually by running the maxscale command or via the service interface.
Upon completion of the configuration process MariaDB MaxScale is ready to be
started for the first time. This may either be done manually by running the
maxscale command or via the service interface.
```
% maxscale
@ -157,7 +213,10 @@ or
% service maxscale start
```
Check the error log in /var/log/maxscale to see if any errors are detected in the configuration file and to confirm MariaDB MaxScale has been started. Also the maxadmin command may be used to confirm that MariaDB MaxScale is running and the services, listeners etc have been correctly configured.
Check the error log in /var/log/maxscale to see if any errors are detected in
the configuration file and to confirm MariaDB MaxScale has been started.
Also the maxadmin command may be used to confirm that MariaDB MaxScale is running
and the services, listeners etc have been correctly configured.
```
% maxadmin list services
@ -180,9 +239,14 @@ dbserv3 | 192.168.2.3 | 3306 | 0 | Running, Synced, Sl
-------------------+-----------------+-------+-------------+--------------------
```
A Galera Cluster is a multi-master clustering technology, however the monitor is able to impose false notions of master and slave roles within a Galera Cluster in order to facilitate the use of Galera as if it were a standard MySQL Replication setup. This is merely an internal MariaDB MaxScale convenience and has no impact on the behavior of the cluster.
A Galera Cluster is a multi-master clustering technology, however the monitor is able
to impose false notions of master and slave roles within a Galera Cluster in order to
facilitate the use of Galera as if it were a standard MySQL Replication setup.
This is merely an internal MariaDB MaxScale convenience and has no impact on the behavior of the cluster.
You can control which Galera node is the master server by using the _priority_ mechanism of the Galera Monitor module. For more details, read the [Galera Monitor](../Monitors/Galera-Monitor.md) documentation.
You can control which Galera node is the master server by using the _priority_
mechanism of the Galera Monitor module. For more details,
read the [Galera Monitor](../Monitors/Galera-Monitor.md) documentation.
```
% maxadmin list listeners
@ -197,4 +261,10 @@ CLI | maxscaled | localhost | 6603 | Running
%
```
MariaDB MaxScale is now ready to start accepting client connections and routing them to the master or slaves within your cluster. Other configuration options are available that can alter the criteria used for routing, such as using weights to obtain unequal balancing operations. These options may be found in the MariaDB MaxScale Configuration Guide. More detail on the use of maxadmin can be found in the document ["MaxAdmin - The MariaDB MaxScale Administration & Monitoring Client Application"](../Reference/MaxAdmin.md).
MariaDB MaxScale is now ready to start accepting client connections and routing
them to the master or slaves within your cluster. Other configuration options are
available that can alter the criteria used for routing, such as using weights to
obtain unequal balancing operations.
These options may be found in the MariaDB MaxScale Configuration Guide.
More detail on the use of maxadmin can be found in the document
["MaxAdmin - The MariaDB MaxScale Administration & Monitoring Client Application"](../Reference/MaxAdmin.md).

81
Documentation/Tutorials/MaxScale-Tutorial.md
View File

@ -1,10 +1,21 @@
# Setting up MariaDB MaxScale
This document is designed as a quick introduction to setting up MariaDB MaxScale in an environment in which you have either a MySQL Master-Slave replication cluster with one master and multiple slave servers or a multi-node Galera cluster. The process of setting and configuring MariaDB MaxScale will be covered within this document.
This document is designed as a quick introduction to setting up MariaDB MaxScale
in an environment in which you have either a MySQL Master-Slave replication cluster
with one master and multiple slave servers or a multi-node Galera cluster.
The process of setting and configuring MariaDB MaxScale will be covered within this document.
The installation and configuration of the MySQL Replication or the Galera cluster will not be covered nor will any discussion of installation management tools to handle automated or semi-automated failover of the replication cluster. The [Setting Up Replication](https://mariadb.com/kb/en/mariadb/setting-up-replication/) article on the MariaDB knowledgebase can help you get started with replication clusters and the [Getting Started With Mariadb Galera Cluster](https://mariadb.com/kb/en/mariadb/getting-started-with-mariadb-galera-cluster/) article will help you set up a Galera cluster.
The installation and configuration of the MySQL Replication or the Galera cluster
will not be covered nor will any discussion of installation management tools
to handle automated or semi-automated failover of the replication cluster.
The [Setting Up Replication](https://mariadb.com/kb/en/mariadb/setting-up-replication/)
article on the MariaDB knowledgebase can help you get started with replication clusters
and the [Getting Started With Mariadb Galera Cluster](https://mariadb.com/kb/en/mariadb/getting-started-with-mariadb-galera-cluster/) article will help you set up a Galera cluster.
This tutorial will assume the user is running from one of the binary distributions available and has installed this in the default location. Building from source code in GitHub is covered in the [Building from Source](../Getting-Started/Building-MaxScale-from-Source-Code.md) document.
This tutorial will assume the user is running from one of the binary distributions
available and has installed this in the default location.
Building from source code in GitHub is covered in the
[Building from Source](../Getting-Started/Building-MaxScale-from-Source-Code.md) document.
## Process
@ -18,19 +29,33 @@ The steps involved in setting up MariaDB MaxScale are:
## Installation
The precise installation process will vary from one distribution to another details of what to do with the RPM and DEB packages can be found on the download site when you select the distribution you are downloading from. The process involves setting up your package manager to include the MariaDB repositories and then running the package manager for your distribution (usually yum or apt-get).
The precise installation process will vary from one distribution to another
details of what to do with the RPM and DEB packages can be found on the download
site when you select the distribution you are downloading from.
The process involves setting up your package manager to include the MariaDB repositories
and then running the package manager for your distribution (usually yum or apt-get).
Upon successful completion of the installation command you will have MariaDB MaxScale installed and ready to be run but without a configuration. You must create a configuration file before you first run MariaDB MaxScale which is covered in a later section.
Upon successful completion of the installation command you will have MariaDB MaxScale
installed and ready to be run but without a configuration.
You must create a configuration file before you first run MariaDB MaxScale
which is covered in a later section.
## Creating Database Users
MariaDB MaxScale needs to connect to the backend databases and run queries for two reasons; one to determine the current state of the database and the other to retrieve the user information for the database cluster. The first pair of credentials will be used by the monitor modules and the second is used by MariaDB MaxScale itself. This may be done either using two separate usernames or with a single user.
MariaDB MaxScale needs to connect to the backend databases and run queries for
two reasons; one to determine the current state of the database and the other to
retrieve the user information for the database cluster. The first pair of
credentials will be used by the monitor modules and the second is used by
MariaDB MaxScale itself. This may be done either using two separate usernames
or with a single user.
The first user required must be able to select data from the table mysql.user, to create this user follow the steps below.
The first user required must be able to select data from the table mysql.user,
to create this user follow the steps below.
1. Connect to the current master server in your replication tree as the root user
2. Create the user, substituting the username, password and host on which maxscale runs within your environment
2. Create the user, substituting the username, password and host on which maxscale
runs within your environment
```
MariaDB [(none)]> create user '*username*'@'*maxscalehost*' identified by '*password*';
@ -42,7 +67,9 @@ MariaDB [(none)]> grant SELECT on mysql.user to '*username*'@'*maxscalehost*';
**Query OK, 0 rows affected (0.03 sec)**
```
Additionally, `SELECT` privileges on the `mysql.db` and `mysql.tables_priv` tables and `SHOW DATABASES` privileges are required in order to load databases name and grants suitable for database name authorization.
Additionally, `SELECT` privileges on the `mysql.db` and `mysql.tables_priv` tables
and `SHOW DATABASES` privileges are required in order to load databases name
and grants suitable for database name authorization.
```
MariaDB [(none)]> GRANT SELECT ON mysql.db TO 'username'@'maxscalehost';
@ -56,7 +83,11 @@ MariaDB [(none)]> GRANT SHOW DATABASES ON *.* TO 'username'@'maxscalehost';
**Query OK, 0 rows affected (0.00 sec)**
```
The second user is used to monitored the state of the cluster. This user, which may be the same username as the first, requires permissions to access the various sources of monitoring data. In order to monitor a replication cluster this user must be granted the role REPLICATION CLIENT. This is only required by the MySQL monitor and Multi-Master monitor modules.
The second user is used to monitored the state of the cluster. This user, which may be
the same username as the first, requires permissions to access the various sources
of monitoring data. In order to monitor a replication cluster this user must be granted
the role REPLICATION CLIENT. This is only required by the MySQL monitor
and Multi-Master monitor modules.
```
MariaDB [(none)]> grant REPLICATION CLIENT on *.* to '*username*'@'*maxscalehost*';
@ -64,17 +95,32 @@ MariaDB [(none)]> grant REPLICATION CLIENT on *.* to '*username*'@'*maxscalehost
**Query OK, 0 rows affected (0.00 sec)**
```
If you wish to use two different usernames for the two different roles of monitoring and collecting user information then create a different username using the first two steps from above.
If you wish to use two different usernames for the two different roles of monitoring
and collecting user information then create a different username using the first
two steps from above.
## Creating additional grants for users
Because MariaDB MaxScale sits between the clients and the backend databases, the backend databases will see all clients as if they were connecting from MariaDB MaxScale's address. This usually requires users to create additional grants for MariaDB MaxScale's hostname. The best way to describe this process is with an example.
Because MariaDB MaxScale sits between the clients and the backend databases,
the backend databases will see all clients as if they were connecting from
MariaDB MaxScale's address.
This usually requires users to create additional grants for MariaDB MaxScale's hostname.
The best way to describe this process is with an example.
User `'jdoe'@'192.168.0.200` has the following grant on the cluster: `GRANT SELECT, INSERT, UPDATE, DELETE ON *.* TO 'jdoe'@'192.168.0.200'`. When the user connects directly to the server it will see it as `'jdoe'@'192.168.0.200` connecting to the server and it will match the grant for `'jdoe'@'192.168.0.200`.
User `'jdoe'@'192.168.0.200` has the following grant on the cluster:
`GRANT SELECT, INSERT, UPDATE, DELETE ON *.* TO 'jdoe'@'192.168.0.200'`.
When the user connects directly to the server it will see it as
`'jdoe'@'192.168.0.200` connecting to the server and it will match
the grant for `'jdoe'@'192.168.0.200`.
If MariaDB MaxScale is at the address `192.168.0.101` and the user `jdoe` connects to this MariaDB MaxScale, the backend server will see the connection as `'jdoe'@'192.168.0.101'`. Since the backend server has no grants for `'jdoe'@'192.168.0.101'`, the connection from MariaDB MaxScale to the server will be refused.
If MariaDB MaxScale is at the address `192.168.0.101` and the user `jdoe`
connects to this MariaDB MaxScale, the backend server will see the connection as
`'jdoe'@'192.168.0.101'`. Since the backend server has no grants for
`'jdoe'@'192.168.0.101'`, the connection from MariaDB MaxScale to the server
will be refused.
We can fix this by either creating a matching grant for user `jdoe` from the MariaDB MaxScale address or by using a wildcard to cover both addresses.
We can fix this by either creating a matching grant for user `jdoe` from
the MariaDB MaxScale address or by using a wildcard to cover both addresses.
The quickest way to do this is by doing a SHOW GRANTS query:
```
@ -95,7 +141,10 @@ MariaDB [(none)]> GRANT SELECT, INSERT, UPDATE, DELETE ON *.* TO 'jdoe'@'192.168
Query OK, 0 rows affected (0.00 sec)
```
The other option is to use a wildcard grant like `GRANT SELECT, INSERT, UPDATE, DELETE ON *.* TO 'jdoe'@'%'`. This is more convenient but also less secure than having specific grants for both the client's address and MariaDB MaxScale's address.
The other option is to use a wildcard grant like
`GRANT SELECT, INSERT, UPDATE, DELETE ON *.* TO 'jdoe'@'%'`.
This is more convenient but also less secure than having specific grants
for both the client's address and MariaDB MaxScale's address.
## Creating the configuration file

85
Documentation/Tutorials/Nagios-Plugins.md
View File

@ -1,49 +1,40 @@
# MariaDB MaxScale Nagios plugins, for Nagios 3.5.1
Massimiliano Pinto
Last Updated: 12th March 2015
## Document History
<table>
<tr>
<td>Date</td>
<td>Change</td>
<td>Who</td>
</tr>
<tr>
<td>10th March 2015</td>
<td>Initial version</td>
<td>Massimiliano Pinto</td>
</tr>
<tr>
<td>20th May 2016</td>
<td>MaxAdmin uses UNIX socket only</td>
<td>Massimiliano Pinto</td>
</tr>
</table>
# Introduction
Nagios® Core™ is an Open Source system and network monitoring application. It watches hosts and services that you specify, alerting you when things go bad and when they get better.
Nagios plugins are compiled executables or scripts (Perl scripts, shell scripts, etc.) that can be run from a command line to check the status or a host or service. Nagios uses the results from plugins to determine the current status of hosts and services on your network.
Nagios core executes a plugin whenever there is a need to check the status of a service or host.
Nagios® Core™ is an Open Source system and network monitoring application.
It watches hosts and services that you specify, alerting you when things go bad
and when they get better.
Nagios plugins are compiled executables or scripts (Perl scripts, shell scripts, etc.)
that can be run from a command line to check the status or a host or service.
Nagios uses the results from plugins to determine the current status of hosts and
services on your network.
Nagios core executes a plugin whenever there is a need to check the status
of a service or host.
While MariaDB MaxScale resources and status can be monitored via CLI using maxadmin commands, Nagios Plugin provides an automated way for system administration and database administrators to monitor MariaDB MaxScale. The diagram below provides view of how Nagios and MariaDB MaxScale interact.
While MariaDB MaxScale resources and status can be monitored via CLI using
maxadmin commands, Nagios Plugin provides an automated way for system administration
and database administrators to monitor MariaDB MaxScale.
The diagram below provides view of how Nagios and MariaDB MaxScale interact.
![Nagios and MariaDB MaxScale interaction](images/HowMaxScaleWorksWithNagios.png)
There are three Nagios plugin scripts that MariaDB MaxScale provides.
1. check_maxscale_threads.pl: This command provides you the status of current running threads and events in the queue on MariaDB MaxScale Server. The Performance data associated with this command current and historic wait time for threads and events
1. check_maxscale_threads.pl: This command provides you the status of current running
threads and events in the queue on MariaDB MaxScale Server.
The Performance data associated with this command current and historic wait time for threads and events
2. check_maxscale_resources.pl: This command provides you status of various resources on MariaDB MaxScale server. The Performance data associated provides details on respective resources.
2. check_maxscale_resources.pl: This command provides you status of various resources
on MariaDB MaxScale server. The Performance data associated provides
details on respective resources.
Current resources are: modules, services, listeners, servers, sessions, filters.
3. check_maxscale_monitor.pl: This command provides you status of the configured monitor modules on MariaDB MaxScale server.
3. check_maxscale_monitor.pl: This command provides you status of the configured
monitor modules on MariaDB MaxScale server.
In order to use these scripts on your Nagios Server, you need to copy them from the MariaDB MaxScale binary package or download them from source tree on GitHub.
In order to use these scripts on your Nagios Server, you need to copy them
from the MariaDB MaxScale binary package or download them from source tree on GitHub.
# MariaDB MaxScale Nagios Plugin Requirements
@ -65,10 +56,13 @@ socket=default
## Prepare Nagios configuration files.
Assuming Nagios installed on a separated server and the plugins are in /usr/lib64/nagios/plugins and configuration files are in /etc/nagios:
Assuming Nagios installed on a separated server and the plugins are
in /usr/lib64/nagios/plugins and configuration files are in /etc/nagios:
* Copy MariaDB MaxScale plugin scripts (./nagios/plugins/check_maxscale_*.pl) to /usr/lib64/nagios/plugins on Nagios Server
* Copy New commands and server1 definition (./nagios/plugins/maxscale_commands.cfg, server1.cfg) to /etc/nagios/objects/ on Nagios Server
* Copy MariaDB MaxScale plugin scripts (./nagios/plugins/check_maxscale_*.pl)
to /usr/lib64/nagios/plugins on Nagios Server
* Copy New commands and server1 definition (./nagios/plugins/maxscale_commands.cfg, server1.cfg)
to /etc/nagios/objects/ on Nagios Server
* Edit /etc/nagios/nagios.cfg on Nagios Server
and add (just after localhost.cfg or commands.cfg)
@ -87,14 +81,18 @@ cfg_file=/etc/nagios/objects/server1.cfg
- The default maxadmin executable path is /usr/bin/maxadmin can be changed by -m option
- default maxadmin socket (/tmp/maxadmin.sock) can be changed with -S option
- maxadmin executable is no longer required to be copied in Nagios server.
- the UNIX user in ssh connection should be also admin user for MariaDB MaxScale admin. First time access or no configured users means the "root" user is the only one that can access MariaDB MaxScale admin interface via UNIX socket.
- the UNIX user in ssh connection should be also admin user for MariaDB MaxScale admin.
First time access or no configured users means the "root" user is the only one that can access
MariaDB MaxScale admin interface via UNIX socket.
Test maxadmin with proper user in maxscale server and later via SSH.
Those checks are strongly recommended before using Nagios scripts.
For additional information about Maxadmin and MariaDB MaxScale administrative interface please refer to [MaxAdmin Utility](../Reference/MaxAdmin.md)
For additional information about Maxadmin and MariaDB MaxScale administrative interface
please refer to [MaxAdmin Utility](../Reference/MaxAdmin.md)
This example shows configuration that needs to be done on Nagios server in order to communicate to MariaDB MaxScale server that is running on host server1.
This example shows configuration that needs to be done on Nagios server in order to
communicate to MariaDB MaxScale server that is running on host server1.
In this example we are using the check_maxscale_resource as the check command
```
@ -109,7 +107,8 @@ In this example we are using the check_maxscale_resource as the check command
```
### Check new running monitors
* Restart Nagios and check new monitors are running in HTTP Interface "Current Status -> Services" on Nagios Server
* Restart Nagios and check new monitors are running in HTTP Interface
"Current Status -> Services" on Nagios Server
* Look for any errors in /var/log/nagios/nagios.log or nagios.debug on Nagios Server
# Nagios Plugin command line usage
@ -127,7 +126,8 @@ In this example we are using the check_maxscale_resource as the check command
-u <user> = username to connect to maxscale host via SSH (same user is used for maxadmin authentication)
-i <identity> = identity file to use for <user> at <host>
-m <maxadmin> = /path/to/maxadmin
-S <socket> = UNIX socket path between maxadmin and maxscale (default is /tmp/maxadmin.sock)
-S <socket> = UNIX socket path between maxadmin and maxscale (default
is /tmp/maxadmin.sock)
(2) ./check_maxscale_resources.pl -h
@ -166,7 +166,10 @@ Example for 'services'
```
#./check_maxscale_resources.pl -r resources
OK: 7 services found | services1=RW_Router;readwritesplit;1;1 services2=RW_Split;readwritesplit;1;1 services3=Test Service;readconnroute;1;1 services4=Master Service;readconnroute;2;2 services5=Debug Service;debugcli;1;1 services6=CLI;cli;2;145 services7=MaxInfo;maxinfo;2;2
OK: 7 services found | services1=RW_Router;readwritesplit;1;1 services2=RW_Split;
readwritesplit;1;1 services3=Test Service;readconnroute;1;1 services4=Master Service;
readconnroute;2;2 services5=Debug Service;debugcli;1;1 services6=CLI;cli;2;145
services7=MaxInfo;maxinfo;2;2
```
Returns OK and the number of services

54
Documentation/Tutorials/RabbitMQ-Setup-And-MaxScale-Integration.md
View File

@ -1,34 +1,42 @@
# Rabbit MQ setup and MariaDB MaxScale Integration
## Introduction
A step by step guide helps installing a RabbitMQ server and testing it before MariaDB MaxScale integration.
A step by step guide helps installing a RabbitMQ server and testing it before
MariaDB MaxScale integration.
New plugin filter and a message consumer application need to be compiled and linked with an external C library, RabbitMQ-c, that provides AMQP protocol integration.
New plugin filter and a message consumer application need to be compiled and
linked with an external C library, RabbitMQ-c, that provides AMQP protocol integration.
Custom configuration, with TCP/IP and Queue parameters, is also detailed here.
The software install setup provides RPM and DEB packaging and traditional compilation steps.
## Step 1 - Get the RabbitMQ binaries
On Centos 6.5 using fedora / RHEL rpm get the rpm from [http://www.rabbitmq.com/](http://www.rabbitmq.com/ "RabbitMQ")
On Centos 6.5 using fedora / RHEL rpm get the rpm from
[http://www.rabbitmq.com/](http://www.rabbitmq.com/ "RabbitMQ")
rabbitmq-server-3.3.4-1.noarch.rpm
Please note, before installing RabbitMQ, you must install Erlang.
Example:
```
yum install erlang
Package erlang-R14B-04.3.el6.x86_64 already installed and latest version
```
## Step 2 - Install and Start the Server
Install the packages using your distribution's package manager and start the server:
```
yum install rabbitmq-server-3.3.4-1.noarch.rpm
systemctl start rabbitmq-server.service
To configure your RabbitMQ server, please refer to the RabbitMQ website: [http://www.rabbitmq.com/](http://www.rabbitmq.com/ RabbitMQ website).
```
To configure your RabbitMQ server, please refer to the RabbitMQ website:
[http://www.rabbitmq.com/](http://www.rabbitmq.com/ RabbitMQ website).
rabbitmqctl is a command line tool for managing a RabbitMQ broker. It performs all actions by connecting to one of the broker's nodes.
rabbitmqctl is a command line tool for managing a RabbitMQ broker.
It performs all actions by connecting to one of the broker's nodes.
```
rabbitmqctl list_queues
@ -87,12 +95,14 @@ make
make install
```
Please note, this will install the packages to /usr. If you do not wish to install them to this location, provide a different value for the CMAKE_INSTALL_PREFIX variable.
Please note, this will install the packages to /usr. If you do not wish to install
them to this location, provide a different value for the CMAKE_INSTALL_PREFIX variable.
### Setup using the EPEL repository
Check how to configure your distribution for the EPEL repository: [https://fedoraproject.org/wiki/EPEL](https://fedoraproject.org/wiki/EPEL EPEL)
Check how to configure your distribution for the EPEL repository:
[https://fedoraproject.org/wiki/EPEL](https://fedoraproject.org/wiki/EPEL EPEL)
Configure your repositories and install the software:
@ -114,7 +124,9 @@ yum install rabbitmq-server
### Basic tests with library
The required library librabbitmq-c is now installed and we continue with basic operations with amqp_* tools, located in the examples/ folder of the build directory, testing client server interaction.
The required library librabbitmq-c is now installed and we continue with basic
operations with amqp_* tools, located in the examples/ folder of the build directory,
testing client server interaction.
Please note, those example applications may not be included in the RPM library packages.
@ -185,7 +197,8 @@ port=5672
logging_trigger=all
```
Logging triggers define whether to log all or a subset of the incoming queries using these options:
Logging triggers define whether to log all or a subset of the incoming queries
using these options:
```
# log only some elements or all
@ -230,9 +243,13 @@ anything targeting my1 table is logged
SELECT NOW(), SELECT MD5(“xyz)” are not logged
```
Please note that if we want to log only the user ‘maxtest’ accessing the schema ‘test’ with target ‘my1’ the option logging_strict must be set to TRUE and if we want to include those selects without schema name the option logging_log_all must be set to TRUE.
Please note that if we want to log only the user ‘maxtest’ accessing the
schema ‘test’ with target ‘my1’ the option logging_strict must be set
to TRUE and if we want to include those selects without schema name the
option logging_log_all must be set to TRUE.
The mqfilter logs into the MariaDB MaxScale TRACE log information about the matched logging triggers and the message delivering:
The mqfilter logs into the MariaDB MaxScale TRACE log information about
the matched logging triggers and the message delivering:
```
2014 09/03 06:22:04 Trigger is TRG_SOURCE: user: testuser = testuser
@ -288,7 +305,9 @@ and finally we can launch it:
# ./consumer
```
If the consumer.cnf file is not in the same directory as the binary file is, you can provide the location of the folder that it is in by passing it the -c flag followed by the path:
If the consumer.cnf file is not in the same directory as the binary file is,
you can provide the location of the folder that it is in by passing
it the -c flag followed by the path:
```
# ./consumer -c path/to/file
@ -298,7 +317,8 @@ and start MariaDB MaxScale as well
## Step 5 - Test the filter and check collected data
Assuming that MariaDB MaxScale and the message consumer are successfully running let’s connect to the service with an active mqfilter:
Assuming that MariaDB MaxScale and the message consumer are successfully
running let’s connect to the service with an active mqfilter:
```
[root@maxscale-02 MaxScale]# mysql -h 127.0.0.1 -P 4506 -uxxx -pyyy
@ -347,8 +367,10 @@ MariaDB [mqpairs]> select * from pairs;
7 rows in set (0.01 sec)
```
The filter send queries to the RabbitMQ server in the canonical format, i.e select RAND(?), RAND(?).
The queries Message Queue Consumer application gets from the server are stored with a counter that quickly shows how many times that normalized query was received:
The filter send queries to the RabbitMQ server in the canonical format,
i.e select RAND(?), RAND(?).
The queries Message Queue Consumer application gets from the server
are stored with a counter that quickly shows how many times that normalized query was received:
```
| 01050106010701080109010a010b010c10d | select RAND(?), RAND(?) | Columns: 2 | 2014-09-02 11:24:37 | 2014-09-02 11:29:15 | 3 |