376 lines
13 KiB
Markdown
376 lines
13 KiB
Markdown
# Rabbit MQ setup and MaxScale Integration
|
|
## Introduction
|
|
A step by step guide helps installing a RabbitMQ server and testing it before MaxScale integration.
|
|
|
|
New plugin filter and a message consumer application need to be compiled and linked with an external C library, RabbitMQ-c, that provides AMQP protocol integration.
|
|
Custom configuration, with TCP/IP and Queue parameters, is also detailed here.
|
|
The software install setup provides RPM and DEB packaging and traditional compilation steps.
|
|
|
|
## Step 1 - Get the RabbitMQ binaries
|
|
|
|
On Centos 6.5 using fedora / RHEL rpm get the rpm from [http://www.rabbitmq.com/](http://www.rabbitmq.com/ "RabbitMQ")
|
|
|
|
rabbitmq-server-3.3.4-1.noarch.rpm
|
|
|
|
Please note, before installing RabbitMQ, you must install Erlang.
|
|
|
|
Example:
|
|
|
|
yum install erlang
|
|
Package erlang-R14B-04.3.el6.x86_64 already installed and latest version
|
|
|
|
## Step 2 - Install and Start the Server
|
|
|
|
Install the packages using your distribution's package manager and start the server:
|
|
|
|
yum install rabbitmq-server-3.3.4-1.noarch.rpm
|
|
systemctl start rabbitmq-server.service
|
|
|
|
To configure your RabbitMQ server, please refer to the RabbitMQ website: [http://www.rabbitmq.com/](http://www.rabbitmq.com/ RabbitMQ website).
|
|
|
|
rabbitmqctl is a command line tool for managing a RabbitMQ broker. It performs all actions by connecting to one of the broker's nodes.
|
|
|
|
rabbitmqctl list_queues
|
|
rabbitmqctl list_queues | list_exchanges| cluster_status | list_bindings | list_connections | list_consumers | status
|
|
|
|
Example output:
|
|
|
|
[root@maxscale-02 MaxScale]# rabbitmqctl status
|
|
Status of node 'rabbit@maxscale-02' ...
|
|
[{pid,12251},
|
|
{running_applications,[{rabbit,"RabbitMQ","3.3.4"},
|
|
{os_mon,"CPO CXC 138 46","2.2.7"},
|
|
{xmerl,"XML parser","1.2.10"},
|
|
{mnesia,"MNESIA CXC 138 12","4.5"},
|
|
{sasl,"SASL CXC 138 11","2.1.10"},
|
|
{stdlib,"ERTS CXC 138 10","1.17.5"},
|
|
{kernel,"ERTS CXC 138 10","2.14.5"}]},
|
|
{os,{unix,linux}},
|
|
{erlang_version,"Erlang R14B04 (erts-5.8.5) [source] [64-bit] [smp:2:2] [rq:2] [async-threads:30] [kernel-poll:true]\n"},
|
|
...
|
|
{listeners,[{clustering,25672,"::"},{amqp,5672,"::"}]},
|
|
...
|
|
...done.
|
|
|
|
|
|
[root@maxscale-02 MaxScale]# rabbitmqctl list_bindings
|
|
Listing bindings ...
|
|
x1 exchange q1 queue k1 []
|
|
...done.
|
|
|
|
Interaction with the server may require stop & reset at some point:
|
|
|
|
rabbitmqctl stop_app
|
|
rabbitmqctl reset
|
|
rabbitmqctl start_app
|
|
|
|
## Step 3 - Install and test the client libraries
|
|
|
|
The selected library for MaxScale integration of RabbitMQ is:
|
|
[https://github.com/alanxz/rabbitmq-c](https://github.com/alanxz/rabbitmq-c RabbitMQ-C)
|
|
|
|
### Manual software compilation
|
|
|
|
To compile the RabbitMQ-C libraries manually:
|
|
|
|
git clone https://github.com/alanxz/rabbitmq-c.git
|
|
cd rabbitmq-c
|
|
cmake -DCMAKE_INSTALL_PREFIX=/usr .
|
|
make
|
|
make install
|
|
|
|
Please note, this will install the packages to /usr. If you do not wish to install them to this location, provide a different value for the CMAKE_INSTALL_PREFIX variable.
|
|
|
|
|
|
### Setup using the EPEL repository
|
|
|
|
Check how to configure your distribution for the EPEL repository: [https://fedoraproject.org/wiki/EPEL](https://fedoraproject.org/wiki/EPEL EPEL)
|
|
|
|
Configure your repositories and install the software:
|
|
|
|
yum install librabbitmq.x86_64
|
|
|
|
you might also like to install:
|
|
|
|
librabbitmq-tools.x86_64, librabbitmq-devel.x86_64
|
|
|
|
Please note you may also install the rabbitmq server from the EPEL repository:
|
|
|
|
yum install rabbitmq-server
|
|
|
|
|
|
|
|
|
|
### Basic tests with library
|
|
|
|
The required library librabbitmq-c is now installed and we continue with basic operations with amqp_* tools, located in the examples/ folder of the build directory, testing client server interaction.
|
|
|
|
Please note, those example applications may not be included in the RPM library packages.
|
|
|
|
#### Test 1 - create the exchange
|
|
|
|
[root@maxscale-02 examples]# ./amqp_exchange_declare
|
|
Usage: amqp_exchange_declare host port exchange exchangetype
|
|
|
|
Declare the exchange:
|
|
|
|
[root@maxscale-02 examples]# ./amqp_exchange_declare 127.0.0.1 5672 foo direct
|
|
|
|
#### Test 2 - Listen to exchange with selected binding key
|
|
|
|
[root@maxscale-02 examples]# ./amqp_listen
|
|
Usage: amqp_listen host port exchange bindingkey
|
|
|
|
Start the listener:
|
|
|
|
[root@maxscale-02 examples]# ./amqp_listen 127.0.0.1 5672 foo k1 &
|
|
|
|
#### Test 3 - Send a message …
|
|
|
|
[root@maxscale-02 examples]# ./amqp_sendstring
|
|
Usage: amqp_sendstring host port exchange routingkey messagebody
|
|
|
|
[root@maxscale-02 examples]# ./amqp_sendstring 127.0.0.1 5672 foo k1 “This is a new message”
|
|
|
|
... and watch the listener output
|
|
|
|
Delivery 1, exchange foo routingkey k1
|
|
Content-type: text/plain
|
|
|
|
|
|
## Step 4 - MaxScale integration with librabbitmq-c
|
|
|
|
A new filter (mqfilter.c) is implemented in order to send messages to the rabbitmq server and a message consumer (rabbitmq_consumer/consumer.c) program will get messages and store them into a MySQL/MariaDB database.
|
|
A quick way to install MaxScale with the RabbitMQ filter is to go to the MaxScale source directory and run the following commands:
|
|
|
|
mkdir build
|
|
cd build
|
|
cmake .. -DBUILD_RABBITMQ=Y
|
|
make
|
|
make install
|
|
|
|
To build the RabbitMQ filter CMake needs an additional parameter:
|
|
|
|
-DBUILD_RABBITMQ=Y
|
|
|
|
If the librabbitmq-c library is manually compiled it may be necessary to manually pass the location of the libraries and header files to CMake.
|
|
|
|
Libraries:
|
|
|
|
-DRABBITMQ_LIBRARIES=<path to RabbitMQ-c libraries>
|
|
|
|
Headers:
|
|
|
|
-DRABBITMQ_HEADERS=<path to RabbitMQ-c headers>
|
|
|
|
|
|
Please note, Message Queue Consumer (consumer.c) also needs to be compiled with MySQL/MariaDB client libraries in addition to the RabbitMQ-c libraries. If you have your MySQL/MariaDB client libraries and headers in non-standard locations, you can pass them manually to CMake:
|
|
|
|
Libraries:
|
|
|
|
-DMYSQLCLIENT_LIBRARIES=<path to libraries>
|
|
|
|
Headers:
|
|
|
|
-DMYSQLCLIENT_HEADERS=<path to headers>
|
|
|
|
The message queue consumer must be also built as a separate task, it’s not built as part of MaxScale build system. To build it, run the following commands in the rabbitmq_consumer directory in the MaxScale source folder:
|
|
|
|
mkdir build
|
|
cd build
|
|
cmake ..
|
|
make
|
|
|
|
To install it:
|
|
|
|
make install
|
|
|
|
To build packages:
|
|
|
|
make package
|
|
|
|
This generates RPM or DEB packages based on your system. These packages can then be installed on remote systems for easy access to the data generated by the consumer client.
|
|
|
|
## Step 5 - Configure new applications
|
|
|
|
The new filter needs to be configured in MaxScale.cnf.
|
|
|
|
[Test Service]
|
|
type=service
|
|
router=readconnroute
|
|
router_options=slave
|
|
servers=server1,server2,server3,server5,server4
|
|
user=massi
|
|
passwd=massi
|
|
filters=MQ
|
|
|
|
[MQ]
|
|
type=filter
|
|
module=mqfilter
|
|
exchange=x1
|
|
key=k1
|
|
queue=q1
|
|
hostname=127.0.0.1
|
|
port=5672
|
|
logging_trigger=all
|
|
|
|
|
|
|
|
Logging triggers define whether to log all or a subset of the incoming queries using these options:
|
|
|
|
# log only some elements or all
|
|
logging_trigger=[all,source,schema,object]
|
|
|
|
# Whether to log only SELECT, UPDATE, INSERT and DELETE queries or all possible queries
|
|
logging_log_all=true|false
|
|
|
|
|
|
|
|
# Log only when any of the trigger parameters match or only if all parameters match
|
|
logging_strict=true|false
|
|
|
|
# specify objects
|
|
logging_object=mytable,another_table
|
|
|
|
# specify logged users
|
|
logging_source_user=testuser,testuser
|
|
|
|
|
|
# specify source addresses
|
|
logging_source_host=127.0.0.1,192.168.10.14
|
|
|
|
# specify schemas
|
|
logging_schema=employees,orders,catalog
|
|
|
|
|
|
Example:
|
|
|
|
logging_trigger=object,schema,source
|
|
logging_strict=false
|
|
logging_log_all=false
|
|
logging_object=my1
|
|
logging_schema=test
|
|
logging_source_user=maxtest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The logging result of the example is:
|
|
|
|
if user maxtest does something, it's logged
|
|
and all queries in test schema are logged
|
|
anything targeting my1 table is logged
|
|
SELECT NOW(), SELECT MD5(“xyz)” are not logged
|
|
|
|
Please note that if we want to log only the user ‘maxtest’ accessing the schema ‘test’ with target ‘my1’ the option logging_strict must be set to TRUE and if we want to include those selects without schema name the option logging_log_all must be set to TRUE.
|
|
|
|
The mqfilter logs into the MaxScale TRACE log information about the matched logging triggers and the message delivering:
|
|
|
|
2014 09/03 06:22:04 Trigger is TRG_SOURCE: user: testuser = testuser
|
|
2014 09/03 06:22:04 Trigger is TRG_SCHEMA: test = test
|
|
2014 09/03 06:22:04 Trigger is TRG_OBJECT: test.t1 = t1
|
|
2014 09/03 06:22:04 Routing message to: 127.0.0.1:5672 / as guest/guest, exchange: x1<direct> key:k1 queue:q1
|
|
|
|
The consumer application needs to be configured as well:
|
|
|
|
|
|
#The options for the consumer are:
|
|
#hostname RabbitMQ hostname
|
|
#port RabbitMQ port
|
|
#vhost RabbitMQ virtual host
|
|
#user RabbitMQ username
|
|
#passwd RabbitMQ password
|
|
|
|
|
|
#queue Name of the queue to use
|
|
#dbserver SQL server name
|
|
#dbport SQL server port
|
|
#dbname Name of the database to use
|
|
#dbuser SQL server username
|
|
#dbpasswd SQL server password
|
|
#logfile Message log filename
|
|
|
|
[consumer]
|
|
hostname=127.0.0.1
|
|
port=5672
|
|
vhost=/
|
|
user=guest
|
|
passwd=guest
|
|
queue=q1
|
|
dbserver=127.0.0.1
|
|
dbport=3308
|
|
dbname=mqpairs
|
|
dbuser=xxx
|
|
dbpasswd=yyy
|
|
|
|
We may probably need to modify LD_LIBRARY_PATH before launching ‘consumer’:
|
|
|
|
# export LD_LIBRARY_PATH=/packages/rabbitmq-c/rabbitmq-c/librabbitmq:/packages/mariadb_client-2.0.0-Linux/lib/mariadb:/usr/lib64
|
|
|
|
and finally we can launch it:
|
|
|
|
# ./consumer
|
|
|
|
If the consumer.cnf file is not in the same directory as the binary file is, you can provide the location of the folder that it is in by passing it the -c flag followed by the path:
|
|
|
|
# ./consumer -c path/to/file
|
|
|
|
and start maxScale as well
|
|
|
|
## Step 6 - Test the filter and check collected data
|
|
Assuming that MaxScale and the message consumer are successfully running let’s connect to the service with an active mqfilter:
|
|
|
|
[root@maxscale-02 MaxScale]# mysql -h 127.0.0.1 -P 4506 -uxxx -pyyy
|
|
...
|
|
MariaDB [(none)]> select RAND(3), RAND(5);
|
|
+--------------------+---------------------+
|
|
| RAND(3) | RAND(5) |
|
|
+--------------------+---------------------+
|
|
| 0.9057697559760601 | 0.40613597483014313 |
|
|
+--------------------+---------------------+
|
|
1 row in set (0.01 sec)
|
|
|
|
…
|
|
MariaDB [(none)]> select RAND(3544), RAND(11);
|
|
|
|
|
|
|
|
we can check the consumer output in the terminal where it was started:
|
|
|
|
--------------------------------------------------------------
|
|
Received: 1409671452|select @@version_comment limit ?
|
|
Received: 1409671452|Columns: 1
|
|
...
|
|
Received: 1409671477|select RAND(?), RAND(?)
|
|
Received: 1409671477|Columns: 2
|
|
|
|
We query now the database for the content collected so far:
|
|
|
|
MariaDB [(none)]> use mqpairs;
|
|
Database changed
|
|
|
|
|
|
|
|
MariaDB [mqpairs]> select * from pairs;
|
|
|
|
+-------------------------------------+----------------------------------+------------+---------------------+---------------------+---------+
|
|
| tag | query | reply | date_in | date_out | counter |
|
|
+-------------------------------------+----------------------------------+------------+---------------------+---------------------+---------+
|
|
| 006c006d006e006f007000710072007374 | select @@version_comment limit ? | Columns: 1 | 2014-09-02 11:14:51 | 2014-09-02 11:26:38 | 3 |
|
|
| 00750076007700780079007a007b007c7d | SELECT DATABASE() | Columns: 1 | 2014-09-02 11:14:56 | 2014-09-02 11:27:06 | 3 |
|
|
| 007e007f00800081008200830084008586 | show databases | Columns: 1 | 2014-09-02 11:14:56 | 2014-09-02 11:27:06 | 3 |
|
|
| 008700880089008a008b008c008d008e8f | show tables | Columns: 1 | 2014-09-02 11:14:56 | 2014-09-02 11:27:06 | 3 |
|
|
| 0090009100920093009400950096009798 | select * from mqpairs.pairs | Columns: 6 | 2014-09-02 11:15:00 | 2014-09-02 11:27:00 | 12 |
|
|
| 00fc00fd00fe00ff0100010101020103104 | select NOW() | Columns: 1 | 2014-09-02 11:24:23 | 2014-09-02 11:24:23 | 1 |
|
|
| 01050106010701080109010a010b010c10d | select RAND(?), RAND(?) | Columns: 2 | 2014-09-02 11:24:37 | 2014-09-02 11:24:37 | 1 |
|
|
+-------------------------------------+----------------------------------+------------+---------------------+---------------------+---------+
|
|
7 rows in set (0.01 sec)
|
|
|
|
The filter send queries to the RabbitMQ server in the canonical format, i.e select RAND(?), RAND(?).
|
|
The queries Message Queue Consumer application gets from the server are stored with a counter that quickly shows how many times that normalized query was received:
|
|
|
|
| 01050106010701080109010a010b010c10d | select RAND(?), RAND(?) | Columns: 2 | 2014-09-02 11:24:37 | 2014-09-02 11:29:15 | 3 |
|