diff --git a/Documentation/Tutorials/RabbitMQ-Setup-And-MaxScale-Integration.md b/Documentation/Tutorials/RabbitMQ-Setup-And-MaxScale-Integration.md index 5047731e8..7ffff76c2 100644 --- a/Documentation/Tutorials/RabbitMQ-Setup-And-MaxScale-Integration.md +++ b/Documentation/Tutorials/RabbitMQ-Setup-And-MaxScale-Integration.md @@ -29,11 +29,14 @@ systemctl start rabbitmq-server.service To configure your RabbitMQ server, please refer to the RabbitMQ website: [http://www.rabbitmq.com/](http://www.rabbitmq.com/ RabbitMQ website). rabbitmqctl is a command line tool for managing a RabbitMQ broker. It performs all actions by connecting to one of the broker's nodes. + ``` rabbitmqctl list_queues rabbitmqctl list_queues | list_exchanges| cluster_status | list_bindings | list_connections | list_consumers | status ``` + Example output: + ``` [root@maxscale-02 MaxScale]# rabbitmqctl status Status of node 'rabbit@maxscale-02' ... @@ -58,12 +61,15 @@ Example output: x1 exchange q1 queue k1 [] ...done. ``` + Interaction with the server may require stop & reset at some point: + ``` rabbitmqctl stop_app rabbitmqctl reset rabbitmqctl start_app ``` + ## Step 3 - Install and test the client libraries The selected library for MaxScale integration of RabbitMQ is: @@ -72,6 +78,7 @@ The selected library for MaxScale integration of RabbitMQ is: ### Manual software compilation To compile the RabbitMQ-C libraries manually: + ``` git clone https://github.com/alanxz/rabbitmq-c.git cd rabbitmq-c @@ -79,6 +86,7 @@ cmake -DCMAKE_INSTALL_PREFIX=/usr . make make install ``` + Please note, this will install the packages to /usr. If you do not wish to install them to this location, provide a different value for the CMAKE_INSTALL_PREFIX variable. @@ -87,14 +95,19 @@ Please note, this will install the packages to /usr. If you do not wish to insta Check how to configure your distribution for the EPEL repository: [https://fedoraproject.org/wiki/EPEL](https://fedoraproject.org/wiki/EPEL EPEL) Configure your repositories and install the software: + ``` yum install librabbitmq.x86_64 ``` + you might also like to install: + ``` librabbitmq-tools.x86_64, librabbitmq-devel.x86_64 ``` + Please note you may also install the rabbitmq server from the EPEL repository: + ``` yum install rabbitmq-server ``` @@ -106,31 +119,42 @@ The required library librabbitmq-c is now installed and we continue with basic o Please note, those example applications may not be included in the RPM library packages. #### Test 1 - create the exchange + ``` [root@maxscale-02 examples]# ./amqp_exchange_declare Usage: amqp_exchange_declare host port exchange exchangetype ``` + Declare the exchange: + ``` [root@maxscale-02 examples]# ./amqp_exchange_declare 127.0.0.1 5672 foo direct ``` + #### Test 2 - Listen to exchange with selected binding key + ``` [root@maxscale-02 examples]# ./amqp_listen Usage: amqp_listen host port exchange bindingkey ``` + Start the listener: + ``` [root@maxscale-02 examples]# ./amqp_listen 127.0.0.1 5672 foo k1 & ``` + #### Test 3 - Send a message … + ``` [root@maxscale-02 examples]# ./amqp_sendstring Usage: amqp_sendstring host port exchange routingkey messagebody -``` + [root@maxscale-02 examples]# ./amqp_sendstring 127.0.0.1 5672 foo k1 “This is a new message” ``` + ... and watch the listener output + ``` Delivery 1, exchange foo routingkey k1 Content-type: text/plain @@ -140,6 +164,7 @@ Content-type: text/plain A new filter (mqfilter.c) is implemented in order to send messages to the rabbitmq server and a message consumer (rabbitmq_consumer/consumer.c) program will get messages and store them into a MySQL/MariaDB database. A quick way to install MaxScale with the RabbitMQ filter is to go to the MaxScale source directory and run the following commands: + ``` mkdir build cd build @@ -147,17 +172,23 @@ cmake .. -DBUILD_RABBITMQ=Y make make install ``` + To build the RabbitMQ filter CMake needs an additional parameter: + ``` -DBUILD_RABBITMQ=Y ``` + If the librabbitmq-c library is manually compiled it may be necessary to manually pass the location of the libraries and header files to CMake. Libraries: + ``` -DRABBITMQ_LIBRARIES= ``` + Headers: + ``` -DRABBITMQ_HEADERS= ``` @@ -165,33 +196,44 @@ Headers: Please note, Message Queue Consumer (consumer.c) also needs to be compiled with MySQL/MariaDB client libraries in addition to the RabbitMQ-c libraries. If you have your MySQL/MariaDB client libraries and headers in non-standard locations, you can pass them manually to CMake: Libraries: + ``` -DMYSQLCLIENT_LIBRARIES= ``` + Headers: + ``` -DMYSQLCLIENT_HEADERS= ``` + The message queue consumer must be also built as a separate task, it’s not built as part of MaxScale build system. To build it, run the following commands in the rabbitmq_consumer directory in the MaxScale source folder: + ``` mkdir build cd build cmake .. make ``` + To install it: + ``` make install ``` + To build packages: + ``` make package ``` + This generates RPM or DEB packages based on your system. These packages can then be installed on remote systems for easy access to the data generated by the consumer client. ## Step 5 - Configure new applications The new filter needs to be configured in maxscale.cnf. + ``` [Test Service] type=service @@ -214,6 +256,7 @@ logging_trigger=all ``` Logging triggers define whether to log all or a subset of the incoming queries using these options: + ``` # log only some elements or all logging_trigger=[all,source,schema,object] @@ -238,6 +281,7 @@ logging_schema=employees,orders,catalog ``` Example: + ``` logging_trigger=object,schema,source logging_strict=false @@ -247,28 +291,29 @@ logging_schema=test logging_source_user=maxtest ``` - - - - - The logging result of the example is: + ``` if user maxtest does something, it's logged and all queries in test schema are logged anything targeting my1 table is logged SELECT NOW(), SELECT MD5(“xyz)” are not logged ``` + Please note that if we want to log only the user ‘maxtest’ accessing the schema ‘test’ with target ‘my1’ the option logging_strict must be set to TRUE and if we want to include those selects without schema name the option logging_log_all must be set to TRUE. The mqfilter logs into the MaxScale TRACE log information about the matched logging triggers and the message delivering: + ``` 2014 09/03 06:22:04 Trigger is TRG_SOURCE: user: testuser = testuser 2014 09/03 06:22:04 Trigger is TRG_SCHEMA: test = test 2014 09/03 06:22:04 Trigger is TRG_OBJECT: test.t1 = t1 2014 09/03 06:22:04 Routing message to: 127.0.0.1:5672 / as guest/guest, exchange: x1 key:k1 queue:q1 + ``` + The consumer application needs to be configured as well: + ``` #The options for the consumer are: @@ -300,22 +345,31 @@ dbname=mqpairs dbuser=xxx dbpasswd=yyy ``` + We may probably need to modify LD_LIBRARY_PATH before launching ‘consumer’: + ``` # export LD_LIBRARY_PATH=/packages/rabbitmq-c/rabbitmq-c/librabbitmq:/packages/mariadb_client-2.0.0-Linux/lib/mariadb:/usr/lib64 ``` + and finally we can launch it: + ``` # ./consumer ``` + If the consumer.cnf file is not in the same directory as the binary file is, you can provide the location of the folder that it is in by passing it the -c flag followed by the path: +``` # ./consumer -c path/to/file +``` and start maxScale as well ## Step 6 - Test the filter and check collected data + Assuming that MaxScale and the message consumer are successfully running let’s connect to the service with an active mqfilter: + ``` [root@maxscale-02 MaxScale]# mysql -h 127.0.0.1 -P 4506 -uxxx -pyyy ... @@ -326,13 +380,12 @@ MariaDB [(none)]> select RAND(3), RAND(5); | 0.9057697559760601 | 0.40613597483014313 | +--------------------+---------------------+ 1 row in set (0.01 sec) - … MariaDB [(none)]> select RAND(3544), RAND(11); ``` - we can check the consumer output in the terminal where it was started: + ``` -------------------------------------------------------------- Received: 1409671452|select @@version_comment limit ? @@ -362,9 +415,11 @@ MariaDB [mqpairs]> select * from pairs; | 01050106010701080109010a010b010c10d | select RAND(?), RAND(?) | Columns: 2 | 2014-09-02 11:24:37 | 2014-09-02 11:24:37 | 1 | +-------------------------------------+----------------------------------+------------+---------------------+---------------------+---------+ 7 rows in set (0.01 sec) -``` +``` + The filter send queries to the RabbitMQ server in the canonical format, i.e select RAND(?), RAND(?). The queries Message Queue Consumer application gets from the server are stored with a counter that quickly shows how many times that normalized query was received: + ``` | 01050106010701080109010a010b010c10d | select RAND(?), RAND(?) | Columns: 2 | 2014-09-02 11:24:37 | 2014-09-02 11:29:15 | 3 | ```