Plugin development manual
Plenty of additions and restructuring compared to the first version. Add it to Documentation.
This commit is contained in:
parent
b2cd90bb08
commit
37f80a7dd8
@ -187,6 +187,7 @@ add_subdirectory(plugins)
|
||||
add_subdirectory(query_classifier)
|
||||
add_subdirectory(server)
|
||||
add_subdirectory(include/maxscale)
|
||||
add_subdirectory(Documentation)
|
||||
if(NOT WITHOUT_MAXADMIN)
|
||||
add_subdirectory(client)
|
||||
endif()
|
||||
|
1
Documentation/CMakeLists.txt
Normal file
1
Documentation/CMakeLists.txt
Normal file
@ -0,0 +1 @@
|
||||
install_custom_file(Plugin-development-guide.md ${CMAKE_INSTALL_INCLUDEDIR}/maxscale/Documentation devel)
|
566
Documentation/Plugin-development-guide.md
Normal file
566
Documentation/Plugin-development-guide.md
Normal file
@ -0,0 +1,566 @@
|
||||
# MariaDB MaxScale plugin development guide
|
||||
|
||||
This document and the attached example code explain prospective plugin
|
||||
developers the MariaDB MaxScale plugin API and also present and explain some
|
||||
best practices and possible pitfalls in module development. We predict that
|
||||
*filters* and *routers* are the module types developers are most likely to work
|
||||
on, so the APIs of these two are discussed in detail.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
1. [Introduction](#introduction)
|
||||
* [Module categories](#module-categories)
|
||||
* [Common definitions and headers](#common-definitions-and-headers)
|
||||
1. [Module information container](#module-information-container)
|
||||
* [Module API](#module-api)
|
||||
1. [Overview](#overview)
|
||||
* [General module management](#general-module-management)
|
||||
* [Protocol](#protocol)
|
||||
* [Authenticator](#authenticator)
|
||||
* [Filter and Router](#filter-and-router)
|
||||
* [Monitor](#monitor)
|
||||
* [Compiling, installing and running](#compiling-installing-and-running)
|
||||
1. [Hands-on example: RoundRobinRouter](#hands-on-example-roundrobinrouter)
|
||||
* [Summary and conclusion](#summary-and-conclusion)
|
||||
|
||||
## Introduction
|
||||
|
||||
MariaDB MaxScale is designed to be an extensible program. Much, if not most, of
|
||||
the actual processing is done by plugin modules. Plugins receive network data,
|
||||
process it and relay it to its destination. The MaxScale core loads plugins,
|
||||
manages client sessions and threads and, most importantly, offers a selection of
|
||||
functions for the plugins to call upon. This collection of functions is called
|
||||
the *MaxScale Public Interface* or just **MPI** for short.
|
||||
|
||||
The plugin modules are shared libraries (.so-files) implementing a set of
|
||||
interface functions, the plugin API. Different plugin types have different APIs,
|
||||
although there are similarities. The MPI is a set of C and C++ header files,
|
||||
from which the module code includes the ones required. MariaDB MaxScale is
|
||||
written in C/C++ and the plugin API is in pure C. Although it is possible
|
||||
to write plugins in any language capable of exposing a C interface and
|
||||
dynamically binding to the core program, in this document we assume plugin
|
||||
modules are written in C++.
|
||||
|
||||
The *RoundRobinRouter* is a practical example of a simple router plugin. The
|
||||
RoundRobinRouter is compiled, installed and ran in [section
|
||||
5.1](#hands-on-example:-roundrobinrouter). The source for the router is included
|
||||
in the plugin development package.
|
||||
|
||||
## Module categories
|
||||
|
||||
This section lists all the module types and summarises their core tasks. The
|
||||
modules are listed in the order a client packet would typically travel through.
|
||||
For more information about a particular module type, see the corresponding
|
||||
folder in `MaxScale/Documentation/`, located in the main MariaDB MaxScale
|
||||
repository.
|
||||
|
||||
**Protocol** modules implement I/O between clients and MaxScale, and between
|
||||
MaxScale and backend servers. Protocol modules read and write to socket
|
||||
descriptors using raw I/O functions provided by the MPI, and implement
|
||||
protocol-specific I/O functions to be used through a common interface. The
|
||||
Protocol module API is defined in `protocol.h`. Currently, the only implemented
|
||||
database protocol is *MySQL*. Other protocols currently in use include *HTTPD*
|
||||
and *maxscaled*, which are used by the MaxInfo and MaxAdmin modules.
|
||||
|
||||
**Authenticator** modules retrieve user account information from the backend
|
||||
databases, store it and use it to authenticate connecting clients. MariaDB
|
||||
MaxScale includes authenticators for MySQL (normal and GSSApi). The
|
||||
authenticator API is defined in `authenticator.h`.
|
||||
|
||||
**Filter** modules process data from clients before routing. A data buffer may
|
||||
travel through multiple filters before arriving in a router. For a data buffer
|
||||
going from a backend to the client, the router receives it first and the
|
||||
filters receive it in reverse order. MaxScale includes a healthly selection of
|
||||
filters ranging from logging, overwriting query data and caching. The filter
|
||||
API is defined in `filter.h`.
|
||||
|
||||
**Router** modules route packets from the last filter in the filter chain to
|
||||
backends and reply data from backends to the last filter. The routing decisions
|
||||
may be based on a variety of conditions; typically packet contents and backend
|
||||
status are the most significant factors. Routers are often used for load
|
||||
balancing, dividing clients and even individual queries between backends.
|
||||
Routers use protocol functions to write to backends, making them somewhat
|
||||
protocol-agnostic. The router API is defined in `router.h`.
|
||||
|
||||
**Monitor** modules do not process data flowing through MariaDB MaxScale, but
|
||||
support the other modules in their operation by updating the status of the
|
||||
backend servers. Monitors are ran in their own threads to minimize
|
||||
interference to the worker threads. They periodically connect to all their
|
||||
assigned backends, query their status and write the results in global structs.
|
||||
The monitor API is defined in `monitor.h`.
|
||||
|
||||
## Common definitions and headers
|
||||
|
||||
Generally, most type definitions, macros and functions exposed by the MPI to be
|
||||
used by modules are prefixed with **MXS**. This should avoid name collisions
|
||||
in the case a module includes many symbols from the MPI.
|
||||
|
||||
Every compilation unit in a module should begin with `#define MXS_MODULE_NAME
|
||||
"<name>"`. This definition will be used by log macros for clarity, prepending
|
||||
`<name>` to every log message. Next, the module should
|
||||
`#include <maxscale/cppdefs.h>` (for C++) or `#include <maxscale/cdefs.h>` (for
|
||||
C). These headers contain compilation environment dependent definitions and
|
||||
global constants, and include some generally useful headers. Including one of
|
||||
them first in every source file enables later global redefinitions across all
|
||||
MaxScale modules. If your module is composed of multiple source files, the above
|
||||
should be placed to a common header file included in the beginning of the source
|
||||
files. The file with the module API definition should also include the header
|
||||
for the module type, e.g. `filter.h`.
|
||||
|
||||
Other common MPI header files required by most modules are listed in the table
|
||||
below.
|
||||
|
||||
Header | Contents
|
||||
-------|----------
|
||||
`alloc.h` | Malloc, calloc etc. replacements
|
||||
`buffer.h` | Packet buffer management
|
||||
`config.h` | Configuration settings
|
||||
`dcb.h` | I/O using descriptor control blocks
|
||||
`debug.h` | Debugging macros
|
||||
`modinfo.h` | Module information structure
|
||||
`server.h` | Backend server information
|
||||
`service.h` | Service definition
|
||||
`session.h` | Client session definition
|
||||
`logmanager.h` | Logging macros and functions
|
||||
|
||||
### Module information container
|
||||
|
||||
A module must implement the `MXS_CREATE_MODULE()`-function, which returns a
|
||||
pointer to a `MXS_MODULE`-structure. This function is called by the module
|
||||
loader during program startup. `MXS_MODULE` (type defined in `modinfo.h`)
|
||||
contains function pointers to further module entrypoints, miscellaneous
|
||||
information about the module and the configuration parameters accepted by the
|
||||
module. This function must be exported without C++ name mangling, so in C++ code
|
||||
it should be defined `extern "C"`.
|
||||
|
||||
The information container describes the module in general and is constructed
|
||||
once during program excecution. A module may have multiple *instances* with
|
||||
different values for configuration parameters. For example, a filter module can
|
||||
be used with two different configurations in different services (or even in the
|
||||
same service). In this case the loader uses the same module information
|
||||
container for both but creates two module instances.
|
||||
|
||||
The MariaDB MaxScale configuration file `maxscale.cnf` is parsed by the core.
|
||||
The core also checks that all the defined parameters are of the correct type for
|
||||
the module. For this, the `MXS_MODULE`-structure includes a list of parameters
|
||||
accepted by the module, defining parameter names, types and default values. In
|
||||
the actual module code, parameter values should be extracted using functions
|
||||
defined in `config.h`.
|
||||
|
||||
## Module API
|
||||
|
||||
### Overview
|
||||
|
||||
This section explains some general concepts encountered when implementing a
|
||||
module API. For more detailed information, see the module specific subsection,
|
||||
header files or the doxygen documentation.
|
||||
|
||||
Modules with configuration data define an *INSTANCE* object, which is created by
|
||||
the module code in a `createInstance`-function or equivalent. The instance
|
||||
creation function is called during MaxScale startup, usually when creating
|
||||
services. MaxScale core holds the module instance data in the
|
||||
`SERVICE`-structure (or other higher level construct) and gives it as a
|
||||
parameter when calling functions from the module in question. The instance
|
||||
structure should contain all non-client-specific information required by the
|
||||
functions of the module. The core does not know what the object contains (since
|
||||
it is defined by the module itself), nor will it modify the pointer or the
|
||||
referenced object in any way.
|
||||
|
||||
Modules dealing with client-specific data require a *SESSION* object for every
|
||||
client. As with the instance data, the definition of the module session
|
||||
structure is up to the module writer and MaxScale treats it as an opaque type.
|
||||
Usually the session contains status indicators and any resources required by the
|
||||
client. MaxScale core has its own `MXS_SESSION` object, which tracks a variety
|
||||
of client related information. The `MXS_SESSION` is given as a parameter to
|
||||
module-specific session creation functions and is required for several typical
|
||||
operations such as connecting to backends.
|
||||
|
||||
Descriptor control blocks (`DCB`), are generalized I/O descriptor types. DCBs
|
||||
store the file descriptor, state, remote address, username, session, and other
|
||||
data. DCBs are created whenever a new socket is created. Typically this happens
|
||||
when a new client connects or MaxScale connects the client session to backend
|
||||
servers. The module writer should use DCB handling functions provided by the MPI
|
||||
to manage connections instead of calling general networking libraries. This
|
||||
ensures that I/O is handled asynchronously by epoll. In general, module code
|
||||
should avoid blocking I/O, *sleep*, *yield* or other potentially costly
|
||||
operations, as the same thread is typically used for many client sessions.
|
||||
|
||||
Network data such as client queries and backend replies are held in a buffer
|
||||
container called `GWBUF`. Multiple GWBUFs can form a linked list with type
|
||||
information and properties in each GWBUF-node. Each node includes a pointer to a
|
||||
reference counted shared buffer (`SHARED_BUF`), which finally points to a slice
|
||||
of the actual data. In effect, multiple GWBUF-chains can share some data while
|
||||
keeping some parts private. This construction is meant to minimize the need for
|
||||
data copying and makes it easy to append more data to partially received data
|
||||
packets. Plugin writers should use the MPI to manipulate GWBUFs. For more
|
||||
information on the GWBUF, see [Filter and Router](#filter-and-router).
|
||||
|
||||
### General module management
|
||||
|
||||
```java
|
||||
int process_init()
|
||||
void process_finish()
|
||||
int thread_init()
|
||||
void thread_finish()
|
||||
```
|
||||
|
||||
These four functions are present in all `MXS_MODULE` structs and are not part of
|
||||
the API of any individual module type. `process_init` and `process_finish` are
|
||||
called by the module loader right after loading a module and just before
|
||||
MaxScale terminates, respectively. Usually, these can be set to null in
|
||||
`MXS_MODULE` unless the module needs some general initializations before
|
||||
creating any instances. `thread_init` and `thread_finish` are thread-specific
|
||||
equivalents.
|
||||
|
||||
```java
|
||||
void diagnostics(INSTANCE *instance, DCB *dcb)
|
||||
```
|
||||
|
||||
A diagnostics printing routine is present in nearly all module types, although
|
||||
with varying signatures. This entrypoint should print various statistics and
|
||||
status information about the module instance `instance` in string form. The
|
||||
target of the printing is the given DCB, and printing should be implemented by
|
||||
calling `dcb_printf`. The diagnostics function is used by the *MaxInfo* and
|
||||
*MaxAdmin* features.
|
||||
|
||||
### Protocol
|
||||
|
||||
```java
|
||||
int32_t read(struct dcb *)
|
||||
int32_t write(struct dcb *, GWBUF *)
|
||||
int32_t write_ready(struct dcb *)
|
||||
int32_t error(struct dcb *)
|
||||
int32_t hangup(struct dcb *)
|
||||
int32_t accept(struct dcb *)
|
||||
int32_t connect(struct dcb *, struct server *, struct session *)
|
||||
int32_t close(struct dcb *)
|
||||
int32_t listen(struct dcb *, char *)
|
||||
int32_t auth(struct dcb *, struct server *, struct session *, GWBUF *)
|
||||
int32_t session(struct dcb *, void *)
|
||||
char auth_default()
|
||||
int32_t connlimit(struct dcb *, int limit)
|
||||
```
|
||||
|
||||
Protocol modules are laborous to implement due to their low level nature. Each
|
||||
DCB maintains pointers to the correct protocol functions to be used with it,
|
||||
allowing the DCB to be used in a protocol-independent manner.
|
||||
|
||||
`read`, `write_ready`, `error` and `hangup` are *epoll* handlers for their
|
||||
respective events. `write` implements writing and is usually called in a router
|
||||
module. `accept` is a listener socker handler. `connect` is used during session
|
||||
creation when connecting to backend servers. `listen` creates a listener socket.
|
||||
`close` closes a DCB created by `accept`, `connect` or `listen`.
|
||||
|
||||
In the ideal case modules other than the protocol modules themselves should not
|
||||
be protocol-specific. This is currently difficult to achieve, since many actions
|
||||
in the modules are dependent on protocol-speficic details. In the future,
|
||||
protocol modules may be expanded to implement a generic query parsing and
|
||||
information API, allowing filters and routers to be used with different SQL
|
||||
variants.
|
||||
|
||||
### Authenticator
|
||||
|
||||
```java
|
||||
void* initialize(char **options)
|
||||
void* create(void* instance)
|
||||
int extract(struct dcb *, GWBUF *)
|
||||
bool connectssl(struct dcb *)
|
||||
int authenticate(struct dcb *)
|
||||
void free(struct dcb *)
|
||||
void destroy(void *)
|
||||
int loadusers(struct servlistener *)
|
||||
void diagnostic(struct dcb*, struct servlistener *)
|
||||
int reauthenticate(struct dcb *, const char *user, uint8_t *token,
|
||||
size_t token_len, uint8_t *scramble, size_t scramble_len,
|
||||
uint8_t *output, size_t output_len);
|
||||
```
|
||||
|
||||
Authenticators must communicate with the client or the backends and implement
|
||||
authentication. The authenticators can be divided to client and backend modules,
|
||||
although the two types are linked and must be used together. Authenticators are
|
||||
also dependent on the protocol modules.
|
||||
|
||||
### Filter and Router
|
||||
|
||||
Filter and router APIs are nearly identical and are presented together. Since
|
||||
these are the modules most likely to be implemented by plugin developers, their
|
||||
APIs are discussed in more detail.
|
||||
|
||||
```java
|
||||
INSTANCE* createInstance(SERVICE* service, char** options)
|
||||
void destroyInstance(INSTANCE* instance)
|
||||
```
|
||||
|
||||
`createInstance` should read the `options` and initialize an instance object for
|
||||
use with `service`. Often, simply saving the configuration values to fields is
|
||||
enough. `destroyInstance` is called when the service using the module is
|
||||
deallocated. It should free any resources claimed by the instance. All sessions
|
||||
created by this instance should be closed before calling the destructor.
|
||||
|
||||
```java
|
||||
SESSION* newSession(INSTANCE* instance, MXS_SESSION* mxs_session)
|
||||
void closeSession(INSTANCE* instance, SESSION* session)
|
||||
void freeSession(INSTANCE* instance, SESSION* session)
|
||||
```
|
||||
|
||||
These functions manage sessions. `newSession` should allocate a router or filter
|
||||
session attached to the client session represented by `mxs_session`. MaxScale
|
||||
will pass the returned pointer to all the API entrypoints that process user data
|
||||
for the particular client. `closeSession` should close connections the session
|
||||
has opened and release any resources specific to the served client. The
|
||||
*SESSION* structure allocated in `newSession` should not be deallocated by
|
||||
`closeSession` but in `freeSession`. These two are called in succession
|
||||
by the core.
|
||||
|
||||
```java
|
||||
int routeQuery(INSTANCE *instance, SESSION session, GWBUF* queue) void
|
||||
clientReply(INSTANCE* instance, SESSION session, GWBUF* queue, DCB *backend_dcb)
|
||||
uint64_t getCapabilities(INSTANCE* instance)
|
||||
```
|
||||
|
||||
`routeQuery` is called for client requests which should be routed to backends,
|
||||
and `clientReply` for backend reply packets which should be routed to the
|
||||
client. For some modules, MaxScale itself is the backend. For filters, these can
|
||||
be NULL, in which case the filter will be skipped for that packet type.
|
||||
|
||||
`routeQuery` is often the most complicated function in a router, as it
|
||||
implements the routing logic. It typically considers the client request `queue`,
|
||||
the router settings in `instance` and the session state in `session` when making
|
||||
a routing decision. For filters aswell, `routeQuery` typically implements the
|
||||
main logic, although the routing target is constant. For router modules,
|
||||
`routeQuery` should send data forward with `dcb->func.write()`. Filters should
|
||||
directly call `routeQuery` for the next filter or router in the chain.
|
||||
|
||||
`clientReply` processes data flowing from backend back to client. For routers,
|
||||
this function is often much simpler than `routeQuery`, since there is only one
|
||||
client to route to. Depending on the router, some packets may not be routed to
|
||||
the client. For example, if a client query was routed to multiple backends,
|
||||
MaxScale will receive multiple replies while the client only expects one.
|
||||
Routers should pass the reply packet to the last filter in the chain (reversed
|
||||
order) using the macro `MXS_SESSION_ROUTE_REPLY`. Filters should call the
|
||||
`clientReply` of the previous filter in the chain. There is no need for filters
|
||||
to worry about being the first filter in the chain, as this is handled
|
||||
transparently by the session creation routine.
|
||||
|
||||
Application data is not always received in complete packets from the network
|
||||
stack. How partial packets are handled by the receiving protocol module depends
|
||||
on the attached filters and the router, communicated by their
|
||||
`getCapabilities`-functions. `getCapabilities` should return a bitfield
|
||||
resulting from ORring the individual capabilities. `routing.h` lists the allowed
|
||||
capability flags.
|
||||
|
||||
If a router or filter sets no capabilities, `routeQuery` or `clientReply` may be
|
||||
called to route partial packets. If the routing logic does not require any
|
||||
information on the contents of the packets or even tracking the number of
|
||||
packets, this may be fine. For many cases though, receiving a data packet in a
|
||||
complete GWBUF chain or in one contiguos GWBUF is required. The former can be
|
||||
requested by `getCapabilities` returning *RCAP_TYPE_STMT*, the latter by
|
||||
*RCAP_TYPE_CONTIGUOUS*. Separate settings exist for queries and replies. For
|
||||
replies, an additional value, *RCAP_TYPE_RESULTSET_OUTPUT* is defined. This
|
||||
requests the protocol module to gather partial results into one result set.
|
||||
Enforcing complete packets will delay processing, since the protocol module will
|
||||
have to wait for the entire data packet to arrive before sending it down the
|
||||
processing chain.
|
||||
|
||||
```java
|
||||
void handleError(INSTANCE* instance,SESSION* session, GWBUF* errmsgbuf,
|
||||
DCB* problem_dcb, mxs_error_action_t action, bool* succp);
|
||||
```
|
||||
|
||||
This router-only entrypoint is called if `routeQuery` returns an error value or
|
||||
if an error occurs in one of the connections listened to by the session. The
|
||||
steps an error handler typically takes depend on the nature of the `problem_dcb`
|
||||
and the error encountered. If `problem_dcb` is a client socket, then the session
|
||||
is lost and should be closed. The error handler should not do this by itself and
|
||||
just report the failure by setting `succp` to false. If `problem_dcb` is a
|
||||
backend socket, then the error handler should try to connect to another backend
|
||||
if the routing logic allows this. If the error is simply a failed authentication
|
||||
on the backend, then it is usually best to send the message directly to the
|
||||
client and close the session.
|
||||
|
||||
### Monitor
|
||||
|
||||
```java
|
||||
MONITOR* startMonitor(MXS_MONITOR *monitor, const MXS_CONFIG_PARAMETER *params)
|
||||
void stopMonitor(MXS_MONITOR *monitor)
|
||||
void diagnostics(DCB *, const MXS_MONITOR *)
|
||||
```
|
||||
|
||||
Monitor modules typically run a repeated monitor routine with a used defined
|
||||
interval. The `MXS_MONITOR` is a standard monitor definition used for all
|
||||
monitors and contains a void pointer for storing module specific data.
|
||||
`startMonitor` should create a new thread for itself using functions in the MPI
|
||||
and have it regularly run a monitor loop. In the beginning of every monitor
|
||||
loop, the monitor should lock the `SERVER`-structures of its servers. This
|
||||
prevents any administrative action from interfering with the monitor during its
|
||||
pass.
|
||||
|
||||
## Compiling, installing and running
|
||||
|
||||
The requirements for compiling a module are:
|
||||
* The public headers (MPI)
|
||||
* A compatible compiler, typically GCC
|
||||
* Libraries required by the public headers
|
||||
|
||||
Some of the public header files themselves include headers from other libraries.
|
||||
These libraries need to be installed and it may be required to point out their
|
||||
location to gcc. Some of the more commonly required libraries are:
|
||||
* *MySQL Connector-C*, used by the MySQL protocol module
|
||||
* *pcre2 regular expressions* (libpcre2-dev), used for example by the header
|
||||
`modutil.h`
|
||||
|
||||
After all dependencies are accounted for, the module should compile with a
|
||||
command similar to
|
||||
```
|
||||
gcc -I /usr/local/include/mariadb -shared -fPIC -g -o libmymodule.so mymodule.cpp
|
||||
```
|
||||
Large modules composed of several source files and using additional libraries
|
||||
may require a more complicated compilation scheme, but that is outside the scope
|
||||
of this document. The result of compiling a plugin should be a single shared
|
||||
library file.
|
||||
|
||||
The compiled .so-file needs to be copied to the MaxScale library folder, which
|
||||
is `/usr/local/lib/maxscale` by default. MaxScale expects the filename to be
|
||||
`lib<name>.so`, where `<name>` must match the module name given in the
|
||||
configuration file.
|
||||
|
||||
### Hands-on example: RoundRobinRouter
|
||||
|
||||
In this example, the RoundRobinRouter is compiled, installed and tested. The
|
||||
software environment this section was written and tested is listed below. Any
|
||||
recent Linux setup should be applicaple.
|
||||
|
||||
* Linux Mint 18
|
||||
* gcc 5.4.0, glibc 2.23
|
||||
* MariaDB MaxScale 2.1.0 debug build (binaries in `usr/local/maxscale`, modules in
|
||||
`/usr/local/lib/maxscale`)
|
||||
* MariaDB Connector-C 2.3.2 (installed to `/usr/local/lib/mariadb`, headers in
|
||||
`/usr/local/include/mariadb`)
|
||||
* `roundrobinrouter.cpp` in the current directory
|
||||
* MaxScale plugin development headers (in `usr/include/maxscale`)
|
||||
|
||||
**Step 1** Compile RoundRobinRouter with `$gcc -I /usr/local/include/mariadb
|
||||
-shared -fPIC -g -o libroundrobinrouter.so roundrobinrouter.cpp`.
|
||||
Assuming all headers were found, the shared library `libroundrobinrouter.so`
|
||||
is produced.
|
||||
|
||||
**Step 2** Copy the compiled module to the MaxScale module directory: `$sudo cp
|
||||
libroundrobinrouter.so /usr/local/lib/maxscale`.
|
||||
|
||||
**Step 3** Modify the MaxScale configuration file to use the RoundRobinRouter as
|
||||
a router. Example service and listener definitions are below. The *servers*
|
||||
and *write_backend*-lines should be configured according to the actual backend
|
||||
configuration.
|
||||
|
||||
```
|
||||
[RR Service]
|
||||
type=service
|
||||
router=roundrobinrouter
|
||||
servers=LocalMaster1,LocalSlave1,LocalSlave2
|
||||
user=maxscale
|
||||
passwd=maxscale
|
||||
filters=MyLogFilter1
|
||||
max_backends=10
|
||||
write_backend=LocalMaster1
|
||||
print_on_routing=true
|
||||
dummy_setting=two
|
||||
|
||||
[RR Listener]
|
||||
type=listener
|
||||
service=RR Service
|
||||
protocol=MySQLClient
|
||||
port=4009
|
||||
```
|
||||
|
||||
**Step 4** Start MaxScale: `$ maxscale -d`. Output:
|
||||
```
|
||||
MariaDB Corporation MaxScale 2.1.0 Mon Feb 20 17:22:18 2017
|
||||
------------------------------------------------------
|
||||
Info : MaxScale will be run in the terminal process.
|
||||
See the log from the following log files :
|
||||
|
||||
Configuration file : /etc/maxscale.cnf
|
||||
Log directory : /var/log/maxscale
|
||||
Data directory : /var/lib/maxscale
|
||||
Module directory : /usr/local/lib/maxscale
|
||||
Service cache : /var/cache/maxscale
|
||||
```
|
||||
|
||||
**Step 5** Test with a MySQL client. The RoundRobinRouter has been tested with both a command line and a GUI client. With `DEBUG_RRROUTER` defined and `print_on_routing` enabled, the `/var/log/maxscale/maxscale.log` file will report nearly every action taken by the router.
|
||||
|
||||
```
|
||||
2017-02-21 10:37:23 notice : [RoundRobinRouter] Creating instance.
|
||||
2017-02-21 10:37:23 notice : [RoundRobinRouter] Settings read:
|
||||
2017-02-21 10:37:23 notice : [RoundRobinRouter] 'max_backends': 10
|
||||
2017-02-21 10:37:23 notice : [RoundRobinRouter] 'write_backend': 0xf0ce70
|
||||
2017-02-21 10:37:23 notice : [RoundRobinRouter] 'print_on_routing': 1
|
||||
2017-02-21 10:37:23 notice : [RoundRobinRouter] 'dummy_setting': 2
|
||||
.
|
||||
.
|
||||
.
|
||||
2017-02-21 10:37:37 notice : [RoundRobinRouter] Session with 4 connections created.
|
||||
2017-02-21 10:37:37 notice : [RoundRobinRouter] QUERY: SHOW VARIABLES WHERE Variable_name in ('max_allowed_packet', 'system_time_zone', 'time_zone', 'sql_mode')
|
||||
2017-02-21 10:37:37 notice : [RoundRobinRouter] Routing statement of length 110u to backend 'LocalMaster1'.
|
||||
2017-02-21 10:37:37 notice : [RoundRobinRouter] Replied to client.
|
||||
2017-02-21 10:37:37 notice : [RoundRobinRouter] QUERY: set session autocommit=1,sql_mode='NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES'
|
||||
2017-02-21 10:37:37 notice : [RoundRobinRouter] Routing statement of length 103u to 4 backends.
|
||||
2017-02-21 10:37:37 notice : [RoundRobinRouter] Replied to client.
|
||||
2017-02-21 10:37:37 notice : [RoundRobinRouter] QUERY: SET @ApplicationName='DBeaver 3.8.5 - Main'
|
||||
2017-02-21 10:37:37 notice : [RoundRobinRouter] Routing statement of length 48u to 4 backends.
|
||||
2017-02-21 10:37:37 notice : [RoundRobinRouter] Replied to client.
|
||||
2017-02-21 10:37:37 notice : [RoundRobinRouter] QUERY: select @@lower_case_table_names
|
||||
2017-02-21 10:37:37 notice : [RoundRobinRouter] Routing statement of length 36u to backend 'LocalSlave1'.
|
||||
2017-02-21 10:37:37 notice : [RoundRobinRouter] Replied to client.
|
||||
```
|
||||
|
||||
**Step 5** Connect with MaxAdmin, print diagnostics and call a custom command.
|
||||
```
|
||||
$sudo maxadmin
|
||||
MaxScale> show service "RR Service"
|
||||
Service: RR Service
|
||||
Router: roundrobinrouter
|
||||
State: Started
|
||||
Queries routed successfully: 37
|
||||
Failed routing attempts: 0
|
||||
Client replies routed: 38
|
||||
Started: Tue Feb 21 11:52:08 2017
|
||||
Root user access: Disabled
|
||||
Filter chain: MyLogFilter1
|
||||
Backend databases:
|
||||
127.0.0.1:3001 Protocol: MySQLBackend Name: LocalMaster1
|
||||
127.0.0.1:3002 Protocol: MySQLBackend Name: LocalSlave1
|
||||
127.0.0.1:3003 Protocol: MySQLBackend Name: LocalSlave2
|
||||
Total connections: 2
|
||||
Currently connected: 2
|
||||
MaxScale> call command rrrouter test_command "one" 0
|
||||
```
|
||||
|
||||
The result of the `test_command "one" 0` is printed to the terminal MaxScale is
|
||||
running in:
|
||||
```
|
||||
RoundRobinRouter wishes the Admin a good day.
|
||||
The module got 2 arguments.
|
||||
Argument 0: type 'string' value 'one'
|
||||
Argument 1: type 'boolean' value 'false'
|
||||
```
|
||||
|
||||
## Summary and conclusion
|
||||
|
||||
Plugins offer a way to extend MariaDB MaxScale whenever the standard modules are
|
||||
found insufficient. The plugins need only implement a set API, can be
|
||||
independently compiled and installation is simply a file copy with some
|
||||
configuration file modifications.
|
||||
|
||||
Out of the different plugin types, filters are the easiest to implement. They
|
||||
work independently and have few requirements. Protocol and authenticator modules
|
||||
require indepth knowledge of the database protocol they implement. Router module
|
||||
complexity depends on the routing logic requirements.
|
||||
|
||||
The provided RoundRobinRouter example code should serve as a valid starting
|
||||
point for both filters and routers. Studying the MaxScale Public Interface
|
||||
headers to get a general idea of what services the core provides for plugins,
|
||||
is also highly recommeded.
|
||||
|
||||
Lastly, MariaDB MaxScale is an open-source project, so code contributions can be
|
||||
accepted if they fulfill the
|
||||
[requirements](https://github.com/mariadb-corporation/MaxScale/wiki/Contributing).
|
@ -11,6 +11,7 @@ add_subdirectory(maxinfo)
|
||||
add_subdirectory(readconnroute)
|
||||
add_subdirectory(readwritesplit)
|
||||
add_subdirectory(schemarouter)
|
||||
add_subdirectory(roundrobinrouter)
|
||||
|
||||
if(BUILD_TESTS)
|
||||
add_subdirectory(testroute)
|
||||
|
6
server/modules/routing/roundrobinrouter/CMakeLists.txt
Normal file
6
server/modules/routing/roundrobinrouter/CMakeLists.txt
Normal file
@ -0,0 +1,6 @@
|
||||
add_library(roundrobinrouter SHARED roundrobinrouter.cpp)
|
||||
target_link_libraries(roundrobinrouter maxscale-common)
|
||||
set_target_properties(roundrobinrouter PROPERTIES VERSION "1.0.0")
|
||||
install_module(roundrobinrouter core)
|
||||
|
||||
install_custom_file(roundrobinrouter.cpp ${CMAKE_INSTALL_INCLUDEDIR}/maxscale/Documentation devel)
|
897
server/modules/routing/roundrobinrouter/roundrobinrouter.cpp
Normal file
897
server/modules/routing/roundrobinrouter/roundrobinrouter.cpp
Normal file
@ -0,0 +1,897 @@
|
||||
/*
|
||||
* Copyright (c) 2017 MariaDB Corporation Ab
|
||||
*
|
||||
* Use of this software is governed by the Business Source License included
|
||||
* in the LICENSE.TXT file and at www.mariadb.com/bsl.
|
||||
*
|
||||
* Change Date: 2019-07-01
|
||||
*
|
||||
* On the date above, in accordance with the Business Source License, use
|
||||
* of this software will be governed by version 2 or later of the General
|
||||
* Public License.
|
||||
*/
|
||||
|
||||
/**
|
||||
* @file roundrobinrouter.c - Round-robin router load balancer
|
||||
*
|
||||
* This is an implementation of a simple query router that balances reads on a
|
||||
* query level. The router is configured with a set of slaves and optionally
|
||||
* a master. The router balances the client read queries over the set of slave
|
||||
* servers, sending write operations to the master. Session-operations are sent
|
||||
* to all slaves and the master. The read query balancing is done in round robin
|
||||
* style: in each session, the slave servers (and the master if inserted into the
|
||||
* slave list) take turns processing read queries.
|
||||
*
|
||||
* This router is intended to be a rather straightforward example on how to
|
||||
* program a module for MariaDB MaxScale. The router does not yet support all
|
||||
* SQL-commands and there are bound to be various limitations yet unknown. It
|
||||
* does work on basic reads and writes.
|
||||
*
|
||||
*/
|
||||
|
||||
/* The log macros use this definition. */
|
||||
#define MXS_MODULE_NAME "RoundRobinRouter"
|
||||
|
||||
#include <maxscale/cppdefs.hh>
|
||||
|
||||
#include <vector>
|
||||
#include <iostream>
|
||||
#include <string>
|
||||
#include <iterator>
|
||||
|
||||
#include <maxscale/alloc.h>
|
||||
#include <maxscale/buffer.h>
|
||||
#include <maxscale/dcb.h>
|
||||
#include <maxscale/modinfo.h>
|
||||
#include <maxscale/modulecmd.h>
|
||||
#include <maxscale/modutil.h>
|
||||
#include <maxscale/protocol/mysql.h>
|
||||
#include <maxscale/query_classifier.h>
|
||||
#include <maxscale/router.h>
|
||||
|
||||
//#define DEBUG_RRROUTER
|
||||
#undef DEBUG_RROUTER
|
||||
|
||||
#ifdef DEBUG_RRROUTER
|
||||
#define RR_DEBUG(msg, ...) MXS_NOTICE(msg, ##__VA_ARGS__)
|
||||
#else
|
||||
#define RR_DEBUG(msg, ...)
|
||||
#endif
|
||||
|
||||
/* This router handles different query types in a different manner. Some queries
|
||||
* require that a "write_backend" is set. */
|
||||
const uint32_t q_route_to_rr = (QUERY_TYPE_LOCAL_READ | QUERY_TYPE_READ |
|
||||
QUERY_TYPE_MASTER_READ | QUERY_TYPE_USERVAR_READ |
|
||||
QUERY_TYPE_SYSVAR_READ | QUERY_TYPE_GSYSVAR_READ |
|
||||
QUERY_TYPE_SHOW_DATABASES | QUERY_TYPE_SHOW_TABLES);
|
||||
|
||||
const uint32_t q_route_to_all = (QUERY_TYPE_SESSION_WRITE | QUERY_TYPE_USERVAR_WRITE |
|
||||
QUERY_TYPE_GSYSVAR_WRITE | QUERY_TYPE_ENABLE_AUTOCOMMIT |
|
||||
QUERY_TYPE_DISABLE_AUTOCOMMIT);
|
||||
|
||||
const uint32_t q_trx_begin = QUERY_TYPE_BEGIN_TRX;
|
||||
|
||||
const uint32_t q_trx_end = (QUERY_TYPE_ROLLBACK | QUERY_TYPE_COMMIT);
|
||||
|
||||
const uint32_t q_route_to_write = (QUERY_TYPE_WRITE | QUERY_TYPE_PREPARE_NAMED_STMT |
|
||||
QUERY_TYPE_PREPARE_STMT | QUERY_TYPE_EXEC_STMT |
|
||||
QUERY_TYPE_CREATE_TMP_TABLE | QUERY_TYPE_READ_TMP_TABLE);
|
||||
|
||||
const char MAX_BACKENDS[] = "max_backends";
|
||||
const char WRITE_BACKEND[] = "write_backend";
|
||||
const char PRINT_ON_ROUTING[] = "print_on_routing";
|
||||
const char DUMMY[] = "dummy_setting";
|
||||
|
||||
/* Enum setting definition example */
|
||||
static const MXS_ENUM_VALUE enum_example[] =
|
||||
{
|
||||
{"two", 2},
|
||||
{"zero", 0},
|
||||
{NULL} /* Last must be NULL */
|
||||
};
|
||||
|
||||
static modulecmd_arg_type_t custom_cmd_args[] =
|
||||
{
|
||||
{MODULECMD_ARG_STRING, "Example string"},
|
||||
{(MODULECMD_ARG_BOOLEAN | MODULECMD_ARG_OPTIONAL), "This is an optional bool parameter"}
|
||||
};
|
||||
|
||||
bool custom_cmd_example(const MODULECMD_ARG *argv);
|
||||
|
||||
using std::string;
|
||||
using std::cout;
|
||||
|
||||
typedef std::vector<DCB*> DCB_VEC;
|
||||
|
||||
class RRRouter;
|
||||
class RRRouterSession;
|
||||
|
||||
/* Each service using this router will have a router object instance. */
|
||||
class RRRouter
|
||||
{
|
||||
private:
|
||||
SERVICE* m_service; /* Service this router is part of */
|
||||
/* Router settings */
|
||||
unsigned int m_max_backends; /* How many backend servers to use */
|
||||
SERVER* m_write_server; /* Where to send write etc. "unsafe" queries */
|
||||
bool m_print_on_routing; /* Print a message on every packet routed? */
|
||||
uint64_t m_example_enum; /* Not used */
|
||||
|
||||
void decide_target(RRRouterSession* rses, GWBUF* querybuf, DCB*& target, bool& route_to_all);
|
||||
public:
|
||||
/* Statistics, written to by multiple threads */
|
||||
volatile unsigned long int m_routing_s; /* Routing success */
|
||||
volatile unsigned long int m_routing_f; /* Routing fail */
|
||||
volatile unsigned long int m_routing_c; /* Client packets routed */
|
||||
|
||||
/* Methods */
|
||||
RRRouter(SERVICE* service);
|
||||
~RRRouter();
|
||||
RRRouterSession* create_session(MXS_SESSION* session);
|
||||
int route_query(RRRouterSession* rses, GWBUF* querybuf);
|
||||
void client_reply(RRRouterSession* rses, GWBUF* buf, DCB* backend_dcb);
|
||||
void handle_error(RRRouterSession* rses, GWBUF* message, DCB* problem_dcb,
|
||||
mxs_error_action_t action, bool* succp);
|
||||
};
|
||||
|
||||
/* Every client connection has a corresponding session. */
|
||||
class RRRouterSession
|
||||
{
|
||||
public:
|
||||
bool m_closed; /* true when closeSession is called */
|
||||
DCB_VEC m_backend_dcbs; /* backends */
|
||||
DCB* m_write_dcb; /* write backend */
|
||||
DCB* m_client_dcb; /* client */
|
||||
unsigned int m_route_count; /* how many packets have been routed */
|
||||
bool m_on_transaction; /* Is the session in transaction mode? */
|
||||
unsigned int m_replies_to_ignore; /* Counts how many replies should be ignored. */
|
||||
|
||||
RRRouterSession(DCB_VEC&, DCB*, DCB*);
|
||||
~RRRouterSession();
|
||||
void close();
|
||||
};
|
||||
|
||||
RRRouter::RRRouter(SERVICE* service)
|
||||
: m_service(service)
|
||||
, m_routing_s(0)
|
||||
, m_routing_f(0)
|
||||
, m_routing_c(0)
|
||||
{
|
||||
RR_DEBUG("Creating instance.");
|
||||
/* Read options specific to round robin router. */
|
||||
MXS_CONFIG_PARAMETER* params = service->svc_config_param;
|
||||
m_max_backends = config_get_integer(params, MAX_BACKENDS);
|
||||
m_write_server = config_get_server(params, WRITE_BACKEND);
|
||||
m_print_on_routing = config_get_bool(params, PRINT_ON_ROUTING);
|
||||
m_example_enum = config_get_enum(params, DUMMY, enum_example);
|
||||
|
||||
RR_DEBUG("Settings read:");
|
||||
RR_DEBUG("'%s': %d", MAX_BACKENDS, m_max_backends);
|
||||
RR_DEBUG("'%s': %p", WRITE_BACKEND, m_write_server);
|
||||
RR_DEBUG("'%s': %d", PRINT_ON_ROUTING, m_print_on_routing);
|
||||
RR_DEBUG("'%s': %lu", DUMMY, m_example_enum);
|
||||
}
|
||||
RRRouter::~RRRouter()
|
||||
{
|
||||
RR_DEBUG("Deleting router instance.");
|
||||
RR_DEBUG("Queries routed successfully: %lu", m_routing_s);
|
||||
RR_DEBUG("Failed routing attempts: %lu", m_routing_f);
|
||||
RR_DEBUG("Client replies: %lu", m_routing_c);
|
||||
}
|
||||
|
||||
RRRouterSession* RRRouter::create_session(MXS_SESSION* session)
|
||||
{
|
||||
DCB_VEC backends;
|
||||
DCB* write_dcb = NULL;
|
||||
RRRouterSession* rses = NULL;
|
||||
try
|
||||
{
|
||||
/* Try to connect to as many backends as required. */
|
||||
SERVER_REF* sref;
|
||||
for (sref = m_service->dbref; sref != NULL; sref = sref->next)
|
||||
{
|
||||
if (SERVER_REF_IS_ACTIVE(sref) && (backends.size() < m_max_backends))
|
||||
{
|
||||
/* Connect to server */
|
||||
DCB* conn = dcb_connect(sref->server, session,
|
||||
sref->server->protocol);
|
||||
if (conn)
|
||||
{
|
||||
/* Success */
|
||||
atomic_add(&sref->connections, 1);
|
||||
conn->service = session->service;
|
||||
backends.push_back(conn);
|
||||
} /* Any error by dcb_connect is reported by the function itself */
|
||||
}
|
||||
}
|
||||
if (m_write_server)
|
||||
{
|
||||
/* Connect to write backend server. This is not essential. */
|
||||
write_dcb = dcb_connect(m_write_server, session, m_write_server->protocol);
|
||||
if (write_dcb)
|
||||
{
|
||||
/* Success */
|
||||
write_dcb->service = session->service;
|
||||
}
|
||||
}
|
||||
if (backends.size() < 1)
|
||||
{
|
||||
MXS_ERROR("Session creation failed, could not connect to any "
|
||||
"read backends.");
|
||||
}
|
||||
else
|
||||
{
|
||||
rses = new RRRouterSession(backends, write_dcb, session->client_dcb);
|
||||
RR_DEBUG("Session with %lu connections created.",
|
||||
backends.size() + (write_dcb ? 1 : 0));
|
||||
}
|
||||
}
|
||||
catch (const std::exception& x)
|
||||
{
|
||||
MXS_ERROR("Caught exception: %s", x.what());
|
||||
/* Close any connections already made */
|
||||
for (unsigned int i = 0; i < backends.size(); i++)
|
||||
{
|
||||
DCB* dcb = backends[i];
|
||||
dcb_close(dcb);
|
||||
atomic_add(&(m_service->dbref->connections), -1);
|
||||
}
|
||||
backends.clear();
|
||||
if (write_dcb)
|
||||
{
|
||||
dcb_close(write_dcb);
|
||||
}
|
||||
}
|
||||
return rses;
|
||||
}
|
||||
|
||||
int RRRouter::route_query(RRRouterSession* rses, GWBUF* querybuf)
|
||||
{
|
||||
int rval = 0;
|
||||
const bool print = m_print_on_routing;
|
||||
DCB* target = NULL;
|
||||
bool route_to_all = false;
|
||||
|
||||
if (!rses->m_closed)
|
||||
{
|
||||
decide_target(rses, querybuf, target, route_to_all);
|
||||
}
|
||||
|
||||
/* Target selection done, write to dcb. */
|
||||
if (target)
|
||||
{
|
||||
/* We have one target backend */
|
||||
if (print)
|
||||
{
|
||||
MXS_NOTICE("Routing statement of length %du to backend '%s'.",
|
||||
gwbuf_length(querybuf), target->server->unique_name);
|
||||
}
|
||||
/* Do not use dcb_write() to output to a dcb. dcb_write() is used only
|
||||
* for raw write in the procol modules. */
|
||||
rval = target->func.write(target, querybuf);
|
||||
/* After write, the buffer points to non-existing data. */
|
||||
querybuf = NULL;
|
||||
}
|
||||
else if (route_to_all)
|
||||
{
|
||||
int n_targets = rses->m_backend_dcbs.size() + (rses->m_write_dcb ? 1 : 0);
|
||||
if (print)
|
||||
{
|
||||
MXS_NOTICE("Routing statement of length %du to %d backends.",
|
||||
gwbuf_length(querybuf), n_targets);
|
||||
}
|
||||
int route_success = 0;
|
||||
for (unsigned int i = 0; i < rses->m_backend_dcbs.size(); i++)
|
||||
{
|
||||
DCB* dcb = rses->m_backend_dcbs[i];
|
||||
/* Need to clone the buffer since write consumes it */
|
||||
GWBUF* copy = gwbuf_clone(querybuf);
|
||||
if (copy)
|
||||
{
|
||||
route_success += dcb->func.write(dcb, copy);
|
||||
}
|
||||
}
|
||||
if (rses->m_write_dcb)
|
||||
{
|
||||
GWBUF* copy = gwbuf_clone(querybuf);
|
||||
if (copy)
|
||||
{
|
||||
route_success += rses->m_write_dcb->func.write(rses->m_write_dcb,
|
||||
copy);
|
||||
}
|
||||
}
|
||||
rses->m_replies_to_ignore += route_success - 1;
|
||||
rval = (route_success == n_targets) ? 1 : 0;
|
||||
gwbuf_free(querybuf);
|
||||
}
|
||||
else
|
||||
{
|
||||
MXS_ERROR("Could not find a valid routing backend. Either the "
|
||||
"'%s' is not set or the command is not recognized.",
|
||||
WRITE_BACKEND);
|
||||
gwbuf_free(querybuf);
|
||||
}
|
||||
if (rval == 1)
|
||||
{
|
||||
/* Non-atomic update of shared data, but contents are non-essential */
|
||||
m_routing_s++;
|
||||
}
|
||||
else
|
||||
{
|
||||
m_routing_f++;
|
||||
}
|
||||
return rval;
|
||||
}
|
||||
|
||||
void RRRouter::client_reply(RRRouterSession* rses, GWBUF* buf, DCB* backend_dcb)
|
||||
{
|
||||
if (rses->m_replies_to_ignore > 0)
|
||||
{
|
||||
/* In this case MaxScale cloned the message to many backends but the client
|
||||
* expects just one reply. Assume that client does not send next query until
|
||||
* previous has been answered.
|
||||
*/
|
||||
rses->m_replies_to_ignore--;
|
||||
gwbuf_free(buf);
|
||||
return;
|
||||
}
|
||||
|
||||
MXS_SESSION_ROUTE_REPLY(backend_dcb->session, buf);
|
||||
|
||||
m_routing_c++;
|
||||
if (m_print_on_routing)
|
||||
{
|
||||
MXS_NOTICE("Replied to client.\n");
|
||||
}
|
||||
}
|
||||
void RRRouter::handle_error(RRRouterSession* rses, GWBUF* message,
|
||||
DCB* problem_dcb, mxs_error_action_t action, bool* succp)
|
||||
{
|
||||
/* Don't handle same error twice on same DCB */
|
||||
if (problem_dcb->dcb_errhandle_called)
|
||||
{
|
||||
/* Assume that previous call succeed. */
|
||||
*succp = true;
|
||||
return;
|
||||
}
|
||||
else
|
||||
{
|
||||
problem_dcb->dcb_errhandle_called = true;
|
||||
}
|
||||
|
||||
MXS_SESSION* session = problem_dcb->session;
|
||||
DCB* client_dcb = session->client_dcb;
|
||||
mxs_session_state_t sesstate = session->state;
|
||||
|
||||
/* If the erroneous dcb is a client handler, close it. Setting succp to
|
||||
* false will cause the entire attached session to be closed.
|
||||
*/
|
||||
if (problem_dcb->dcb_role == DCB_ROLE_CLIENT_HANDLER)
|
||||
{
|
||||
dcb_close(problem_dcb);
|
||||
*succp = false;
|
||||
}
|
||||
else
|
||||
{
|
||||
switch (action)
|
||||
{
|
||||
case ERRACT_REPLY_CLIENT:
|
||||
{
|
||||
/* React to failed authentication, send message to client */
|
||||
if (sesstate == SESSION_STATE_ROUTER_READY)
|
||||
{
|
||||
/* Send error report to client */
|
||||
GWBUF* copy = gwbuf_clone(message);
|
||||
if (copy)
|
||||
{
|
||||
client_dcb->func.write(client_dcb, copy);
|
||||
}
|
||||
}
|
||||
*succp = false;
|
||||
}
|
||||
break;
|
||||
case ERRACT_NEW_CONNECTION:
|
||||
{
|
||||
/* React to a failed backend */
|
||||
if (problem_dcb->dcb_role == DCB_ROLE_BACKEND_HANDLER)
|
||||
{
|
||||
if (problem_dcb == rses->m_write_dcb)
|
||||
{
|
||||
dcb_close(rses->m_write_dcb);
|
||||
rses->m_write_dcb = NULL;
|
||||
}
|
||||
else
|
||||
{
|
||||
/* Find dcb in the list of backends */
|
||||
DCB_VEC::iterator iter = rses->m_backend_dcbs.begin();
|
||||
while (iter != rses->m_backend_dcbs.end())
|
||||
{
|
||||
if (*iter == problem_dcb)
|
||||
{
|
||||
dcb_close(*iter);
|
||||
rses->m_backend_dcbs.erase(iter);
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/* If there is still backends remaining, return true since
|
||||
* router can still function.
|
||||
*/
|
||||
*succp = (rses->m_backend_dcbs.size() > 0) ? true : false;
|
||||
}
|
||||
}
|
||||
break;
|
||||
default:
|
||||
ss_dassert(!true);
|
||||
*succp = false;
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
RRRouterSession::RRRouterSession(DCB_VEC& backends, DCB* write, DCB* client)
|
||||
: m_closed(false)
|
||||
, m_route_count(0)
|
||||
, m_on_transaction(false)
|
||||
, m_replies_to_ignore(0)
|
||||
{
|
||||
m_backend_dcbs = backends;
|
||||
m_write_dcb = write;
|
||||
m_client_dcb = client;
|
||||
}
|
||||
|
||||
RRRouterSession::~RRRouterSession()
|
||||
{
|
||||
/* Shouldn't happen. */
|
||||
ss_dassert(m_closed);
|
||||
}
|
||||
|
||||
void RRRouterSession::close()
|
||||
{
|
||||
if (!m_closed)
|
||||
{
|
||||
/**
|
||||
* Mark router session as closed. @c m_closed is checked at the start
|
||||
* of most API functions to quickly stop the processing of closed sessions.
|
||||
*/
|
||||
m_closed = true;
|
||||
int closed_conns = 0;
|
||||
for (unsigned int i = 0; i < m_backend_dcbs.size(); i++)
|
||||
{
|
||||
DCB* dcb = m_backend_dcbs[i];
|
||||
SERVER_REF* sref = dcb->service->dbref;
|
||||
dcb_close(dcb);
|
||||
closed_conns++;
|
||||
atomic_add(&(sref->connections), -1);
|
||||
}
|
||||
m_backend_dcbs.clear();
|
||||
if (m_write_dcb)
|
||||
{
|
||||
dcb_close(m_write_dcb);
|
||||
m_write_dcb = NULL;
|
||||
closed_conns++;
|
||||
}
|
||||
RR_DEBUG("Session with %d connections closed.", closed_conns);
|
||||
}
|
||||
}
|
||||
|
||||
void RRRouter::decide_target(RRRouterSession* rses, GWBUF* querybuf, DCB*& target, bool& route_to_all)
|
||||
{
|
||||
/* Extract the command type from the SQL-buffer */
|
||||
mysql_server_cmd_t cmd_type = MYSQL_GET_COMMAND(GWBUF_DATA(querybuf));
|
||||
/* The "query_types" is only really valid for query-commands but let's use
|
||||
* it here for all command types.
|
||||
*/
|
||||
uint32_t query_types = 0;
|
||||
|
||||
switch (cmd_type)
|
||||
{
|
||||
case MYSQL_COM_QUERY:
|
||||
{
|
||||
/* Use the inbuilt query_classifier to get information about
|
||||
* the query. The default qc works with mySQL-queries.
|
||||
*/
|
||||
query_types = qc_get_type_mask(querybuf);
|
||||
|
||||
#ifdef DEBUG_RRROUTER
|
||||
char* zSql_query = NULL;
|
||||
int length = 0;
|
||||
modutil_extract_SQL(querybuf, &zSql_query, &length);
|
||||
string sql_query(zSql_query, length);
|
||||
RR_DEBUG("QUERY: %s", sql_query.c_str());
|
||||
#endif
|
||||
}
|
||||
break;
|
||||
case MYSQL_COM_INIT_DB:
|
||||
query_types = q_route_to_all;
|
||||
RR_DEBUG("MYSQL_COM_INIT_DB");
|
||||
break;
|
||||
case MYSQL_COM_QUIT:
|
||||
query_types = q_route_to_all;
|
||||
RR_DEBUG("MYSQL_COM_QUIT");
|
||||
break;
|
||||
case MYSQL_COM_FIELD_LIST:
|
||||
query_types = q_route_to_rr;
|
||||
RR_DEBUG("MYSQL_COM_FIELD_LIST");
|
||||
break;
|
||||
default:
|
||||
/* TODO: Add support for other commands if needed. */
|
||||
/* This error message will only print the number of the cmd. */
|
||||
MXS_ERROR("Received unexpected sql command type: '%d'.", cmd_type);
|
||||
break;
|
||||
}
|
||||
|
||||
if ((query_types & q_route_to_write) != 0)
|
||||
{
|
||||
target = rses->m_write_dcb;
|
||||
}
|
||||
else
|
||||
{
|
||||
/* This is not yet sufficient for handling transactions. */
|
||||
if ((query_types & q_trx_begin) != 0)
|
||||
{
|
||||
rses->m_on_transaction = true;
|
||||
}
|
||||
if (rses->m_on_transaction)
|
||||
{
|
||||
/* If a transaction is going on, route all to write backend */
|
||||
target = rses->m_write_dcb;
|
||||
}
|
||||
if ((query_types & q_trx_end) != 0)
|
||||
{
|
||||
rses->m_on_transaction = false;
|
||||
}
|
||||
|
||||
if (!target && ((query_types & q_route_to_rr) != 0))
|
||||
{
|
||||
/* Round robin backend. */
|
||||
unsigned int index = (rses->m_route_count++) % rses->m_backend_dcbs.size();
|
||||
target = rses->m_backend_dcbs[index];
|
||||
}
|
||||
/* Some commands and queries are routed to all backends. */
|
||||
else if (!target && ((query_types & q_route_to_all) != 0))
|
||||
{
|
||||
route_to_all = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
* The functions implementing the router module API. These do not need to be
|
||||
* "extern C", but they do need to be callable from C code.
|
||||
*/
|
||||
static MXS_ROUTER* createInstance(SERVICE* service, char** options);
|
||||
static void* newSession(MXS_ROUTER* instance, MXS_SESSION* session);
|
||||
static void closeSession(MXS_ROUTER* instance, void* session);
|
||||
static void freeSession(MXS_ROUTER* instance, void* session);
|
||||
static int routeQuery(MXS_ROUTER* instance, void* session, GWBUF* querybuf);
|
||||
static void diagnostics(MXS_ROUTER* instance, DCB* dcb);
|
||||
static void clientReply(MXS_ROUTER* instance, void* router_session, GWBUF* resultbuf,
|
||||
DCB* backend_dcb);
|
||||
static void handleError(MXS_ROUTER* instance, void* router_session,
|
||||
GWBUF* errmsgbuf, DCB* backend_dcb,
|
||||
mxs_error_action_t action, bool* succp);
|
||||
static uint64_t getCapabilities(MXS_ROUTER *instance);
|
||||
static void destroyInstance(MXS_ROUTER* instance);
|
||||
/* The next two entry points are usually optional. */
|
||||
static int process_init();
|
||||
static void process_finish();
|
||||
|
||||
/*
|
||||
* This is called by the module loader during MaxScale startup. A module
|
||||
* description, including entrypoints and allowed configuration parameters,
|
||||
* is returned. This function must be exported.
|
||||
*/
|
||||
extern "C" MXS_MODULE* MXS_CREATE_MODULE();
|
||||
MXS_MODULE* MXS_CREATE_MODULE()
|
||||
{
|
||||
static MXS_ROUTER_OBJECT entryPoints =
|
||||
{
|
||||
createInstance,
|
||||
newSession,
|
||||
closeSession,
|
||||
freeSession,
|
||||
routeQuery,
|
||||
diagnostics,
|
||||
clientReply,
|
||||
handleError,
|
||||
getCapabilities,
|
||||
destroyInstance
|
||||
};
|
||||
|
||||
static MXS_MODULE moduleObject =
|
||||
{
|
||||
MXS_MODULE_API_ROUTER, /* Module type */
|
||||
MXS_MODULE_BETA_RELEASE, /* Release status */
|
||||
MXS_ROUTER_VERSION, /* Implemented module API version */
|
||||
"A simple round robin router", /* Description */
|
||||
"V1.1.0", /* Module version */
|
||||
&entryPoints, /* Defined above */
|
||||
process_init, /* Process init, can be null */
|
||||
process_finish, /* Process finish, can be null */
|
||||
NULL, /* Thread init */
|
||||
NULL, /* Thread finish */
|
||||
{
|
||||
/* Next is an array of MODULE_PARAM structs, max 64 items. These define all
|
||||
* the possible parameters that this module accepts. This is required
|
||||
* since the module loader also parses the configuration file for the module.
|
||||
* Any unrecognised parameters in the config file are discarded.
|
||||
*
|
||||
* Note that many common parameters, such as backend servers, are
|
||||
* already set to the upper level "service"-object.
|
||||
*/
|
||||
{ /* For simple types, only 3 of the 5 struct fields need to be
|
||||
* defined. */
|
||||
MAX_BACKENDS, /* Setting identifier in maxscale.cnf */
|
||||
MXS_MODULE_PARAM_INT, /* Setting type */
|
||||
"0" /* Default value */
|
||||
},
|
||||
{PRINT_ON_ROUTING, MXS_MODULE_PARAM_BOOL, "false"},
|
||||
{WRITE_BACKEND, MXS_MODULE_PARAM_SERVER, NULL},
|
||||
{ /* Enum types require an array with allowed values. */
|
||||
DUMMY,
|
||||
MXS_MODULE_PARAM_ENUM,
|
||||
"the_answer",
|
||||
MXS_MODULE_OPT_NONE,
|
||||
enum_example
|
||||
},
|
||||
{MXS_END_MODULE_PARAMS}
|
||||
}
|
||||
};
|
||||
return &moduleObject;
|
||||
}
|
||||
|
||||
/**
|
||||
* @brief Create an instance of the router (API).
|
||||
*
|
||||
* Create an instance of the round robin router. One instance of the router is
|
||||
* created for each service that is defined in the configuration as using this
|
||||
* router. One instance of the router will handle multiple connections
|
||||
* (router sessions).
|
||||
*
|
||||
* @param service The service this router is being created for
|
||||
* @param options The options for this query router
|
||||
* @return NULL in failure, pointer to router in success.
|
||||
*/
|
||||
static MXS_ROUTER* createInstance(SERVICE* service, char** options)
|
||||
{
|
||||
RRRouter* instance = NULL;
|
||||
/* The core of MaxScale is written in C and does not understand exceptions.
|
||||
* The macro catches all exceptions. Add custom handling here if required. */
|
||||
MXS_EXCEPTION_GUARD(instance = new RRRouter(service));
|
||||
/* The MXS_ROUTER is just a void*. Only the router module will dereference
|
||||
* the pointer. */
|
||||
|
||||
/* Register a custom command */
|
||||
if (!modulecmd_register_command("rrrouter", "test_command", custom_cmd_example,
|
||||
2, custom_cmd_args))
|
||||
{
|
||||
MXS_ERROR("Module command registration failed.");
|
||||
}
|
||||
return (MXS_ROUTER*)instance;
|
||||
}
|
||||
|
||||
/**
|
||||
* @brief Create a new router session for this router instance (API).
|
||||
*
|
||||
* Connect a client session to the router instance and return a router session.
|
||||
* The router session stores all client specific data required by the router.
|
||||
*
|
||||
* @param instance The router object instance
|
||||
* @param session The MaxScale session (generic client connection data)
|
||||
* @return Client specific data for this router
|
||||
*/
|
||||
static void* newSession(MXS_ROUTER* instance, MXS_SESSION* session)
|
||||
{
|
||||
RRRouter* router = (RRRouter*)instance;
|
||||
RRRouterSession* rses = NULL;
|
||||
MXS_EXCEPTION_GUARD(rses = router->create_session(session));
|
||||
return (void*)rses;
|
||||
}
|
||||
/**
|
||||
* @brief Close an existing router session for this router instance (API).
|
||||
*
|
||||
* Close a client session attached to the router instance. This function should
|
||||
* close connections and release other resources allocated in "newSession" or
|
||||
* otherwise held by the router session. This function should NOT free the
|
||||
* session object itself.
|
||||
*
|
||||
* @param instance The router object instance
|
||||
* @param session The router session
|
||||
*/
|
||||
static void closeSession(MXS_ROUTER* instance, void* session)
|
||||
{
|
||||
RRRouterSession* rses = (RRRouterSession*)session;
|
||||
MXS_EXCEPTION_GUARD(rses->close());
|
||||
}
|
||||
|
||||
/**
|
||||
* @brief Free a router session (API).
|
||||
*
|
||||
* When a router session has been closed, freeSession may be called to free
|
||||
* allocated resources.
|
||||
*
|
||||
* @param instance The router instance the session belongs to
|
||||
* @param session Router client session
|
||||
*
|
||||
*/
|
||||
static void freeSession(MXS_ROUTER* instance, void* session)
|
||||
{
|
||||
RRRouterSession* rses = (RRRouterSession*)session;
|
||||
delete rses;
|
||||
}
|
||||
|
||||
/**
|
||||
* @brief Route a packet (API)
|
||||
*
|
||||
* The routeQuery function receives a packet and makes the routing decision
|
||||
* based on the contents of the router instance, router session and the query
|
||||
* itself. It then sends the query to the target backend(s).
|
||||
*
|
||||
* @param instance Router instance
|
||||
* @param session Router session associated with the client
|
||||
* @param buffer Buffer containing the query (or command)
|
||||
* @return 1 on success, 0 on error
|
||||
*/
|
||||
static int routeQuery(MXS_ROUTER* instance, void* session, GWBUF* buffer)
|
||||
{
|
||||
RRRouter* router = (RRRouter*)instance;
|
||||
RRRouterSession* rses = (RRRouterSession*)session;
|
||||
int rval = 0;
|
||||
MXS_EXCEPTION_GUARD(rval = router->route_query(rses, buffer));
|
||||
return rval;
|
||||
}
|
||||
|
||||
/**
|
||||
* @brief Diagnostics routine (API)
|
||||
*
|
||||
* Print router statistics to the DCB passed in. This is usually called by the
|
||||
* MaxInfo or MaxAdmin modules.
|
||||
*
|
||||
* @param instance The router instance
|
||||
* @param dcb The DCB for diagnostic output
|
||||
*/
|
||||
static void diagnostics(MXS_ROUTER* instance, DCB* dcb)
|
||||
{
|
||||
RRRouter* router = (RRRouter*)instance;
|
||||
dcb_printf(dcb, "\t\tQueries routed successfully: %lu\n", router->m_routing_s);
|
||||
dcb_printf(dcb, "\t\tFailed routing attempts: %lu\n", router->m_routing_f);
|
||||
dcb_printf(dcb, "\t\tClient replies routed: %lu\n", router->m_routing_c);
|
||||
}
|
||||
|
||||
/**
|
||||
* @brief Client Reply routine (API)
|
||||
*
|
||||
* This routine receives a packet from a backend server meant for the client.
|
||||
* Often, there is little logic needed and the packet can just be forwarded to
|
||||
* the next element in the processing chain.
|
||||
*
|
||||
* @param instance The router instance
|
||||
* @param session The router session
|
||||
* @param backend_dcb The backend DCB (data source)
|
||||
* @param queue The GWBUF with reply data
|
||||
*/
|
||||
static void clientReply(MXS_ROUTER* instance, void* session, GWBUF* queue,
|
||||
DCB* backend_dcb)
|
||||
{
|
||||
RRRouter* router = (RRRouter*)instance;
|
||||
RRRouterSession* rses = (RRRouterSession*)session;
|
||||
MXS_EXCEPTION_GUARD(router->client_reply(rses, queue, backend_dcb));
|
||||
}
|
||||
|
||||
/**
|
||||
* Error Handler routine (API)
|
||||
*
|
||||
* This routine will handle errors that occurred with the session. This function
|
||||
* is called if routeQuery() returns 0 instead of 1. The client or a backend
|
||||
* unexpectedly closing a connection also triggers this routine.
|
||||
*
|
||||
* @param instance The router instance
|
||||
* @param session The router session
|
||||
* @param message The error message to reply
|
||||
* @param problem_dcb The DCB related to the error
|
||||
* @param action The action: ERRACT_NEW_CONNECTION or ERRACT_REPLY_CLIENT
|
||||
* @param succp Output result of action, true if router can continue
|
||||
*/
|
||||
static void handleError(MXS_ROUTER* instance, void* session,
|
||||
GWBUF* message, DCB* problem_dcb,
|
||||
mxs_error_action_t action, bool* succp)
|
||||
{
|
||||
RRRouter* router = (RRRouter*)instance;
|
||||
RRRouterSession* rses = (RRRouterSession*)session;
|
||||
MXS_EXCEPTION_GUARD(router->handle_error(rses, message, problem_dcb, action, succp));
|
||||
}
|
||||
|
||||
/**
|
||||
* @brief Get router capabilities (API)
|
||||
*
|
||||
* Return a bit map indicating the characteristics of this router type.
|
||||
* In this case, the only bit set indicates that the router wants to receive
|
||||
* data for routing as whole SQL statements.
|
||||
*
|
||||
* @param instance The router instance
|
||||
* @return RCAP_TYPE_CONTIGUOUS_INPUT | RCAP_TYPE_STMT_OUTPUT
|
||||
*/
|
||||
static uint64_t getCapabilities(MXS_ROUTER *instance)
|
||||
{
|
||||
/* This router needs to parse client queries, so it should set RCAP_TYPE_CONTIGUOUS_INPUT.
|
||||
* For output, parsing is not required but counting SQL replies is. RCAP_TYPE_RESULTSET_OUTPUT
|
||||
* should be sufficient.
|
||||
*/
|
||||
return RCAP_TYPE_CONTIGUOUS_INPUT | RCAP_TYPE_RESULTSET_OUTPUT;
|
||||
}
|
||||
|
||||
/**
|
||||
* @brief Destroy router instance (API)
|
||||
*
|
||||
* A destroy-function is rarely required, since this is usually called only at
|
||||
* program exit.
|
||||
*
|
||||
* @param instance Router instance
|
||||
*/
|
||||
static void destroyInstance(MXS_ROUTER* instance)
|
||||
{
|
||||
RRRouter* router = (RRRouter*)instance;
|
||||
delete router;
|
||||
}
|
||||
|
||||
/**
|
||||
* Make any initializations required by the router module as a whole and not
|
||||
* specific to any individual router instance.
|
||||
*
|
||||
* @return 0 on success
|
||||
*/
|
||||
static int process_init()
|
||||
{
|
||||
RR_DEBUG("Module loaded.");
|
||||
return 0;
|
||||
}
|
||||
/**
|
||||
* Undo module initializations.
|
||||
*/
|
||||
static void process_finish()
|
||||
{
|
||||
RR_DEBUG("Module unloaded.");
|
||||
}
|
||||
|
||||
/**
|
||||
* A function executed as a custom module command through MaxAdmin
|
||||
* @param argv The arguments
|
||||
*/
|
||||
bool custom_cmd_example(const MODULECMD_ARG *argv)
|
||||
{
|
||||
cout << MXS_MODULE_NAME << " wishes the Admin a good day.\n";
|
||||
int n_args = argv->argc;
|
||||
cout << "The module got " << n_args << " arguments.\n";
|
||||
for (int i = 0; i < n_args; i++)
|
||||
{
|
||||
arg_node node = argv->argv[i];
|
||||
string type_str;
|
||||
string val_str;
|
||||
switch (MODULECMD_GET_TYPE(&node.type))
|
||||
{
|
||||
case MODULECMD_ARG_STRING:
|
||||
{
|
||||
type_str = "string";
|
||||
val_str.assign(node.value.string);
|
||||
}
|
||||
break;
|
||||
case MODULECMD_ARG_BOOLEAN:
|
||||
{
|
||||
type_str = "boolean";
|
||||
val_str.assign((node.value.boolean) ? "true" : "false");
|
||||
}
|
||||
break;
|
||||
default:
|
||||
{
|
||||
type_str = "other";
|
||||
val_str.assign("unknown");
|
||||
}
|
||||
break;
|
||||
}
|
||||
cout << "Argument " << i << ": type '" << type_str << "' value '" << val_str <<
|
||||
"'\n";
|
||||
}
|
||||
return true;
|
||||
}
|
Loading…
x
Reference in New Issue
Block a user