diff --git a/Documentation/Documentation-Contents.md b/Documentation/Documentation-Contents.md new file mode 100644 index 000000000..b6cbeb08a --- /dev/null +++ b/Documentation/Documentation-Contents.md @@ -0,0 +1,17 @@ +# Contents + +## Filters + + - [Query Log All](filters/Query-Log-All-Filter.md) + - [Regex Filter](filters/Regex-Filter.md) + - [Tee Filter](filters/Tee-Filter.md) + - [Top N Filter](filters/Top-N-Filter.md) + +## Earlier Release Notes + + - [MaxScale 0.5 Release Notes](Release-Notes/MaxScale-0.5-Release-Notes.md) + - [MaxScale 0.6 Release Notes](Release-Notes/MaxScale-0.6-Release-Notes.md) + - [MaxScale 0.7 Release Notes](Release-Notes/MaxScale-0.7-Release-Notes.md) + - [MaxScale 1.0 Release Notes](Release-Notes/MaxScale-1.0-Release-Notes.md) + - [MaxScale 1.0.1 Release Notes](Release-Notes/MaxScale-1.0.1-Release-Notes.md) + - [MaxScale 1.0.3 Release Notes](Release-Notes/MaxScale-1.0.3-Release-Notes.md) diff --git a/Documentation/Release-Notes/MaxScale-0.5-Release-Notes.md b/Documentation/Release-Notes/MaxScale-0.5-Release-Notes.md new file mode 100644 index 000000000..403bf1d92 --- /dev/null +++ b/Documentation/Release-Notes/MaxScale-0.5-Release-Notes.md @@ -0,0 +1,336 @@ +MaxScale Release Notes + +0.5 Alpha + +This document details the changes in version 0.5 since the release of the 0.4 alpha of the MaxScale product. + +# New Features + +## Read/Write Splitter Routing Module + +In previous versions the read/write splitter routing module has had a number of limitations on it use, in the alpha release the router now removes the most important restrictions. + +### Session Commands + +Session commands are those statements that make some change to the user’s login session that may cause different effects from subsequent statements executed. Since the read/write splitter executes statements on either a master server or a slave server, depending upon the statement to execute, it is important that these session modifications are executed on all connections to both slave and master servers. This is resolved in release 0.5 such that session modification commands are executed on all active connections and a single return is forward back to the client that made the request. + +### Transaction Support + +Transaction support has been added into this version of the read/write splitter, there is one known outstanding limitation. If autocommit is enabled inside an active transaction it is not considered as commit in read/write splitter. Once a transaction has started all statements are routed to a master until the transaction is committed or rolled back. + +## Authentication + +A number of issues and shortcomings in the authentication performed by MaxScale have been resolved by this release. + +### Host Considered in Authentication + +Previously MaxScale did not follow the same rules as MySQL when authenticating a login request, it would always use the wildcard password entries and would not check the incoming host was allowed to connect. MaxScale now checks the incoming IP address for a connection request and verifies this against the authentication data loaded from the backend servers. The same rules are applied when choosing the password entry to authenticate with. Note however that authentication from MaxScale to the backend database will fail if the MaxScale host is not allowed to login using the matching password for the user. + +### Stale Authentication Data + +In previous releases of MaxScale the authentication data would be read at startup time only and would not be refreshed. Therefore if a user was added or modified in the backend server this will not be picked up by MaxScale and that user would be unable to connect via MaxScale. MaxScale now reloads user authentication data when a failure occurs and will refresh its internal tables if the data has changed in the backend. Please note that this reload process is rate limited to prevent incorrect logins to MaxScale being used for a denial of service attack on the backend servers. + +### Enable Use Of "root" User + +Previously MaxScale would prevent the use of the root user to login to the backend servers via MaxScale. This may be enabled on a per service basis by adding an "enable_root_user" options in the service entry to enable it in the MaxScale configuration file. This allows the use of root to be controlled on a per service basis. + +## Network Support + +### Unix Domain Sockets + +MaxScale now supports Unix domain sockets for connecting to a local MaxScale server. The use of a Unix domain socket is controlled by adding a "socket" entry in the listener configuration entry for a service. + +### Network Interface Binding + +MaxScale has added the ability to bind a listener for a service to a network address via an "address" entry in the configuration file. + +# Server Version + +The server version reported when connected to a database via MaxScale has now been altered. This now shows the MaxScale name and version together with the backend server name. An example of this can be seen below for the 0.5 release. + +-bash-4.1$ mysql -h 127.0.0.1 -P 4006 -uxxxx -pxxxx Welcome to the MariaDB monitor. Commands end with ; or \g. Your MySQL connection id is 22320 Server version: MaxScale 0.5.0 MariaDB Server Copyright (c) 2000, 2012, Oracle, Monty Program Ab and others. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. MySQL [(none)]> \s -------------- mysql Ver 15.1 Distrib 5.5.28a-MariaDB, for Linux (i686) using readline 5.1 ... Server: MySQL Server version: MaxScale 0.5.0 MariaDB Server ... -------------- MySQL [(none)]> + +# Bug Fixes + +A number of bug fixes have been applied between the 0.4 alpha and this alpha release. The table below lists the bugs that have been resolved. The details for each of these may be found in bugs.skysql.com. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
IDSummary
141No "delete user" command in debugcli
175Buffer leak in dcb_read from Coverity run
178Uninitialised variables from Coverity run
179open with O_CREAT in second argument needs 3 arguments
363simple_mutex "name" memory handling ...
126"reload config" in debug interface causes maxscale server to segfault
149It is possible to delete all maxscale users
218there is no way to understand what is going on if MAXSCALE_HOME is incorrect
137"show users" and "reload users" refer to very different things in debugcli
154readwritesplit does not use router_options
160telnetd leaks memory
169Galera monitor is actually never compiled ....
172Several compile errors in galera_mon.c
174Resource leak in server.c
176Resource leak in gw_utils.c
362possible datadir_cleanup() problems ...
124readconnroute does not validate router_options
153MaxScale fails when max connections are exceeded
133MaxScale leaves lots of "data" directories sitting around $MAXSCALE_HOME
166readwritesplit causes MaxScale segfault when starting up
207Quitting telnet session causes maxscale to fail
161Memory leak in load_mysql_users.
177Resource leak in secrets.c
182On Startup logfiles are empty
135MaxScale unsafely handles empty passwords in getUsers
145.secret file for encrypted passwords cyclicly searched
171ifndef logic in build_gateway.inc doesn't work, MARIADB_SRC_PATH from env not picked up
173Resource leak in adminusers.c found by Coverity
376Confusing Server Version
370maxscale binary returns zero exit status on failures
150telnetd listener should bind to 127.0.0.1 by default
152listener configuration should support bind address
373Documentation: it's not clear what privileges the maxscale user needs
128Maxscale prints debug information to terminal session when run in background
129MaxScale refuses to connect to server and reports nonsense error as a result
147Maxscale's hashtable fails to handle deletion of entries.
148users data structure's stats have incorrect values.
384MaxScale crashes if backend authentication fails
210Bad timing in freeing readconnrouter's dcbs cause maxscale crash
403gwbuf_free doesn't protect freeing shared buffer
371If router module load fails, MaxScale goes to inifinite loop
385MaxScale (DEBUG-version) dasserts if backend dcb is closed in the middle of client dcb performing close_dcb
386Starting MaxScale with -c pointing at existing file causes erroneous behavior
209Error in backend hangs client connection
194maxscale crashes at start if module load fails
369typo in "QUERY_TYPE_UNKNWON"
163MaxScale crashes with multiple threads
162threads parameter in configuration file is not effective
400hastable_get_stats returns value of uninitialized value in 'nelems'
212Failing write causes maxscale to fail
222Double freeing mutex corrupts log
208current_connection_count is decreased multiple times per session, thus breaking load balancing logic
378Misspelling maxscale section name in config file crashes maxscale
399Every row in log starts with 0x0A00
205MaxScale crashes due SEGFAULT because return value of dcb_read is not checked
220Maxscale crash if socket listening fails in startup
372Log manager hangs MaxScale if log string (mostly query length) exceeds block size
397Free of uninitialised pointer if MAXSCALE_HOME is not set
402gw_decode_mysql_server_handshake asserts with mysql 5.1 backend
345MaxScale don't find backend servers if they are started after MaxScale
406Memory leak in dcb_alloc()
360MaxScale passwd option
151Get parse_sql failed on array INSERT
216Backend error handling doesn't update server's connection counter
127MaxScale should handle out-of-date backend auth data more gracefully
146"show dbusers" argument not documented
125readconnroute causes maxscale server crash if no slaves are available
375Tarball contains UID and maxscale base dir
+ + diff --git a/Documentation/Release-Notes/MaxScale-0.6-Release-Notes.md b/Documentation/Release-Notes/MaxScale-0.6-Release-Notes.md new file mode 100644 index 000000000..ff7033e4d --- /dev/null +++ b/Documentation/Release-Notes/MaxScale-0.6-Release-Notes.md @@ -0,0 +1,31 @@ +MaxScale Release Notes + +0.6 Alpha + +This document details the changes in version 0.6 since the release of the 0.5 alpha of the MaxScale product. The 0.6 version is merely a set of bug fixes based on the previous 0.5 version. + +# Bug Fixes + +A number of bug fixes have been applied between the 0.5 alpha and this alpha release. The table below lists the bugs that have been resolved. The details for each of these may be found in bugs.skysql.com. + + + + + + + + + + + + + + +
IDSummary
423The new "version_string" parameter has been added to service section. +This allows a specific version string to be set for each service, this version string is used in the MySQL handshake from MaxScale to clients and is reported as the server version to clients. + +The version_string is optional, the default value will be taken from the embedded MariaDB library which supplies the parser to MaxScale.
418Statements are not routed to master if a transaction is started implicitly by setting autocommit=0. In such cases statements were previously routed as if they were not part of a transaction. + +This fix changes the behavior so that is autocommit is disabled, all statements are routed to the master and in case of session variable updates, to both master and slave.
+ + diff --git a/Documentation/Release-Notes/MaxScale-0.7-Release-Notes.md b/Documentation/Release-Notes/MaxScale-0.7-Release-Notes.md new file mode 100644 index 000000000..bc43ee00d --- /dev/null +++ b/Documentation/Release-Notes/MaxScale-0.7-Release-Notes.md @@ -0,0 +1,158 @@ +MaxScale Release Notes + +0.7 Alpha + +This document details the changes in version 0.7 since the release of the 0.6 alpha of the MaxScale product. + +# New Features + +## Galera Support + +Enhanced support for Galera cluster to allow Galera to be used as a High Available Cluster with no write contention between the nodes.. + +MaxScale will control access to a Galera Cluster such that one node is designated as the master node to which all write operations will be sent. Read operations will be sent to any of the remaining nodes that are part of the cluster. Should the currently elected master node fail MaxScale will automatically promote one of the remaining nodes to become the new master node. + +## Multiple Slave Connections + +The Read/Write Split query router has been enhanced to allow multiple slaves connections to be created. The number of slave connections is configurable via a parameter in the MaxScale configuration file. + +Adding multiple connections allows for better load balancing between the slaves and in a pre-requisite for providing improved fault tolerance within the Read/Write Splitter. The selection of which slave to use for a particular read operation can be controlled via options in the router configuration. + +## Debug Interface Enhancements + +A number of new list commands have been added to the debug interface to allow more concise tabular output of certain object types within the interface. + +**MaxScale>** help list + +Available options to the list command: + + filters List all the filters defined within MaxScale + + listeners List all the listeners defined within MaxScale + + modules Show all currently loaded modules + + services List all the services defined within MaxScale + + servers List all the servers defined within MaxScale + + sessions List all the active sessions within MaxScale + +**MaxScale>** + +Those objects that are defined in the configuration file can now be referenced by the names used in the configuration file rather than by using memory addresses. This means that services, servers, monitors and filters can all now be referenced using meaningful names provided by the user. Internal objects such as DCB’s and sessions, which are not named in the configuration file still require the use of memory addresses. + +Two modes of operation of the interface are now available, user mode and developer mode. The user mode restricts access to the feature that allow arbitrary structures to be examined and checks all memory address for validity before allowing access. + +## Maintenance Mode for Servers + +MaxScale now provides a maintenance mode for servers, this mode allows servers to be set such that no new connections will be opened to that server. Also, servers in maintenance mode are not monitored by MaxScale. This allows an administrator to set a server into maintenance mode when it is required to be taken out of use. The connections will then diminish over time and since no new connections are created, the administrator can remove the node from use to perform some maintenance activities. + +Nodes are placed into maintenance mode via the debug interface using the set server command. + +**MaxScale>** set server datanode3 maintenance + +Nodes are taken out of maintenance using the clear server command. + +**MaxScale>** clear server datanode3 maintenance + +## Configurable Monitoring Interval + +All monitor plugins now provide a configuration parameter that can be set to control how frequently the MaxScale monitoring is performed. + +## Replication Lag Heartbeat Monitor + +The mysqlmon monitor module now implements a replication heartbeat protocol that is used to determine the lag between updates to the master and those updates being applied to the slave. This information is then made available to routing modules and may be used to determine if a particular slave node may be used or which slave node is most up to date. + +## Filters API + +The first phase of the filter API is available as part of this release. This provides filtering for the statements from the client application to the router. Filtering for the returned results has not yet been implemented and will be available in a future version. + +Three example filters are including in the release + +1. Statement counting Filter - a simple filter that counts the number of SQL statements executed within a session. Results may be viewed via the debug interface. + +2. Query Logging Filter - a simple query logging filter that write all statements for a session into a log file for that session. + +3. Query Rewrite Filter - an example of how filters can alter the query contents. This filter allows a regular expression to be defined, along with replacement text that should be substituted for every match of that regular expression. + +## MariaDB 10 Replication Support + +The myqlmon monitor module has been updated to support the new syntax for show all slaves status in MariaDB in order to correctly determine the master and slave state of each server being monitor. Determination of MariaDB 10 is automatically performed by the monitor and no configuration is required. + +## API Versioning + +The module interface has been enhanced to allow the API version in use to be reported, along with the status of the module and a short description of the module. The status allows for differentiation of the release status of a plugin to be identified independently of the core of MaxScale. plugins may be designated as "in development", “alpha”, “beta” or “GA”. + +**MaxScale>** list modules + +Module Name | Module Type | Version | API | Status + +---------------------------------------------------------------- + +regexfilter | Filter | V1.0.0 | 1.0.0 | Alpha + +MySQLBackend | Protocol | V2.0.0 | 1.0.0 | Alpha + +telnetd | Protocol | V1.0.1 | 1.0.0 | Alpha + +MySQLClient | Protocol | V1.0.0 | 1.0.0 | Alpha + +mysqlmon | Monitor | V1.2.0 | 1.0.0 | Alpha + +readwritesplit | Router | V1.0.2 | 1.0.0 | Alpha + +readconnroute | Router | V1.0.2 | 1.0.0 | Alpha + +debugcli | Router | V1.1.1 | 1.0.0 | Alpha + +**MaxScale>** + +# Bug Fixes + +A number of bug fixes have been applied between the 0.6 alpha and this alpha release. The table below lists the bugs that have been resolved. The details for each of these may be found in bugs.skysql.com. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
IDSummary
443mysql/galera monitors hang when backend fails
424Read/Write Splitter closes connection without sending COM_QUIT
438Internal thread deadlock
436Sessions in invalid state
359Router options for Read/Write Split module
435Some automated tests have invalid SQL syntax
431rwsplit.sh test script has incorrect bash syntax
425MaxScale crashes after prolonged use
+ + +# Linking + +Following reported issues with incompatibilities between MaxScale and the shared library used by MySQL this version of MaxScale will be statically linked with the MariaDB 5.5 embedded library that it requires. This library is used for internal purposes only and does not result in MaxScale support for other versions of MySQL or MariaDB being affected. + diff --git a/Documentation/Release-Notes/MaxScale-1.0-Release-Notes.md b/Documentation/Release-Notes/MaxScale-1.0-Release-Notes.md new file mode 100644 index 000000000..a15757e0f --- /dev/null +++ b/Documentation/Release-Notes/MaxScale-1.0-Release-Notes.md @@ -0,0 +1,124 @@ +MaxScale Release Notes + +1.0 Beta + +This document details the changes in version 1.0 since the release of the 0.7 alpha of the MaxScale product. + +# New Features + +## Complex Replication Structures + +The MaxScale monitor module for Master/Slave replication is now able to correctly identify tree structured replication environments and route write statements to the master server at the root level of the tree. Isolated database instances and now also correctly identified as external to the replication tree. + +## Read/Write Splitter Enhancements + +### Support For Prepared Statements + +Prepared statements are now correctly recognised by MaxScale, with the prepare stage being sent to all the eligible servers that could eventually run the statement. Statements are then execute on a single server. + +### Slave Failure Resilience + +The Read/Write splitter can not be used to establish multiple connections to different slave servers. The read load will be distributed across these slaves and slave failure will be masked from the application as MaxScale will automatically failover to another slave when one fails. + +### Configurable Load Balancing Options + +It is now possible to configure the criteria that the Read/Write Splitter uses for load balancing, the options are: + +* The total number of connections to the servers, from this MaxScale instance + +* The number of connections to the server for this particular MaxScale service + +* The number of statements currently being executed on the server on behalf of this MaxScale instance + +* Route statements to the slave that has the least replication lag + +### Replication Consistency + +The Read/Write splitter may now be configured to exclude nodes that are currently showing a replication lag greater than a configurable threshold. The replication lag is measured using the MySQL Monitor module of MaxScale. + +Alternatively it is possible to define that read operations should be routed to the slave that has the least measured replication lag. + +## Weighted Routing Options + +The distribution of connections and statement across the set of nodes can be controlled by attaching arbitrary parameters to the servers and then configuring the router to use that parameter value as a weighting factor when deciding which of the valid servers to which to connect or route queries. + +Several parameters may be used on each host and different routers may choose to use different parameters as the weighting parameter for that router. The use of weighting is optional, if no weighting parameter is given in the service definition then all eligible servers will have an equal distribution applied. + +Server weighting is supported by both the Read/Write Splitter and the connection router. + +## MaxAdmin Client + +A new administrative interface has been added that uses a MaxScale specific client application to interact with MaxScale to control and monitor the MaxScale activities. This client application may be used interactively or within scripts, passing commands to MaxScale via command line arguments. Command scripts are available, allowing command sets of commands to be stored in script files. + +MaxAdmin also supports command history via libedit on those distributions that support the libedit library. This allows for the use of the up and down cursor keys or selection of previous commands and editing of lines using vi or emacs style editing commands. + +## Pacemaker Support + +MaxScale now ships with an init.d script that is compatible with the use of Pacemaker and Heartbeat to provide for a highly available implementation of MaxScale. A tutorial on setting up MaxScale under Pacemaker control is included in the Documentation directory. + +## Filter API Enhancements + +The filter API has now been enhanced to operate not just on downstream query filtering but also upstream result set filtering. + +## Enhanced and New Filters + +Addition of new filters and enhancements to those existing filters that appeared in 0.7 of MaxScale. + +### Top Filter + +A new filter to capture and log the longest running queries within a client session. The filter can be configured to capture a specific number of queries that take the longest time between the query being submitted to the database server and the first result being returned. + +The queries captured can be defined using regular expressions to include and exclude queries that match these expressions. In addition the inclusion of a session may be based on the user name used to connect to the database or the source address of the client session. + +### Tee Filter + +A filter to optionally duplicate requests received from the client and send them to other services within MaxScale. This allows a single statement sent by a client to be routed to multiple storage backends via MaxScale. + +The queries duplicated can be defined using regular expressions to include and exclude queries that match these expressions. In addition the inclusion of a session may be based on the user name used to connect to the database or the source client session. + +### QLA and Regex Filter Improvements + +These filters have been enhanced to provide for the inclusion of sessions by specifying the username used to connect to the database or the source of the client connection as a criteria to trigger the use of these filters for particular sessions connected to the MaxScale service. + +# Bug Fixes + +A number of bug fixes have been applied between the 0.6 alpha and this alpha release. The table below lists the bugs that have been resolved. The details for each of these may be found in bugs.skysql.com. + + + + + + + + + + + + + + + + + + + + + + + + + + +
IDSummary
441Possible failure to return a value in setipaddress
396Build instruction suggest forcing install of RPM’s
452Make install copies the modules to an incorrect directory
450Read/Write splitter does not balance load between multiple slaves
449The router clientReply function does not handle GWBUF structures correctly
+ + +# Packaging + +Both RPM and Debian packages are available for MaxScale in addition to the tar based releases previously distributed we now provide + +* CentOS/RedHat 5 RPM + +* CentOS/RedHat 6 RPM + +* Ubuntu 14.04 package + diff --git a/Documentation/Release-Notes/MaxScale-1.0.1-Release-Notes.md b/Documentation/Release-Notes/MaxScale-1.0.1-Release-Notes.md new file mode 100644 index 000000000..6a34ec3a9 --- /dev/null +++ b/Documentation/Release-Notes/MaxScale-1.0.1-Release-Notes.md @@ -0,0 +1,332 @@ +MaxScale Release Notes + +1.0.1 Beta + +This document details the changes in version 1.0.1 since the release of the 1.0 beta of the MaxScale product. + +# New Features + +## CMake build system + +Building MaxScale is now easier than ever thanks to the introduction of CMake into the build process. Building with CMake removes the need to edit files, specify directory locations or change build flags, in all but the rarest of the cases, and building with non-standard configurations is a lot easier thanks to the easy configuration of all the build parameters. + +Here’s a short list of the most common build parameters,their functions and default values. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
VariablePurposeDefault value
INSTALL_DIRRoot location of the MaxScale install/usr/local/skysql/maxscale
STATIC_EMBEDDEDWhether to use the static or the dynamic version of the embedded libraryNo
OLEVELLevel of optimization used when compilingNo optimization
INSTALL_SYSTEM_FILESIf startup scripts should be installed into /etc/init.d and ldconfig configuration files to /etc/ld.so.conf.dYes
BUILD_TYPEThe type of the build. ‘None’ for normal, ‘Debug’ for debugging and ‘Optimized’ for an optimized build.None
+ + +Details on all the configurable parameters and instructions on how to use CMake can be found in the README file. + +## Enhancements + +The polling mechanism in MaxScale has been modified to overcome a flaw which mean that connections with a heavy I/O load could starve other connections within MaxScale and prevent query execution. This has been resolved with a more fairer event scheduling mechanism within the MaxScale polling subsystem. This has led to improve overall performance in high load situations. + +# Bug Fixes + +A number of bug fixes have been applied between the 1.0 beta release and this release candidate. The table below lists the bugs that have been resolved. The details for each of these may be found in bugs.skysql.com. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
IDSummary
462Testall target fails in server/test to invalid MAXSCALE_HOME path specification
467max_slave_replication lag is not effective after session creation
468query_classifier : if parsing fails, parse tree and thread context are freed but used
469rwsplit counts every connection twice in master - connection counts leak
466hint_next_token doesn't detect = pair if there are no spaces around '='
470Maxscale crashes after a normal query if a query with named hint was used before
473Entering a hint with route server target as '=(' causes a crash
472Using a named hint after its initial use causes a crash
471Routing Hints route to server sometimes doesn't work
463MaxScale hangs receiving more than 16K in input
476mysql_common.c:protocol_archive_srv_command leaks memory and accesses freed memory
479Undefined filter reference in MaxScale.cnf causes a crash
410MaxScale.cnf server option is not parsed for spaces
417Galera monitor freezes on network failure of a server
488SHOW VARIABLES randomly failing with "Lost connection to MySQL server"
484Hashtable does not always release write lock during add
485Hashtable not locked soon enough in iterator get next item
493Can have same section name multiple times without warning
510Embedded library crashes on a call to free_embedded_thd
511Format strings in log_manager.cc should be const char*
509rw-split sensitive to order of terms in field list of SELECT
507rw-split router does not send last_insert_id() to master
490session handling for non-determinstic user variables broken
489@@hostname and @@server_id treated differently from @@wsrep_node_address
528Wrong service name in tee filter crashes maxscale on connect
530MaxScale socket permission
536log_manager doesn't write buffers to disk in the order they are written
447Error log is flooded with same warning if there are no slaves present
475The end comment tag in hints isn't properly detected.
181Missing log entry if server not reachable
486Hashtable problems when created with size less than one
516maxadmin CLI client sessions are not closed?
495Referring to a nonexisting server in servers=... doesn't even raise a warning
538maxscale should expose details of "Down" server
539MaxScale crashes in session_setup_filters
494The service 'CLI' is missing a definition of the servers that provide the service
180Documentation: No information found in the documentation about firewall settings
524Connecting to MaxScale from localhost tries matching @127.0.0.1 grant
481MySQL monitor doesn't set master server if the replication is broken
437Failure to detect MHA master switch
541Long queries cause MaxScale to block
492In dcb.c switch fallthrough appears to be used without comment
439Memory leak in getUsers
545RWSplit: session modification commands weren't routed to all if executed inside open transaction
543RWSplit router statistics counters are not updated correctly
544server with weight=0 gets one connection
525Crash when saving post in Wordpress
533Drupal installer hangs
497Can’t enable debug/trace logs in configuration file
430Temporary tables not working in MaxScale
527No signal handler for segfault etc
546Use of weightby router parameter causes error log write
506Don’t write shm/tmpfs by default without telling the user or giving a way to override it
552Long argument options to maxadmin and maxscale broke maxadmin commands
521Many commands in maxadmin client simply hang
478Parallel session command processing fails
499make clean leavessoem .o files behind
500"depend: no such file warnings during make
501log_manager, query classifier rebuilds unconditionally
502log_manager and query_classifier builds always rebuild utils
504clean rule for Documentation directory in wrong makefile
505utils/makefile builds stuff unconditionally, misses "depend" target
548MaxScale accesses freed client DCB and crashes
550modutil functions process length incorrectly
+ + +# Packaging + +Both RPM and Debian packages are available for MaxScale in addition to the tar based releases previously distributed we now provide + +* CentOS/RedHat 5 RPM + +* CentOS/RedHat 6 RPM + +* Ubuntu 14.04 package + diff --git a/Documentation/Release-Notes/MaxScale-1.0.3-Release-Notes.md b/Documentation/Release-Notes/MaxScale-1.0.3-Release-Notes.md new file mode 100644 index 000000000..67c5e3ece --- /dev/null +++ b/Documentation/Release-Notes/MaxScale-1.0.3-Release-Notes.md @@ -0,0 +1,136 @@ +MaxScale Release Notes + +1.0.3 GA + +This document details the changes in version 1.0.3 since the release of the 1.0.2 Release Candidate of the MaxScale product. + +# New Features + +No new features have been introduced since the released candidate was released. + +# Bug Fixes + +A number of bug fixes have been applied between the 0.6 alpha and this alpha release. The table below lists the bugs that have been resolved. The details for each of these may be found in bugs.mariadb.com. + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
IDSummary
644Buffered that were cloned using the gwbuf_clone routine failed to initialise the buffer lock structure correctly.
643Recursive filter definitions in the configuration file could cause MaxScale to loop
665An access to memory that had already been freed could be made within the MaxScale core
664MySQL Authentication code could access memory that had already been freed.
673MaxScale could crash if it had an empty user table and the MaxAdmin show dbusers command was run
670The tee filter could lose statement on the branch service if the branch service was significantly slower at executing statements compared with the main service.
653Memory corruption could occur with extremely long hostnames in the mysql.user table.
657If the branch service of a tee filter shutdown unexpectedly then MaxScale could fail
654Missing quotes in MaxAdmin show dbusers command could cause MaxAdmin to crash
677A race condition existed in the tee filter client reply handling
658The readconnroute router did not correctly close sessions when a backend database failed
662MaxScale startup hangs if no backend servers respond
676MaxScale writes a log entry, "Write to backend failed. Session closed." when changing default database via readwritesplit with max_slave_connections != 100%
650Tee filter does not correctly detect missing branch service
645Tee filter can hang MaxScale if the read/write splitter is used
678Tee filter does not always send full query to branch service
679A shared pointer in the service was leading to misleading service states
680The Read/Write Splitter can not load users if there are no databases available at startup
681The Read/Write Splitter could crash is the value of max_slave_connections was set to a low percentage and only a small number of backend servers are available
+ + +# Known Issues + +There are a number bugs and known limitations within this version of MaxScale, the most serious of this are listed below. + +* The SQL construct "LOAD DATA LOCAL INFILE" is not fully supported. + +* The Read/Write Splitter is a little too strict when it receives errors from slave servers during execution of session commands. This can result in sessions being terminated in situation in which MaxScale could recover without terminating the sessions. + +* MaxScale can not manage authentication that uses wildcard matching in hostnames in the mysql.user table of the backend database. The only wildcards that can be used are in IP address entries. + +* When users have different passwords based on the host from which they connect MaxScale is unable to determine which password it should use to connect to the backend database. This results in failed connections and unusable usernames in MaxScale. + +# Packaging + +Both RPM and Debian packages are available for MaxScale in addition to the tar based releases previously distributed we now provide + +* CentOS/RedHat 5 + +* CentOS/RedHat 6 + +* CentOS/RedHat 7 + +* Debian 6 + +* Debian 7 + +* Ubuntu 12.04 LTS + +* Ubuntu 13.10 + +* Ubuntu 14.04 LTS + +* Fedora 19 + +* Fedora 20 + +* OpenSuSE 13 + diff --git a/Documentation/filters/Query-Log-All-Filter.md b/Documentation/filters/Query-Log-All-Filter.md new file mode 100644 index 000000000..07067c0d0 --- /dev/null +++ b/Documentation/filters/Query-Log-All-Filter.md @@ -0,0 +1,79 @@ +# Query Log All Filter + +## Overview + +The Query Log All (QLA) filter is a filter module for MaxScale that is able to log all query content on a per client session basis. Logs are written in a csv format file that lists the time submitted and the SQL statement text. + +## Configuration + +The configuration block for the QLA filter requires the minimal filter options in it's section within the MaxScale.cnf file, stored in $MAXSCALE_HOME/etc/MaxScale.cnf. + + [MyLogFilter] + type=filter + module=qlafilter + +## Filter Options + +The QLA filter accepts one option value, this is the name that is used for the log files that are written. The file that is created appends the session number to the name given in the options entry. For example: + + options=/tmp/QueryLog + +would create log files /tmp/QueryLog.1 etc. + +Note, this is included for backward compatibility with the version of the QLA filter that was provided in the initial filters implementation preview in 0.7 of MaxScale. The filebase parameter can now be used and will take precedence over the filter option. + +## Filter Parameters + +The QLA filter accepts a number of optional parameters, these were introduced in the 1.0 release of MaxScale. + +### Filebase + +The basename of the output file created for each session. A session index is added to the filename for each file written. + + filebase=/tmp/SqlQueryLog + +The filebase may also be set as the filter, the mechanism to set the filebase via the filter option is superseded by the parameter. If both are set the parameter setting will be used and the filter option ignored. + +### Match + +An optional parameter that can be used to limit the queries that will be logged by the QLA filter. The parameter value is a regular expression that is used to match against the SQL text. Only SQL statements that matches the text passed as the value of this parameter will be logged. + + match=select.*from.*customer.*where + +All regular expressions are evaluated with the option to ignore the case of the text, therefore a match option of select will match both select, SELECT and any form of the word with upper or lowercase characters. + +### Exclude + +An optional parameter that can be used to limit the queries that will be logged by the QLA filter. The parameter value is a regular expression that is used to match against the SQL text. SQL statements that match the text passed as the value of this parameter will be excluded from the log output. + + exclude=where + +All regular expressions are evaluated with the option to ignore the case of the text, therefore an exclude option of select will exclude statements that contain both select, SELECT or any form of the word with upper or lowercase characters. + +### Source + +The optional source parameter defines an address that is used to match against the address from which the client connection to MaxScale originates. Only sessions that originate from this address will be logged. + + source=127.0.0.1 + +### User + +The optional user parameter defines a user name that is used to match against the user from which the client connection to MaxScale originates. Only sessions that are connected using this username are logged. + + user=john + +## Examples + +### Example 1 - Query without primary key + +Imagine you have observed an issue with a particular table and you want to determine if there are queries that are accessing that table but not using the primary key of the table. Let's assume the table name is PRODUCTS and the primary key is called PRODUCT_ID. Add a filter with the following definition: + + [ProductsSelectLogger] + type=filter + module=qlafilter + match=SELECT.*from.*PRODUCTS .* + exclude=WHERE.*PRODUCT_ID.* + filebase=/var/logs/qla/SelectProducts + +The result of then putting this filter into the service used by the application would be a log file of all select queries that mentioned the table but did not mention the PRODUCT_ID primary key in the predicates for the query. + \ No newline at end of file diff --git a/Documentation/filters/Regex-Filter.md b/Documentation/filters/Regex-Filter.md new file mode 100644 index 000000000..76ddb6fad --- /dev/null +++ b/Documentation/filters/Regex-Filter.md @@ -0,0 +1,72 @@ +Regex Filter + +# Overview + +The regex filter is a filter module for MaxScale that is able to rewrite query content using regular expression matches and text substitution. + +# Configuration + +The configuration block for the Regex filter requires the minimal filter options in it’s section within the MaxScale.cnf file, stored in $MAXSCALE_HOME/etc/MaxScale.cnf. + +[MyRegexFilter] + +type=filter + +module=regexfilter + +match=some string + +replace=replacement string + +## Filter Options + +The regex filter accepts the options ignorecase or case. These define if the pattern text should take the case of the string it is matching against into consideration or not. + +## Filter Parameters + +The Regex filter requires two mandatory parameters to be defined. + +### Match + +A parameter that can be used to match text in the SQL statement which should be replaced. + +match=TYPE[ ]*= + +If the filter option ignorecase is used all regular expressions are evaluated with the option to ignore the case of the text, therefore a match option of select will match both type, TYPE and any form of the word with upper or lowercase characters. + +### Replace + +The replace parameter defines the text that should replace the text in the SQL text which matches the match. + +replace=ENGINE = + +### Source + +The optional source parameter defines an address that is used to match against the address from which the client connection to MaxScale originates. Only sessions that originate from this address will have the match and replacement applied to them. + +source=127.0.0.1 + +### User + +The optional user parameter defines a user name that is used to match against the user from which the client connection to MaxScale originates. Only sessions that are connected using this username will have the match and replacement applied to them. + +user=john + +## Examples + +### Example 1 - Replace MySQL 5.1 create table syntax with that for later versions + +MySQL 5.1 used the parameter TYPE = to set the storage engine that should be used for a table. In later versions this changed to be ENGINE =. Imagine you have an application that you can not change for some reason, but you wish to migrate to a newer version of MySQL. The regexfilter can be used to transform the create table statments into the form that could be used by MySQL 5.5 + +[CreateTableFilter] + +type=filter + +module=regexfilter + +options=ignorecase + +match=TYPE[ ]*= + +replace=ENGINE= + diff --git a/Documentation/filters/Tee-Filter.md b/Documentation/filters/Tee-Filter.md new file mode 100644 index 000000000..be77e83b8 --- /dev/null +++ b/Documentation/filters/Tee-Filter.md @@ -0,0 +1,128 @@ +TEE Filter + +# Overview + +The tee filter is a filter module for MaxScale is a "plumbing" fitting in the MaxScale filter toolkit. It can be used in a filter pipeline of a service to make a copy of requests from the client and dispatch a copy of the request to another service within MaxScale. + +# Configuration + +The configuration block for the TEE filter requires the minimal filter parameters in it’s section within the MaxScale.cnf file, stored in $MAXSCALE_HOME/etc/MaxScale.cnf, that defines the filter to load and the service to send the duplicates to. + +[DataMartFilter] + +type=filter + +module=tee + +service=DataMart + +## Filter Options + +The tee filter does not support any filter options. + +## Filter Parameters + +The tee filter requires a mandatory parameter to define the service to replicate statements to and accepts a number of optional parameters. + +### Match + +An optional parameter that can be used to limit the queries that will be replicated by the tee filter. The parameter value is a regular expression that is used to match against the SQL text. Only SQL statements that matches the text passed as the value of this parameter will be sent to the service defined in the filter section. + +match=insert.*into.*order* + +All regular expressions are evaluated with the option to ignore the case of the text, therefore a match option of select will match both insert, INSERT and any form of the word with upper or lowercase characters. + +### Exclude + +An optional parameter that can be used to limit the queries that will be replicated by the tee filter. The parameter value is a regular expression that is used to match against the SQL text. SQL statements that match the text passed as the value of this parameter will be excluded from the replication stream. + +exclude=select + +All regular expressions are evaluated with the option to ignore the case of the text, therefore an exclude option of select will exclude statements that contain both select, SELECT or any form of the word with upper or lowercase characters. + +### Source + +The optional source parameter defines an address that is used to match against the address from which the client connection to MaxScale originates. Only sessions that originate from this address will be replicated. + +source=127.0.0.1 + +### User + +The optional user parameter defines a user name that is used to match against the user from which the client connection to MaxScale originates. Only sessions that are connected using this username are replicated. + +user=john + +## Examples + +### Example 1 - Replicate all inserts into the orders table + +Assume an order processing system that has a table called orders. You also have another database server, the datamart server, that requires all inserts into orders to be replicated to it. Deletes and updates are not however required. + +Set up a service in MaxScale, called Orders, to communicate with the order processing system with the tee filter applied to it. Also set up a service to talk the datamart server, using the DataMart service. The tee filter woudl have as it’s service entry the DataMart service, by adding a match parameter of "insert into orders" would then result in all requests being sent to the order processing system, and insert statements that include the orders table being additionally sent to the datamart server. + +[Orders] + +type=service + +router=readconnroute + +servers=server1, server2, server3, server4 + +user=massi + +passwd=6628C50E07CCE1F0392EDEEB9D1203F3 + +filters=ReplicateOrders + +[ReplicateOrders] + +type=filter + +module=tee + +service=DataMart + +match=insert[ ]*into[ ]*orders + +[DataMart] + +type=service + +router=readconnroute + +servers=datamartserver + +user=massi + +passwd=6628C50E07CCE1F0392EDEEB9D1203F3 + +filters=QLA_DataMart + +[QLA_DataMart] + +type=filter + +module=qlafilter + +options=/var/log/DataMart/InsertsLog + +[Orders Listener] + +type=listener + +service=Orders + +protocol=MySQLClient + +port=4011 + +[DataMart Listener] + +type=listener + +service=DataMart + +protocol=MySQLClient + +port=4012 + diff --git a/Documentation/filters/Top-N-Filter.md b/Documentation/filters/Top-N-Filter.md new file mode 100644 index 000000000..afd960d87 --- /dev/null +++ b/Documentation/filters/Top-N-Filter.md @@ -0,0 +1,182 @@ +Top Filter + +# Overview + +The top filter is a filter module for MaxScale that monitors every SQL statement that passes through the filter. It measures the duration of that statement, the time between the statement being sent and the first result being returned. The top N times are kept, along with the SQL text itself and a list sorted on the execution times of the query is written to a file upon closure of the client session. + +# Configuration + +The configuration block for the TOP filter requires the minimal filter options in it’s section within the MaxScale.cnf file, stored in $MAXSCALE_HOME/etc/MaxScale.cnf. + +[MyLogFilter] + +type=filter + +module=topfilter + +## Filter Options + +The top filter does not support any filter options currently. + +## Filter Parameters + +The top filter accepts a number of optional parameters. + +### Filebase + +The basename of the output file created for each session. A session index is added to the filename for each file written. + +filebase=/tmp/SqlQueryLog + +The filebase may also be set as the filter, the mechanism to set the filebase via the filter option is superseded by the parameter. If both are set the parameter setting will be used and the filter option ignored. + +### Count + +The number of SQL statements to store and report upon. + +count=30 + +The default vakue for the numebr of statements recorded is 10. + +### Match + +An optional parameter that can be used to limit the queries that will be logged by the top filter. The parameter value is a regular expression that is used to match against the SQL text. Only SQL statements that matches the text passed as the value of this parameter will be logged. + +match=select.*from.*customer.*where + +All regular expressions are evaluated with the option to ignore the case of the text, therefore a match option of select will match both select, SELECT and any form of the word with upper or lowercase characters. + +### Exclude + +An optional parameter that can be used to limit the queries that will be logged by the top filter. The parameter value is a regular expression that is used to match against the SQL text. SQL statements that match the text passed as the value of this parameter will be excluded from the log output. + +exclude=where + +All regular expressions are evaluated with the option to ignore the case of the text, therefore an exclude option of select will exclude statements that contain both where, WHERE or any form of the word with upper or lowercase characters. + +### Source + +The optional source parameter defines an address that is used to match against the address from which the client connection to MaxScale originates. Only sessions that originate from this address will be logged. + +source=127.0.0.1 + +### User + +The optional user parameter defines a user name that is used to match against the user from which the client connection to MaxScale originates. Only sessions that are connected using this username will result in results being gebnerated. + +user=john + +## Examples + +### Example 1 - Heavily Contended Table + +You have an order system and believe the updates of the PRODUCTS table is causing some performance issues for the rest of your application. You would like to know which of the many updates in your application is causing the issue. + +Add a filter with the following definition; + +[ProductsUpdateTop20] + +type=filter + +module=topfilter + +count=20 + +match=UPDATE.*PRODUCTS.*WHERE + +exclude=UPDATE.*PRODUCTS_STOCK.*WHERE + +filebase=/var/logs/top/ProductsUpdate + +Note the exclude entry, this is to prevent updates to the PRODUCTS_STOCK table from being included in the report. + +### Example 2 - One Application Server is Slow + +One of your applications servers is slower than the rest, you believe it is related to database access but you not not sure what is taking the time. + +Add a filter with the following definition; + +[SlowAppServer] + +type=filter + +module=topfilter + +count=20 + +source=192.168.0.32 + +filebase=/var/logs/top/SlowAppServer + +In order to produce a comparison with an unaffected application server you can also add a second filter as a control. + +[ControlAppServer] + +type=filter + +module=topfilter + +count=20 + +source=192.168.0.42 + +filebase=/var/logs/top/ControlAppServer + +In the router definition add both filters + +filters=SlowAppServer | ControlAppServer + +You will then have two sets of logs files written, one which profiles the top 20 queries of the slow application server and another that gives you the top 20 queries of your control application server. These two sets of files can then be compared to determine what if anythign is different between the two. + +# Output Report + +The following is an example report for a number of fictitious queries executed against the employees exaple database available for MySQL. + +-bash-4.1$ cat /var/logs/top/Employees-top-10.137 + +Top 10 longest running queries in session. + +========================================== + +Time (sec) | Query + +-----------+----------------------------------------------------------------- + + 22.985 | select sum(salary), year(from_date) from salaries s, (select distinct year(from_date) as y1 from salaries) y where (makedate(y.y1, 1) between s.from_date and s.to_date) group by y.y1 + + 5.304 | select d.dept_name as "Department", y.y1 as "Year", count(*) as "Count" from departments d, dept_emp de, (select distinct year(from_date) as y1 from dept_emp order by 1) y where d.dept_no = de.dept_no and (makedate(y.y1, 1) between de.from_date and de.to_date) group by y.y1, d.dept_name order by 1, 2 + + 2.896 | select year(now()) - year(birth_date) as age, gender, avg(salary) as "Average Salary" from employees e, salaries s where e.emp_no = s.emp_no and ("1988-08-01" between from_date AND to_date) group by year(now()) - year(birth_date), gender order by 1,2 + + 2.160 | select dept_name as "Department", sum(salary) / 12 as "Salary Bill" from employees e, departments d, dept_emp de, salaries s where e.emp_no = de.emp_no and de.dept_no = d.dept_no and ("1988-08-01" between de.from_date AND de.to_date) and ("1988-08-01" between s.from_date AND s.to_date) and s.emp_no = e.emp_no group by dept_name order by 1 + + 0.845 | select dept_name as "Department", avg(year(now()) - year(birth_date)) as "Average Age", gender from employees e, departments d, dept_emp de where e.emp_no = de.emp_no and de.dept_no = d.dept_no and ("1988-08-01" between from_date AND to_date) group by dept_name, gender + + 0.668 | select year(hire_date) as "Hired", d.dept_name, count(*) as "Count" from employees e, departments d, dept_emp de where de.emp_no = e.emp_no and de.dept_no = d.dept_no group by d.dept_name, year(hire_date) + + 0.249 | select moves.n_depts As "No. of Departments", count(moves.emp_no) as "No. of Employees" from (select de1.emp_no as emp_no, count(de1.emp_no) as n_depts from dept_emp de1 group by de1.emp_no) as moves group by moves.n_depts order by 1 + + 0.245 | select year(now()) - year(birth_date) as age, gender, count(*) as "Count" from employees group by year(now()) - year(birth_date), gender order by 1,2 + + 0.179 | select year(hire_date) as "Hired", count(*) as "Count" from employees group by year(hire_date) + + 0.160 | select year(hire_date) - year(birth_date) as "Age", count(*) as Count from employees group by year(hire_date) - year(birth_date) order by 1 + +-----------+----------------------------------------------------------------- + +Session started Wed Jun 18 18:41:03 2014 + +Connection from 127.0.0.1 + +Username massi + +Total of 24 statements executed. + +Total statement execution time 35.701 seconds + +Average statement execution time 1.488 seconds + +Total connection time 46.500 seconds + +-bash-4.1$ +