56 Commits

Author SHA1 Message Date
Markus Makela
7a8c307515 MXS-171: https://mariadb.atlassian.net/browse/MXS-171
Added option which allows the master server to be used for reads.
2015-06-24 19:23:43 +03:00
Markus Makela
21ad6fba2a Added option to disable slave recovery in readwritesplit. 2015-04-24 05:55:06 +03:00
Markus Makela
8d4f301545 Added the option to disable session command history to readwritesplit. 2015-04-23 20:12:04 +03:00
Markus Makela
7cfc1d3cc9 Chaged MaxScale default installation directory to /usr/local/mariadb/maxscale and MaxAdmin password to 'mariadb'. 2015-03-23 14:53:31 +02:00
Markus Makela
c937568624 Fixed a possible memory leak when session command history limits are used. 2015-03-05 20:46:55 +02:00
Markus Makela
2d08bc4eda Fix to bug 736: http://bugs.mariadb.com/show_bug.cgi?id=736
Added a cofigurable limit on the number of session commands the readwritesplit router stores.
2015-03-01 02:27:47 +02:00
Markus Makela
22f7137440 Fix to bug 600: http://bugs.mariadb.com/show_bug.cgi?id=600
readwritesplit now drops slave connections that fail a session command.
2015-02-20 06:38:38 +02:00
VilhoRaatikka
ff5fe23ce6 dcb.b:dcb_final_free:replaced ((dcb->flags & DCBF_CLONE)==0) with macro !DCB_IS_CLONE(dcb)
readwritesplit.h:Removed invalid macros which assumed that ROUTER_CLIENT_SES->rses_backend_ref always pointed to valid and used backend reference and thus included potential risk of NULL-pointer refernce.
mysql_backend.c and mysql_client.c:avoid executing CHK_PROTOCOL(p) after original DCB has been released the memory.
readwritesplit.c:Replaced RSES_CLEINT_DCB macro with a function which returns client DCB for a given router client session.
2014-12-23 16:10:27 +02:00
VilhoRaatikka
a4968f2521 Complete fix candidate for #645, http://bugs.skysql.com/show_bug.cgi?id=645 and #648, http://bugs.skysql.com/show_bug.cgi?id=648
tee.c:closeSession removed unnecessary dcb_free, router/service closes all backend DCBs and the client DCB, and client DCB is the one that was tried to free in closeSession.
readwritesplit.c:routeQuery now handles untyped and typed GWBUFs. Untyped means that read buffer may consist of incomplete and multiple MySQL packets. Typed buffer always consists of a single MySQL packet (which can be split to many buffers inside GWBUF).
Fixed Coverity cases #84840 and #84841
2014-12-14 11:26:15 +02:00
VilhoRaatikka
c9b679359f Made RWSplit to use slave selection criteria also during client session. Previously the criteria was use when slaves were selected for use of client session. 2014-11-17 13:40:18 +02:00
VilhoRaatikka
848c7aa0b8 Fixes to Coverity cases 72678 (retry), 72693 2014-10-31 11:38:57 +02:00
Markus Makela
b0683d9b4d Updated the licensing year from 2013 to 2013-2014 2014-10-01 14:37:12 +03:00
Markus Makela
f4e591e382 Changed 'SkySQL Gateway' to 'MariaDB Corporation MaxScale' 2014-09-30 13:15:03 +03:00
Markus Makela
c344231f80 Renamed all occurences of SkySQL to MariaDB Corporation 2014-09-30 13:02:10 +03:00
VilhoRaatikka
5ec1a83f3b Fix to #478, http://bugs.skysql.com/show_bug.cgi?id=478
Execution of session commands failed because session commands and normal sql statements were executed in a wrong order if backend was executing previous session command while new sql stmt was routed to that backend. There was a window where ordering went wrong.
It is possible that one normal sql stmt arrives while previous sescmds are still being executed.

Introduced a new member in backend_ref_t structure, bref_pending_cmd where new sql stmt pointer is stored in that case. When sescmds are executed completely, that command is automatically executed next.
2014-09-26 15:40:32 +03:00
MassimilianoPinto
8b5b4a17fe Merge from Z3
Merge from Z3
2014-09-11 12:51:16 +02:00
Mark Riddoch
27e0ccd641 Fix for bug 447 - Error log flooded with same warnings if there are no slaves present 2014-09-11 11:03:53 +01:00
VilhoRaatikka
1154b25bff Fixed typo which made use_sql_variables_in paramter ineffective. 2014-09-10 21:20:12 +03:00
VilhoRaatikka
025f920ddb Replaced RWSplit parameters write_ses_variables_to_all and read_ses_variables_from_slaves with
use_sql_variables_in=[master|all] (default all)
Modified MaxScale Configuration And Usage Scenarios-Z3.pdf and MaxScale_template.cnf accordingly.
Fixed typo in server/modules/routing/readwritesplit/test/rwsplit.sh
2014-09-10 11:32:53 +03:00
VilhoRaatikka
ba6ad3e240 Added description of new rwsplit variables, read_ses_variables_from_slaves and write_ses_variables_to_all to Configuration document.
Added description of new rwsplit variables, read_ses_variables_from_slaves and write_ses_variables_to_all to MaxScale_template.cnf
readwritesplit.h:Set default values for write_ses_variables_to_all=true, and read_ses_variables_from_slaves=false
2014-09-08 22:18:00 +03:00
VilhoRaatikka
92889ad216 config.c: Added configuration parameter processing for read_ses_variables_from_slaves and for write_ses_variables_to_all . The values are read from config file, qualified and stored to service. Values are loaded when instance is created. This is limitation in current implementation and will change so that configuration is dynamically changeable. 2014-09-07 23:48:16 +03:00
VilhoRaatikka
5e5c59d322 Merge branch 'MAX-237' into Z3 2014-09-03 17:50:56 +03:00
Markus Makela
a17e584adb Merge remote-tracking branch 'origin/Z3' into MAX-237
Conflicts:
	query_classifier/query_classifier.cc
	query_classifier/query_classifier.h
	server/modules/routing/readwritesplit/readwritesplit.c
2014-09-01 11:05:10 +03:00
Markus Makela
7629c455a6 partial implementation 2014-08-30 08:27:05 +03:00
VilhoRaatikka
531d8d7b47 query_classifier.cc: added detection for CREATE TEMPORARY TABLE and setting a new query type QUERY_TYPE_CREATE_TMP_TABLE for it.
query_classifier.h: added QUERY_TYPE_CREATE_TMP_TABLE and QUERY_TYPE_READ_TMP_TABLE for use of temporary table support.
hashtable.c:Added variant of hashtable which is 'flat', that is, stored to existing memory instead of allocating memory as a part of the call. Existing function declarations don't change but added hashtable_alloc_flat for the purpose. Both hashtable_alloc and hashtable_alloc_flat now call the real allocation function, hashtable_alloc_real. hashtable_free only frees memory which is allocated in hashtable_alloc_real.
hashtable.h: added a flag to HASHTABLE struct to indicate whether hashtable owns its memory or not.
readwritesplit.h: Added RSES_PROP_TYPE_TMPTABLES property type to be used for keeping the hashtable for tablenames.
readwritesplit.c: Added comments about temporary table support implementation.
2014-08-29 10:08:48 +03:00
VilhoRaatikka
65b121889b config.h: Added enumerated types MAX_RLAG_AVAILABLE and MAX_RLAG_UNDEFINED to be used instead of magic numbers as max_slave_replication_lag values.
readwritesplit.h: Added TARGET_RLAG_MAX to the list of hint targets. It is used in case of hint max_slave_replication_lag=<value>
readwritesplit.c: get_dcb is called when readwritesplit router chooses target backend for a query. Backend is selected from those the router has connection to. get_dcb now takes additional parameters, name and max_rlag. Name is used if there is a hint route to server <server name>. max_rlag is used for hint max_slave_replication_lag=<value> or if it doesn't exist, then for configure parameter max_slave_replication_lag=<configured value> or if both are absent, ignored.
2014-08-01 16:33:29 +03:00
VilhoRaatikka
4f3d746f4e MAX-157. Added support for hints in rwsplit router.
buffer.c: added memory release for hint of a GWBUF
hint.c: added bool hint_exists()
hint.h: added placeholder for hint type HINT_ROUTE_TO_ALL which doesn't have implementation yet.
filter/Makefile: fixed dependency issue
hintparser.c: added const char* token_get_keyword(), hint_parser:added NULL check, hint_next_token: fixed off-by-one bug
readwritesplit.h: added bitfield for hints' use, which includes route targets and flag for case where user hinted to route to named backend server.
readwritesplit.c: added function route_target_t get_route_target() for resolving route target based on 1) query type (from query_classifier) 2) transaction state (active/not) and 3) hints. Modified get_dcb, which is called in routeQuery to provide pointer to correct backend DCB. Now get_dcb also takes server unique name as a parameter if such a hint was found. for hints' use, which includes  enter the commit message for your changes.
2014-07-31 23:40:02 +03:00
Mark Riddoch
eaaac58b44 Addition of server weighting to the read/write splitter
Tidy up the output of show server and show servers so that columsn align
2014-07-01 11:06:28 +01:00
VilhoRaatikka
57a977bdc0 Merge branch 'develop' of https://github.com/skysql/MaxScale into develop 2014-06-30 22:38:15 +03:00
VilhoRaatikka
b875936a21 MAX-94, Added configuration parameter, max_slave_replication_lag=<longest allowed replication lag in seconds> to router section.
Parameter can be changed runtime but it requires reloading of config.
2014-06-30 22:35:27 +03:00
VilhoRaatikka
6d672cb967 Added status bit BREF_QUERY_ACTIVE to indicate if a query, other than session command, sent to backend for execution. The flag is cleared when the first packet belonging to the response arrives.
The flag is part of the active operation counting, which is utilized in load balancing. The active operation count per backend is used by default as criteria when router chooses to which backend a query should be routed.
2014-06-30 13:44:34 +03:00
VilhoRaatikka
fcf67716fd Added mechanism for choosing slave for a query based on the current load in all connected slaves. Counting operations is not correctly done here. Reading values and choosing accordingly is done.
Fixed several things in session command reply processing.
2014-06-29 22:21:30 +03:00
VilhoRaatikka
7ff14e23a5 Support for prepared statement, namely support for following comands : COM_STMT_PREPARE, COM_STMT_EXECUTE, COM_STMT_SEND_LONG_DATA, COM_STMT_RESET, COM_STMT_CLOSE, SQLCOM_PREPARE, SQLCOM_EXECUTE, SQLCOM_DEALLOCATE_PREPARE (DEALLOCATE/DROP PREPARE stmt).
All prepare commands are executed in every backend server currently connected.

All executes are routed to master. If stmt type was recorded in prepare phase in rwsplit router, read-only stmts could be routed to slaves.

COM_STMT_PREPARE gets arbitrary number of response packets from backend database. Since statements are prepared in every backend server and only one multi-packet response can be replied to client, redundant multi-packet responses are discarded. This is done in router. Mechanisms from session command handling are utilized with little changes: router must identify when response consists of multiple packets so that it knows to calculate the number of packets in response and that it is able to discard correct number of packets.

Information to the reply-handling router is provided by backend protocol, which includes a ordered list of commands of commands sent to protocol-owning backend server. A command is stored to protocol struct in mysql_backend.c:gw_MySQLWrite_backend if the statement buffer's type has GWBUF_TYPE_SINGLE_STMT set in mysql_client.c:route_by_statement. GWBUF_TYPE_SINGLE_STATEMENT indicates that there is single statement in the buffer, as opposite to Read Connection router, which accepts streaming input from client.
2014-06-25 17:15:46 +03:00
VilhoRaatikka
4f03c775e5 sescmd_cursor_reset didn't reset cursor's property pointer which made reset ineffective. 2014-06-16 13:40:45 +03:00
VilhoRaatikka
dfc9141a38 Whenever monitored server's state changes, a callback, router_handle_state_switch is called for each DCB in MaxScale session. The DCB referring to the server in question will be passed as parameter to backend DCB's hangup function.
The logic that solves the situation is not in place yet.
2014-06-13 13:30:50 +03:00
VilhoRaatikka
15ff1fd26a Fixed many error handling issues regading to timing and multiple threads.
Added flags to those backend references which have sent something to backend which causes the backend to send results or reply back. Didn't add removal of the flag since there's currently no way to tell whether response from backend contains anything else than session command reply - which aren't counted when BREF_WAITING_RESULT is set and cleared.
2014-06-12 23:22:51 +03:00
VilhoRaatikka
ad744962b2 Merge branch 'develop' into MAX-11
Conflicts:
	server/include/server.h
	server/modules/monitor/mysql_mon.c
	server/modules/protocol/mysql_client.c
	server/modules/routing/readwritesplit/readwritesplit.c
	utils/skygw_debug.h
2014-06-07 00:50:08 +03:00
VilhoRaatikka
7e6cb7afc2 Snapshot of failure tolerance changes.
Added a lot of logging to error, trace and message logs which should help the user to handle errors which can't be automatically resolved, like attempt to use nonexisting database.
2014-06-06 23:32:04 +03:00
VilhoRaatikka
46495bb59e Bug #443, http://bugs.skysql.com/show_bug.cgi?id=443
monitors used mysql_ping without MYSQL_OPT_READ_TIMEOUT which caused read to block. Fixed in mysql and galera monitor.
Added log writing per each status change of each server and repeatedly if server is not running.

Removed SERVER_IS_JOINED checks from rwsplit router.
2014-06-03 19:39:50 +03:00
VilhoRaatikka
c28052d5a4 Manual merge from master to develop. Incliding fix for bug #425 http://bugs.skysql.com/show_bug.cgi?id=425 2014-05-21 15:41:39 +03:00
VilhoRaatikka
c5ec3cbc57 Added new criteria for slave selection: LEAST_ROUTER_CONNECTIONS which causes the servers having least connections from same router to become chosen.
Added function pointer array which is indexed by enumerated selection criterias.
Selection criteria can now be switched by modifying config file and reloading it.
2014-05-12 17:03:22 +03:00
VilhoRaatikka
b7ae1c1784 Added router_option slave_selection_criteria=[LEAST_GLOBAL_CONNECTIONS(default)|LEAST_BEHIND_MASTER] to readwritesplit section.
The option is reloadable in the same way as max_slave_connections parameter. Option value determines which sort function is called before to sort backend server pointers before selecting them and establishing connections to backends.
2014-05-11 23:59:16 +03:00
VilhoRaatikka
9d1aa06376 Added default value for max_slave_connections=1. It is set when router instance is created. Default value is overwritten if configuration parameter is found.
Added missing makefile to server/modules/routing/test . It is needed for system testing. Prior this fix, 'make testall' failed because the makefile was missing.
2014-05-10 23:27:28 +03:00
VilhoRaatikka
a1361d9c9e Related to MAX-95, added code which makes it possible to change max_slave_connections parameter value during run time of MaxScale. The change will be effective in rwsplit sessions created after the configuration modification and reload. Note that in order to config modification to be effective, the user must connect the debug interface of MaxScale (telnet) and execute 'reload config'.
In practice, 'reload config' reads config file and updates config parameters of rwsplit router service. In addition to that there is a version number indicating what generation of configuration service holds. When router instance is created (when MaxScale is started) service's config version is copied to router intance. Whenever new client connection (rwsplit session) is started, router instance's config version is checked against that of service's. If versions differ, service's config data is copied to router instance. New session will be started with router instance's config values.
2014-05-02 16:35:40 +03:00
VilhoRaatikka
c927057b5c Fixed two bugs of which one was older.
1. in query_classifier.cc autocommit_enabled, and transaction_active variables maintained their values across different sessions. Now those values are stored in each router_client_ses object.
2. As a part of implementation of MAX-95 session variables were added to BACKEND struct which is shared with all sessions using the SERVICE which the particular BACKEND serves. Now each router_client_ses object has a backend reference struct which includes pointer to BACKEND, DCB and to session command cursor.

Added test - set_autocommit_disabled.sql, test_after_autocommit_disabled.sql - to check that session variable is discarded when session where it belongs terminates.
2014-04-28 23:33:49 +03:00
VilhoRaatikka
28bc3509cc Added new parameter RW Split Router.max_slave_connections=[<int>|<int>%] which specifies the maximum number of slaves which read/write split router connects in each routing session.
Parameter it read from config file to CONFIG_CONTEXT's parameter list. It is qualified in service.c:service_set_slave_conn_limit and if qualified, the qualified integer value and the value type are copied to the CONFIG_PARAMETER structure.

This CONFIG_PARAMETER struct is cloned (=copied to different memory area) and linked to RW Split SERVICE struct.

When RW Split router_instance is created in readwritesplit.c:createInstance, the value is copied to (new) rwsplit_config_t structure from SERVICE's parameter list.

When new routing session is created in readwritesplit.c:newSession, the rwsplit_config_t structure is copied to ROUTER_CLIENT_SES struct and the actual max_nslaves value is calculated from the config value (if percentage is used).

Tests and many error handling branches are missing but functionality seems to be working.
2014-04-24 22:05:26 +03:00
VilhoRaatikka
04313caf82 Initial implementation of the support for multiple slaves in read/write split router session. Omits configuration changes. 2014-04-23 14:55:04 +03:00
VilhoRaatikka
fcf7d37114 Added support for router option 'synced' in read write split router. If router option is specified, backend server must be in 'joined' state to be eligible backend candidate. 2014-03-19 22:29:02 +02:00
VilhoRaatikka
a3f7eebdc9 Extended session command support to cover COM_CHANGE_USER, and COM_INIT_DB.
This implementation doesn't guarantee execution order between session commands and queries
if other backend server lags behind in session command execution.

In poll.c : moved processing of EPOLLERR and EPOLLHUP after processing of EPOLLIN and EPOLLOUT.
This ensures that COM_QUIT messages are read and routed forward before signals arrive (from local client/backend).
2014-03-14 13:25:37 +02:00
VilhoRaatikka
cb6a976555 Router has now capability value which currently tells whether router session expects stream or individual, complete statements. With read con
nection router stream is used and with read/write split router individual statements are passed to router.
Added new function to ROUTER_OBJECT : uint8_t (*getCapabilities)(ROUTER *instance, void* router_session); which is implemented in every route
r.

Added support for multi-statement packets in rwsplit router. In other words, if network packet includes multiple mysql statements, they are separated and passed to router one by one.

Multi-packet statements (those which exceeds network packet boundaries) are _not_ supported yet.
2014-03-11 23:12:11 +02:00