Commit Graph

377 Commits

Author SHA1 Message Date
57a977bdc0 Merge branch 'develop' of https://github.com/skysql/MaxScale into develop 2014-06-30 22:38:15 +03:00
b875936a21 MAX-94, Added configuration parameter, max_slave_replication_lag=<longest allowed replication lag in seconds> to router section.
Parameter can be changed runtime but it requires reloading of config.
2014-06-30 22:35:27 +03:00
0b89245a4b Merge branch 'develop' into MAX-167
Conflicts:
	server/modules/monitor/mysql_mon.c
	server/modules/routing/readwritesplit/readwritesplit.c
2014-06-30 14:17:00 +03:00
6d672cb967 Added status bit BREF_QUERY_ACTIVE to indicate if a query, other than session command, sent to backend for execution. The flag is cleared when the first packet belonging to the response arrives.
The flag is part of the active operation counting, which is utilized in load balancing. The active operation count per backend is used by default as criteria when router chooses to which backend a query should be routed.
2014-06-30 13:44:34 +03:00
862d504a74 Enabled set and clear the flag BREF_WAITING_RESULT. It is actually a counter which is modified with atomic operations only. Setting and clearing BREF_WAITING_RESULT also includes atomic increment and decrement of corresponding backend server's current operations counter, backend_server->stats.n_current_ops. 2014-06-30 11:00:04 +03:00
aa828e8751 Added missing comment to get_root_master
Added missing comment to get_root_master
2014-06-30 09:00:43 +02:00
fcf67716fd Added mechanism for choosing slave for a query based on the current load in all connected slaves. Counting operations is not correctly done here. Reading values and choosing accordingly is done.
Fixed several things in session command reply processing.
2014-06-29 22:21:30 +03:00
5cbc7b9f7d root server IN_MAINT returns NULL to caller
the replication tree returns NULL if the root server is maintenance mode
2014-06-27 17:55:46 +02:00
f246219beb Fix compile errors and warnings
Fix compile errors and warnings
2014-06-26 18:20:33 +02:00
1dbb4b1780 removed duplicated routine code
removed duplicated routine code
2014-06-26 17:12:08 +02:00
1a33277f72 Merge from develop
Merge from develop
2014-06-26 17:08:21 +02:00
dbf12f6183 Added new rules for Master selection
Added new rules for Master selection using replication tree
2014-06-25 18:38:33 +02:00
1d1da398d4 Merge branch 'develop' into MAX-99
Conflicts:
	server/modules/routing/readwritesplit/readwritesplit.c
2014-06-25 17:50:29 +03:00
7ff14e23a5 Support for prepared statement, namely support for following comands : COM_STMT_PREPARE, COM_STMT_EXECUTE, COM_STMT_SEND_LONG_DATA, COM_STMT_RESET, COM_STMT_CLOSE, SQLCOM_PREPARE, SQLCOM_EXECUTE, SQLCOM_DEALLOCATE_PREPARE (DEALLOCATE/DROP PREPARE stmt).
All prepare commands are executed in every backend server currently connected.

All executes are routed to master. If stmt type was recorded in prepare phase in rwsplit router, read-only stmts could be routed to slaves.

COM_STMT_PREPARE gets arbitrary number of response packets from backend database. Since statements are prepared in every backend server and only one multi-packet response can be replied to client, redundant multi-packet responses are discarded. This is done in router. Mechanisms from session command handling are utilized with little changes: router must identify when response consists of multiple packets so that it knows to calculate the number of packets in response and that it is able to discard correct number of packets.

Information to the reply-handling router is provided by backend protocol, which includes a ordered list of commands of commands sent to protocol-owning backend server. A command is stored to protocol struct in mysql_backend.c:gw_MySQLWrite_backend if the statement buffer's type has GWBUF_TYPE_SINGLE_STMT set in mysql_client.c:route_by_statement. GWBUF_TYPE_SINGLE_STATEMENT indicates that there is single statement in the buffer, as opposite to Read Connection router, which accepts streaming input from client.
2014-06-25 17:15:46 +03:00
4d257897d3 Merge branch 'develop' of github.com:skysql/MaxScale into develop
Conflicts:
	server/core/session.c
	server/modules/protocol/mysql_backend.c
	server/modules/routing/readwritesplit/readwritesplit.c

Resolved.

Addition of user in topfilter report and general tidyup
2014-06-18 17:45:57 +01:00
e7fa80a591 Code clean up 2014-06-17 16:15:19 +03:00
4f03c775e5 sescmd_cursor_reset didn't reset cursor's property pointer which made reset ineffective. 2014-06-16 13:40:45 +03:00
0200ca4e24 Some things weren't switched off from product build. 2014-06-16 00:31:14 +03:00
5bcae64538 When protocol closes DCB it calls dcb_close instead of dcb->func.close. dcb_close then calls dcb->func.close. This is now changed to all protocols and routers.
Rwsplit handles ERRACT_NEW_CONNECTION by clearing backend reference, removing callbacks and associating backend reference with new backend server. If it succeeds and the router session can continue, handleError returns true. Otherwise false. When ever false is returned it means that session must be closed.

Rwsplit now tolerates backend failures in a way that it searches new backends when monitor, backend, or client operation fails due to backend failure.
2014-06-15 23:44:07 +03:00
09d20d1059 Modified dcb_call_foreach so that it doesn't hold spinlock (dcbspin) continuously because it caused deadlock when new slaves were tried to connect with.
Fixed a few smallish things.
2014-06-13 14:52:04 +03:00
dfc9141a38 Whenever monitored server's state changes, a callback, router_handle_state_switch is called for each DCB in MaxScale session. The DCB referring to the server in question will be passed as parameter to backend DCB's hangup function.
The logic that solves the situation is not in place yet.
2014-06-13 13:30:50 +03:00
9eda859724 Added callback for rwsplit router which traverses through every rwsplit router session and for each rses, all backend references. Each bref is checked whether it is connected to non responsive server and if it is flagged to be waiting for response from the non-responsive backend. For matching ones, backend protocol's hangup function is called. 2014-06-13 08:01:26 +03:00
15ff1fd26a Fixed many error handling issues regading to timing and multiple threads.
Added flags to those backend references which have sent something to backend which causes the backend to send results or reply back. Didn't add removal of the flag since there's currently no way to tell whether response from backend contains anything else than session command reply - which aren't counted when BREF_WAITING_RESULT is set and cleared.
2014-06-12 23:22:51 +03:00
77e1426dbf Initial upstream fitlering implementation.
Test filter with up and down stream filtering. Keeps top N queries
and prints a report of these queries on session close.
2014-06-09 21:13:28 +01:00
889bdd4f8c In dcb.c:dcb_close DCB is removed either before or after the call dcb->func.close. Since mysql backend protocol sends COM_QUIT and thus, writes to backend DCB, it is kept in DCB_STATE_POLLING until the write is completed.
dcb.h: define ERRHAND temporarily since changes are still behind that macro
Defined two error handling actions in router.h: ERRACT_NEW_COMMECTION and ERRACT_REPLY_CLIENT.
Failed database is logged at expanding frequence to error and to message log due changes in mysql_mon.c. Added two new members in MONITOR_SERVERS: mon_err_count, and mon_prev_status so that each backend can be treated individually.

Error handling: if mysql_backend.c:dcb_read fails, router's handleError is called instead of closing session.
	If mysql_client.c:SESSION_ROUTE_QUERY fails router's handleError is called instead of sending error to client.

	readwritesplit.c:select_connect_backend_servers is modified so that in can be called during active router session. When called, it attempts to find one master and maximum number of configured slaves in correct state if necessary.
	When handleError needs to replace failed unit it now calls select_connect_backend_servers.
2014-06-08 19:36:12 +03:00
ad744962b2 Merge branch 'develop' into MAX-11
Conflicts:
	server/include/server.h
	server/modules/monitor/mysql_mon.c
	server/modules/protocol/mysql_client.c
	server/modules/routing/readwritesplit/readwritesplit.c
	utils/skygw_debug.h
2014-06-07 00:50:08 +03:00
7e6cb7afc2 Snapshot of failure tolerance changes.
Added a lot of logging to error, trace and message logs which should help the user to handle errors which can't be automatically resolved, like attempt to use nonexisting database.
2014-06-06 23:32:04 +03:00
7e863657db Removed references to SERVER_IS_JOINED status from readwritesplit.c because rwsplit for Galera doesn't use that status at all. 2014-06-04 14:49:05 +03:00
46495bb59e Bug #443, http://bugs.skysql.com/show_bug.cgi?id=443
monitors used mysql_ping without MYSQL_OPT_READ_TIMEOUT which caused read to block. Fixed in mysql and galera monitor.
Added log writing per each status change of each server and repeatedly if server is not running.

Removed SERVER_IS_JOINED checks from rwsplit router.
2014-06-03 19:39:50 +03:00
1245fba35b Addition of the module info structure to allow module information to
be extracted from the modules.

This gives a way to verify the API that the module provides as well
as the version of that API. The hope is that this will make it possible
for MaxScale to detect out of date plugins and either adapt to use them
or reject loading them.

Also added the ability to set a release state on a per module basis.
This allows for production ready and non-production ready plugins to
be identified.
2014-06-02 17:10:05 +01:00
1a2b8e5475 Router spinlock wasn't used to protect routing. As a consequence router could have been closed in thread #1 while thread #2 was in a middle of execution of router code. Solved by holding router lock so that it covered whole routing operation. 2014-05-28 18:29:02 +03:00
c28052d5a4 Manual merge from master to develop. Incliding fix for bug #425 http://bugs.skysql.com/show_bug.cgi?id=425 2014-05-21 15:41:39 +03:00
c5ec3cbc57 Added new criteria for slave selection: LEAST_ROUTER_CONNECTIONS which causes the servers having least connections from same router to become chosen.
Added function pointer array which is indexed by enumerated selection criterias.
Selection criteria can now be switched by modifying config file and reloading it.
2014-05-12 17:03:22 +03:00
b7ae1c1784 Added router_option slave_selection_criteria=[LEAST_GLOBAL_CONNECTIONS(default)|LEAST_BEHIND_MASTER] to readwritesplit section.
The option is reloadable in the same way as max_slave_connections parameter. Option value determines which sort function is called before to sort backend server pointers before selecting them and establishing connections to backends.
2014-05-11 23:59:16 +03:00
b8618ee7cf Backend, and server connection counters were updated erroneously. As a consequent, backend selection worked wrong and negative connection count values were possible. 2014-05-11 17:25:56 +03:00
9d1aa06376 Added default value for max_slave_connections=1. It is set when router instance is created. Default value is overwritten if configuration parameter is found.
Added missing makefile to server/modules/routing/test . It is needed for system testing. Prior this fix, 'make testall' failed because the makefile was missing.
2014-05-10 23:27:28 +03:00
ffce479780 backend connection counter was decreased in freeSession even for those backends which weren't connected. This resulted negative counts and affected on selection algorithm. 2014-05-10 22:21:30 +03:00
8be4aba223 Added new state to SESSION: SESSION_STATE_STOPPING, which is set in protocol module before calling closeSession (router). THe new state tells that session is closing and DCBs included may not be polling anymore.
Fixed some crash scenarios.
2014-05-08 23:17:35 +03:00
a1361d9c9e Related to MAX-95, added code which makes it possible to change max_slave_connections parameter value during run time of MaxScale. The change will be effective in rwsplit sessions created after the configuration modification and reload. Note that in order to config modification to be effective, the user must connect the debug interface of MaxScale (telnet) and execute 'reload config'.
In practice, 'reload config' reads config file and updates config parameters of rwsplit router service. In addition to that there is a version number indicating what generation of configuration service holds. When router instance is created (when MaxScale is started) service's config version is copied to router intance. Whenever new client connection (rwsplit session) is started, router instance's config version is checked against that of service's. If versions differ, service's config data is copied to router instance. New session will be started with router instance's config values.
2014-05-02 16:35:40 +03:00
b5e9428ff7 log_manager.cc fixed memory leak, block buffer mutex names weren't freed.
query_classifier.cc use of uninitialized value in skygw_stmt_causes_implicit_commit
config.c crashed if module load failed, use of unitialized value
load_utils.c pretty-printed error
service.c use of uninitialized value in service_add_qualified_param
modules.h function prototype
readwritesplit.c memory leaks
2014-04-29 14:50:09 +03:00
c927057b5c Fixed two bugs of which one was older.
1. in query_classifier.cc autocommit_enabled, and transaction_active variables maintained their values across different sessions. Now those values are stored in each router_client_ses object.
2. As a part of implementation of MAX-95 session variables were added to BACKEND struct which is shared with all sessions using the SERVICE which the particular BACKEND serves. Now each router_client_ses object has a backend reference struct which includes pointer to BACKEND, DCB and to session command cursor.

Added test - set_autocommit_disabled.sql, test_after_autocommit_disabled.sql - to check that session variable is discarded when session where it belongs terminates.
2014-04-28 23:33:49 +03:00
28bc3509cc Added new parameter RW Split Router.max_slave_connections=[<int>|<int>%] which specifies the maximum number of slaves which read/write split router connects in each routing session.
Parameter it read from config file to CONFIG_CONTEXT's parameter list. It is qualified in service.c:service_set_slave_conn_limit and if qualified, the qualified integer value and the value type are copied to the CONFIG_PARAMETER structure.

This CONFIG_PARAMETER struct is cloned (=copied to different memory area) and linked to RW Split SERVICE struct.

When RW Split router_instance is created in readwritesplit.c:createInstance, the value is copied to (new) rwsplit_config_t structure from SERVICE's parameter list.

When new routing session is created in readwritesplit.c:newSession, the rwsplit_config_t structure is copied to ROUTER_CLIENT_SES struct and the actual max_nslaves value is calculated from the config value (if percentage is used).

Tests and many error handling branches are missing but functionality seems to be working.
2014-04-24 22:05:26 +03:00
04313caf82 Initial implementation of the support for multiple slaves in read/write split router session. Omits configuration changes. 2014-04-23 14:55:04 +03:00
bbf1bc2afa Fix for bug #418
Increased skygw_query_type_t to 16 bits, and corrected the way how those bit fields are checked.
Added tests for cases where autocommit is disabled and corrected old tests.
2014-04-10 17:09:42 +03:00
2c17dc3edf Bug #418, added functions to query classifier to detect if SET autocommit is called.
Note: this compiles but doesn't work yet properly.
2014-04-09 23:43:03 +03:00
7c3a354fd8 Query classifier ignored implicit commits in cases of write commands. Fixed it.
Added more tests for transaction support. Mostly different cases where some command triggers implicit commit in the middle of transaction.
2014-03-28 00:16:18 +02:00
d660d3590d Cannot compile in readwritesplit.c 2014-03-20 20:30:03 +02:00
288ca68677 Changed variable type which includes information of query type returned by query classifier.
As a consequence, if autocommit is enabled, active transaction(s) are implicitly committed and MaxScale detects that implicit commit.
2014-03-20 12:15:57 +02:00
fcf7d37114 Added support for router option 'synced' in read write split router. If router option is specified, backend server must be in 'joined' state to be eligible backend candidate. 2014-03-19 22:29:02 +02:00
d6a9a5c1d0 MAX-10, Transaction support for MaxScale.
Naive implementation, which routes all statements to master between BEGIN|START TRANSACTION <options> and ROLLBACK|COMMIT
2014-03-18 23:41:32 +02:00