Commit Graph

489 Commits

Author SHA1 Message Date
dbf12f6183 Added new rules for Master selection
Added new rules for Master selection using replication tree
2014-06-25 18:38:33 +02:00
1d1da398d4 Merge branch 'develop' into MAX-99
Conflicts:
	server/modules/routing/readwritesplit/readwritesplit.c
2014-06-25 17:50:29 +03:00
7ff14e23a5 Support for prepared statement, namely support for following comands : COM_STMT_PREPARE, COM_STMT_EXECUTE, COM_STMT_SEND_LONG_DATA, COM_STMT_RESET, COM_STMT_CLOSE, SQLCOM_PREPARE, SQLCOM_EXECUTE, SQLCOM_DEALLOCATE_PREPARE (DEALLOCATE/DROP PREPARE stmt).
All prepare commands are executed in every backend server currently connected.

All executes are routed to master. If stmt type was recorded in prepare phase in rwsplit router, read-only stmts could be routed to slaves.

COM_STMT_PREPARE gets arbitrary number of response packets from backend database. Since statements are prepared in every backend server and only one multi-packet response can be replied to client, redundant multi-packet responses are discarded. This is done in router. Mechanisms from session command handling are utilized with little changes: router must identify when response consists of multiple packets so that it knows to calculate the number of packets in response and that it is able to discard correct number of packets.

Information to the reply-handling router is provided by backend protocol, which includes a ordered list of commands of commands sent to protocol-owning backend server. A command is stored to protocol struct in mysql_backend.c:gw_MySQLWrite_backend if the statement buffer's type has GWBUF_TYPE_SINGLE_STMT set in mysql_client.c:route_by_statement. GWBUF_TYPE_SINGLE_STATEMENT indicates that there is single statement in the buffer, as opposite to Read Connection router, which accepts streaming input from client.
2014-06-25 17:15:46 +03:00
4d257897d3 Merge branch 'develop' of github.com:skysql/MaxScale into develop
Conflicts:
	server/core/session.c
	server/modules/protocol/mysql_backend.c
	server/modules/routing/readwritesplit/readwritesplit.c

Resolved.

Addition of user in topfilter report and general tidyup
2014-06-18 17:45:57 +01:00
e7fa80a591 Code clean up 2014-06-17 16:15:19 +03:00
4f03c775e5 sescmd_cursor_reset didn't reset cursor's property pointer which made reset ineffective. 2014-06-16 13:40:45 +03:00
0200ca4e24 Some things weren't switched off from product build. 2014-06-16 00:31:14 +03:00
5bcae64538 When protocol closes DCB it calls dcb_close instead of dcb->func.close. dcb_close then calls dcb->func.close. This is now changed to all protocols and routers.
Rwsplit handles ERRACT_NEW_CONNECTION by clearing backend reference, removing callbacks and associating backend reference with new backend server. If it succeeds and the router session can continue, handleError returns true. Otherwise false. When ever false is returned it means that session must be closed.

Rwsplit now tolerates backend failures in a way that it searches new backends when monitor, backend, or client operation fails due to backend failure.
2014-06-15 23:44:07 +03:00
8103ac6052 Merge branch 'develop' into MAX-65 2014-06-14 12:54:08 +01:00
09d20d1059 Modified dcb_call_foreach so that it doesn't hold spinlock (dcbspin) continuously because it caused deadlock when new slaves were tried to connect with.
Fixed a few smallish things.
2014-06-13 14:52:04 +03:00
dfc9141a38 Whenever monitored server's state changes, a callback, router_handle_state_switch is called for each DCB in MaxScale session. The DCB referring to the server in question will be passed as parameter to backend DCB's hangup function.
The logic that solves the situation is not in place yet.
2014-06-13 13:30:50 +03:00
9eda859724 Added callback for rwsplit router which traverses through every rwsplit router session and for each rses, all backend references. Each bref is checked whether it is connected to non responsive server and if it is flagged to be waiting for response from the non-responsive backend. For matching ones, backend protocol's hangup function is called. 2014-06-13 08:01:26 +03:00
15ff1fd26a Fixed many error handling issues regading to timing and multiple threads.
Added flags to those backend references which have sent something to backend which causes the backend to send results or reply back. Didn't add removal of the flag since there's currently no way to tell whether response from backend contains anything else than session command reply - which aren't counted when BREF_WAITING_RESULT is set and cleared.
2014-06-12 23:22:51 +03:00
e95b6cc0d9 dcb.c, gw_utils.c, mysql_server_protocol.h, mysql_client.c : Replaced gw_read_gwbuff with dcb_read in mysql_client.c:gw_read_client_event.
rwsplit.sh, test_sescmd.sql : Added test case for session commands.
2014-06-12 19:02:47 +03:00
02330093cc Added test for session commands. 2014-06-11 15:33:42 +03:00
77e1426dbf Initial upstream fitlering implementation.
Test filter with up and down stream filtering. Keeps top N queries
and prints a report of these queries on session close.
2014-06-09 21:13:28 +01:00
889bdd4f8c In dcb.c:dcb_close DCB is removed either before or after the call dcb->func.close. Since mysql backend protocol sends COM_QUIT and thus, writes to backend DCB, it is kept in DCB_STATE_POLLING until the write is completed.
dcb.h: define ERRHAND temporarily since changes are still behind that macro
Defined two error handling actions in router.h: ERRACT_NEW_COMMECTION and ERRACT_REPLY_CLIENT.
Failed database is logged at expanding frequence to error and to message log due changes in mysql_mon.c. Added two new members in MONITOR_SERVERS: mon_err_count, and mon_prev_status so that each backend can be treated individually.

Error handling: if mysql_backend.c:dcb_read fails, router's handleError is called instead of closing session.
	If mysql_client.c:SESSION_ROUTE_QUERY fails router's handleError is called instead of sending error to client.

	readwritesplit.c:select_connect_backend_servers is modified so that in can be called during active router session. When called, it attempts to find one master and maximum number of configured slaves in correct state if necessary.
	When handleError needs to replace failed unit it now calls select_connect_backend_servers.
2014-06-08 19:36:12 +03:00
ad744962b2 Merge branch 'develop' into MAX-11
Conflicts:
	server/include/server.h
	server/modules/monitor/mysql_mon.c
	server/modules/protocol/mysql_client.c
	server/modules/routing/readwritesplit/readwritesplit.c
	utils/skygw_debug.h
2014-06-07 00:50:08 +03:00
7e6cb7afc2 Snapshot of failure tolerance changes.
Added a lot of logging to error, trace and message logs which should help the user to handle errors which can't be automatically resolved, like attempt to use nonexisting database.
2014-06-06 23:32:04 +03:00
7e863657db Removed references to SERVER_IS_JOINED status from readwritesplit.c because rwsplit for Galera doesn't use that status at all. 2014-06-04 14:49:05 +03:00
46495bb59e Bug #443, http://bugs.skysql.com/show_bug.cgi?id=443
monitors used mysql_ping without MYSQL_OPT_READ_TIMEOUT which caused read to block. Fixed in mysql and galera monitor.
Added log writing per each status change of each server and repeatedly if server is not running.

Removed SERVER_IS_JOINED checks from rwsplit router.
2014-06-03 19:39:50 +03:00
1245fba35b Addition of the module info structure to allow module information to
be extracted from the modules.

This gives a way to verify the API that the module provides as well
as the version of that API. The hope is that this will make it possible
for MaxScale to detect out of date plugins and either adapt to use them
or reject loading them.

Also added the ability to set a release state on a per module basis.
This allows for production ready and non-production ready plugins to
be identified.
2014-06-02 17:10:05 +01:00
1a2b8e5475 Router spinlock wasn't used to protect routing. As a consequence router could have been closed in thread #1 while thread #2 was in a middle of execution of router code. Solved by holding router lock so that it covered whole routing operation. 2014-05-28 18:29:02 +03:00
698e63eaf4 Changed bash-related comparison equality operator '==' to generic '=' 2014-05-23 22:08:57 +03:00
a6eb7bd743 Added space character between '--' and comment text in sql scripts. 2014-05-23 22:07:32 +03:00
c28052d5a4 Manual merge from master to develop. Incliding fix for bug #425 http://bugs.skysql.com/show_bug.cgi?id=425 2014-05-21 15:41:39 +03:00
c5ec3cbc57 Added new criteria for slave selection: LEAST_ROUTER_CONNECTIONS which causes the servers having least connections from same router to become chosen.
Added function pointer array which is indexed by enumerated selection criterias.
Selection criteria can now be switched by modifying config file and reloading it.
2014-05-12 17:03:22 +03:00
b7ae1c1784 Added router_option slave_selection_criteria=[LEAST_GLOBAL_CONNECTIONS(default)|LEAST_BEHIND_MASTER] to readwritesplit section.
The option is reloadable in the same way as max_slave_connections parameter. Option value determines which sort function is called before to sort backend server pointers before selecting them and establishing connections to backends.
2014-05-11 23:59:16 +03:00
b8618ee7cf Backend, and server connection counters were updated erroneously. As a consequent, backend selection worked wrong and negative connection count values were possible. 2014-05-11 17:25:56 +03:00
9d1aa06376 Added default value for max_slave_connections=1. It is set when router instance is created. Default value is overwritten if configuration parameter is found.
Added missing makefile to server/modules/routing/test . It is needed for system testing. Prior this fix, 'make testall' failed because the makefile was missing.
2014-05-10 23:27:28 +03:00
ffce479780 backend connection counter was decreased in freeSession even for those backends which weren't connected. This resulted negative counts and affected on selection algorithm. 2014-05-10 22:21:30 +03:00
8be4aba223 Added new state to SESSION: SESSION_STATE_STOPPING, which is set in protocol module before calling closeSession (router). THe new state tells that session is closing and DCBs included may not be polling anymore.
Fixed some crash scenarios.
2014-05-08 23:17:35 +03:00
a1361d9c9e Related to MAX-95, added code which makes it possible to change max_slave_connections parameter value during run time of MaxScale. The change will be effective in rwsplit sessions created after the configuration modification and reload. Note that in order to config modification to be effective, the user must connect the debug interface of MaxScale (telnet) and execute 'reload config'.
In practice, 'reload config' reads config file and updates config parameters of rwsplit router service. In addition to that there is a version number indicating what generation of configuration service holds. When router instance is created (when MaxScale is started) service's config version is copied to router intance. Whenever new client connection (rwsplit session) is started, router instance's config version is checked against that of service's. If versions differ, service's config data is copied to router instance. New session will be started with router instance's config values.
2014-05-02 16:35:40 +03:00
b5e9428ff7 log_manager.cc fixed memory leak, block buffer mutex names weren't freed.
query_classifier.cc use of uninitialized value in skygw_stmt_causes_implicit_commit
config.c crashed if module load failed, use of unitialized value
load_utils.c pretty-printed error
service.c use of uninitialized value in service_add_qualified_param
modules.h function prototype
readwritesplit.c memory leaks
2014-04-29 14:50:09 +03:00
c927057b5c Fixed two bugs of which one was older.
1. in query_classifier.cc autocommit_enabled, and transaction_active variables maintained their values across different sessions. Now those values are stored in each router_client_ses object.
2. As a part of implementation of MAX-95 session variables were added to BACKEND struct which is shared with all sessions using the SERVICE which the particular BACKEND serves. Now each router_client_ses object has a backend reference struct which includes pointer to BACKEND, DCB and to session command cursor.

Added test - set_autocommit_disabled.sql, test_after_autocommit_disabled.sql - to check that session variable is discarded when session where it belongs terminates.
2014-04-28 23:33:49 +03:00
28bc3509cc Added new parameter RW Split Router.max_slave_connections=[<int>|<int>%] which specifies the maximum number of slaves which read/write split router connects in each routing session.
Parameter it read from config file to CONFIG_CONTEXT's parameter list. It is qualified in service.c:service_set_slave_conn_limit and if qualified, the qualified integer value and the value type are copied to the CONFIG_PARAMETER structure.

This CONFIG_PARAMETER struct is cloned (=copied to different memory area) and linked to RW Split SERVICE struct.

When RW Split router_instance is created in readwritesplit.c:createInstance, the value is copied to (new) rwsplit_config_t structure from SERVICE's parameter list.

When new routing session is created in readwritesplit.c:newSession, the rwsplit_config_t structure is copied to ROUTER_CLIENT_SES struct and the actual max_nslaves value is calculated from the config value (if percentage is used).

Tests and many error handling branches are missing but functionality seems to be working.
2014-04-24 22:05:26 +03:00
04313caf82 Initial implementation of the support for multiple slaves in read/write split router session. Omits configuration changes. 2014-04-23 14:55:04 +03:00
81af54730a Added missing test files for rwsplit router tests. 2014-04-16 10:25:32 +03:00
c994c81a0e Changed the way how autocommit is searched from parse tree. Switched to use typelib instead of explicitly converting and comparing user value. Added more tests for cases where user uses literal values in SET autocommit command. 2014-04-15 14:32:47 +03:00
7b40c1ee70 Changed how MaxScale unit test works.
All tests are run by executing 'make testall' in root directory. As a result all directories which contain tests will be entered and tests executed.
After tests, each directory have a test log including the last run's logs only.
Created a global test log, which is specified in test.inc. Test logs from subdirectories are concatenated to this file.
2014-04-10 19:08:28 +03:00
b9a8f2ce65 Merge branch 'develop' of https://github.com/skysql/MaxScale into develop 2014-04-10 17:11:45 +03:00
bbf1bc2afa Fix for bug #418
Increased skygw_query_type_t to 16 bits, and corrected the way how those bit fields are checked.
Added tests for cases where autocommit is disabled and corrected old tests.
2014-04-10 17:09:42 +03:00
2c17dc3edf Bug #418, added functions to query classifier to detect if SET autocommit is called.
Note: this compiles but doesn't work yet properly.
2014-04-09 23:43:03 +03:00
c86e25eeb8 Added a test for implicitly started trasnaction. 2014-04-07 17:00:31 +03:00
ce5dc19ecc Added read/write split router tests to a part of makefile structure. 2014-04-01 14:11:48 -07:00
7c3a354fd8 Query classifier ignored implicit commits in cases of write commands. Fixed it.
Added more tests for transaction support. Mostly different cases where some command triggers implicit commit in the middle of transaction.
2014-03-28 00:16:18 +02:00
9b1035751a Made rwsplit test select_for_var_set.sql generic so that master server's id is filled in test.inc. 2014-03-27 17:17:01 +02:00
e9d1f04bc6 Added test: set var @a via select @@server_id
Added test: set  var @a via select @@server_id
2014-03-27 11:45:47 +01:00
10fdccdeee Added two tests for testing readwrite split router's ability to route read to master if in active transaction and to slave when outside transaction. 2014-03-27 10:27:19 +02:00
75549c6a20 Added $(ROOT_PATH)/test.inc where test parameters can be added. For example, MaxScale port numbers, usernames, passwords. The file can be included in every */test/makefile so that all test directories can use same parameters if necessary.
Moved checking test results to rwsplit.sh so that tests can use easily different criterias to evaluate test success.
2014-03-27 10:06:31 +02:00