236 Commits

Author SHA1 Message Date
VilhoRaatikka
7558abb6fa Bug #468, http://bugs.skysql.com/show_bug.cgi?id=468, Query classifier accessed freed thread context. If parsing fails thd doesn't need to be freed because it holds correct information about command type.
session.c:session_setup_filters : fixed memory leak
hintparser.c: added token_free for HINT_TOKENs and fixed a few memory leaks.
mysql_client_server_protocol.h: added mysql_protocol_done which frees memory blocks pointed to by protocol members. Those can't be freed in dcb.c because dcb.c doesn't know about protocol's members.
mysql_backend.c:gw_backend_close: fixed memory leak
mysql_client.c: gw_client_close: fixed memory leak
mysql_common.c: added implementation of mysql_protocol_done
    :protocol_archive_srv_command: tried to fix memory leak. Some memory is still leaking according to valgrind. Removed use of uninitialized local variable len.
readwritesplit.c:execute_sescmd_in_backend: fixed a memory leak - visible only in DEBUG=Y build.
2014-08-05 10:42:13 +03:00
VilhoRaatikka
6b3c7041e3 Bug #468, http://bugs.skysql.com/show_bug.cgi?id=468, Query classifier accessed freed thread context. If parsing fails thd doesn't need to be freed because it holds correct information about command type.
session.c:session_setup_filters : fixed memory leak
hintparser.c: added token_free for HINT_TOKENs and fixed a few memory leaks.
mysql_client_server_protocol.h: added mysql_protocol_done which frees memory blocks pointed to by protocol members. Those can't be freed in dcb.c because dcb.c doesn't know about protocol's members.
mysql_backend.c:gw_backend_close: fixed memory leak
mysql_client.c: gw_client_close: fixed memory leak
mysql_common.c: added implementation of mysql_protocol_done
	:protocol_archive_srv_command: tried to fix memory leak. Some memory is still leaking according to valgrind. Removed use of uninitialized local variable len.
readwritesplit.c:execute_sescmd_in_backend: fixed a memory leak - visible only in DEBUG=Y build.
2014-08-05 09:31:10 +03:00
VilhoRaatikka
65b121889b config.h: Added enumerated types MAX_RLAG_AVAILABLE and MAX_RLAG_UNDEFINED to be used instead of magic numbers as max_slave_replication_lag values.
readwritesplit.h: Added TARGET_RLAG_MAX to the list of hint targets. It is used in case of hint max_slave_replication_lag=<value>
readwritesplit.c: get_dcb is called when readwritesplit router chooses target backend for a query. Backend is selected from those the router has connection to. get_dcb now takes additional parameters, name and max_rlag. Name is used if there is a hint route to server <server name>. max_rlag is used for hint max_slave_replication_lag=<value> or if it doesn't exist, then for configure parameter max_slave_replication_lag=<configured value> or if both are absent, ignored.
2014-08-01 16:33:29 +03:00
VilhoRaatikka
4f3d746f4e MAX-157. Added support for hints in rwsplit router.
buffer.c: added memory release for hint of a GWBUF
hint.c: added bool hint_exists()
hint.h: added placeholder for hint type HINT_ROUTE_TO_ALL which doesn't have implementation yet.
filter/Makefile: fixed dependency issue
hintparser.c: added const char* token_get_keyword(), hint_parser:added NULL check, hint_next_token: fixed off-by-one bug
readwritesplit.h: added bitfield for hints' use, which includes route targets and flag for case where user hinted to route to named backend server.
readwritesplit.c: added function route_target_t get_route_target() for resolving route target based on 1) query type (from query_classifier) 2) transaction state (active/not) and 3) hints. Modified get_dcb, which is called in routeQuery to provide pointer to correct backend DCB. Now get_dcb also takes server unique name as a parameter if such a hint was found. for hints' use, which includes  enter the commit message for your changes.
2014-07-31 23:40:02 +03:00
Mark Riddoch
2dea68b9b3 Hint parsing 2014-07-25 16:33:49 +01:00
Mark Riddoch
87e66a0ea8 Revert "Implementation of HTTPD protocol, gwbuf properties and a dmeo web application"
This reverts commit 6fd5dff34902051f38932947493280e1e62dbeb1.
2014-07-15 17:53:39 +01:00
Mark Riddoch
6fd5dff349 Implementation of HTTPD protocol, gwbuf properties and a dmeo web application
interface to test the new httpd.
2014-07-15 17:39:31 +01:00
VilhoRaatikka
39679447c9 Bugzilla #455, MaxScale crashed due missing buffer type information in multi-buffer responses. 2014-07-03 19:20:45 +03:00
VilhoRaatikka
e944ae2e70 http://bugs.skysql.com/show_bug.cgi?id=453
Fixed bug in session command resul handling. In case where backend sent error message the session command cursor wasn't updated properly.

Added check to bref_clear_state, if bref's waiter counter would go negative, decrement to global operation counter is skipped.
2014-07-03 00:43:30 +03:00
Mark Riddoch
eaaac58b44 Addition of server weighting to the read/write splitter
Tidy up the output of show server and show servers so that columsn align
2014-07-01 11:06:28 +01:00
VilhoRaatikka
57a977bdc0 Merge branch 'develop' of https://github.com/skysql/MaxScale into develop 2014-06-30 22:38:15 +03:00
VilhoRaatikka
b875936a21 MAX-94, Added configuration parameter, max_slave_replication_lag=<longest allowed replication lag in seconds> to router section.
Parameter can be changed runtime but it requires reloading of config.
2014-06-30 22:35:27 +03:00
VilhoRaatikka
0b89245a4b Merge branch 'develop' into MAX-167
Conflicts:
	server/modules/monitor/mysql_mon.c
	server/modules/routing/readwritesplit/readwritesplit.c
2014-06-30 14:17:00 +03:00
VilhoRaatikka
6d672cb967 Added status bit BREF_QUERY_ACTIVE to indicate if a query, other than session command, sent to backend for execution. The flag is cleared when the first packet belonging to the response arrives.
The flag is part of the active operation counting, which is utilized in load balancing. The active operation count per backend is used by default as criteria when router chooses to which backend a query should be routed.
2014-06-30 13:44:34 +03:00
VilhoRaatikka
fcf67716fd Added mechanism for choosing slave for a query based on the current load in all connected slaves. Counting operations is not correctly done here. Reading values and choosing accordingly is done.
Fixed several things in session command reply processing.
2014-06-29 22:21:30 +03:00
Mark Riddoch
25d4f784bc Implementation of server routing generic functionality and for readconnrouter 2014-06-27 12:07:47 +01:00
VilhoRaatikka
7ff14e23a5 Support for prepared statement, namely support for following comands : COM_STMT_PREPARE, COM_STMT_EXECUTE, COM_STMT_SEND_LONG_DATA, COM_STMT_RESET, COM_STMT_CLOSE, SQLCOM_PREPARE, SQLCOM_EXECUTE, SQLCOM_DEALLOCATE_PREPARE (DEALLOCATE/DROP PREPARE stmt).
All prepare commands are executed in every backend server currently connected.

All executes are routed to master. If stmt type was recorded in prepare phase in rwsplit router, read-only stmts could be routed to slaves.

COM_STMT_PREPARE gets arbitrary number of response packets from backend database. Since statements are prepared in every backend server and only one multi-packet response can be replied to client, redundant multi-packet responses are discarded. This is done in router. Mechanisms from session command handling are utilized with little changes: router must identify when response consists of multiple packets so that it knows to calculate the number of packets in response and that it is able to discard correct number of packets.

Information to the reply-handling router is provided by backend protocol, which includes a ordered list of commands of commands sent to protocol-owning backend server. A command is stored to protocol struct in mysql_backend.c:gw_MySQLWrite_backend if the statement buffer's type has GWBUF_TYPE_SINGLE_STMT set in mysql_client.c:route_by_statement. GWBUF_TYPE_SINGLE_STATEMENT indicates that there is single statement in the buffer, as opposite to Read Connection router, which accepts streaming input from client.
2014-06-25 17:15:46 +03:00
VilhoRaatikka
2d128de85f Replaced MySQL command macros with enumerated type. Each command has prefix 'MYSQL_' otherwise they match with those listed in mariadb-5.5 mysql_com.h.
Added server command list structure which is included in MySQLProtocol. It holds command and number of response packets the command causes backend server to send as a response. Added set of functions related to protocol command and response packets counting etc.
2014-06-25 15:48:55 +03:00
VilhoRaatikka
49163a4c43 Merge branch 'develop' into MAX-11 2014-06-16 19:53:19 +03:00
VilhoRaatikka
4f03c775e5 sescmd_cursor_reset didn't reset cursor's property pointer which made reset ineffective. 2014-06-16 13:40:45 +03:00
VilhoRaatikka
5bcae64538 When protocol closes DCB it calls dcb_close instead of dcb->func.close. dcb_close then calls dcb->func.close. This is now changed to all protocols and routers.
Rwsplit handles ERRACT_NEW_CONNECTION by clearing backend reference, removing callbacks and associating backend reference with new backend server. If it succeeds and the router session can continue, handleError returns true. Otherwise false. When ever false is returned it means that session must be closed.

Rwsplit now tolerates backend failures in a way that it searches new backends when monitor, backend, or client operation fails due to backend failure.
2014-06-15 23:44:07 +03:00
Mark Riddoch
719503e471 Addition of new client utility, maxadmin.
Supporting protocol for the admin interface

New routing module, cli, which shares source with debugcli

Tidyup output of lsit commands
2014-06-13 23:40:07 +01:00
VilhoRaatikka
dfc9141a38 Whenever monitored server's state changes, a callback, router_handle_state_switch is called for each DCB in MaxScale session. The DCB referring to the server in question will be passed as parameter to backend DCB's hangup function.
The logic that solves the situation is not in place yet.
2014-06-13 13:30:50 +03:00
VilhoRaatikka
15ff1fd26a Fixed many error handling issues regading to timing and multiple threads.
Added flags to those backend references which have sent something to backend which causes the backend to send results or reply back. Didn't add removal of the flag since there's currently no way to tell whether response from backend contains anything else than session command reply - which aren't counted when BREF_WAITING_RESULT is set and cleared.
2014-06-12 23:22:51 +03:00
VilhoRaatikka
e95b6cc0d9 dcb.c, gw_utils.c, mysql_server_protocol.h, mysql_client.c : Replaced gw_read_gwbuff with dcb_read in mysql_client.c:gw_read_client_event.
rwsplit.sh, test_sescmd.sql : Added test case for session commands.
2014-06-12 19:02:47 +03:00
Mark Riddoch
2963a8448b Implement non-blocking alternative to mutexes for read serialisation 2014-06-10 17:59:49 +01:00
Mark Riddoch
f7a177dac8 Improved diagnostics 2014-06-08 13:52:48 +01:00
Mark Riddoch
fda9b1c706 Merge branch 'develop' into blr
Conflicts:
	server/core/dcb.c
	server/modules/include/blr.h
	server/modules/routing/binlog/STATUS
	server/modules/routing/binlog/blr.c
	server/modules/routing/binlog/blr_file.c
	server/modules/routing/binlog/blr_master.c
	server/modules/routing/binlog/blr_slave.c
2014-06-08 00:50:36 +01:00
Mark Riddoch
13e95ffc53 Improved diagnostics
Added master reconnect on failure

Added EPOLLRDHUP events
2014-06-07 21:01:11 +01:00
VilhoRaatikka
ad744962b2 Merge branch 'develop' into MAX-11
Conflicts:
	server/include/server.h
	server/modules/monitor/mysql_mon.c
	server/modules/protocol/mysql_client.c
	server/modules/routing/readwritesplit/readwritesplit.c
	utils/skygw_debug.h
2014-06-07 00:50:08 +03:00
VilhoRaatikka
7e6cb7afc2 Snapshot of failure tolerance changes.
Added a lot of logging to error, trace and message logs which should help the user to handle errors which can't be automatically resolved, like attempt to use nonexisting database.
2014-06-06 23:32:04 +03:00
Mark Riddoch
bb0e6c3858 Support non-default binlog filenames
Add support for bin log file names that are shorter than the default.
Handle events per more than 2 packets
2014-06-04 18:37:41 +01:00
VilhoRaatikka
46495bb59e Bug #443, http://bugs.skysql.com/show_bug.cgi?id=443
monitors used mysql_ping without MYSQL_OPT_READ_TIMEOUT which caused read to block. Fixed in mysql and galera monitor.
Added log writing per each status change of each server and repeatedly if server is not running.

Removed SERVER_IS_JOINED checks from rwsplit router.
2014-06-03 19:39:50 +03:00
Mark Riddoch
4b5f801ff9 Misc fixes for bin log rotate issues 2014-05-28 23:38:54 +01:00
Mark Riddoch
97bb1638ad Delete swp file 2014-05-27 12:17:09 +01:00
MassimilianoPinto
378d00983d merge from branch 'cenh'
merge from branch ‘cenh’
2014-05-26 12:15:34 +02:00
MassimilianoPinto
f0f6413b39 Update
Update
2014-05-26 11:17:57 +02:00
MassimilianoPinto
d732a397aa struct router_client_session update
struct router_client_session update
2014-05-26 10:56:51 +02:00
Mark Riddoch
45d5d521b8 Merge branch 'blr' into cenh
Conflicts:
	Documentation/MaxScale Configuration And Usage Scenarios.pdf
	server/core/config.c
	server/core/dcb.c
	server/core/service.c
	server/modules/routing/Makefile
	server/modules/routing/debugcmd.c
	server/modules/routing/readwritesplit/readwritesplit.c
	server/modules/routing/readwritesplit/test/rwsplit.sh
2014-05-23 16:39:39 +01:00
Mark Riddoch
7fe50a311f Addition of developer and user modes to debugcli
In order to be able to protect the interface so that mistyped argument
do not endanger MaxScale yet still allow the developer access a router
option has been added that gives the debugcli two modes.
2014-05-23 13:29:58 +01:00
VilhoRaatikka
c28052d5a4 Manual merge from master to develop. Incliding fix for bug #425 http://bugs.skysql.com/show_bug.cgi?id=425 2014-05-21 15:41:39 +03:00
Mark Riddoch
d52f761b2d Fix missing GWBUF queue 2014-05-19 15:18:09 +01:00
Mark Riddoch
9777e83a5a Merge branch 'master' into blr 2014-05-19 13:46:47 +01:00
Mark Riddoch
29d247319d Fixed issue with bin log event straddling buffers 2014-05-17 07:25:03 +01:00
VilhoRaatikka
bb364f531b Bug #425, http://bugs.skysql.com/show_bug.cgi?id=425
mysqlk_client.c now assumes that mysql packets may arrive in separate pieces.
gw_read_client_event reads client data, and if packet is incomplete it is added to new dcb->dcb_readqueue. When next packet is combined with data in readqueue.

mysql_common.c:gw_MySQL_get_next_packet can now return mysql packet from read buffer although it would span multiple gwbuf buffers.
2014-05-16 17:01:10 +02:00
Mark Riddoch
1d08b0100f Initial binlog prototype
Supports pulling binlogs from the master, caching to a local file and relaying to a slave.

Only tested with a single slave and a single MaxScale thread.
2014-05-13 16:06:05 +01:00
VilhoRaatikka
c5ec3cbc57 Added new criteria for slave selection: LEAST_ROUTER_CONNECTIONS which causes the servers having least connections from same router to become chosen.
Added function pointer array which is indexed by enumerated selection criterias.
Selection criteria can now be switched by modifying config file and reloading it.
2014-05-12 17:03:22 +03:00
VilhoRaatikka
b7ae1c1784 Added router_option slave_selection_criteria=[LEAST_GLOBAL_CONNECTIONS(default)|LEAST_BEHIND_MASTER] to readwritesplit section.
The option is reloadable in the same way as max_slave_connections parameter. Option value determines which sort function is called before to sort backend server pointers before selecting them and establishing connections to backends.
2014-05-11 23:59:16 +03:00
VilhoRaatikka
9d1aa06376 Added default value for max_slave_connections=1. It is set when router instance is created. Default value is overwritten if configuration parameter is found.
Added missing makefile to server/modules/routing/test . It is needed for system testing. Prior this fix, 'make testall' failed because the makefile was missing.
2014-05-10 23:27:28 +03:00
VilhoRaatikka
9af5d9fb06 Commit fix to bug #418 http://bugs.skysql.com/show_bug.cgi?id=418 2014-05-08 13:13:42 +03:00