Commit Graph

391 Commits

Author SHA1 Message Date
81e1dd8719 Merge branch 'develop' into blr
Add instrumentation

Remove mutexes

Improve gwbuf_append performance

Conflicts:
	server/core/dcb.c
	server/modules/protocol/mysql_backend.c
2014-08-28 11:41:26 +01:00
677a44f497 Merge from release-1.0beta to Z2 2014-08-15 18:00:39 +03:00
d3a79ce7c4 Partial fix to #463, http://bugs.skysql.com/show_bug.cgi?id=463
log_manager.cc: fixed block buffer overflow. Queries are logged to trace log and long queries exceed the bufsize in length. Those were written beyond allocated memory areas.
mysql_client_server_protocol.h: added mysql_protocol_state_t to indicate whether MySQL protocol object is allocated, usable or freed. Freed means that memory allocations made by the protocol are freed. That is, command history etc.
mysql_backend.c: gw_backend_hangup and gw_error_backend_event used to call error handling function although session was already closing. Added check for session state.
mysql_client.c: route_by_statement lost some packets in case where query was sent in multiple packets.
mysql_common.c: gw_MySQL_get_next_packet failed in packet handling with route_by_statement. When multi-packet query was merged into one, packet type wasn't copied.
	protocol_archive_srv_command and mysql_protocol_done didn't have proper locking in place which lead to occasional crashes.
2014-08-14 13:05:05 +03:00
86a4c3ba90 Fix to #472, http://bugs.skysql.com/show_bug.cgi?id=472
Fix to #473, http://bugs.skysql.com/show_bug.cgi?id=473

Several memory issues. Read the code.
In general one-off hint is only added to gwbuf and freed in gwbuf_free. Stacked hint is copied to stack and to gwbuf. gwbuf is freed after routing but stacked hint is freed either in stop or when session is closed. All this applies to named hint as well except that in addition, it has one more copy in named hint struct which is emptied when session is closed.
2014-08-08 01:28:07 +03:00
08b99c121e Bug #468, http://bugs.skysql.com/show_bug.cgi?id=468, Query classifier accessed freed thread context. If parsing fails thd doesn't need to be freed because it holds correct information about command type.
session.c:session_setup_filters : fixed memory leak
hintparser.c: added token_free for HINT_TOKENs and fixed a few memory leaks.
mysql_client_server_protocol.h: added mysql_protocol_done which frees memory blocks pointed to by protocol members. Those can't be freed in dcb.c because dcb.c doesn't know about protocol's members.
mysql_backend.c:gw_backend_close: fixed memory leak
mysql_client.c: gw_client_close: fixed memory leak
mysql_common.c: added implementation of mysql_protocol_done
        :protocol_archive_srv_command: tried to fix memory leak. Some memory is still leaking according to valgrind. Removed use of uninitialized local variable len.
readwritesplit.c: Fix to bug #469, http://bugs.skysql.com/show_bug.cgi?id=469,  rwsplit counts every connection twice in master - counnection counts leak
	execute_sescmd_in_backend: fixed a memory leak - visible only in DEBUG=Y build.
readwritesplit/test/makefile: added target for hints tests
2014-08-05 18:14:06 +03:00
7558abb6fa Bug #468, http://bugs.skysql.com/show_bug.cgi?id=468, Query classifier accessed freed thread context. If parsing fails thd doesn't need to be freed because it holds correct information about command type.
session.c:session_setup_filters : fixed memory leak
hintparser.c: added token_free for HINT_TOKENs and fixed a few memory leaks.
mysql_client_server_protocol.h: added mysql_protocol_done which frees memory blocks pointed to by protocol members. Those can't be freed in dcb.c because dcb.c doesn't know about protocol's members.
mysql_backend.c:gw_backend_close: fixed memory leak
mysql_client.c: gw_client_close: fixed memory leak
mysql_common.c: added implementation of mysql_protocol_done
    :protocol_archive_srv_command: tried to fix memory leak. Some memory is still leaking according to valgrind. Removed use of uninitialized local variable len.
readwritesplit.c:execute_sescmd_in_backend: fixed a memory leak - visible only in DEBUG=Y build.
2014-08-05 10:42:13 +03:00
6b3c7041e3 Bug #468, http://bugs.skysql.com/show_bug.cgi?id=468, Query classifier accessed freed thread context. If parsing fails thd doesn't need to be freed because it holds correct information about command type.
session.c:session_setup_filters : fixed memory leak
hintparser.c: added token_free for HINT_TOKENs and fixed a few memory leaks.
mysql_client_server_protocol.h: added mysql_protocol_done which frees memory blocks pointed to by protocol members. Those can't be freed in dcb.c because dcb.c doesn't know about protocol's members.
mysql_backend.c:gw_backend_close: fixed memory leak
mysql_client.c: gw_client_close: fixed memory leak
mysql_common.c: added implementation of mysql_protocol_done
	:protocol_archive_srv_command: tried to fix memory leak. Some memory is still leaking according to valgrind. Removed use of uninitialized local variable len.
readwritesplit.c:execute_sescmd_in_backend: fixed a memory leak - visible only in DEBUG=Y build.
2014-08-05 09:31:10 +03:00
65b121889b config.h: Added enumerated types MAX_RLAG_AVAILABLE and MAX_RLAG_UNDEFINED to be used instead of magic numbers as max_slave_replication_lag values.
readwritesplit.h: Added TARGET_RLAG_MAX to the list of hint targets. It is used in case of hint max_slave_replication_lag=<value>
readwritesplit.c: get_dcb is called when readwritesplit router chooses target backend for a query. Backend is selected from those the router has connection to. get_dcb now takes additional parameters, name and max_rlag. Name is used if there is a hint route to server <server name>. max_rlag is used for hint max_slave_replication_lag=<value> or if it doesn't exist, then for configure parameter max_slave_replication_lag=<configured value> or if both are absent, ignored.
2014-08-01 16:33:29 +03:00
4f3d746f4e MAX-157. Added support for hints in rwsplit router.
buffer.c: added memory release for hint of a GWBUF
hint.c: added bool hint_exists()
hint.h: added placeholder for hint type HINT_ROUTE_TO_ALL which doesn't have implementation yet.
filter/Makefile: fixed dependency issue
hintparser.c: added const char* token_get_keyword(), hint_parser:added NULL check, hint_next_token: fixed off-by-one bug
readwritesplit.h: added bitfield for hints' use, which includes route targets and flag for case where user hinted to route to named backend server.
readwritesplit.c: added function route_target_t get_route_target() for resolving route target based on 1) query type (from query_classifier) 2) transaction state (active/not) and 3) hints. Modified get_dcb, which is called in routeQuery to provide pointer to correct backend DCB. Now get_dcb also takes server unique name as a parameter if such a hint was found. for hints' use, which includes  enter the commit message for your changes.
2014-07-31 23:40:02 +03:00
2dea68b9b3 Hint parsing 2014-07-25 16:33:49 +01:00
87e66a0ea8 Revert "Implementation of HTTPD protocol, gwbuf properties and a dmeo web application"
This reverts commit 6fd5dff34902051f38932947493280e1e62dbeb1.
2014-07-15 17:53:39 +01:00
6fd5dff349 Implementation of HTTPD protocol, gwbuf properties and a dmeo web application
interface to test the new httpd.
2014-07-15 17:39:31 +01:00
39679447c9 Bugzilla #455, MaxScale crashed due missing buffer type information in multi-buffer responses. 2014-07-03 19:20:45 +03:00
e944ae2e70 http://bugs.skysql.com/show_bug.cgi?id=453
Fixed bug in session command resul handling. In case where backend sent error message the session command cursor wasn't updated properly.

Added check to bref_clear_state, if bref's waiter counter would go negative, decrement to global operation counter is skipped.
2014-07-03 00:43:30 +03:00
eaaac58b44 Addition of server weighting to the read/write splitter
Tidy up the output of show server and show servers so that columsn align
2014-07-01 11:06:28 +01:00
57a977bdc0 Merge branch 'develop' of https://github.com/skysql/MaxScale into develop 2014-06-30 22:38:15 +03:00
b875936a21 MAX-94, Added configuration parameter, max_slave_replication_lag=<longest allowed replication lag in seconds> to router section.
Parameter can be changed runtime but it requires reloading of config.
2014-06-30 22:35:27 +03:00
0b89245a4b Merge branch 'develop' into MAX-167
Conflicts:
	server/modules/monitor/mysql_mon.c
	server/modules/routing/readwritesplit/readwritesplit.c
2014-06-30 14:17:00 +03:00
6d672cb967 Added status bit BREF_QUERY_ACTIVE to indicate if a query, other than session command, sent to backend for execution. The flag is cleared when the first packet belonging to the response arrives.
The flag is part of the active operation counting, which is utilized in load balancing. The active operation count per backend is used by default as criteria when router chooses to which backend a query should be routed.
2014-06-30 13:44:34 +03:00
fcf67716fd Added mechanism for choosing slave for a query based on the current load in all connected slaves. Counting operations is not correctly done here. Reading values and choosing accordingly is done.
Fixed several things in session command reply processing.
2014-06-29 22:21:30 +03:00
25d4f784bc Implementation of server routing generic functionality and for readconnrouter 2014-06-27 12:07:47 +01:00
7ff14e23a5 Support for prepared statement, namely support for following comands : COM_STMT_PREPARE, COM_STMT_EXECUTE, COM_STMT_SEND_LONG_DATA, COM_STMT_RESET, COM_STMT_CLOSE, SQLCOM_PREPARE, SQLCOM_EXECUTE, SQLCOM_DEALLOCATE_PREPARE (DEALLOCATE/DROP PREPARE stmt).
All prepare commands are executed in every backend server currently connected.

All executes are routed to master. If stmt type was recorded in prepare phase in rwsplit router, read-only stmts could be routed to slaves.

COM_STMT_PREPARE gets arbitrary number of response packets from backend database. Since statements are prepared in every backend server and only one multi-packet response can be replied to client, redundant multi-packet responses are discarded. This is done in router. Mechanisms from session command handling are utilized with little changes: router must identify when response consists of multiple packets so that it knows to calculate the number of packets in response and that it is able to discard correct number of packets.

Information to the reply-handling router is provided by backend protocol, which includes a ordered list of commands of commands sent to protocol-owning backend server. A command is stored to protocol struct in mysql_backend.c:gw_MySQLWrite_backend if the statement buffer's type has GWBUF_TYPE_SINGLE_STMT set in mysql_client.c:route_by_statement. GWBUF_TYPE_SINGLE_STATEMENT indicates that there is single statement in the buffer, as opposite to Read Connection router, which accepts streaming input from client.
2014-06-25 17:15:46 +03:00
2d128de85f Replaced MySQL command macros with enumerated type. Each command has prefix 'MYSQL_' otherwise they match with those listed in mariadb-5.5 mysql_com.h.
Added server command list structure which is included in MySQLProtocol. It holds command and number of response packets the command causes backend server to send as a response. Added set of functions related to protocol command and response packets counting etc.
2014-06-25 15:48:55 +03:00
49163a4c43 Merge branch 'develop' into MAX-11 2014-06-16 19:53:19 +03:00
4f03c775e5 sescmd_cursor_reset didn't reset cursor's property pointer which made reset ineffective. 2014-06-16 13:40:45 +03:00
5bcae64538 When protocol closes DCB it calls dcb_close instead of dcb->func.close. dcb_close then calls dcb->func.close. This is now changed to all protocols and routers.
Rwsplit handles ERRACT_NEW_CONNECTION by clearing backend reference, removing callbacks and associating backend reference with new backend server. If it succeeds and the router session can continue, handleError returns true. Otherwise false. When ever false is returned it means that session must be closed.

Rwsplit now tolerates backend failures in a way that it searches new backends when monitor, backend, or client operation fails due to backend failure.
2014-06-15 23:44:07 +03:00
719503e471 Addition of new client utility, maxadmin.
Supporting protocol for the admin interface

New routing module, cli, which shares source with debugcli

Tidyup output of lsit commands
2014-06-13 23:40:07 +01:00
dfc9141a38 Whenever monitored server's state changes, a callback, router_handle_state_switch is called for each DCB in MaxScale session. The DCB referring to the server in question will be passed as parameter to backend DCB's hangup function.
The logic that solves the situation is not in place yet.
2014-06-13 13:30:50 +03:00
15ff1fd26a Fixed many error handling issues regading to timing and multiple threads.
Added flags to those backend references which have sent something to backend which causes the backend to send results or reply back. Didn't add removal of the flag since there's currently no way to tell whether response from backend contains anything else than session command reply - which aren't counted when BREF_WAITING_RESULT is set and cleared.
2014-06-12 23:22:51 +03:00
e95b6cc0d9 dcb.c, gw_utils.c, mysql_server_protocol.h, mysql_client.c : Replaced gw_read_gwbuff with dcb_read in mysql_client.c:gw_read_client_event.
rwsplit.sh, test_sescmd.sql : Added test case for session commands.
2014-06-12 19:02:47 +03:00
2963a8448b Implement non-blocking alternative to mutexes for read serialisation 2014-06-10 17:59:49 +01:00
f7a177dac8 Improved diagnostics 2014-06-08 13:52:48 +01:00
fda9b1c706 Merge branch 'develop' into blr
Conflicts:
	server/core/dcb.c
	server/modules/include/blr.h
	server/modules/routing/binlog/STATUS
	server/modules/routing/binlog/blr.c
	server/modules/routing/binlog/blr_file.c
	server/modules/routing/binlog/blr_master.c
	server/modules/routing/binlog/blr_slave.c
2014-06-08 00:50:36 +01:00
13e95ffc53 Improved diagnostics
Added master reconnect on failure

Added EPOLLRDHUP events
2014-06-07 21:01:11 +01:00
ad744962b2 Merge branch 'develop' into MAX-11
Conflicts:
	server/include/server.h
	server/modules/monitor/mysql_mon.c
	server/modules/protocol/mysql_client.c
	server/modules/routing/readwritesplit/readwritesplit.c
	utils/skygw_debug.h
2014-06-07 00:50:08 +03:00
7e6cb7afc2 Snapshot of failure tolerance changes.
Added a lot of logging to error, trace and message logs which should help the user to handle errors which can't be automatically resolved, like attempt to use nonexisting database.
2014-06-06 23:32:04 +03:00
bb0e6c3858 Support non-default binlog filenames
Add support for bin log file names that are shorter than the default.
Handle events per more than 2 packets
2014-06-04 18:37:41 +01:00
46495bb59e Bug #443, http://bugs.skysql.com/show_bug.cgi?id=443
monitors used mysql_ping without MYSQL_OPT_READ_TIMEOUT which caused read to block. Fixed in mysql and galera monitor.
Added log writing per each status change of each server and repeatedly if server is not running.

Removed SERVER_IS_JOINED checks from rwsplit router.
2014-06-03 19:39:50 +03:00
4b5f801ff9 Misc fixes for bin log rotate issues 2014-05-28 23:38:54 +01:00
97bb1638ad Delete swp file 2014-05-27 12:17:09 +01:00
378d00983d merge from branch 'cenh'
merge from branch ‘cenh’
2014-05-26 12:15:34 +02:00
f0f6413b39 Update
Update
2014-05-26 11:17:57 +02:00
d732a397aa struct router_client_session update
struct router_client_session update
2014-05-26 10:56:51 +02:00
45d5d521b8 Merge branch 'blr' into cenh
Conflicts:
	Documentation/MaxScale Configuration And Usage Scenarios.pdf
	server/core/config.c
	server/core/dcb.c
	server/core/service.c
	server/modules/routing/Makefile
	server/modules/routing/debugcmd.c
	server/modules/routing/readwritesplit/readwritesplit.c
	server/modules/routing/readwritesplit/test/rwsplit.sh
2014-05-23 16:39:39 +01:00
7fe50a311f Addition of developer and user modes to debugcli
In order to be able to protect the interface so that mistyped argument
do not endanger MaxScale yet still allow the developer access a router
option has been added that gives the debugcli two modes.
2014-05-23 13:29:58 +01:00
c28052d5a4 Manual merge from master to develop. Incliding fix for bug #425 http://bugs.skysql.com/show_bug.cgi?id=425 2014-05-21 15:41:39 +03:00
d52f761b2d Fix missing GWBUF queue 2014-05-19 15:18:09 +01:00
9777e83a5a Merge branch 'master' into blr 2014-05-19 13:46:47 +01:00
29d247319d Fixed issue with bin log event straddling buffers 2014-05-17 07:25:03 +01:00
bb364f531b Bug #425, http://bugs.skysql.com/show_bug.cgi?id=425
mysqlk_client.c now assumes that mysql packets may arrive in separate pieces.
gw_read_client_event reads client data, and if packet is incomplete it is added to new dcb->dcb_readqueue. When next packet is combined with data in readqueue.

mysql_common.c:gw_MySQL_get_next_packet can now return mysql packet from read buffer although it would span multiple gwbuf buffers.
2014-05-16 17:01:10 +02:00