Commit Graph

564 Commits

Author SHA1 Message Date
c28052d5a4 Manual merge from master to develop. Incliding fix for bug #425 http://bugs.skysql.com/show_bug.cgi?id=425 2014-05-21 15:41:39 +03:00
c5ec3cbc57 Added new criteria for slave selection: LEAST_ROUTER_CONNECTIONS which causes the servers having least connections from same router to become chosen.
Added function pointer array which is indexed by enumerated selection criterias.
Selection criteria can now be switched by modifying config file and reloading it.
2014-05-12 17:03:22 +03:00
b7ae1c1784 Added router_option slave_selection_criteria=[LEAST_GLOBAL_CONNECTIONS(default)|LEAST_BEHIND_MASTER] to readwritesplit section.
The option is reloadable in the same way as max_slave_connections parameter. Option value determines which sort function is called before to sort backend server pointers before selecting them and establishing connections to backends.
2014-05-11 23:59:16 +03:00
b8618ee7cf Backend, and server connection counters were updated erroneously. As a consequent, backend selection worked wrong and negative connection count values were possible. 2014-05-11 17:25:56 +03:00
9d1aa06376 Added default value for max_slave_connections=1. It is set when router instance is created. Default value is overwritten if configuration parameter is found.
Added missing makefile to server/modules/routing/test . It is needed for system testing. Prior this fix, 'make testall' failed because the makefile was missing.
2014-05-10 23:27:28 +03:00
ffce479780 backend connection counter was decreased in freeSession even for those backends which weren't connected. This resulted negative counts and affected on selection algorithm. 2014-05-10 22:21:30 +03:00
8be4aba223 Added new state to SESSION: SESSION_STATE_STOPPING, which is set in protocol module before calling closeSession (router). THe new state tells that session is closing and DCBs included may not be polling anymore.
Fixed some crash scenarios.
2014-05-08 23:17:35 +03:00
a1361d9c9e Related to MAX-95, added code which makes it possible to change max_slave_connections parameter value during run time of MaxScale. The change will be effective in rwsplit sessions created after the configuration modification and reload. Note that in order to config modification to be effective, the user must connect the debug interface of MaxScale (telnet) and execute 'reload config'.
In practice, 'reload config' reads config file and updates config parameters of rwsplit router service. In addition to that there is a version number indicating what generation of configuration service holds. When router instance is created (when MaxScale is started) service's config version is copied to router intance. Whenever new client connection (rwsplit session) is started, router instance's config version is checked against that of service's. If versions differ, service's config data is copied to router instance. New session will be started with router instance's config values.
2014-05-02 16:35:40 +03:00
b5e9428ff7 log_manager.cc fixed memory leak, block buffer mutex names weren't freed.
query_classifier.cc use of uninitialized value in skygw_stmt_causes_implicit_commit
config.c crashed if module load failed, use of unitialized value
load_utils.c pretty-printed error
service.c use of uninitialized value in service_add_qualified_param
modules.h function prototype
readwritesplit.c memory leaks
2014-04-29 14:50:09 +03:00
c927057b5c Fixed two bugs of which one was older.
1. in query_classifier.cc autocommit_enabled, and transaction_active variables maintained their values across different sessions. Now those values are stored in each router_client_ses object.
2. As a part of implementation of MAX-95 session variables were added to BACKEND struct which is shared with all sessions using the SERVICE which the particular BACKEND serves. Now each router_client_ses object has a backend reference struct which includes pointer to BACKEND, DCB and to session command cursor.

Added test - set_autocommit_disabled.sql, test_after_autocommit_disabled.sql - to check that session variable is discarded when session where it belongs terminates.
2014-04-28 23:33:49 +03:00
28bc3509cc Added new parameter RW Split Router.max_slave_connections=[<int>|<int>%] which specifies the maximum number of slaves which read/write split router connects in each routing session.
Parameter it read from config file to CONFIG_CONTEXT's parameter list. It is qualified in service.c:service_set_slave_conn_limit and if qualified, the qualified integer value and the value type are copied to the CONFIG_PARAMETER structure.

This CONFIG_PARAMETER struct is cloned (=copied to different memory area) and linked to RW Split SERVICE struct.

When RW Split router_instance is created in readwritesplit.c:createInstance, the value is copied to (new) rwsplit_config_t structure from SERVICE's parameter list.

When new routing session is created in readwritesplit.c:newSession, the rwsplit_config_t structure is copied to ROUTER_CLIENT_SES struct and the actual max_nslaves value is calculated from the config value (if percentage is used).

Tests and many error handling branches are missing but functionality seems to be working.
2014-04-24 22:05:26 +03:00
04313caf82 Initial implementation of the support for multiple slaves in read/write split router session. Omits configuration changes. 2014-04-23 14:55:04 +03:00
81af54730a Added missing test files for rwsplit router tests. 2014-04-16 10:25:32 +03:00
c994c81a0e Changed the way how autocommit is searched from parse tree. Switched to use typelib instead of explicitly converting and comparing user value. Added more tests for cases where user uses literal values in SET autocommit command. 2014-04-15 14:32:47 +03:00
7b40c1ee70 Changed how MaxScale unit test works.
All tests are run by executing 'make testall' in root directory. As a result all directories which contain tests will be entered and tests executed.
After tests, each directory have a test log including the last run's logs only.
Created a global test log, which is specified in test.inc. Test logs from subdirectories are concatenated to this file.
2014-04-10 19:08:28 +03:00
b9a8f2ce65 Merge branch 'develop' of https://github.com/skysql/MaxScale into develop 2014-04-10 17:11:45 +03:00
bbf1bc2afa Fix for bug #418
Increased skygw_query_type_t to 16 bits, and corrected the way how those bit fields are checked.
Added tests for cases where autocommit is disabled and corrected old tests.
2014-04-10 17:09:42 +03:00
2c17dc3edf Bug #418, added functions to query classifier to detect if SET autocommit is called.
Note: this compiles but doesn't work yet properly.
2014-04-09 23:43:03 +03:00
c86e25eeb8 Added a test for implicitly started trasnaction. 2014-04-07 17:00:31 +03:00
ce5dc19ecc Added read/write split router tests to a part of makefile structure. 2014-04-01 14:11:48 -07:00
7c3a354fd8 Query classifier ignored implicit commits in cases of write commands. Fixed it.
Added more tests for transaction support. Mostly different cases where some command triggers implicit commit in the middle of transaction.
2014-03-28 00:16:18 +02:00
9b1035751a Made rwsplit test select_for_var_set.sql generic so that master server's id is filled in test.inc. 2014-03-27 17:17:01 +02:00
e9d1f04bc6 Added test: set var @a via select @@server_id
Added test: set  var @a via select @@server_id
2014-03-27 11:45:47 +01:00
10fdccdeee Added two tests for testing readwrite split router's ability to route read to master if in active transaction and to slave when outside transaction. 2014-03-27 10:27:19 +02:00
75549c6a20 Added $(ROOT_PATH)/test.inc where test parameters can be added. For example, MaxScale port numbers, usernames, passwords. The file can be included in every */test/makefile so that all test directories can use same parameters if necessary.
Moved checking test results to rwsplit.sh so that tests can use easily different criterias to evaluate test success.
2014-03-27 10:06:31 +02:00
f49df89a0c Changed how query classifier determines which statements trigger implicit commit.
Changed test makefile and rwsplit.sh script and added two example sql scripts.
2014-03-26 19:10:35 +02:00
df02926321 Added structure which can be cloned for multiple test cases. 2014-03-25 23:19:24 +02:00
f320c17834 Added test for readwrite split: SET and TRANSACTION - COMMIT
Added test for readwrite split: SET and TRANSACTION - COMMIT
2014-03-24 15:15:21 +01:00
d660d3590d Cannot compile in readwritesplit.c 2014-03-20 20:30:03 +02:00
288ca68677 Changed variable type which includes information of query type returned by query classifier.
As a consequence, if autocommit is enabled, active transaction(s) are implicitly committed and MaxScale detects that implicit commit.
2014-03-20 12:15:57 +02:00
fcf7d37114 Added support for router option 'synced' in read write split router. If router option is specified, backend server must be in 'joined' state to be eligible backend candidate. 2014-03-19 22:29:02 +02:00
d6a9a5c1d0 MAX-10, Transaction support for MaxScale.
Naive implementation, which routes all statements to master between BEGIN|START TRANSACTION <options> and ROLLBACK|COMMIT
2014-03-18 23:41:32 +02:00
35c460457a Fixed compile error. 2014-03-18 11:21:51 +02:00
526762bcc7 Fixed reference to backend servers. 2014-03-18 10:54:19 +02:00
72a17e5fe2 Merging SESvars with develop. 2014-03-18 10:28:34 +02:00
c356b396a6 Merge branch 'SESvars' of https://github.com/skysql/MaxScale into SESvars
Conflicts:
	server/core/dcb.c
	server/core/poll.c
	server/modules/include/mysql_client_server_protocol.h
	server/modules/routing/readwritesplit/readwritesplit.c
2014-03-18 10:28:06 +02:00
67d9b3afb9 Fixed bug in how response messages from backends were handled. Now messages are examined if session command cursor is active
. That means that (at least) the first packet  in the message is response to session command. Then if the response is alread
y forwarded to client it is discarded. Otherwise it is routed to client and the command is marked as responded so that the o
ther backend knows to discard its duplicate response.
2014-03-17 22:26:32 +02:00
ae2379be31 Server connection counter is updated in closeSession
Server connection counter is updated in closeSession
2014-03-17 11:26:35 +01:00
90f701be8e readwritesplit.c : router is changed so that it guarantees to keep the execution order of session commands and queries when they are routed to backend servers. In the same way it maintains the order of response packets and discards duplicate responses.
For each session command a sescmd property is created and added to the end of list of session commands. List is owned by router client session and it includes all session commands from the beginning of router session.
Router maintains an individual session command cursor for each backend. A cursor refers to the first session command which the corresponding backend server haven't yet responded yet.
When response message arrives at any time from backend, first it is checked whether backend's cursor is active. Cursor is active if a session command is routed to backend and the backend haven't responded to it yet. If cursor is active, next it is checked whether the current session command property has been responded by other backend. If both are true, then response message is sent to client as is. If session command response is routed to client already, the arrived response is discarded.
2014-03-16 19:43:49 +02:00
a3f7eebdc9 Extended session command support to cover COM_CHANGE_USER, and COM_INIT_DB.
This implementation doesn't guarantee execution order between session commands and queries
if other backend server lags behind in session command execution.

In poll.c : moved processing of EPOLLERR and EPOLLHUP after processing of EPOLLIN and EPOLLOUT.
This ensures that COM_QUIT messages are read and routed forward before signals arrive (from local client/backend).
2014-03-14 13:25:37 +02:00
cb6a976555 Router has now capability value which currently tells whether router session expects stream or individual, complete statements. With read con
nection router stream is used and with read/write split router individual statements are passed to router.
Added new function to ROUTER_OBJECT : uint8_t (*getCapabilities)(ROUTER *instance, void* router_session); which is implemented in every route
r.

Added support for multi-statement packets in rwsplit router. In other words, if network packet includes multiple mysql statements, they are separated and passed to router one by one.

Multi-packet statements (those which exceeds network packet boundaries) are _not_ supported yet.
2014-03-11 23:12:11 +02:00
c28892323a Added support for session commands to readwrite split router.
Added support for multi-statement packets.

This is an intermediate commit to save work. Code is not cleaned and there are debug prints and prototypes to be removed.
2014-03-07 20:53:33 +02:00
3e111534a1 Support for mysql session variable commands, for example, SET AUTOCOMMIT=0
Session commands are identified by query clasisfier, and added to the session command property list in router client session object.
Session commands are then executed in existing backend servers but only one of them will reply to client.
2014-02-26 20:07:09 +02:00
3edea0afc4 Updated gitignore files 2014-02-05 14:49:05 +01:00
c444bf454b Bug #385 http://bugs.skysql.com/show_bug.cgi?id=385
dcb.c:dcb_write accept also dcb state DCB_STATE_NOPOLLING since it only means that dcb has been removed from epoll set but it is still possible to write to it.

Bug #384 http://bugs.skysql.com/show_bug.cgi?id=384
session.h:added new state for SESSION, SESSION_STATE_ROUTER_READY which follows SESSION_STATE_READY. The difference is that ROUTER_READY is set only after router session is successfully created while READY means that session still lacks router.
session.c:set SESSION_STATE_ROUTER_READY when router is created.
mysql_backend.c:gw_read_backend_event, added SESSION_STATE_ROUTER_READY check before router session is closed. Changed chec kso that it doesn't block in infinite loop (although it shouldn't be possible anyway).
mysql_backend.c:gw_error_backend_event, added similar check before session is closed.
2014-01-27 22:56:33 +02:00
d403018fd9 Surrounded every log writing command with LOGIF macro, which tests if the given logfile id matches with those enabled. The comparison is done against local variable and expensive function call is avoided. 2013-12-12 16:14:33 +02:00
a673fc9c48 Read-write split router doesn't handle QUERY_TYPE_SESSION_WRITE in any special way. Behavior defaults to same as in the case of QUERY_TYPE_WRITE. That is, modification is executed in master only. 2013-12-09 15:30:47 +02:00
f50de7a084 build_gateway.inc
Removed MARIADB_SRC_PATH
	Added MYSQL_ROOT - root directory where MariaDB headers are installed
	Added MYSQL_HEADERS - -I notation for three header directories
	Added EMBEDDED_LIB - for embedded library directory
	Added ERRMSG - for errmsg.sys file

log_manager.cc
	Little fixes

log_manager/makefile
	Updated header include directive

makefile.inc
	Removed DEBUGGER and BACKGR as unnecessary
	Added LIB for embedded library file name

query_classifier/makefile
	Updated header include directives and embedded library directory

query_classifier/query_classifier.cc
	clean up

server/core/Makefile
	Updated header include directives and embedded library directory
	Added libaio and install of errmsg.sys

server/core/dcb.c
	clean up

server/core/gateway.c
	Added --language and --skip-innodb to mysql_library_init command-line arguments list
	clean up

server/core/load_utils.c
	clean up

server/modules/monitor/Makefile
server/modules/routing/readwritesplit/Makefile
	Updated header include directives and embedded library directory

utils/skygw_debug.h
	http://bugs.skysql.com/show_bug.cgi?id=369

utils/skygw_types.h
utils/skygw_utils.cc
	clean up
2013-12-06 00:12:41 +02:00
9ba7a0d955 log_manager.cc
Added new argument '-s' which takes additional argument composed of list of logfile identifiers. Logfiles listed with '-s' will be written on main memory instead of disk. In practice, the log file in question will be written in /dev/shm but corresponding symlink is added to log directory. In the case of name conflicts with log files and links, a differentiating sequence number is included in hte name of the file. This, however, is done only when existing file is not writable or is of different type (symlink <> file). 
	Added new logfile LOGFILE_DEBUG whose contents will be largerly what was included up to date in trace log. 

	Disabled feature which spreads writes to log files to others because of bug (#338) in the way block buffers are managed.

	Changed log manager parameters to match with the current implementation. List of arguments:
                "-h - help\n"
                "-a <debug prefix>   ............(\"skygw_debug\")\n"
                "-b <debug suffix>   ............(\".log\")\n"
                "-c <trace prefix>   ............(\"skygw_trace\")\n"
                "-d <trace suffix>   ............(\".log\")\n"
                "-e <message prefix> ............(\"skygw_msg\")\n"
                "-f <message suffix> ............(\".log\")\n"
                "-g <error prefix>   ............(\"skygw_err\")\n"
                "-i <error suffix>   ............(\".log\")\n"
                "-j <log path>       ............(\"/tmp\")\n"
                "-s <shmem log file ids> ........(no default)\n";

dcb.c
	dcb_add_to_zombieslist, add dcb to the front of zombies list instead of inserting to the end of it.

gateway.c
	Renamed shutdown_gateway to shutdown_server (Bug #131)
	Call skygw_logmanager_init so that trace and debug logs are written to shared memory.

poll.c 
dcb.h
	Removed some dead code and references to unneeded mutexes.

debugcmd.c
	Added enable/disable log command for debug log.

skygw_utils.cc
	skygw_file_init now takes optional symlink name as a second argument. Symlink is created to point to the file being created.
2013-11-08 12:56:39 +02:00
3769a02957 query_classifier.cc
resolve_query_type, added GSYSVAR_FUNC type for functions that read system variables and can be executed in Maxscale instead of backend server.

dcb.c
	dcb_read, if read returns value <= 0 and if error is EAGAIN/EWOULDBLOCK so there was nothing to read in sthe socket, that is not an error because some other thread may have read the data that was expected to be available.

mysql_backend.c
	gw_read_backend_event, used dcb->authlock to ensure that dcb's protocol state is read and modified serially. This removes issues with false authentication failures which may happen when two threads modify and read the state without any control.

mysql_client.c 
	removed dead code.

readconnroute.c, readwritesplit.c
	removed invalid assert which assumed that spinlock can not have larger value than one when itis locked.
2013-10-31 22:24:51 +02:00