66 Commits

Author SHA1 Message Date
VilhoRaatikka
5a2d492fe8 Fixed use of uninitialized memory. 2014-10-31 23:47:33 +02:00
Markus Makela
52c0d0c768 Merge remote-tracking branch 'origin/develop' into header_rename 2014-10-01 05:24:02 +03:00
VilhoRaatikka
c055a3c0d2 Addition to fix of #557, Removal of confusing and erroneous log message. 2014-09-30 17:04:43 +03:00
VilhoRaatikka
ca7ae0bb64 Fix to bug #557, http://bugs.mariadb.com/show_bug.cgi?id=557
query_classifier.cc: added function for printing combined query type from a bit field.
query_classifier.h: identify query types SHOW DATABASE, and SHOW TABLES to make log entries more understandable.
mysql_common.c: moved some trace log commands to debug log.
readwritesplit.c: moved some trace logs to debug log, added trace log commands to gather routing, query type and session information to one log entry.
skygw_debug.h: added string macros for several query and hint types.
2014-09-30 16:31:30 +03:00
Markus Makela
f4e591e382 Changed 'SkySQL Gateway' to 'MariaDB Corporation MaxScale' 2014-09-30 13:15:03 +03:00
Markus Makela
c344231f80 Renamed all occurences of SkySQL to MariaDB Corporation 2014-09-30 13:02:10 +03:00
VilhoRaatikka
2b147a9ea4 Monitors waited monitor check interval time before checking backend servers for the first time. Removed wait for the first check.
Added extra debugging to query_classifier to assist in issue resolution regarding to optimized MaxScale builds and pthread_mutex_lock in sql/sql_class.h
2014-09-24 12:03:04 +03:00
VilhoRaatikka
a41a8d6060 Fix to bug #543, http://bugs.skysql.com/show_bug.cgi?id=543
All counters are now updated in routeQuery
Fix to bug #545, http://bugs.skysql.com/show_bug.cgi?id=545
All sql variable and session modification statements, such as autocommit-, and set <db> commands are routed to all nodes.
2014-09-15 19:01:04 +03:00
VilhoRaatikka
d799331c50 Added support for fine-grained query classification which is needed for configurable support for user- and system variable execution. This is partial implementation to task MAX-242. Tests passed with make -C server/modules/routing/readwritesplit/test but this is intermediate commit. 2014-09-05 22:50:35 +03:00
Markus Makela
c00e51fd00 Merge branch 'Z3' into Z3_rabbit_partial
Conflicts:
	server/modules/routing/readwritesplit/readwritesplit.c
2014-09-04 13:52:12 +03:00
Markus Makela
4e11ea9b06 Merged some of the rabbitmq branch changes
query_classifier.cc: updated skygw_get_table_names to allow for partial or full table names
readwritesplit.c: transferred temporary table detection to separate functions
2014-09-04 13:31:20 +03:00
VilhoRaatikka
7fcf950f8f Merge branch 'Z3' of https://github.com/skysql/MaxScale into Z3
Conflicts:
	server/modules/routing/readwritesplit/readwritesplit.c
2014-09-03 22:54:32 +03:00
VilhoRaatikka
cda39a62fb Fixes to
Includes imprvements to hints processing. If hint can't be followed query is routed possibly to slave, and eventually to master if other attempts fail.
2014-09-03 22:09:50 +03:00
VilhoRaatikka
5e5c59d322 Merge branch 'MAX-237' into Z3 2014-09-03 17:50:56 +03:00
Markus Makela
cc5922b218 Merge branch 'MAX-237' into Z3 2014-09-02 16:58:20 +03:00
Markus Makela
52f3adbf20 fixed temporary tables looking for database drops instead of table drops 2014-09-01 19:50:25 +03:00
VilhoRaatikka
0fed5c2c5b Fix to bug #510, http://bugs.skysql.com/show_bug.cgi?id=510, made every MaxScale thread to call mysql_thread_init() before entering poll_waitevents. Also main thread does this before starting services. Removed all calls to mysql_thread_init() and to mysql_thread_end() from elsewhere than from poll.c:poll_waitevents and from gateway.c:main
skygw_utils.cc: replace_literal: fixed memory leak
2014-09-01 19:37:31 +03:00
VilhoRaatikka
3c1abf4b64 Cleanup 2014-09-01 13:57:31 +03:00
Markus Makela
0e53633a37 Merge remote-tracking branch 'origin/Z3' into MAX-237 2014-09-01 13:44:24 +03:00
VilhoRaatikka
20abbbdf57 query_classifier.cc:skygw_get_canonical: Fixed bug in how strings were passed to replace_literal function. Changed to use item->str_ptr which stores all strings in the same way unlike item->name in which some string values have double quotation.
skygw_utils.cc:Fixed regexp so that it detects literals at the end of line too.
2014-09-01 13:23:04 +03:00
Markus Makela
a17e584adb Merge remote-tracking branch 'origin/Z3' into MAX-237
Conflicts:
	query_classifier/query_classifier.cc
	query_classifier/query_classifier.h
	server/modules/routing/readwritesplit/readwritesplit.c
2014-09-01 11:05:10 +03:00
Markus Makela
067ce3c886 removed unneeded QUERY_TYPE_DROP_TABLE type from query_classifier.h 2014-09-01 10:35:48 +03:00
VilhoRaatikka
40f85f9cad Fix to bug http://bugs.skysql.com/show_bug.cgi?id=488. SHOW VARIABLES was treated as if it was session write command. Instead it is a read-only query. Changed to route that to master. 2014-08-31 22:56:30 +03:00
Markus Makela
58e8c05c8a added detection of drop table targeting a temporary table 2014-08-31 20:19:47 +03:00
Markus Makela
7629c455a6 partial implementation 2014-08-30 08:27:05 +03:00
Markus Makela
7ea53f0141 Merge remote-tracking branch 'origin/MAX-160' into MAX-237
Conflicts:
	query_classifier/query_classifier.cc
	query_classifier/query_classifier.h
2014-08-29 11:02:03 +03:00
VilhoRaatikka
531d8d7b47 query_classifier.cc: added detection for CREATE TEMPORARY TABLE and setting a new query type QUERY_TYPE_CREATE_TMP_TABLE for it.
query_classifier.h: added QUERY_TYPE_CREATE_TMP_TABLE and QUERY_TYPE_READ_TMP_TABLE for use of temporary table support.
hashtable.c:Added variant of hashtable which is 'flat', that is, stored to existing memory instead of allocating memory as a part of the call. Existing function declarations don't change but added hashtable_alloc_flat for the purpose. Both hashtable_alloc and hashtable_alloc_flat now call the real allocation function, hashtable_alloc_real. hashtable_free only frees memory which is allocated in hashtable_alloc_real.
hashtable.h: added a flag to HASHTABLE struct to indicate whether hashtable owns its memory or not.
readwritesplit.h: Added RSES_PROP_TYPE_TMPTABLES property type to be used for keeping the hashtable for tablenames.
readwritesplit.c: Added comments about temporary table support implementation.
2014-08-29 10:08:48 +03:00
VilhoRaatikka
69104d7dee skygw_utils.cc:replace_literal: Fixed regular expression which, for example, accepted "200" with needle "2"
query_classifier.cc: fixed invalid argument list in logging command.
input.sql: added a few previously failed cases for canonical query test
2014-08-26 11:08:05 +03:00
VilhoRaatikka
c5fbb1f295 query_classifier.cc:parsing_info_done: calling mysql_thread_end caused segfauls when same thread tried next time call free_embedded_thd because thread's sysvar was set to NULL in mysql_thread_end.
Added a few lines to canonical query test's input script which are not handled correctly.
2014-08-25 22:17:21 +03:00
VilhoRaatikka
ee52ac64a9 query_classifier.cc:skygw_get_canonical: if the item to be replaced is an empty string "", it is processed differently from the other cases due to difficulties to get wanted result by adding special rule for that to regex.
query_classifier.h: added parsing information structure to query classifier away from buffer.h.
buffer.c:introduced a buffer object which includes object pointer and a clean-up call-back function. Buffer objects form a list which is cleaned up by the last referrer of the buffer, as a part of gwbuf_free. Buffer object list is protected by a spinlock gwbuf_lock.
	Also added identifier type, bufobj_id_t which is enumerated type and currently includes one value only, GWBUF_PARSING_INFO. Added also a bitfield for information about the buffer. It currently has one type only, GWBUF_INFO_PARSED indicating that buffer content is parsed and there is buffer object of type GWBUF_PARSING_INFO.
skygw_utils.cc:replace_literal:changed regexec matching to case insensitive because user-defined literals are sometimes converted to upper-case ones.
2014-08-22 19:01:56 +03:00
VilhoRaatikka
3bc88e4eb9 skygw_get_canonical didn't return NULL pointerin cases where parsing info didn't exist or something went wrong. 2014-08-21 23:31:23 +03:00
VilhoRaatikka
3a5b8ef64c query_classifier.cc: cleaned up and simplified skygw_get_canonical
skygw_util.cc:fixed memory allocation issue where terminating byte wasn't counted. Added some error checks.
2014-08-21 23:08:21 +03:00
VilhoRaatikka
fa2189373d query_classifier.cc: query_is_parsed is now global function and can be used to check whether parsing information already exists in the buffer.
skygw_utils.cc: removed replace_str and implemented replace_literal which replaces user-provided literals in query with predefined string "?". Replacing is done one by one, so it is suboptimal ipmlementation since all literals could be passed to replacement function in one call and processed all before returning.
2014-08-21 22:28:23 +03:00
VilhoRaatikka
c501d4d4e1 Changes related to canonical query format implementation.
query_classifier.cc: Now query can be parsed outside query_classifier_get_type by calling function parse_query. It creates parsing_info_t struct which is then added to the GWBUF which also includes the query. Parsing information follows the buffered query and it is freed at the same time with query buffer, in gwbuf_free.
buffer.c: additions of parsing information to gwbuf struct.
modutil.c: added function which returns query from GWBUF in plain text string.
readwritesplit.c:routeQuery now only calls query_classifier_get_type to get the query type instead of extracting plain text query from the GWBUF buffer.
2014-08-20 22:10:36 +03:00
VilhoRaatikka
fb3a950a18 Completed skygw_get_canonical by adding NULL checks and debug assertions. Replacable literal types are now INT_ITEM, STRING_ITEM, DECIMAL_ITEM, REAL_ITEM, VARBIN_ITEM and NULL_ITEM. 2014-08-19 09:07:18 +03:00
VilhoRaatikka
544e64a301 query_classifier: implemented skygw_get_canonical which returns a copy of original string with given substrings being replaced with questionmarks.
skygw_utils.cc: added string replacement function for use of skygw_get_canonical
2014-08-18 14:19:46 +03:00
VilhoRaatikka
7558abb6fa Bug #468, http://bugs.skysql.com/show_bug.cgi?id=468, Query classifier accessed freed thread context. If parsing fails thd doesn't need to be freed because it holds correct information about command type.
session.c:session_setup_filters : fixed memory leak
hintparser.c: added token_free for HINT_TOKENs and fixed a few memory leaks.
mysql_client_server_protocol.h: added mysql_protocol_done which frees memory blocks pointed to by protocol members. Those can't be freed in dcb.c because dcb.c doesn't know about protocol's members.
mysql_backend.c:gw_backend_close: fixed memory leak
mysql_client.c: gw_client_close: fixed memory leak
mysql_common.c: added implementation of mysql_protocol_done
    :protocol_archive_srv_command: tried to fix memory leak. Some memory is still leaking according to valgrind. Removed use of uninitialized local variable len.
readwritesplit.c:execute_sescmd_in_backend: fixed a memory leak - visible only in DEBUG=Y build.
2014-08-05 10:42:13 +03:00
VilhoRaatikka
7ff14e23a5 Support for prepared statement, namely support for following comands : COM_STMT_PREPARE, COM_STMT_EXECUTE, COM_STMT_SEND_LONG_DATA, COM_STMT_RESET, COM_STMT_CLOSE, SQLCOM_PREPARE, SQLCOM_EXECUTE, SQLCOM_DEALLOCATE_PREPARE (DEALLOCATE/DROP PREPARE stmt).
All prepare commands are executed in every backend server currently connected.

All executes are routed to master. If stmt type was recorded in prepare phase in rwsplit router, read-only stmts could be routed to slaves.

COM_STMT_PREPARE gets arbitrary number of response packets from backend database. Since statements are prepared in every backend server and only one multi-packet response can be replied to client, redundant multi-packet responses are discarded. This is done in router. Mechanisms from session command handling are utilized with little changes: router must identify when response consists of multiple packets so that it knows to calculate the number of packets in response and that it is able to discard correct number of packets.

Information to the reply-handling router is provided by backend protocol, which includes a ordered list of commands of commands sent to protocol-owning backend server. A command is stored to protocol struct in mysql_backend.c:gw_MySQLWrite_backend if the statement buffer's type has GWBUF_TYPE_SINGLE_STMT set in mysql_client.c:route_by_statement. GWBUF_TYPE_SINGLE_STATEMENT indicates that there is single statement in the buffer, as opposite to Read Connection router, which accepts streaming input from client.
2014-06-25 17:15:46 +03:00
VilhoRaatikka
619aeb4afa Query classifier can keep parse tree after the function that created it returns. Parse tree can then be queried from outside. Added skygw_query_classifier_get_stmtname which returns the name of statement, and skygw_query_classifier_free which frees mysql handle, parse tree and thread context (thd). 2014-06-24 21:45:00 +03:00
VilhoRaatikka
7e6cb7afc2 Snapshot of failure tolerance changes.
Added a lot of logging to error, trace and message logs which should help the user to handle errors which can't be automatically resolved, like attempt to use nonexisting database.
2014-06-06 23:32:04 +03:00
VilhoRaatikka
b5e9428ff7 log_manager.cc fixed memory leak, block buffer mutex names weren't freed.
query_classifier.cc use of uninitialized value in skygw_stmt_causes_implicit_commit
config.c crashed if module load failed, use of unitialized value
load_utils.c pretty-printed error
service.c use of uninitialized value in service_add_qualified_param
modules.h function prototype
readwritesplit.c memory leaks
2014-04-29 14:50:09 +03:00
VilhoRaatikka
c927057b5c Fixed two bugs of which one was older.
1. in query_classifier.cc autocommit_enabled, and transaction_active variables maintained their values across different sessions. Now those values are stored in each router_client_ses object.
2. As a part of implementation of MAX-95 session variables were added to BACKEND struct which is shared with all sessions using the SERVICE which the particular BACKEND serves. Now each router_client_ses object has a backend reference struct which includes pointer to BACKEND, DCB and to session command cursor.

Added test - set_autocommit_disabled.sql, test_after_autocommit_disabled.sql - to check that session variable is discarded when session where it belongs terminates.
2014-04-28 23:33:49 +03:00
VilhoRaatikka
c994c81a0e Changed the way how autocommit is searched from parse tree. Switched to use typelib instead of explicitly converting and comparing user value. Added more tests for cases where user uses literal values in SET autocommit command. 2014-04-15 14:32:47 +03:00
VilhoRaatikka
26c6c331fd Changed is_autocommit_stmt so that client character set can very and the value is compared to strings '0','1','off','on' which are converted to match with the cs of the client prior compare. 2014-04-11 20:02:43 +03:00
VilhoRaatikka
bbf1bc2afa Fix for bug #418
Increased skygw_query_type_t to 16 bits, and corrected the way how those bit fields are checked.
Added tests for cases where autocommit is disabled and corrected old tests.
2014-04-10 17:09:42 +03:00
VilhoRaatikka
2c17dc3edf Bug #418, added functions to query classifier to detect if SET autocommit is called.
Note: this compiles but doesn't work yet properly.
2014-04-09 23:43:03 +03:00
VilhoRaatikka
7c3a354fd8 Query classifier ignored implicit commits in cases of write commands. Fixed it.
Added more tests for transaction support. Mostly different cases where some command triggers implicit commit in the middle of transaction.
2014-03-28 00:16:18 +02:00
VilhoRaatikka
f380c707f9 In resolve_query_type type can be QUERY_TYPE_COMMIT even if lex->option_type is OPT_DEFAULT. 2014-03-26 22:33:57 +02:00
VilhoRaatikka
f49df89a0c Changed how query classifier determines which statements trigger implicit commit.
Changed test makefile and rwsplit.sh script and added two example sql scripts.
2014-03-26 19:10:35 +02:00
VilhoRaatikka
a137196de0 Added trace for spotting out commands which cause implicit commit before or after they are executed. 2014-03-25 11:41:46 +02:00