520 Commits

Author SHA1 Message Date
Markus Makela
a17e584adb Merge remote-tracking branch 'origin/Z3' into MAX-237
Conflicts:
	query_classifier/query_classifier.cc
	query_classifier/query_classifier.h
	server/modules/routing/readwritesplit/readwritesplit.c
2014-09-01 11:05:10 +03:00
Markus Makela
067ce3c886 removed unneeded QUERY_TYPE_DROP_TABLE type from query_classifier.h 2014-09-01 10:35:48 +03:00
VilhoRaatikka
40f85f9cad Fix to bug http://bugs.skysql.com/show_bug.cgi?id=488. SHOW VARIABLES was treated as if it was session write command. Instead it is a read-only query. Changed to route that to master. 2014-08-31 22:56:30 +03:00
Markus Makela
58e8c05c8a added detection of drop table targeting a temporary table 2014-08-31 20:19:47 +03:00
Markus Makela
7629c455a6 partial implementation 2014-08-30 08:27:05 +03:00
VilhoRaatikka
ed94f08b4e Added rule version.h: $(ROOT_PATH)/VERSION 2014-08-29 11:37:41 +03:00
Markus Makela
7ea53f0141 Merge remote-tracking branch 'origin/MAX-160' into MAX-237
Conflicts:
	query_classifier/query_classifier.cc
	query_classifier/query_classifier.h
2014-08-29 11:02:03 +03:00
VilhoRaatikka
531d8d7b47 query_classifier.cc: added detection for CREATE TEMPORARY TABLE and setting a new query type QUERY_TYPE_CREATE_TMP_TABLE for it.
query_classifier.h: added QUERY_TYPE_CREATE_TMP_TABLE and QUERY_TYPE_READ_TMP_TABLE for use of temporary table support.
hashtable.c:Added variant of hashtable which is 'flat', that is, stored to existing memory instead of allocating memory as a part of the call. Existing function declarations don't change but added hashtable_alloc_flat for the purpose. Both hashtable_alloc and hashtable_alloc_flat now call the real allocation function, hashtable_alloc_real. hashtable_free only frees memory which is allocated in hashtable_alloc_real.
hashtable.h: added a flag to HASHTABLE struct to indicate whether hashtable owns its memory or not.
readwritesplit.h: Added RSES_PROP_TYPE_TMPTABLES property type to be used for keeping the hashtable for tablenames.
readwritesplit.c: Added comments about temporary table support implementation.
2014-08-29 10:08:48 +03:00
VilhoRaatikka
cf5821d4ef Added expected results for missing queries. 2014-08-26 11:14:51 +03:00
VilhoRaatikka
69104d7dee skygw_utils.cc:replace_literal: Fixed regular expression which, for example, accepted "200" with needle "2"
query_classifier.cc: fixed invalid argument list in logging command.
input.sql: added a few previously failed cases for canonical query test
2014-08-26 11:08:05 +03:00
VilhoRaatikka
fd125f4a42 Merge branch 'MAX-160' of https://github.com/skysql/MaxScale into MAX-160
Conflicts:
	query_classifier/test/canonical_tests/input.sql
2014-08-25 22:22:44 +03:00
VilhoRaatikka
c5fbb1f295 query_classifier.cc:parsing_info_done: calling mysql_thread_end caused segfauls when same thread tried next time call free_embedded_thd because thread's sysvar was set to NULL in mysql_thread_end.
Added a few lines to canonical query test's input script which are not handled correctly.
2014-08-25 22:17:21 +03:00
Markus Makela
4ab7112db7 more test cases for canonical queries and makefile typo fixes 2014-08-25 15:24:23 +03:00
Markus Makela
662b4b00ee the errmsg.sys file is now copied during test building 2014-08-22 20:50:54 +03:00
Markus Makela
73707cc9da canonical query tests for query classifier 2014-08-22 20:44:26 +03:00
VilhoRaatikka
ee52ac64a9 query_classifier.cc:skygw_get_canonical: if the item to be replaced is an empty string "", it is processed differently from the other cases due to difficulties to get wanted result by adding special rule for that to regex.
query_classifier.h: added parsing information structure to query classifier away from buffer.h.
buffer.c:introduced a buffer object which includes object pointer and a clean-up call-back function. Buffer objects form a list which is cleaned up by the last referrer of the buffer, as a part of gwbuf_free. Buffer object list is protected by a spinlock gwbuf_lock.
	Also added identifier type, bufobj_id_t which is enumerated type and currently includes one value only, GWBUF_PARSING_INFO. Added also a bitfield for information about the buffer. It currently has one type only, GWBUF_INFO_PARSED indicating that buffer content is parsed and there is buffer object of type GWBUF_PARSING_INFO.
skygw_utils.cc:replace_literal:changed regexec matching to case insensitive because user-defined literals are sometimes converted to upper-case ones.
2014-08-22 19:01:56 +03:00
VilhoRaatikka
3bc88e4eb9 skygw_get_canonical didn't return NULL pointerin cases where parsing info didn't exist or something went wrong. 2014-08-21 23:31:23 +03:00
VilhoRaatikka
3a5b8ef64c query_classifier.cc: cleaned up and simplified skygw_get_canonical
skygw_util.cc:fixed memory allocation issue where terminating byte wasn't counted. Added some error checks.
2014-08-21 23:08:21 +03:00
VilhoRaatikka
fa2189373d query_classifier.cc: query_is_parsed is now global function and can be used to check whether parsing information already exists in the buffer.
skygw_utils.cc: removed replace_str and implemented replace_literal which replaces user-provided literals in query with predefined string "?". Replacing is done one by one, so it is suboptimal ipmlementation since all literals could be passed to replacement function in one call and processed all before returning.
2014-08-21 22:28:23 +03:00
VilhoRaatikka
c501d4d4e1 Changes related to canonical query format implementation.
query_classifier.cc: Now query can be parsed outside query_classifier_get_type by calling function parse_query. It creates parsing_info_t struct which is then added to the GWBUF which also includes the query. Parsing information follows the buffered query and it is freed at the same time with query buffer, in gwbuf_free.
buffer.c: additions of parsing information to gwbuf struct.
modutil.c: added function which returns query from GWBUF in plain text string.
readwritesplit.c:routeQuery now only calls query_classifier_get_type to get the query type instead of extracting plain text query from the GWBUF buffer.
2014-08-20 22:10:36 +03:00
VilhoRaatikka
fb3a950a18 Completed skygw_get_canonical by adding NULL checks and debug assertions. Replacable literal types are now INT_ITEM, STRING_ITEM, DECIMAL_ITEM, REAL_ITEM, VARBIN_ITEM and NULL_ITEM. 2014-08-19 09:07:18 +03:00
VilhoRaatikka
544e64a301 query_classifier: implemented skygw_get_canonical which returns a copy of original string with given substrings being replaced with questionmarks.
skygw_utils.cc: added string replacement function for use of skygw_get_canonical
2014-08-18 14:19:46 +03:00
Hartmut Holzgraefe
d355e07e2d gitignore cleanup
* ignore typical backup files created by common editors
* move general ignore rules like "*.o" or "depend.mk" to top level gitignore
* ignore executables and test directories in target dir gitignore
  as these are local and there's no general catch-all pattern for them
2014-08-18 09:39:29 +00:00
Hartmut Holzgraefe
fb87f68368 consistently use DEL instead of rm in Makefiles 2014-08-17 21:02:24 +00:00
VilhoRaatikka
4ef1df4e9d Bug #468, http://bugs.skysql.com/show_bug.cgi?id=468, Query classifier accessed freed thread context. If parsing fails thd doesn't need to be freed because it holds correct information about command type. 2014-08-05 13:44:04 +03:00
VilhoRaatikka
b40a1a9ed8 Merge branch 'develop' of https://github.com/skysql/MaxScale into develop 2014-07-04 11:30:14 +03:00
VilhoRaatikka
307bb6ad98 Disabled query classifier tests from testset until tests are fixed. Current version doesn't even compile. 2014-07-04 11:29:26 +03:00
Mark Riddoch
5251244ddd Fix missing argument to skygw_query_classifier_get_type in test code 2014-07-04 08:44:42 +01:00
VilhoRaatikka
7ff14e23a5 Support for prepared statement, namely support for following comands : COM_STMT_PREPARE, COM_STMT_EXECUTE, COM_STMT_SEND_LONG_DATA, COM_STMT_RESET, COM_STMT_CLOSE, SQLCOM_PREPARE, SQLCOM_EXECUTE, SQLCOM_DEALLOCATE_PREPARE (DEALLOCATE/DROP PREPARE stmt).
All prepare commands are executed in every backend server currently connected.

All executes are routed to master. If stmt type was recorded in prepare phase in rwsplit router, read-only stmts could be routed to slaves.

COM_STMT_PREPARE gets arbitrary number of response packets from backend database. Since statements are prepared in every backend server and only one multi-packet response can be replied to client, redundant multi-packet responses are discarded. This is done in router. Mechanisms from session command handling are utilized with little changes: router must identify when response consists of multiple packets so that it knows to calculate the number of packets in response and that it is able to discard correct number of packets.

Information to the reply-handling router is provided by backend protocol, which includes a ordered list of commands of commands sent to protocol-owning backend server. A command is stored to protocol struct in mysql_backend.c:gw_MySQLWrite_backend if the statement buffer's type has GWBUF_TYPE_SINGLE_STMT set in mysql_client.c:route_by_statement. GWBUF_TYPE_SINGLE_STATEMENT indicates that there is single statement in the buffer, as opposite to Read Connection router, which accepts streaming input from client.
2014-06-25 17:15:46 +03:00
VilhoRaatikka
619aeb4afa Query classifier can keep parse tree after the function that created it returns. Parse tree can then be queried from outside. Added skygw_query_classifier_get_stmtname which returns the name of statement, and skygw_query_classifier_free which frees mysql handle, parse tree and thread context (thd). 2014-06-24 21:45:00 +03:00
VilhoRaatikka
7e6cb7afc2 Snapshot of failure tolerance changes.
Added a lot of logging to error, trace and message logs which should help the user to handle errors which can't be automatically resolved, like attempt to use nonexisting database.
2014-06-06 23:32:04 +03:00
VilhoRaatikka
b5e9428ff7 log_manager.cc fixed memory leak, block buffer mutex names weren't freed.
query_classifier.cc use of uninitialized value in skygw_stmt_causes_implicit_commit
config.c crashed if module load failed, use of unitialized value
load_utils.c pretty-printed error
service.c use of uninitialized value in service_add_qualified_param
modules.h function prototype
readwritesplit.c memory leaks
2014-04-29 14:50:09 +03:00
VilhoRaatikka
c927057b5c Fixed two bugs of which one was older.
1. in query_classifier.cc autocommit_enabled, and transaction_active variables maintained their values across different sessions. Now those values are stored in each router_client_ses object.
2. As a part of implementation of MAX-95 session variables were added to BACKEND struct which is shared with all sessions using the SERVICE which the particular BACKEND serves. Now each router_client_ses object has a backend reference struct which includes pointer to BACKEND, DCB and to session command cursor.

Added test - set_autocommit_disabled.sql, test_after_autocommit_disabled.sql - to check that session variable is discarded when session where it belongs terminates.
2014-04-28 23:33:49 +03:00
VilhoRaatikka
c994c81a0e Changed the way how autocommit is searched from parse tree. Switched to use typelib instead of explicitly converting and comparing user value. Added more tests for cases where user uses literal values in SET autocommit command. 2014-04-15 14:32:47 +03:00
VilhoRaatikka
26c6c331fd Changed is_autocommit_stmt so that client character set can very and the value is compared to strings '0','1','off','on' which are converted to match with the cs of the client prior compare. 2014-04-11 20:02:43 +03:00
VilhoRaatikka
7b40c1ee70 Changed how MaxScale unit test works.
All tests are run by executing 'make testall' in root directory. As a result all directories which contain tests will be entered and tests executed.
After tests, each directory have a test log including the last run's logs only.
Created a global test log, which is specified in test.inc. Test logs from subdirectories are concatenated to this file.
2014-04-10 19:08:28 +03:00
VilhoRaatikka
bbf1bc2afa Fix for bug #418
Increased skygw_query_type_t to 16 bits, and corrected the way how those bit fields are checked.
Added tests for cases where autocommit is disabled and corrected old tests.
2014-04-10 17:09:42 +03:00
VilhoRaatikka
2c17dc3edf Bug #418, added functions to query classifier to detect if SET autocommit is called.
Note: this compiles but doesn't work yet properly.
2014-04-09 23:43:03 +03:00
VilhoRaatikka
7c3a354fd8 Query classifier ignored implicit commits in cases of write commands. Fixed it.
Added more tests for transaction support. Mostly different cases where some command triggers implicit commit in the middle of transaction.
2014-03-28 00:16:18 +02:00
VilhoRaatikka
f380c707f9 In resolve_query_type type can be QUERY_TYPE_COMMIT even if lex->option_type is OPT_DEFAULT. 2014-03-26 22:33:57 +02:00
VilhoRaatikka
f49df89a0c Changed how query classifier determines which statements trigger implicit commit.
Changed test makefile and rwsplit.sh script and added two example sql scripts.
2014-03-26 19:10:35 +02:00
VilhoRaatikka
a137196de0 Added trace for spotting out commands which cause implicit commit before or after they are executed. 2014-03-25 11:41:46 +02:00
VilhoRaatikka
288ca68677 Changed variable type which includes information of query type returned by query classifier.
As a consequence, if autocommit is enabled, active transaction(s) are implicitly committed and MaxScale detects that implicit commit.
2014-03-20 12:15:57 +02:00
VilhoRaatikka
d6a9a5c1d0 MAX-10, Transaction support for MaxScale.
Naive implementation, which routes all statements to master between BEGIN|START TRANSACTION <options> and ROLLBACK|COMMIT
2014-03-18 23:41:32 +02:00
VilhoRaatikka
91afdffa9a Added -lrt 2014-02-07 12:23:50 +02:00
VilhoRaatikka
75af5ee03c Merge https://github.com/skysql/MaxScale 2014-02-07 11:11:47 +02:00
VilhoRaatikka
e8ce2268e0 Test system for MaxScale.
Every test/makefile have the following targets:
 cleantests 	- clean local and subdirectories' tests
 buildtests	- build all local and subdirectories' tests
 runtests	- run all local tests
 testall	- clean, build and run local and subdirectories' tests

Tests for directory random_dir are always in its subdirectory, in this case in random_dir/test . If random_dir has subdirectories with tests, random_dir/child_dir, for example, tests of child_dir can be started from random_dir/test/makefile where make -C child_dir/test <test target> is called.

See MAXSCALE_HOME/test/README for further information.
2014-02-07 11:06:37 +02:00
Mark Riddoch
3edea0afc4 Updated gitignore files 2014-02-05 14:49:05 +01:00
VilhoRaatikka
0a13bf9989 Revert previous change to query_classifier.cc, which was triggered due to bug # 391, http://bugs.skysql.com/show_bug.cgi?id=391
Bug was identified by test, but it couldn't be validated. Using RW Split router currently may lead to situation where after setting system variable there is a backend server which doesn't have the value set. This is an incomplete feature at the moment rather than a bug.
2014-01-30 12:51:11 +02:00
VilhoRaatikka
89de86320e Added detection for USE <db> (= DATABASE()), which is a system function which changes session variable. Therefore it must be treted as SESION write and executed in both master and slave. 2014-01-29 23:29:32 +02:00