Commit Graph

227 Commits

Author SHA1 Message Date
8a0455c2fe MXS-1196: Allow delimiters to be multi-character
In some new test files, the delimiter is 2 characters.
2017-06-28 21:33:04 +02:00
cd6e9fa44b MXS-1196: Extend flags of compare
Sometimes you want to know whether the parsing using different
queryclassifiers differs, irrespective of whether they agree
upon the outcome or not.

With -R it is now possible to cause a difference in the return
value of qc_parse() to be printed.
2017-06-28 21:33:04 +02:00
50b2316fa7 MXS-1249: Test json recognition
If the version if >= 10.2.3 recognized/builtin Json function should
be typed as QUERY_TYPE_READ while unrecognized ones should be typed
as QUERY_TYPE_READ | QUERY_TYPE_WRITE.
2017-06-21 13:28:59 +03:00
08a0883ec8 Fix missed change dates
A few files were missed.
2017-06-01 19:39:41 +03:00
f546a17e77 Update change date of 2.2 2017-06-01 10:24:20 +03:00
8e2c7fd952 Merge branch '2.0' into 2.1 2017-05-10 09:12:42 +03:00
d70dad260a Implement collecting behaviour in qc_sqlite
By default, only the essentials - the type and the operation - of
a statement will be collected and only if fields, tables, functions
and databases are explicitly asked for, will they be collected.
However, a statement will be parsed at most twice; if parsing is
needed a second time then all information will be collected.

If it is known that some particular information is needed, then
qc_parse() can be called explicitly to ensure it is collected
at first parsing.
2017-03-22 11:09:36 +02:00
d0a9571da0 Introduce new qc_parse() prototype
It is now possible to specify what information the caller is interested
in. With this the cost for collecting information during the query parsing
that nobody is interested in can be avoided.
2017-03-22 11:09:36 +02:00
a58f944f23 Allow query classifier to initialize itself
The process and thread initialization/finalization of the query
classifier plugins is handled using the process and thread
initialization/finalization functions in the module object.

However, the top-level query classifier will also need to perform
process and thread initialization when transaction boundaries are
detected using regular expressions.
2017-03-15 09:35:15 +02:00
20d89717c7 Remove unused headers
The my_config.h headers are not used.
2017-03-09 13:02:23 +02:00
a6ae60c808 Merge branch '2.0' into 2.1 2017-03-07 11:45:49 +02:00
dd07feda07 MXS-1025: Parse SAVEPOINT statements
The various forms of the SAVEPOINT/RELEASE/ROLLBACK statements are now
fully parsed.
2017-03-07 08:44:21 +02:00
5648f708af Update license to BSL 1.1 2017-02-14 21:42:28 +02:00
6e38276a20 Rename files, removing "gw"-prefix
Also gwdirs.h.in -> paths.h.in
2017-01-20 12:55:52 +02:00
16a76fcd28 Rename qc_get_type() to qc_get_type_mask()
To make it plain that it is a mask that is returned and not a
specific value from the enum qc_query_type_t.
2017-01-18 08:34:53 +02:00
20034b6fd6 QC: Exclude statement that qc_sqlite does not parse 2017-01-17 14:51:19 +02:00
3d92263cb3 Format query classifier modules
Formatted query classifier modules with Astyle.
2017-01-17 14:48:43 +02:00
cc6a3e629e Implement qc_get_preparable_stmt
Now returns the preparable statement as a GWBUF on which the
other qc-functions can be used.
2017-01-17 10:10:38 +02:00
8a95a0f045 Remove qc_get_prepare_operation
Since the whole preparable statement will be available, it is
superfluous to provide a function using which the operation of
the preparable statement can be obtained.
2017-01-17 10:10:38 +02:00
7fee90a399 Streamline query classifier API interface
- Only types of fixed size used in API interface
- The actual function return value specifies whether the parsing
  process succeeded, while "logical" return values are returned
  as out arguments.

The wrapper function currently ignores the function return value.
2017-01-17 10:10:38 +02:00
0ce7632f57 QC: Remove qc_is_real_query
Not used.
2017-01-16 14:17:56 +02:00
482fbe6400 qc: Implement qc_get_function_info for qc_mysqlembedded
MXS-1070

Now both qc_mysqlembedded and qc_sqlite return the same stuff
for the same statement, and both include also operators in
addition to pure functions. Whether that is the right approach,
is still subject to debate.

However, if we want to make it possible to disable e.g. the
use of concat as in "select concat(a) from t", where a is a string,
to prevent the bypassing of the masking filter, then conceptually
it should be possible to prevent "select a+0 from t", where a is an
int, as well.
2017-01-09 09:17:38 +02:00
d078f56dc5 qc: Reveal function usage
MXS-1070

Just interface changes. Implementation follow in separate changes.
2017-01-09 09:17:38 +02:00
8fc5bdc2f1 Perform query classifier initialization implicitly
The setting up and the initialization of the query classifier has
now been separated. The gateway explicitly sets up the query
classifier (i.e. chooses which one to use and what arguments to
provide), but the actual initialization is performed as part of
the general module initialization.
2017-01-09 09:16:35 +02:00
530c0e9617 QC: Setup and initialization separated
This is in preparation for using the general process/thread
initialization mechanism.
2017-01-09 09:16:35 +02:00
aa2de52054 Factor out .test-reading capability
The capability for reading MySQL/MariaDB .test-files has now been
factored out from the compare.cc test program. That way, the
functionality can be used from other test programs as well.
2016-12-13 12:15:57 +02:00
994299b4f1 MXS-1043: Handle @@identity like @@last_insert_id
The type of @@identity, @@last_insert_id and last_insert_id() is
now the same, that is, QUERY_TYPE_READ|QUERY_TYPE_MASTER_READ.
2016-12-02 11:38:27 +02:00
0218ac9e9c MXS-1043: Handle @@identity like @@last_insert_id
The type of @@identity, @@last_insert_id and last_insert_id() is
now the same, that is, QUERY_TYPE_READ|QUERY_TYPE_MASTER_READ.
2016-12-02 11:35:50 +02:00
3ba8525063 Remove old .gitignore files
The files were used to deal with old Makefile builds.
2016-11-28 14:37:26 +02:00
d9642dd5ae MXS-1025: All logging now behing the log_level
Earlier a successful parsing but failure to classify was always
logged.
2016-11-23 14:47:24 +02:00
35620f8dae Build compare using embedded library
For whatever reason compare fails to load qc_mysqlembedded if
it has not been built with the embedded library. Needs to be
sorted out at some point.
2016-11-23 14:35:46 +02:00
dcd98900ea Remove dependency on embedded library
The test programs of the query classifier do not need the
embedded library headers.
2016-11-23 10:14:50 +02:00
9e4ee0323d Merge branch '2.0' into develop 2016-11-21 12:23:30 +02:00
d640291023 Add crashing test-case
With 2.0.1 or earlier, if a statement contains a trailing NULL,
the statement will inside qc_sqlite.c incorrectly be assumed not
to be the one to be classified with a crash being indirectly the
result.
2016-11-18 16:00:57 +02:00
59ee5a78c9 MXS-969: Report user var modifications properly
User variable modifications are now reported as QUERY_TYPE_USERVAR_WRITE
and not as QUERY_TYPE_GSYSVAR_WRITE as earlier.
2016-11-16 15:26:06 +02:00
15e9652c46 qc: Provide information about field usage
Together with the field names, now qc_get_field_info also returns
field usage information, that is, in what context a field is used.
This allows, for instance, the cache to take action if a a particular
field is selected (SELECT a FROM ...), but not if it is used in a
GROUP BY clause (...GROUP BY a).

This caused a significant modifications of qc_mysqlembedded that
earlier did not walk the parse-tree, but instead looped over of a
list of st_select_lex instances that, the name notwithstanding,
also contain information about other things but SELECTs. The former
approach lost all contextual information, so it was not possible
to know where a particular field was used.

Now the parse tree is walked, which means that the contextual
information is known, and thus the field usage can be updated.
2016-11-14 09:35:16 +02:00
bf62f8950a Remove qc_get_affected_fields
Function is no longer used and it was quite unoptimal, so now
removed.

qc_get_prepare_name, qc_get_prepare_operation and qc_get_field_info
that were missing from qc_dummy added at the same time.
2016-11-07 12:28:58 +02:00
568153efb9 Fix buflen calculation in classify
qc_sqlite.c is now strict as far as buffers and payloads goes.
2016-11-07 10:10:26 +02:00
c71acecce3 MXS-884: qc_mysqlembedded now supports qc_get_field_info
Initial implementation, one failure in the tests still to be
sorted out. Plus some cleanup.
2016-11-07 10:10:26 +02:00
fa5a858582 MXS-884: Implement qc_get_fields_infos.
We now collect more information about a particular field and
then, if necessary, copy the data over into affected_fields if
someone is interested in that.

The comparison program is extended as well, but qc_get_fields_infos()
is not tested, because qc_mysqlembedded does not implement this yet.
2016-11-07 10:10:26 +02:00
91e36ecaed Make PREPARE operation explicit
The operation of the statement to be prepared is no longer
reported as the operation of the PREPARE statement.

Instead, when the type of the statement is
QUERY_TYPE_PREPARE_NAMED_STMT, the operation can be obtained
using qc_get_prepare_operation().

The qc_mysqlembedded implementation will be provided in a
subsequent commit.
2016-10-31 13:44:48 +02:00
7499c1e1bf qc: Add minimal support for qc_get_prepare_name
If the query is a PREPARE or EXECUTE statement, then qc_get_prepare_name
will return the name of the prepared statement.
2016-10-26 14:36:37 +03:00
dc97de57c2 qc: Some minor cleanup
- qc_types_to_string renamed to qc_typemask_to_string
- qc_get_qtype_str removed, duplicate of qc_typemask_to_string
2016-10-26 10:21:11 +03:00
8f8823dc41 qc_sqlite: Protect against misuse
- Ensure contiguous buffer of expected size.
- Ensure COM_QUERY content.
2016-10-25 20:56:14 +03:00
e1ccc8afe0 MXS-938: Recognize transaction start
Now more information about a transaction start is provided. When
a transaction start statement is parsed, the type of the statement
with be QUERY_TYPE_BEGIN_TRX anded with QUERY_TYPE_READ or
QUERY_TYPE_WRITE if the transaction was explicitly started as READ
ONLY or READ WRITE.

Now also BEGIN WORK and [COMMIT|ROLLBACK] WORK are recognized.
"AND CHAIN" will still need to be recognized.
2016-10-20 21:21:38 +03:00
1333da0712 Remove skygw_utils.h
The general purpose stuff in skygw_utils.h was moved to utils.h
and the corresponding implementation from skygw_utils.cc to utils.c.
Includes updated accordingly.

Skygw_utils.h is now only used by log_manager and by mlist, which
is only used by log_manager. Consequently, skygw_utils.h was moved
to server/maxscale.

Utils.h needs a separate overhaul.
2016-10-14 19:50:54 +03:00
03dbc6df80 Remove dependency on skygw_utils.h
- STRERROR_BUFLEN moved to cdefs.h and renamed to MXS_STRERROR_BUFLEN.
  Better would be to provide a 'const char* mxs_strerror(int errno)'
  that would have a thread specific buffer for the error message.
- MIN and MAX also moved to defs.h as MXS_MIN and MXS_MAX.
- Now only mlist.h of the headers depend upon skygw_utils.h.
2016-10-14 15:42:02 +03:00
3a18b64bdd mysql_client_server_protocol.h moved.
Moved to include/maxscale/protocol/mysql.h
2016-10-13 22:59:39 +03:00
e41589be10 Move headers from server/include to include/maxscale
- Headers now to be included as <maxscale/xyz.h>
- First step, no cleanup of headers has been made. Only moving
  from one place to another + necessary modifications.
2016-10-13 16:19:20 +03:00
dc7f7b6fb4 All changes 2.0.0...develop
973b983 Merge branch 'release-2.0.0' into develop
255dd23 Make spinlock functions take const argument
6e23bab Fix bitmask reallocation
338c189 Rename and clean up slavelag filter
3ea8f28 Fix possible NULL pointer dereference
bfe6738 MXS-830: Add module information to logged messages
1fad962 Fix strncat usage
d38997a Adjust log throttling policy
0be4e4b Add hashtable_item_strcasecmp
726100e Take hashtable convenience functions into use
5e7744a Fix typo in maxadmin.md
c5778c8 Merge branch 'release-2.0.0' into develop
b5762af Move from tmpnam to mkstemp
d6f2c71 Add convenience functions to hashtable
359058a MXS-825: Add support for --execdir
636347c Enable runtime reconfiguration of log throttling
ef9fba9 Improve log throttling documentation
aef917a Implement log throttling
e3a5349 Remove shardrouter.c
8051e80 Remove custom qc_sqlite allocation functions
fd34d60 Initial implementation of the learning firewall
a8752a8 Removed "filestem option" from example
1ef2519 Removed "filestem option" from example
0815cc8 Cleanup spinlock.h
ab4dc99 Clean up hashtable.h
ef2c078 Add prototypes for hashtable copy and free functions
fb5cfaf Add 'log_throttling' configuration entry
300d823 Add proper prototypes for hashtable hash and cmp functions
1c649aa qc_mysqlembedded: Include skygw_...h without path.
d276160 Add missing RPM scripts
e70e644 Fix HTTPAuth installation
1b2b389 Combine utils into the server directory
3ff9913 Add missing utils headers to devel package
407efb2 Fix minor packaging problems
99aa6ad Split MaxScale into core, experimental and devel packages
1290386 Merge branch 'develop' of ssh://github.com/mariadb-corporation/maxscale-new into develop
e59f148 Make scripts POSIX sh compatible
7319266 Fixed SHOW SLAVE STATUS in bonlog router
f8d760a Update Binlogrouter.md
0a904ed Update Replication-Proxy-Binlog-Router-Tutorial.md
75d4202 Update Replication-Proxy-Binlog-Router-Tutorial.md
b8651fc Add missing newline in listmanager.h
c7ad047 Add note about user data caches to release notes
70ccc2b Merge branch 'release-2.0.0' into develop
575d1b6 Mistake - dummy session needs list markers set.
8364508 Merge branch 'develop' into binlog_server_semisync
868b902 Update MaxScale limitations
2c8b327 Store listener caches in separate directories
6e183ec Create unique user data caches for each listeners
f643685 Don't free orphaned tee filter sessions
4179afa Allow binlogrouter to be used without a listener
7ad79af Add function for freeing a listener
677a0a2 Move authentication data from services to listeners
4f12af7 Merge remote-tracking branch 'origin/MXS-677' into develop
1419b81 Semi-Sync support to binlog server: code review updtate
0ea0f01 Semi-Sync support to binlog server: added missing routine
4aad909 Semi-Sync support to binlog server
b824e1e Add authenticator support to httpd.c
705a688 Change tabs to spaces
d0c419e Change method of adding list fields to e.g. DCB
25504fc Document the changed routing priority of hints
41666d1 Remove use_ssl_if_enabled global option
a3584e9 Make routing hints have highest priority
34a1d24 Updated document with new binlog router option
01eedc5 Updated documentation with SSL usage
8a4c0f6 Update Replication-Proxy-Binlog-Router-Tutorial.md
4e374aa Update Replication-Proxy-Binlog-Router-Tutorial.md
f3f3c57 Update Replication-Proxy-Binlog-Router-Tutorial.md
617b79f Binlog Server: error messages typo fix
fa8dfae Binlog Server: error messages review
1b8819c Fix freeing of schemarouter session memory
07f49e1 MXS-788: new code review fix
1fd3b09 MXS-788: show services now displays SSL info
6ca2584 MXS-788 code review fix
ae6a7d0 MXS-788 code review
43d3474 Master server SSL connection
90b2377 Use correct variable in listmanager pre-allocation
9a5b238 Fix listmanager pre-allocation
9c78625 Fix a memory leak when backend authentication fails
e59a966 Fix hang in list_find_free
ff30223 Fix freeing of shared data in schemarouter
fc8f9d3 Add missing include in luafilter
ecf7f53 Add missing NULL value to filter parameter array
636d849 Update memory allocation approach
f0d1d38 Add new allocation functions
97d00a0 Fix writing of uninitialized data to logs
e72c9b2 Merge branch 'release-2.0.0' into develop
cf2b712 Merge branch 'release-2.0.0' into develop
8917c5c Change the logic behind valid list entry checks
c10deff Improve documentation about version_string
f59f1f7 Merge branch 'develop' of ssh://github.com/mariadb-corporation/maxscale-new into develop
c88edb3 Backend authentication failure improvement
abd5bee Revert "Backend authentication failure improvement"
5bb3107 Backend authentication failure improvement
b7f434a Add new allocation functions
3f022fa Fix stupid mistake
99c4317 Merge remote-tracking branch 'origin/MXS-677' into develop
3c1ded6 Added connection/authentication failure error reporting in SHOW SLAVE STATUS
0a60f7b Tidy up and deal with review points.
ba103ff blr_slave.c: Update strncpy usage
467331e blr_master.c: Strncpy usage updates
d2b7c0c Merge remote-tracking branch 'origin/develop-nullauth-merge' into develop
5a8c1d0 qc: Measure execution time at the right place.
bccdb93 Merge branch 'NullAuthDeny' into develop
2e6511c Add 5.5.5 prefix to all version strings that lack it
314655a Improve DCB and session initialization and list handling
e1c43f0 MXS-655: Make MaxScale logging logrotate(8) compatible
ce36afd MXS-626: Don't log a header unless maxlog enabled
dcd47a7 blr_file.c: Replace uses of strncpy
6b8f576 bls_slave.c: Replace strncpy with memcpy
68a0039 Add list preallocation, tidy up, simplify init.
cb37d1b Fix copyright etc headers.
11a400d Tidy; comment; fix bad copies and mistakes.
7e36ec4 Add list manager files.
c4794e3 Initial code for list manager.
1b42e25 Merge remote-tracking branch 'origin/MXS-765' into develop
d50f617 Fix problems, extend tests, respond to review.
dcb4a91 Filter test folder removed
0b60dbe Add a couple of comments.
83cdba0 Fix overwriting problem.
ba5d353 Fix overwriting problem.
53671cb Small fixes in response to review.
173d049 blr.c: Review strncpy usage
4ff6ef2 binlog_common.c: Replace strncpy with memcpy
f238e03 maxbinlogcheck.s: Replace strncpy
9807f8d harness: Replace unnecessary use of strncpy
8c7fe6a avro: Modify strncpy usage
9b8008e Small improvements.
b7f784f Fix mistakes in testqueuemanager.c
cc26962 Restore missing poll.c code; add testqueuemanager.c.
2e91806 Format the filter harness
22059e6 Initial implementation connection queueing.
c604dc2 readwritesplit.c: Improve COM_INIT_DB handling
454d920 schemarouter.c: Replace strncpy with strcpy
8e85d66 sharding_common.c: Too long a database name handled explicitly
77f4446 Astyle schemarouter
491f7c2 maxinfo.c: Replace strncpy with memcpy
6b98105 maxinfo: Reformat with astyle
c1dbf08 Handle oversize user and database names
5fa4a0f Merge branch 'develop' of ssh://github.com/mariadb-corporation/maxscale-new into develop
706963b BLR_DBUSERS_TAIL new var in blr.h
d75b9af Tweak comments, remove trailing blanks.
ab2400a Optimise statistics gathering by inline & simpler fns.
fb59ddc Remove unnecessary strncpy/strncat usage in Binlog Server
bdcd551 resultset.c: Change strncpy to memcpy
c6b1c5e Reject rather than cut too long a path
6d8f112 Remove unnecessary strncpy/strncat usage
18bf5ed Remove unnecessary strncpy usage
dc0e2db Make maxpasswd more userfriendly
c9c8695 Fix calculation of padded_len in encryptPassword
2cfd2c6 dbusers.c: Check strncpy usage
7ab9342 Make more thorough checks in secrets_readKeys
be7d593 Format cli.c debugcli.c testroute.c webserver.c
1ee5efb config.c: Check usage of strncpy
3043b12 gq_utils.c: Unnecessary use of strncpy removed
77874ac Add help to maxkeys
38392a3 Update secrets_writeKeys documentation
2d1325c Make SSL optional in MaxScale's own communication
bda00da Fix avro build failures
b2cb31a Add more OOM macros
41ccf17 Fix strdup usage
a48f732 Fix realloc calls
20771f6 Add forgotten extern "C" block
8faf35a Add maxscale allocation functions
bb47890 Add macros for OOM logging
afea388 Fix silly mistakes.
6dafd22 Make deny default for null auth; move code from common to auth.
2016-08-17 10:06:35 +03:00