Commit Graph

7490 Commits

Author SHA1 Message Date
7469c5be52 Add functionality for bypassing MySQL whitespace/comments
This functionality is needed both in the query classifier, where
it was first created, and in the cache.
2017-03-17 10:27:16 +02:00
6db3cc380b Add cache parameter 'selects' 2017-03-17 09:06:27 +02:00
987a52b398 Further simplify key generation
The original approach was made for RocksDB where it is beneficial
to keep keys of stuff related to each other close to each other.
However, as RocksDB is no longer the primary focus, it just causes
additional cost to dig out the table names.

The key is a 64-bit integer, but crc32 only gives us a 32-bit one.
We create an other 32-bit value by running crc32 over the same SQL,
using the first crc value as adler.

I think that further reduces the chance for clashes:

    uint32_t crc0 = crc32(0, Z_NULL, 0);
    uint32_t crc1;
    uint32_t crc2;

    crc1 = crc32(crc0, "codding", 7) => 1774765869
    crc2 = crc32(crc1, "codding", 7) => 1409592046

    crc1 = crc32(crc0, "gnu", 3)     => 1774765869
    crc2 = crc32(crc1, "gnu", 3)     => 1213798908

Note that the first value is the same, but the second is not.
2017-03-16 18:39:10 +02:00
daf3774329 Use simpler function for calculating hash
Now the key is a 64-bit integer and it is created by the cache
component and can not be unique for the storage.
2017-03-16 16:47:15 +02:00
9388dff7cf Merge branch 'develop' into MXS-1075 2017-03-15 18:12:00 +01:00
f384050d45 Assign stale master status to standalone masters
When a standalone master server is detected, it should receive the stale
status to prevent it from losing the master status if another server is
started and allow_cluster_recovery is enabled.
2017-03-15 17:37:25 +02:00
3396741c21 Improve UTF-8 handling in avrorouter
The json_stringn function should be used instead of the json_string to
allow null characters as well as non-null terminated strings to be
embedded in the JSON values.

The CDC example Python programs now decode the data as UTF-8 instead of
ASCII.
2017-03-15 17:37:25 +02:00
67590d59ac Fix connector_plugindir usage with basedir
The connector_plugindir wasn't set correctly when basedir was used.
2017-03-15 17:37:25 +02:00
c4d08a6936 Update build dependencies and documentation
The libaio is not required by MaxScale so the check for it is no longer
needed.

Updated documentation to match the current requirements to build MaxScale.
2017-03-15 17:37:25 +02:00
9dbabb666a Rebase NamedServerFilter to develop
Fix conflicts due to ipv6 changes.
Change SourceHost->m_address to string.
SourceHost generation is now cleaner.
2017-03-15 16:19:53 +02:00
ab7b1c227e Refactor nameserverfilter, now uses filter template
Move most of the functions under the filter or session-classes.
Class definitions moved to a header file. General cleanup. Some features
are still incoming.
2017-03-15 16:19:53 +02:00
32fe220a1b NamedServerFilter multiple "match"-"server"-pairs
The filter now accepts (in addition to the old "match" and "server")
parameters of the form "matchN" and "serverN", where N is a decimal
number with two digits, i.e 01, 02, 03 .. 20. When routing queries,
the regural expressions will be tested one by one, and the servers
from the first match will be added as hints. Also, a single "server"-
setting may contain multiple servers separated by ','. The server
names are not verified to be actual servers, this is up to the user.
2017-03-15 16:19:53 +02:00
3129efb3f7 Add new parameter type MXS_MODULE_PARAM_SERVERLIST
This is a list of servers, separated by commas. When queried as a
config setting, returns a null-terminated array of SERVER*:s. The
commit includes a serverlist parsing function, which should probably
be used anywhere a similarly formed string is parsed.
2017-03-15 11:48:46 +02:00
5e39268e37 Remove support for TrxBoundaryMatcher
For the general case, regex matching simply will not do. The
regex becomes so hairy so it turns write-only, i.e. unmaintainable.
Regex matching is also slower than a handwritten custom parser.
2017-03-15 10:39:26 +02:00
c235e181ce Null filter no long requires 'capabilities' argument
- If no capabilities are provided, then the filter will have none.
- Now logs its capabilities as startup (useful for debugging).
2017-03-15 10:26:31 +02:00
1c8205e005 Add missing flag handling 2017-03-15 10:25:02 +02:00
c51cd8858d Minor optimizations to TrxBoundaryParser 2017-03-15 10:20:35 +02:00
96f0321b7e Add program for profiling TrxBoundaryParser 2017-03-15 09:35:15 +02:00
9a33cf4d3e Add smoke-test for qc_get_trx_type_mask_using
Use the query classifier test files and compare that the
result returned by qc_sqlite and the custom parser agree
for every statement.
2017-03-15 09:35:15 +02:00
cab176b149 Fix handling of AUTOCOMMIT=ON 2017-03-15 09:35:15 +02:00
87d84bc81c Ignore implicit commits 2017-03-15 09:35:15 +02:00
c2add97e30 Recognise various autocommit syntaxes
set autocommit=1
set global autocommit=1
set session autocommit=1
set @@global.autocommit=1
set @@session.autocommit=1
2017-03-15 09:35:15 +02:00
470de51e22 Handle whitespace in TrxBoundaryParser
TheBoundaryMatcher is not updated as it is likely it will be removed
altogether. A regex that accepts comments in all relevant places becomes
so hairy it is unmaintainable. It seems that the only working solution
would be to first remove all comments and then perform the regex.
2017-03-15 09:35:15 +02:00
931f765df5 Enhance transaction tracking test 2017-03-15 09:35:15 +02:00
f8a0ca25a4 Regex accept WITH CONSISTENT SNAPSHOT 2017-03-15 09:35:15 +02:00
212be78ece Make first a crude check, then a detailed one
Before checking whether a particular regex matches a statement
we check with one general one whether it at all is possible that
a statement might match.

Since most statements will not be transaction related it makes
sense to first check whether it at all is possible that the
statement might be transaction related.
2017-03-15 09:35:15 +02:00
d5fc54a9de Handle WITH CONSISTENT SNAPSHOT as well
TrsBoundaryParser now capable of parsing WITH CONSISTENT
SNAPSHOT transaction statements as well.
2017-03-15 09:35:15 +02:00
35132f3afe Take TrxBoundaryMatcher into use
The code that used to be in query_classifier.cc is now in
trxboundarymatcher.cc.
2017-03-15 09:35:15 +02:00
faa0dea2ce Add TrxBoundaryMatcher
A class capble of detecting statements that change the transaction
state and autocommit mode. The detection is done using regexes.

There is still some expanding and optimization to be done.
2017-03-15 09:35:15 +02:00
a85ce2e915 Take TrxBoundaryParser into use 2017-03-15 09:35:15 +02:00
74fe9fb911 Add class TrxBoundaryParser
A class capble of detecting statements that change the transaction
state and autocommit mode.

There are still some expansion and optimization to be done.
2017-03-15 09:35:15 +02:00
ddca19767c Compile query_classifier.c as C++
In preparation for some modifications that are easier done using
C++.
2017-03-15 09:35:15 +02:00
8e81941058 Enable trx boundary detection using regexes
Transaction boundaries can now be detected using regexes.
All else being equal, it gives a 10% performance improvement
compared to qc-based detection.

In a subsequent change, mysql_client.c will be modified to use
qc_get_trx_type_mask() instead of qc_get_type_mask().

Currently the use of regex matching is turned on using an
environment variable. That will change.
2017-03-15 09:35:15 +02:00
0985a18a7e Perform internal qc initialization 2017-03-15 09:35:15 +02:00
a58f944f23 Allow query classifier to initialize itself
The process and thread initialization/finalization of the query
classifier plugins is handled using the process and thread
initialization/finalization functions in the module object.

However, the top-level query classifier will also need to perform
process and thread initialization when transaction boundaries are
detected using regular expressions.
2017-03-15 09:35:15 +02:00
669d6e95f4 Change 'localhost_from_socket' to 'localhost'
With the former you cannot log in over domain sockets.
2017-03-15 09:34:12 +02:00
78b6b4d4e4 Add MXS-1032 to the release notes
Added MXS-1032 to the rease notes.
2017-03-15 09:11:18 +02:00
b7e29fb6f0 Add example use of match and ignore parameters
The CCRFilter documentation now has examples for both `match` and
`ignore`.
2017-03-15 09:11:18 +02:00
9efcea4cec Fix false error message
The error message for failure to connect to any server was always logged
when the schemarouter was used.
2017-03-15 09:11:18 +02:00
d19e5b376d MXS-1032: Add configurable connector plugin directory
The connector plugin directory can now be controlled with the
`connector_plugindir` argument and configuration option. This should allow
the connector to use the system plugins if the versions are binary
compatible.

Replaced calls to mysql_options to mysql_optionsv as the former is
deprecated in Connector-C 3.0 and the latter is supported in Connector-C
2.3.
2017-03-15 09:11:18 +02:00
61627e035c Correct release note link 2017-03-14 12:51:26 +02:00
2bcb57365b Update 2.1.1 release date 2017-03-14 12:51:26 +02:00
e572743f02 Correct release note link 2017-03-14 12:46:58 +02:00
e1165a85bb Update 2.1.1 release date 2017-03-14 12:38:11 +02:00
dd82c46596 Remove two checks that always return false
At the point this function is called, the servers and services are
not yet created.
2017-03-14 11:09:54 +02:00
8c7618035b Merge branch '2.1' into develop 2017-03-14 10:45:25 +02:00
7b3c287ac3 Close sessions in MaxScale core
The core now provides a simple function to close a session. This removes
the need for the modules to directly call the API entry points when the
session should be closed. It is also in line with the style that other
objects, namely the DCBs, use. This makes the new session_close very
similar to dcb_close.
2017-03-14 10:45:10 +02:00
29be8436e5 Remove unused readconnroute code
The state change code was not used.
2017-03-14 10:45:10 +02:00
9952676052 Move UNIX domain socket creation to utils.c
The network socket code is now completely inside utils.c.
2017-03-14 10:45:10 +02:00
573615889b Binlog Server doesn't ask for any maxwell query if the option is not set
If ‘maxwell-compatibility’ option is not set, no SET/SELECT are sent to
master during slave registration phase
2017-03-14 09:28:04 +01:00