Commit Graph

3464 Commits

Author SHA1 Message Date
61241f9e07 Remove old feedback system
The feedback system wasn't used and was starting to cause problems on
Debian 9 where the libcurl required different version of OpenSSL than what
MaxScale was linked against.
2017-07-10 16:03:50 +03:00
33e1878fe1 MXS-1302: addition of CaptureRule class
Addition of CaptureRule class, derived from Rule class
2017-07-10 14:10:21 +02:00
f10cc156a5 MXS-1302: added tests for "obfuscate"
Tests updated
2017-07-10 13:47:29 +02:00
f3547d14a1 Masking filter: removed create_rule_from_elements() routines
Masking filter cleanup: create_rule_from_elements() removed.
2017-07-10 09:49:20 +02:00
e70852d10b Code cleanup in ReplaceRule::create_from()
Code cleanup in ReplaceRule::create_from()
2017-07-10 09:49:20 +02:00
8b5c6f7286 Merge branch 'MXS-1302' into develop 2017-07-07 17:37:16 +02:00
9490af2fb1 MXS-1302: Addition of ObfuscateRule derived class and 'obfuscation' rule
New rule ‘obfuscate’ is being added:
{
"obfuscate": {
"column": “p_name”,
"database": "test",
"table": "masking"
}
},
"applies_to": [ ... ],
"exempted": [ ... ]
2017-07-07 17:31:37 +02:00
b26e2d8189 MXS-1302: masking filter value replacement is done by new ReplaceRule, derived form base class Rule
The value replacement is now done by new ReplaceRule, derived form base
class Rule
2017-07-07 15:02:59 +02:00
96b98845a2 Allow filters to be used with binlogrouter
Filters can now be used with the binlogrouter to modify the contents of
the binlog stream now that the correct function call is used.
2017-07-07 10:37:30 +03:00
ef73f3e217 Merge branch '2.1' into develop 2017-07-06 11:25:36 +03:00
9618e63b5f MXS-1310: Add deterministic conflict resolution
The schemarouter can now resolve database mapping conflicts in a
deterministic manner. This will fix the problem of central databases which
are replicated shards being assigned in a non-deterministic manner.
2017-07-05 19:19:49 +03:00
0c33572a96 MXS-1310: Detect implicit database usage
When the current database is implicitly used in a query that also uses an
explicit database, it must be routed to the shard which has the current
database.

As cross-shard joins are not supported, the safest, and possibly the most
expected course of action to take, is to route it to the so-called default
shard. The default shard is the shard that contains the database that is
currently set as the active database with a COM_INIT_DB, a text protocol
USE <database> query or it was set at connection time.
2017-07-04 23:21:04 +03:00
c6b579619a MXS-1310: Route implicit table queries to the current shard
When a query that implicitly uses the current database is detected, it
will be routed to the shard which has the current database.
2017-07-04 23:04:27 +03:00
374c204a04 Only print text protocol statements in schemarouter
The schemarouter info level logging treated all queries as something that
contain SQL statements. This, for somewhat obvious reasons, cannot be
expected.
2017-07-04 22:26:24 +03:00
71bc67b694 Use unordered_map in schemarouter
The shard map doesn't have to be in order and an unordered map will do
just fine.
2017-07-04 13:01:04 +03:00
11d494ef33 Fix avro CREATE TABLE message
The string was assumed to be null-terminated when in reality it was not.
2017-07-04 12:40:22 +03:00
285086ba7c Fix ALTER TABLE regression in avrorouter
The avrorouter failed to detect ALTER TABLE statements which caused a
regression. Extended the alter table tests to parse the JSON for more
strict validation of test results.
2017-07-04 12:40:22 +03:00
5f8c426578 MXS-1308: Fix crash with multimaster=true
The multimaster node detection uses stacks to sort the node groups. The
size of this stack was always assumed to be positive but it was possible
that it dropped down to -1 causing a crash when the stack was accessed
with the index number.
2017-07-04 12:40:22 +03:00
5ed6483f01 Fix avro CREATE TABLE message
The string was assumed to be null-terminated when in reality it was not.
2017-07-04 12:21:25 +03:00
5c50045227 Fix ALTER TABLE regression in avrorouter
The avrorouter failed to detect ALTER TABLE statements which caused a
regression. Extended the alter table tests to parse the JSON for more
strict validation of test results.
2017-07-04 09:31:11 +03:00
729e9dbbae Added FAKE_ROTATE event info log
Added FAKE_ROTATE event info log.

Changed: log of Request file and pos only if the binlog_name is not
empty (it can be empty when setting the GTID value before CHANGE MASTER
TO)
2017-07-03 18:14:20 +02:00
ff22243955 MXS-1308: Fix crash with multimaster=true
The multimaster node detection uses stacks to sort the node groups. The
size of this stack was always assumed to be positive but it was possible
that it dropped down to -1 causing a crash when the stack was accessed
with the index number.
2017-07-03 02:44:37 +03:00
7913f0ddf1 Develop Merge
Develop Merge
2017-06-30 10:48:13 +02:00
674b3887c9 MXS-1301: function matches functionless queries in accept mode
A function type rule matches a query if the query uses a function defined
in the rule. This is the desired behavior for blacklist mode operation
with `action=block`.

When in whitelist mode, all queries must match a rule to be allowed
through. For function type rules, this fact is problematic as queries that
don't use functions are blocked. The desired behavior is to allow the use
of certain functions while preventing the use of others.

The allowed set of functions should always contain the "empty set" (no
functions are used) when the filter is in whitelist mode.
2017-06-30 11:07:18 +03:00
809dea34e0 Fix overlap of router->binlog_name in blr_file_create
A call to strcpy was made in blr_file_create where the function was given
the same pointer as both parameters. To avoid this, the file name is now
copied to a local variable before the router variables are modified.
2017-06-30 11:07:18 +03:00
9a42709da4 Properly terminate 'ping' subcommand list
The 'ping' subcommand list was missing the terminating value of
`{EMPTY_OPTION}`.
2017-06-30 11:07:18 +03:00
a19d93380e Make block_size a size parameter
The block_size signifies a size in bytes so it makes sense to change it to
the size type.
2017-06-30 11:07:18 +03:00
25fd5a8a93 Fix crash on session allocation failure
The the return value was used before it was checked.
2017-06-30 11:07:18 +03:00
322fae8326 Add wrapper functions for buffer and original IV access
Added functions for accessing the buffer and original IV. This hides the
changes introduced in OpenSSL 1.1.
2017-06-30 10:46:05 +03:00
a47d4c40f5 Abstract EVP cipher context creation
The EVP_CIPHER_CTX is now created inside a wrapper function to add support
for OpenSSL 1.1. Also fixed improper use of the EVP_CIPHER_CTX internals
in binlogrouter.
2017-06-30 10:46:04 +03:00
cb57e10761 Develop merge
Develop merge
2017-06-29 15:34:22 +02:00
5597db255b Add wrapper functions for buffer and original IV access
Added functions for accessing the buffer and original IV. This hides the
changes introduced in OpenSSL 1.1.
2017-06-29 15:55:44 +03:00
09fb336403 Abstract EVP cipher context creation
The EVP_CIPHER_CTX is now created inside a wrapper function to add support
for OpenSSL 1.1. Also fixed improper use of the EVP_CIPHER_CTX internals
in binlogrouter.
2017-06-29 15:55:44 +03:00
8fb9b79dbb Make RWSplit::config() const
The function can be const.
2017-06-29 15:53:50 +03:00
3b3799889e MXS-852: Fix PS storage when locker to the master
When the router session is locked to the master, the storage of prepared
statements is not necessary as no other server can be used. The ID sent by
the client can be used without modification as it will always be the same
on the master.
2017-06-29 15:53:50 +03:00
fd9a08b9f4 Make Slave of External Server status configurable
The assignment of the Slave status with Slave of External Server can now
be controlled with the allow_external_slaves parameter.
2017-06-29 15:53:50 +03:00
1f8072f3d6 Fix temporary table detection in readwritesplit
The column information does not contain the used tables and the correct
function to use is qc_get_table_names.

Added a utility function for iterating over the set of tables. This
removes the need to have the same table iteration code in multiple places
and makes the code easier to comprehend.
2017-06-29 15:53:50 +03:00
5384162fb1 Fix slave server candidate ordering
The return value of the comparison was misinterpreted so that the
selection process preferred the new candidate over the current one if both
were equal. This was not an intended change and only a better candidate
should be chosen over the current candidate.
2017-06-29 15:53:50 +03:00
f7500cce9e Clean up route target resolution function
Cleaned up the function that resolves to which type of a server the query
should go.
2017-06-29 15:53:50 +03:00
dd05b44d38 Always extract the current command from the buffer
As readwritesplit queues queries, it needs to extract the command from the
buffer when the queued queries are routed. It cannot rely on the client
protocol command when the rerouting is taking place.
2017-06-29 15:53:50 +03:00
b0023e1689 Clean up readwritesplit API functions
Cleaned up some of the comments and documentation for the
functions. Renamed some variables and moved parts of the error handling
logging into a subfunction.
2017-06-29 15:53:50 +03:00
f2b199b3b0 MXS-852: Refactor gathering of routing information
The routing information is now gathered into a struct before the routing
process is started. This allows the requirements of the query to be
gathered before the actual target is selected.
2017-06-29 15:53:50 +03:00
a9a1291703 Split readwritesplit sources into separate parts
The common readwritesplit header was split into three distinct parts; the
instance, session and prepared statement headers. The definitions of any
members were moved to .cc files away from the headers.

The RWSplitSession, with its RWBackend class, is declared in the
rwsplitsession.hh header with all relevant definitions in
rwsplitsession.cc.

The PSManager class and all prepared statement related functions are now
located in the rwsplit_ps.hh header.

The old readwritesplit.hh header now contains the instance level
structures and all common classes used by the router. The
rwsplit_internal.hh header could be absorbed into the three newly created
headers with new headers for distinct parts of the router.
2017-06-29 15:53:50 +03:00
013b081b9e Rename readwritesplit router session class
Renamed the struct to RWSplitSession and removed the obsole next pointer.
2017-06-29 15:53:50 +03:00
917fe21f72 Refactor readwritesplit instance class
Changed the ROUTER_INSTANCE struct to a class and added functions for
common operations.

Renamed configuration and statistics structures and added constructors.

Moved objects around in readwritesplit.hh to be ready for a split into
multiple headers.
2017-06-29 15:53:50 +03:00
8ed16fd9d2 Reduce readwritesplit debug verbosity
Moved the LOG_RS output down to debug level. The output is not useful for
the end users.
2017-06-29 15:53:50 +03:00
16201592a2 MXS-852: Fix execution of COM_STMT_FETCH
The COM_STMT_FETCH queries are always executed when the result has not
been fully read. This means that the statement queuing code needs to allow
COM_STMT_FETCH commands to pass if a statement is being executed but the
command queue is empty.
2017-06-29 15:53:50 +03:00
296c1001a2 MXS-852: Track COM_STMT_EXECUTE by statement ID
The COM_STMT_EXECUTE targets are now tracked per statement ID. This should
theoretically allow parallel execution of COM_STMT_EXECUTE commands that
use cursors but the current implementation of the reply state processing
does not yet allow it.
2017-06-29 15:53:50 +03:00
d0b8ccd6b8 MXS-852: Track target of last COM_STMT_EXECUTE
Tracking of the node where the last COM_STMT_EXECUTE was sent allows
routing of all following COM_STMT_FETCH to the same node. This is required
for cursors to work.

MariaDB/MySQL does not support multiple active cursors so the
COM_STMT_FETCH will always refer to the latest COM_STMT_EXECUTE and using
a different ID goes against the protocol. If/when the support for multiple
active cursors is added, the tracking should be done for each
COM_STMT_EXECUTE statement. This should be relatively easily to achieve
but currently it is unnecessary.
2017-06-29 15:53:50 +03:00
52b7fb9340 MXS-852: Handle one-way session commands
Session commands that will not return a response can be completed
immediately. This requires some special code in the readwritesplit Backend
class implementation as well as a small addition to the Backend class
itself.

Since not all commands expect a response from the server, the queued query
routing function needs some adjustment. The routing of queued queries
should be attempted until a command which expects a response is found or
the queue is empty.

By properly handling these types of session commands, the router can
enable the execution of COM_STMT_CLOSE and COM_STMT_RESET on all
servers. This will prevent resource leakages in the server and allow
proper handling of COM_STMT type command.
2017-06-29 15:53:50 +03:00