12509 Commits

Author SHA1 Message Date
Markus Mäkelä
45ba520057
Update csmon documentation
The required functionality for automatic "master" detection is in 1.2.
2019-03-25 18:48:26 +02:00
Johan Wikman
fe359593ba MXS-2407 Report database for SHOW TABLES FROM db
The database used in 'SHOW TABLES FROM db' is now available via
qc_get_database_names().
2019-03-25 17:02:36 +02:00
Johan Wikman
8b4961261f MXS-2403 Update documentation 2019-03-25 15:26:17 +02:00
Johan Wikman
3c7e64181b MXS-2403 Check only what is necessary
The amount of checking is minimized and what can be checked in one
go, is checked in one go.
2019-03-25 15:26:17 +02:00
Johan Wikman
f9f9599e74 MXS-2403 Reject if masked filed is used in subquery
There's some overlapping work being done now. To be cleanup up
in a subsequent commit.
2019-03-25 15:26:17 +02:00
Johan Wikman
8b329f7f33 MXS-2403 Add 'check_subqueries' parameter
Only documentation and parameter handling.
2019-03-25 15:26:17 +02:00
Johan Wikman
3870fac600 MXS-2402 MXS-2396 '*' should be blocked only if there are rules
If there are no rules for the current user, then '*' should be
allowed in UNIONS and when defining variables.
2019-03-25 15:24:35 +02:00
Johan Wikman
25f8931384 MXS-2402 Reject if masked field is referred to in UNION select
If the second or subsequent select of a UNION statement refers to
a field to be masked, then the statement must be rejected.
2019-03-25 15:24:35 +02:00
Johan Wikman
ca8350ae35 MXS-2402 Document and handle 'check_unions' parameter
Only parameter handling, no actual functionality.
2019-03-25 15:24:35 +02:00
Johan Wikman
f37340e9fd MXS-2397 Provide context information for fields
The query classifier now returns contextual information for a fields;
does it appear in the (right hand side) of a UNION or in a SUBQUERY.
2019-03-25 15:24:35 +02:00
Esa Korhonen
fadbdc7514 Merge branch '2.2' into 2.3 2019-03-22 12:11:24 +02:00
Esa Korhonen
5cdba97ec7 Merge commit '216eb904c557509ea5a3216e68e274df957ab807' into 2.3 2019-03-22 10:48:31 +02:00
Markus Mäkelä
9bc721afb6
Merge commit '11ee74bad327e7fb15e8388d20e7838b9e49cadf' into 2.3 2019-03-21 17:52:42 +02:00
Markus Mäkelä
3448a36b68
Fix runtime config error generation
The error string was passed as the format string.
2019-03-21 17:48:47 +02:00
Esa Korhonen
9236ace077 MXS-2355 If client is using the wrong authenticator, attempt a switch
Some SQL clients may default to a different authentication plugin than
"mysql_native_password". Since this is the only one supported by MySQL-
authenticator, the client is instructed to swap its plugin.
2019-03-21 17:07:09 +02:00
Esa Korhonen
216eb904c5 MXS-1991 Allow replication_user and replication_password be set runtime
Because runtime changes are performed one at a time, adding replication credentials
to a mariadbmon which didn't have any would cause an error to be printed, and
the monitor would not start.

This is now fixed by allowing replication_user without replication_password. This
is not an ideal solution as a configuration file with only replication_user would be
accepted. Also, when adding the credentials to a monitor, replication_user must be
given first to avoid the error.
2019-03-21 17:06:24 +02:00
Johan Wikman
559b786533 MXS-2398 Handle MariaDB specific comments
Same approach as with regular comments:
- /*M! STMT */ are always parsed.
- /*M!###### STMT */ are never parsed.
2019-03-21 14:57:24 +02:00
Markus Mäkelä
11ee74bad3
Free the readwritesplit query queue
If the queue isn't empty when the session closes, the queue would leak.
2019-03-21 11:22:40 +02:00
Esa Korhonen
2c4228db47 MXS-2400 Extend examplefilter
Added more comments. Also the filter now demonstrates how to
1) read configuration parameters
2) react to queries and replies
3) handle shared filter data
4) print diagnostics output
5) add log entries

As the filter is already built and moved to the library directory,
it is immediately usable. This should be helpful with assignments.
2019-03-21 11:02:16 +02:00
Johan Wikman
65b4ac7c1b MXS-2389 Handle MariaDB comment correctly
A non version specific executable comment, such as "/*! SELECT 1; */"
is during classification handled as if it would not be a comment. That
is, the contained statement will *always* be parsed.

A version specific executable comment, such as "/*!99999 CREATE PROCEDURE
 bypass BEGIN */ SELECT ... " is during classification handled as it would
be a general comment. That is, the contained statement will *never* be
parsed.

In addition, in the latter case the parse result will never be better than
QC_QUERY_PARTIALLY_PARSED. The rationale is that since the comment is version
specific, we cannot know how the server will actually interpret the statement.

This will have an impact on the masking filter and the database firewall that
now will reject statements containing _version specific_ executable comments.
2019-03-20 16:17:50 +02:00
Johan Wikman
9b27d7f24b MXS-2396 Reject use of masked var in variable definition
E.g.

   set @a = (SELECT fld FROM table);

will be rejected if 'fld' is a field that should be masked.
2019-03-20 15:11:58 +02:00
Johan Wikman
eb0f3ebe81 MXS-2396 Cleanup conceptually 2019-03-20 15:11:58 +02:00
Johan Wikman
7bb3cc6535 MXS-2396 Make preparations for adding additional checks 2019-03-20 15:11:58 +02:00
Johan Wikman
76ac63096c MXS-2396 Add check_user_variables configuration parameter
The masking filter should check for things like

    set @ssn = (SELECT ssn FROM users WHERE id = 1);

so that

    select @ssn;

is not possible.
2019-03-20 15:11:58 +02:00
Markus Mäkelä
0b77c3f05f
MXS-2324: Prevent stack overflow with large results
If a result consists of only OK packets, they would be processed
recursively which most of the time leads to a stack overflow. This can be
prevented by consuming all OK packets in the result in one go.
2019-03-20 10:29:03 +02:00
Johan Wikman
a6f52b008f MXS-2390 Firewall should reject statements prepared from var
Currently no practical way for checking what the statement
actually does and thus blanket rejection is the only alternative.
2019-03-19 15:53:46 +02:00
Johan Wikman
09d9570c7e MXS-2392 MXS-2390 Add new test cases
Checks that prepared statements are checked.
2019-03-19 13:09:16 +02:00
Johan Wikman
8e0ee34a94 MXS-2392 MXS-2390 Handle prepared statements in masking filter
- Check textual prepared statements for use of columns to be masked
  in conjunction with functions.
- Check binary prepared statements for use of columns to be masked
  in conjunction with functions.
- Prevent creation of textual prepared statement from variable.
2019-03-19 12:31:08 +02:00
Johan Wikman
7f1c44cf62 MXS-2393 Reject statement that cannot be parsed completely
As the masking filter must reject statements that use columns to be
masked as arguments to functions, a statement that cannot be fully
parsed must be rejected. Unless a statement can be fully parsed we
cannot know whether such usage occurs.
2019-03-19 10:47:37 +02:00
Johan Wikman
c393d633fe MXS-2394 Accept substitute_variables
'substitute_variables' was not listed amongst the symbols that
are pre-parsed, which meant that it was always rejected.
2019-03-19 09:35:18 +02:00
Markus Mäkelä
805e7975f5
Fix log truncation
Syslog wasn't truncated which caused massive disk space usage when the
full test set was run. Also removed the creation of empty log files if no
messages were logged during the run.
2019-03-18 13:58:04 +02:00
Markus Mäkelä
85c2d3c9ee
MXS-2374: Set next_pos for skipped events
By always setting the position of the next event, the replication cannot
break if the slave server has to reconnect after receiving an event that
was generated by the binlogfilter.
2019-03-18 13:58:04 +02:00
Markus Mäkelä
17245c4ce3
Fix false debug assertion
Given the following query:

PREPARE ps FROM 'PREPARE ps2 FROM \'SELECT 1\'';

The debug assertion is hit even though this is valid, albeit unsupported,
SQL. An optimization would be to ignore the query if the prepared
statement type is another prepared statement.
2019-03-18 12:17:48 +02:00
Markus Mäkelä
dd99cadfd2
Prevent unintended sharing of parsing info
When a single GWBUF was split into two with gwbuf_split, the new GWBUF
would point to the start of the shared data and the old one to the
end. Data-wise, this is fine but as the parsing info for queries is stored
in the shared buffer it causes problems when multiple packets get read in
one network payload. The end result would be that only the first query in
the lot would get parsed and the rest would get the same classification as
the first one.

To properly fix this without the need to deep clone the buffer would
require a reorganization of the buffer mechanism in MaxScale.

This commit alone doesn't fix the queued query routing problems in
readwritesplit. The commit from 2.2 which fixes the ordering problems with
queued queries is also required for a fully functional queued query
mechanism.
2019-03-18 12:17:48 +02:00
Markus Mäkelä
4dda31ffe3
Merge branch '2.2' into 2.3 2019-03-16 09:30:56 +02:00
Markus Mäkelä
fd21bed5aa
Fix queued query handling
If a routing of a queued query caused it to be put back on the query
queue, the order in which the queue was reorganized was wrong. The first
query would get appended as the last query which caused the order to be
reversed.
2019-03-15 20:35:17 +02:00
Esa Korhonen
a3af8ffdd0 MXS-2325 Add test case
Tests that a disabled event is not enabled.
2019-03-15 15:36:41 +02:00
Markus Mäkelä
14890cb232
Merge branch '2.2' into 2.3 2019-03-15 12:26:59 +02:00
Markus Mäkelä
4697128b3c
Update 2.2 maintenance version 2019-03-15 11:42:41 +02:00
Markus Mäkelä
4400876b46
Update release date 2019-03-15 11:41:13 +02:00
Markus Mäkelä
29b8d12be3
Always ignore EPIPE errors
There's no reason to not ignore EPIPE errors in debug builds.
2019-03-14 20:08:13 +02:00
Markus Mäkelä
f9853431ee
Simplify maxrows test config
The configuration now uses only one server to prevent replication lag from
affecting the results.
2019-03-14 12:22:25 +02:00
Markus Mäkelä
09dc92973e
Discard connections as the last step
Th discarding of connections in maintenance mode must be done after any
results have been written to them. This prevents closing of the connection
before the actual result is returned.
2019-03-14 12:15:30 +02:00
Markus Mäkelä
e5edb5c78f
Fix reads into bad memory in readwritesplit
The candidate selection code used default values that would cause reads
past buffers. The code could also dereference the end iterator which
causes undefined behavior.
2019-03-14 12:15:30 +02:00
Markus Mäkelä
b537176248
Fix parsing of non-query packets
Packets that do not contain SQL should not be parsed.
2019-03-13 15:44:02 +02:00
Markus Mäkelä
e83555f126
Update release documentation
The update to VERSIONxx.cmake was not documented.
2019-03-13 14:09:00 +02:00
Markus Mäkelä
0df7228e5e
Add 2.2.20 release notes 2019-03-13 13:57:16 +02:00
Markus Mäkelä
e2ac52826a
MXS-2375: Provide aliases for [start|stop] maxscale
The `start services` and `stop services` command aliases now map to the
same command as `start maxscale` and `stop maxscale`. This should help
alleviate the confusion that the command naming caused.
2019-03-13 13:40:37 +02:00
Timofey Turenko
84e456eee2 ignore truncate() return in fwf_syntax test 2019-03-13 12:53:12 +02:00
Esa Korhonen
040562f718 MXS-2342 Run MariaDBMonitor diagnostics concurrent with the monitor loop
This fixes some situations where MaxAdmin/MaxCtrl would block and wait
until a monitor operation or tick is complete. This also fixes a deadlock
caused by calling monitor diagnostics inside a monitor script.

Concurrency is enabled by adding one mutex per server object to protect
array-like fields from concurrent reading/writing.
2019-03-12 10:50:16 +02:00