Commit Graph

9080 Commits

Author SHA1 Message Date
d2c80d2e91 MXS-1461 Add QueryClassifierModule 2017-11-20 13:14:02 +02:00
531a8bafbd MXS-1461 Add mock classes Session and Dcb 2017-11-20 13:14:02 +02:00
65cf491350 MXS-1461 Add general purpose mocking functions 2017-11-20 13:14:02 +02:00
e434c7ec66 MXS-1461 Add template Module
This is the first in a series of commits that introduce components
that allow filters to be tested without MaxScale or backends.
Obviously the environment is not suited for all cases, but allow e.g.
the behaviour of the firewall filter to be tested in isolation.

The environment consists of helper classes loading and manipulating
modules and filters and mock-classes that provide the needed
scaffolding around a filter.

Currently all components exist under the test directory of the firewall
filter, but once proven they can be moved to a more general purpose
location.
2017-11-20 13:14:02 +02:00
8077d97e25 MXS-1513: Work around the backend_read_timeout-setting
The setting limits the maximum time a MASTER_GTID_WAIT-function
can wait. To work around this limitation, the function is now called
in a loop such that the total timeout is approximately equal to
the requested timeout.
2017-11-20 12:31:11 +02:00
59616b5f3e MXS-1490: Cleanup error printing, add json error
Slave redirection is a special case, as there the total failure
is only known after all redirects have been attempted. In the
failure case, all errors from connections are gathered to one
message.
2017-11-20 11:33:47 +02:00
204fa17322 MXS-1519 Prevent "inheritance" of fw rules
By using an the value of a global incremented integer as the
initial version number of a Dbfw instance it is ensured that every
such instance does not "inherit" any rules from a previous instance
that happened to exist in the same place as the new instance to be
created.
2017-11-20 10:11:23 +02:00
07e58444f6 Improve error message for zero monitor timeout values
The error message was not 100% accurate about the value. In addition to
that, neither the value itself nor the monitor or parameter names were
printed in the error message.
2017-11-17 18:05:03 +02:00
84d1ea0bff MXS-1490: Perform failover only after failcount monitor loops
The same failcount variable is used for the detect_standalone_master-
feature.
2017-11-17 10:12:33 +02:00
703230a930 Only write monitor journal when it changes
The state of the monitored servers is only persisted if the states of the
servers have changed. This removes the unnecessary disk IO caused by the
writing on the monitor journal.
2017-11-16 15:38:13 +02:00
db61ec0d7e Add missing initialization in readwritesplit
A value was not always initialized for all queries when info level logging
was enabled.
2017-11-16 15:38:13 +02:00
e0b8ab0b9c Fix memory leak when users are dumped
When users are dumped to file, the JSON object wasn't freed.
2017-11-16 15:38:12 +02:00
decd5578bc MXS-1525 Check exact match first when checking host
Given a rule like

  ...
  users %@127.0.0.1 match any rules ...

the code started with %@127.0.0.%, which meant that the exact
match rule would be missed.
2017-11-16 14:09:28 +02:00
11a240cb15 MXS-1519 Use fw instance specific rules and users
The rules and users need to be distinct not only for each thread
but for each instance and thread.
2017-11-16 14:06:50 +02:00
b63c6504a3 MXS-1513: Switchover script
First version of switchover script. Unsafe to run as it has no
timeouts for most queries. Also, removed code launching the
previous switchover_script.
2017-11-16 10:51:12 +02:00
032b2f3846 MXS-1333: QLAFilter, add option to log execution time
Adds an option 'reply_time' to log_data. When enabled, query execution time
in milliseconds is logged. Execution time is measured as the time between
routeQuery and the first clientReply.
2017-11-15 16:56:39 +02:00
fca11f992a Fix verify_master_failure
The test expected the wrong error message to be in the log. To better cope
with changes in the text, the test now uses a regular expression to do the
matching.
2017-11-14 16:53:09 +02:00
f41111b4bd MXS-1517: Retain stale master bit even on master failure
If a server goes down and it has the stale master bit enabled, all other
bits for the server are cleared. This allows failed masters that have been
replaced to be first detected and then reintroduced into the replication
topology.
2017-11-14 16:53:09 +02:00
b80f394cd0 Use explicit types
Use uint64_t instead of unsigned. This guarantees that the size of the
type is the same across all platforms.
2017-11-14 16:53:09 +02:00
9d686c4c41 MXS-1461 Create error message only when blocking
Only if the filter is in blocking mode, will a match cause an error
to be returned. Thus, no point in creating the error message unless
the filter is in blocking mode.
2017-11-14 13:03:07 +02:00
ea0f8978b2 MXS-1461 Update log messages of firewall rules
Whether or not a match causes the query to be blocked or to be allowed
depends on whether the filter is in white-listing or black-listing mode,
so better if the text is neutral.
2017-11-14 13:03:07 +02:00
3eba77f43e MXS-1461 Implement not_function
If the instance represents `not_function` then it is a match if the
used function is not found amongst the listed functions.
2017-11-14 13:03:07 +02:00
3993ece209 MXS-1461 Remove special handling for function + allow
No need for this when 'not_function' is available.
2017-11-14 13:03:07 +02:00
3215458b30 MXS-1461 Introduce 'not_function' rule
Using 'not_function' it is possible to match functions other than
a specific set of ones. That will make it significantly easier to
allow certain functions to be used with certain columns.

The special handling of no arguments to `function` in conjunction
with an allowing filter has been removed. The same effect can now
be achieved, without special handling, using `not_function`, no
arguments and a blocking filter.

Implementation will follow in a subsequent commit.
2017-11-14 13:03:07 +02:00
ced93acc20 MXS-1514: Add failover test
The test is composed of a few parts.
1: Test that failover happens on master failure.
2: Test that a server with slave sql thread stopped is not promoted.
3: Test that a server with log_slave_updates=1 is promoted before others.
2017-11-14 10:01:37 +02:00
b2b72474d9 Initialize thread as well in test_trxtracking 2017-11-10 14:45:54 +02:00
6cc8bc36c1 Initialize in test_dcb
Make necessary initializations in test_dcb.
2017-11-10 14:45:54 +02:00
f2b9525032 Initialize thread as well in test_trxcompare 2017-11-10 14:45:54 +02:00
f152c118cb Name core tests consistently
Now, given a concept xyz,
* the main test file is called test_xyz.cc
* the executable is called test_xyz, and
* the ctest test is called text_xyz.
2017-11-10 14:45:54 +02:00
2792f54089 Use explicit script path in test_maxctrl
The path to the script is now explicitly defined. This makes the call
unambiguous as the home directory depends on the current user.
2017-11-10 09:37:51 +02:00
5acdfab5e6 Start MaxScale as maxscale user in run_ctrl_c.sh
MaxScale should not be started as root.
2017-11-09 14:15:47 +02:00
dd58a8b3c8 Add missing close definition for examplefilter
The filter session method was not defined.
2017-11-09 14:15:47 +02:00
e68f16b099 Merge branch '2.2' into 2.2-mrm 2017-11-09 09:41:58 +02:00
3dbd201a86 MXS-1478 Provide daemon flag
With "--daemon" or "-n" MaxScale can now be told to run in daemon
mode, that is, it forks and the parent exits. This is the default
behaviour, but a flag to this effect is needed if the default
behaviour is changed.
2017-11-09 09:34:23 +02:00
cbb6739e00 Always open the SQLite database in binlogrouter
The SQLite database was mistakenly opened only when master GTID
registration was used.
2017-11-08 18:41:14 +02:00
0558a2195d MXS-1479 By default, refuse to run as root
MaxScale now refuses to run as root. However, it is possible to
start MaxScale as root, as long as a user to run MaxScale as is
provided as a command line argument.

It is possible to run as root by invoking MaxScale as root and
by specifying the MaxScale user to be root.
2017-11-08 16:03:05 +02:00
3a13469691 MXS-1490 Fix bug with gtid_io_pos change check
The conditional was opposite to intention.
2017-11-08 10:46:51 +02:00
a1a5947d61 MXS-1490: Parse Gtid_IO_Pos only when using Gtid
First check "Using_Gtid", as that should be always valid. If set to
"Slave_Pos", parse "Gtid_IO_Pos".
2017-11-08 10:46:51 +02:00
3a35d49ffe Merge branch '2.2' into 2.2-mrm 2017-11-08 10:44:35 +02:00
42d9064b04 Remoeve queuemanager.[h|cc] 2017-11-08 10:44:14 +02:00
b1b78a5be7 Remove references to QUEUE_CONFIG
Only used in conjunction with queued connections, which are not
enabled anyway. Once that comes on the table again, better to use
some standard data structures.
2017-11-08 10:44:14 +02:00
4cf01fa88f Remove 'failover_script' parameter
As the failover is now internal to MySQL Monitor, no failover
script parameter is needed.
2017-11-07 16:05:44 +02:00
dce073a684 MXS-1496: Don't assign slave status for masters
The slave and stale slave status bits should be cleared from a master if
it still has them.

Also used the correct functions to manipulate the bits instead of directly
setting them in the monitor.
2017-11-07 15:52:28 +02:00
84e95cee96 MXS-1490: Query gtid_slave_pos only during failover
The value of the global gtid_slave_pos is only needed during
failover, so querying it every monitor loop is unnecessary. The
value is now only requested when deciding on a new master server
or when waiting for the selected promotion target to clear its
relay logs.

Also, when waiting for the logs to clear, gtid_io_pos must stay
constant or failover is cancelled. Io_pos advancing indicates that
the server is still receiving events from the old master.
2017-11-07 13:09:51 +02:00
f44020496a Merge branch '2.2' into 2.2-mrm 2017-11-07 12:05:33 +02:00
d7b8e95234 MXS-1489 Create mechanism for running concurrent tasks
This commit introduces maxscale::future, maxscale::packaged_task
and maxscale::thread that are modeled after C++11 std::future,
std::packaged_task and std::thread as described here:
http://en.cppreference.com/w/cpp/thread

The standard classes rely upon rvalue references (and move
constructors) introduced by C++11. As the C++ compilers we must use
are pre-C++11 that feature is obviously not present. The absence of
rvalue references is circumvented by implementing regular copy
constructors and assignment operators as if the arguments were rvalue
references.

In practice the above means that when one of these objects are copied,
the state is _moved_ rendering the copied object in default initialized
state. Some care is needed to ensure that unintended copying does not
occur.
2017-11-07 12:03:09 +02:00
a96c8e1326 Install explicit version of pkg
The installed version of `pkg` was not locked a specific release.
2017-11-07 12:03:09 +02:00
5db02586b1 Use the external CDC connector
The CDC connector was moved to its own repository and some changes to its
interface were made. Updated build scripts, deleted old connector and
fixed code to use new interfaces.
2017-11-07 12:03:09 +02:00
ce19fe3248 Move Jansson compilation into main test suite
The test suite now compiles Jansson instead of letting the CDC connector
do it. This way the connector can be used as a very simple static library
with a dependency on the Jansson library.
2017-11-07 12:03:09 +02:00
96d9ec1b6d Update the CDC connector
The CDC connector now uses a non-blocking socket for the reads. This
allows the possibility of adding read timeouts.

Added some utility functions for dealing with GTIDs and delayed the
reading of the first row.
2017-11-07 12:03:09 +02:00