Commit Graph

1293 Commits

Author SHA1 Message Date
e36c7efa25 MXS-2785: Report PCRE2 errors
The substitution string could cause errors.
2019-12-09 17:53:59 +02:00
689a284892 MXS-2785: Make rewrite_src a regular expression
The use of a regular expression allows multiple rewrite rules to be
combined into one. This allows more versatile conversions but, given the
simple nature of regular expressions, also makes accidental changes more
likely.

Addd mxs::pcre2_substitute that is a more C++-friendly version of
mxs_pcre2_substitute to make. This makes string replacement a lot easier
to do when the source and destination are not C strings.
2019-12-09 17:53:59 +02:00
d23f0366a8 MXS-2785: Prevent broken replication setups
When rewrite_src and rewrite_dest have different lengths, the slave must
use GTID based replication. This removes the need for one-to-one matching
between the slave's relay log and the master's binlog which gets broken
when event lengths are modified due to event rewriting.
2019-12-09 17:53:59 +02:00
3de5e4edcd MXS-2785: Allow event size changes due to rewrites
The replication events use a redundant format that has both the length of
the event and the position of the next event. The length can be modified
so that the next event position of the previous event and the length of
the curren event can be different. This includes overlap of the events
where the next event position of an event is "inside" the current event.

The next event position must retain its original value as that allows
replication slaves to reconnect with the correct position when file and
position based replication is used. For GTID replication, the slave asks
for the coordinates from the master and uses those.

When a slave receives a heartbeat event from a master, it checks that the
binlog name matches and that the next event position in the event is not
behind the slave's relay log position. These events must be modified to
contain a fake next event position that will never be reached by the
slave. This makes sure that the simple sanity checks never fail even if
we've caused the slave's relay log to be ahead of the master's binlog.
2019-12-09 17:53:59 +02:00
e829cae8b1 MXS-2785: Add rewrite_src and rewrite_dest parameters
The parameters allow rudimentary database rewriting in the replication
stream. This is still very limited as the replacement must have the same
length as the original. In theory it could be shorter without causing
problems but making it longer is not easy.
2019-12-09 17:53:58 +02:00
c480a44852 Merge branch '2.3' into 2.4 2019-12-05 16:54:44 +02:00
e673c3eab0 MXS-2788 Use case-insensitive string comparisons
When names are matched in rules and in resultsets, case insensitive
matching must be used.
2019-12-05 13:44:18 +02:00
52eafb7926 Fix binlogfilter capabilities
The binlogfilter needs to read results one packet at a time but it needs
resultsets to be collected into a single buffer. This behavior is
guaranteed implicitly when the binlogrouter is used but is not present
when it is used without it. To support the use of the binlogfilter with
readconnroute, the filter must properly declare the capabilities.
2019-12-03 11:06:52 +02:00
b1961042a7 MXS-2776: Fix unwanted regex checks on COMMIT
The default database check would be done even on transaction management
commands which would cause them to always be skipped when a match was
defined.
2019-12-02 09:08:00 +02:00
231f68b6dc MXS-2727 Load storage from build-time location
Otherwise it might not be found.
2019-11-13 15:10:56 +02:00
0ec505b301 MXS-2727 Fix test-program 2019-11-13 13:06:07 +02:00
80ce599cf8 Remove cache_stress from the build 2019-11-13 08:37:17 +02:00
f6731a898d Update change date 2019-11-13 08:37:17 +02:00
1c28eefbd0 MXS-2727 Add fix
If an existing cache-entry should be updated, but the new value
is larger that the maximum size of the cache, then the cache can
not be updated, but the old value must be removed.

Whether or not we succeed in removing the entry, an error result
must be returned. Earlier OK was returned, but no node was
allocated, which then caused a crash.
2019-11-12 15:27:31 +02:00
6a319dc655 MXS-2727 Add test that reveals problem 2019-11-12 15:20:32 +02:00
521960e6f9 MXS-2727 Add program for causing cache stress 2019-11-12 15:18:36 +02:00
fdfbf3e133 Update 2.4.3 change date 2019-11-05 12:21:00 +02:00
a1e8287265 Add global to ccrfilter
The `global` parameter causes the time window defined by the `time`
parameter to be applied at the instance level instead of the session
level. This means that a write from one connection will cause all other
connections to use the master for a certain period of time.

Using a configurable time window for consistency is not good as it is not
absolute and cannot adjust to how servers behave.

One example that demonstrates this is when a slave is normally lagging
behind by less than a second but some event causes the lag to spike up to
several seconds. In this case the configured time window would no longer
guarantee consistency.

Another reason to avoid a "static" time window is the fact taht it
prevents load balancing in the cases where slaves catch up to the master
within time window. This happens when time is configured to a higher value
to avoid inconsistencies at all costs.

Added a test case that verified the feature works.
2019-11-04 08:32:17 +02:00
861e27eb00 Merge branch '2.3' into 2.4 2019-10-29 14:04:31 +02:00
df6c56e7ca Update 2.3.13 Change Date 2019-10-29 12:51:31 +02:00
446a3fac15 MXS-2720: Fix service session count
The number of sessions wasn't always incremented but it was always
decremented. This happened primarily when authentication failed. By making
the management of the counters a part of the object lifecycle, this
problem goes away.
2019-10-29 09:26:41 +02:00
dc895e41ad Merge branch '2.3' into 2.4 2019-10-11 09:51:48 +03:00
183673b026 MXS-2720: Fix service session count
The number of sessions wasn't always incremented but it was always
decremented. This happened primarily when authentication failed. By making
the management of the counters a part of the object lifecycle, this
problem goes away.
2019-10-10 21:34:03 +03:00
ee8042c40b MXS-2613 Access correct argument
The cache show-command accessed the wrong argument. That is why

    [maxadmin|maxctrl] call command cache show TheCache

did not work.
2019-07-30 13:00:37 +03:00
c592328ea2 MXS-2613 Access correct argument
The cache show-command accessed the wrong argument. That is why

    [maxadmin|maxctrl] call command cache show TheCache

did not work.
2019-07-30 12:29:01 +03:00
59e7b2b09d Deprecate mqfilter 2019-06-26 10:08:15 +03:00
2ab9aa9a94 Update 2.4.0 Change Date 2019-06-25 09:19:55 +03:00
9f771ffc2b Revert "MXS-2513 Deprecate firewall and masking filters"
This reverts commit de3c5721e95d5d5be909a56faeb121ad09a2f6fe.
2019-06-24 10:21:01 +03:00
470968a296 Fix unit test failures
The new net_write_timeout value needed to be explicitly defined for the
tests.
2019-06-11 21:38:47 +03:00
04fdaf1fdb MXS-2556 Make config::Configuration aware of its object
The name of the object (i.e. the section name from the configuration
file), is now stored in the configuration object for that object.

That way, more contextual and hence morfe user friendly errors and
warnings can be generated.
2019-06-11 11:05:15 +03:00
68af4cb62e MXS-2556 Rename configure -> post_configure
Rename config::Configuration::configure() to
config::Configuration::post_configure(). Latter name makes it
unambiguously clear at what point the function is called.
2019-06-11 09:46:04 +03:00
4efa9dbeea Remove maxscale/alloc.h
The remaining contents were moved to maxbase/alloc.h.
2019-06-10 14:11:25 +03:00
655e5fab5b Move length-encoded integer/string functions to maxsql 2019-06-07 11:32:46 +03:00
44d1b821c3 Merge branch '2.3' into develop 2019-06-03 13:54:55 +03:00
9481992bb9 Fix dbfwfilter release mode build failure
GCC 9 complained about null format strings even though the argument given
was never null.
2019-05-31 14:01:15 +03:00
de3c5721e9 MXS-2513 Deprecate firewall and masking filters 2019-05-31 11:53:36 +03:00
5b3dbdbf16 Fix test_cacheoptions and test_dbfwfilter
Due to listener changes we cannot just keep on creating new
listeners with the same port over and over again.

Easiest to simply use a different port each time.
2019-05-14 15:01:15 +03:00
17d3beba3b MXS-2385 Fix the problem with the USE command and tpmfilter
Trying to get the type mask of the QWBUF caused debug assertion in the query
classifier when using some commands. Now type mask is checked only when the
buffer contains MXS_COM_QUERY command.
2019-05-14 13:40:37 +03:00
fbd61967a7 Format filters 2019-05-10 10:31:12 +03:00
3813c728b1 Move listener parameter handling into Listener::create
The Listener::create method now takes a set of configuration parameters
from which it constructs a listener. This removes the duplicated code and
makes the behavior of listener creation similar to other objects in
MaxScale. It also allows the configuration parameters to be stored in the
listener object itself.
2019-05-10 09:21:52 +03:00
6baf970140 Fix test_dbfwfilter 2019-05-03 16:12:26 +03:00
9a10f64840 Fix test_cacheoptions 2019-05-03 16:09:14 +03:00
fb0745e3de Merge branch '2.3' into develop 2019-05-03 13:48:57 +03:00
a3cf1d22c0 MXS-2457 Streamline logging 2019-05-03 13:38:12 +03:00
279edce16e MXS-2457 Add treat_string_as_field to firewall
Necessary if the firewall should be able to block columns when
'ANSI_QUOTES' as enabled and " instead of backticks are used.

Without this, the following

    > set @@sql_mode='ANSI_QUOTES';
    > select "ssn" from person;

will not be blocked if the database firewall has been configured
to block the column ssn.
2019-05-03 13:38:12 +03:00
5833c39a8a MXS-2457 Disable query classifier cache in masking
As the canonicalization is also not aware of 'ANSI_QUOTES', the
cache must be disabled if the masking filter has
'treat_string_arg_as_field' enabled.
2019-05-03 13:38:12 +03:00
fe5160a714 MXS-2457 Add 'treat_string_arg_as_field' to DB Firewall 2019-05-03 13:38:12 +03:00
3a5a8b13b9 MXS-2457 Treat string args as fields
The masking filter will now consider all string arguments to
functions to be fields. This in order to prevent bypassing of
the masking with

    > set @@sql_mode='ANSI_QUOTES';
    > select concat("ssn") from masking;

This may lead to false positives, but no can do.
2019-05-03 13:38:12 +03:00
8a250a8b13 MXS-2329 Make duration misuse harder
Now the desired type must be specified when getting a duration.
The type also dictates how durations without suffixes should be
interpreted.

That removes the need for remembering that to convert a returned
millisecond duration to a second duration.
2019-04-30 13:02:53 +03:00
de3f4fe5a8 MXS-2346 Ttl should be stored as milliseconds
cacheconfig.hh had not been pushed earlier.
2019-04-12 15:48:28 +03:00