Commit Graph

5933 Commits

Author SHA1 Message Date
cb4e43b05a MXS-2794: Log changes in loaded users
By checking whether the users have changed whenever they are reloaded, we
improve the visibility of the user reloading process. Using a checksum
allows us to easily compress the information with acceptable loss of
accuracy. Using a CAS loop prevents duplicate messages without losing any
updates even if multiple user reloads result in different outcomes.
2019-12-10 09:41:42 +02:00
e36c7efa25 MXS-2785: Report PCRE2 errors
The substitution string could cause errors.
2019-12-09 17:53:59 +02:00
689a284892 MXS-2785: Make rewrite_src a regular expression
The use of a regular expression allows multiple rewrite rules to be
combined into one. This allows more versatile conversions but, given the
simple nature of regular expressions, also makes accidental changes more
likely.

Addd mxs::pcre2_substitute that is a more C++-friendly version of
mxs_pcre2_substitute to make. This makes string replacement a lot easier
to do when the source and destination are not C strings.
2019-12-09 17:53:59 +02:00
d23f0366a8 MXS-2785: Prevent broken replication setups
When rewrite_src and rewrite_dest have different lengths, the slave must
use GTID based replication. This removes the need for one-to-one matching
between the slave's relay log and the master's binlog which gets broken
when event lengths are modified due to event rewriting.
2019-12-09 17:53:59 +02:00
3de5e4edcd MXS-2785: Allow event size changes due to rewrites
The replication events use a redundant format that has both the length of
the event and the position of the next event. The length can be modified
so that the next event position of the previous event and the length of
the curren event can be different. This includes overlap of the events
where the next event position of an event is "inside" the current event.

The next event position must retain its original value as that allows
replication slaves to reconnect with the correct position when file and
position based replication is used. For GTID replication, the slave asks
for the coordinates from the master and uses those.

When a slave receives a heartbeat event from a master, it checks that the
binlog name matches and that the next event position in the event is not
behind the slave's relay log position. These events must be modified to
contain a fake next event position that will never be reached by the
slave. This makes sure that the simple sanity checks never fail even if
we've caused the slave's relay log to be ahead of the master's binlog.
2019-12-09 17:53:59 +02:00
e829cae8b1 MXS-2785: Add rewrite_src and rewrite_dest parameters
The parameters allow rudimentary database rewriting in the replication
stream. This is still very limited as the replacement must have the same
length as the original. In theory it could be shorter without causing
problems but making it longer is not easy.
2019-12-09 17:53:58 +02:00
c480a44852 Merge branch '2.3' into 2.4 2019-12-05 16:54:44 +02:00
e673c3eab0 MXS-2788 Use case-insensitive string comparisons
When names are matched in rules and in resultsets, case insensitive
matching must be used.
2019-12-05 13:44:18 +02:00
52eafb7926 Fix binlogfilter capabilities
The binlogfilter needs to read results one packet at a time but it needs
resultsets to be collected into a single buffer. This behavior is
guaranteed implicitly when the binlogrouter is used but is not present
when it is used without it. To support the use of the binlogfilter with
readconnroute, the filter must properly declare the capabilities.
2019-12-03 11:06:52 +02:00
b1961042a7 MXS-2776: Fix unwanted regex checks on COMMIT
The default database check would be done even on transaction management
commands which would cause them to always be skipped when a match was
defined.
2019-12-02 09:08:00 +02:00
7a5e50f980 Merge branch '2.3' into 2.4 2019-11-29 16:41:07 +02:00
374d5b28d1 MXS-2759: Optimize user loading query
The SQL for the second recursive CTE table can be optimized by adding a
where condition on the recursive part that rules out users that are not
roles. The functionality remains the same as only roles can be granted to
users.
2019-11-29 16:16:34 +02:00
231f68b6dc MXS-2727 Load storage from build-time location
Otherwise it might not be found.
2019-11-13 15:10:56 +02:00
0ec505b301 MXS-2727 Fix test-program 2019-11-13 13:06:07 +02:00
80ce599cf8 Remove cache_stress from the build 2019-11-13 08:37:17 +02:00
f6731a898d Update change date 2019-11-13 08:37:17 +02:00
1c28eefbd0 MXS-2727 Add fix
If an existing cache-entry should be updated, but the new value
is larger that the maximum size of the cache, then the cache can
not be updated, but the old value must be removed.

Whether or not we succeed in removing the entry, an error result
must be returned. Earlier OK was returned, but no node was
allocated, which then caused a crash.
2019-11-12 15:27:31 +02:00
6a319dc655 MXS-2727 Add test that reveals problem 2019-11-12 15:20:32 +02:00
521960e6f9 MXS-2727 Add program for causing cache stress 2019-11-12 15:18:36 +02:00
d67442356e Merge branch '2.3' into 2.4 2019-11-08 14:13:06 +02:00
5c4ac03c07 Log a message in readconnroute's handleError
This way there's trace that something unexpected happened and that the
session closed unexpectedly.
2019-11-08 13:44:21 +02:00
fdfbf3e133 Update 2.4.3 change date 2019-11-05 12:21:00 +02:00
a1e8287265 Add global to ccrfilter
The `global` parameter causes the time window defined by the `time`
parameter to be applied at the instance level instead of the session
level. This means that a write from one connection will cause all other
connections to use the master for a certain period of time.

Using a configurable time window for consistency is not good as it is not
absolute and cannot adjust to how servers behave.

One example that demonstrates this is when a slave is normally lagging
behind by less than a second but some event causes the lag to spike up to
several seconds. In this case the configured time window would no longer
guarantee consistency.

Another reason to avoid a "static" time window is the fact taht it
prevents load balancing in the cases where slaves catch up to the master
within time window. This happens when time is configured to a higher value
to avoid inconsistencies at all costs.

Added a test case that verified the feature works.
2019-11-04 08:32:17 +02:00
861e27eb00 Merge branch '2.3' into 2.4 2019-10-29 14:04:31 +02:00
df6c56e7ca Update 2.3.13 Change Date 2019-10-29 12:51:31 +02:00
4d51342561 Merge remote-tracking branch 'origin/2.3' into 2.4 2019-10-29 12:29:32 +02:00
639b0af6aa MXS-2734: Check preferred server on table conflict
If the schemarouter detects a table conflict in a query and one of the
servers is the preferred one, don't log an error.
2019-10-29 11:34:09 +02:00
ac5b5d527a MXS-2610: Prevent watchdog timeouts in avrorouter
The file processing can take a very long time and as it is done on a
routing worker, the workaround needs to be used.
2019-10-29 11:33:10 +02:00
446a3fac15 MXS-2720: Fix service session count
The number of sessions wasn't always incremented but it was always
decremented. This happened primarily when authentication failed. By making
the management of the counters a part of the object lifecycle, this
problem goes away.
2019-10-29 09:26:41 +02:00
64cae223f0 MXS-2721: Check that LocalClient creation succeeds
The creation can fail if a network socket cannot be opened and connected
to the host in question.
2019-10-29 09:26:41 +02:00
dc895e41ad Merge branch '2.3' into 2.4 2019-10-11 09:51:48 +03:00
183673b026 MXS-2720: Fix service session count
The number of sessions wasn't always incremented but it was always
decremented. This happened primarily when authentication failed. By making
the management of the counters a part of the object lifecycle, this
problem goes away.
2019-10-10 21:34:03 +03:00
1202c00fc7 MXS-2721: Check that LocalClient creation succeeds
The creation can fail if a network socket cannot be opened and connected
to the host in question.
2019-10-10 21:24:35 +03:00
428bc5740b MXS-2645 Decrement service client count also when authentication fails
The client count was incremented before authentication was complete, and
should be decremented if it fails. Otherwise service connection limit can
be easily reached.
2019-10-04 15:11:09 +03:00
2d9d9356c9 Merge branch '2.3' into 2.4 2019-10-04 12:56:52 +03:00
869effc352 MXS-2707: Retain commands that respond
Only commands that return a response should to be retained. Otherwise the
bookkeeping of m_current_query will be wrong.
2019-10-03 22:17:44 +03:00
b223e6eed3 MXS-2264: Add support for RENAME TABLE
RENAME TABLE is now fully supported and works as expected. With the fix to
table versioning, the new table name will receive the latest version
number.
2019-09-30 16:09:04 +03:00
7767231c8f MXS-2264: Fix table versioning
The table versions are now stored in memory and are only resolved on
startup. This simplifies things and removes the need to know where the
data is stored as that information is not available to the Rpl class.
2019-09-30 11:47:30 +03:00
2eb6a25534 MXS-2264: Add support for ALTER TABLE RENAME
This adds preliminary support for renaming tables. There is still a
problem where the table version will always be set to 1 on a rename. This
should not be done and the version should be set to the largest value that
ever was for that table.
2019-09-30 11:47:30 +03:00
a444484310 MXS-2702: Update m_current_master on session command
When lazy_connect is enabled and the first query is `SET autocommit=0`, a
master connection can be created. If it is, then the m_current_master
pointer must also be updated.

Also fixed the case where a failure to connect to one slave would cause
the connection attempts to stop too early.
2019-09-30 11:47:29 +03:00
27675ed41d MXS-2354: Fix subsecond part of temporal values
TIMESTAMP2, DATETIME2 and TIME2 values with decimal parts are now
correctly converted into their string forms. Previously the decimal part
was ignored but most of the code required to extract it was in place.
2019-09-30 11:47:29 +03:00
72d9e41ccb MXS-2263: Process unsigned integer columns correctly
The unsigned integers that would previously be interpreted as negative
values are now correctly converted into their corresponding avro
values. Due to a limitation in the Avro file format, 64-bit unsigned
integers cannot be represented in their unsigned form.

Since both the signed and unsigned versions of a 32-bit integer cannot fit
into a single Avro int, the type for these was changed to long. This is a
backwards incompatible change which means files generated with older
versions will not convert unsigned values correctly.
2019-09-30 11:47:27 +03:00
e1ff978b80 MXS-2263: Retain integer sign information
The unsignedness of a column is now retained in the Column type as well as
the JSON schema. This allows correct conversion of unsigned integer types
which will be done in a later commit.
2019-09-30 11:45:01 +03:00
117e2e7e88 Deliver notifications to registered clients
The notifications were delivered to all clients regardless of the
registration state.
2019-09-30 11:45:01 +03:00
bcf26fba3e Add missing COMMIT detection for non-transactional engines
Non-transactional engines will use a query event with a COMMIT in it to
signal that the transaction is over.
2019-09-30 11:45:01 +03:00
51d2d6c1f2 Fix GTID file path generation
The path would be invalid if there was no trailing slash.
2019-09-30 11:45:01 +03:00
f9a6649d77 Merge branch '2.3' into 2.4 2019-09-24 21:42:32 +03:00
fcc7642328 MXS-2694: Add support for COM_BINLOG_DUMP
The resultset processing now understands COM_BINLOG_DUMP and treats it as
a never-ending resultset.
2019-09-24 21:41:24 +03:00
9ce788ff0b MXS-2690: Fix duplicate database detection
Empty duplicate databases are now correctly detected since the results
will always return the database name combined with a trailing period.
2019-09-21 21:16:55 +03:00
bd3107e7ca Merge branch '2.3' into 2.4 2019-09-19 15:37:43 +03:00