Commit Graph

49 Commits

Author SHA1 Message Date
c88aa11e11 Copy FDE events in avrorouter
Commit 67386980e327ad063b24cb55971cf44f4930e241 caused the actual events
to be ignored. This meant that the larger event size was assumed for all
events. In most cases this works but it is not the correct way to do it.
2018-06-04 15:17:48 +03:00
3dfb972d87 Merge branch '2.1' into 2.2 2018-01-30 16:28:11 +02:00
6dcc71d862 MXS-1621: Fix minor bugs caused by previous changes
Used the correct value in table_create_alloc and remove unused
parameter. Use the pre-calculated end pointer when looking for events.
Always use the column count of the TABLE_MAP event as all mismatches are
detected earlier.
2018-01-30 15:59:06 +02:00
93923acafb MXS-1621: Skip unneeded ALTER TABLE operations
Some ALTER TABLE operations (e.g. ADD INDEX) are not useful to the
avrorouter. These need to be detected and skipped.
2018-01-30 15:59:05 +02:00
9146a215f7 Fix DDL table identifier parsing
The parsing was inadequate and did not support all forms of quoted
identifiers.
2018-01-30 15:59:05 +02:00
8dfb1d0113 MXS-1621: Add ALTER TABLE ... [FIRST | AFTER col ] parsing
The parser checks whether the FIRST or AFTER keywords are used and, if
AFTER is used, extracts the relevant column name.

Added a test case that checks that the parsing works and detects the
correct column names.
2018-01-30 15:59:05 +02:00
0afe10ffb9 Merge branch '2.1' into 2.2 2018-01-16 09:47:49 +02:00
95ad3aa7df MXS-1543: Log a warning if STATEMENT or MIXED is used
If avrorouter suspects that statement based or mixed replication formats
are being used, it logs a warning.
2018-01-05 12:50:39 +02:00
ff6bed6aeb Merge branch '2.1' into 2.2 2017-12-21 12:25:26 +02:00
044339d37f Add missing detection of CREATE TABLE AS
As MaxScale does not have the actual data returned by the SELECT SQL query
done to create the table, CREATE TABLE AS cannot be supported.
2017-12-14 19:06:17 +02:00
79afaa447e Merge branch '2.1' into 2.2 2017-12-12 13:23:02 +02:00
7c2a173a17 MXS-1536: Always recreate TABLE_MAP objects
When a table map event is read after an alter table, the old TABLE_MAP
object contains old information. Due to this, as well as the added benefit
of making the code easier to read, the recycling of TABLE_MAP objects was
removed. In practice, there were no benefits to re-mapping the tables to a
different ID.
2017-11-30 12:38:43 +02:00
764863cb3f MXS-1536: Add Annotate_rows event processing
The Annotate_rows events were not processed which caused the following
table map event to be ignored.

Also removed a false debug assertion. The byte count can be zero and the
pointer is not guaranteed to point to anything valid.
2017-11-30 12:38:43 +02:00
0ca04cb5d5 MXS-1536: Fix crash in CREATE TABLE
If a CREATE TABLE statement had a quoted keyword as the name of a field,
the calculated column count and actual column counts would differ.

In addition to this, oneline comments before the end of the statement
would truncate the SQL due to the fact that the whitespace was squashed
before the comment removal was done.
2017-11-30 12:38:43 +02:00
895d950da0 Format all source files with Astyle
Formatted all source files Astyle.
2017-09-28 07:04:21 +03:00
3676dcebb1 MXS-1398: Clean up mysql.h header
Cleaned up the MaxScale version of the mysql.h header by removing all
unused includes. This revealed a large amount of dependencies on these
removed includes in other files which needed to be fixed.

Also sorted all includes in changed files by type and alphabetical
order. Removed explicit revision history from modified files.
2017-09-14 15:30:43 +03:00
02b9e0a01d Merge branch '2.1.7' into develop-2.1-merge 2017-09-12 11:08:02 +03:00
67386980e3 MXS-1341: Ignore binlog checksums in avrorouter
If the binlog has binlog checksums enabled, the extra checksum bytes are
removed from the end of the event. The avrorouter assumes that whatever
caused the binlogs to appear in the first place already checked that the
checksums are OK.

Also removed one extra byte being added to the length of all query events.
2017-08-16 13:24:52 +03:00
11d494ef33 Fix avro CREATE TABLE message
The string was assumed to be null-terminated when in reality it was not.
2017-07-04 12:40:22 +03:00
285086ba7c Fix ALTER TABLE regression in avrorouter
The avrorouter failed to detect ALTER TABLE statements which caused a
regression. Extended the alter table tests to parse the JSON for more
strict validation of test results.
2017-07-04 12:40:22 +03:00
5ed6483f01 Fix avro CREATE TABLE message
The string was assumed to be null-terminated when in reality it was not.
2017-07-04 12:21:25 +03:00
5c50045227 Fix ALTER TABLE regression in avrorouter
The avrorouter failed to detect ALTER TABLE statements which caused a
regression. Extended the alter table tests to parse the JSON for more
strict validation of test results.
2017-07-04 09:31:11 +03:00
cb57e10761 Develop merge
Develop merge
2017-06-29 15:34:22 +02:00
40b1739249 MXS-1299: Fix CREATE TABLE t1 LIKE t2 processing
The creation of tables from other tables was not working even though the
information was available inside the avrorouter.
2017-06-28 11:25:57 +03:00
457fc80647 Merge branch '2.1' into develop 2017-06-05 13:25:27 +03:00
285598e0e5 Fix avrorouter type and name handling
The type and name parsing functions could move outside of allocated memory
as they didn't check for the terminating null character. Also fixed the
printf format string used when the list of used tables is being created.

Fixed CDC testing connector to abort on error and added some extra output
to the cdc_datatypes test.
2017-06-05 13:19:47 +03:00
f546a17e77 Update change date of 2.2 2017-06-01 10:24:20 +03:00
322983a5f4 Merge branch '2.1' into develop 2017-05-17 13:48:52 +03:00
ed653ac729 Merge branch '2.0' into 2.1-merge-2.0 2017-05-14 10:25:12 +03:00
415001a4cd Merge branch '2.1' into develop 2017-03-09 10:02:36 +02:00
41ff01d16d Merge branch '2.0' into 2.1 2017-03-09 10:01:38 +02:00
e1a1959bc2 Replace strerror_r with mxs_strerror
The mxs_strerror function requires no local buffer, thus making it simpler
and cleaner to use.
2017-03-09 09:36:57 +02:00
8be44a8cda Merge branch '2.1' into develop 2017-03-06 11:25:44 +02:00
bb22b82461 Merge branch '2.0' into 2.1 2017-03-06 10:38:07 +02:00
c3cc46ae04 Add deflate compression to avrorouter
The avrorouter can now compress the Avro files using the deflate algorithm
provided by zlib.
2017-02-21 16:19:09 +02:00
49cc2b52e3 Merge branch '2.1.0' into 2.1 2017-02-15 08:44:55 +02:00
5648f708af Update license to BSL 1.1 2017-02-14 21:42:28 +02:00
7e23789364 Fix one last compilation error
Fix one ignored return value in Avro, caused compilation to fail.
2017-02-13 14:46:43 +02:00
eb1e163bdf Only update index file if file is flushed to disk
There's no point in indexing the file if it hasn't been synced to
disk. Any attempts to index the file will fail if the file still has an
open data block.
2017-02-12 04:38:24 +02:00
d2c3be8728 Fix compiler flag usage
The compiler flags overwrote the default values that CMake generates
instead of appending to them.
2017-02-10 17:19:46 +02:00
a6698c6f07 Merge branch '2.1.0' into 2.1 2017-02-08 09:30:34 +02:00
c82831cc10 Fix merge conflict
An old define name caused the build to fail.
2017-02-08 09:29:26 +02:00
67888f3596 Merge branch '2.0' into 2.1.0 2017-02-08 09:27:00 +02:00
7149775163 Use correct enum values in avrorouter
The file flushing enums weren't used.
2017-02-07 10:21:58 +02:00
a48be9badf Format router modules
Formatted router modules with Astyle.
2017-01-17 14:48:43 +02:00
2cabcea211 Add definitions of MXS_MODULE_NAME to all modules
All modules now declare a name for the module. This is name is added as a
prefix to all messages logged by a module. The prefix should help
determine which part of the system logs a message.
2017-01-16 11:28:34 +02:00
eaf3633728 Only flush tables to disk at the end of the binlog
Currently, when the avrorouter finishes reading a binlog file or when a
certain number of rows or transactions is reached, it will flush all
tables to disk. This is quite slow as events can easily be written faster
into the binlog than they can be processed by avrorouter.

A solution to this would be to only sync the tables (close the Avro block)
instead of flushing them to disk. This would allow more efficient
processing of the files while still retaining the safe shutdown that
flushing offers.
2017-01-02 09:31:21 +02:00
488a9d24e3 Stop the Avrorouter when MaxScale is shutting down
This allows for a faster controlled shutdown when binary log files are
being indexed.
2016-12-19 10:33:55 +02:00
aef6c7b099 Make directory name equal to the library name
* avro -> avrorouter
* binlog -> binlogrouter
2016-12-01 15:23:42 +02:00