128 Commits

Author SHA1 Message Date
Johan Wikman
8bfb4f231a Merge branch '2.1' into 2.2 2018-03-06 10:38:01 +02:00
Markus Mäkelä
93f3529bb4
MXS-1697: Treat missing libraries as an error
If the avrorouter is being build and the required libraries are not found,
the configuration process should fail. Adding the command to bypass this
into the error message should make it easier to disable this part if it is
not needed.
2018-03-05 13:58:09 +02:00
Johan Wikman
e1f1d8e58a Merge branch '2.1' into 2.2 2018-02-02 16:05:14 +02:00
Markus Mäkelä
7093a5bdf8 Fix CREATE TABLE tokenization
The token skipping function did not check for a period or an opening
parenthesis when parsing the test. Also fixed a debug assertion when only
NULL values were inserted.
2018-02-02 12:24:26 +02:00
Markus Mäkelä
3dfb972d87 Merge branch '2.1' into 2.2 2018-01-30 16:28:11 +02:00
Markus Mäkelä
6dcc71d862 MXS-1621: Fix minor bugs caused by previous changes
Used the correct value in table_create_alloc and remove unused
parameter. Use the pre-calculated end pointer when looking for events.
Always use the column count of the TABLE_MAP event as all mismatches are
detected earlier.
2018-01-30 15:59:06 +02:00
Markus Mäkelä
93923acafb MXS-1621: Skip unneeded ALTER TABLE operations
Some ALTER TABLE operations (e.g. ADD INDEX) are not useful to the
avrorouter. These need to be detected and skipped.
2018-01-30 15:59:05 +02:00
Markus Mäkelä
e14710ab2b Fix ALTER TABLE detection regex
The regular expression expected that the COLUMN keyword was always
present.
2018-01-30 15:59:05 +02:00
Markus Mäkelä
df86ee3579 Fix buffer overflow assertions
The buffer overflow assertions were off by one: The data pointer can be
equal to the end pointer when the last column of the row is processed.
2018-01-30 15:59:05 +02:00
Markus Mäkelä
9146a215f7 Fix DDL table identifier parsing
The parsing was inadequate and did not support all forms of quoted
identifiers.
2018-01-30 15:59:05 +02:00
Markus Mäkelä
b7e475f316 MXS-1621: Detect TABLE_MAP ↔ TABLE_CREATE column count mismatch
If the TABLE_MAP and TABLE_CREATE have different column counts, an error
is logged and the row events are skipped.
2018-01-30 15:59:05 +02:00
Markus Mäkelä
c000b3186c MXS-1575: Fix optional COLUMN keywork handling
The COLUMN keyword is optional and cannot be assumed to exist.
2018-01-30 15:59:05 +02:00
Markus Mäkelä
8dfb1d0113 MXS-1621: Add ALTER TABLE ... [FIRST | AFTER col ] parsing
The parser checks whether the FIRST or AFTER keywords are used and, if
AFTER is used, extracts the relevant column name.

Added a test case that checks that the parsing works and detects the
correct column names.
2018-01-30 15:59:05 +02:00
Johan Wikman
6410b4f19a MXS-1633 Turn off collecting of sqlite3 memstats
According to customer reports collecting the statistics has a significant
impact on the performance. As we don't need that information we can just
as well turn off that.

Further, since maxscale-common now links to the sqlite3-library, no
module needs to do that explicitly.
2018-01-30 13:58:37 +02:00
Markus Mäkelä
0afe10ffb9 Merge branch '2.1' into 2.2 2018-01-16 09:47:49 +02:00
Markus Mäkelä
9ca6d586d1 MXS-1575: Fix Avro schema versioning
The versions were not updated if the table was dropped and created again.
2018-01-15 16:15:20 +02:00
Markus Mäkelä
dbad6b737d Detect redundant table map events
When two identical tables are mapped into the same slot, the newer one is
ignored.
2018-01-15 16:15:20 +02:00
Markus Mäkelä
5e4b7ae7c7 Fix memory leak in avrorouter
The TABLE_MAP freeing function leaked memory.
2018-01-15 16:15:20 +02:00
Markus Mäkelä
f2771d5ad8 Remove obsolete message
The warning that a schema already exists is obsolete as mapped tables are
now always opened instead of being reused. This causes the schema checks
to be done for each mapped table.
2018-01-15 16:15:19 +02:00
Markus Mäkelä
3f78dbc923 Improve avrorouter assertion output
When an assertion fails due to an overflow of the event buffer, all
processed values for that event are dumped.

This commit also enables the assertions even for non-debug builds which
should speed up the elimination process for bugs in the avrorouter. The
overhead of doing this is minimal as the output is already gathered for
the INFO level logging.
2018-01-05 13:39:57 +02:00
Markus Mäkelä
ce1c45828a MXS-1575: Fix CREATE TABLE processing
The CREATE TABLE processing failed to identify the explicit database names
that were generated by mysqldump.
2018-01-05 12:50:39 +02:00
Markus Mäkelä
95ad3aa7df MXS-1543: Log a warning if STATEMENT or MIXED is used
If avrorouter suspects that statement based or mixed replication formats
are being used, it logs a warning.
2018-01-05 12:50:39 +02:00
Markus Mäkelä
7d8905d91f Merge branch '2.1' into 2.2 2018-01-03 14:05:31 +02:00
Markus Mäkelä
f5484a1011 MXS-1576: Add purge command to avrorouter
The `purge` command can be used to reset the conversion
process. Currently, executing the `purge` module command and restarting
MaxScale is the only correct way to reset the conversion process.
2018-01-03 09:44:41 +02:00
Markus Mäkelä
a86b81824f MXS-1575: Add extra debug assertions
Added more debug assertions to JSON serialization of the Avro schema. Also
checked that the column count for the TABLE_CREATE object matches that of
the TABLE_MAP object.
2018-01-03 09:44:41 +02:00
Markus Mäkelä
4fd8ebd9eb Add missing include
The utils header was not included.
2017-12-21 12:28:54 +02:00
Markus Mäkelä
ff6bed6aeb Merge branch '2.1' into 2.2 2017-12-21 12:25:26 +02:00
Markus Mäkelä
044339d37f Add missing detection of CREATE TABLE AS
As MaxScale does not have the actual data returned by the SELECT SQL query
done to create the table, CREATE TABLE AS cannot be supported.
2017-12-14 19:06:17 +02:00
Markus Mäkelä
2f56757839 Fix ENUM/SET string representation
The value is now stored as a hexadecimal value instead of a decimal
value. This also lifts the enum size restriction on more than 8 values
(which was incorrectly documented as 255 values).
2017-12-14 19:06:17 +02:00
Markus Mäkelä
bdcda0f235 Fix empty TIME2 values
The values for TIME2 were always empty as they weren't processed.
2017-12-14 19:06:17 +02:00
Markus Mäkelä
49a5dd9390 Fix BIT size calculation
The size of a BIT field was not calculated correctly.
2017-12-14 19:06:17 +02:00
Markus Mäkelä
79afaa447e Merge branch '2.1' into 2.2 2017-12-12 13:23:02 +02:00
Markus Mäkelä
8ed1ff67c2 Fix avro regression
The fact whether a query explicitly defined a query was ignored by the
statement parsing function and it always assumed the database was
explicitly defined. This caused the TABLE keyword in a CREATE TABLE
statement to be falsely identified as the current database.
2017-12-07 13:19:06 +02:00
Markus Mäkelä
7c2a173a17 MXS-1536: Always recreate TABLE_MAP objects
When a table map event is read after an alter table, the old TABLE_MAP
object contains old information. Due to this, as well as the added benefit
of making the code easier to read, the recycling of TABLE_MAP objects was
removed. In practice, there were no benefits to re-mapping the tables to a
different ID.
2017-11-30 12:38:43 +02:00
Markus Mäkelä
a32789d4ac MXS-1536: Fix ALTER TABLE parsing
The ALTER TABLE parsing didn't properly handle escaped values and it used
the current database unconditionally.
2017-11-30 12:38:43 +02:00
Markus Mäkelä
764863cb3f MXS-1536: Add Annotate_rows event processing
The Annotate_rows events were not processed which caused the following
table map event to be ignored.

Also removed a false debug assertion. The byte count can be zero and the
pointer is not guaranteed to point to anything valid.
2017-11-30 12:38:43 +02:00
Markus Mäkelä
0ca04cb5d5 MXS-1536: Fix crash in CREATE TABLE
If a CREATE TABLE statement had a quoted keyword as the name of a field,
the calculated column count and actual column counts would differ.

In addition to this, oneline comments before the end of the statement
would truncate the SQL due to the fact that the whitespace was squashed
before the comment removal was done.
2017-11-30 12:38:43 +02:00
Markus Mäkelä
a971aa25da Merge branch '2.1' into 2.2 2017-10-30 11:01:19 +02:00
Markus Mäkelä
63cbf56cb2 MXS-1500: Fix real_type values
The characters in the type weren't checked for correctness which caused
the processing to read more characters than was intended.
2017-10-30 10:25:03 +02:00
Markus Mäkelä
895d950da0 Format all source files with Astyle
Formatted all source files Astyle.
2017-09-28 07:04:21 +03:00
Markus Mäkelä
25f9cc14b4 Update avrorouter and the related tutorial
The avrorouter now uses the parameters from the source service. This
removes the need for redundant parameter definition in the avrorouter
service when they are defined in the binlogrouter service as parameters.

Added some missing configuration sanity checks and updated the tutorial to
reflect the new configuration method introduced in 2.1.
2017-09-25 10:46:55 +03:00
Markus Mäkelä
3676dcebb1 MXS-1398: Clean up mysql.h header
Cleaned up the MaxScale version of the mysql.h header by removing all
unused includes. This revealed a large amount of dependencies on these
removed includes in other files which needed to be fixed.

Also sorted all includes in changed files by type and alphabetical
order. Removed explicit revision history from modified files.
2017-09-14 15:30:43 +03:00
Johan Wikman
02b9e0a01d Merge branch '2.1.7' into develop-2.1-merge 2017-09-12 11:08:02 +03:00
Markus Mäkelä
f98d4c1dbf MXS-1365: Fix buffer reallocation in avrorouter
Buffer size calculation is now done correctly.
2017-08-17 13:39:20 +03:00
Markus Mäkelä
67386980e3 MXS-1341: Ignore binlog checksums in avrorouter
If the binlog has binlog checksums enabled, the extra checksum bytes are
removed from the end of the event. The avrorouter assumes that whatever
caused the binlogs to appear in the first place already checked that the
checksums are OK.

Also removed one extra byte being added to the length of all query events.
2017-08-16 13:24:52 +03:00
Markus Mäkelä
26bf342e6b Merge branch '2.1' into develop 2017-07-24 12:37:25 +03:00
Markus Mäkelä
fd77b18c49 Remove false debug assertion in avrorouter
The asserted value can be false without it being an error. When a table is
re-mapped to a different position, there is no guarantee that the previous
value has not been reused by another table.
2017-07-13 12:30:24 +03:00
Markus Mäkelä
11d494ef33 Fix avro CREATE TABLE message
The string was assumed to be null-terminated when in reality it was not.
2017-07-04 12:40:22 +03:00
Markus Mäkelä
285086ba7c Fix ALTER TABLE regression in avrorouter
The avrorouter failed to detect ALTER TABLE statements which caused a
regression. Extended the alter table tests to parse the JSON for more
strict validation of test results.
2017-07-04 12:40:22 +03:00
Markus Mäkelä
5ed6483f01 Fix avro CREATE TABLE message
The string was assumed to be null-terminated when in reality it was not.
2017-07-04 12:21:25 +03:00