Commit Graph

3108 Commits

Author SHA1 Message Date
057ba4156d MXS-1278: Turn on ORCALE mode dynamically
If statement input is required for a session, we check the input
for "SET SQL_MODE..." statements and adjust the query classifier
accordingly.
2017-06-28 21:36:08 +02:00
12a291919a MXS-1275: SetSqlModeParser moved under MySQLClient
That's where it belongs as it is only the mysql client protocol that
will use it. It's a bit unfortunate that the qc test program compare
now needs to include a file from a protocol module directory, but
the fact is that the query classifier implementation and the test
programs should actually be *under* the mysql client protocol module.
2017-06-28 21:36:08 +02:00
b385e862b0 MXS-1275: Move SetSqlModeParser to .../server/core/maxscale
The parser is needed not only by mysql_client but also by the test
program of the qc-plugins, so it need to be available someplace
common.

Not that nice that both a qc-component and mysql_client will include
stuff from server/core/maxscale, but at least temporarily ok.
2017-06-28 21:36:08 +02:00
4bf2b27b3e MXS-1275: Accept @@[SESSION|GLOBAL].sql_mode = ... 2017-06-28 21:36:08 +02:00
21c1845798 MXS-1275: Accept LOCAL as a synonum for SESSION 2017-06-28 21:36:08 +02:00
2483f95be2 MXS-1275: Test comment handling 2017-06-28 21:36:08 +02:00
1c6c19d6ae MXS-1275: Inherit TrxBoundaryParser from CustomParser 2017-06-28 21:36:08 +02:00
9386143437 MXS-1275: Test also using non-contiguous statements 2017-06-28 21:33:04 +02:00
ee27efd62e MXS-1275: Add custom parser for "SET SQL_MODE=..."
SetSqlModeParser is a custom parser for detecting "SET SQL_MODE=..."
statements. It will be used in mysql_client and the result will be
used for updating the sql_mode of a session (that is to be added).

Thereafter, whenever a statement arrives, the value of that sql_mode
will be used for setting the mode of the query_classifier.
2017-06-28 21:33:04 +02:00
cd1e3f26cf MXS-1275: Compile mysql_client as C++
A custom parser for detecting "set sql_mode=ORACLE" is needed
and writing that is C++ is more convenient than writing it in
C. Consequently, so as to be able to use that parser, mysql_client
must be compiled as C++.
2017-06-28 21:33:04 +02:00
40b1739249 MXS-1299: Fix CREATE TABLE t1 LIKE t2 processing
The creation of tables from other tables was not working even though the
information was available inside the avrorouter.
2017-06-28 11:25:57 +03:00
84925b3f10 Merge branch '2.0' into 2.1 2017-06-28 10:59:06 +03:00
9b0631f30b MXS-1289: Fix crash on TABLE_MAP with ID > 1024
The active table map was acquired with a modulo operation on the size of
the array instead of the number of elements.
2017-06-28 08:49:57 +03:00
1df8fda93f Merge branch '2.0' into 2.1 2017-06-27 20:52:58 +03:00
bd1da82c96 Fix cdc_schema.py
There's no need to use Python 3 for the script and using it introduces
problems due to the poor availability of MySQL Connector/Python for
Python3.

ENUM, SET and DECIMAL values should have a length of -1 as the length is
meaningless for these types.
2017-06-23 11:10:41 +03:00
469ed1de1b MXS-1271: Remove unnecessary encode call
The trailing newline can be removed from the string without encoding it
first.
2017-06-20 10:40:22 +03:00
b7a6e4b7b0 Fix HintParser line break detection
The HintParser wrongly ignored linebreaks, causing parsing faults
e.g. parsing too far or accepting invalid comments. Now, the parser
detects a line break and terminates comments unless they started with
'/*'. Also, fixed a memory leak when parsing parameter-value-combinations.
2017-06-19 10:49:47 +03:00
567fbce735 MXS-1287: Assign Slave status for all external slaves
Whenever a server which is a slave of an external master is detected, it
will be assigned the slave status. This will allow the status to be used
the way it was intended to be used.
2017-06-16 22:26:33 +03:00
7e59254454 Change modutil_get_SQL() to modutil_extract_SQL() + length limit
The modutil_get_SQL()-function allocates storage, while
modutil_extract_SQL() does not. The strings given by the latter
are not 0-terminated so require a length limit when matched using
regexec().

This commit changes the used function in those cases where the
sql-string is not modified nor is the pointer saved for later use.
2017-06-16 10:07:27 +03:00
4c9fc1a39e Merge branch '2.0' into 2.1 2017-06-14 13:36:40 +03:00
8d5c6ca00f Disallow all prepared statement commands with dbfwfilter
All binary protocol prepared statements should be rejected by the
dbfwfilter in 2.0 as they aren't supported.
2017-06-06 12:34:46 +03:00
c3ffc0ba9c Improve cdc.py error handling
When the CDC service reponds with an error, the program prints the error
and exits.
2017-06-05 20:40:29 +03:00
d93edf02e2 Fix SQL tokenization in avrorouter
There were minor bugs in the token processing which caused ALTER TABLE
statements to be skipped.
2017-06-05 19:59:25 +03:00
5964622839 Fix ALTER TABLE handling in avrorouter
The addition of field types and lengths wasn't added to the avrorouter
ALTER TABLE handler. This caused crashes when an alter table was done and
new rows were inserted afterwards.
2017-06-05 14:42:05 +03:00
09a445f1b7 Fix monitor credentials parameter names
The `monitoruser` and `monitorpw` parameters were mislabeled as `monuser`
and `monpw`. To allow backwards compatibility, the `monuser` and `monpw`
still work as aliases for the correct commands.
2017-06-05 13:20:16 +03:00
539474a28d Disable RocksDB tests in 2.1
The RocksDB based cache storage is not built by default so the tests
should be disabled.
2017-06-05 13:20:15 +03:00
4c78e2c99a Fix avrorouter memory leaks
The loading of the JSON schemas leaked memory as functions that increment
the reference count were used.
2017-06-05 13:20:14 +03:00
d5a72d1b75 Fix avrorouter client counts
The avro client numbers were decremented twice for each client.
2017-06-05 13:19:48 +03:00
285598e0e5 Fix avrorouter type and name handling
The type and name parsing functions could move outside of allocated memory
as they didn't check for the terminating null character. Also fixed the
printf format string used when the list of used tables is being created.

Fixed CDC testing connector to abort on error and added some extra output
to the cdc_datatypes test.
2017-06-05 13:19:47 +03:00
a37a665bd2 MXS-1271: Remove redundant JSON parsing
As the cdc.py script outputs newline delimited JSON, there's no need to
parse the JSON in the cdc_kafka_producer.py script.
2017-06-05 13:17:12 +03:00
9124de86df Add default value to MaskingFilter "fill" setting
MXS-1261. If "fill" is not defined, an "X" is used as default.
Setting an empty string as "fill" or "value" is disallowed.
2017-06-05 11:30:06 +03:00
d9a6b6f324 Fix avrorouter memory leaks
The loading of the JSON schemas leaked memory as functions that increment
the reference count were used.
2017-06-02 13:36:56 +03:00
4b6c5caa47 Fix avrorouter client counts
The avro client numbers were decremented twice for each client.
2017-06-02 10:55:33 +03:00
919d08af1e Fix avrorouter type and name handling
The type and name parsing functions could move outside of allocated memory
as they didn't check for the terminating null character. Also fixed the
printf format string used when the list of used tables is being created.

Fixed CDC testing connector to abort on error and added some extra output
to the cdc_datatypes test.
2017-06-02 10:26:12 +03:00
a3cfabf37b MXS-1271: Remove redundant JSON parsing
As the cdc.py script outputs newline delimited JSON, there's no need to
parse the JSON in the cdc_kafka_producer.py script.
2017-05-30 15:42:09 +03:00
b5d1504e6a Merge branch '2.0' into 2.1 2017-05-26 15:22:43 +03:00
ad109408b5 MXS-1216: Add real_type and length to cdc_schema.py
The schema generator program needs to add the real_type and length fields
if the data types define them.

Also fixed a bug where the real_type and length fields were checked for
generated fields.
2017-05-16 13:27:46 +03:00
0c9409ae7f Merge branch '2.0' into 2.1-merge-2.0 2017-05-15 10:35:44 +03:00
a12d19591e MXS-1216: Store field real type and length in Avro schema
The avro schema allows custom properties to be defined for the schema
fields. The avrorouter stored extra information about the table into the
schema for later use.

Currently, this information is only generated by the avrorouter
itself. Further improvements to the schema generator scripts need to be
done.
2017-05-15 10:25:12 +03:00
6b6a7fa4a1 Do checkpoint processing at end of binlog
When the binlog has been read, it needs to be treated as if the
transaction or row limit has been hit. This will cause all tables to be
flushed to disk before the files are indexed.
2017-05-15 10:25:11 +03:00
8a288110a9 Rename conflicting Avro fields
When a user defined field conflicts with an internal MaxScale field, the
field is suffixed with an underscore.
2017-05-15 10:25:11 +03:00
926930e241 MXS-1216: Correct CHAR(n) handling
The field length was wrongly compared to less than 255 for two byte field
lengths. In addition to that, the metadata was interpreted in the wrong
way.
2017-05-15 10:25:09 +03:00
c988735a03 MXS-1216: Fix crash on MariaDB 10.0 DATETIME(n)
When a MariaDB 10.0 DATETIME field with a custom length was defined, the
field offsets weren't calculated properly.

As there is no metadata for pre-10.1 DATETIME types with decimal
precision, the metadata (i.e. decimal count) needs to be gathered from the
CREATE TABLE statement. This information is then used to calculate the
correct field length when the value is decoded.

This change does not fix the incorrect interpretation of the old DATETIME
value. The converted values are still garbled due to the fact that the
value needs to be shifted out of the decimal format before it can be
properly converted.
2017-05-15 10:25:06 +03:00
ed653ac729 Merge branch '2.0' into 2.1-merge-2.0 2017-05-14 10:25:12 +03:00
c464bb4ee1 MXS-1216: Store field real type and length in Avro schema
The avro schema allows custom properties to be defined for the schema
fields. The avrorouter stored extra information about the table into the
schema for later use.

Currently, this information is only generated by the avrorouter
itself. Further improvements to the schema generator scripts need to be
done.
2017-05-12 16:47:20 +03:00
85b49520a2 Do checkpoint processing at end of binlog
When the binlog has been read, it needs to be treated as if the
transaction or row limit has been hit. This will cause all tables to be
flushed to disk before the files are indexed.
2017-05-12 11:30:19 +03:00
9ed972e1ac Rename conflicting Avro fields
When a user defined field conflicts with an internal MaxScale field, the
field is suffixed with an underscore.
2017-05-12 11:28:07 +03:00
da2f7b1efe MXS-1216: Correct CHAR(n) handling
The field length was wrongly compared to less than 255 for two byte field
lengths. In addition to that, the metadata was interpreted in the wrong
way.
2017-05-12 11:27:02 +03:00
30bd869f1c MXS-1216: Fix crash on MariaDB 10.0 DATETIME(n)
When a MariaDB 10.0 DATETIME field with a custom length was defined, the
field offsets weren't calculated properly.

As there is no metadata for pre-10.1 DATETIME types with decimal
precision, the metadata (i.e. decimal count) needs to be gathered from the
CREATE TABLE statement. This information is then used to calculate the
correct field length when the value is decoded.

This change does not fix the incorrect interpretation of the old DATETIME
value. The converted values are still garbled due to the fact that the
value needs to be shifted out of the decimal format before it can be
properly converted.
2017-05-12 11:22:04 +03:00
8e2c7fd952 Merge branch '2.0' into 2.1 2017-05-10 09:12:42 +03:00