4481 Commits

Author SHA1 Message Date
Markus Mäkelä
0c33572a96 MXS-1310: Detect implicit database usage
When the current database is implicitly used in a query that also uses an
explicit database, it must be routed to the shard which has the current
database.

As cross-shard joins are not supported, the safest, and possibly the most
expected course of action to take, is to route it to the so-called default
shard. The default shard is the shard that contains the database that is
currently set as the active database with a COM_INIT_DB, a text protocol
USE <database> query or it was set at connection time.
2017-07-04 23:21:04 +03:00
Markus Mäkelä
5ed6483f01 Fix avro CREATE TABLE message
The string was assumed to be null-terminated when in reality it was not.
2017-07-04 12:21:25 +03:00
Markus Mäkelä
5c50045227 Fix ALTER TABLE regression in avrorouter
The avrorouter failed to detect ALTER TABLE statements which caused a
regression. Extended the alter table tests to parse the JSON for more
strict validation of test results.
2017-07-04 09:31:11 +03:00
Markus Mäkelä
ff22243955 MXS-1308: Fix crash with multimaster=true
The multimaster node detection uses stacks to sort the node groups. The
size of this stack was always assumed to be positive but it was possible
that it dropped down to -1 causing a crash when the stack was accessed
with the index number.
2017-07-03 02:44:37 +03:00
Markus Mäkelä
322fae8326 Add wrapper functions for buffer and original IV access
Added functions for accessing the buffer and original IV. This hides the
changes introduced in OpenSSL 1.1.
2017-06-30 10:46:05 +03:00
Markus Mäkelä
a47d4c40f5 Abstract EVP cipher context creation
The EVP_CIPHER_CTX is now created inside a wrapper function to add support
for OpenSSL 1.1. Also fixed improper use of the EVP_CIPHER_CTX internals
in binlogrouter.
2017-06-30 10:46:04 +03:00
Markus Mäkelä
469b432ebe Detect and handle OpenSSL 1.1
OpenSSL 1.1 supports most of the native threading libraries, including
pthread. This means that only versions before 1.1 need the thread handling
code.
2017-06-30 10:42:20 +03:00
Markus Mäkelä
2bc5188dae MXS-1304: Don't null-terminate buffers in gw_str_xor
The gw_str_xor function is used only for doing XOR operations on binary
buffers.
2017-06-28 11:36:17 +03:00
Markus Mäkelä
40b1739249 MXS-1299: Fix CREATE TABLE t1 LIKE t2 processing
The creation of tables from other tables was not working even though the
information was available inside the avrorouter.
2017-06-28 11:25:57 +03:00
Markus Mäkelä
84925b3f10 Merge branch '2.0' into 2.1 2017-06-28 10:59:06 +03:00
Markus Mäkelä
9b0631f30b MXS-1289: Fix crash on TABLE_MAP with ID > 1024
The active table map was acquired with a modulo operation on the size of
the array instead of the number of elements.
2017-06-28 08:49:57 +03:00
Markus Mäkelä
dc849d1c0f MXS-1291: Attempt to bind on 0.0.0.0 when :: fails
If binding on the IPv6 all interfaces address fails, MaxScale will attempt
to bind on the IPv4 address.
2017-06-27 21:27:20 +03:00
Markus Mäkelä
1df8fda93f Merge branch '2.0' into 2.1 2017-06-27 20:52:58 +03:00
Markus Mäkelä
cd09e65714 MXS-1296: Always use a case-insensitive parser
The two cases where the case-sensitive parser functions were used don't
appear to hold any special meaning. The case-insensitive function should
be used as it implements a superset of functionality compared to the
case-sensitive version.
2017-06-26 21:05:42 +03:00
Johan Wikman
500f79a150 MXS-1296: Test that lc statements are parsed correctly
The transaction boundary parser - TrxBoundaryParser - must handle
upper and lowercase statements correctly.
2017-06-26 10:57:27 +03:00
Markus Mäkelä
bd1da82c96 Fix cdc_schema.py
There's no need to use Python 3 for the script and using it introduces
problems due to the poor availability of MySQL Connector/Python for
Python3.

ENUM, SET and DECIMAL values should have a length of -1 as the length is
meaningless for these types.
2017-06-23 11:10:41 +03:00
Markus Mäkelä
469ed1de1b MXS-1271: Remove unnecessary encode call
The trailing newline can be removed from the string without encoding it
first.
2017-06-20 10:40:22 +03:00
Esa Korhonen
b7a6e4b7b0 Fix HintParser line break detection
The HintParser wrongly ignored linebreaks, causing parsing faults
e.g. parsing too far or accepting invalid comments. Now, the parser
detects a line break and terminates comments unless they started with
'/*'. Also, fixed a memory leak when parsing parameter-value-combinations.
2017-06-19 10:49:47 +03:00
Markus Mäkelä
567fbce735 MXS-1287: Assign Slave status for all external slaves
Whenever a server which is a slave of an external master is detected, it
will be assigned the slave status. This will allow the status to be used
the way it was intended to be used.
2017-06-16 22:26:33 +03:00
Esa Korhonen
7e59254454 Change modutil_get_SQL() to modutil_extract_SQL() + length limit
The modutil_get_SQL()-function allocates storage, while
modutil_extract_SQL() does not. The strings given by the latter
are not 0-terminated so require a length limit when matched using
regexec().

This commit changes the used function in those cases where the
sql-string is not modified nor is the pointer saved for later use.
2017-06-16 10:07:27 +03:00
Markus Mäkelä
4c9fc1a39e Merge branch '2.0' into 2.1 2017-06-14 13:36:40 +03:00
Markus Mäkelä
8d5c6ca00f Disallow all prepared statement commands with dbfwfilter
All binary protocol prepared statements should be rejected by the
dbfwfilter in 2.0 as they aren't supported.
2017-06-06 12:34:46 +03:00
Markus Mäkelä
c3ffc0ba9c Improve cdc.py error handling
When the CDC service reponds with an error, the program prints the error
and exits.
2017-06-05 20:40:29 +03:00
Markus Mäkelä
d93edf02e2 Fix SQL tokenization in avrorouter
There were minor bugs in the token processing which caused ALTER TABLE
statements to be skipped.
2017-06-05 19:59:25 +03:00
Markus Mäkelä
5964622839 Fix ALTER TABLE handling in avrorouter
The addition of field types and lengths wasn't added to the avrorouter
ALTER TABLE handler. This caused crashes when an alter table was done and
new rows were inserted afterwards.
2017-06-05 14:42:05 +03:00
Markus Mäkelä
09a445f1b7 Fix monitor credentials parameter names
The `monitoruser` and `monitorpw` parameters were mislabeled as `monuser`
and `monpw`. To allow backwards compatibility, the `monuser` and `monpw`
still work as aliases for the correct commands.
2017-06-05 13:20:16 +03:00
Markus Mäkelä
539474a28d Disable RocksDB tests in 2.1
The RocksDB based cache storage is not built by default so the tests
should be disabled.
2017-06-05 13:20:15 +03:00
Markus Mäkelä
4c78e2c99a Fix avrorouter memory leaks
The loading of the JSON schemas leaked memory as functions that increment
the reference count were used.
2017-06-05 13:20:14 +03:00
Markus Mäkelä
d5a72d1b75 Fix avrorouter client counts
The avro client numbers were decremented twice for each client.
2017-06-05 13:19:48 +03:00
Markus Mäkelä
285598e0e5 Fix avrorouter type and name handling
The type and name parsing functions could move outside of allocated memory
as they didn't check for the terminating null character. Also fixed the
printf format string used when the list of used tables is being created.

Fixed CDC testing connector to abort on error and added some extra output
to the cdc_datatypes test.
2017-06-05 13:19:47 +03:00
Markus Mäkelä
a37a665bd2 MXS-1271: Remove redundant JSON parsing
As the cdc.py script outputs newline delimited JSON, there's no need to
parse the JSON in the cdc_kafka_producer.py script.
2017-06-05 13:17:12 +03:00
Esa Korhonen
9124de86df Add default value to MaskingFilter "fill" setting
MXS-1261. If "fill" is not defined, an "X" is used as default.
Setting an empty string as "fill" or "value" is disallowed.
2017-06-05 11:30:06 +03:00
Markus Mäkelä
d9a6b6f324 Fix avrorouter memory leaks
The loading of the JSON schemas leaked memory as functions that increment
the reference count were used.
2017-06-02 13:36:56 +03:00
Markus Mäkelä
4b6c5caa47 Fix avrorouter client counts
The avro client numbers were decremented twice for each client.
2017-06-02 10:55:33 +03:00
Markus Mäkelä
919d08af1e Fix avrorouter type and name handling
The type and name parsing functions could move outside of allocated memory
as they didn't check for the terminating null character. Also fixed the
printf format string used when the list of used tables is being created.

Fixed CDC testing connector to abort on error and added some extra output
to the cdc_datatypes test.
2017-06-02 10:26:12 +03:00
Markus Mäkelä
a3cfabf37b MXS-1271: Remove redundant JSON parsing
As the cdc.py script outputs newline delimited JSON, there's no need to
parse the JSON in the cdc_kafka_producer.py script.
2017-05-30 15:42:09 +03:00
Markus Mäkelä
b5d1504e6a Merge branch '2.0' into 2.1 2017-05-26 15:22:43 +03:00
Markus Mäkelä
ad109408b5 MXS-1216: Add real_type and length to cdc_schema.py
The schema generator program needs to add the real_type and length fields
if the data types define them.

Also fixed a bug where the real_type and length fields were checked for
generated fields.
2017-05-16 13:27:46 +03:00
Markus Mäkelä
4050cd20d8 Merge branch '2.1-merge-2.0' into 2.1 2017-05-15 10:36:58 +03:00
Markus Mäkelä
0c9409ae7f Merge branch '2.0' into 2.1-merge-2.0 2017-05-15 10:35:44 +03:00
Markus Mäkelä
5a0d2c54bd MXS-1216: Fix DATETIME(n) value interpretation
The DATETIME(n) values generated by a MariaDB 10.0 server were not
interpreted correctly as the wrong algorithm was used to extract the
values.

DATETIME(0) values still do not work properly and they require further
debugging and changes to the code.
2017-05-15 10:25:12 +03:00
Markus Mäkelä
a12d19591e MXS-1216: Store field real type and length in Avro schema
The avro schema allows custom properties to be defined for the schema
fields. The avrorouter stored extra information about the table into the
schema for later use.

Currently, this information is only generated by the avrorouter
itself. Further improvements to the schema generator scripts need to be
done.
2017-05-15 10:25:12 +03:00
Markus Mäkelä
6b6a7fa4a1 Do checkpoint processing at end of binlog
When the binlog has been read, it needs to be treated as if the
transaction or row limit has been hit. This will cause all tables to be
flushed to disk before the files are indexed.
2017-05-15 10:25:11 +03:00
Markus Mäkelä
8a288110a9 Rename conflicting Avro fields
When a user defined field conflicts with an internal MaxScale field, the
field is suffixed with an underscore.
2017-05-15 10:25:11 +03:00
Markus Mäkelä
926930e241 MXS-1216: Correct CHAR(n) handling
The field length was wrongly compared to less than 255 for two byte field
lengths. In addition to that, the metadata was interpreted in the wrong
way.
2017-05-15 10:25:09 +03:00
Markus Mäkelä
c988735a03 MXS-1216: Fix crash on MariaDB 10.0 DATETIME(n)
When a MariaDB 10.0 DATETIME field with a custom length was defined, the
field offsets weren't calculated properly.

As there is no metadata for pre-10.1 DATETIME types with decimal
precision, the metadata (i.e. decimal count) needs to be gathered from the
CREATE TABLE statement. This information is then used to calculate the
correct field length when the value is decoded.

This change does not fix the incorrect interpretation of the old DATETIME
value. The converted values are still garbled due to the fact that the
value needs to be shifted out of the decimal format before it can be
properly converted.
2017-05-15 10:25:06 +03:00
Markus Mäkelä
ed653ac729 Merge branch '2.0' into 2.1-merge-2.0 2017-05-14 10:25:12 +03:00
Markus Mäkelä
70db289c60 MXS-1216: Fix DATETIME(n) value interpretation
The DATETIME(n) values generated by a MariaDB 10.0 server were not
interpreted correctly as the wrong algorithm was used to extract the
values.

DATETIME(0) values still do not work properly and they require further
debugging and changes to the code.
2017-05-12 16:50:41 +03:00
Markus Mäkelä
c464bb4ee1 MXS-1216: Store field real type and length in Avro schema
The avro schema allows custom properties to be defined for the schema
fields. The avrorouter stored extra information about the table into the
schema for later use.

Currently, this information is only generated by the avrorouter
itself. Further improvements to the schema generator scripts need to be
done.
2017-05-12 16:47:20 +03:00
Markus Mäkelä
85b49520a2 Do checkpoint processing at end of binlog
When the binlog has been read, it needs to be treated as if the
transaction or row limit has been hit. This will cause all tables to be
flushed to disk before the files are indexed.
2017-05-12 11:30:19 +03:00