Commit Graph

5858 Commits

Author SHA1 Message Date
fd2df3a863 Always open files with configured block size
If the Avro file already exists, it should be opened with the configured
block size instead of the default block size of 16 kilobytes.
2017-03-08 20:03:45 +02:00
60c69d0f7f Update change log and release notes for 2.0.5 2017-03-07 10:33:50 +02:00
4f8fdca6ae Update version to 2.0.5 2017-03-07 10:33:50 +02:00
dd07feda07 MXS-1025: Parse SAVEPOINT statements
The various forms of the SAVEPOINT/RELEASE/ROLLBACK statements are now
fully parsed.
2017-03-07 08:44:21 +02:00
f2fc9b9d9f Add workaround for null value handling in Avro C API
The Avro C API fails to write bytes of size zero. A workaround is to write
a single zero byte for each NULL field of type bytes.

Also added an option to configure the Avro block size in case very large
records are written.
2017-03-04 10:12:18 +02:00
09df0acb00 Fix binlog rotation detection
The rotations of binlogs weren't detected as the file names weren't
compared.

Moved the indexing of the binlogs to the end of the binlog
processing. This way the files can be flushed multiple times before they
are indexed.
2017-03-04 00:31:07 +02:00
dadc0d6a9d Fix DATETIME and BLOB processing
The old DATETIME format wasn't processed properly which caused a
corruption of following events.

A BLOB type value could be non-NULL but still have no data. In this case,
the value should be stored as a null Avro value.
2017-03-04 00:26:51 +02:00
075ca42482 Expand slave_selection_criteria documentation
Added a section about how `slave_selection_criteria` and
`max_slave_connections` affect each other.
2017-03-02 13:33:45 +02:00
e2869052bd MXS-1081: Fix VARCHAR field processing
The data length is stored in the field metadata instead of the data being
encoded as a length encoded string.
2017-03-02 13:14:07 +02:00
e01a6a0d58 Fix field name parsing
The fix to field name parsing didn't properly break the loop when the
backtick character was detected.
2017-03-02 13:12:45 +02:00
e4b4dad94a Add more debug assertions to avro row processing
The debug assertions check that the event pointer isn't moving beyond
allocated memory.
2017-03-02 13:11:52 +02:00
97115fae6a Fix threads documentation
The description of the `auto` value was wrong. The algorithm is:

    CPU count - 1
2017-03-02 12:21:06 +02:00
0de9561b5a MXS-1146: Fix command tracking for large packets
The current command was updated with invalid data when the packet size
exceeded 2^24 bytes.
2017-03-02 12:21:05 +02:00
f7cc548093 Format mysql_binlog.c
The lines were too long.
2017-02-13 16:23:26 +02:00
7a35ec71da Fix DECIMAL conversion
The DECIMAL type was not correctly converted to positive integers. The
0x80 bit was set only for negative numbers when it needed to be XOR-ed for
all values.
2017-02-13 16:23:26 +02:00
16ddefc686 Remove DECIMAL from avrorouter limitations
The router can now handle DECIMAL types.

Backported to the 2.0 branch.
2017-02-13 16:23:26 +02:00
f9732d7041 Fix DECIMAL handling in Avrorouter
The DECIMAL value type is now properly handled in Avrorouter. It is
processed into an Avro double value when before it was ignored and
replaced with a zero integer.

Backported to the 2.0 branch.
2017-02-13 16:23:26 +02:00
4bd743d3ce Use correct Avro function
The avrorouter used the 32-bit function to store 64-bit integers. This
caused incorrect values to be stored.
2017-02-13 16:23:26 +02:00
65caaab77b Fix maxavro block verification errors
An error was logged when the end of file was reached. The error should
only be logged when a partial sync marker is read and the end of file has
not been reached.
2017-02-13 16:23:26 +02:00
0b892c9714 Add missing newline to avrorouter output
The schema was not terminated with a newline.
2017-02-13 16:23:22 +02:00
10e74225c0 Add missing error detection
If the file fails to seek to the correct position, the indexing should
stop.
2017-02-13 16:18:43 +02:00
fb21b99983 Fix DATE and DATETIME months
The month values for DATE and DATETIME were off by one.
2017-02-09 16:50:14 +02:00
47a1cdad5d Fix binlog integer conversion
The binlog integers were stored as unsigned values instead of signed ones.
2017-02-09 16:01:03 +02:00
526e48e459 Fix floats being read as doubles
The avro floating point numbers were processed as 8 byte values when they
are 4 bytes values.
2017-02-09 15:00:19 +02:00
e3bed424ea MXS-1123: Fix connection_timeout causing constant disconnections
In a configuration with multiple services, one with connection_timeout and
others without it, the connections to non-connection_timeout services
would get immediately closed due to integer overflow.
2017-02-06 23:03:57 +02:00
c0f5124f6f Fix field name parsing in avrorouter
The backtick was copied to the field name and converted to an underscore
when the name was transformed into a valid Avro identifier. This caused
one extra character to appear in the field name in the Avro schema files.
2017-02-06 16:35:40 +02:00
6ee257dc5f Only log an error if the binlog file exists
The avrorouter logged an error every time it tried to open a file even if
the file doesn't exist.
2017-02-03 08:16:12 +02:00
eb5e284a90 Rephrase the schemarouter limitations
The text was not very clear as to whether the prepared statements were
actually supported.
2017-02-01 17:52:00 +02:00
3d02343a40 Update date in release notes 2017-02-01 15:07:53 +02:00
a8780b8924 Add PS limitation of schemarouter to limitations
As prepared satements aren't parsed by the schemarouter, it can't support
prepared statements.
2017-01-31 13:48:51 +02:00
00f16e1fa5 Make parameters for command_is_mandatory const
The parameters can be const as they aren't modified.
2017-01-31 10:16:33 +02:00
82105d20e1 Fix a typo in readwritesplit documentation
Fixed a typo in readwritesplit documentation. The cluster should be in
plural form.
2017-01-31 10:10:25 +02:00
6cd16d26b8 Flush the producer after every new record
As the cdc_kafka_producer script is an example, it should flush the
producer after every new record. This should make it easier to see that
events from MaxScale are sent to Kafka.
2017-01-30 20:14:27 +02:00
4bbd513b1e Update release notes
Added dbfwfilter changes and fixed bugs to release notes.
2017-01-30 19:26:18 +02:00
443af1aee3 Use preliminary 2.3.3 release of Connector/C
The new version has fixed a bug which MaxScale depends on.
2017-01-30 19:20:46 +02:00
fdee329198 MXS-1111: Allow COM_PING and other commands to pass the firewall
The firewall filter should allow COM_PING and other similar commands to
pass through as they are mainly used to check the status of the backend
server or to display statistics. The COM_PROCESS_KILL is the exception as
it affects the state of the backend server. This is better controlled with
permissions in the server than in the firewall filter.

Commands that require special grants aren't allowed to pass as they are
mainly for maintenance purposes and these should not be done through the
firewall.
2017-01-30 19:16:33 +02:00
122027e4a3 Fix binlogrouter test
The test used the wrong working directory. blr_slave.c didn't check the
input for NULL values.
2017-01-30 16:46:24 +02:00
22698fdf21 MXS-951: Backport fix to 2.0
Backported the fix for 2.0.4.
2017-01-27 16:17:39 +02:00
78545b2c5c Read complete lines with Kafka producer
There's no need to process the JSON twice as the Kafka producer is
expected to be used with the Python CDC client which already splits the
JSON with newlines.
2017-01-27 13:06:25 +02:00
81b9d51aab Fix crash on startup
The Avro file was initialized in the wrong order and uninitialized values
were used.
2017-01-27 12:55:07 +02:00
8da655b7cb Improve maxavro failure handling and error messages
When the creation of the Avro schema would fail for a file that is being
opened, the errors wouldn't be handled properly. Also free all allocated
memory on failure.

All errors that set errno are now properly logged with the error number
and message.
2017-01-26 16:16:41 +02:00
84040be182 Add missing error handling to Avro file handling
Some of the JSON errors weren't handled which could cause problems when a
malformed schema definition is read.

Also added more error messages for situations when opening of the files
fails.
2017-01-26 10:23:26 +02:00
c2b03100f0 Update changelog and release notes of 2.0.4 2017-01-25 15:30:57 +02:00
98c2d37156 MXS-1080: Document max_slave_replication_lag behavior
The max_slave_replication_lag parameter for readwritesplit only works for
monitors that detect replication lag. As the MySQL monitor is the only one
that implements this functionality, the parameter only has meaning when
used with master-slave clusters.
2017-01-19 13:10:46 +02:00
fed61fde98 Disable prepared statement execution in dbfwfilter
The dbfwfilter doesn't parse prepared statements for all rules which
requires that they are disabled.
2017-01-19 08:57:15 +02:00
3793f685ac Store DATETIME correctly in avrorouter
DATETIME was stored as a raw year value when it should've been stored as
the raw year value minus 1900.
2017-01-19 08:57:15 +02:00
8fc74e7b37 Update version for 2.0.4 2017-01-18 12:42:46 +02:00
68965639d2 Document zero weightby values
The behavior when a server had a zero value for the weighting parameter
was not documented.
2016-12-20 02:32:48 +02:00
7a08ea99af Change README to Markdown and fix Travis for 2.0
Converting the README into Markdown format makes it a lot easier to
comprehent. Also cleaned up the formatting.

The 2.0 branch had a broken Travis configuration. Fixed it and changed
links to point to 2.0 branch.
2016-12-19 10:55:28 +02:00
9e27e6c002 Don't ask for unlimited files
There's a bug reported on Launchpad
(https://bugs.launchpad.net/ubuntu/+source/upstart/+bug/1420640) that
describes a similar situation. The solution is to request a lower limit of
open files for the process.
2016-12-16 15:51:17 +02:00