454 Commits

Author SHA1 Message Date
Markus Makela
affba3cd65 Merge branch '1.2.1-binlog_router_trx' into release-1.4.2 2016-04-15 11:15:01 +03:00
Markus Makela
2855389fe2 Non-critical errors in binlogrouter are now properly logged as warnings
The event distribution error messages were not truly errors but only rare
corner cases that usually don't happen. This doesn't mean that they should
be treated as errors and treating them as warnings is more helpful.
2016-04-15 10:37:21 +03:00
Markus Makela
aabcc4fb12 Added clarifying comments to event distribution code
When a slave transitions from catchup mode to up-to-date mode, it is possible
that events are being distributed and the slave appears to be lagging behind.
The added comment explain why it happens and how it is handled to make the code
clearer.
2016-04-15 10:18:50 +03:00
Markus Makela
25f4238231 Merge branch '1.2.1-binlog_router_trx' into release-1.4.2 2016-04-14 18:52:36 +03:00
Markus Makela
edb1c18e83 Added missing check for error packets
A missing else branch caused the binlogrouter to crash when it
requests non-existent binlog file.
2016-04-14 17:51:42 +03:00
Markus Makela
bb2d549a00 Slaves that are behind the master no longer cause an error to be logged
When a slave transitions from catchup mode to up-to-date mode, an error
message is logged because the slave is at an unexpected position. This
error message should not be logged because it is a possible and an expected
situation.
2016-04-14 12:01:02 +03:00
Markus Makela
6988c0bfed Fixed rotate events never being distributed
The check for rotate event conditions was wrong which led to false error
message about unexpected binlog file and position combinations.

The position of the last event was reset every time a file was opened which
caused problems when the binlog file was rotated. The slave's current positions
were compared to the position where the last event started and because the
last_event_pos variable didn't point to the rotate event of the previous binlog,
the slave's never got the rotate event.
2016-04-14 11:02:03 +03:00
Markus Makela
440bc049c0 Added more information to binlogrouter error messages
The name of the binlog file was added to the log message where a slave
is behind the master but the same file is used. This makes debugging the problem
a bit easier.
2016-04-14 11:02:03 +03:00
Markus Makela
df9f8eba82 Improved binlogrouter event distribution error messages
Added master position and current safe event to log messages about unexpected
slave binlog file positions.
2016-04-14 11:02:03 +03:00
MassimilianoPinto
335b1e095f Added master pos to log message
Added master pos to log message
2016-04-14 11:02:03 +03:00
Markus Makela
52ef8987cb Made sending of events more strict
The decision to send an event to a slave can now only be made in one place.
This will force all events to pass the same checks before they are sent to
the slaves.
2016-04-14 11:02:03 +03:00
Markus Makela
aa7e3461c8 Fixed event position updating
The position of the next event to be written was used as the position
of the current event. This caused the checks for the position of the current
safe event to fail and the non-transaction safe version was used.

This only happened with events that are not done inside a transaction i.e.
DDL statements.
2016-04-14 11:02:03 +03:00
Markus Makela
cf62ac0e6c Added more information to the incomplete transaction error message
The error message now logs the caller role, current safe event and the event
type.
2016-04-14 11:02:01 +03:00
Markus Makela
fe84f9599b Removed messages about incomplete transaction events
The message is logged when a DDL statement is executed. It should not be
logged if trx_safe is on since the current_safe_event should always point
at the event we are sending. The current_safe_event is set to the wrong value
which causes this message to be logged.

Due to the false positives caused by this, the message is removed.
2016-04-07 12:51:38 +03:00
Markus Makela
bb94ffb9ea Added more information to duplicate event error message
The message now states the location where it was called from and the amount
of events received from the master. In addition to this, new logging was
added when unsafe events are sent to slaves when transaction safety is enabled.
2016-04-06 17:30:25 +03:00
Markus Makela
0ce48474eb Added logging for safe event and current event mismatch
If the position being currently processed is not the current safe event,
a log message is written.
2016-04-05 16:57:39 +03:00
Markus Makela
ed9356562c Added DCB write queue to error message
The duplicate event error message now logs the length of the slave's
write queue. This will tell how much data is still buffered inside MaxScale
when duplicate events are detected.
2016-04-05 15:01:30 +03:00
Johan Wikman
0493196ecc Disconnect slave if duplicate event detected
If a duplicate event is detected the state of the slave is set
to BLRS_ERRORED and the connection is closed. That way the
duplicate event will not break the slave, and it will pick
up its state when it reconnects.
2016-03-31 10:57:23 +03:00
Johan Wikman
51e60000dd Add duplicate event detection & logging.
When an event is sent to a slave, we store information about the
event and who sent it, so that we can detect if the same event is
sent twice. If a duplicate event is detected, we log information
about it.
2016-03-31 10:50:47 +03:00
Johan Wikman
5070b81473 Reformat binlog router. 2016-03-15 12:56:41 +02:00
Markus Makela
7bae03f3b8 Added missing PCRE link flags to binlogrouter
MariaDB 10.0 might require it to be linked against system PCRE. This will allow
the binlogrouter to be built with MariaDB 10.0 embedded libraries.
2016-03-05 06:23:35 +02:00
Johan Wikman
a86903f3d0 Remove obsolete declarations.
And dummy definitions as well.
2016-03-04 14:10:34 +02:00
Markus Makela
d3da98b58f Merge branch 'MXS-544' into develop-MXS-544-merge 2016-03-03 21:39:39 +02:00
Johan Wikman
1d4779215d Ensure symbols are resolved at link-time.
With the linker flags "-Wl,-z,defs", all symbols used by a library
are resolved at link-time. Otherwise they will be resolved at runtime.
The use of these flags ensures that missing symbols are found as
early as possible.

Case in point, the binlog router test-cases failed, because the loading
of the binlog router failed due to missing symbols my_uuid_init and
my_uuid. The reason was that when maxscale no longer was linked with
the embedded library, those symbols were not available.

Now we know that the loading of the binlog router will not fail due
to missing symbols.
2016-03-01 21:29:13 +02:00
Johan Wikman
9b5550c053 Link binlog router with embedded lib.
Binlog router uses my_uuid_init and my_uuid, which are non-public
functions available in the embedded library. Consequently, blr
must currently be linked with the embedded library.

A custom implementation of these functions should be provided, in
order to break that dependency.
2016-03-01 21:25:47 +02:00
Johan Wikman
9a0534d0f6 Use maxscale_uptime().
MaxScaleUptime() has been renamed to maxscale_uptime(). Take
prototype from include file.
2016-03-01 21:23:43 +02:00
Markus Makela
bf24f18805 Merge branch 'develop' into MXS-544 2016-02-29 10:40:33 +02:00
counterpoint
395cbdc103 Simplify handling of "client data" by having a pointer to it only in DCB and not in session structure. Change name of session->client to session->client_dcb for greater clarity. Temporary free of client data in DCB, to be moved to authenticator when it becomes a module. Fix incorrect name of listener_alloc. 2016-02-23 11:13:21 +00:00
Markus Makela
28bd7c2202 Merge branch '1.2.1-binlog_router_trx' into develop 2016-02-19 18:51:59 +02:00
Markus Makela
59f5880898 Added missing OK byte to payload size calculation
The OK byte was not taken into notice when the total size of all the payloads
in all the packets was calculated.
2016-02-17 08:30:50 +02:00
Markus Makela
63ce9fe6bc Fixed formatting and added more error checks
Added log messages when ftruncate fails and cleaned up formatting.
2016-02-16 13:06:25 +02:00
Markus Makela
cd2af6ffef Cleaned up the code based on the code review
Added missing error condition checks and cleaned up code.
2016-02-16 13:06:25 +02:00
Markus Makela
a55f017c75 Fixed packets with a length of one being ignored
The packets were not written into the binlogs which caused binlog corruption.
2016-02-16 13:06:25 +02:00
Markus Makela
9306b9d68c Added detection of checksums split across two packets
The checksums should now be processed properly event if the event is in more than
one packet.
2016-02-16 13:06:25 +02:00
Markus Makela
36896afcbd Fixed missing NULL check when reading records
If the binlog record was not found, a NULL pointer is returned. There was no
check for the return value and it assumed that it was always non-NULL.
2016-02-16 13:06:25 +02:00
Markus Makela
74c8b5e296 Fixed events larger than 2^24 failing without transaction safety
If transaction safety was disabled and a large event sent in multiple SQL
packets was received, the distribution of that event to the slaves would fail.
2016-02-16 13:06:25 +02:00
Markus Makela
2b7e2d3043 Added checksum calculations for events larger than 2^24 bytes
The checksums are now properly calculated for large events that span multiple
SQL packets.
2016-02-16 13:06:25 +02:00
Markus Makela
3609f97ba0 Fixed events which are exactly 0x00fffffe bytes long failing to replicate
The empty packet sent after a large event which fits into exactly one packet
was written to disk and the writing of no bytes caused it to be treated as
an error.
2016-02-16 13:06:25 +02:00
MassimilianoPinto
476691eda1 Removed log message for event larger than 16MB
The log message used during tests is now removed
2016-02-16 13:06:25 +02:00
Markus Makela
3e04a36ac3 Added support for distribution of packets larger than 2^24 bytes
Moved the the sending of the replication events to a different function
and added support for events that span multiple MySQL packets.
2016-02-16 13:06:25 +02:00
Markus Makela
12ee568978 Fixed last_written being set to the size of the event
The addition used =+ instead of += which caused it to be an assignment.
2016-02-16 13:06:25 +02:00
Markus Makela
d2b4713d27 Added missing condition to else clause
This fixes all packets being considered as large packets.
2016-02-16 13:06:25 +02:00
Markus Makela
ae33df3cbc Large events are now processed in chuncks
The router->last_written is used to store the position where the last event was
written. The replication header is also stored in a separate structure in
the router which is used later when the last packet of a multi-packet event
arrives.
2016-02-16 13:06:24 +02:00
MassimilianoPinto
d3e1d4dd2f First fix for 16MB handling in the master part
First fix for 16MB handling in the master part.

Distribute events to up to date slave is not included yet
2016-02-16 13:06:24 +02:00
MassimilianoPinto
ab1fb90d86 Fix for 16MB transmission in the slave part
Fix for 16MB transmission in the slave part
2016-02-16 13:06:24 +02:00
MassimilianoPinto
e777828bd0 When creating heartbeat packet too many bytes were copied.
The memory area ‘ptr’ points to contains now the right data
2016-02-11 17:12:10 +01:00
MassimilianoPinto
4eccc1acb6 When creating heartbeat packet too many bytes were copied.
The memory area ‘ptr’ points to contains now the right data
2016-02-11 17:06:13 +01:00
Johan Wikman
4a4f22c9a7 Make query_classifier part of core.
The query classifier interface is now built as an integral
part of MaxScale core.
2016-02-08 11:44:07 +02:00
Markus Makela
081bacdc64 Switched to MySQL client library headers for MaxScale
The include directories previously used by MaxScale were from the embedded
library. All parts of MaxScale apart from the query classifier now use
the client libraries.
2016-02-02 14:04:59 +02:00
MassimilianoPinto
0ab9733393 The router->rotating is no longer part of Unsafe Pos check
In blr_read_binlog the router->rotating is no longer used for Unsafe
Pos check
2016-02-01 09:12:48 +01:00