Commit Graph

2816 Commits

Author SHA1 Message Date
95935991ab MXS-1881: Move Rpl related code out of Avro files
Organized the Rpl and Avro code in a way that they aren't mixed.
2018-06-16 23:29:40 +03:00
a3a4bb3da6 MXS-1881: Add TableCreateEvent JSON serialization
Serializing the object allows replicated events to be processed even if
the CREATE TABLE statement is not in the binary logs.
2018-06-16 23:29:40 +03:00
3417842e75 MXS-1881: Add table creation method to RowEventHandler
The new method is called for each new CREATE TABLE statement that is
processed as well as all ALTER TABLE statemets that modify the table
structure.

Right now the entry point not in use but it opens up the possibility of
persisting the CREATE TABLE statements at creation time. Currently the
tables are only persisted when the first actual event for the table is
received.
2018-06-16 23:29:39 +03:00
c27529c9a4 MXS-1881: Expand GTID helper class
Added string conversion methods to the gtid_pos_t class that can be used
to store and load a GTID value.

Also added the missing rpl.cc file that previously only had the Rpl class
constructor in it.
2018-06-16 23:29:39 +03:00
ac6370afcf Clean up common binlogrouter headers
Cleaned up blr_constants.h and moved the REP_HEADER construction helper
into binlog_common.h.
2018-06-16 23:29:39 +03:00
e74591cfe5 MXS-1881: Delegate event processing to the Rpl class
The actual processing of the replicated events is now delegated to the Rpl
class. This class only deals with the raw binary format log events which
allows it to be used for both binlogs stored on disk as well as binlogs
that have just been replicated.
2018-06-16 23:29:39 +03:00
4def9382f2 MXS-1881: Remove unneeded code
The code that checks for stop events is not needed as it is only used for
log messages. These aren't really useful to the end user so they can be
removed.

Moved the modification of the event size in case binlog checksums are
enabled outside of the handle_one_event function. This will make it
possible to move most of the processing done inside it into a reusable
class of its own.

Also fixed the memory leak for the event data.
2018-06-16 23:29:39 +03:00
c39fa4009e MXS-1881: Pass the RowEventConverer as a parameter
The RowEventConverter is now passed as a parameter to the Avro
instance. Wrapped the value in an std::auto_ptr to make the cleanup
automatic (when it is implemented).

Fixed a typo in the event handler member variable and removed the unused
stats member.
2018-06-16 23:29:39 +03:00
8f76cf5f5a MXS-1881: Remove file indexing
The file indexing provided very little benefit for the intended purpose of
the router. Removing it makes the whole system more robust and simplifies
the code by a large amount.
2018-06-16 23:29:38 +03:00
2005164222 Fix slave reconnection logic
Allowing calls to select_connect_backend_servers even when all slaves are
connected solves the debug assertion in select_connect_backend_servers
that happens when the execution of a queued query causes a new connection
to be created.
2018-06-15 16:16:53 +03:00
3ed6411741 Fix debug assert on reconnection with session commands
When a query was routed to a server that must first be connected to, the
expected response count was not updated for the executed session commands.
2018-06-15 16:16:53 +03:00
d2e7be2ca0 Acknowledge one-way commands immediately
The Backend class response state tracking was not updated when a one-way
command was executed. This caused the logic in handleError to break if a
master was executing a command that wouldn't create a response.
2018-06-15 16:16:53 +03:00
1ea94501e4 Prefer servers that are not busy executing session commands
Readwritesplit now prefers servers that are not busy executing session
commands. This should give the best responsiveness for reads.
2018-06-15 16:16:53 +03:00
70104f957e Fix readwritesplit hang when query is queued
Readwritesplit would hang when the query execution is postponed due to the
fact that the target server is executing a session command. The number of
expected responses was incremented when no response was expected.
2018-06-15 16:16:53 +03:00
34c33ce798 Remove unnecessary includes of hashtable.h 2018-06-15 09:55:44 +03:00
bbeaaa97b5 Merge branch '2.2' into develop 2018-06-13 23:18:52 +03:00
315738f279 Merge branch '2.1' into 2.2 2018-06-13 23:15:11 +03:00
53177c30de Fix runtime listener creation
The number of arguments to createListener was incremented but the maximum
count was not. Also fixed the parameter types for createListener and
alterServer.
2018-06-13 23:13:16 +03:00
143882ddce Merge branch '2.1' into 2.2 2018-06-13 21:04:49 +03:00
1ea7c65861 Fix runtime alteration of servers
The server runtime alteration was broken by commit
c850336199c3c19508a3d280fb7000291d66b80c when it increased the maximum
argument count of the `alter server` command to 14.
2018-06-13 14:29:28 +03:00
8094c67ac2 Merge branch '2.2' into develop 2018-06-13 00:25:56 +03:00
0627e8dc86 Resolve all symbols for routers at link time
The routers should not have undefied references. Exceptions to this are
the modules that use the maxscale_shutdown() function defined in
gateway.cc.
2018-06-12 19:40:34 +03:00
1946cb2876 Take a lambda function into use in readwritesplit
To test whether the compiler conforms to C++11, we can try to use a lambda
function.
2018-06-12 19:40:34 +03:00
cd136b11a7 MXS-1891: dealloc named prepare should route to all (#174)
* dealloc named prepare route to all

* add newline

* erase from ps manager too

* little refactor
2018-06-12 10:45:11 +03:00
96ed651466 Merge branch '2.2' into develop 2018-06-11 11:28:36 +03:00
1a24f0a956 Merge branch '2.1' into 2.2 2018-06-11 10:36:57 +03:00
1e1734f42e MXS-1910: Only require ssl_ca_cert for servers
Servers in MaxScale can encrypt the connections without client keys and
certificates. As keys and certificates are no longer required, the CA
certificate must always be initialized.
2018-06-09 00:15:48 +03:00
8c22131827 Remove redundant or unused avrorouter code
The code in avrorouter that returned the current transaction was not very
useful and it can be acquired via the REST API in a more convenient
format.

The number of created sessions is tracked on the service level so there is
no need to track it in the avrorouter.

Removed declarations for functions that do not exist and moved code around
to reduce the scope.
2018-06-08 12:18:14 +03:00
47d84cab81 Fix annotate_rows and binlog name processing
The four extra bytes added by the binlog checksums weren't ignored. This
caused the info messages to display the binary data of the checksum.
2018-06-08 12:18:14 +03:00
7c18696608 Abstract all row event processing
The code that handles the Avro files is now fully abstracted behind the
AvroConverter class that implements the RowEventHandler interface.

The code still has some avro specific behavior in a few places (parsing of
JSON files into TableCreate objects). This can be replaced, if needed, by
querying the master server for the CREATE TABLE statements.
2018-06-08 12:18:14 +03:00
d094e93209 Rename conflicting objects
The START_ENCRYPTION_EVENT is the name of an object that is exposed by the
Connector-C.
2018-06-08 12:18:13 +03:00
5268d032c5 Process file and data events separately
The various file operation related binlog events are now processed on the
upper level. This makes the actual data event processing simpler and
easier to comprehend.
2018-06-08 12:18:13 +03:00
9ec6293e3f Implement avrorouter in a separate file
The module implementation is now contained in one file. This makes it
easier to see which parts depend on each other.
2018-06-08 12:18:13 +03:00
b3cc65c516 Add missing binlog_checksum initialization
The variable was not initialized.
2018-06-08 12:18:13 +03:00
1786fb59bb Reorganize avrorouter event processing
Reorganized and cleaned up the binlog event processing code. Moved some of
the sanity checks into subfunctions and placed file related checks into a
separate section.
2018-06-08 12:18:12 +03:00
8fab725413 MXS-1881: Move RBR objects into a separate header
The RBR event handling related objects are now all in the rpl_events.hh
header. The intention is to combine all replication processing related
events used in the binlogrouter and avrorouter into this header to make
them reusable.

Also fixed the TableCreateEvent constructor to use an rvalue instead of
stealing an lvalue.
2018-06-08 12:18:12 +03:00
f574703f8e MXS-1881: Abstract the final event processing
The final part of the row event processing is now done by an
implementation of the EventConverter class. This makes the implementation
of different storage types easier as only the actual storage operation
needs to be implemented.
2018-06-08 12:18:12 +03:00
c56b2063aa MXS-1881: Rename event processing related objects
The main objects involved in replication event processing now use camel
case.
2018-06-08 12:18:11 +03:00
69855f9214 MXS-1881: Refactor AVRO_TABLE
Moved initialization into the constructor and removed unused member
variables.
2018-06-08 12:18:11 +03:00
f61c56228c MXS-1881: Refactor TABLE_MAP initialization
Changed TABLE_MAP to use STL containers and types. The initialization is
now done in the constructor. Removed unnecessary linkage between TABLE_MAP
and TABLE_CREATE.
2018-06-08 12:18:11 +03:00
e35d9dfc10 Move client functionality into AvroSession
Most of the code is unchanged. Some of the code that relied on modifiable
char pointers was modified to use std::string.
2018-06-08 12:18:11 +03:00
0d73530ff3 Merge branch '2.2' into develop 2018-06-08 11:30:55 +03:00
a7dd2127d6 Merge branch '2.1' into 2.2 2018-06-08 11:21:35 +03:00
c850336199 MXS-1907: Allow ssl_verify_peer_certificate when creating listener
When a listener is created at runtime or SSL is being enabled for an
already created listener, the ssl_verify_peer_certificate parameter can
now be defined.
2018-06-08 10:11:43 +03:00
6ba423cf94 Dump statements on unexpected internal state
If a server responds when no response was expected, dump stored
statements. This should help deduce root causes of problems relating to
unexpected responses.
2018-06-08 07:45:18 +03:00
880db1353a Merge branch '2.2' into develop 2018-06-07 14:39:16 +03:00
0dd7448586 MXS-1709 Fix memory leaks in unit tests 2018-06-06 22:59:52 +03:00
445eece95b MXS-1507: Fix replaying of empty transactions
If the starting of a transaction was interrupted by a server failure, the
query needs to be retried. This needs to be done as a transaction replay
to keep the routing logic consistent and simple.

When a non-autocommit transaction is interrupted, there will be no query
in progress and no replaying is needed. To handle this case, the replay
initialization logic needed to be altered to treat truly empty
transactions as a success case.
2018-06-04 19:26:36 +03:00
4a3216d483 Merge branch '2.2' into develop 2018-06-04 16:00:19 +03:00
9d0b1be08b Improve unexpected response error message
The message now logs the current command and the query if it is available.
2018-06-04 15:17:48 +03:00