2346 Commits

Author SHA1 Message Date
Markus Mäkelä
8094c67ac2
Merge branch '2.2' into develop 2018-06-13 00:25:56 +03:00
Markus Mäkelä
0627e8dc86
Resolve all symbols for routers at link time
The routers should not have undefied references. Exceptions to this are
the modules that use the maxscale_shutdown() function defined in
gateway.cc.
2018-06-12 19:40:34 +03:00
Markus Mäkelä
1946cb2876
Take a lambda function into use in readwritesplit
To test whether the compiler conforms to C++11, we can try to use a lambda
function.
2018-06-12 19:40:34 +03:00
dapeng huang
cd136b11a7 MXS-1891: dealloc named prepare should route to all (#174)
* dealloc named prepare route to all

* add newline

* erase from ps manager too

* little refactor
2018-06-12 10:45:11 +03:00
Markus Mäkelä
96ed651466
Merge branch '2.2' into develop 2018-06-11 11:28:36 +03:00
Markus Mäkelä
1a24f0a956
Merge branch '2.1' into 2.2 2018-06-11 10:36:57 +03:00
Markus Mäkelä
1e1734f42e
MXS-1910: Only require ssl_ca_cert for servers
Servers in MaxScale can encrypt the connections without client keys and
certificates. As keys and certificates are no longer required, the CA
certificate must always be initialized.
2018-06-09 00:15:48 +03:00
Markus Mäkelä
8c22131827
Remove redundant or unused avrorouter code
The code in avrorouter that returned the current transaction was not very
useful and it can be acquired via the REST API in a more convenient
format.

The number of created sessions is tracked on the service level so there is
no need to track it in the avrorouter.

Removed declarations for functions that do not exist and moved code around
to reduce the scope.
2018-06-08 12:18:14 +03:00
Markus Mäkelä
47d84cab81
Fix annotate_rows and binlog name processing
The four extra bytes added by the binlog checksums weren't ignored. This
caused the info messages to display the binary data of the checksum.
2018-06-08 12:18:14 +03:00
Markus Mäkelä
7c18696608
Abstract all row event processing
The code that handles the Avro files is now fully abstracted behind the
AvroConverter class that implements the RowEventHandler interface.

The code still has some avro specific behavior in a few places (parsing of
JSON files into TableCreate objects). This can be replaced, if needed, by
querying the master server for the CREATE TABLE statements.
2018-06-08 12:18:14 +03:00
Markus Mäkelä
d094e93209
Rename conflicting objects
The START_ENCRYPTION_EVENT is the name of an object that is exposed by the
Connector-C.
2018-06-08 12:18:13 +03:00
Markus Mäkelä
5268d032c5
Process file and data events separately
The various file operation related binlog events are now processed on the
upper level. This makes the actual data event processing simpler and
easier to comprehend.
2018-06-08 12:18:13 +03:00
Markus Mäkelä
9ec6293e3f
Implement avrorouter in a separate file
The module implementation is now contained in one file. This makes it
easier to see which parts depend on each other.
2018-06-08 12:18:13 +03:00
Markus Mäkelä
b3cc65c516
Add missing binlog_checksum initialization
The variable was not initialized.
2018-06-08 12:18:13 +03:00
Markus Mäkelä
1786fb59bb
Reorganize avrorouter event processing
Reorganized and cleaned up the binlog event processing code. Moved some of
the sanity checks into subfunctions and placed file related checks into a
separate section.
2018-06-08 12:18:12 +03:00
Markus Mäkelä
8fab725413
MXS-1881: Move RBR objects into a separate header
The RBR event handling related objects are now all in the rpl_events.hh
header. The intention is to combine all replication processing related
events used in the binlogrouter and avrorouter into this header to make
them reusable.

Also fixed the TableCreateEvent constructor to use an rvalue instead of
stealing an lvalue.
2018-06-08 12:18:12 +03:00
Markus Mäkelä
f574703f8e
MXS-1881: Abstract the final event processing
The final part of the row event processing is now done by an
implementation of the EventConverter class. This makes the implementation
of different storage types easier as only the actual storage operation
needs to be implemented.
2018-06-08 12:18:12 +03:00
Markus Mäkelä
c56b2063aa
MXS-1881: Rename event processing related objects
The main objects involved in replication event processing now use camel
case.
2018-06-08 12:18:11 +03:00
Markus Mäkelä
69855f9214
MXS-1881: Refactor AVRO_TABLE
Moved initialization into the constructor and removed unused member
variables.
2018-06-08 12:18:11 +03:00
Markus Mäkelä
f61c56228c
MXS-1881: Refactor TABLE_MAP initialization
Changed TABLE_MAP to use STL containers and types. The initialization is
now done in the constructor. Removed unnecessary linkage between TABLE_MAP
and TABLE_CREATE.
2018-06-08 12:18:11 +03:00
Markus Mäkelä
e35d9dfc10
Move client functionality into AvroSession
Most of the code is unchanged. Some of the code that relied on modifiable
char pointers was modified to use std::string.
2018-06-08 12:18:11 +03:00
Markus Mäkelä
0d73530ff3
Merge branch '2.2' into develop 2018-06-08 11:30:55 +03:00
Markus Mäkelä
a7dd2127d6
Merge branch '2.1' into 2.2 2018-06-08 11:21:35 +03:00
Markus Mäkelä
c850336199
MXS-1907: Allow ssl_verify_peer_certificate when creating listener
When a listener is created at runtime or SSL is being enabled for an
already created listener, the ssl_verify_peer_certificate parameter can
now be defined.
2018-06-08 10:11:43 +03:00
Markus Mäkelä
6ba423cf94
Dump statements on unexpected internal state
If a server responds when no response was expected, dump stored
statements. This should help deduce root causes of problems relating to
unexpected responses.
2018-06-08 07:45:18 +03:00
Marko
880db1353a Merge branch '2.2' into develop 2018-06-07 14:39:16 +03:00
Marko
0dd7448586 MXS-1709 Fix memory leaks in unit tests 2018-06-06 22:59:52 +03:00
Markus Mäkelä
445eece95b
MXS-1507: Fix replaying of empty transactions
If the starting of a transaction was interrupted by a server failure, the
query needs to be retried. This needs to be done as a transaction replay
to keep the routing logic consistent and simple.

When a non-autocommit transaction is interrupted, there will be no query
in progress and no replaying is needed. To handle this case, the replay
initialization logic needed to be altered to treat truly empty
transactions as a success case.
2018-06-04 19:26:36 +03:00
Markus Mäkelä
4a3216d483
Merge branch '2.2' into develop 2018-06-04 16:00:19 +03:00
Markus Mäkelä
9d0b1be08b
Improve unexpected response error message
The message now logs the current command and the query if it is available.
2018-06-04 15:17:48 +03:00
Markus Mäkelä
d22f6d7b1c
MXS-1743: Fix load balancing with router_options=master,slave
The code that selects the candidate backend always returned the root
master if the server bitmask contained the master bit. This should only be
done if the master bit is the only bit in the bitmask and when there are
other bits, the normal candidate selection code should be used.

Also added a query to the expanded test case to make sure the connection
actually works.
2018-06-04 15:17:48 +03:00
Markus Mäkelä
cddcc6d7d5
MXS-1896: Distinct LOAD DATA LOCAL INFILE from LOAD DATA INFILE
The two operations return different types of results and need to be
treated differently in order for them to be handled correctly in 2.2.

This fixes the unexpected internal state errors that happened in all 2.2
versions due to a wrong assumption made by readwritesplit. This fix is not
necessary for newer versions as the LOAD DATA LOCAL INFILE processing is
done with a simpler, and more robust, method.
2018-06-04 15:17:48 +03:00
Markus Mäkelä
c88aa11e11
Copy FDE events in avrorouter
Commit 67386980e327ad063b24cb55971cf44f4930e241 caused the actual events
to be ignored. This meant that the larger event size was assumed for all
events. In most cases this works but it is not the correct way to do it.
2018-06-04 15:17:48 +03:00
Johan Wikman
2ed631ab63 Merge branch '2.2' into develop 2018-06-01 13:46:03 +03:00
Johan Wikman
089be56103 MXS-1889 Only turn on master bit for slaves
The bitmask ensures that the master bit would be ignored in
cases where it is not relevant, but nicer if it is set only
when it is relevant.
2018-06-01 13:43:02 +03:00
Johan Wikman
d77a9a3040 MXS-1889 Handle master used as slave
Up until 2.1.12, if it in the configuration file said
'router_options=slave', the master was used if there were no
slaves at session creation time.

That broke in 2.1.13 as a side-effect of MXS-1516 that checks
at routing time whether the server initially selected as master
still is the master.

Now the required server status is stored separately for each
session, so that if the master was chosen, even though we have
'router_options=slave', we can turn on the SERVER_MASTER bit.

That allows us to handle the case correctly in connection_is_valid().
2018-06-01 11:00:13 +03:00
Esa Korhonen
c039821467 MXS-1883 Maintenance is now the only user-modifiable bit for a monitored server
The request to turn maintenance off/on is a separate flag, although the actual
status is still contained in the status bitfield.
2018-05-30 10:09:15 +03:00
Markus Mäkelä
2bbf1271c9
Fix large packet execution
The number of expected responses was not correctly tracked for large
packets.
2018-05-28 13:51:05 +03:00
Markus Mäkelä
be098538e5
MXS-1881: Read event header in a separate function
Reading the binlog event header in a separate function makes it easier to
comprehend. Cleaned up some unused variables and code that would is not
used.
2018-05-28 10:32:18 +03:00
Markus Mäkelä
5a3bbf0d15
Move binlog event processing into a separate file
This clarifies what parts of the router are specific to the binlogrouter
and what are common between the binlogrouter and avrorouter.

Ideally, the two modules would use the same infrastructure to handle the
processing of replication events. This is the first, albeit small, step
towards making the code in the binlogrouter the common infrastructure.
2018-05-28 10:32:18 +03:00
Markus Mäkelä
14a3b0052b
MXS-1881: Perform task management in main worker
The creation and cancellation of delayed tasks must be done on the worker
where the task is executed.
2018-05-28 10:32:18 +03:00
Markus Mäkelä
ec919c367b
Move Avro creation into Avro::create
The Avro instance is now created inside a static class method. This brings
it in line with how other modules create instances.

Converted all strings to std::string and updated their usage.
2018-05-28 10:32:17 +03:00
Markus Mäkelä
232d86e67a
Make active table map array dynamic
By storing the table maps in a std::unordered_map, the storage of mapped
tables is made dynamic. This also makes the management of mapped tables a
lot more robust.
2018-05-28 10:32:17 +03:00
Markus Mäkelä
f8ceb875a0
Refactor TABLE_CREATE allocation
Using std::string for names removes the need to handle memory
allocation. Moving the column attributes into a class of its own greatly
simplifies the creation of the TABLE_CREATE as well as modifications that
are done to it.
2018-05-28 10:32:17 +03:00
Markus Mäkelä
d5760f4301
Move AvroSession initialization inside the class
The AvroSession now has the AvroSession::create method for creation new
sessions. The destructor handles the freeing of all allocated resources.
2018-05-28 10:32:16 +03:00
Markus Mäkelä
fb53bbf4da
Use STL containers in avrorouter
The HASHTABLE can be replaced with std::unordered_map. This simplifies the
management by making the deletion of old objects automatic.

More cleanup and refactoring is needed to make the contained classes
cleaner.
2018-05-28 10:32:16 +03:00
Markus Mäkelä
a6eef98597
Initialize all AvroSession variables
This is done to get tests to pass before further refactoring is done.
2018-05-28 10:32:16 +03:00
Markus Mäkelä
6159f863ce
Remove unnecessary locking
The lock didn't really protect anything but since it was not explicitly
initialized, it caused a hang.
2018-05-28 10:32:16 +03:00
Markus Mäkelä
3fce61a615
Do avro conversion in main thread
The avrorouter no longer uses the housekeeper for the conversion
task. This prevents the deadlock which could occur when clients were
notified at the same time that the binlogrouter was adding a master
reconnection task.
2018-05-28 10:32:16 +03:00
Markus Mäkelä
16d2ff9564
Set read and execute permissions for all paths
If a path requires read or execute permission, it is granted to all. This
keeps path parameters in line with other directories that MaxScale
creates.
2018-05-28 10:32:16 +03:00