2382 Commits

Author SHA1 Message Date
Johan Wikman
dc4a555c70 Merge branch '2.2' into develop 2018-06-28 10:34:41 +03:00
Markus Mäkelä
f97f422379
Don't use closed backends
Only use backends that are still in use. The COM_STMT_EXECUTE and
COM_STMT_FETCH relationship caused unused backends to be used.
2018-06-27 08:49:21 +03:00
Johan Wikman
8ea7d8898a MXS-1915 Remove id from mxs::Worker
The id has now been moved from mxs::Worker to mxs::RoutingWorker
and the implications are felt in many places.

The primary need for the id was to be able to access worker specfic
data, maintained outside of a routing worker, when given a worker
(the id is used to index into an array). Slightly related to that
was the need to be able to iterate over all workers. That obviously
implies some kind of collection.

That causes all sorts of issues if there is a need for being able
to create and destroy a worker at runtime. With the id removed from
mxs::Worker all those issues are gone, and its perfectly ok to create
and destory mxs::Workers as needed.

Further, while there is a need to broadcast a particular message to
all _routing_ workers, it hardly makes sense to broadcast a particular
message too _all_ workers. Consequently, only routing workers are kept
in a collection and all static member functions dealing with all
workers (e.g. broadcast) have now been moved to mxs::RoutingWorker.

Now, instead of passing the id around we instead deal directly
with the worker pointer. Later the data in all those external arrays
will be moved into mxs::[Worker|RoutingWorker] so that worker related
data is maintained in exactly one place.
2018-06-26 09:19:46 +03:00
Johan Wikman
86b5238aaf MXS-1915 Replace worker id with worker pointer
To get rid of the need that a Worker must have an id, we store
in the MXS_POLL_DATA structure a pointer to the owning worker
instead of the id of the owning worker. This also allows some
further cleanup as the need for switching back and forth between
the id and the worker disappears.

The id will be moved from Worker to RoutingWorker as there
currently is a fair amount of code that assumes that the id of
routing workers start from 0.
2018-06-26 09:19:46 +03:00
Johan Wikman
cc0299aee6 Update change date of 2.3 2018-06-25 10:07:52 +03:00
Markus Mäkelä
b8e0c31872
Merge branch '2.2' into develop 2018-06-25 09:37:18 +03:00
Markus Mäkelä
6183fab79b
MXS-1938: Log query for inconsistent replies
If a session command produces a different result on the slave than it did
on the master, a warning is logged. This warning now also logs the query
that was being executed to make investigation of the problem easier.
2018-06-25 08:44:13 +03:00
Marko
f308dd281a MXS-1849 Combine table and database mapping
Previously schemarouter only mapped databases to the servers
they were resided on. Now all the tables are also mapped to allow the
router to route queries to the right server based on the tables used in
that query.
2018-06-24 22:26:36 +03:00
Marko
b38cac4939 MXS-1849 Add functions for mapping tables to servers 2018-06-24 22:26:36 +03:00
Markus Mäkelä
b96228f95c
Merge branch '2.2' into develop 2018-06-22 10:44:39 +03:00
Markus Mäkelä
e561c3995c
Use correct write in Backend::execute_session_command
Backend::execute_session_command would use the overridden write method
instead of the Backend::write method that it intended to use. This caused
session commands that did not expect a response to be in a state that
expected a result.

Also fixed RWBackend::write pass the response_type value to
Backend::write.
2018-06-22 10:37:11 +03:00
Markus Mäkelä
9e4b86ff0a
MXS-1935: Accent NULL from qc_get_preparable_stmt in readwritesplit
If the prepared statement cannot be extracted from the query, treat it as
a query with an unknown type.
2018-06-21 10:04:34 +03:00
Markus Mäkelä
6278f27ab6
Merge branch '2.2' into develop 2018-06-20 10:26:29 +03:00
Markus Mäkelä
980caa5be5
MXS-1926: Ignore server shutdown errors
If the server sends a server shutdown error, it is safe for readwritesplit
to ignore it. When the TCP connection is closed, the router error handling
will discard the connection, optionally replacing it.
2018-06-20 00:34:16 +03:00
Markus Mäkelä
1f166482b2
Fix slave reconnection regression
The state of the backend needs to be checked before any pending session
commands are executed on it.

Added debug assertions to catch invalid use of the status functions of
closed backends.
2018-06-18 14:25:05 +03:00
Markus Mäkelä
bd6155b950
MXS-1914: Resolve symbols at link time
Enabled link time symbol resolution to administrative modules.
2018-06-18 12:58:51 +03:00
Markus Mäkelä
95935991ab
MXS-1881: Move Rpl related code out of Avro files
Organized the Rpl and Avro code in a way that they aren't mixed.
2018-06-16 23:29:40 +03:00
Markus Mäkelä
a3a4bb3da6
MXS-1881: Add TableCreateEvent JSON serialization
Serializing the object allows replicated events to be processed even if
the CREATE TABLE statement is not in the binary logs.
2018-06-16 23:29:40 +03:00
Markus Mäkelä
3417842e75
MXS-1881: Add table creation method to RowEventHandler
The new method is called for each new CREATE TABLE statement that is
processed as well as all ALTER TABLE statemets that modify the table
structure.

Right now the entry point not in use but it opens up the possibility of
persisting the CREATE TABLE statements at creation time. Currently the
tables are only persisted when the first actual event for the table is
received.
2018-06-16 23:29:39 +03:00
Markus Mäkelä
c27529c9a4
MXS-1881: Expand GTID helper class
Added string conversion methods to the gtid_pos_t class that can be used
to store and load a GTID value.

Also added the missing rpl.cc file that previously only had the Rpl class
constructor in it.
2018-06-16 23:29:39 +03:00
Markus Mäkelä
ac6370afcf
Clean up common binlogrouter headers
Cleaned up blr_constants.h and moved the REP_HEADER construction helper
into binlog_common.h.
2018-06-16 23:29:39 +03:00
Markus Mäkelä
e74591cfe5
MXS-1881: Delegate event processing to the Rpl class
The actual processing of the replicated events is now delegated to the Rpl
class. This class only deals with the raw binary format log events which
allows it to be used for both binlogs stored on disk as well as binlogs
that have just been replicated.
2018-06-16 23:29:39 +03:00
Markus Mäkelä
4def9382f2
MXS-1881: Remove unneeded code
The code that checks for stop events is not needed as it is only used for
log messages. These aren't really useful to the end user so they can be
removed.

Moved the modification of the event size in case binlog checksums are
enabled outside of the handle_one_event function. This will make it
possible to move most of the processing done inside it into a reusable
class of its own.

Also fixed the memory leak for the event data.
2018-06-16 23:29:39 +03:00
Markus Mäkelä
c39fa4009e
MXS-1881: Pass the RowEventConverer as a parameter
The RowEventConverter is now passed as a parameter to the Avro
instance. Wrapped the value in an std::auto_ptr to make the cleanup
automatic (when it is implemented).

Fixed a typo in the event handler member variable and removed the unused
stats member.
2018-06-16 23:29:39 +03:00
Markus Mäkelä
8f76cf5f5a
MXS-1881: Remove file indexing
The file indexing provided very little benefit for the intended purpose of
the router. Removing it makes the whole system more robust and simplifies
the code by a large amount.
2018-06-16 23:29:38 +03:00
Markus Mäkelä
2005164222
Fix slave reconnection logic
Allowing calls to select_connect_backend_servers even when all slaves are
connected solves the debug assertion in select_connect_backend_servers
that happens when the execution of a queued query causes a new connection
to be created.
2018-06-15 16:16:53 +03:00
Markus Mäkelä
3ed6411741
Fix debug assert on reconnection with session commands
When a query was routed to a server that must first be connected to, the
expected response count was not updated for the executed session commands.
2018-06-15 16:16:53 +03:00
Markus Mäkelä
d2e7be2ca0
Acknowledge one-way commands immediately
The Backend class response state tracking was not updated when a one-way
command was executed. This caused the logic in handleError to break if a
master was executing a command that wouldn't create a response.
2018-06-15 16:16:53 +03:00
Markus Mäkelä
1ea94501e4
Prefer servers that are not busy executing session commands
Readwritesplit now prefers servers that are not busy executing session
commands. This should give the best responsiveness for reads.
2018-06-15 16:16:53 +03:00
Markus Mäkelä
70104f957e
Fix readwritesplit hang when query is queued
Readwritesplit would hang when the query execution is postponed due to the
fact that the target server is executing a session command. The number of
expected responses was incremented when no response was expected.
2018-06-15 16:16:53 +03:00
Johan Wikman
34c33ce798 Remove unnecessary includes of hashtable.h 2018-06-15 09:55:44 +03:00
Markus Mäkelä
bbeaaa97b5
Merge branch '2.2' into develop 2018-06-13 23:18:52 +03:00
Markus Mäkelä
315738f279
Merge branch '2.1' into 2.2 2018-06-13 23:15:11 +03:00
Markus Mäkelä
53177c30de
Fix runtime listener creation
The number of arguments to createListener was incremented but the maximum
count was not. Also fixed the parameter types for createListener and
alterServer.
2018-06-13 23:13:16 +03:00
Markus Mäkelä
143882ddce
Merge branch '2.1' into 2.2 2018-06-13 21:04:49 +03:00
Markus Mäkelä
1ea7c65861
Fix runtime alteration of servers
The server runtime alteration was broken by commit
c850336199c3c19508a3d280fb7000291d66b80c when it increased the maximum
argument count of the `alter server` command to 14.
2018-06-13 14:29:28 +03:00
Markus Mäkelä
8094c67ac2
Merge branch '2.2' into develop 2018-06-13 00:25:56 +03:00
Markus Mäkelä
0627e8dc86
Resolve all symbols for routers at link time
The routers should not have undefied references. Exceptions to this are
the modules that use the maxscale_shutdown() function defined in
gateway.cc.
2018-06-12 19:40:34 +03:00
Markus Mäkelä
1946cb2876
Take a lambda function into use in readwritesplit
To test whether the compiler conforms to C++11, we can try to use a lambda
function.
2018-06-12 19:40:34 +03:00
dapeng huang
cd136b11a7 MXS-1891: dealloc named prepare should route to all (#174)
* dealloc named prepare route to all

* add newline

* erase from ps manager too

* little refactor
2018-06-12 10:45:11 +03:00
Markus Mäkelä
96ed651466
Merge branch '2.2' into develop 2018-06-11 11:28:36 +03:00
Markus Mäkelä
1a24f0a956
Merge branch '2.1' into 2.2 2018-06-11 10:36:57 +03:00
Markus Mäkelä
1e1734f42e
MXS-1910: Only require ssl_ca_cert for servers
Servers in MaxScale can encrypt the connections without client keys and
certificates. As keys and certificates are no longer required, the CA
certificate must always be initialized.
2018-06-09 00:15:48 +03:00
Markus Mäkelä
8c22131827
Remove redundant or unused avrorouter code
The code in avrorouter that returned the current transaction was not very
useful and it can be acquired via the REST API in a more convenient
format.

The number of created sessions is tracked on the service level so there is
no need to track it in the avrorouter.

Removed declarations for functions that do not exist and moved code around
to reduce the scope.
2018-06-08 12:18:14 +03:00
Markus Mäkelä
47d84cab81
Fix annotate_rows and binlog name processing
The four extra bytes added by the binlog checksums weren't ignored. This
caused the info messages to display the binary data of the checksum.
2018-06-08 12:18:14 +03:00
Markus Mäkelä
7c18696608
Abstract all row event processing
The code that handles the Avro files is now fully abstracted behind the
AvroConverter class that implements the RowEventHandler interface.

The code still has some avro specific behavior in a few places (parsing of
JSON files into TableCreate objects). This can be replaced, if needed, by
querying the master server for the CREATE TABLE statements.
2018-06-08 12:18:14 +03:00
Markus Mäkelä
d094e93209
Rename conflicting objects
The START_ENCRYPTION_EVENT is the name of an object that is exposed by the
Connector-C.
2018-06-08 12:18:13 +03:00
Markus Mäkelä
5268d032c5
Process file and data events separately
The various file operation related binlog events are now processed on the
upper level. This makes the actual data event processing simpler and
easier to comprehend.
2018-06-08 12:18:13 +03:00
Markus Mäkelä
9ec6293e3f
Implement avrorouter in a separate file
The module implementation is now contained in one file. This makes it
easier to see which parts depend on each other.
2018-06-08 12:18:13 +03:00
Markus Mäkelä
b3cc65c516
Add missing binlog_checksum initialization
The variable was not initialized.
2018-06-08 12:18:13 +03:00