23 Commits

Author SHA1 Message Date
Johan Wikman
cc0299aee6 Update change date of 2.3 2018-06-25 10:07:52 +03:00
Markus Mäkelä
e74591cfe5
MXS-1881: Delegate event processing to the Rpl class
The actual processing of the replicated events is now delegated to the Rpl
class. This class only deals with the raw binary format log events which
allows it to be used for both binlogs stored on disk as well as binlogs
that have just been replicated.
2018-06-16 23:29:39 +03:00
Markus Mäkelä
c39fa4009e
MXS-1881: Pass the RowEventConverer as a parameter
The RowEventConverter is now passed as a parameter to the Avro
instance. Wrapped the value in an std::auto_ptr to make the cleanup
automatic (when it is implemented).

Fixed a typo in the event handler member variable and removed the unused
stats member.
2018-06-16 23:29:39 +03:00
Markus Mäkelä
8f76cf5f5a
MXS-1881: Remove file indexing
The file indexing provided very little benefit for the intended purpose of
the router. Removing it makes the whole system more robust and simplifies
the code by a large amount.
2018-06-16 23:29:38 +03:00
Markus Mäkelä
8c22131827
Remove redundant or unused avrorouter code
The code in avrorouter that returned the current transaction was not very
useful and it can be acquired via the REST API in a more convenient
format.

The number of created sessions is tracked on the service level so there is
no need to track it in the avrorouter.

Removed declarations for functions that do not exist and moved code around
to reduce the scope.
2018-06-08 12:18:14 +03:00
Markus Mäkelä
7c18696608
Abstract all row event processing
The code that handles the Avro files is now fully abstracted behind the
AvroConverter class that implements the RowEventHandler interface.

The code still has some avro specific behavior in a few places (parsing of
JSON files into TableCreate objects). This can be replaced, if needed, by
querying the master server for the CREATE TABLE statements.
2018-06-08 12:18:14 +03:00
Markus Mäkelä
5268d032c5
Process file and data events separately
The various file operation related binlog events are now processed on the
upper level. This makes the actual data event processing simpler and
easier to comprehend.
2018-06-08 12:18:13 +03:00
Markus Mäkelä
9ec6293e3f
Implement avrorouter in a separate file
The module implementation is now contained in one file. This makes it
easier to see which parts depend on each other.
2018-06-08 12:18:13 +03:00
Markus Mäkelä
8fab725413
MXS-1881: Move RBR objects into a separate header
The RBR event handling related objects are now all in the rpl_events.hh
header. The intention is to combine all replication processing related
events used in the binlogrouter and avrorouter into this header to make
them reusable.

Also fixed the TableCreateEvent constructor to use an rvalue instead of
stealing an lvalue.
2018-06-08 12:18:12 +03:00
Markus Mäkelä
c56b2063aa
MXS-1881: Rename event processing related objects
The main objects involved in replication event processing now use camel
case.
2018-06-08 12:18:11 +03:00
Markus Mäkelä
69855f9214
MXS-1881: Refactor AVRO_TABLE
Moved initialization into the constructor and removed unused member
variables.
2018-06-08 12:18:11 +03:00
Markus Mäkelä
f61c56228c
MXS-1881: Refactor TABLE_MAP initialization
Changed TABLE_MAP to use STL containers and types. The initialization is
now done in the constructor. Removed unnecessary linkage between TABLE_MAP
and TABLE_CREATE.
2018-06-08 12:18:11 +03:00
Markus Mäkelä
e35d9dfc10
Move client functionality into AvroSession
Most of the code is unchanged. Some of the code that relied on modifiable
char pointers was modified to use std::string.
2018-06-08 12:18:11 +03:00
Markus Mäkelä
ec919c367b
Move Avro creation into Avro::create
The Avro instance is now created inside a static class method. This brings
it in line with how other modules create instances.

Converted all strings to std::string and updated their usage.
2018-05-28 10:32:17 +03:00
Markus Mäkelä
232d86e67a
Make active table map array dynamic
By storing the table maps in a std::unordered_map, the storage of mapped
tables is made dynamic. This also makes the management of mapped tables a
lot more robust.
2018-05-28 10:32:17 +03:00
Markus Mäkelä
f8ceb875a0
Refactor TABLE_CREATE allocation
Using std::string for names removes the need to handle memory
allocation. Moving the column attributes into a class of its own greatly
simplifies the creation of the TABLE_CREATE as well as modifications that
are done to it.
2018-05-28 10:32:17 +03:00
Markus Mäkelä
d5760f4301
Move AvroSession initialization inside the class
The AvroSession now has the AvroSession::create method for creation new
sessions. The destructor handles the freeing of all allocated resources.
2018-05-28 10:32:16 +03:00
Markus Mäkelä
fb53bbf4da
Use STL containers in avrorouter
The HASHTABLE can be replaced with std::unordered_map. This simplifies the
management by making the deletion of old objects automatic.

More cleanup and refactoring is needed to make the contained classes
cleaner.
2018-05-28 10:32:16 +03:00
Markus Mäkelä
a6eef98597
Initialize all AvroSession variables
This is done to get tests to pass before further refactoring is done.
2018-05-28 10:32:16 +03:00
Markus Mäkelä
6159f863ce
Remove unnecessary locking
The lock didn't really protect anything but since it was not explicitly
initialized, it caused a hang.
2018-05-28 10:32:16 +03:00
Markus Mäkelä
3fce61a615
Do avro conversion in main thread
The avrorouter no longer uses the housekeeper for the conversion
task. This prevents the deadlock which could occur when clients were
notified at the same time that the binlogrouter was adding a master
reconnection task.
2018-05-28 10:32:16 +03:00
Markus Mäkelä
6898397d49
Clean up avrorouter.hh
Cleaned up the header and removed most of unused member variables.
2018-05-28 10:32:15 +03:00
Markus Mäkelä
3095b18b14
Convert avrorouter objects to C++
The instance and session objects are now C++ structs. The next pointers
for the sessions was removed as it is not the appropriate place to store
this information. This means that the client notification functionality is
broken in this commit.
2018-05-28 10:32:15 +03:00