203 Commits

Author SHA1 Message Date
Markus Mäkelä
ab4f870927
MXS-2067: Replace most SPINLOCKs
Replaced SPINLOCK with std::mutex where possible, leaving out the more
complex cases. The big offenders remaining are the binlogrouter and the
gateway.cc OpenSSL locks.
2018-09-28 12:18:23 +03:00
Markus Mäkelä
4e41978069
Merge branch '2.2' into develop 2018-09-14 10:41:43 +03:00
Markus Mäkelä
bee7cc2002
MXS-2046: Fix additional memory leaks
A set of memory leaks were revealed by Valgrind.
2018-09-13 14:27:21 +03:00
Markus Mäkelä
d11c78ad80
Format all sources with Uncrustify
Formatted all sources and manually tuned some files to make the code look
neater.
2018-09-10 13:22:49 +03:00
Niclas Antti
c447e5cf15 Uncrustify maxscale
See script directory for method. The script to run in the top level
MaxScale directory is called maxscale-uncrustify.sh, which uses
another script, list-src, from the same directory (so you need to set
your PATH). The uncrustify version was 0.66.
2018-09-09 22:26:19 +03:00
Markus Mäkelä
0e5d827f7a
Merge branch '2.2' into develop 2018-09-08 18:58:57 +03:00
Markus Mäkelä
d7dd636c9a
Add table filtering to avrorouter
The filtering is implemented with PCRE2 regular expressions and as such is
not the most user-friendly interface.
2018-09-04 10:50:31 +03:00
Markus Mäkelä
8b653133a7
Add shutdown detection
The maxscale_is_shutting_down function is used to detect when MaxScale
should stop. This fixes a race condition in the code where the workers has
not yet been initialized but a termination signal has been received. It
also replaces the misuse of the service_should_stop variable with a proper
function.
2018-08-29 11:06:11 +03:00
Markus Mäkelä
9a6f1b2044
Merge branch '2.2' into develop 2018-08-28 16:06:23 +03:00
Markus Mäkelä
4480995089
MXS-1996: Remove misleading errors
The errors are expected if the file is empty.
2018-08-22 14:23:25 +03:00
Johan Wikman
ab9a9f92cb MXS-2020 Remove maxscale/debug.h
- Removed from all files.
- maxbase/assert.h included where necessary.
2018-08-22 11:35:35 +03:00
Johan Wikman
3f53eddbde MXS-2020 Replace ss[_info]_dassert with mxb_assert[_message] 2018-08-22 11:34:59 +03:00
Johan Wikman
b1e405442f MXS-2020 Replace ss_debug with MXB_AT_DEBUG 2018-08-22 11:34:06 +03:00
Niclas Antti
24ab3c099c Move top of the file "#pragma once" to after the following comment (swap them). If the comment is a BPL update it to the latest one 2018-08-21 13:13:15 +03:00
Johan Wikman
88f1795412 MXS-2008 Move Worker and MessageQueue to maxbase 2018-08-21 10:02:39 +03:00
Markus Mäkelä
930be8d77a
Fix CMake problems in system test
The paths were wrong in the scripts used to run tests. The same problem
was in the CMake files that used CMAKE_SOURCE_DIR instead of
CMAKE_CURRENT_SOURCE_DIR.

Added missing check for BUILD_SYSTEM_TESTS in avrorouter for the
workaround to building without all dependencies present.
2018-08-20 13:56:58 +03:00
Johan Wikman
8f257a51fe MXS-2008 Remove unused headers from worker.hh
Add corresponding headers to files that depended on those headers.
2018-08-20 11:15:14 +03:00
Markus Mäkelä
57334153fe
MXS-1880: Fix crash after restart
The offsets in the events were wrong if MaxScale was restarted in the
middle of a binlog. This was caused by the fact that the FDE event
contains information on binlog checksums which is required for a correct
interpretation of the binlog. The FDE event was only read when the binlog
was first opened when it needs to be read every time the binlog is opened.
2018-08-20 08:06:59 +03:00
Markus Mäkelä
d6ed10975e
MXS-2015: Add missing newlines
The newlines were missing from a few of the avrorouter responses.
2018-08-20 08:06:59 +03:00
Markus Mäkelä
1c733bf450
MXS-2015: Always check for error responses
As a JSON object can never start with the ERR prefix, it is safe to check
it for all requests. Also fixed the missing newline in the avrorouter
error message.
2018-08-20 08:06:59 +03:00
Johan Wikman
cf0aeed516 MXS-2014 Rename log_manager.h to log.h
There's nothing resembling a manager anymore.
2018-08-17 10:59:37 +03:00
Johan Wikman
f3f802cabe MXS-2008 Move maxscale/atomic.h to maxbase/atomic.h 2018-08-15 08:44:39 +03:00
Markus Mäkelä
2852530893
Use standard RNG
The custom random number generator can now be replaced with a C++11
RNG. This greatly improves the reliability and trustworthiness of it.

In addition to this, the conversion of the RNG to a thread-local object
removes the race condition that was present in the previous
implementation. It also theoretically improves performance by a tiny bit.
2018-08-13 10:37:48 +03:00
Johan Wikman
e9758ebaf1 MXS-2002 Rename Worker::post() to Worker::execute()
The main point is that tasks/functions are executed, not that
they are posted.
2018-08-13 08:30:05 +03:00
Johan Wikman
6335d3776c MXS-2002 Replace auto_ptr with unique_ptr
Given 'Derived : public Base', a unique_ptr<Derived> converts
implictly to a unique_ptr<Base>.
2018-08-13 08:30:05 +03:00
Johan Wikman
9cfd451a1d MXS-2002 Make Worker excecution mode explicit
This is the first step in some cleanup of the Worker interface.
The execution mode must now be explicitly specified, but that is
just a temporary step. Further down the road, _posting_ will
*always* mean via the message loop while _executing_ will optionally
and by default mean direct execution if the calling thread is that
of the worker.
2018-08-13 08:30:05 +03:00
Markus Mäkelä
a252b45f18
Don't use auto&& in non-template code
Use explicit types instead.
2018-08-06 21:20:29 +03:00
Markus Mäkelä
359f61c73b
Improve shutdown signal safeness
The signal handler no longer acquires the service list lock which removes
a number of deadlock possibilities from the shutdown process. Instead, a
global shutdown flag is set that serves the same purpose as the individual
service shutdown flags did.
2018-08-01 16:48:05 +03:00
Markus Mäkelä
cbb8c68770
Remove router_options
Relaced router_options with configuration parameters in the createInstance
router entry point. The same needs to be done for the filter API as barely
any filters use the feature.

Some routers (binlogrouter) still support router_options but using it is
deprecated. This had to be done as their use wasn't deprecated in 2.2.
2018-07-11 14:08:56 +03:00
Markus Mäkelä
77a1417479
Replace TR1 headers with standard headers
Now that the C++11 standard is the default one, we can remove the TR1
headers and classes.
2018-07-11 14:08:46 +03:00
Johan Wikman
cc0299aee6 Update change date of 2.3 2018-06-25 10:07:52 +03:00
Markus Mäkelä
95935991ab
MXS-1881: Move Rpl related code out of Avro files
Organized the Rpl and Avro code in a way that they aren't mixed.
2018-06-16 23:29:40 +03:00
Markus Mäkelä
a3a4bb3da6
MXS-1881: Add TableCreateEvent JSON serialization
Serializing the object allows replicated events to be processed even if
the CREATE TABLE statement is not in the binary logs.
2018-06-16 23:29:40 +03:00
Markus Mäkelä
3417842e75
MXS-1881: Add table creation method to RowEventHandler
The new method is called for each new CREATE TABLE statement that is
processed as well as all ALTER TABLE statemets that modify the table
structure.

Right now the entry point not in use but it opens up the possibility of
persisting the CREATE TABLE statements at creation time. Currently the
tables are only persisted when the first actual event for the table is
received.
2018-06-16 23:29:39 +03:00
Markus Mäkelä
c27529c9a4
MXS-1881: Expand GTID helper class
Added string conversion methods to the gtid_pos_t class that can be used
to store and load a GTID value.

Also added the missing rpl.cc file that previously only had the Rpl class
constructor in it.
2018-06-16 23:29:39 +03:00
Markus Mäkelä
ac6370afcf
Clean up common binlogrouter headers
Cleaned up blr_constants.h and moved the REP_HEADER construction helper
into binlog_common.h.
2018-06-16 23:29:39 +03:00
Markus Mäkelä
e74591cfe5
MXS-1881: Delegate event processing to the Rpl class
The actual processing of the replicated events is now delegated to the Rpl
class. This class only deals with the raw binary format log events which
allows it to be used for both binlogs stored on disk as well as binlogs
that have just been replicated.
2018-06-16 23:29:39 +03:00
Markus Mäkelä
4def9382f2
MXS-1881: Remove unneeded code
The code that checks for stop events is not needed as it is only used for
log messages. These aren't really useful to the end user so they can be
removed.

Moved the modification of the event size in case binlog checksums are
enabled outside of the handle_one_event function. This will make it
possible to move most of the processing done inside it into a reusable
class of its own.

Also fixed the memory leak for the event data.
2018-06-16 23:29:39 +03:00
Markus Mäkelä
c39fa4009e
MXS-1881: Pass the RowEventConverer as a parameter
The RowEventConverter is now passed as a parameter to the Avro
instance. Wrapped the value in an std::auto_ptr to make the cleanup
automatic (when it is implemented).

Fixed a typo in the event handler member variable and removed the unused
stats member.
2018-06-16 23:29:39 +03:00
Markus Mäkelä
8f76cf5f5a
MXS-1881: Remove file indexing
The file indexing provided very little benefit for the intended purpose of
the router. Removing it makes the whole system more robust and simplifies
the code by a large amount.
2018-06-16 23:29:38 +03:00
Markus Mäkelä
0627e8dc86
Resolve all symbols for routers at link time
The routers should not have undefied references. Exceptions to this are
the modules that use the maxscale_shutdown() function defined in
gateway.cc.
2018-06-12 19:40:34 +03:00
Markus Mäkelä
8c22131827
Remove redundant or unused avrorouter code
The code in avrorouter that returned the current transaction was not very
useful and it can be acquired via the REST API in a more convenient
format.

The number of created sessions is tracked on the service level so there is
no need to track it in the avrorouter.

Removed declarations for functions that do not exist and moved code around
to reduce the scope.
2018-06-08 12:18:14 +03:00
Markus Mäkelä
47d84cab81
Fix annotate_rows and binlog name processing
The four extra bytes added by the binlog checksums weren't ignored. This
caused the info messages to display the binary data of the checksum.
2018-06-08 12:18:14 +03:00
Markus Mäkelä
7c18696608
Abstract all row event processing
The code that handles the Avro files is now fully abstracted behind the
AvroConverter class that implements the RowEventHandler interface.

The code still has some avro specific behavior in a few places (parsing of
JSON files into TableCreate objects). This can be replaced, if needed, by
querying the master server for the CREATE TABLE statements.
2018-06-08 12:18:14 +03:00
Markus Mäkelä
5268d032c5
Process file and data events separately
The various file operation related binlog events are now processed on the
upper level. This makes the actual data event processing simpler and
easier to comprehend.
2018-06-08 12:18:13 +03:00
Markus Mäkelä
9ec6293e3f
Implement avrorouter in a separate file
The module implementation is now contained in one file. This makes it
easier to see which parts depend on each other.
2018-06-08 12:18:13 +03:00
Markus Mäkelä
b3cc65c516
Add missing binlog_checksum initialization
The variable was not initialized.
2018-06-08 12:18:13 +03:00
Markus Mäkelä
1786fb59bb
Reorganize avrorouter event processing
Reorganized and cleaned up the binlog event processing code. Moved some of
the sanity checks into subfunctions and placed file related checks into a
separate section.
2018-06-08 12:18:12 +03:00
Markus Mäkelä
8fab725413
MXS-1881: Move RBR objects into a separate header
The RBR event handling related objects are now all in the rpl_events.hh
header. The intention is to combine all replication processing related
events used in the binlogrouter and avrorouter into this header to make
them reusable.

Also fixed the TableCreateEvent constructor to use an rvalue instead of
stealing an lvalue.
2018-06-08 12:18:12 +03:00
Markus Mäkelä
f574703f8e
MXS-1881: Abstract the final event processing
The final part of the row event processing is now done by an
implementation of the EventConverter class. This makes the implementation
of different storage types easier as only the actual storage operation
needs to be implemented.
2018-06-08 12:18:12 +03:00