1200 Commits

Author SHA1 Message Date
Markus Makela
dd9f7fbbb7 Fixed rwsplit length calculations for multi-statements
The calculations used the amount of bytes read instead of amount of bytes
available and also read one byte too much.
2016-03-04 09:56:18 +02:00
Markus Makela
d1837e0e4a Added more error logging and fixed parsing of packets in readwritesplit
Readwritesplit tried to parse COM_PING packets which only have one byte of
payload. This would cause the queries to fail to parsed.
2016-03-04 07:55:03 +02:00
Markus Makela
d3da98b58f Merge branch 'MXS-544' into develop-MXS-544-merge 2016-03-03 21:39:39 +02:00
Markus Makela
76f06572ed Fix to multi-statement processing
Renamed is_mysql_comment_start to is_mysql_statement_end because it checks
whether a statement truly ends instead of just checking comment block starts.

The calculations for buffer length in readwritesplit now use the payload size
instead of the buffer size.
2016-03-03 10:44:11 +02:00
Markus Makela
8b6595aa68 Improvements and fixes to strchr_esc_mysql
The C style comments were not ignored and the -- style comments
did not check for the trailing whitespace and made return values char*.

The creation of a stored procedure would prevent sessions from using
any of the slave servers because readwritesplit would interpret
the creation statement as a multi-statement query.

Parts of modutil and readwritesplit now compare pointers to pointers instead of
converting pointers to integers.
2016-03-02 21:34:22 +02:00
Markus Makela
2589630b8e Unknown SQL commands are treated as writes
This will prevent any possible damage to a slave by new or unexpected commands.
2016-03-02 21:27:34 +02:00
Markus Makela
9bfef5c7c1 Improved multi-statement detection in readwritesplit
Now comment blocks and identifiers quoted with backticks are taken into
consideration when multi-statements are processed.
2016-03-02 21:27:34 +02:00
Johan Wikman
1d4779215d Ensure symbols are resolved at link-time.
With the linker flags "-Wl,-z,defs", all symbols used by a library
are resolved at link-time. Otherwise they will be resolved at runtime.
The use of these flags ensures that missing symbols are found as
early as possible.

Case in point, the binlog router test-cases failed, because the loading
of the binlog router failed due to missing symbols my_uuid_init and
my_uuid. The reason was that when maxscale no longer was linked with
the embedded library, those symbols were not available.

Now we know that the loading of the binlog router will not fail due
to missing symbols.
2016-03-01 21:29:13 +02:00
Johan Wikman
9b5550c053 Link binlog router with embedded lib.
Binlog router uses my_uuid_init and my_uuid, which are non-public
functions available in the embedded library. Consequently, blr
must currently be linked with the embedded library.

A custom implementation of these functions should be provided, in
order to break that dependency.
2016-03-01 21:25:47 +02:00
Johan Wikman
9a0534d0f6 Use maxscale_uptime().
MaxScaleUptime() has been renamed to maxscale_uptime(). Take
prototype from include file.
2016-03-01 21:23:43 +02:00
Johan Wikman
cb3213af63 server/include/poll.h renamed to server/include/maxscale/poll.h
Due to an include conflict between /usr/include/poll.h and
maxscale/server/include/poll.h the latter was moved to
maxscale/server/include/maxscale/poll.h.

Once 1.4 is out, all maxscale header files will be moved to that
same sub-directory. That will prevent include conflicts and also
make it easy to see which include files belong to maxscale and
which do not.
2016-03-01 12:56:39 +02:00
Markus Makela
bf1a789376 Added configurable multistatement behavior
Whether all queries should be routed to the master after a multistatement
query is executed can now be controlled with the `strict_multi_stmt` option.

When the option is disabled queries executed after a multistatement query will
be routed normally.
2016-03-01 10:45:08 +02:00
Markus Makela
ac007fa8f5 MXS-497: Multi-statement queries are always routed to the master
This will prevent the routing of queries that modify data to the slaves.

In the future a more intricate solution can done where all the statements
are parsed and the destination is resolved based on the actual contents.
2016-03-01 09:32:55 +02:00
Markus Makela
b88d66357e Shard router is no longer built by default
The shardrouter module is not finished and should not be built for releases.
2016-02-29 17:18:20 +02:00
Markus Makela
bf24f18805 Merge branch 'develop' into MXS-544 2016-02-29 10:40:33 +02:00
counterpoint
395cbdc103 Simplify handling of "client data" by having a pointer to it only in DCB and not in session structure. Change name of session->client to session->client_dcb for greater clarity. Temporary free of client data in DCB, to be moved to authenticator when it becomes a module. Fix incorrect name of listener_alloc. 2016-02-23 11:13:21 +00:00
Markus Makela
28bd7c2202 Merge branch '1.2.1-binlog_router_trx' into develop 2016-02-19 18:51:59 +02:00
Markus Makela
59f5880898 Added missing OK byte to payload size calculation
The OK byte was not taken into notice when the total size of all the payloads
in all the packets was calculated.
2016-02-17 08:30:50 +02:00
Markus Makela
63ce9fe6bc Fixed formatting and added more error checks
Added log messages when ftruncate fails and cleaned up formatting.
2016-02-16 13:06:25 +02:00
Markus Makela
cd2af6ffef Cleaned up the code based on the code review
Added missing error condition checks and cleaned up code.
2016-02-16 13:06:25 +02:00
Markus Makela
a55f017c75 Fixed packets with a length of one being ignored
The packets were not written into the binlogs which caused binlog corruption.
2016-02-16 13:06:25 +02:00
Markus Makela
9306b9d68c Added detection of checksums split across two packets
The checksums should now be processed properly event if the event is in more than
one packet.
2016-02-16 13:06:25 +02:00
Markus Makela
36896afcbd Fixed missing NULL check when reading records
If the binlog record was not found, a NULL pointer is returned. There was no
check for the return value and it assumed that it was always non-NULL.
2016-02-16 13:06:25 +02:00
Markus Makela
74c8b5e296 Fixed events larger than 2^24 failing without transaction safety
If transaction safety was disabled and a large event sent in multiple SQL
packets was received, the distribution of that event to the slaves would fail.
2016-02-16 13:06:25 +02:00
Markus Makela
2b7e2d3043 Added checksum calculations for events larger than 2^24 bytes
The checksums are now properly calculated for large events that span multiple
SQL packets.
2016-02-16 13:06:25 +02:00
Markus Makela
3609f97ba0 Fixed events which are exactly 0x00fffffe bytes long failing to replicate
The empty packet sent after a large event which fits into exactly one packet
was written to disk and the writing of no bytes caused it to be treated as
an error.
2016-02-16 13:06:25 +02:00
MassimilianoPinto
476691eda1 Removed log message for event larger than 16MB
The log message used during tests is now removed
2016-02-16 13:06:25 +02:00
Markus Makela
3e04a36ac3 Added support for distribution of packets larger than 2^24 bytes
Moved the the sending of the replication events to a different function
and added support for events that span multiple MySQL packets.
2016-02-16 13:06:25 +02:00
Markus Makela
12ee568978 Fixed last_written being set to the size of the event
The addition used =+ instead of += which caused it to be an assignment.
2016-02-16 13:06:25 +02:00
Markus Makela
d2b4713d27 Added missing condition to else clause
This fixes all packets being considered as large packets.
2016-02-16 13:06:25 +02:00
Markus Makela
ae33df3cbc Large events are now processed in chuncks
The router->last_written is used to store the position where the last event was
written. The replication header is also stored in a separate structure in
the router which is used later when the last packet of a multi-packet event
arrives.
2016-02-16 13:06:24 +02:00
MassimilianoPinto
d3e1d4dd2f First fix for 16MB handling in the master part
First fix for 16MB handling in the master part.

Distribute events to up to date slave is not included yet
2016-02-16 13:06:24 +02:00
MassimilianoPinto
ab1fb90d86 Fix for 16MB transmission in the slave part
Fix for 16MB transmission in the slave part
2016-02-16 13:06:24 +02:00
Markus Makela
26cbcb7365 MXS-511: Made log messages about master status coherent
The checking of the master status and the possible error logging were done
in two different steps. This led to confusing error messages when the state
of the server changed between the check and the logging of the error message.
2016-02-12 21:18:57 +02:00
MassimilianoPinto
e777828bd0 When creating heartbeat packet too many bytes were copied.
The memory area ‘ptr’ points to contains now the right data
2016-02-11 17:12:10 +01:00
MassimilianoPinto
4eccc1acb6 When creating heartbeat packet too many bytes were copied.
The memory area ‘ptr’ points to contains now the right data
2016-02-11 17:06:13 +01:00
Markus Makela
cfefc046e9 Merge branch 'release-1.3.0' into develop 2016-02-11 10:23:00 +02:00
Markus Makela
9acdd4e70c Fixed tests failing to build
The tests were not linking against the maxscale-common library.
2016-02-09 10:59:11 +02:00
Johan Wikman
4a4f22c9a7 Make query_classifier part of core.
The query classifier interface is now built as an integral
part of MaxScale core.
2016-02-08 11:44:07 +02:00
Markus Makela
379cb6c781 MXS-564: Slave server states were not properly cleared after session commands
When a command that changes the session state is executed on all servers and
a slave server goes down, an error message was sent to the client even though
one wasn't expected. These appeared as odd connection errors on the client side.
2016-02-05 13:54:24 +02:00
Markus Makela
9432967da5 Don't assert the master server state when routing query to slave
When a query is being routed to the slave, it is not necessary to assert that
the master server's state is still master. It is possible that the master server
changes states while the query is being routed but that doesn't affect the query
being currently routed.
2016-02-04 09:57:27 +02:00
Markus Makela
e731f6a29e Fixed double freeing of buffers
The buffers in readwritesplit are handled at the routeQuery level and freeing
them at lower levels cause a double free of the used buffers.
2016-02-04 09:57:27 +02:00
Markus Makela
d9eeb72198 Added missing check for closed master backend reference into readwritesplit
The master DCB was checked for NULL-ness but the proper way is to check if
the backend reference is closed. This will fix a debug assertion in addition
to possibly preventing crashes.
2016-02-04 09:57:27 +02:00
Markus Makela
e7fb2640d3 Fixed authentication failing due to unknown database even though it exists
When the shard maps are being updated they are set into a stale state. This
means that one client connection is updating the shard maps and the information
in the shard map is not the most recent. This does mean that the information
is valid and authentication should succeed even if the shard map is stale.
2016-02-03 00:47:06 +02:00
Markus Makela
081bacdc64 Switched to MySQL client library headers for MaxScale
The include directories previously used by MaxScale were from the embedded
library. All parts of MaxScale apart from the query classifier now use
the client libraries.
2016-02-02 14:04:59 +02:00
MassimilianoPinto
0ab9733393 The router->rotating is no longer part of Unsafe Pos check
In blr_read_binlog the router->rotating is no longer used for Unsafe
Pos check
2016-02-01 09:12:48 +01:00
Johan Wikman
7e36e1cbb1 Link against Connector-C.
Now, qc_mysqlembedded is linked against MySQL's embedded library,
and MaxScale itself against Connector-C.

So, in order to build MaxScale, Connector-C must be installed.
This has been tested with Connector-C 2.2.1.

The build variable MYSQLCLIENT_LIBRARIES is no longer used.
2016-01-29 16:50:31 +02:00
Johan Wikman
e840f92cbb Make query_classifier a wrapper.
The query_classifier library is now only a wrapper that loads an
actual query classifier implementation. Currently it is hardwired
to load qc_mysqlembedded, which implements the query classifier
API using MySQL embedded.

This will be changed, so that the library to load is specified
when qc_init() is called. That will then allow the query classifier
to be specified in the config file.

Currently there seems to be a conflict between the mysql_library_end()
call made in qc_mysqlembedded and the mysql_library_end() call made in
gateway.c. The reason is that they both finalize a shared library.
For the time being mysql_library_end() is not called in gateway.c.
This problem is likely to go away by switching from the client
library to the connector-c library.
2016-01-29 16:35:03 +02:00
Johan Wikman
d9b022db10 Protect updating of router when rotating.
When rotating, all state variables of router are now updated while
protected by the router->binlog_lock lock.
2016-01-28 15:23:22 +02:00
Markus Makela
379004a2a4 Removed corrupted error messages from readwritesplit
When a slave server fails to execute a session command, the log message printed
the command that was being executed as if the ERR packet was a COM_QUERY packet.
This caused corrupt strings to be printed into the error logs.
2016-01-28 13:04:23 +02:00