Commit Graph

8043 Commits

Author SHA1 Message Date
44900882a5 Merge branch '2.1' into develop 2017-06-30 11:11:34 +03:00
ff278df3bd Call non_native_setup in scripts that use mysqltest
A few tests didn't call it explicitly and relied on the implicit
initialization that was done in the test driver.
2017-06-30 11:07:18 +03:00
674b3887c9 MXS-1301: function matches functionless queries in accept mode
A function type rule matches a query if the query uses a function defined
in the rule. This is the desired behavior for blacklist mode operation
with `action=block`.

When in whitelist mode, all queries must match a rule to be allowed
through. For function type rules, this fact is problematic as queries that
don't use functions are blocked. The desired behavior is to allow the use
of certain functions while preventing the use of others.

The allowed set of functions should always contain the "empty set" (no
functions are used) when the filter is in whitelist mode.
2017-06-30 11:07:18 +03:00
902013e4f8 Fix off-by-one false positive in maxavro
The float and double types were calculated to exceed the internal buffer
sizes even though the buffer was of the correct size.
2017-06-30 11:07:18 +03:00
809dea34e0 Fix overlap of router->binlog_name in blr_file_create
A call to strcpy was made in blr_file_create where the function was given
the same pointer as both parameters. To avoid this, the file name is now
copied to a local variable before the router variables are modified.
2017-06-30 11:07:18 +03:00
9a42709da4 Properly terminate 'ping' subcommand list
The 'ping' subcommand list was missing the terminating value of
`{EMPTY_OPTION}`.
2017-06-30 11:07:18 +03:00
308a7d63d4 Fix double initialization of tests
The tests that used the mysqltest driver were initialized twice.
2017-06-30 11:07:18 +03:00
a19d93380e Make block_size a size parameter
The block_size signifies a size in bytes so it makes sense to change it to
the size type.
2017-06-30 11:07:18 +03:00
25fd5a8a93 Fix crash on session allocation failure
The the return value was used before it was checked.
2017-06-30 11:07:18 +03:00
322fae8326 Add wrapper functions for buffer and original IV access
Added functions for accessing the buffer and original IV. This hides the
changes introduced in OpenSSL 1.1.
2017-06-30 10:46:05 +03:00
a47d4c40f5 Abstract EVP cipher context creation
The EVP_CIPHER_CTX is now created inside a wrapper function to add support
for OpenSSL 1.1. Also fixed improper use of the EVP_CIPHER_CTX internals
in binlogrouter.
2017-06-30 10:46:04 +03:00
469b432ebe Detect and handle OpenSSL 1.1
OpenSSL 1.1 supports most of the native threading libraries, including
pthread. This means that only versions before 1.1 need the thread handling
code.
2017-06-30 10:42:20 +03:00
f91df4617a MXS-1248: Add simple cte test 2017-06-30 08:36:19 +02:00
84c5aa9934 MXS-1248: Report fields from CTE expressions
Now field and function information of CTE expressions are
collected.
2017-06-30 08:36:19 +02:00
48a67eeef2 qc_mysqlembedded: Collect ... WHERE EXISTS ...
Information was not collected from the subselect in a situation
e.g. like

    SELECT ... WHERE NOT EXISTS (SELECT ...)
2017-06-30 08:36:19 +02:00
5597db255b Add wrapper functions for buffer and original IV access
Added functions for accessing the buffer and original IV. This hides the
changes introduced in OpenSSL 1.1.
2017-06-29 15:55:44 +03:00
09fb336403 Abstract EVP cipher context creation
The EVP_CIPHER_CTX is now created inside a wrapper function to add support
for OpenSSL 1.1. Also fixed improper use of the EVP_CIPHER_CTX internals
in binlogrouter.
2017-06-29 15:55:44 +03:00
750f2ef96c Detect and handle OpenSSL 1.1
OpenSSL 1.1 supports most of the native threading libraries, including
pthread. This means that only versions before 1.1 need the thread handling
code.
2017-06-29 15:55:44 +03:00
8fb9b79dbb Make RWSplit::config() const
The function can be const.
2017-06-29 15:53:50 +03:00
1c5e1b705c Don't delete but call destructor on timeout
Deleting a stack allocated object will cause an immediate crash whereas
only calling the destructor will successfully complete the call with a
possibility of leaked memory or a crash later on.
2017-06-29 15:53:50 +03:00
3b3799889e MXS-852: Fix PS storage when locker to the master
When the router session is locked to the master, the storage of prepared
statements is not necessary as no other server can be used. The ID sent by
the client can be used without modification as it will always be the same
on the master.
2017-06-29 15:53:50 +03:00
fd9a08b9f4 Make Slave of External Server status configurable
The assignment of the Slave status with Slave of External Server can now
be controlled with the allow_external_slaves parameter.
2017-06-29 15:53:50 +03:00
1f8072f3d6 Fix temporary table detection in readwritesplit
The column information does not contain the used tables and the correct
function to use is qc_get_table_names.

Added a utility function for iterating over the set of tables. This
removes the need to have the same table iteration code in multiple places
and makes the code easier to comprehend.
2017-06-29 15:53:50 +03:00
5384162fb1 Fix slave server candidate ordering
The return value of the comparison was misinterpreted so that the
selection process preferred the new candidate over the current one if both
were equal. This was not an intended change and only a better candidate
should be chosen over the current candidate.
2017-06-29 15:53:50 +03:00
f7500cce9e Clean up route target resolution function
Cleaned up the function that resolves to which type of a server the query
should go.
2017-06-29 15:53:50 +03:00
dd05b44d38 Always extract the current command from the buffer
As readwritesplit queues queries, it needs to extract the command from the
buffer when the queued queries are routed. It cannot rely on the client
protocol command when the rerouting is taking place.
2017-06-29 15:53:50 +03:00
b0023e1689 Clean up readwritesplit API functions
Cleaned up some of the comments and documentation for the
functions. Renamed some variables and moved parts of the error handling
logging into a subfunction.
2017-06-29 15:53:50 +03:00
14692cd70c MXS-852: Expand binary PS cursor test
The test now tests that read-write splitting works both inside and outside
transactions.
2017-06-29 15:53:50 +03:00
f2b199b3b0 MXS-852: Refactor gathering of routing information
The routing information is now gathered into a struct before the routing
process is started. This allows the requirements of the query to be
gathered before the actual target is selected.
2017-06-29 15:53:50 +03:00
70123e3fae MXS-852: Add binary protocol cursor test
The test uses binary protocol prepared statements with cursors. The second
part of the test will not pass as the test uses parallel execution of
prepared statements.
2017-06-29 15:53:50 +03:00
a9a1291703 Split readwritesplit sources into separate parts
The common readwritesplit header was split into three distinct parts; the
instance, session and prepared statement headers. The definitions of any
members were moved to .cc files away from the headers.

The RWSplitSession, with its RWBackend class, is declared in the
rwsplitsession.hh header with all relevant definitions in
rwsplitsession.cc.

The PSManager class and all prepared statement related functions are now
located in the rwsplit_ps.hh header.

The old readwritesplit.hh header now contains the instance level
structures and all common classes used by the router. The
rwsplit_internal.hh header could be absorbed into the three newly created
headers with new headers for distinct parts of the router.
2017-06-29 15:53:50 +03:00
013b081b9e Rename readwritesplit router session class
Renamed the struct to RWSplitSession and removed the obsole next pointer.
2017-06-29 15:53:50 +03:00
917fe21f72 Refactor readwritesplit instance class
Changed the ROUTER_INSTANCE struct to a class and added functions for
common operations.

Renamed configuration and statistics structures and added constructors.

Moved objects around in readwritesplit.hh to be ready for a split into
multiple headers.
2017-06-29 15:53:50 +03:00
8ed16fd9d2 Reduce readwritesplit debug verbosity
Moved the LOG_RS output down to debug level. The output is not useful for
the end users.
2017-06-29 15:53:50 +03:00
16201592a2 MXS-852: Fix execution of COM_STMT_FETCH
The COM_STMT_FETCH queries are always executed when the result has not
been fully read. This means that the statement queuing code needs to allow
COM_STMT_FETCH commands to pass if a statement is being executed but the
command queue is empty.
2017-06-29 15:53:50 +03:00
5c94610b68 MXS-852: Update readwritesplit limitations
Updated limitations with notes about parallel execution of binary protocol
prepared statements. Remove limitations that have been lifted in 2.2.
2017-06-29 15:53:50 +03:00
296c1001a2 MXS-852: Track COM_STMT_EXECUTE by statement ID
The COM_STMT_EXECUTE targets are now tracked per statement ID. This should
theoretically allow parallel execution of COM_STMT_EXECUTE commands that
use cursors but the current implementation of the reply state processing
does not yet allow it.
2017-06-29 15:53:50 +03:00
40120264e7 MXS-852: Add simple PS routing test
The test checks that reads go to slaves and writes to the master.
2017-06-29 15:53:50 +03:00
d0b8ccd6b8 MXS-852: Track target of last COM_STMT_EXECUTE
Tracking of the node where the last COM_STMT_EXECUTE was sent allows
routing of all following COM_STMT_FETCH to the same node. This is required
for cursors to work.

MariaDB/MySQL does not support multiple active cursors so the
COM_STMT_FETCH will always refer to the latest COM_STMT_EXECUTE and using
a different ID goes against the protocol. If/when the support for multiple
active cursors is added, the tracking should be done for each
COM_STMT_EXECUTE statement. This should be relatively easily to achieve
but currently it is unnecessary.
2017-06-29 15:53:50 +03:00
52b7fb9340 MXS-852: Handle one-way session commands
Session commands that will not return a response can be completed
immediately. This requires some special code in the readwritesplit Backend
class implementation as well as a small addition to the Backend class
itself.

Since not all commands expect a response from the server, the queued query
routing function needs some adjustment. The routing of queued queries
should be attempted until a command which expects a response is found or
the queue is empty.

By properly handling these types of session commands, the router can
enable the execution of COM_STMT_CLOSE and COM_STMT_RESET on all
servers. This will prevent resource leakages in the server and allow
proper handling of COM_STMT type command.
2017-06-29 15:53:50 +03:00
5fc30740b7 MXS-852: Store the internal ID in the buffer
If the internal ID is stored in the buffer when it is moving inside the
readwritesplit router, the RWBackend can manage the execution of all
commands with a statement ID by replacing the stored ID with the correct
value.
2017-06-29 15:53:49 +03:00
3c4e1e3b4b MXS-852: Use stored query type for COM_STMT_EXECUTE
When a COM_STMT_EXECUTE or a COM_STMT_SEND_LONG_DATA command is executed,
the query type of the prepared statement is used. This allows read-only
prepared statements to be load balanced across slaves.
2017-06-29 15:53:49 +03:00
77f78c4b20 MXS-852: Map client PS handle to internal ID
Mapping the handles returned to the client to a session command ID allows
the mapping of client handle to the backend specific handle. Currently,
the mapping is used for diagnostic output only.
2017-06-29 15:53:49 +03:00
0aa0fa82b7 MXS-852: Add PS manager class
The class manages both text and binary protocol prepared statement ID to
type mapping. The text protocol statements are mapped by their plaintext
name and the binary protocol statements are mapped by the session command
ID of the prepared statement.

By mapping the binary protocol prepared statement type to the session
command identifier, we can store the types for both styles of prepared
statements in a very similar manner. When the prepared statement handle is
received from the backend and is sent to the client, the client handle to
session command ID mapping can be done. This allows the mapping of both
client and backend PS handles to internal session command IDs.
2017-06-29 15:53:49 +03:00
3eac28248d MXS-852: Store prepared statement handles in RWBackend
The readwritesplit Backend implementation maps the returned PS handles to
session command identifiers. This allows the handles to be retrieven later
on when the prepared statements are executed.
2017-06-29 15:53:49 +03:00
dd90ad01b6 MXS-852: Add PS response and command extraction functions
The mxs_mysql_extract_ps_response function extracts the binary protocol
prepared statement components and stores them in a common structure.

The mxs_mysql_get_command extracts the command byte from a GWBUF object
containing a complete MySQL packet.
2017-06-29 15:53:49 +03:00
6cd6ded3d8 Merge branch '2.1-oracle-compat' into develop-new-merge-oracle 2017-06-29 09:39:55 +02:00
7f1a949198 Firewall must use its own operator codes
The firewall uses a bitmask for representing what operations a
particular rule should be applied to. Consquently it cannot use
the query classifier qc_query_op_t enumerator values as such,
as they are consecutive numbers.
2017-06-28 21:36:08 +02:00
f0ddbc5c8f MXS-1258: Remove non-critical failing test-cases
"SELECT LENGTH(_utf8 0xC39F), LENGTH(CHAR(14844588 USING utf8));"

Type and operation collected correctly, but function names are not
2017-06-28 21:36:08 +02:00
bea551092d MXS-1278: Change modes when 'set sql_mode=' is encountered.
The initial setting of sql_mode affects how MaxScale initially
behaves with respect to autocommit.

When 'set sql_mode=[default|oracle];" is encountered, the query
classifier and autocommit modes are adjusted accordingly.
2017-06-28 21:36:08 +02:00