8235 Commits

Author SHA1 Message Date
Markus Mäkelä
750f2ef96c Detect and handle OpenSSL 1.1
OpenSSL 1.1 supports most of the native threading libraries, including
pthread. This means that only versions before 1.1 need the thread handling
code.
2017-06-29 15:55:44 +03:00
Markus Mäkelä
8fb9b79dbb Make RWSplit::config() const
The function can be const.
2017-06-29 15:53:50 +03:00
Markus Mäkelä
1c5e1b705c Don't delete but call destructor on timeout
Deleting a stack allocated object will cause an immediate crash whereas
only calling the destructor will successfully complete the call with a
possibility of leaked memory or a crash later on.
2017-06-29 15:53:50 +03:00
Markus Mäkelä
3b3799889e MXS-852: Fix PS storage when locker to the master
When the router session is locked to the master, the storage of prepared
statements is not necessary as no other server can be used. The ID sent by
the client can be used without modification as it will always be the same
on the master.
2017-06-29 15:53:50 +03:00
Markus Mäkelä
fd9a08b9f4 Make Slave of External Server status configurable
The assignment of the Slave status with Slave of External Server can now
be controlled with the allow_external_slaves parameter.
2017-06-29 15:53:50 +03:00
Markus Mäkelä
1f8072f3d6 Fix temporary table detection in readwritesplit
The column information does not contain the used tables and the correct
function to use is qc_get_table_names.

Added a utility function for iterating over the set of tables. This
removes the need to have the same table iteration code in multiple places
and makes the code easier to comprehend.
2017-06-29 15:53:50 +03:00
Markus Mäkelä
5384162fb1 Fix slave server candidate ordering
The return value of the comparison was misinterpreted so that the
selection process preferred the new candidate over the current one if both
were equal. This was not an intended change and only a better candidate
should be chosen over the current candidate.
2017-06-29 15:53:50 +03:00
Markus Mäkelä
f7500cce9e Clean up route target resolution function
Cleaned up the function that resolves to which type of a server the query
should go.
2017-06-29 15:53:50 +03:00
Markus Mäkelä
dd05b44d38 Always extract the current command from the buffer
As readwritesplit queues queries, it needs to extract the command from the
buffer when the queued queries are routed. It cannot rely on the client
protocol command when the rerouting is taking place.
2017-06-29 15:53:50 +03:00
Markus Mäkelä
b0023e1689 Clean up readwritesplit API functions
Cleaned up some of the comments and documentation for the
functions. Renamed some variables and moved parts of the error handling
logging into a subfunction.
2017-06-29 15:53:50 +03:00
Markus Mäkelä
14692cd70c MXS-852: Expand binary PS cursor test
The test now tests that read-write splitting works both inside and outside
transactions.
2017-06-29 15:53:50 +03:00
Markus Mäkelä
f2b199b3b0 MXS-852: Refactor gathering of routing information
The routing information is now gathered into a struct before the routing
process is started. This allows the requirements of the query to be
gathered before the actual target is selected.
2017-06-29 15:53:50 +03:00
Markus Mäkelä
70123e3fae MXS-852: Add binary protocol cursor test
The test uses binary protocol prepared statements with cursors. The second
part of the test will not pass as the test uses parallel execution of
prepared statements.
2017-06-29 15:53:50 +03:00
Markus Mäkelä
a9a1291703 Split readwritesplit sources into separate parts
The common readwritesplit header was split into three distinct parts; the
instance, session and prepared statement headers. The definitions of any
members were moved to .cc files away from the headers.

The RWSplitSession, with its RWBackend class, is declared in the
rwsplitsession.hh header with all relevant definitions in
rwsplitsession.cc.

The PSManager class and all prepared statement related functions are now
located in the rwsplit_ps.hh header.

The old readwritesplit.hh header now contains the instance level
structures and all common classes used by the router. The
rwsplit_internal.hh header could be absorbed into the three newly created
headers with new headers for distinct parts of the router.
2017-06-29 15:53:50 +03:00
Markus Mäkelä
013b081b9e Rename readwritesplit router session class
Renamed the struct to RWSplitSession and removed the obsole next pointer.
2017-06-29 15:53:50 +03:00
Markus Mäkelä
917fe21f72 Refactor readwritesplit instance class
Changed the ROUTER_INSTANCE struct to a class and added functions for
common operations.

Renamed configuration and statistics structures and added constructors.

Moved objects around in readwritesplit.hh to be ready for a split into
multiple headers.
2017-06-29 15:53:50 +03:00
Markus Mäkelä
8ed16fd9d2 Reduce readwritesplit debug verbosity
Moved the LOG_RS output down to debug level. The output is not useful for
the end users.
2017-06-29 15:53:50 +03:00
Markus Mäkelä
16201592a2 MXS-852: Fix execution of COM_STMT_FETCH
The COM_STMT_FETCH queries are always executed when the result has not
been fully read. This means that the statement queuing code needs to allow
COM_STMT_FETCH commands to pass if a statement is being executed but the
command queue is empty.
2017-06-29 15:53:50 +03:00
Markus Mäkelä
5c94610b68 MXS-852: Update readwritesplit limitations
Updated limitations with notes about parallel execution of binary protocol
prepared statements. Remove limitations that have been lifted in 2.2.
2017-06-29 15:53:50 +03:00
Markus Mäkelä
296c1001a2 MXS-852: Track COM_STMT_EXECUTE by statement ID
The COM_STMT_EXECUTE targets are now tracked per statement ID. This should
theoretically allow parallel execution of COM_STMT_EXECUTE commands that
use cursors but the current implementation of the reply state processing
does not yet allow it.
2017-06-29 15:53:50 +03:00
Markus Mäkelä
40120264e7 MXS-852: Add simple PS routing test
The test checks that reads go to slaves and writes to the master.
2017-06-29 15:53:50 +03:00
Markus Mäkelä
d0b8ccd6b8 MXS-852: Track target of last COM_STMT_EXECUTE
Tracking of the node where the last COM_STMT_EXECUTE was sent allows
routing of all following COM_STMT_FETCH to the same node. This is required
for cursors to work.

MariaDB/MySQL does not support multiple active cursors so the
COM_STMT_FETCH will always refer to the latest COM_STMT_EXECUTE and using
a different ID goes against the protocol. If/when the support for multiple
active cursors is added, the tracking should be done for each
COM_STMT_EXECUTE statement. This should be relatively easily to achieve
but currently it is unnecessary.
2017-06-29 15:53:50 +03:00
Markus Mäkelä
52b7fb9340 MXS-852: Handle one-way session commands
Session commands that will not return a response can be completed
immediately. This requires some special code in the readwritesplit Backend
class implementation as well as a small addition to the Backend class
itself.

Since not all commands expect a response from the server, the queued query
routing function needs some adjustment. The routing of queued queries
should be attempted until a command which expects a response is found or
the queue is empty.

By properly handling these types of session commands, the router can
enable the execution of COM_STMT_CLOSE and COM_STMT_RESET on all
servers. This will prevent resource leakages in the server and allow
proper handling of COM_STMT type command.
2017-06-29 15:53:50 +03:00
Markus Mäkelä
5fc30740b7 MXS-852: Store the internal ID in the buffer
If the internal ID is stored in the buffer when it is moving inside the
readwritesplit router, the RWBackend can manage the execution of all
commands with a statement ID by replacing the stored ID with the correct
value.
2017-06-29 15:53:49 +03:00
Markus Mäkelä
3c4e1e3b4b MXS-852: Use stored query type for COM_STMT_EXECUTE
When a COM_STMT_EXECUTE or a COM_STMT_SEND_LONG_DATA command is executed,
the query type of the prepared statement is used. This allows read-only
prepared statements to be load balanced across slaves.
2017-06-29 15:53:49 +03:00
Markus Mäkelä
77f78c4b20 MXS-852: Map client PS handle to internal ID
Mapping the handles returned to the client to a session command ID allows
the mapping of client handle to the backend specific handle. Currently,
the mapping is used for diagnostic output only.
2017-06-29 15:53:49 +03:00
Markus Mäkelä
0aa0fa82b7 MXS-852: Add PS manager class
The class manages both text and binary protocol prepared statement ID to
type mapping. The text protocol statements are mapped by their plaintext
name and the binary protocol statements are mapped by the session command
ID of the prepared statement.

By mapping the binary protocol prepared statement type to the session
command identifier, we can store the types for both styles of prepared
statements in a very similar manner. When the prepared statement handle is
received from the backend and is sent to the client, the client handle to
session command ID mapping can be done. This allows the mapping of both
client and backend PS handles to internal session command IDs.
2017-06-29 15:53:49 +03:00
Markus Mäkelä
3eac28248d MXS-852: Store prepared statement handles in RWBackend
The readwritesplit Backend implementation maps the returned PS handles to
session command identifiers. This allows the handles to be retrieven later
on when the prepared statements are executed.
2017-06-29 15:53:49 +03:00
Markus Mäkelä
dd90ad01b6 MXS-852: Add PS response and command extraction functions
The mxs_mysql_extract_ps_response function extracts the binary protocol
prepared statement components and stores them in a common structure.

The mxs_mysql_get_command extracts the command byte from a GWBUF object
containing a complete MySQL packet.
2017-06-29 15:53:49 +03:00
MassimilianoPinto
4993fd683c Merge branch 'develop' into MXS-1266 2017-06-29 11:31:56 +02:00
Johan Wikman
6cd6ded3d8 Merge branch '2.1-oracle-compat' into develop-new-merge-oracle 2017-06-29 09:39:55 +02:00
Johan Wikman
7f1a949198 Firewall must use its own operator codes
The firewall uses a bitmask for representing what operations a
particular rule should be applied to. Consquently it cannot use
the query classifier qc_query_op_t enumerator values as such,
as they are consecutive numbers.
2017-06-28 21:36:08 +02:00
Johan Wikman
f0ddbc5c8f MXS-1258: Remove non-critical failing test-cases
"SELECT LENGTH(_utf8 0xC39F), LENGTH(CHAR(14844588 USING utf8));"

Type and operation collected correctly, but function names are not
2017-06-28 21:36:08 +02:00
Johan Wikman
bea551092d MXS-1278: Change modes when 'set sql_mode=' is encountered.
The initial setting of sql_mode affects how MaxScale initially
behaves with respect to autocommit.

When 'set sql_mode=[default|oracle];" is encountered, the query
classifier and autocommit modes are adjusted accordingly.
2017-06-28 21:36:08 +02:00
Johan Wikman
8128c74341 MXS-1196: Update Oracle tests 2017-06-28 21:36:08 +02:00
Johan Wikman
a14755a403 MXS-1278: Add 'sql_mode' configuration parameter
Using the 'sql_mode' configuration parameter, the initial sql mode
can be specified.
2017-06-28 21:36:08 +02:00
Johan Wikman
83f8010c4b MXS-1278: Add 'sql_mode' configuration parameter
Using the 'sql_mode' configuration parameter, the initial sql mode
can be specified.
2017-06-28 21:36:08 +02:00
Johan Wikman
a690b44919 MXS-1278: Sql mode must be specified explicitly
The default sql mode must now be provided explicitly when the query
classifier is setup. This is in preparation for "sql_mode" becoming
a global configuration parameter of MaxScale.
2017-06-28 21:36:08 +02:00
Johan Wikman
057ba4156d MXS-1278: Turn on ORCALE mode dynamically
If statement input is required for a session, we check the input
for "SET SQL_MODE..." statements and adjust the query classifier
accordingly.
2017-06-28 21:36:08 +02:00
Johan Wikman
12a291919a MXS-1275: SetSqlModeParser moved under MySQLClient
That's where it belongs as it is only the mysql client protocol that
will use it. It's a bit unfortunate that the qc test program compare
now needs to include a file from a protocol module directory, but
the fact is that the query classifier implementation and the test
programs should actually be *under* the mysql client protocol module.
2017-06-28 21:36:08 +02:00
Johan Wikman
25314b6662 MXS-1275: Add client protocol data to MXS_SESSION
The current sql mode will be tracked in that variable. Since one
thread will handle multiple sessions, we need to track the sql mode
of a session, so that the sql mode of the query classifier can be
set whenever we handle a request of a particular session.

As the sql-mode will be managed by the client protocol, the data
can be private to it as well.
2017-06-28 21:36:08 +02:00
Johan Wikman
51452ecb3b MXS-1275: No need to be explicit about the sql mode
As statements as "set sql_mode=oracle" are recognized and the
behaviour of the classifiers is configured accordingly, the mode
need not the hardwired.
2017-06-28 21:36:08 +02:00
Johan Wikman
d9448e1550 MXS-1275: Function name mappings need to follow sql_mode 2017-06-28 21:36:08 +02:00
Johan Wikman
a60b6473ed MXS-1275: Check for "set sql_mode=ORACLE" and act accordingly 2017-06-28 21:36:08 +02:00
Johan Wikman
021fe09edb MXS-1275: qc_mysqlembedded, add support for qc_[get|set]_sql_mode 2017-06-28 21:36:08 +02:00
Johan Wikman
368ae65779 MXS-1275: // is not an mysql comment token 2017-06-28 21:36:08 +02:00
Johan Wikman
b385e862b0 MXS-1275: Move SetSqlModeParser to .../server/core/maxscale
The parser is needed not only by mysql_client but also by the test
program of the qc-plugins, so it need to be available someplace
common.

Not that nice that both a qc-component and mysql_client will include
stuff from server/core/maxscale, but at least temporarily ok.
2017-06-28 21:36:08 +02:00
Johan Wikman
3084e95772 MXS-1275: Implement qc_[get|set]_sql_mode for qc_sqlite 2017-06-28 21:36:08 +02:00
Johan Wikman
9ae0526efb MXS-1275: Extend QC-API to allow setting of sql mode
Only API changes, implementation will follow.
2017-06-28 21:36:08 +02:00
Johan Wikman
4bf2b27b3e MXS-1275: Accept @@[SESSION|GLOBAL].sql_mode = ... 2017-06-28 21:36:08 +02:00