490 Commits

Author SHA1 Message Date
Johan Wikman
6cd6ded3d8 Merge branch '2.1-oracle-compat' into develop-new-merge-oracle 2017-06-29 09:39:55 +02:00
Johan Wikman
bea551092d MXS-1278: Change modes when 'set sql_mode=' is encountered.
The initial setting of sql_mode affects how MaxScale initially
behaves with respect to autocommit.

When 'set sql_mode=[default|oracle];" is encountered, the query
classifier and autocommit modes are adjusted accordingly.
2017-06-28 21:36:08 +02:00
Johan Wikman
057ba4156d MXS-1278: Turn on ORCALE mode dynamically
If statement input is required for a session, we check the input
for "SET SQL_MODE..." statements and adjust the query classifier
accordingly.
2017-06-28 21:36:08 +02:00
Johan Wikman
12a291919a MXS-1275: SetSqlModeParser moved under MySQLClient
That's where it belongs as it is only the mysql client protocol that
will use it. It's a bit unfortunate that the qc test program compare
now needs to include a file from a protocol module directory, but
the fact is that the query classifier implementation and the test
programs should actually be *under* the mysql client protocol module.
2017-06-28 21:36:08 +02:00
Johan Wikman
b385e862b0 MXS-1275: Move SetSqlModeParser to .../server/core/maxscale
The parser is needed not only by mysql_client but also by the test
program of the qc-plugins, so it need to be available someplace
common.

Not that nice that both a qc-component and mysql_client will include
stuff from server/core/maxscale, but at least temporarily ok.
2017-06-28 21:36:08 +02:00
Johan Wikman
4bf2b27b3e MXS-1275: Accept @@[SESSION|GLOBAL].sql_mode = ... 2017-06-28 21:36:08 +02:00
Johan Wikman
21c1845798 MXS-1275: Accept LOCAL as a synonum for SESSION 2017-06-28 21:36:08 +02:00
Johan Wikman
2483f95be2 MXS-1275: Test comment handling 2017-06-28 21:36:08 +02:00
Johan Wikman
1c6c19d6ae MXS-1275: Inherit TrxBoundaryParser from CustomParser 2017-06-28 21:36:08 +02:00
Johan Wikman
9386143437 MXS-1275: Test also using non-contiguous statements 2017-06-28 21:33:04 +02:00
Johan Wikman
ee27efd62e MXS-1275: Add custom parser for "SET SQL_MODE=..."
SetSqlModeParser is a custom parser for detecting "SET SQL_MODE=..."
statements. It will be used in mysql_client and the result will be
used for updating the sql_mode of a session (that is to be added).

Thereafter, whenever a statement arrives, the value of that sql_mode
will be used for setting the mode of the query_classifier.
2017-06-28 21:33:04 +02:00
Johan Wikman
cd1e3f26cf MXS-1275: Compile mysql_client as C++
A custom parser for detecting "set sql_mode=ORACLE" is needed
and writing that is C++ is more convenient than writing it in
C. Consequently, so as to be able to use that parser, mysql_client
must be compiled as C++.
2017-06-28 21:33:04 +02:00
Markus Mäkelä
705d29ea41 MXS-852: Fix prepared statement collection
If a prepared statement response was collected into one buffer, it doesn't
need to be processed again. By jumping directly to the routing of the
collected result, we prevent the unnecessary splitting of buffers that
appears to happend with continuous preparations of prepared statements.
2017-06-22 10:40:19 +03:00
Markus Mäkelä
bd7a26d830 Abstract the session default database
The default database can now be manipulated with a set of functions
exposed by the maxscale/protocol/mysql.h header. This removes the need to
handle the structures themselves in the modules and is a step towards
moving the dcb->data contents inside the session.
2017-06-22 10:40:16 +03:00
Johan Wikman
4696c7e662 MXS-1249: Set the server version for qc 2017-06-21 13:28:59 +03:00
Johan Wikman
3e39ec906e Change server version from char pointer to char array
With this change, it is no longer possible that the server version
is deallocated at the very moment it is read. There is still a race,
but it's mostly harmless.
2017-06-19 14:05:26 +03:00
Markus Mäkelä
d59e98e238 MXS-1267: Remove DCB cloning
The DCB cloning is no longer needed as the tee filter now does the session
branching by simulating a local client connection.
2017-06-02 13:07:02 +03:00
Markus Mäkelä
e43cdcf741 MXS-1267: Expose MySQL backend authentication
Refactored the backend authentication functions so that they can be
exposed to the tee filter. This allows the tee filter to use the same
functions as the protocol modules use without having to reimplement them
inside the tee filter.
2017-06-02 13:07:02 +03:00
Johan Wikman
f546a17e77 Update change date of 2.2 2017-06-01 10:24:20 +03:00
Esa Korhonen
dbfd631fed Change session registry to a template class
The template class wraps a HashMap such that only a few operations
are allowed. Usage requires specializing a RegistryTraits class
template for each entry type.
2017-05-19 10:16:37 +03:00
Markus Mäkelä
17ba824d9f Only close valid sessions
When a session is being closed, the state needs to be checked. If the
session creation failed, the session need to be only freed.
2017-05-16 11:39:57 +03:00
Markus Mäkelä
4117dcf410 Only store established connections in the pool
If a connection has not been fully established (i.e. authentication has
been completed) then it should not be considered as a connection pool
candidate.
2017-05-15 19:32:02 +03:00
Esa Korhonen
39da11763b Add test for parse_kill_query() 2017-05-11 12:41:13 +03:00
Esa Korhonen
4add7a14fc parse_kill_query() now also expects the "KILL"
This was originally removed, since it was checking the same word
twice. However, the parsing is clearer with it and the cost is only
paid when the KILL is detected, which is very rare.

Also, fix some incorrect parsing.
2017-05-11 12:06:42 +03:00
Esa Korhonen
a0cd067a03 KILL [CONNECTION | QUERY] support, part3
The text-version of "KILL CONNECTION" command is now supported. To keep the
overhead low, only minimal parsing is done on the query. The query
needs to start in the beginning of the mysql-packet, have no comments
and have limited whitespace as the total length of the query is limited.
Both "KILL 123" and "KILL CONNECTION 123" are accepted.

"KILL QUERY 123" is also accepted but not acted on, as it requires larger
changes.
2017-05-10 13:43:28 +03:00
Esa Korhonen
aebe839990 Change session id to 64bit
The server internal session id may be larger than 4 bytes (MariaDB uses 8)
but only 4 are sent in the handshake. The full value can be queried
from the server, but this query is not supported by MaxScale yet. In any
case, both the protocol and MXS_SESSION now have 64 bit counters. Only the
low 32 bits are sent in the handshake, similar to server.
2017-05-10 13:26:25 +03:00
Esa Korhonen
ee20191645 KILL [CONNECTION | QUERY] support, part2B
Various small changes to part2, as suggested by comments and otherwise.
Mostly renaming, working logic should not change.

Exception: session id changed to 64bit in the container and associated
functions. Another commit will change it to 64bit in the session itself.
2017-05-08 09:58:02 +03:00
Esa Korhonen
17f6e94cba KILL [CONNECTION | QUERY] support, part2
MySQL sessions are added to a hasmap when created, removed when closed.
MYSQL_COM_PROCESS_KILL is now detected, the thread_id is read and the kill
command sent to all worker threads to find the correct session. If found, a
fake hangup even is created for the client dcb.

As is, this function is of little use since the client could just disconnect
itself instead. Later on, additional commands of this nature will be  added.
2017-05-08 09:51:07 +03:00
Markus Mäkelä
e62be5034a Use constant sized arrays for some service strings
The `user`, `password`, `version_string` and `weightby` values should be
allocated as a part of the service structure. This allows them to be
modified at runtime without having to worry about memory allocation
problems.

Although this removes the problem of reallocation, it still does not make
the updating of the strings thread-safe. This can cause invalid values to
be read from the service strings.
2017-05-04 09:17:48 +03:00
Esa Korhonen
bfd94c2b31 KILL [CONNECTION | QUERY] support, part1
Preparation for adding KILL syntax support.
Session id changed to uint32 everywhere. Added atomic op.
Session id can be acquired before session_alloc().
Added session_alloc_with_id(), which is given a session id number.
Worker object has a session_id->SESSION* mapping, not used yet.
2017-05-02 10:29:55 +03:00
Esa Korhonen
8f50269338 Add proxy protocol support
Adds a server-specific parameter, "use_proxy_protocol". If enabled,
a header string is sent to the backend when a routing session connection
changes state to MXS_AUTH_STATE_CONNECTED. The string contains the real
client IP and port.
2017-04-27 12:29:46 +03:00
Markus Mäkelä
36d06960bf Combine query preparation into one function
The same operations of protocol state and inspections of the buffer were
done in multiple places. Combining these into one function removes the
duplicated code.
2017-04-20 14:22:55 +03:00
Markus Mäkelä
73dd9bd025 Allow collection of prepared statement responses
The backend MySQL protocol can now collect prepared statement preparation
responses as well as result sets. This removes the need to parse and
collect the preparation responses at the router level.
2017-04-20 14:22:55 +03:00
Markus Mäkelä
a88e98035f Clean up MySQL protocol debug logging
Removed pthread_self calls from the backend modules. This makes the debug
logging easier to parse when the messages aren't prefixed with the verbose
thread ID.
2017-04-20 14:22:55 +03:00
Markus Mäkelä
8fe31f360d Remove multi-packet additions to response parsing
The additions to the packet parsing code weren't necessary once the
statement output change was reverted.
2017-04-20 14:18:40 +03:00
Markus Mäkelä
5037646846 Remove redundant flag value definitions
The response status flag values are already declared in the mysql_com.h
header.
2017-04-20 14:18:40 +03:00
Markus Mäkelä
d7258fffd0 MXS-1203: Improve resultset processing functions
The functions used to track the resultset EOF packets now expose the
position of the end of the result set. This allows the modules that use
them to check if more results exist in the same buffer.

Added the status bits for OK and EOF packets to the mysql.h protocol
header. This can be used to check for various state changes that happen in
the session. Currently the status bits are only used to detect if more
results are expected.
2017-04-20 14:18:40 +03:00
Markus Mäkelä
a1c7ee438d MXS-1203: Fix current command tracking with statement routing
When statement based routing was used, it was possible that the current
statement being executed wasn't properly updated. Readwritesplit requires
it to track whether a command will create a response.
2017-04-20 14:18:40 +03:00
Markus Mäkelä
66cf571412 MXS-1203: Better handling of batch queries
When batched queries are done through readwritesplit, it will now handle
them one by one. This allows batched queries to be used with
readwritesplit but it does impose a performance penalty when compared to
direct execution on the backend.
2017-04-20 14:18:39 +03:00
Markus Mäkelä
ad1c05b015 Merge branch '2.1' into develop 2017-04-05 11:35:13 +03:00
Markus Mäkelä
860b14526c Fix mxs_mysql_is_result_set
The function assumed that the packet payload was always at least two
bytes.
2017-04-04 15:28:39 +03:00
Markus Mäkelä
1ff83150f0 Revert "Route statements in larger batches"
This reverts commit f3c83770903151a0a3b53593c3e05fa0af94cd5f. The
functionality was used implicitly by modules that declare the
RCAP_TYPE_CONTIGUOUS_OUTPUT capability.
2017-04-04 09:56:51 +03:00
Markus Mäkelä
f3c8377090 Route statements in larger batches
The RCAP_TYPE_STMT_OUTPUT is not used in its previous form. It can be
altered to route only complete packets back to the client. This allows
routers to do safer parsing on the results.
2017-04-03 12:07:45 +03:00
Markus Mäkelä
cbc1e864d9 Use RFC 3986 compliant addresses in log messages
When log messages are written with both address and port information, IPv6
addresses can cause confusion if the normal address:port formatting is
used. The RFC 3986 suggests that all IPv6 addresses are expressed as a
bracket enclosed address optionally followed by the port that is separate
from the address by a colon.

In practice, the "all interfaces" address and port number 3306 can be
written in IPv4 numbers-and-dots notation as 0.0.0.0:3306 and in IPv6
notation as [::]:3306. Using the latter format in log messages keeps the
output consistent with all types of addresses.

The details of the standard can be found at the following addresses:

     https://www.ietf.org/rfc/rfc3986.txt

     https://www.rfc-editor.org/std/std66.txt
2017-03-31 14:12:58 +03:00
Markus Mäkelä
c2857b976a Add collectable resultset buffer type
The new type allows routers to send queries and get complete result sets
as a response. This allows the routers to easily send commands that create
result sets and which are parsed by the router.

Currently only the schemarouter benefits from this new capability as it
generates the database mappings by parsing the output of a SHOW DATABASES
query.
2017-03-31 14:12:02 +03:00
Markus Mäkelä
5c1c89c835 Remove unused buffer types
A part of the buffer types weren't used or provided no real functionality.
2017-03-31 14:12:02 +03:00
Markus Mäkelä
1ba399a62a Use size_t in all functions
Some of the protocol modules use ssize_t instead of size_t.

Split the function that counts the number of response packets a session
command will receive into two parts. This allows it to be reused
elsewhere.
2017-03-31 14:12:00 +03:00
Markus Mäkelä
509a5c6ed3 Fix error handling for binlogrouter
The binlogrouter error handling closed the DCB twice. This was caused by
the change in the way the DCB error handling is done.

The protocol modules now also call the error handling routine even if the
router session is NULL. This enables the binlogrouter to manage
authentication failures correctly instead of trying to reconnect again.
2017-03-31 14:11:46 +03:00
Markus Mäkelä
710012ac5d MXS-827: Add connection keepalive
The readwritesplit now sends COM_PING queries to backend servers that have
been idle for too long. The option is configured with the
`connection_keepalive` parameter.
2017-03-26 13:57:25 +03:00
Markus Mäkelä
7bd05d4581 Merge branch '2.1' into develop 2017-03-22 15:20:21 +02:00