Commit Graph

987 Commits

Author SHA1 Message Date
0df326d581 MXS-1824: Track only the opening of cursors
Whether a cursor is open or not does not matter as long as the attempt to
open a cursor is detected.
2018-04-26 16:02:09 +03:00
092532745d MXS-1507: Add transaction size limit
Added the `transaction_replay_max_size` parameter that controls the
maximum size of a transaction that can be replayed. If the limit is
exceeded, the stored statements are released thus preventing the
transaction from being replayed.

This limitation prevents accidental misuse of the transaction replaying
system when autocommit is disabled. It also allows the user to control the
amount of memory that MaxScale will use.
2018-04-26 13:44:26 +03:00
7de7e7b2fe MXS-1507: Replay read-only transaction
Read-only transactions are now replayed if the node in question fails.
2018-04-26 13:44:25 +03:00
2848f96945 MXS-1507: Make transaction replay configurable
The transaction retrying behavior is now configurable and documented. The
`transaction_replay` parameter implicitly enables the required
functionality in the router that it needs.
2018-04-26 13:44:25 +03:00
c1c942a058 MXS-1507: Retry interrupted queries in transactions
As the current query was added to the transaction log before it finished,
the m_current_query contained a duplicate of the latest transaction log
entry. To correctly log only successful transactions, the statement should
be added only after it has successfully completed. This change also
removed the unnecessary cloning that took place when the statement was
added to the log before it finished.

With the fixed transaction logging, the value of m_current_query can be
stashed for later retrying while the replay process is happening. If the
replay completes successfully and the checksums match, the interrupted
query is retried.

Also added a clarifying comment to can_retry_query to explain why a query
inside a transaction cannot be retried.
2018-04-26 13:44:25 +03:00
01bf5cc8b0 MXS-1507: Add initial implementation of transaction replay
Added the initial implementation of transaction replay. Transactions are
only replayed if the master fails when no statement is being executed.

The validity of the replayed transaction is done by verifying that the
checksums of the returned results are equal.

Added a close function into the Trx class to make resetting its state
easier. Also changed the return type of the pop_stmt to GWBUF* as the
places where it is used expect a raw GWBUF pointer.
2018-04-26 13:44:25 +03:00
daecb6980b MXS-1507: Use same mechanism for all delayed routing
Passing the delay to the retry_query function allows the same function to
be used in all cases. This removes duplicate code.
2018-04-26 13:44:25 +03:00
050af8fb52 MXS-1810: Create Trx class
The class encapsulates the relevant information of a transaction.
2018-04-26 13:44:24 +03:00
94038933d8 MXS-1810: Store transaction contents
The queries that make up the transaction are now stored in the router
session while the transaction is in progress. For the time being, the
queries are only used to log extra information about the transaction
contents.
2018-04-26 13:44:24 +03:00
53a5685dc2 MXS-1810: Add transaction checksums
Readwritesplit now calculates checksums for all successful and failed
transactions. This checksum is not of any practical use until the
transaction replaying is implemented.
2018-04-26 13:44:23 +03:00
a2b78d40cf Merge branch '2.2' into develop 2018-04-18 08:00:48 +03:00
3a1c2119fb MXS-1804: Fix hanging of large session commands
Large session commands weren't properly handled which caused the router to
think that the trailing end of a multi-packet query was actually a new
query.

This cannot be confidently solved in 2.2 which is why the router session
is now closed the moment a large session command is noticed.
2018-04-17 15:04:12 +03:00
232f807ef3 MXS-1808: Only store SQL statements for retrying
Only commands that can contain an SQL statements should be stored for
retrying (COM_QUERY and COM_EXECUTE). Other commands are either session
commands or do not work with query retrying.
2018-04-17 11:25:56 +03:00
12bd34c8d3 MXS-1625 Remove the PS manager from RWS
Not used as it now is in QueryClassifier
2018-04-17 10:09:37 +03:00
7d97bf76ea MXS-1625 Remove duplicate function
Correct test in remaining function.
2018-04-17 10:09:37 +03:00
ec33fcf87d Merge branch '2.2' into develop 2018-04-13 14:53:00 +03:00
802b16f709 MXS-1786: Fix hang on COM_STATISTICS
The commands needs to be handled separately from the rest of the result
types.

Added a test case that reproduces the problem and verifies that the change
in code fixes it.
2018-04-12 20:00:00 +03:00
1a293c0093 MXS-1785: Don't assume empty packet is for LDLI
When a LOAD DATA LOCAL INFILE finishes, the client sends an empty
packet. The second case when the client sends an empty packet when the
previous packet was exactly 0xffffff bytes long. These two packets were
confused which caused the internal state to temporarily flip from inactive
to ending and back to inactive.

The aforementioned flip-flopping didn't have any practical differences but
it was caught by a debug assertion.
2018-04-12 19:36:34 +03:00
526d57049b Add backend name to info level messages
The messaged that logged the URI of the backend now also log the name of
the backend.
2018-04-12 11:20:37 +03:00
dc3c848df8 Pick servers that can be connected to as candidates
Only servers that qualify to be connected should be considered as
candidate servers. This triggered a debug assertion when a slave server
failed to execute a session command but it was chosen as a candidate
server later on.
2018-04-12 10:42:39 +03:00
da03c73373 MXS-1776: Fix COM_STMT_EXECUTE flag extraction
The code extracted the statement id, not the flags.
2018-04-12 10:25:10 +03:00
fab8477c05 MXS-1776: Fix utility functions
The COM_STMT_FETCH command will create a response. This was a
readwritesplit-specific interpretation of the command and it was wrong.

Also record the currently executed command event for session commands.
2018-04-12 09:44:28 +03:00
ad5458f0e7 MXS-1776: Initialize RWBackend::m_command
The variable was not initialized which caused COM_CHANGE_USER to produce
unexpected behavior.
2018-04-12 09:43:38 +03:00
311adf817f MXS-1776: Handle recursive COM_STMT_EXECUTE commands
Readwritesplit would not handle multiple overlapping COM_STMT_EXECUTE
commands properly if they opened cursors. This was due to the fact that
the result would not be marked as complete and COM_STMT_FETCH commands
were executed as if they did not return results.

The correct implementation is to consider a COM_STMT_EXECUTE that opens a
cursor complete only when the first EOF packet is read (that is, when the
resultset header is read). This allows subsequent COM_STMT_FETCH commands
to be handled separately.

The separate COM_STMT_FETCH handling must count the number of packets that
are being fetched. This allows correct tracking of the state of a
COM_STMT_FETCH by checking that the number of packets is correct or the
second EOF/ERR packet is read.
2018-04-11 15:26:37 +03:00
1eefb46e68 MXS-1773: Update internal state when LOAD DATA LOCAL INFILE fails
When a LOAD DATA LOCAL INFILE is actively rejected by the server, the
server sends an error to the client. This error was not detected and the
router was stuck in the special mode that handles LOAD DATA LOCAL INFILE.
2018-04-11 09:25:23 +03:00
1da33c4423 MXS-1625 Remove RWS RouteInfo
Now uses the one in QueryClassifier directly.
2018-04-10 17:41:59 +03:00
9be98df41c MXS-1625 Move RouteInfo to QueryClassifier 2018-04-10 17:41:59 +03:00
91b1ce39b8 MXS-1625 Move routing target bits to QueryClassifier 2018-04-10 17:41:59 +03:00
948ff9b5f1 MXS-1506: Skip error logging if retrying query
If the query is about to be retried, error logging must be skipped.
2018-04-10 15:32:23 +03:00
7f05d0ae05 MXS-1506: Refactor causal read reply processing
The state could be factored out into a boolean variable as the reply
processing can be in two states: Either waiting for the response to
MASTER_GTID_WAIT or updating packet numbers.

The packet number updating can always be done as long as a buffer is
available.  The discard_master_wait_gtid_result function discards the OK
packet before the packet numbers are updated so any trailing packets get
corrected properly.
2018-04-10 15:32:23 +03:00
f124e388fa MXS-1506: Fix handle_causal_read_reply
The function did not return the changed buffer to the caller of the
function.
2018-04-10 15:32:23 +03:00
52c55a365e MXS-1506: Store query if delayed_retry is enabled
The delayed retrying depends on the current query being stored.
2018-04-10 15:32:23 +03:00
997835c7f8 MXS-1506: Remove mxs::Buffer wrappers from RWSplitSession
Now that the mxs::Buffer has a copy_from for GWBUFs, there's no need to
use wrapper functions. This removes the duplicate code in the wrappers.
2018-04-10 15:32:23 +03:00
90eeba45df MXS-1506: Use mxs::Buffer for current query
The mxs::Buffer class already implemented most of the code that was in the
RWSplitSession class.
2018-04-10 15:32:22 +03:00
196543ef39 MXS-1506: Retry interrupted writes
If a query is interrupted that was sent to the master, it is now
retried. This allows all autocommit queries to be transparently retried if
the server in question fails.
2018-04-10 15:31:51 +03:00
ed0f20708f MXS-1506: Release stored buffer
When the session closes, the current query buffer needs to be freed.
2018-04-10 15:31:51 +03:00
c8d25f293f MXS-1506: Clean up various functions
Reduced variable scopes and removed unused code.
2018-04-10 15:31:51 +03:00
d80f60e0af MXS-1506: Remove delayed_retry_interval
Configuring the parameter current doesn't have enough of an effect to
warrant having a configuration option for it.

Also took mxs::extract_sql into use in the INFO level message.
2018-04-10 15:31:51 +03:00
1ab8f7a4bf MXS-1506: Add delayed_retry parameter
By adding a boolean parameter, the feature can be enabled with sensible
default values. Renamed the query_retry parameters and set the defaults to
acceptable values.

Added new parameters to diagnostic output. Also did some minor renaming.
2018-04-10 15:31:50 +03:00
4786d88413 MXS-1506: Use correct buffer
The GWBUF passed to RWBackend::write must be treated as freed memory if
the write succeeds. This means that the original buffer should be stored,
not the clone.
2018-04-10 15:31:50 +03:00
e1e01c029c MXS-1506: Allow immediate query re-routing
Now that the readwritesplit uses the same mechanism for both
retry_failed_reads and delayed query retries, the re-routing function
should accept a delay of 0 seconds. This makes the mechanism more suitable
for other uses e.g. delaying of queries in filters.
2018-04-10 15:31:50 +03:00
450b31dd8c MXS-1506: Store queries inside RWSplitSession
As the readwritesplit is the only thing that uses the statement storage,
it can be integrated into RWSplitSession. This makes the code a lot
simpler.
2018-04-10 15:31:50 +03:00
53dec5323d Add missing parameters to RWSplit diagnostic output
The new parameters weren't added to the diagnostic output.
2018-04-10 15:31:12 +03:00
926d029373 MXS-1506: Remove explicit pointer use from diagnostics
As the diagnostics functions are a part of the RWSplit class, there's no
need to use the self pointer.
2018-04-10 15:31:12 +03:00
ff5b2c85d6 MXS-1506: Simplify logging in connection creation
The logging in the RWSplit::select_connect_backend_servers was quite
cumbersome and doing the logging when the connection is created makes for
a more information rich output (we know what servers were taken into use
and when).

Changed the log messages to use the names of the servers instead of the
address and port.
2018-04-10 15:31:12 +03:00
d6f98784f7 MXS-1506: Use session_delay_routing with retry_failed_reads
Using the same functionality with the failed read retrying mechanism
removes the need to have multiple versions of the target deduction logic.
2018-04-10 15:31:11 +03:00
34008082e5 Rename to has_session_commands
This way it is similar to other member functions.
2018-04-10 15:30:28 +03:00
951a55ef43 Format readwritesplit source code
Formatted all files with Astyle.
2018-04-10 15:30:27 +03:00
dc338ff3f2 Move causal read reply processing into sub-function
This makes the clientReply a lot easier to comprehend.
2018-04-10 15:29:31 +03:00
465a17d439 MXS-1503: Remove false debug assertion
It is possible that the routing fails even if master_reconnection is
enabled and a second master is available. This is the case when a
transaction is open or autocommit is disabled which is what the
mxs359_master_switch test tests.
2018-04-10 15:29:31 +03:00