The protocol could leak memory in rare cases where several commands were
queued at the same time. Readwritesplit also didn't free the memory it
acquired via qc_get_table_names.
When a valid target was not found, no error message was logged by the
router. This would cause the "Routing the query failed. Session will be
closed." message to be logged with no explanation as to why the routing
failed.
In addition to the above-mentioned case, no message would be logged if the
target for a COM_STMT_FETCH was not in use.
If a session command produces a different result on the slave than it did
on the master, a warning is logged. This warning now also logs the query
that was being executed to make investigation of the problem easier.
Backend::execute_session_command would use the overridden write method
instead of the Backend::write method that it intended to use. This caused
session commands that did not expect a response to be in a state that
expected a result.
Also fixed RWBackend::write pass the response_type value to
Backend::write.
If the server sends a server shutdown error, it is safe for readwritesplit
to ignore it. When the TCP connection is closed, the router error handling
will discard the connection, optionally replacing it.
If a server responds when no response was expected, dump stored
statements. This should help deduce root causes of problems relating to
unexpected responses.
The two operations return different types of results and need to be
treated differently in order for them to be handled correctly in 2.2.
This fixes the unexpected internal state errors that happened in all 2.2
versions due to a wrong assumption made by readwritesplit. This fix is not
necessary for newer versions as the LOAD DATA LOCAL INFILE processing is
done with a simpler, and more robust, method.
To work around the limitation in the session command handling and
multi-part results, all session commands are now treated as gathered
results. This allows session commands which return result sets to be used
with MaxScale.
This change should not cause problems with practical workloads as they
usually do not return massive resultsets for session commands.
The optimal way to handle the multi-part responses would be to integrate
it into the result completion tracking process. This would allow the
prepared statement IDs to be extracted while the command is being
processed.
The individual servers were missing a statistic that would give an
estimated query count. As there is no simple way to count queries for all
modules, counting the number of routed protocol packets is a suitable
substitute.
The resultset processing for MySQL requires some extra work as it lacks
the proper SERVER_MORE_RESULTS_EXIST flag in the last EOF packet. Instead,
the first EOF packet has the SERVER_PS_OUT_PARAMS flag which needs to be
interpreted as a SERVER_MORE_RESULTS_EXIST flag for the second EOF packet.
Also corrected the EOF packet handling to do the flag checks in the code
that deals with the EOF packets.
As the modutil_state parameter is now used for more than large packet
tracking, the correct solution is to store this state object in the
readwritesplit session instead of interpreting it to a boolean value.
Large session commands weren't properly handled which caused the router to
think that the trailing end of a multi-packet query was actually a new
query.
This cannot be confidently solved in 2.2 which is why the router session
is now closed the moment a large session command is noticed.
Only commands that can contain an SQL statements should be stored for
retrying (COM_QUERY and COM_EXECUTE). Other commands are either session
commands or do not work with query retrying.
The commands needs to be handled separately from the rest of the result
types.
Added a test case that reproduces the problem and verifies that the change
in code fixes it.
When a LOAD DATA LOCAL INFILE finishes, the client sends an empty
packet. The second case when the client sends an empty packet when the
previous packet was exactly 0xffffff bytes long. These two packets were
confused which caused the internal state to temporarily flip from inactive
to ending and back to inactive.
The aforementioned flip-flopping didn't have any practical differences but
it was caught by a debug assertion.
The COM_STMT_FETCH command will create a response. This was a
readwritesplit-specific interpretation of the command and it was wrong.
Also record the currently executed command event for session commands.
Readwritesplit would not handle multiple overlapping COM_STMT_EXECUTE
commands properly if they opened cursors. This was due to the fact that
the result would not be marked as complete and COM_STMT_FETCH commands
were executed as if they did not return results.
The correct implementation is to consider a COM_STMT_EXECUTE that opens a
cursor complete only when the first EOF packet is read (that is, when the
resultset header is read). This allows subsequent COM_STMT_FETCH commands
to be handled separately.
The separate COM_STMT_FETCH handling must count the number of packets that
are being fetched. This allows correct tracking of the state of a
COM_STMT_FETCH by checking that the number of packets is correct or the
second EOF/ERR packet is read.
When a LOAD DATA LOCAL INFILE is actively rejected by the server, the
server sends an error to the client. This error was not detected and the
router was stuck in the special mode that handles LOAD DATA LOCAL INFILE.
When a multi-statement query consisting completely of UPDATE statements is
received, the packets can be received in two separate buffers. To cope
with this situation, the state change into REPLY_STATE_RSET_COLDEF must
only be done if the buffer contains more than a single packet.
When readwritesplit receives a reply from a backend, an info level log
message is now logged. This allows easier debugging of situations where
replies aren't properly returned by the router.
After a temporary table is created, readwritesplit will check whether a
query drops or targets that temporary table. The check for query type was
missing from the table dropping part of the code. The temporary table read
part was checking that the query is a text form query.
Added a debug assertion to the query parsing function in qc_sqlite to
catch this type of interface misuse.
When a BEGIN statement is prepared using the binary protocol, it returns a
single OK packet. Due to a bug in the code that deals with multi-statement
results and EOF packets, the response was never sent to the client.
Also added back the error messages of failed session commands to the INFO
level. This way it's still possible to see why a session command fails but
the log isn't flooded by them in normal usage.
The responses of slaves that arrived before the master were always
compared to the empty value of 0x00. If the slave connection replied after
the master, the comparison was correct.
This commit introduces a map of slaves and their responses that
are handled once the master's response arrives.
Removed false error message about failed session commands. An error in
response to a session command is a perfectly valid result.
Also added the explicit commands that the master and slave return to the
warning that is logged when the results differ.
The debug assertion wasn't well placed as it is perfectly possible that a
master connnection exists but it is not in use. This can be further
checked by asserting that the master is indeed closed and not in use.
Moved the original debug assertion into a separate branch that should
catch any errors in the routing logic.
As chages to the transaction state are detected by the protocol level
mini-parser, there's no need to fully classify queries inside read-only
transactions. This should be a good performance boost for loads that
heavily use read-only transactions.
Make all modules lowercase and make module loading case
insensitive. Further, make command invocation case insensitive,
as far as the module name is conserned.
The default value for strict_multi_stmt prevents compound statements and
atomic multi-statement commands from being executed without completely
disabling load balancing. As the new default value will have no practical
effect on all correct uses of readwritesplit, this is a relatively safe
thing to change.
When binary data was processed, it was possible that the values were
misinterpreted as OK packets which caused debug assertions to trigger.
In addition to this, readwritesplit did not handle the case when all
packets were routed individually.
A multi-statements can return multiple resultsets in one response. To
accommodate for this, both the readwritesplit and modutil code must be
altered.
By ignoring complete resultsets in readwritesplit, the code can deduce
whether a result is complete or not.
When LEAST_BEHIND_MASTER routing criteria was used, the info level logging
function would fall through to the default case. In debug builds, this
would trigger a debug assertion.
Returning the results of a query as a set of packets is currently more
efficient. This is mainly due to the fact that each individual packet for
single packet routing is allocated from the heap which causes a
significant loss in performance.
Took the new capability into use in readwritesplit and modified the
reply_is_complete function to work with non-contiguous results.
