The process of appending to a std::string always includes a size check in
case the internal storage needs to expand. Given that we know a
canonicalized version of a query string is never larger than the original
string and that we pre-allocate enough memory to cope with the worst-case
scenario, the extra logic in std::string::push_back is unnecessary and an
extra cost. Writing directly into the string avoids this cost and improves
the performance.
Switched from default character type functions to ones that use lookup
tables. Eliminated the internal state and replaced with in-place iteration
of the query. Added code to allow single-lookup detection of normal
characters.
Since we know the worst-case size of a canonical statement is the size of
the query string, we can reduce the number of memory allocations to one in
the get_canonical function.
Replaced SPINLOCK with std::mutex where possible, leaving out the more
complex cases. The big offenders remaining are the binlogrouter and the
gateway.cc OpenSSL locks.
See script directory for method. The script to run in the top level
MaxScale directory is called maxscale-uncrustify.sh, which uses
another script, list-src, from the same directory (so you need to set
your PATH). The uncrustify version was 0.66.
The resultset processing for MySQL requires some extra work as it lacks
the proper SERVER_MORE_RESULTS_EXIST flag in the last EOF packet. Instead,
the first EOF packet has the SERVER_PS_OUT_PARAMS flag which needs to be
interpreted as a SERVER_MORE_RESULTS_EXIST flag for the second EOF packet.
Also corrected the EOF packet handling to do the flag checks in the code
that deals with the EOF packets.
As the modutil_state parameter is now used for more than large packet
tracking, the correct solution is to store this state object in the
readwritesplit session instead of interpreting it to a boolean value.
Readwritesplit would not handle multiple overlapping COM_STMT_EXECUTE
commands properly if they opened cursors. This was due to the fact that
the result would not be marked as complete and COM_STMT_FETCH commands
were executed as if they did not return results.
The correct implementation is to consider a COM_STMT_EXECUTE that opens a
cursor complete only when the first EOF packet is read (that is, when the
resultset header is read). This allows subsequent COM_STMT_FETCH commands
to be handled separately.
The separate COM_STMT_FETCH handling must count the number of packets that
are being fetched. This allows correct tracking of the state of a
COM_STMT_FETCH by checking that the number of packets is correct or the
second EOF/ERR packet is read.
The canonicalization process now strips non-executable comments from the
SQL and replaces all constants in executable comments.
Enabled the comment test and updated expected output of the select and
alter tests.
Updated tests with new expected output. Also took new function into use
and removed the old one.
Since the comment removal isn't added yet, one of the tests is expected to
fail and it is temporarily disabled.
The new canonicalization function now processes negative numbers
correctly. It uses a look-behind operation to detect whether the operation
is a negation of a number or a subtraction from another value.
The function uses a hand-written parser that picks out the values and
replaces them with question marks. This function is not yet able to
process negative values; a `-?` token will be generated for all negative
numbers.
The new function will canonicalize the query in such a way that string and
number constants cannot be distinguished which means that it won't pass
the current test cases.
When a BEGIN statement is prepared using the binary protocol, it returns a
single OK packet. Due to a bug in the code that deals with multi-statement
results and EOF packets, the response was never sent to the client.
Also added back the error messages of failed session commands to the INFO
level. This way it's still possible to see why a session command fails but
the log isn't flooded by them in normal usage.
When binary data was processed, it was possible that the values were
misinterpreted as OK packets which caused debug assertions to trigger.
In addition to this, readwritesplit did not handle the case when all
packets were routed individually.
If multiple queries that only generate OK packets were executed, the
result returned by the server would consist of a chain of OK packets. This
special case needs to be handled by the modutil_count_signal_packets.
The current implementation is very ugly as it simulates a result with at
least one resultset in it. A better implementation would hide it behind a
simple boolean return value and an internal state object.
A multi-statements can return multiple resultsets in one response. To
accommodate for this, both the readwritesplit and modutil code must be
altered.
By ignoring complete resultsets in readwritesplit, the code can deduce
whether a result is complete or not.
The original offset needs to be separately tracked to assert that an OK
packet is not the first packet in the buffer. The functional offset into
the buffer is modified to reduce the need to iterate over buffers that
have already been processed.
Returning the results of a query as a set of packets is currently more
efficient. This is mainly due to the fact that each individual packet for
single packet routing is allocated from the heap which causes a
significant loss in performance.
Took the new capability into use in readwritesplit and modified the
reply_is_complete function to work with non-contiguous results.
The enums exposed by the connector are not intended to be used by the
users of the library. The fact that the protocol, and other, modules used
it was in violation of how the library is intended to be used.
Adding an internal mapping into MaxScale also removes some of the
dependencies that the core has on the connector.
Cleaned up the MaxScale version of the mysql.h header by removing all
unused includes. This revealed a large amount of dependencies on these
removed includes in other files which needed to be fixed.
Also sorted all includes in changed files by type and alphabetical
order. Removed explicit revision history from modified files.
The EOF packet calculation function in modutil.cc didn't handle the case
where the payload exceeded maximum packet size and could mistake binary
data for a ERR packet.
The state of a multi-packet payload is now exposed by the
modutil_count_signal_packets function. This allows proper handling of
large multi-packet payloads.
Added minor improvements to mxs1110_16mb to handle testing of this change.
If a prepared statement response was collected into one buffer, it doesn't
need to be processed again. By jumping directly to the routing of the
collected result, we prevent the unnecessary splitting of buffers that
appears to happend with continuous preparations of prepared statements.
