This allows the same verbose information to be logged in the cases where
it is of use. Mostly this information can be used to figure out why a
certain session was closed.
By doing the reconnection only when a new query arrives, we prevent the
excessive reconnecting that is done when a server's actual and monitored
states are in conflict.
If the execution of a session command fails on a master, it is retried
again. If the master is not available, the response will be returned from
one of the slaves.
If a client requests an unknown binary protocol prepared statement handle,
a custom error shows the actual ID used instead of the "empty" ID of 0
that the backend sends.
Now considers other routing hints if first one fails. The order is inverted compared
to e.g. namedserver filter settings because of how routing hints are stored. If all hints
are unsuccessful, route to any slave.
Backported the changes that convert the query queue in readwritesplit into
a proper queue. This changes combines both
5e3198f8313b7bb33df386eb35986bfae1db94a3 and
6042a53cb31046b1100743723567906c5d8208e2 into one commit.
This commit adds a new parameter that, when enabled, prunes the session
command history to a known length. This makes it possible to keep a
client-side pooled connection open indefinitely at the cost of making
reconnections theoretically unsafe. In practice the maximum history length
can be set to a value that encompasses a single session using the pooled
connection with no risk to session state integrity. The default history
length of 50 commands is quite likely to be adequate for the majority of
use-cases.
By storing the server statistics object in side the session, the lookup
involved in getting a worker-local value is avoided. Since the lookup is
done multiple times for a single query, it is beneficial to store it in
the session.
As the worker-local value is never deleted, it is safe to store a
reference to it in the session. It is also never updated concurrently so
no atomic operations are necessary.
The code now only checks the need for a keepalive ping once every
keepalive interval. Reduced the number of mxs_clock calls to one so that
all servers use the same value.
By resetting the replay state the transaction replay can start again on a
new server. This allows the replay process work when a master server is
shutting down.
With causal_reads enabled, the query would return with an error if the
slave was not able to catch up to the master fast enough. By automatically
retrying the query on the master, we're guaranteed that a valid result is
always returned to the client.
The readwritesplit transaction management was a large part of the
clientReply function. Moving it into a separate function clarifies the
clientReply function by hiding the comments and details of the transaction
management.
See script directory for method. The script to run in the top level
MaxScale directory is called maxscale-uncrustify.sh, which uses
another script, list-src, from the same directory (so you need to set
your PATH). The uncrustify version was 0.66.
- The ones that were not used were removed.
- The ones that were used were moved close to the actual type.
In most cases some values were missing and if the definition is
close to the type there is a remote chance that they will stay
in sync. If detached, they surely will not.
The configuration doesn't need to be contained in shared pointer as each
session holds its own version of it. This removes most of the overhead in
configuration reloading. The only thing that's left is any overhead added
by the use of thread-local storage.
The transaction migration in the case of a changed master never worked as
transaction replay would only be triggered when the master fails. To cover
this case, the transaction replay just needs to be started when the need
for a transaction migration is detected.
To help diagnose the behavior, the Trx class no longer logs a message when
a transaction is closed. This is now done by readwritesplit which has more
knowledge of the context in which the transaction is closed.
Moved transaction statistics calculations into a member function and
placed all target type specific processing into their respective
functions.
Also inverted the connection keepalive check to also cover hinted queries.
If transaction replaying was enabled and a result was returned in more
than one call to clientReply, a NULL value would be added to the statement
which in turn would trigger a debug assertion.
Similarly any following statements in the transaction would be executed
regardless of whether the result was complete.
Renamed the statement execution function to better describe what it does.
Extended the basic functional test case to cover this.
By using a shared pointer instead of a plain object, we can replace the
router configuration without it affecting existing sessions. This is a
change that is required to enable runtime reconfiguration of
readwritesplit.
If two or more session commands contain identical buffers, the buffer of
the first session command is shared between the others. This reduces the
amount of memory used to store repeated executions of session commands.
The purging of session command history in readwritesplit was replaced with
session command de-duplication. This was done to prevent problems that
could arise when the order of session commands plays a significant role.
When the `optimistic_trx` mode is enabled, all transactions are started on
a slave server. If the client executes a query inside the transaction that
is not of a read-only nature, the transaction is rolled back and replayed
on the master.
Formatted readwritesplit with Astyle. Changed the initialization of
Backend::m_modutil_state to use curly braces to cope with Astyle's lack of
support for curly braces inside parentheses.
The MariaDB implementation allows the last GTID to be tracked with the
`last_gtid` variable. To do this, the configuration option
`session_track_system_variables=last_gtid` must be used or it must be
enabled at runtime.
Session commands that span multiple packets are now allowed and will
work. However, if one is executed the session command history is disabled
as no interface for appending to session commands exists.
The backend protocol modules now also correctly track the current
command. This was a pre-requisite for large session commands as they
needed to be gathered into a single buffer and to do this the current
command had to be accurate.
Updated tests to expect success instead of failure for large prepared
statements.
The resultset processing for MySQL requires some extra work as it lacks
the proper SERVER_MORE_RESULTS_EXIST flag in the last EOF packet. Instead,
the first EOF packet has the SERVER_PS_OUT_PARAMS flag which needs to be
interpreted as a SERVER_MORE_RESULTS_EXIST flag for the second EOF packet.
Also corrected the EOF packet handling to do the flag checks in the code
that deals with the EOF packets.
As the modutil_state parameter is now used for more than large packet
tracking, the correct solution is to store this state object in the
readwritesplit session instead of interpreting it to a boolean value.
Added the `transaction_replay_max_size` parameter that controls the
maximum size of a transaction that can be replayed. If the limit is
exceeded, the stored statements are released thus preventing the
transaction from being replayed.
This limitation prevents accidental misuse of the transaction replaying
system when autocommit is disabled. It also allows the user to control the
amount of memory that MaxScale will use.
As the current query was added to the transaction log before it finished,
the m_current_query contained a duplicate of the latest transaction log
entry. To correctly log only successful transactions, the statement should
be added only after it has successfully completed. This change also
removed the unnecessary cloning that took place when the statement was
added to the log before it finished.
With the fixed transaction logging, the value of m_current_query can be
stashed for later retrying while the replay process is happening. If the
replay completes successfully and the checksums match, the interrupted
query is retried.
Also added a clarifying comment to can_retry_query to explain why a query
inside a transaction cannot be retried.
Added the initial implementation of transaction replay. Transactions are
only replayed if the master fails when no statement is being executed.
The validity of the replayed transaction is done by verifying that the
checksums of the returned results are equal.
Added a close function into the Trx class to make resetting its state
easier. Also changed the return type of the pop_stmt to GWBUF* as the
places where it is used expect a raw GWBUF pointer.
The queries that make up the transaction are now stored in the router
session while the transaction is in progress. For the time being, the
queries are only used to log extra information about the transaction
contents.
Readwritesplit now calculates checksums for all successful and failed
transactions. This checksum is not of any practical use until the
transaction replaying is implemented.
