The master down verification through slaves won't work with this commit. It needs to be
redesigned to handle multiple slave connections or removed. Also, only the first row of
slave status data is used by the monitor, so multiple slave connections are still
incorrectly handled.
The possibility to have multiple cache rules in a cache
configuration file is now handled throughout the cache
filter.
The major difference is that while you earlier directly
queried the Cache whether data should be stored to the
cache and whether data in the cache should be used, you
now query the Cache whether data should be stored to the
cache and, if so, get a CacheRules object from which you
subsequently query whether data from the cache should
be used.
It's now possible to have a rules file with an array of rule
objects, e.g.
[
{
store: [ ... ],
use: [ ... ]
},
{
store: [ ... ],
use: [ ... ]
}
]
This commit only contains the low-level modifications for
supporting that; the upper-level modifications are made in
another commit.
The resultset processing for MySQL requires some extra work as it lacks
the proper SERVER_MORE_RESULTS_EXIST flag in the last EOF packet. Instead,
the first EOF packet has the SERVER_PS_OUT_PARAMS flag which needs to be
interpreted as a SERVER_MORE_RESULTS_EXIST flag for the second EOF packet.
Also corrected the EOF packet handling to do the flag checks in the code
that deals with the EOF packets.
As the modutil_state parameter is now used for more than large packet
tracking, the correct solution is to store this state object in the
readwritesplit session instead of interpreting it to a boolean value.
Added the `transaction_replay_max_size` parameter that controls the
maximum size of a transaction that can be replayed. If the limit is
exceeded, the stored statements are released thus preventing the
transaction from being replayed.
This limitation prevents accidental misuse of the transaction replaying
system when autocommit is disabled. It also allows the user to control the
amount of memory that MaxScale will use.
The transaction retrying behavior is now configurable and documented. The
`transaction_replay` parameter implicitly enables the required
functionality in the router that it needs.
As the current query was added to the transaction log before it finished,
the m_current_query contained a duplicate of the latest transaction log
entry. To correctly log only successful transactions, the statement should
be added only after it has successfully completed. This change also
removed the unnecessary cloning that took place when the statement was
added to the log before it finished.
With the fixed transaction logging, the value of m_current_query can be
stashed for later retrying while the replay process is happening. If the
replay completes successfully and the checksums match, the interrupted
query is retried.
Also added a clarifying comment to can_retry_query to explain why a query
inside a transaction cannot be retried.
Added the initial implementation of transaction replay. Transactions are
only replayed if the master fails when no statement is being executed.
The validity of the replayed transaction is done by verifying that the
checksums of the returned results are equal.
Added a close function into the Trx class to make resetting its state
easier. Also changed the return type of the pop_stmt to GWBUF* as the
places where it is used expect a raw GWBUF pointer.
The queries that make up the transaction are now stored in the router
session while the transaction is in progress. For the time being, the
queries are only used to log extra information about the transaction
contents.
Readwritesplit now calculates checksums for all successful and failed
transactions. This checksum is not of any practical use until the
transaction replaying is implemented.
The Checksum class defines an interface which the SHA1Checksum and
CRC32Checksum implement.
Added test unit test cases to verify that the checksums work and perform
as expected.
5.1 to 5.3 are officially not supported anymore, so support can be removed from
the monitor. This allows removing the config parameter "mysql51_replication".
In this case, the server was already a slave and is not being demoted. Also, the file may
contain queries which cannot be ran while a slave connection is running.
Fixed string truncation warnings by reducing max parameter lengths by one
where applicable. The binlogrouter filename lengths are slightly different
so using memcpy to work around the warnings is an adequate "solution"
until the root of the problem is solved.
Removed unnecessary CMake policy settings from qc_sqlite. Adding a
self-dependency on the source file of an external project has no effect
and only caused warnings to be logged.
When providing pointer to instance and pointer to member function
of the class of the instance, the pointer to the member function
should be first and the pointer to the instance second.
There's now double bookkeeping:
- All delayed calls are in a map whose key is the next
invocation time. Since it's a map (and not an unordered_map)
it's sorted just the way we want to have it.
- In addition, there's an unordered set for each tag.
With this arrangement we can easily invoke the delayed calls
in the right order and be able to efficiently remove all
delayed calls related to a particular tag.
When canceling, a DelayedCall instance must be removed from the
collection holding all delayed calls. Consequently priority_queue
cannot be used as it 1) does not provide access to the underlying
collection and 2) the underlying collection (vector or deque)
is a bad choise if items in the middle needs to be removed.
The interface for canceling calls is now geared towards the needs
of sessions. Basically the idea is as follows:
class MyFilterSession : public maxscale::FilterSession
{
...
int MyFilterSession::routeQuery(GWBUF* pPacket)
{
...
if (needs_to_be_delayed())
{
Worker* pWorker = Worker::current();
void* pTag = this;
pWorker->delayed_call(5000, pTag, this,
&MyFilterSession::delayed_routeQuery,
pPacket);
return 1;
}
...
}
bool MyFilterSession::delayed_routeQuery(Worker::Call:action_t action,
GWBUF* pPacket)
{
if (action == Worker::Call::EXECUTE)
{
routeQuery(pPacket);
}
else
{
ss_dassert(action == Worker::Call::CANCEL);
gwbuf_free(pPacket);
}
return false;
}
~MyFilterSession()
{
void* pTag = this;
Worker::current()->cancel_delayed_calls(pTag);
}
}
The alternative, returning some key that the caller must keep
around seems more cumbersome for the general case.
It's now possible to provide Worker with a function to call
at a later time. It's possible to provide a function or a
member function (with the object), taking zero or one argument
of any kind. The argument must be copyable.
There's currently no way to cancel a call, which must be added
as typically the delayed calling is associated with a session
and if the session is closed before the delayed call is made,
bad things are likely to happen.
Worker::Timer class and Worker::DelegatingTimer templates are
timers built on top of timerfd_create(2). As such they consume
descriptor and hence cannot be created independently for each
timer need.
Each Worker has now a private timer member variable on top of
which a general timer mechanism will be provided.
The maximum number of workers and routing workers are now
hardwired to 128 and 100, respectively. It is still so that
all workers must be created at startup and destroyed at
shutdown, creating/destorying workers at runtime is not
possible.
