Services can now be destroyed if they have no active listeners and they
are not linked to servers. When these conditions are met, the service will
be destroyed when the last session for the service is closed.
The closing of a service will close all listeners that were once assigned
to the service. This allows closing of the ports at runtime which
previously was done only on shutdown.
Exposed the command through the REST API but not through MaxAdmin as it is
deprecated.
When the query queue does not contain a complete packet
(i.e. modutil_get_next_MySQL_packet return NULL), an informative dump of
how many bytes and what is stored is logged.
By aborting the process if memory runs out when a buffer needs to be made
contiguous, we rule out other, more subtle, errors. Failing as soon as a
possible when memory allocation fails gives better error messages.
The LocalClient micro-client required a reference to the session that was
valid at construction time. This is the reason why the previous
implementation used dcb_foreach to first gather the targets and then
execute queries on them. By replacing this reference with pointers to the
raw data it requires, we lift the requirement of the orignating session
being alive at construction time.
Now that the LocalClient no longer holds a reference to the session, the
killing of the connection does not have to be done on the same thread that
started the process. This prevents the deadlock that occurred when
concurrect dcb_foreach calls were made.
Replaced the unused dcb_foreach_parallel with a version of dcb_foreach
that allows iteration of DCBs local to this worker. The dcb_foreach_local
is the basis upon which all DCB access outside of administrative tasks
should be built on.
This change will introduce a regression in functionality: The client will
no longer receive an error if no connections match the KILL query
criteria. This is done to avoid having to synchronize the workers after
they have performed the killing of their own connections.
The dcb_foreach function is not safe to use from multiple threads at the
same time. This should be asserted by checking that the function is called
only from the main worker.
The addition of this assertion also implies that only administrative
operations should use the dcb_foreach function. To accommodate this
change, the KILL command iteration needs to be adjusted.
MariaDBMonitor diagnostics printing is unsafe as some of the read
fields are arrays. To be on the safe side, the fields are now read
in the monitor worker thread.
Since diagnostics must work even for stopped monitors, a worker task
is used. In practice, it usually runs when the monitor is sleeping.
With the removal of the old session command implementation, the code that
used it can be removed or replaced with newer constructs. As a result, the
backend protocol no longer does any session command processing.
The three buffer types, GWBUF_TYPE_SESCMD_RESPONSE,
GWBUF_TYPE_RESPONSE_END and GWBUF_TYPE_SESCMD as well as their related
macros are no longer used and can be removed.
The test cases allocated servers in a way that doesn't comfortably suit
the way the servers are now allocated. Adding a helper C++ class to load
module defaults makes it easier to do explicit server initialization in
tests.
The binlogrouter was also fixed in this commit as it uses servers much
like a test would use.
The configuration system that modules use allows the SSL parameter
validation to be simplified. It should also provide more consistent error
messages for similar types of errors.
The SSL_LISTENER initialization is now done in one step. There was no good
reason to do it in two separate steps for listeners but in one step for
servers.
The `ssl` parameter now also accepts boolean values. As the parameter
behaves like a boolean and looks like a boolean, it ought to be a
boolean. It still accepts the custom `required` and `disabled` values
simply for backwards compatibility.
Also added the missing freeing functions for the SSL_LISTENER type. This
prevents failed SSL_LISTENER creations from leaking memory.
The same mechanism that is used for modules can be used for the
configuration of the core objects. This removes the need for the redundant
code that validates various values that is already present in the code
that modules use.
Relaced router_options with configuration parameters in the createInstance
router entry point. The same needs to be done for the filter API as barely
any filters use the feature.
Some routers (binlogrouter) still support router_options but using it is
deprecated. This had to be done as their use wasn't deprecated in 2.2.
The mxs::strtok function splits a string into a vector of strings based on
a set of delimiters. The function makes it easier to iterate over a set of
values given as a delimited string.
Added a new router API entry point that allows configuration changes after
the instance has been created. This makes alterations to most service
parameters at runtime possible.
An option to reconfiguration would have been the creation of a new
service and the eventual destruction of the old one. This would be a more
complicated and costly method but from an architectural point of view it
is interesting.
The actual implementation of the configuration change is left to the
router. Currently, only readwritesplit performs reconfiguration as
implementing it with versioned configurations is very easy.
Versioned configurations can be considered an adequate first step but it
is not an optimal solution as it causes a bottleneck in the reference
counting of the shared configuration. Thread-specific configuration
definitions would make for a more efficient solution but the
implementation is more complex.
With the global configuration parameter 'query_classifier_cache'
the query classification cache can be turned on. At the moment it
does not matter what value it has; its presence simply enables the
caching.
Eventually you will be able to specify how much memory the cache
is allowed to consume.
Now takes a structure that, if present, enables the query
classification caching and specifies the properties of the
cache.
For the time being no actual properties are yet available.
The mapping from a canonical statement to the query classification
result is maintained by the class QCInfoCache of which there exist
an instance per thread. That way no locking is needed but the
information will be cached multiple times (but that is a smaller
price to pay). Currently the information is stored in a regular
std::unordered_map, which means that the consumed amount of
memory will just keep on growing unless the number of canonical
statements used by clients happens to have an upper bound.
The LRU cache (that provides means for putting a bound on the
amount of memory used and number of items) used in the cache filter
will be generalized and be taken into use here as well.
The key is now the canonical statement itself, which means that
a fair amount of memory will be used. To preserve memory it might
make sense to use a hashed value instead, although that at least
in principle opens up the possibility for unintended collisions.
This feature will also be made configurable.
The query classifier stores information about the statement carried
by a GWBUF in the GWBUF itself. We need to be able to store that
object out side the lifetime of the GWBUF. So, we require that a
query classifier is capable of duplicating references to that object.
The evq_length file held the returned number of descriptors from
the last epoll_wait() call. As such it is highly temporal and not
particularly meaningful.
That has now been removed and the instead the average number of
returned descriptors is maintained. That information changes slowly
and thus carries some meaning.
If two or more session commands contain identical buffers, the buffer of
the first session command is shared between the others. This reduces the
amount of memory used to store repeated executions of session commands.
The purging of session command history in readwritesplit was replaced with
session command de-duplication. This was done to prevent problems that
could arise when the order of session commands plays a significant role.
When the `optimistic_trx` mode is enabled, all transactions are started on
a slave server. If the client executes a query inside the transaction that
is not of a read-only nature, the transaction is rolled back and replayed
on the master.
The characteristics of a transaction can now be tracked by the query
classifier. This allows read-only and read-write transaction statistics to
be calculated.
The id has now been moved from mxs::Worker to mxs::RoutingWorker
and the implications are felt in many places.
The primary need for the id was to be able to access worker specfic
data, maintained outside of a routing worker, when given a worker
(the id is used to index into an array). Slightly related to that
was the need to be able to iterate over all workers. That obviously
implies some kind of collection.
That causes all sorts of issues if there is a need for being able
to create and destroy a worker at runtime. With the id removed from
mxs::Worker all those issues are gone, and its perfectly ok to create
and destory mxs::Workers as needed.
Further, while there is a need to broadcast a particular message to
all _routing_ workers, it hardly makes sense to broadcast a particular
message too _all_ workers. Consequently, only routing workers are kept
in a collection and all static member functions dealing with all
workers (e.g. broadcast) have now been moved to mxs::RoutingWorker.
Now, instead of passing the id around we instead deal directly
with the worker pointer. Later the data in all those external arrays
will be moved into mxs::[Worker|RoutingWorker] so that worker related
data is maintained in exactly one place.
To get rid of the need that a Worker must have an id, we store
in the MXS_POLL_DATA structure a pointer to the owning worker
instead of the id of the owning worker. This also allows some
further cleanup as the need for switching back and forth between
the id and the worker disappears.
The id will be moved from Worker to RoutingWorker as there
currently is a fair amount of code that assumes that the id of
routing workers start from 0.
It's no longer necessary to inherit from Worker in order to use
it, but it can now be used in a stand-alone fashion. This fits
the MonitorInstance use-case better.
The function implemented redundant functionality and replacement with
modutil_get_next_MySQL_packet was planned.
When faced with a packet header spread over multiple buffers, the packet
length calculation would read past the buffer end. This is fixed by taking
modutil_get_next_MySQL_packet into use.
Identical behavior to the old function is achieved by calling
gwbuf_make_contiguous for each packet to store them in a contiguous area
of memory. This should be either removed and only done when
RCAP_TYPE_CONTIGUOUS_INPUT is requested or be made an innate feature of
statement based routing.
The state of the backend needs to be checked before any pending session
commands are executed on it.
Added debug assertions to catch invalid use of the status functions of
closed backends.
