All whitespace in object names is now converted to a compacted format with
whitespace replaced with hyphens. If a conversion takes place, a warning
is logged.
Converted some of the tests into C++ as they directly include .cc files.
Monitors can now be printed in JSON format. The REST API resource
`/monitors` accepts GET requests and returns a JSON representation of the
monitors as a response.
The functions don't modify the filters when the JSON is generated. This
means that the parameters can be const.
Minor formatting fixes to service sources.
Both the filters and services can be queried via the REST API now that
these resources can be expressed in JSON format.
As with the other resources, these directly call the functions that
generate the data. This will be done via the inter-thread messaging system
once it's in place.
The REST API now prints individual sessions and servers. It also lists all
servers if no specific server is given.
The functions directly call the printing functions when they should be
using the inter-thread messaging system. When the messaging system is
ready, these functions should be updated.
The admin thread now uses blocking IO. This is not optimal but it
simplifies the code by some amount.
Fixed option processing removing one extra character from key name.
Use correct member variable when checking for the option map end.
Preparation for adding KILL syntax support.
Session id changed to uint32 everywhere. Added atomic op.
Session id can be acquired before session_alloc().
Added session_alloc_with_id(), which is given a session id number.
Worker object has a session_id->SESSION* mapping, not used yet.
Adds a server-specific parameter, "use_proxy_protocol". If enabled,
a header string is sent to the backend when a routing session connection
changes state to MXS_AUTH_STATE_CONNECTED. The string contains the real
client IP and port.
The function was no longer thread-safe as it used the obsolete per-thread
spinlocks to iterate over the DCBs. Now the function uses the newly added
WorkerTask class to iterate over them.
Since the new WorkerTask mechanism is far superion to dcb_foreach, the
latter is now deprecated.
The backend MySQL protocol can now collect prepared statement preparation
responses as well as result sets. This removes the need to parse and
collect the preparation responses at the router level.
The functions used to track the resultset EOF packets now expose the
position of the end of the result set. This allows the modules that use
them to check if more results exist in the same buffer.
Added the status bits for OK and EOF packets to the mysql.h protocol
header. This can be used to check for various state changes that happen in
the session. Currently the status bits are only used to detect if more
results are expected.
When batched queries are done through readwritesplit, it will now handle
them one by one. This allows batched queries to be used with
readwritesplit but it does impose a performance penalty when compared to
direct execution on the backend.
Now the statistics is in a single structure and the property of the
Worker instance in question. Methods are provided for obtaining the
statistics of all workers in one go.
Showing dcb addresses, the number of fds and the events is
meaningless as the information is completely transient and
is likely to have changed the moment is was displayed.
Just like the thread stats and poll stats earlier, the queue stats
are now moved to worker.
A litte refactoring still, and the polling will only work on local
data.
Each worker now has a separate structure for collecting the
polling statistics that is passed to epoll_waitevents(). When
the stats are asked for, we loop over all separate stats and
combine them. So, instead of having every statistics of each
thread one cacheline apart, each thread has all its statistics
in one lump that, for obvious reasons, are going to be apart.
The primary purpose of this excersize is to remove the hardwired
nature of the statistics collection. For instance, the admin
thread will be doing I/O but that I/O should not be included
in the statistics of the workers.
The Worker no longer creates a pipe and implements the cross
worker/thread message mechanism itself. Instead it has a
MessageQueue instance variable for that purpose.
MessageQueue encapsulates a message queue built on top of a
pipe. The message queue needs a handler for receiving messages
and must be added to a worker for pumping messages through the
pipe.
Each Worker will have an instance of MessageQueue.
This is the first step in turning the worker mechanism and everything
around it into a set of C++ classes. In this change, the original C
API is still present, but in subsequent changes that will be removed.
This is not globally safe yet, but all other access is directly or
indirectly related to maxadmin, which is irrelevant as far as
performance testing is concerned.
Now possible to send a function and arguments to a specific worker
thread for execution.
In particular, this will be used for transferring the injection of
fake hangup events into DCBs, related to a particular server, from
the monitor thread to the worker threads, thus removing the need
for locks.
The shutdown is now performed so that a shutdown message is
sent to all workers. When the workers receive that message, they
turn on a shutdown flag, which subsequently is checked in the poll
loop.
MXS_WORKER is an abstraction of a worker aka worker thread.
It has a pipe whose read descriptor is added to the worker/thread
specific poll set and a write descriptor used for sending messages
to the worker.
The worker exposes a function mxs_worker_post_message using which
messages can be sent to the worker. These messages can be sent from
any thread but will be delivered on the thread dedicated for the
worker.
To illustrate how it works, maxadmin has been provided with a new
command "ping workers" that sends a message to every worker, which
then logs a message to the log.
Additional refactoring are needed, since there currently are overlaps
and undesirable interactions between the poll mechanism, the thread
mechanism and the worker mechanism.
This is visible currently, for instance, by it not being possible to
shut down MaxScale. The reason is that the workers should be shut down
first, then the poll mechanism and finally the threads. The shutdown
need to be arranged so that a shutdown message is sent to the workers
who then cause the polling loop to exit, which will cause the threads
to exit.
That can be arranged cleanly by making poll_waitevents() a "method"
of the worker, which implies that the poll set becomes a "member
variable" of the worker.
To be continued.
The whole worker thread mechanism assumes EPOLLET and non-blocking
descriptors, so that should be the default.
TODO: In debug mode, check that the provided file descriptor indeed
is non-blocking.
The handler callback should now return a bitmask with bits set
according to what it did when it was called. That way the actual
statistics gathering can be done in poll_waitevents() and the
handler need not be aware of any thread structs.
Actually, the only thing that needs any assistance is accept handling,
because in poll_waitevents() we do not know whether a READ event
relates to a listening or a normal socket, that is, should the
event be counted as an accept or as a read.
