The service, filter and monitor resources now have a "parameters" value
which contains a set of all configuration parameters for that object. This
set contains both standard and non-standard parameters.
Also fixed a mistake in the constant name definitions for the monitor
parameters "events" and "script".
A self link to the resource itself provides a convenient way for the
client to request a resource, modify it and call the self link to update
it. This removes some of the burden on the client to keep track of the
resource links by placing these in the resource itself.
The server can now be modified with a PUT request of a modified server
resource. The server resource was reorganized to have the parameters as a
separate entity from the other more general entities of the resource.
The PUT/POST functions return a more appropriate error message when no
request body is provided.
Moved some of the constant names used in server.cc into the config.h
header.
The actual list of headers is not known when the request is first
generated. This prevents the headers from being generated in admin.cc
which handles things on a lower level.
The moving of the header generation is done with the OPTIONS method in
mind. This header needs to be generated inside the RootResource class
which manages the navigation of the resources.
Cleaned up various parts of the resource, request and response class
headers.
Moved `using` declarations into .cc files.
Made the Resource class non-copyable as it isn't really meant to be
copied.
The data was not processed correctly and instead an error was sent to the
client as soon as the request arrived.
Created a class that somewhat abstracts the internals of the client
request processing.
The resources are now declared by simply typing the path that they match
and providing a callback for that path. This makes it easier to add new
resources without having to create a new class for each added resource
endpoint. It also removes some of the pathname processing that was done
inside the child class handler functions.
The standard response headers are now generated at a higher level. This
reduces the scope of the HttpResponse class making it a leaner wrapper
around a few simple variables, namely the JSON body of the response.
The HttpRequest now exposes the Host header that the client sent. This
allows resource relations to be real links that work without modification.
The HTTP side of the REST API is better handled with an actual
library. The libmicrohttpd library provides a convenient way of handling
the HTTP traffic between the clients and MaxScale.
The admin thread now uses blocking IO. This is not optimal but it
simplifies the code by some amount.
Fixed option processing removing one extra character from key name.
Use correct member variable when checking for the option map end.
The resource handler system is now usable but it doesn't perform anything
useful. Although, this will allows it to be tested for correctness.
Minor fixes to HttpResponse output and renaming of functions.
The Resource class is intended to be an abstraction of a resource
tree. Each node in the tree can perform actions. The tree is traversed
depth first so that deeper command paths resolve to the correct nodes.
Currently all the base resources defined in the REST API documents are
implemented in a way that they return a 200 OK response to all
requests. When the internal data can be represented as JSON, the resources
can be hooked up to functions that generate JSON.
When a client requests a resource, the HttpRequest class now splits the
requested resource into parts. This should help with the resource
validation and navigation.
Added test that checks that the resources are correctly split into the
correct number of arguments and that the argument contents are correct.
The options of a request are now parsed and exposed by the HttpRequest
class.
Added tests for the request options.
Also added missing error handling of invalid requests.
The class now generates default headers. The ETag and Last-Modified tags
do not represent any actual modification time or resource hash.
The basic functionality of the HTTP responses is tested by the core test
suite. More advanced testing of the whole REST API is still required.
Removed the static `create` functions as only the JSON parsing version
could generated errors and even then the errors were unlikely. By
replacing the static creator function with a normal constructor, the
HttpResponse class can now also be created on the stack making its use
easier.
The HTTP response class simplifies the response creation. The next step is
to add generation of all the default headers that are needed by the REST
API.
The HTTP request body is expected to be a valid JSON object. All other
requests are considered malformed requests and result in a HTTP 400 error.
Added the Jansson license to the LICENSE-THIRDPARTY.TXT file. Imported
some of the tests from the Jansson test suite to the HttpParser test.
The HttpParser class was renamed to HttpRequest as it parses and processes
only HTTP requests. A second class that creates a HTTP response needs to
be created to handle the response generation.
Moved some of the HTTP constants and helper functions to a separate
http.hh header.
The admin requests are now processed in blocking mode. The timing out of
connecttions is handled by a specific timeout thread that checks the state
of each admin request.
The simplification will help with the JSON parsing with PUT/POST
commands. If non-blocking IO is used, the network reading code and JSON
parsing needs a lot more work to handle partial reads.
If the administrative interface requires higher performance and
concurrency, a multi-threaded solution could be created.
The HTTP parser parses HTTP/1.1 messages into easily manageable data
structures. This should make it easier to map the HTTP requests into
actual commands in MaxScale.
When MaxScale is started, a separate thread for the administrative
interface is started. This allows the worker threads to handle client
requests while the administrative thread handles the lower priority
administrative requests.
The administrative interface responds to all request with a 200 OK HTTP
response. This allows the administrative interface itself to be tested.
The old polling message system is obsolete now that the worker messages
are implemented. The old system was only used to clean up the persistent
connection pool of a server.
It is now possible to define whether tasks are executed immediately or put
on the event queue of the worker thread. If task execution is in automatic
mode and the current executing thread is a worker thread, the
Task->execute method is called immediately.
This allows tasks to be posted from within worker threads. This is
intended to be used when purging of stale persistent connections and
printing diagnostic output via MaxAdmin. All of these actions are done
from within a worker thread.
- Posting a task to a worker for execution (without implicit wait)
is called "post".
- Posting a task to every worker for execution (without implicit wait)
is called "broadcast".
In these cases the task must be provided as a pointer or auto_ptr, to
indicate that the provided pointer must remain alive for longer than
the duration of the function call.
- Posting a task to a worker for execution *and* waiting for all workers
to have executed the task is called "execute" and the two variants are
now called "execute_concurrently" and "execute_serially".
In these cases the task is provided as a reference, since the functions
will return only when all workers have (in concurrent or serial fashion)
executed the task. That is, it need not remain alive for longer than the
duration of the function call.
Concurrently executing a task on all workers *and* waiting until
all workers have executed the task seems to be common enough to
warrant a helper function for that purpose.
Preparation for adding KILL syntax support.
Session id changed to uint32 everywhere. Added atomic op.
Session id can be acquired before session_alloc().
Added session_alloc_with_id(), which is given a session id number.
Worker object has a session_id->SESSION* mapping, not used yet.
A Worker::Task is an object that can be sent to a worker for
execution. The task is sent to the worker using the messaging
mechanism where the `execute` function of the task will be
called in the thread context of the worker.
There are two kinds of tasks; regular tasks and disposable tasks.
The former are just sent to the worker for execution while the
latter are sent and subsequently disposed of, once the task has
been executed.
A disposable task can be sent to either one worker or to all
workers. In the latter case, the task will be deleted once it
has been executed by all workers.
A semaphore can be associated with a regular task. Once the task
has been executed by the worker, the semaphore will automatically
be posted. That way, it is trivial to send a task for execution
to a worker and wait until the task has been executed. For instance:
Semaphore sem;
MyTask task;
pWorker->execute(&task, &sem);
sem.wait();
const MyResult& result = task.result();
The low level mechanism for posting and broadcasting messages will
be removed.
All workers share an epoll instance that is added level-triggered
to the epoll instance of each Worker. This is intended to be used
together with listening sockets.
When a listening socket is added to the shared epoll instance the
effect is that EPOLLIN will be active for it whenever there is a
connection pending on a listening socket added to that epoll
instance.
When that occurs all workers in their epoll_wait()-calls will return.
When the workers subsequently call epoll_wait() on the shared epoll
instance, that will return with an event provided some other thread(s)
has not yet called accept() on the listening socket.
As each worker extracts just one event at a time and calls accept just
once before calling epoll_wait(), it means that the client connections
will be distributed evenly across all workers, provided the load on
the workers is roughly the same. If it isn't then a worker with less
load will get more connections to handle (which will even out the load).
Now the statistics is in a single structure and the property of the
Worker instance in question. Methods are provided for obtaining the
statistics of all workers in one go.
Just like the thread stats and poll stats earlier, the queue stats
are now moved to worker.
A litte refactoring still, and the polling will only work on local
data.
Each worker now has a separate structure for collecting the
polling statistics that is passed to epoll_waitevents(). When
the stats are asked for, we loop over all separate stats and
combine them. So, instead of having every statistics of each
thread one cacheline apart, each thread has all its statistics
in one lump that, for obvious reasons, are going to be apart.
The primary purpose of this excersize is to remove the hardwired
nature of the statistics collection. For instance, the admin
thread will be doing I/O but that I/O should not be included
in the statistics of the workers.
The Worker no longer creates a pipe and implements the cross
worker/thread message mechanism itself. Instead it has a
MessageQueue instance variable for that purpose.
MessageQueue encapsulates a message queue built on top of a
pipe. The message queue needs a handler for receiving messages
and must be added to a worker for pumping messages through the
pipe.
Each Worker will have an instance of MessageQueue.
Now the epoll instance of the Worker is used when polling. The
work is still done in poll.cc and the worker provides the descriptor
and thread id.
The comments of poll_waitevents() have been removed; they were not
accurate anymore so better to let the code speak for itself.
And add the Worker header...
The epoll instance is not used yet, but the common creation of epoll
instances in poll.cc will be removed and the epoll instances of each
worker used instead.
This is the first step in turning the worker mechanism and everything
around it into a set of C++ classes. In this change, the original C
API is still present, but in subsequent changes that will be removed.
