Commit Graph

2505 Commits

Author SHA1 Message Date
c937457738 MXS-1220: Add request body parsing
The HTTP request body is expected to be a valid JSON object. All other
requests are considered malformed requests and result in a HTTP 400 error.

Added the Jansson license to the LICENSE-THIRDPARTY.TXT file. Imported
some of the tests from the Jansson test suite to the HttpParser test.
2017-05-04 09:10:32 +03:00
4eb121ce35 MXS-1220: Rename and reorganize HttpParser
The HttpParser class was renamed to HttpRequest as it parses and processes
only HTTP requests. A second class that creates a HTTP response needs to
be created to handle the response generation.

Moved some of the HTTP constants and helper functions to a separate
http.hh header.
2017-05-04 09:10:32 +03:00
e34b65658e MXS-1220: Simplify admin request handling
The admin requests are now processed in blocking mode. The timing out of
connecttions is handled by a specific timeout thread that checks the state
of each admin request.

The simplification will help with the JSON parsing with PUT/POST
commands. If non-blocking IO is used, the network reading code and JSON
parsing needs a lot more work to handle partial reads.

If the administrative interface requires higher performance and
concurrency, a multi-threaded solution could be created.
2017-05-04 09:10:32 +03:00
439d67d129 MXS-1220: Add test for HttpParser
The test simulates various HTTP requests and checks that they are parsed
properly.
2017-05-04 09:10:32 +03:00
605fed7839 MXS-1220: Add HTTP parser
The HTTP parser parses HTTP/1.1 messages into easily manageable data
structures. This should make it easier to map the HTTP requests into
actual commands in MaxScale.
2017-05-04 09:10:31 +03:00
23b6fb3e6d MXS-1220: Create separate admin thread
When MaxScale is started, a separate thread for the administrative
interface is started. This allows the worker threads to handle client
requests while the administrative thread handles the lower priority
administrative requests.

The administrative interface responds to all request with a 200 OK HTTP
response. This allows the administrative interface itself to be tested.
2017-05-04 09:10:31 +03:00
dd8a10f466 Remove old polling message system
The old polling message system is obsolete now that the worker messages
are implemented. The old system was only used to clean up the persistent
connection pool of a server.
2017-05-03 14:16:35 +03:00
4c6e0c75a5 Allow execution of tasks from within worker threads
It is now possible to define whether tasks are executed immediately or put
on the event queue of the worker thread. If task execution is in automatic
mode and the current executing thread is a worker thread, the
Task->execute method is called immediately.

This allows tasks to be posted from within worker threads. This is
intended to be used when purging of stale persistent connections and
printing diagnostic output via MaxAdmin. All of these actions are done
from within a worker thread.
2017-05-03 14:03:23 +03:00
6f24c04a4f Compile core unit tests as C++
Since the core source is C++, the tests should also be C++.
2017-05-03 14:02:25 +03:00
ab31cd4b1a Use O_DIRECT, but only if available.
TODO: The kernel version should be looked up at startup and made
      generally available so that it is readily available for
      anybody interested.
2017-05-03 13:39:26 +03:00
00b6c10089 Adjust Worker terminology
- Posting a task to a worker for execution (without implicit wait)
  is called "post".
- Posting a task to every worker for execution (without implicit wait)
  is called "broadcast".

In these cases the task must be provided as a pointer or auto_ptr, to
indicate that the provided pointer must remain alive for longer than
the duration of the function call.

- Posting a task to a worker for execution *and* waiting for all workers
  to have executed the task is called "execute" and the two variants are
  now called "execute_concurrently" and "execute_serially".

In these cases the task is provided as a reference, since the functions
will return only when all workers have (in concurrent or serial fashion)
executed the task. That is, it need not remain alive for longer than the
duration of the function call.
2017-05-02 11:32:08 +03:00
1b58a75f42 Add concurrent execution helper to Worker
Concurrently executing a task on all workers *and* waiting until
all workers have executed the task seems to be common enough to
warrant a helper function for that purpose.
2017-05-02 10:54:29 +03:00
bfd94c2b31 KILL [CONNECTION | QUERY] support, part1
Preparation for adding KILL syntax support.
Session id changed to uint32 everywhere. Added atomic op.
Session id can be acquired before session_alloc().
Added session_alloc_with_id(), which is given a session id number.
Worker object has a session_id->SESSION* mapping, not used yet.
2017-05-02 10:29:55 +03:00
ab33f05f7a Use correct __atomic builtins for atomic_add
The atomic_add function should return the old value.
2017-04-28 16:27:20 +03:00
601976dede Use __atomic builtins only where they are supported
Use the newer __atomic builtins only with GCC >= 4.7. The older __sync
builtins are used elsewhere.
2017-04-28 14:53:37 +03:00
6a3d99afe5 Prevent adding DCBs on non-worker threads 2017-04-28 12:56:38 +03:00
8f50269338 Add proxy protocol support
Adds a server-specific parameter, "use_proxy_protocol". If enabled,
a header string is sent to the backend when a routing session connection
changes state to MXS_AUTH_STATE_CONNECTED. The string contains the real
client IP and port.
2017-04-27 12:29:46 +03:00
728c780187 Expose current worker id to c-files 2017-04-26 13:17:23 +03:00
48ed7792a5 Arrange so that startup connections are handled by main Worker
When the Worker mechanism has been initialized the current_worker_id
of the calling thread is set to 0. That way, connections can be created
after Worker::init() has been called, but before the workers have been
started. Such connections will be handled by the worker that is running
in the main thread.
2017-04-26 10:57:50 +03:00
4cdb7bc49b Use Semaphore multi-wait
The dcb_foreach_parallel now uses the correct method of the Semaphore
class.
2017-04-25 15:05:24 +03:00
c3df805b22 Make dcb_foreach thread-safe
The function was no longer thread-safe as it used the obsolete per-thread
spinlocks to iterate over the DCBs. Now the function uses the newly added
WorkerTask class to iterate over them.

Since the new WorkerTask mechanism is far superion to dcb_foreach, the
latter is now deprecated.
2017-04-25 15:04:42 +03:00
963ff0216d Allow serial execution of worker tasks
The Worker::execute_on_all_wait is intended to be used with dcb_foreach
which expects a single-threaded context for its function.
2017-04-25 15:04:42 +03:00
ea39b15bbb Add safety check and rename ref mgmt functions 2017-04-25 13:20:14 +03:00
55011c2951 Add safety check and rename ref mgmt functions 2017-04-25 13:10:01 +03:00
1eb409a666 Add possibility to wait several times on a semaphore 2017-04-25 13:10:01 +03:00
19cf8c489e Rename atomic store and load functions
The atomic store and load functions are now called atomic_store_X and
atomic_load_X where X is one of int32, int64 or uint64.
2017-04-24 16:11:34 +03:00
f91d415be1 Add simple test for atomic operations
The test runs some simple tests with the atomic operations in MaxScale.
2017-04-24 15:58:28 +03:00
122337569c Add atomic store and load operations
Added abstractions for storing and loading 32-bit and 64-bit values
atomically. The functions currently use the GCC __atomic builtin atomics.
2017-04-24 15:58:28 +03:00
8174690f77 Introduce concept of Worker tasks
A Worker::Task is an object that can be sent to a worker for
execution. The task is sent to the worker using the messaging
mechanism where the `execute` function of the task will be
called in the thread context of the worker.

There are two kinds of tasks; regular tasks and disposable tasks.
The former are just sent to the worker for execution while the
latter are sent and subsequently disposed of, once the task has
been executed.

A disposable task can be sent to either one worker or to all
workers. In the latter case, the task will be deleted once it
has been executed by all workers.

A semaphore can be associated with a regular task. Once the task
has been executed by the worker, the semaphore will automatically
be posted. That way, it is trivial to send a task for execution
to a worker and wait until the task has been executed. For instance:

    Semaphore sem;
    MyTask task;

    pWorker->execute(&task, &sem);
    sem.wait();

    const MyResult& result = task.result();

The low level mechanism for posting and broadcasting messages will
be removed.
2017-04-24 14:52:54 +03:00
ade2cef852 MXS-1244: added 'detect_stale_slave' in monitor params
'detect_stale_slave' added to monitor params
2017-04-24 10:34:13 +02:00
172cdbc5a3 Update comment regarding use of level-triggered events 2017-04-24 10:51:45 +03:00
b0922576be Add Semaphore class
A simple Semaphore class that makes it simpler and less
erroprone to use a semaphore.
2017-04-23 18:54:18 +03:00
c5fd2bdb81 Add listening sockets to shared epoll instance
All sorts of additional cleaning up can now be performed.
2017-04-22 17:02:52 +03:00
56b411aea9 Add shared epoll instance to workers
All workers share an epoll instance that is added level-triggered
to the epoll instance of each Worker. This is intended to be used
together with listening sockets.

When a listening socket is added to the shared epoll instance the
effect is that EPOLLIN will be active for it whenever there is a
connection pending on a listening socket added to that epoll
instance.

When that occurs all workers in their epoll_wait()-calls will return.
When the workers subsequently call epoll_wait() on the shared epoll
instance, that will return with an event provided some other thread(s)
has not yet called accept() on the listening socket.

As each worker extracts just one event at a time and calls accept just
once before calling epoll_wait(), it means that the client connections
will be distributed evenly across all workers, provided the load on
the workers is roughly the same. If it isn't then a worker with less
load will get more connections to handle (which will even out the load).
2017-04-21 21:24:20 +03:00
a27bec5e88 Add return value to Worker::init()
Cleaner to return status than exit upon failure.
2017-04-21 19:57:45 +03:00
cf2272f479 Clean up DCB debug logging
All debug messages from dcb.cc were prefixed with the pthread ID of the
current thread. If the thread ID is needed, it should be logged by the log
manager.
2017-04-20 14:21:35 +03:00
8fe31f360d Remove multi-packet additions to response parsing
The additions to the packet parsing code weren't necessary once the
statement output change was reverted.
2017-04-20 14:18:40 +03:00
5037646846 Remove redundant flag value definitions
The response status flag values are already declared in the mysql_com.h
header.
2017-04-20 14:18:40 +03:00
d7258fffd0 MXS-1203: Improve resultset processing functions
The functions used to track the resultset EOF packets now expose the
position of the end of the result set. This allows the modules that use
them to check if more results exist in the same buffer.

Added the status bits for OK and EOF packets to the mysql.h protocol
header. This can be used to check for various state changes that happen in
the session. Currently the status bits are only used to detect if more
results are expected.
2017-04-20 14:18:40 +03:00
a8b42d24b7 Handle fake events when they are delivered
Since fake events are delivered via the event loop they can and
indeed should be processed immediately.
2017-04-20 13:51:16 +03:00
db3153ee4e Move statistics to Worker
Now the statistics is in a single structure and the property of the
Worker instance in question. Methods are provided for obtaining the
statistics of all workers in one go.
2017-04-20 13:51:16 +03:00
722d6da46f The thread state is now a property of the worker
A worker's state is not statistics, but transient information.
2017-04-20 13:51:16 +03:00
effa2f5674 poll_waitevents moved to Worker
A direct move without any non-essential modifications. Poll_waitevents will
be turned into a regular methods using instance variables.
2017-04-20 13:51:16 +03:00
4ed615a6c1 Cleanup of poll.cc
Unnecessary/unused functions, variable and headers removed.
2017-04-20 13:51:16 +03:00
2bbb67b4f5 Remove old load calculation
The existing load calculation does not fit the 2.0 thread approach
that well. So it is removed entirely now, to be replaced with some
new approach later.
2017-04-20 13:51:16 +03:00
f742a98119 Remove queue event reporting
Was not updated, output meaningless.
2017-04-20 13:51:16 +03:00
b265ca457d Remove various obsolete stuff 2017-04-20 13:51:16 +03:00
3ac619bfec Fold thread state into poll stats 2017-04-20 13:51:16 +03:00
30d7f52852 Remove meaningsless stats
Showing dcb addresses, the number of fds and the events is
meaningless as the information is completely transient and
is likely to have changed the moment is was displayed.
2017-04-20 13:51:16 +03:00
1464230714 Fix stats calculation error 2017-04-20 13:51:16 +03:00