58 Commits

Author SHA1 Message Date
Johan Wikman
56ec56f7a0 MXS-2363 Enable the serial execution of function objects
Make it possible to run function objects serially on all
routing workers. Sometimes that is exactly what you want.
2019-03-29 11:31:35 +02:00
Markus Mäkelä
203bba0e1d
Add support for multiple runtime error messages
Storing all the runtime errors makes it possible to return all of them
them via the REST API. MaxAdmin will still only show the latest error but
MaxCtrl will now show all errors if more than one error occurs.
2019-03-21 18:19:10 +02:00
Markus Mäkelä
494a8a660a
Improve RoutingWorker utilities
Added an overload to execute_concurrently that takes an std::function as a
parameter and added a const version of operator* for rworker_local. Also
removed the std::move of the return value in rworker_local::values as it
can prevent RVO from taking place.
2019-03-21 18:19:09 +02:00
Johan Wikman
72d02d33c0 MXS-2314 Remove unnecessary worker states
The removed states were not used for anything useful.
2019-02-11 13:03:18 +02:00
Johan Wikman
d50ae1fb8a MXS-2218 Treat all routing workers the same way
All routing workers are now started an stopped the same way.
2019-01-08 16:01:36 +02:00
Markus Mäkelä
7aa60b4a24
MXS-2196: Start listeners after worker threads
If the startup of the listeners requires communication with all of the
workers, the workers must be up and running for that to happen.

Due to the fact that the main thread is still a worker thread, the
initialization code is not extra straightforward. By queuing an event to
the main worker, the startup of all listeners is done at a fully
operational state with all workers fully functional.

The service initialization code was also flawed in the sense that it would
cause a deadlock if any of the threads would have to check for the user
permissions. This is mainly a problem with the authenticator modules but
the benefits of the per service pre-loading of users is most likely
superficial. In theory startup will be faster as each thread now queries
the users in parallel.
2018-12-04 11:50:43 +02:00
Markus Mäkelä
694d4a4003
MXS-2196: Make DCB a C++ struct
Allocating DCB with new allows the use of C++ objects in the DCB
struct. Also the explicit poll field can be replaced by inheriting from
MXB_POLL_DATA.
2018-12-04 11:50:43 +02:00
Esa Korhonen
1701f87f00 MXS-2205 Convert internal/maxscale.h and internal/dcb.h to .hh 2018-12-03 13:49:08 +02:00
Esa Korhonen
36792607d8 MXS-2205 Combine internal/modules.h with internal/modules.hh 2018-12-03 13:36:25 +02:00
Markus Mäkelä
77585bdb8c
MXS-2197: Make config.h and service.h C++ headers
This is the first step into converting the other headers into C++.
2018-11-30 12:15:57 +02:00
Johan Wikman
f1a113aff0 MXS-2178 Reduce accessbility of workarounds
Some rearrangements to ensure that what should be private
can be kept private.

- WatchdogNotifier made a friend.
- WatchdogWorkaround defined in RoutingWorker and made a friend.
- mxs::WatchdogWorker defined with 'using'.
2018-11-21 13:31:49 +02:00
Johan Wikman
be12cab16d MXS-2178: Provide workaround for watchdog notifications
The systemd watchdog mechanism requries notifications at
regular intervals. If a synchronous operation of some kind
is performed by a worker, then those notfications will not
be generated.

This change provides each worker with a secondary thread that
can be used for triggering those notifications when the worker
itself is busy doing other stuff. The effect is that there will
be an additional thread for each worker, but most of the time
that thread will be idle.

Sofar only the mechanism; in subsequent changes the mechanism
will be taken into use.
2018-11-21 13:31:49 +02:00
Niclas Antti
77f8a3b71b MXS-2057 System test 2018-11-14 16:20:42 +02:00
Niclas Antti
e371964f8b MXS-2057 Add log output for system test, and two random code fixes 2018-11-14 16:20:42 +02:00
Niclas Antti
1108132cbd MXS-2057 Do not require systemd libraries
Exclude systemd usage if the library is not installed.
Only excluding what is necessary. This keeps the object size the
same and still compiles most of the code.
2018-11-09 16:45:59 +02:00
Niclas Antti
f29e5b65de MXS-2057 systemd watchdog
Systemd wathdog notification at a little more than 2/3 of the
systemd configured time. In the service config (maxscale.service)
add e.g. WatchdogSec=30s to set and enable the watchdog.
For building: install libsystemd-dev.

The next commit will modify cmake configuration and code to
conditionally compile the new code based on existence of libsystemd-dev.
2018-11-09 16:45:59 +02:00
Johan Wikman
809d3549ae MXS-2149 Add REST-API watchdog
This will simply cause a task to be posted to each worker.
If the workers are running normally, the task will reach the
workers and the associated semaphore posted, and the REST-API
call will return. If any worker is not running normally, the
task will not be processed and the REST-API call will hang.
2018-11-08 12:13:02 +02:00
Esa Korhonen
2c280e27f2 Free qc thread resources at worker exit 2018-11-06 11:44:50 +02:00
Johan Wikman
6dc038aa4e MXS-2137 Remove meaningsless statistics
The removed statistics variables have no meaning anymore and
were not updated.
Decided to simply drop the variable from the JSON output. It
gets far too rigid if fields of objects cannot be changed without
bumping the REST-API version.
2018-11-06 09:49:25 +02:00
Markus Mäkelä
75ea1b6ea1
Fix formatting of new(std::nothrow)
The code previously formatted everything as `new( std::nothrow)`.
2018-10-04 21:50:44 +03:00
Markus Mäkelä
2e069fa892
MXS-1632: Take mxb::atomic::add into use
The function now mostly replaces the use of atomic_add_ functions declared
in atomic.h.
2018-09-18 15:21:54 +03:00
Markus Mäkelä
f066d154c6
MXS-1632: Use statistics.hh functions in routingworker.cc
The custom code for accessing member methods can be removed.
2018-09-18 15:21:53 +03:00
Markus Mäkelä
94a567f4ca
Remove redundant Worker method
The Worker::get_local_statistics was a duplicate of Worker::statistics.
2018-09-10 14:14:21 +03:00
Markus Mäkelä
58c0b4f5f4
MXS-1632: Remove the unused statistics code
The statistics code was almost completely unused.
2018-09-10 14:14:20 +03:00
Niclas Antti
c447e5cf15 Uncrustify maxscale
See script directory for method. The script to run in the top level
MaxScale directory is called maxscale-uncrustify.sh, which uses
another script, list-src, from the same directory (so you need to set
your PATH). The uncrustify version was 0.66.
2018-09-09 22:26:19 +03:00
Markus Mäkelä
19b059d303
Add std::function overload to RoutingWorker::broadcast
This makes it easier to collect distributed results from multiple workers.
2018-09-04 13:04:08 +03:00
Markus Mäkelä
26dfb19ea4
Make routingworker.hh public
Given that worker.hh was public, it made sense to make routingworker.hh
public as well. This removes the need to include private headers in
modules and allows C++ constructs to be used in C++ code when previously
only the C API was available.
2018-09-04 13:04:08 +03:00
Johan Wikman
3f53eddbde MXS-2020 Replace ss[_info]_dassert with mxb_assert[_message] 2018-08-22 11:34:59 +03:00
Johan Wikman
88f1795412 MXS-2008 Move Worker and MessageQueue to maxbase 2018-08-21 10:02:39 +03:00
Johan Wikman
40a5ae81c7 MXS-2008 Store worker as MXB_WORKER* and not void* 2018-08-20 11:15:14 +03:00
Johan Wikman
e0cb11151f MXS-2008 Move maxscale/worker.h to maxbase/worker.h 2018-08-20 11:15:14 +03:00
Johan Wikman
932956d5f6 MXS-2008 Add maxbase/poll.[h|hh]
Remove maxscale/poll_core.h
2018-08-20 11:15:14 +03:00
Johan Wikman
aaad63ee5e MXS-2008 Move content of maxscale/platform.h to maxbase/cdefs.h
No good reason for keeping that platform adjustments anywhere
else but in cdefs.h.
2018-08-20 11:15:14 +03:00
Johan Wikman
d8255d0cac MXS-2017 Move maxscale::Semaphore to maxbase::Semaphore 2018-08-17 15:53:54 +03:00
Johan Wikman
f3f802cabe MXS-2008 Move maxscale/atomic.h to maxbase/atomic.h 2018-08-15 08:44:39 +03:00
Johan Wikman
47b53aae55 MXS-2004 Replace THREAD with std::thread in Worker 2018-08-13 13:38:39 +03:00
Johan Wikman
e9758ebaf1 MXS-2002 Rename Worker::post() to Worker::execute()
The main point is that tasks/functions are executed, not that
they are posted.
2018-08-13 08:30:05 +03:00
Johan Wikman
6335d3776c MXS-2002 Replace auto_ptr with unique_ptr
Given 'Derived : public Base', a unique_ptr<Derived> converts
implictly to a unique_ptr<Base>.
2018-08-13 08:30:05 +03:00
Johan Wikman
9cfd451a1d MXS-2002 Make Worker excecution mode explicit
This is the first step in some cleanup of the Worker interface.
The execution mode must now be explicitly specified, but that is
just a temporary step. Further down the road, _posting_ will
*always* mean via the message loop while _executing_ will optionally
and by default mean direct execution if the calling thread is that
of the worker.
2018-08-13 08:30:05 +03:00
Johan Wikman
90569e3a28 MXS-1992 Expose qc stats in threads info
In principle it would be better if the qc information were
obtained via a specific query_classifier resource. However,
there are multiple problems with that (e.g. the qc has no way
of safely accessing information of another thread) and hence
the worker specific qc cache statistics is reported as part of
the worker statistics.
2018-08-08 09:16:19 +03:00
Johan Wikman
fbd3b08c1e MXS-1992 Make preparations for REST-API 2018-08-08 09:16:19 +03:00
Johan Wikman
b76cdc944b MXS-1992 RoutingWorker provides access to QC stats
This will be used in the context of MaxAdmin and MaxCtrl.
2018-08-03 07:33:12 +03:00
Johan Wikman
ae27ea92a0 MXS-1992 Thread-initialize internal QC module 2018-08-03 07:33:12 +03:00
Markus Mäkelä
8a248dd930
Copy callbacks by value in mxs_rworker_broadcast
Copying by reference will not work as the references will refer to the
values passed to the function, not the actual callbacks.
2018-08-01 16:48:04 +03:00
Markus Mäkelä
ec420332ea
MXS-1929: Take ResultSet into use
Replaced the previous RESULTSET with the new implementation. As the new
ResultSet doesn't have a JSON streaming capability, the MaxInfo JSON
interface has been removed. This should not be a big problem as the REST
API offers the same information in a more secure and structured way.
2018-07-31 22:50:08 +03:00
Markus Mäkelä
ff07009d8c
MXS-1929: Add worker local storage of data
Data can now be stored on thread-local storage of the worker. By acquiring
a unique handle from the worker, a module can store a thread-local
value.

This functionality will be used to store configurations that are sometimes
updated at runtime but are largely read-only. By avoiding shared data
altogether, performance is not affected. The only synchronization that is
done is on update.

Also added a helper functions for broadcasting tasks on all routing
workers. With the old mxs_rworker_broadcast_message function, if a
function call was broadcasted it was always queued for execution. The
mxs_rworker_broadcast will immediately execute the task on the local
worker and queue it for execution of other routing workers.
2018-07-31 09:41:14 +03:00
Markus Mäkelä
cca7757090
MXS-1929: Take internal Service struct into use
The internals now mostly refer to the Service struct instead of the public
SERVICE struct.
2018-07-31 09:41:13 +03:00
Markus Mäkelä
a11d83679e
Fix REST API regression
The threads resource was not pass the schema validation test; no ID field
was defined.
2018-07-09 12:10:36 +03:00
Johan Wikman
c89bdb9626 MXS-1823 Replace meaningless eventq info with meaningful
The evq_length file held the returned number of descriptors from
the last epoll_wait() call. As such it is highly temporal and not
particularly meaningful.

That has now been removed and the instead the average number of
returned descriptors is maintained. That information changes slowly
and thus carries some meaning.
2018-07-06 11:22:42 +03:00
Markus Mäkelä
44ef4912e8
Merge branch '2.2' into develop 2018-07-03 21:13:41 +03:00