Commit Graph

1074 Commits

Author SHA1 Message Date
ecc7442358 Detect manual commands faster
Previous, MariaDBMonitor would wait until the next monitor interval before detecting
a new manual command. The commands are now checked every 100 ms.
2018-11-08 19:12:00 +02:00
809d3549ae MXS-2149 Add REST-API watchdog
This will simply cause a task to be posted to each worker.
If the workers are running normally, the task will reach the
workers and the associated semaphore posted, and the REST-API
call will return. If any worker is not running normally, the
task will not be processed and the REST-API call will hang.
2018-11-08 12:13:02 +02:00
32f2e769f4 MXS-1780 Make retain_last_statements service specific 2018-11-08 12:08:42 +02:00
c899f00541 MXS-1780 Collect server response information
As the router is the only one that knows what backends a particular
statement has been sent to, it is the responsibility of the router
to keep the session bookkeeping up to date. If it doesn't we will
know what statements a session has received (provided at least some
component in the routing chain has RCAP_TYPE_STMT_INPUT capability),
but not how long their processing took. Currently only readwritesplit
does that.

All queries are stored and not just COM_QUERY as that makes the
overall bookkeeping simpler; at clientReply() time we do not need to
know whether or not to bookkeep information, we can just do it.

When session information is queried for, we report as much information
we have available.
2018-11-08 12:04:55 +02:00
5175d2b2d7 MXS-2078 Add support for holding router specific server data.
New class to hold the statistics, part of which is currently in
RWSplitSession. Simple API in Backend to create session
specific data.
2018-11-08 10:44:32 +02:00
a349f0ad0d MXS-1779 Implement the classify REST-API 2018-10-31 10:19:40 +02:00
3631388f75 MXS-1779 Add initial support for classification URL
This commit introduces the plumbing support for obtaining
classification information of a statement using the REST-API.
It introduces a URL like

    /v1/maxscale/query_classifier/classify?sql=SELECT+1

that in the response will return a JSON object with the
information. Subsequent commits will provide the actual
information.
2018-10-31 10:19:40 +02:00
06d1189e94 Remove unused and confusing variable
The global version_string variable was never set.
2018-10-31 09:13:07 +02:00
7deed35587 Merge branch '2.2' into 2.3 2018-10-31 09:03:09 +02:00
a8eda7342f Make writeqlen and others uint64_t
This prevents them from overflowing with buffered results larger than can
be expressed with 32 bits.
2018-10-20 11:53:57 +03:00
4be5d9267d Fix writeq_high_water and writeq_low_water
The parameters were never read at startup and could not be modified at
runtime. Also the values were only read once at startup.
2018-10-20 11:53:57 +03:00
f518b38364 Initialize disk_space_checked
Was reading an uninitialized value.
2018-10-16 16:09:38 +03:00
75ea1b6ea1 Fix formatting of new(std::nothrow)
The code previously formatted everything as `new( std::nothrow)`.
2018-10-04 21:50:44 +03:00
415c7fa2d7 MXS-1777 MXS-1777 ResponseStat improvements (take 2).
Fix comments.
Fix a bug in make_valid().
Change sync time (when the average should be pushed to the server EMA)
to only depend on time, not use sample_max. This decreases the amount of
sync calls, and allows for a much shorter sync time. Testing shows this to be
more stabel and allows to make sample_max adaptive .
2018-10-04 17:53:57 +03:00
a4747f5b03 Revert the last commit, and an additional fix to the
"Fix code for warnings:" commit.
2018-10-03 17:22:10 +03:00
5892ef18af MXS-1777 ResponseStat improvements
Fix comments.
Fix a bug in make_valid().
Change sync time (when the average should be pushed to the server EMA)
to only depend on time, not use sample_max. This decreases the amount of
sync calls, and allows for a much shorter sync time. Testing shows this to be
more stabel and allow better control of the sample_max.
2018-10-03 16:35:06 +03:00
8b9fdaa0cb Remove HTTP functionality
It wasn't used so it can be removed.
2018-10-03 08:41:45 +03:00
e10f34d6aa Remove size restrictions from network reads
The reads now read as much of the data as is available to reduce the
number of distinct malloc calls that need to be made. The SSL_read also
now allocates the buffer before reading into it so that the amount of
copying is reduced.

Also removed some of the not quite helpful debug messages.
2018-10-03 08:41:43 +03:00
1ed708559d Assert that buffers are thread-local
Cross-thread usage of buffers should not happen and the debug assertions
help verify it.
2018-10-03 08:41:43 +03:00
75f9921ca2 Merge branch '2.2' into develop 2018-10-02 10:18:09 +03:00
288fbc8c33 MXS-2075: Only enable stdout redirection for maxscale
The stdout redirection must not be enabled for the other programs as they
are not run as daemon processes.
2018-10-02 09:44:02 +03:00
93d866c6a7 MXS-2073: Add missing TCP_NODELAY for client sockets
The client connections had the Nagle algorithm enabled which could cause
bad performance with smaller workloads. The common network configuration
code in utils.cc, currently used by the backend connections, sets it
properly.
2018-10-02 09:44:00 +03:00
9278da1f54 MXS-2067: Remove spinlock.h
Removed the spinlock.h header and replaced with plain pthread types and
functions.
2018-09-28 12:18:24 +03:00
3ee5d9a8ea MXS-2067: Move server lock into Server class
The lock for the server is now only visible to the MaxScale core. Changing
the type to std::mutex also allows the use of RAII lock guards.
2018-09-28 12:18:23 +03:00
ab4f870927 MXS-2067: Replace most SPINLOCKs
Replaced SPINLOCK with std::mutex where possible, leaving out the more
complex cases. The big offenders remaining are the binlogrouter and the
gateway.cc OpenSSL locks.
2018-09-28 12:18:23 +03:00
50451166bb MXS-2067: Remove spinlock.hh
Replaced the C++ versions with standard library mutexes.
2018-09-28 12:18:23 +03:00
fc1e36429c MXS-2067: Replace SPINLOCK with pthread_mutex_t
Replaced the SPINLOCK implementation with pthread_mutex_t. The SPINLOCK
interface is still used and will be removed later on.
2018-09-28 12:18:23 +03:00
dd9ff27743 MXS-1845 Rewrite server promotion code
In progress, does not yet overwrite existing code.

The new promotion mechanism automatically retries queries which timed out. It also
handles multimaster situations correctly.
2018-09-26 13:20:29 +03:00
92832c1ec4 MXS-1777 Remove selection of servers with historically lower number of connections.
Removed the almost equal comparison and subsequent selection based on historical number of connections.
The effect of it was this: Select the server that has historically, weights or not, been slower. Tested this with 2.2
with maxscale on one server and mariadb:s on two servers with different network lags. The tests with historical
selects were clearly slower.
2018-09-26 12:05:48 +03:00
7d231e5328 MXS-1777 Changing server weights to match 2.2 behavior.
Match 2.2, changed the weights back to non-inverse because 0-weight
is a special case. Renamed to server_weight for greppability.
2018-09-26 12:05:48 +03:00
548d121699 MXS-2068: Use RWBackend in schemarouter
The schemarouter now uses the RWBackend to track the response states. This
fixes the debug assertions that happened with the mxs1113_schemarouter_ps
test.
2018-09-26 09:43:26 +03:00
24b438c9b6 MXS-2068: Split reply_is_complete into two functions
By splitting the processing and state querying into two separate
functions, the result can be inspected multiple times without triggering
the result processing.
2018-09-26 09:43:25 +03:00
60cb8127a0 MXS-2068: Remove unused code 2018-09-26 09:43:25 +03:00
a32361e894 MXS-2068: Move common functionality into RWBackend
The RWBackend now updates the internal state when a new write is done in
addition to acknowledging it when the reply is complete.
2018-09-26 09:43:25 +03:00
09a64753f1 MXS-2068: Move RWBackend into mysqlcommon
This cleanly allows multiple modules to use it.
2018-09-26 09:43:25 +03:00
047242a721 MXS-701: Improve binlogfilter
Added `match` and `exclude` functionality. This allows versatile filtering
without a large investment of development time by leveraging the benefits
of PCRE2 regular expressions.

Also cleaned up the filter and removed the single table matching and
active parameter that were obsoleted by the regular expression parameters.
2018-09-26 09:43:01 +03:00
2e069fa892 MXS-1632: Take mxb::atomic::add into use
The function now mostly replaces the use of atomic_add_ functions declared
in atomic.h.
2018-09-18 15:21:54 +03:00
a2761be953 MXS-1632: Rename value_type to ValueType
Renamed the type as value_type is too close to the member definition of
value_type in various STL containers. Also formatted with Uncrustify.
2018-09-18 15:21:53 +03:00
805840dcdc MXS-1632: Add per-server read-write statistics
The read-write distribution in readwritesplit is now stored in a map
partitioned by the servers that the router has used. Currently, the
statistics for removed servers aren't dropped so some filtering still
needs to be added.
2018-09-18 15:21:53 +03:00
708dabd773 MXS-1632: Add statistics functions
The statistic.hh header defines a set of functions that complement the
standard library numeric functions. They differ from the standard library
functions in that they take a container reference and a pointer-to-member
as parameters and calculate the statistic based on the pointed-to member.
2018-09-18 15:21:52 +03:00
9b0f016aba Fix minor formatting oversights
Code that was not formatted was merged into develop unformatted.
2018-09-12 08:58:15 +03:00
1e7d088aed Make combination of rworker_local values easier
The `values` helper method provides a convenient way to access all of the
internal values as a vector.
2018-09-10 14:14:21 +03:00
58c0b4f5f4 MXS-1632: Remove the unused statistics code
The statistics code was almost completely unused.
2018-09-10 14:14:20 +03:00
108638b0cf Format with Uncrustify 0.67 2018-09-10 13:31:39 +03:00
d11c78ad80 Format all sources with Uncrustify
Formatted all sources and manually tuned some files to make the code look
neater.
2018-09-10 13:22:49 +03:00
03017c6aad Fix develop build failures
The macro in testrules.cc needed to be chagned to use string literals to
work correctly with uncrustify. Also change <cmath> to <math.h> in C
headers.
2018-09-10 11:34:09 +03:00
c81173e320 Move C++ code out of C headers
The additions into the server.h header used C++ language which caused C
programs to fail to compile. Moved the implementation of the EMAverage
class into the private Server class in the server.hh header and exposed it
via functions in the server.h header. Also temporarily moved
almost_equal_server_scores into the public server.hh as there is no
service.hh header.
2018-09-10 11:21:06 +03:00
c447e5cf15 Uncrustify maxscale
See script directory for method. The script to run in the top level
MaxScale directory is called maxscale-uncrustify.sh, which uses
another script, list-src, from the same directory (so you need to set
your PATH). The uncrustify version was 0.66.
2018-09-09 22:26:19 +03:00
fa7ec95069 MXS-1777 Tune code for cases with slow, or new servers.
Changes that allow slow or new servers to quickly apply samples towards the
server average. The most important changes are to not ignore the first N samples,
and apply an average to the server as soon as there is one available.
The new ResponseStat::make_valid() will use filter samples to add an average,
if no averages have yet been added, even if the number of  filter samples is less
than the filter limit.
2018-09-09 14:17:40 +03:00
91acbc0994 MXS-1777 Turn server weights to their inverses, and make them doubles.
The math becomes simpler when the weight is inverted, i.e. a simple multiplication
to get the (inverse) score. Inverse weights are normalized to the range [0..1] where a lower
number is a higher weight,
The enum select_criteria_t is used to provide a std::function that takes the backends
as vector  (rather than the prior pairwise compares) and returns the best backend.
2018-09-05 17:45:28 +03:00