Commit Graph

8145 Commits

Author SHA1 Message Date
c643f9bc8d Merge branch '2.3' into develop 2019-04-12 13:23:49 +03:00
1652b18a7b Fix whitespace in canonicalized queries
Trailing whitespace was not removed and whitespace wasn't normalized to
spaces.
2019-04-12 09:18:07 +03:00
d2ecaa83a6 Move result start handling into separate function
The largest part of the code deals with the start of a response. Moving
this into a subfunction makes the function clearer as the switch statement
inside a switch statement is removed.
2019-04-12 09:18:07 +03:00
746bd53668 Simplify RWBackend result handling
By processing the packets one at a time, the reply state is updated
correctly regardless of how many packets are received. This removes the
need for the clunky code that used modutil_count_signal_packets to detect
the end of the result set.
2019-04-12 09:18:05 +03:00
e6526dd9ea Add extra info logging to readwritesplit
Added logging into RWBackend reply state processing code to know more.
2019-04-12 09:17:48 +03:00
139651c092 MXS-2253 Runtime ttl changes are made in seconds
Internally durations are stored in milliseconds but runtime changes
using SQL are made in seconds. Consequently, the provided value must
be multiplied by 1000 before being stored.
2019-04-11 15:53:45 +03:00
74634abc80 MXS-1662 Move PAM authentication function into maxbase
The same code can be used for REST-API authentication.
2019-04-09 14:41:40 +03:00
9a5b60a071 Add forced maintenance mode tests
Tested that the force option works and is accepted.
2019-04-09 10:00:50 +03:00
0cb15976e8 Backport: Add force option to set endpoint
The new `force=yes` option closes all connections to the server that is
being put into maintenance mode. This will immediately close all open
connections to the server without allowing results to return.
2019-04-09 09:43:18 +03:00
7fb840ac9e Sort CN_ definitions 2019-04-09 09:43:18 +03:00
05515cca16 MXS-2259: Limit size of client reads
Given the assumption that queries are rarely 16MB long and that
realistically the only time that happens is during a large dump of data,
we can limit the size of a single read to at most one MariaDB/MySQL packet
at a time. This change allows the network throttling to engage a lot
sooner and reduces the maximum overshoot of throtting to 16MB.
2019-04-05 22:48:16 +03:00
aad29404c6 Fix parameter value error
The argumets were given in the wrong order.
2019-04-05 13:33:16 +03:00
b54e67223f MXS-2423: Add missing parameters to maxscale endpoint
Also updated the REST API documentation to include the newer output
(automating this update would be valuable).
2019-04-05 13:33:16 +03:00
9722c0887a Log connection ID when reading server handshake
By logging the connection ID for each created connection, failures can be
traced back from the backend server all the way up to the client
application.
2019-04-05 13:33:16 +03:00
ec890b33cd Prevent checksum mismatch on second trx replay
If a transaction replay has to be executed twice due to a failure of the
original candidate master, the query queue could contain replayed
queries. The replayed queries would be placed into the queue if a new
connection needs to be created before the transaction replay can start.
2019-04-05 13:33:16 +03:00
6421af1bb4 Backport query queue changes to 2.3
Backported the changes that convert the query queue in readwritesplit into
a proper queue. This changes combines both
5e3198f8313b7bb33df386eb35986bfae1db94a3 and
6042a53cb31046b1100743723567906c5d8208e2 into one commit.
2019-04-05 13:33:16 +03:00
31c93cfe1c Use SHA2-512 for admin users
MD5 is not secure enough with the modern hardware. Upgrading to SHA2-512
helps move the problem to the future.
2019-04-05 01:00:47 +03:00
adba581a4d Fix addition of admin users
The user passwords were stored in plaintext format.
2019-04-05 01:00:44 +03:00
daf5f52c64 Pass raw password to users_auth
By passing the raw password deeper into the authentication code, it can be
used to verify the user can access some systems. Right now, this is not
required by the simple salted password comparison done in MaxScale.
2019-04-05 00:42:00 +03:00
40d73948a9 MXS-1662: Move mxs_crypt into utils
Moved the mxs_crypt function into utils and renamed to mxs::crypt (no C
code used it).
2019-04-05 00:42:00 +03:00
d2f31aab0a MXS-2420 Add debug function for decoding response packet
When debugging you occasionally want to find out what a packet
contains (e.g. delivered to clientReply). Manually looking into
the packet works, but is tedious. With this function you when
the execution has been stopped in GDB examine a protocol packet.

E.g.

Thread 3 "maxscale" hit Breakpoint 6, RWSplitSession::clientReply (this=0x7fffe401ed20, writebuf=0x7fffe401e910, backend_dcb=0x7fffe401dbe0) at /home/wikman/MariaDB/MaxScale/server/modules/routing/readwritesplit/rwsplitsession.cc:567
567	    DCB* client_dcb = backend_dcb->session->client_dcb;
(gdb) p dbg_decode_response(writebuf)
$30 = 0x7ffff0d40d54 "Packet no: 1, Payload len: 44, Command : ERR, Code: 1146, Message : Table 'test.blahasdf' doesn't exist"
2019-04-04 16:05:16 +03:00
2aa3515fc8 Merge commit '09cb4a885f88d30b5108d215dcdaa5163229a230' into develop 2019-04-04 14:34:17 +03:00
b08d4e37b5 MXS-2416: Pass deleter to std::shared_ptr<GWBUF>
As shared_ptr doesn't automatically use std::default_delete<T>, it needs
to be explicitly passed to the constructor.
2019-04-03 12:57:06 +03:00
e3e66f8e90 MXS-2417: Add option to ignore persisted configs
The load_persisted_configs parameter now controls whether persisted
runtime changes are loaded on startup. The changes are still generated as
it persists the current state of MaxScale making problem analysis easier.
2019-04-03 12:57:05 +03:00
a217dde1f0 MXS-2419: Queue queries executed during trx replay
By storing the queries in the query queue and routing it once the
transaction replay is done, we prevent two problems:

* Multiple transaction replays would overwrite the m_interrupted_query
  buffer that was used to store any queries executed during the
  transaction replay.

* Incorrect ordering of queries when the query queue is not empty and a
  new query is executed during transaction replay.
2019-04-03 12:57:05 +03:00
2dfd7d35ac MXS-2418: Crash on trx replay when log_info is enabled
If the session starts with no master but later one becomes available, when
a transaction is started the code would unconditionally use the master's
name in a log message.
2019-04-03 12:57:05 +03:00
5242cd5ebf Readwritesplit: Graceful maintenance mode
By allowing transactions to the master to end even if the server is in
maintenance mode makes it possible to terminate connections at a known
point. This helps prevent interrupted transactions which can help reduce
errors that are visible to the clients.
2019-04-02 14:21:54 +03:00
ebd2162024 Add force option to set endpoint
The new `force=yes` option closes all connections to the server that is
being put into maintenance mode. This will immediately close all open
connections to the server without allowing results to return.
2019-04-02 14:21:54 +03:00
f4c6b648d8 MXS-2271 Remove monitor->m_active field
Deactivated monitors are now moved to a separate, inaccessible list.
2019-04-02 13:08:38 +03:00
d89f0c062b MXS-2271 Change Monitor->m_name to std::string
Also, monitor address is no longer printed.
2019-04-02 13:08:38 +03:00
1417292f0d Destroy monitor only if it's not used
If a service uses a monitor as the source of its servers, it must not be
destroyed before the monitor is removed from all services that use it.
2019-04-02 10:08:59 +03:00
84bf241dd1 MXS-2339 A running Clustrix node is regarded as master
In this context master should be interpreted as "can be read
and written to".

Marking them as master requires less changes in RWS to make it
usable with a Clustrix cluster.
2019-04-02 08:13:50 +03:00
e659c6cc73 Merge branch '2.3' into develop 2019-04-01 13:24:30 +03:00
738ae9178b Fix binlogfilter matching
The matching always checked the default database when it should only check
it if there are no tables in the statement.
2019-04-01 11:29:16 +03:00
5346b24fa4 MXS-2393 Add parameter 'require_fully_parsed'
If set to true and if any of the other blocking related parameters
is true, then a statement that cannot be fully parsed will be blocked.

Default is true.
2019-04-01 10:52:52 +03:00
d7e3dd1a23 MXS-2363 Return QC cache information as collection
More logcal perhaps and Makes it easier to handle the output in maxctrl.
2019-03-29 11:31:35 +02:00
ea6ffe2371 MXS-2363 Add REST-API test 2019-03-29 11:31:35 +02:00
5f5d2ef183 MXS-2363 Implement /v1/maxscale/query_classifier/cache
That URL will now return information about the statements in
the query classifier cache. The information is collected using
the same map in a serial manner from all routing workers (that
each have their own cache). Since all caches will contains the
same statements, collecting the information in a serial manner
means that the overall memory consumption will be lower than
what it would be if the information was collected in parallel.
2019-03-29 11:31:35 +02:00
64a0327ada MXS-2363 Add string constants needed by new QC functionality 2019-03-29 11:31:35 +02:00
56ec56f7a0 MXS-2363 Enable the serial execution of function objects
Make it possible to run function objects serially on all
routing workers. Sometimes that is exactly what you want.
2019-03-29 11:31:35 +02:00
9ec82932cf MXS-2363 Skeleton of /v1/maxscale/query_classifier/cache
URL routing in place, callback exists, but no actual information.
2019-03-29 11:31:35 +02:00
1a81371346 MXS-2363 Keep track of individual QC cache hits
Useful statistics.
2019-03-29 11:31:35 +02:00
48791c3877 Fix duplication of the router parameter
The service parameter list had two router entries in it due to the
assumption that the parameter list never contained the router itself in
it.
2019-03-29 10:35:41 +02:00
388e952c5f MXS-1951: Fix port conflict detection
With the addition of SO_REUSEPORT support, it is no longer possible to
rely on the network stack to prevent multiple listeners from listening on
the same port. Without explicitly checking for the ports it would be
possible for two listeners from two different services to listen on the
same port in which case the service would be almost randomly chosen.
2019-03-28 16:34:12 +02:00
df3b501563 MXS-1951: Add SO_REUSEPORT support
If SO_REUSEPORT is available and the kernel supports it, listeners will
now listen on separate file descriptors. This removes the need for
cross-worker communication when in normal operation which should make
MaxScale scale better.
2019-03-28 16:34:12 +02:00
cb957200c9 MXS-1951: Detect SO_REUSEPORT support
The support for SO_REUSEPORT is now detected at runtime.
2019-03-28 16:34:12 +02:00
e745d1cffb MXS-2312: Use default network buffer sizes
Removed the hard-coded network buffer sizes. This effectively makes them
configurable via the kernel.
2019-03-28 16:34:12 +02:00
db97c3343d MXS-1951: Move worker selection into Listener
The worker to which the client DCB is assigned to is now chosen by the
Listener. This makes the protocol code simpler which is always a good
thing.
2019-03-28 16:34:12 +02:00
5c7846da3f MXS-1951: Make listener fd worker-local
By storing the file descriptor inside a worker-local variable, it is
possible to handle both unique file descriptors (created with
SO_REUSEPORT) and shared file descriptors with the same code. The way in
which the file descriptor is stored in the rworker_local object determines
the way the listener behaves.
2019-03-28 16:34:12 +02:00
27cc247e47 MXS-1951: Clean up listener creation
The old values that were encoded into a string wasn't necessary as the
data is already present in a processed form in the member variables.
2019-03-28 16:34:12 +02:00