Commit Graph

744 Commits

Author SHA1 Message Date
b36f6faa7e MXS-1754 Reintroduce maxscale::Worker
Worker is now the base class of all workers. It has a message
queue and can be run in a thread of its own, or in the calling
thread. Worker can not be used as such, but a concrete worker
class must be derived from it. Currently there is only one
concrete class RoutingWorker.

There is some overlapping in functionality between Worker and
RoutingWorker, as there is e.g. a need for broadcasting a
message to all routing workers, but not to other workers.

Currently other workers can not be created as the array for
holding the pointers to the workers is exactly as large as
there will be RoutingWorkers. That will be changed so that
the maximum number of threads is hardwired to some ridiculous
value such as 128. That's the first step in the path towards
a situation where the number of worker threads can be changed
at runtime.
2018-04-16 14:53:08 +03:00
ec33fcf87d Merge branch '2.2' into develop 2018-04-13 14:53:00 +03:00
311adf817f MXS-1776: Handle recursive COM_STMT_EXECUTE commands
Readwritesplit would not handle multiple overlapping COM_STMT_EXECUTE
commands properly if they opened cursors. This was due to the fact that
the result would not be marked as complete and COM_STMT_FETCH commands
were executed as if they did not return results.

The correct implementation is to consider a COM_STMT_EXECUTE that opens a
cursor complete only when the first EOF packet is read (that is, when the
resultset header is read). This allows subsequent COM_STMT_FETCH commands
to be handled separately.

The separate COM_STMT_FETCH handling must count the number of packets that
are being fetched. This allows correct tracking of the state of a
COM_STMT_FETCH by checking that the number of packets is correct or the
second EOF/ERR packet is read.
2018-04-11 15:26:37 +03:00
9be98df41c MXS-1625 Move RouteInfo to QueryClassifier 2018-04-10 17:41:59 +03:00
91b1ce39b8 MXS-1625 Move routing target bits to QueryClassifier 2018-04-10 17:41:59 +03:00
23aa9cc492 Add Buffer::copy_from(GWBUF*)
This makes it easier to create a Buffer from GWBUF that the API functions
pass as arguments.
2018-04-10 15:32:23 +03:00
f06b2b5ab9 Fix mxs::Buffer::copy_from
The code failed to compile when the function is used with a warning that
`pBuffer` was used without initialization. This makes sense as the first
conditional block re-declares the same parameter.
2018-04-10 15:32:22 +03:00
b9c0f0ffd2 Add mxs::extract_sql
The helper function extracts the SQL contents as std::string from a
buffer. This should make it easier to log the query in C++ code.
2018-04-10 15:31:51 +03:00
e1e01c029c MXS-1506: Allow immediate query re-routing
Now that the readwritesplit uses the same mechanism for both
retry_failed_reads and delayed query retries, the re-routing function
should accept a delay of 0 seconds. This makes the mechanism more suitable
for other uses e.g. delaying of queries in filters.
2018-04-10 15:31:50 +03:00
450b31dd8c MXS-1506: Store queries inside RWSplitSession
As the readwritesplit is the only thing that uses the statement storage,
it can be integrated into RWSplitSession. This makes the code a lot
simpler.
2018-04-10 15:31:50 +03:00
34008082e5 Rename to has_session_commands
This way it is similar to other member functions.
2018-04-10 15:30:28 +03:00
e5e607908d MXS-1506: Add router to MXS_DOWNSTREAM helper function
The function makes it easier to deal with the delayed query routing as
well as removing redundancy in the code.
2018-04-10 15:29:30 +03:00
cc793b2151 MXS-1506: Remove unused MXS_UPSTREAM variables
The `error` variable was never used. Also added a more convenient typedef
for both the downstream and upstream functions and updated filter API
version.
2018-04-10 15:29:30 +03:00
c70216390f MXS-1506: Combine housekeeper task types
The tasks themselves now control whether they are executed again. To
compare it to the old system, oneshot tasks now return `false` and
repeating tasks return `true`.

Letting the housekeeper remove the tasks makes the code simpler and
removes the possibility of the task being removed while it is being
executed. It does introduce a deadlock possibility if a housekeeper
function is called inside a housekeeper task.
2018-04-10 15:29:30 +03:00
96a0aae7fe MXS-1506: Move all functionality into Housekeeper
The class now does all of the work and the API wraps the calls to the
member methods. Using an STL container makes the list management a lot
more convenient.
2018-04-10 15:29:30 +03:00
67b2f24be1 MXS-1506: Move housekeeping into Housekeeper class
Moved the main task processing code into a class.
2018-04-10 15:29:30 +03:00
b33f464eea MXS-1506: Make heartbeat reads atomic
The old hkheartbeat variable was changed to the mxs_clock() function that
simply wraps an atomic load of the variable. This allows it to be
correctly read by MaxScale as well as opening up the possibility of
converting the value load to a relaxed memory order read.

Renamed the header and associated macros. Removed inclusion of the
heartbeat header from the housekeeper header and added it to the files
that were missing it.
2018-04-10 15:29:29 +03:00
761fda2806 Add delayed query retry prototype
This is a proof-of-concept that validates the query retrying method. The
actual implementation of the query retrying mechanism needs more thought
as using the housekeeper is not very efficient.
2018-04-10 15:29:29 +03:00
d65af04d19 MXS-1625 Make as much private as possible in QueryClassifier
Some stuff that is manipulated from RWS must be left public
for the time being.
2018-04-10 14:14:55 +03:00
563a33ae56 MXS-1625 Move get_target_type() to QueryClassifier 2018-04-10 14:14:55 +03:00
42e72bfb81 MXS-1625 Move handle_multi_temp_and_load to QueryClasifier 2018-04-10 14:14:55 +03:00
369029b612 MXS-1625 Move check_for_multi_stmt to QueryClassifier 2018-04-10 14:14:55 +03:00
9a6d66f104 MXS-1625 Move functions to QueryClassifier 2018-04-10 14:14:55 +03:00
505d6b5889 MXS-1625 Move check_create_tmp_table() to QueryClassifier 2018-04-10 14:14:55 +03:00
c8673bc0b3 MXS-1625 Move determine_query_type() to QueryClassifier 2018-04-10 14:14:55 +03:00
18ca7018b1 MXS-1625 Move log_transaction_status() to QueryClassifier 2018-04-10 14:14:55 +03:00
5689613801 MXS-1625 Hint-handling moved to QueryClassifier 2018-04-10 14:14:55 +03:00
c8961a3d14 MXS-1625 Move QueryClassifier::Handler to RWSplitSession 2018-04-10 14:14:55 +03:00
a102ddf5de MXS-1625 Remove final traces of RWSplitSession
The code for figuring out the where to send a statement does no
longer depend upon RWSplitSession but only on QueryClassifier.
So now the functionality can be moved into QueryClassifier after
which further cleanup and streamlining can be done.
2018-04-10 14:14:55 +03:00
300de4ecc6 MXS-1625 Move internal/external id map to QueryClassifier 2018-04-10 14:14:55 +03:00
304499bd36 Merge branch '2.2' into develop 2018-04-10 14:05:39 +03:00
fad4508fe2 Merge branch '2.1' into 2.2 2018-04-10 13:49:42 +03:00
cddf132d23 MXS-1762: Compare client IP when choosing a connection
When the connection pool is inspected, both the client username and IP
must match. This causes the pool to be partitioned by username and IP,
prevening unintentional sharing of connections between different users.
2018-04-09 13:21:18 +03:00
36bea39b63 MXS-1625 Remove need for RWS session from handle_multi_temp_and_load() 2018-04-05 15:05:04 +03:00
00ee271656 MXS-1625 Remove need for RWS session in log_transaction_status() 2018-04-05 13:54:37 +03:00
08987e3e85 MXS-1625 Remove need for RWS session in check_create_tmp_table 2018-04-05 13:42:39 +03:00
1133173a65 MXS-1625 Move tmp table management to QueryClassifier
Eventually only managed by QueryClassifier.
2018-04-05 13:38:55 +03:00
59d257579c MXS-1625 Move LOAD DATA sent statistics to QueryClassifier
Eventually the tracking will be internal to QueryClassifier.
2018-04-05 12:46:27 +03:00
8c0033ccb2 MXS-1625 Figure out multi statement state at startup
Whether or not multi statements are allowed is checked when
the QueryClassifier instance is created.
2018-04-05 10:16:23 +03:00
dbce77f9ee MXS-1625 Add PSManager to QueryClassifier
A copy of the PSManager of ReadWriteSplit.
2018-04-05 10:16:23 +03:00
c08120846c MXS-1625 large_query flag moved from RWS session to query classifier 2018-04-04 15:20:38 +03:00
bc22790e1b MXS-1625 Move tmp table state variable to mxs::QueryClassifier 2018-04-04 15:18:51 +03:00
8220c31664 MXS-1625 Move load data state to QueryClassifier
Still managed from the outside, but eventually will be managed
entirelly by QueryClassifier.
2018-04-04 15:16:46 +03:00
221658299d MXS-1502: Derive EqualPointees from std::unary_function
This is done to make it clear that the class is intended to be used as an
unary function.
2018-04-03 16:27:18 +03:00
1556bac6e4 MXS-1625 Add QueryClassifier class
This is the first step in providing a QueryClassifier class
that is capable of performing context dependent query classification,
where the context is essentially the session state.
2018-04-03 16:02:18 +03:00
319122e621 MXS-1502: Add have_session_commands helper function
The function conveys the meaning of the call better than using
session_command_count to check whether there are session commands to be
executed.
2018-04-03 13:32:10 +03:00
98fb2cf5ed MXS-1503: Add EqualPointees<T> template
The templated provides an unary predicate for (shared) pointers that
allows containers containing pointers to objects to be searched, not by
pointer comparisons but by comparisons of the values that they point
to. This allows std::find_if to be used with shared pointers in a similar
manner that std::find is used with non-pointer objects.
2018-04-03 13:31:40 +03:00
6ef9e1fd9a Move RWBackend into a separate file
Moved the RWBackend class implementation into its own file. Made some of
the command type functions a part of the <maxscale/protocol/mysql.h>
header to make it reusable.
2018-04-03 13:30:55 +03:00
e57ac4b0a3 MXS-1502: Prune session commands
The session command history is now compacted to contain only the first and
last execution of a session command. This should still allow most of the
more eccentric use-cases of user variables while keeping the session
command history smaller.

Added some convenience functions into the SessionCommand class to make the
pruning process easier.
2018-04-03 13:30:53 +03:00
876187b340 MXS-1503: Enable master reconnection
The `master_reconnection` parameter now controls both the reconnection of
the master server as well as the migration of the master server to another
server. Although these two cases appear to be different, the end result
from readwritesplit's point of view is the same and are thus controlled
with the same parameter.

The RWBackend class now resets its internal state when it is closed. This
allows readwritesplit to handle the case when a result was expected from
the master but the master died before the result was returned. The same
code should also handle slave connection failures mid-result, allowing
Backend reuse.

Added a test case that verifies the new functionality when combined with
`master_failure_mode=error_on_write`.
2018-04-03 13:30:51 +03:00