632 Commits

Author SHA1 Message Date
Markus Mäkelä
0d6c06f33d MXS-1446: Add heartbeat conversion macros
The macros make the conversion from heartbeats to seconds more convenient
and consistent.
2017-09-27 19:44:25 +03:00
Markus Mäkelä
4c3d6f6884 MXS-1446: Add execution of dummy failover command
The failover command is simulated by executing a call to /usr/bin/echo
with all possible monitor parameters. This allows testing of the failover
mechanism without actually using the failover command.
2017-09-27 19:44:21 +03:00
Markus Mäkelä
316f792242 MXS-1446: Make failover_timeout configurable
The time that MaxScale waits for a failover is now configurable.
2017-09-27 19:37:41 +03:00
Markus Mäkelä
ef2ee38ccf MXS-1446: Store more detailed event information
The timestamp of the last change from passive to active is now
tracked. This, with the timestamps of the last master_down and master_up
events, allows detection of cases when MaxScale was failed over but the
failover was not done.

Currently, only a warning is logged if no new master has appeared within
90 seconds of a master_down event and MaxScale was set to active from
passive.

The last event and when the event was triggered is now shown for all
servers. The latest change from passive to active is also shown.
2017-09-27 19:32:58 +03:00
Markus Mäkelä
3e1d89ff17 MXS-1446: Store last triggered event for each server
When an event occurs on a server, it is now stored so that the last event
for each server is known. This allows a state change to trigger an event
even if, at the time of the event, no action was taken.

This change is only cosmetic as no functionality is implemented.
2017-09-27 19:32:58 +03:00
Markus Mäkelä
1d2ba10a68 MXS-1446: Add passive parameter
The `passive` parameter can be given in the configuration file or on the
command line. It is displayed in the diagnostic output and changes to it
are persisted.

The variable itself does nothing.
2017-09-27 19:32:58 +03:00
Markus Mäkelä
2633ad2fe4 Merge branch '2.1' into 2.2 2017-09-26 14:29:14 +03:00
Johan Wikman
4220e3ca6a MXS-1450 Add more string trimming functions
- trim_leading
- trim_trailing

Implemented trim in terms of trim_leading and trim_trailing
2017-09-26 13:36:10 +03:00
Markus Mäkelä
2d590308b6 Set current command when executing queued queries
When readwritesplit is routing any queued queries, the currently executed
command of the protocol modules needs to be adjusted by
readwritesplit. This is not a true fix but more of a workaround to fix the
problems of queued query execution.

The correct solution would be to move the queued query handling into the
client protocol module so that all components see the same state.
2017-09-25 10:46:55 +03:00
Markus Mäkelä
45e0e8bb59 Introduce internal protocol command enum
The enums exposed by the connector are not intended to be used by the
users of the library. The fact that the protocol, and other, modules used
it was in violation of how the library is intended to be used.

Adding an internal mapping into MaxScale also removes some of the
dependencies that the core has on the connector.
2017-09-14 15:30:43 +03:00
Markus Mäkelä
3676dcebb1 MXS-1398: Clean up mysql.h header
Cleaned up the MaxScale version of the mysql.h header by removing all
unused includes. This revealed a large amount of dependencies on these
removed includes in other files which needed to be fixed.

Also sorted all includes in changed files by type and alphabetical
order. Removed explicit revision history from modified files.
2017-09-14 15:30:43 +03:00
Markus Mäkelä
f07cc2b3ef MXS-1398: Update Connector-C to version 3.0.2
Updated the Connector-C to 3.0.2.

Removed the `mysql_com.h` header from `maxscale/protocol/mysql.h` as it is
already included by `mysql.h` and including it causes errors with
Connector-C 3.0.2.

Organized includes in `maxscale/protocol/mysql.h`.
2017-09-14 15:30:42 +03:00
Markus Mäkelä
fe40511d97 MXS-1405: Take script_timeout into use
The script_timeout parameter is now used by all monitors.
2017-09-14 12:34:34 +03:00
Johan Wikman
a2d0d24b2c Merge branch '2.1' into develop
Bringing in MXS-1406
2017-09-13 10:02:37 +03:00
Markus Mäkelä
16d3eab59f Merge branch '2.1' into develop 2017-09-12 22:42:11 +03:00
Johan Wikman
49ab5797a2 MXS-1406 Report CALL operation
Basically it would be trivial to report far more operations
explicitly, but for the fact that the values in qc_query_op_t
currently, quite unnecessarily, form a bitmask.

In 2.2 that is no longer the case, so other operations will be
added there.
2017-09-12 15:38:25 +03:00
Johan Wikman
02b9e0a01d Merge branch '2.1.7' into develop-2.1-merge 2017-09-12 11:08:02 +03:00
Markus Mäkelä
9ceb23dd65 MXS-1396: Fix persistent connection hangs
When a COM_CHANGE_USER was executed, it is possible that the server
responds with a AuthSwitchRequest packet instead of an OK packet. In this
case, the server sends a new scramble which must be used to create the 20
byte hash that is expected as the response.
2017-09-11 12:50:24 +03:00
Markus Mäkelä
730072e2ef Add format checks for printf-style functions
Added format checks to functions that expect printf style arguments and
fixed any broken calls to these functions.
2017-09-11 12:49:11 +03:00
Markus Mäkelä
486024aaf4 MXS-1390: Add missing SSL parameter processing to servers resource
Added missing SSL parameters to servers resource output as well as added
the processing of these parameters when servers are created. It is
preferable to define servers as either encrypter or plain and to prevent
the modification of this at runtime.
2017-09-11 12:49:11 +03:00
Markus Mäkelä
5abb30c357 MXS-1390: Add missing self-link to module command output
The module commands now produce functional self-links. The links most
often require parameters so they are not of great use.
2017-09-11 12:49:11 +03:00
Johan Wikman
402b27ad01 MXS-1392 Remove remnats of DCB reference counting 2017-09-08 12:41:41 +03:00
Johan Wikman
be94066b77 MXS-1392 Manage DCB lifetime using refcounts
Now it is also possible to ensure that a DCB stays alive while
a task referring to it is posted from one worker to another.
That will be implemented in a subsequent commit.
2017-09-08 12:41:41 +03:00
Johan Wikman
bf42d845cf MXS-1392 Remove dcb_free
Not used and with reference counting it must not be available either.
2017-09-08 12:41:41 +03:00
Johan Wikman
7e17e2cd56 MXS-1392 Add reference count to MXS_POLL_DATA
The polling mechanism can now optionally be used for managing
the lifetime of an object placed into the poll set.

If a MXS_POLL_DATA has a non-null 'free', then the reference count
of the data will be increased before calling the handler and
decreased after. In that case, if the reference count reaches 0,
the free function will be called.

Note that the reference counts of *all* MXS_POLL_DATAs returned
by 'epoll_wait' will be increased before the events are delivered
to the handlers individually for each MXS_POLL_DATA, and then once
all events have been delivered, the reference count of each
MXS_POLL_DATA will be decreased.

This ensure that if there are interdependencies between different
MXS_POLL_DATAs returned by one call to 'epoll_wait', the case that
an MXS_POLL_DATA is deleted before its events have been delivered
can be avoided by using the reference count for lifetime management.

In subsequent commits, the reference count will be taken into use
in the lifetime management of DCBs.
2017-09-08 12:41:41 +03:00
Markus Mäkelä
6067c21c1b MXS-1346: Use the filter template in dbfwfilter
The dbfwfilter now uses the MaxScale filter template. Also fixed up some
of the filter template documentation.
2017-09-08 09:31:37 +03:00
Johan Wikman
44db97215f Deliver fake events for the current dcb immediately
If a fake event is added to the current dcb, we arrange things so
that it is delivered immediately when the handling of the event(s)
during which the fake event was added, has been performed.

Otherwise the event is delivered via the event loop.
2017-09-07 13:48:07 +03:00
Markus Mäkelä
0d5d67a050 Use SIGABRT for debug assertions
Raising the signal directly keeps the stack trace smaller. It also makes
sure that the assertion works even if NDEBUG is defined.

The debug assertions are now also printed to stderr to make sure that they
are caught even when running MaxScale directly.
2017-09-07 10:41:15 +03:00
Markus Mäkelä
c542010e67 Fix internal test failures
The adminusers test did not properly initialize all subsystems in
MaxScale. The polling and DCB tests weren't updated with the changes to
the DCB closing.
2017-09-06 13:20:28 +03:00
Johan Wikman
3770b4da95 Name read/fake queue consistently
Now dcb_readqueue and dvb_fakequeue are named readq and fakeq
respectively, to be consistent with the naming of the write
and delay queue.
2017-09-06 11:30:24 +03:00
Johan Wikman
24044a7376 Add dcb functions
dcb_readq_append()
dcb_readq_prepend()
dcb_readq_set()
dcb_readq_has()
dcb_readq_release()
dcb_readq_get()
dcb_readq_length()

No code but for DCB code itself should directly manipulate the
internals of a DCB. Thesse functions will be taken into use in
protocol modules.
2017-09-06 11:12:32 +03:00
Johan Wikman
e51a8b4fb9 Ensure that PRI formatting defines are available
On some systems, without __STDC_FORMAT_MACROS, the PRI defines
for printing 32 and 64 bit integers are not present.
2017-09-04 15:28:17 +03:00
Johan Wikman
b43ab674e3 Add dcb_close_in_owning_thread(DCB*);
Allows a DCB to be closed by a thread other than the owning
thread.
2017-09-04 13:41:16 +03:00
Johan Wikman
61a2c960ec MXS-1378 Provide access to current session
Provide access to current session and session id. This will
be used by the logging mechanism for logging the session id
together with messages.
2017-09-01 13:20:20 +03:00
Johan Wikman
eff5e7431a MXS-1378 Provide access to the current DCB in 2.1
Conceptually this is a cherry-pick of commit
67efd1daeabbc398b8a8fbc0cd02c2af26e4cb6c (2.2), but too much has
changed to actually be able to cherry-pick that commit.
2017-09-01 13:20:20 +03:00
Markus Mäkelä
7cef722282 Handle unexpected responses to COM_CHANGE_USER
When an unexpected response to a COM_CHANGE_USER is received, it is now
processes and discarded instead of treated as an error. This will allow
further analysis of the situation in addition to possibly solving some of
the problems that the persistent connections have.

Added extra info level logging to relevant parts of the code that deal
with the COM_CHANGE_USER reply processing. This information should allow
tracking of the response state for debugging purposes.
2017-09-01 11:28:21 +03:00
Johan Wikman
e7926d770e MXS-1378 Provide access to current session
Provide access to current session and session id. This will
be used by the logging mechanism for logging the session id
together with messages.
2017-08-30 13:58:42 +03:00
Johan Wikman
67efd1daea MXS-1378 Provide access to the current DCB
This will be used by a subsequent `session_get_current()` and
`session_get_current_id()` for obtaining the current SESSION and
session id, respectively. The latter of those will be used by the
logging mechanism for logging the session id in conjunction with
messages.
2017-08-30 13:58:42 +03:00
Markus Mäkelä
e6f78bfcb3 Pass modutil function state as a struct
This allows the type to change without changing it at the call sites.
2017-08-29 22:46:35 +03:00
Johan Wikman
2a346fd061 MXS-1376 Really close a DCB only at the end of event loop
When dcb_close() is called, the DCB is only marked for closing
and the actual closing takes place only after all event handlers
have been called. That way, the state of the DCB will not change
during event processing but only after.

From a handler perspective this should now be just like it was
when the zombie queue was present.

TODO: There are far too many state variables or variables akin to
      state variables - dcb_role, state, persistentstart, n_close -
      in DCB. A cleanup is warranted.
2017-08-29 15:44:50 +03:00
Markus Mäkelä
13f7015e7b Fix EOF packet calculation for large rows
The EOF packet calculation function in modutil.cc didn't handle the case
where the payload exceeded maximum packet size and could mistake binary
data for a ERR packet.

The state of a multi-packet payload is now exposed by the
modutil_count_signal_packets function. This allows proper handling of
large multi-packet payloads.

Added minor improvements to mxs1110_16mb to handle testing of this change.
2017-08-25 17:00:42 +03:00
Johan Wikman
5832352a08 MXS-1376 DCBMM removed
Related to the zombie processing and as zombie processing is not
needed anymore, it can be removed.
2017-08-25 14:48:26 +03:00
Johan Wikman
9c25e6d995 MXS-1376 All zombie related code removed
As dcbs are now closed when dcb_close() is called and there is
no zombie queue, the zombie state can also be removed.
2017-08-25 14:48:16 +03:00
Johan Wikman
170e03fd23 MXS-1376 Update DCB structure documentation 2017-08-25 14:48:16 +03:00
Johan Wikman
ad4e8dad94 MXS-1364 Drop the usage field
But for the most trivial statements did not really provide
useful information.

The arguments of the "function" '=' are now reported.
2017-08-24 09:58:18 +03:00
Johan Wikman
72831c86ac DCBEVENTQ removed
No longer used for anything
2017-08-24 09:57:42 +03:00
Johan Wikman
365371e7b5 MXS-1364 Provide function field usage information
Only API change, implementations follow.
2017-08-22 11:01:50 +03:00
Markus Mäkelä
5b1f8afcd9 MXS-1366: Validate closed connections before pooling them
When a session is being closed in a controlled manner, i.e. a COM_QUIT is
received from the client, it is possible to deduce from this fact that the
backend connections are very likely to be idle. This can be used as an
additional qualification that must be met by all connections before they
can be candidates for connection pooling.

This assumption will not hold with batched and asynchronous queries. In
this case it is possible that the COM_QUIT is received from the client
before even the first result from the backend is read. For this to work,
the protocol module would need to track the number and state of expected
responses.
2017-08-21 10:31:07 +03:00
Markus Mäkelä
b56594470b Add persistent pool statistics
Added number of times a connection was taken from the pool as well as the
availability of the pool as a percentage of connections taken from the
pool.
2017-08-20 21:54:18 +03:00
Markus Mäkelä
6cc3986db5 MXS-1354: Prevent removal of last admin account
Removing the last admin account is now forbidden. This should prevent most
cases where users could lock themselves out of the administrative
interface.

This change does allow a non-root network user to be the last admin
account. In practice this does not prevent the root user from gaining
access to maxadmin. Access can be gained by removing the users file and
restarting MaxScale or by editing the users file by hand.
2017-08-18 10:28:21 +03:00