2695 Commits

Author SHA1 Message Date
Markus Mäkelä
c542010e67 Fix internal test failures
The adminusers test did not properly initialize all subsystems in
MaxScale. The polling and DCB tests weren't updated with the changes to
the DCB closing.
2017-09-06 13:20:28 +03:00
Johan Wikman
3770b4da95 Name read/fake queue consistently
Now dcb_readqueue and dvb_fakequeue are named readq and fakeq
respectively, to be consistent with the naming of the write
and delay queue.
2017-09-06 11:30:24 +03:00
Johan Wikman
24044a7376 Add dcb functions
dcb_readq_append()
dcb_readq_prepend()
dcb_readq_set()
dcb_readq_has()
dcb_readq_release()
dcb_readq_get()
dcb_readq_length()

No code but for DCB code itself should directly manipulate the
internals of a DCB. Thesse functions will be taken into use in
protocol modules.
2017-09-06 11:12:32 +03:00
Johan Wikman
186ee31abf Fix setting of this_thread.current_dcb
Also in the case of fake events, the current dcb must be set.
2017-09-05 16:04:59 +03:00
Johan Wikman
d3f4723c81 Do not report events for closed dcb
Before each event handler is called, it is checked whether the
dcb has been closed. If it has been, then the event handler is
not called.

The check has to be made before each event handler, because any
event handler can close the dcb.
2017-09-05 13:20:22 +03:00
Markus Mäkelä
d931787e2e Use _exit when daemonizing the process
The original process should use _exit if the forking of the child process
is successful. This makes sure that the exit handlers are called only when
the daemon process exits.
2017-09-04 17:56:34 +03:00
Johan Wikman
2da7a93473 Destroy workers after services
The workers need to be destroyed only after services have been
to ensure that they are around in case the destruction of services
involves the closing of dcbs.
2017-09-04 16:01:42 +03:00
Johan Wikman
2f0292fd68 Relax dcb_close assert
If the current worker id is -1, we do not insist that the dcb
is closed by the owning thread. That will happen only for dcbs
that are created before the workers have been started and hence
it is also ok to close them after the workers have exited.
2017-09-04 16:00:27 +03:00
Johan Wikman
b43ab674e3 Add dcb_close_in_owning_thread(DCB*);
Allows a DCB to be closed by a thread other than the owning
thread.
2017-09-04 13:41:16 +03:00
Johan Wikman
ccbff0f6d4 Allow debugging of dcb event handling
Using DCB_LOG_EVENT_HANDLING is defined, a notice will be
logged for each event when it is handled.
2017-09-04 11:29:18 +03:00
Johan Wikman
1a468049b8 Overwrite dcb thread id at final close 2017-09-04 11:09:48 +03:00
Markus Mäkelä
862548666b MXS-1367: Enable MYSQL_OPT_RECONNECT for internal connections
The automatic reconnection feature of the Connector-C was not
enabled. Enabling it should reduce the amount of false positives that the
monitors pick up.
2017-09-04 09:19:20 +03:00
Markus Mäkelä
f9c5578bdd MXS-1384: Remove parameter length limitations
There is no reason to limit configuration parameter names or values to 256
characters.
2017-09-01 22:02:33 +03:00
Markus Mäkelä
d7662a10af MXS-1385: Fix truncation of script variables
The replacement of script variables now uses the maximum string length of
the original and replacement to make sure there's enough buffer space.
2017-09-01 21:51:15 +03:00
Johan Wikman
d0ec6e3762 MXS-1378 Log session id
If the session id is known, it will be logged together with all
messages. If present, the session id appears, enclosed in paranthesis,
right after the message category. E.g.

    2017-08-30 12:20:49   warning: (4711) [masking] The rule ...
2017-09-01 13:22:06 +03:00
Johan Wikman
61a2c960ec MXS-1378 Provide access to current session
Provide access to current session and session id. This will
be used by the logging mechanism for logging the session id
together with messages.
2017-09-01 13:20:20 +03:00
Johan Wikman
eff5e7431a MXS-1378 Provide access to the current DCB in 2.1
Conceptually this is a cherry-pick of commit
67efd1daeabbc398b8a8fbc0cd02c2af26e4cb6c (2.2), but too much has
changed to actually be able to cherry-pick that commit.
2017-09-01 13:20:20 +03:00
Johan Wikman
18cb61f7de Delay closing in dcb_close
If a dcb being closed is the dcb for which events are currently being
processed, the dcb is not closed immediately but only after all events
have been delivered.
2017-09-01 11:03:23 +03:00
Johan Wikman
d7b04fc451 Add static storage to anonymous unit/thread structures
Without static, CentOS6 complains about non-local variable
using anonymous type.
2017-08-31 12:45:16 +03:00
Johan Wikman
68b7d88fa1 Check return value of freopen
Otherwise compilation fails in release mode.
2017-08-30 14:36:51 +03:00
Johan Wikman
63257f1b2a MXS-1378 Log session id
If the session id is known, it will be logged together with all
messages. If present, the session id appears, enclosed in paranthesis,
right after the message category. E.g.

    2017-08-30 12:20:49   warning: (4711) [masking] The rule ...
2017-08-30 13:58:42 +03:00
Johan Wikman
e7926d770e MXS-1378 Provide access to current session
Provide access to current session and session id. This will
be used by the logging mechanism for logging the session id
together with messages.
2017-08-30 13:58:42 +03:00
Johan Wikman
67efd1daea MXS-1378 Provide access to the current DCB
This will be used by a subsequent `session_get_current()` and
`session_get_current_id()` for obtaining the current SESSION and
session id, respectively. The latter of those will be used by the
logging mechanism for logging the session id in conjunction with
messages.
2017-08-30 13:58:42 +03:00
Markus Mäkelä
e6f78bfcb3 Pass modutil function state as a struct
This allows the type to change without changing it at the call sites.
2017-08-29 22:46:35 +03:00
Johan Wikman
2a346fd061 MXS-1376 Really close a DCB only at the end of event loop
When dcb_close() is called, the DCB is only marked for closing
and the actual closing takes place only after all event handlers
have been called. That way, the state of the DCB will not change
during event processing but only after.

From a handler perspective this should now be just like it was
when the zombie queue was present.

TODO: There are far too many state variables or variables akin to
      state variables - dcb_role, state, persistentstart, n_close -
      in DCB. A cleanup is warranted.
2017-08-29 15:44:50 +03:00
Markus Mäkelä
866a15dafa Skip COM_CHANGE_USER for COM_QUIT
As COM_QUIT would terminate the connection, there's no need to initiate
the session reset process. Also make sure all buffers are empty before
putting the DCB into the pool.

Added extra debug assertions for parts of the code that are related to the
COM_CHANGE_USER processing.
2017-08-29 11:15:06 +03:00
Markus Mäkelä
abd99febcf Fix possible crash on show sessions
The function that printed all sessions assumed that all client DCBs had
valid, non-dummy sessions. It is possible that a client with a dummy
session is the list. These sessions should be ignored.
2017-08-25 18:16:07 +03:00
Markus Mäkelä
13f7015e7b Fix EOF packet calculation for large rows
The EOF packet calculation function in modutil.cc didn't handle the case
where the payload exceeded maximum packet size and could mistake binary
data for a ERR packet.

The state of a multi-packet payload is now exposed by the
modutil_count_signal_packets function. This allows proper handling of
large multi-packet payloads.

Added minor improvements to mxs1110_16mb to handle testing of this change.
2017-08-25 17:00:42 +03:00
Johan Wikman
9c25e6d995 MXS-1376 All zombie related code removed
As dcbs are now closed when dcb_close() is called and there is
no zombie queue, the zombie state can also be removed.
2017-08-25 14:48:16 +03:00
Johan Wikman
c11d8fa328 MXS-1376 Close a connection in dcb_close
As each client/server connection will be handled by a specific
thread, all closing activity can take place directly when the
connection is closed and not later when the zombie queue is
processed.

In a subsequent commit the zombie queue will be removed.
2017-08-25 14:48:16 +03:00
Markus Mäkelä
c2a9c4bfc1 Create admin user files if they don't exist
The default users are now inserted into the admin users files if no
existing files are found. This removes the hard-coded checks for admin
user names and simplifies the admin user logic.
2017-08-25 13:54:10 +03:00
Markus Mäkelä
0e89e445f2 Bind admin interface to IPv4 if IPv6 fails
The default interface for the admin interface is the IPv6 address '::'
which corresponds to the IPv4 address '0.0.0.0'. If the system doesn't
support IPv6, then an attempt to bind on IPv4 should be made.
2017-08-25 13:54:10 +03:00
Johan Wikman
ad4e8dad94 MXS-1364 Drop the usage field
But for the most trivial statements did not really provide
useful information.

The arguments of the "function" '=' are now reported.
2017-08-24 09:58:18 +03:00
Johan Wikman
72831c86ac DCBEVENTQ removed
No longer used for anything
2017-08-24 09:57:42 +03:00
Johan Wikman
11dffd0d93 Remove unused variables 2017-08-24 09:57:42 +03:00
Markus Mäkelä
bc1e2a844c MXS-1374: Add persistpoolmax and persistmaxtime to alter server
The alteration of a server's persistpoolmax and persistmaxtime parameters
appeared to work when in reality it did not change the values.
2017-08-24 09:21:00 +03:00
Markus Mäkelä
5b1f8afcd9 MXS-1366: Validate closed connections before pooling them
When a session is being closed in a controlled manner, i.e. a COM_QUIT is
received from the client, it is possible to deduce from this fact that the
backend connections are very likely to be idle. This can be used as an
additional qualification that must be met by all connections before they
can be candidates for connection pooling.

This assumption will not hold with batched and asynchronous queries. In
this case it is possible that the COM_QUIT is received from the client
before even the first result from the backend is read. For this to work,
the protocol module would need to track the number and state of expected
responses.
2017-08-21 10:31:07 +03:00
Markus Mäkelä
b56594470b Add persistent pool statistics
Added number of times a connection was taken from the pool as well as the
availability of the pool as a percentage of connections taken from the
pool.
2017-08-20 21:54:18 +03:00
Markus Mäkelä
6cc3986db5 MXS-1354: Prevent removal of last admin account
Removing the last admin account is now forbidden. This should prevent most
cases where users could lock themselves out of the administrative
interface.

This change does allow a non-root network user to be the last admin
account. In practice this does not prevent the root user from gaining
access to maxadmin. Access can be gained by removing the users file and
restarting MaxScale or by editing the users file by hand.
2017-08-18 10:28:21 +03:00
Markus Mäkelä
828649ba99 MXS-1354: Add user authorization to maxadmin
All commands that modify the internal state of MaxScale now require admin
level authorization.
2017-08-18 10:28:21 +03:00
Markus Mäkelä
9f81f0775f MXS-1354: Rename user account type enum
Renamed the enum and its values.
2017-08-18 10:28:21 +03:00
Markus Mäkelä
6ee7ed6a38 MXS-1354: Add creation of basic/admin users to maxctrl
The type of the created user can now be specified with the --type option.

Expanded tests that cover the user creation. Also added a test case that
checks that basic users are only allowed to read through the REST API.
2017-08-18 10:28:21 +03:00
Markus Mäkelä
9d24a63c10 MXS-1354: Add user account types to REST API
The user accounts can now be created with a specific account type. This
allows read-only users to be created for the REST API.
2017-08-18 10:28:21 +03:00
Markus Mäkelä
ec045b7ab6 MXS-1354: Add creation of basic user via MaxAdmin
MaxAdmin can now create basic users for both network and UNIX domain
socket use. Currently the basic and admin types have the same permissions
in maxadmin but for the REST API, only admin accounts can modify MaxScale.
2017-08-18 10:28:21 +03:00
Markus Mäkelä
2708942762 MXS-1354: Store users in a new format
The users are now stored as an array of JSON objects. Legacy users are
automatically upgraded once they are loaded and a backup of the original
users file is created.

Removed the password parameter from the `remove user` maxadmin command as
well as all of the relevant functions. Requiring that an administrator
knows the password of the account to be deleted is not a sound requirement
now that, at least in theory, two types of accounts can be created.
2017-08-18 10:28:20 +03:00
Markus Mäkelä
a3e7fd4f23 MXS-1354: Add utility functions to User class
Added a utility function for checking if an admin user has been
created. Removed unused promote and demote commands which can be replaced
with a call to remove and add.
2017-08-18 10:28:20 +03:00
Markus Mäkelä
3aebe0f91e MXS-1354: Add JSON serialization of users
The users can now be dumped and loaded as JSON objects. This allows easier
parsing and handling of users while still retaining the possibility to
manually edit the output. Added tests for dumping and loading the JSON
form users.

Also fixed a deadlock in Users::remove() where the same lock was acquired
twice and a faulty test case where failed authentication was expected to
work.
2017-08-18 10:28:20 +03:00
Markus Mäkelä
253d6d211f MXS-1354: Allow creation of basic users
The type of the user being created is defined at creation time. This
allows the creation of basic users.

Although the users can be created internally, they cannot yet be created
via maxadmin or the REST API.
2017-08-18 10:28:20 +03:00
Markus Mäkelä
829d8a1224 MXS-1354: Rename enums to better represent their use
Renamed permission types to PERM_BASIC and PERM_ADMIN to better represent
how the permission types are used.
2017-08-18 10:28:20 +03:00
Markus Mäkelä
4a179d973b MXS-1354: Add missing locking to the Users class
The Users class now performs locking when a method is called. This will
prevent concurrent access to the internal map of users.

Added missing const versions of SpinLockGuard.
2017-08-18 10:28:20 +03:00