3361 Commits

Author SHA1 Message Date
Johan Wikman
7f24f12cfc cache: Update documentation and add rule handling
The concept of 'allowed_references' was removed from the
documentation and the code. Now that COM_INIT_DB is tracked,
we will always know what the default database is and hence
we can create a cache key that distinguises between identical
queries targeting different default database (that is not
implemented yet in this change).

The rules for the cache is expressed using a JSON object.
There are two decisions to be made; when to store data to the
cache and when to use data from the cache. The latter is
obviously dependent on the former.

In this change, the 'store' handling is implemented; 'use'
handling will be in a subsequent change.
2016-09-28 19:38:11 +03:00
Johan Wikman
20b57b1577 cache: Remove default db only is use db succeeds 2016-09-28 19:38:11 +03:00
counterpoint
2ae4fb7745 Add missing comment blocks. 2016-09-28 09:33:28 +01:00
Markus Makela
c919511ba7 Implement simple failover mode into mysqlmon
The mysqlmon simple failover mode allows it to direct write traffic to a
secondary node. This enables a very simple failover mode with MaxScale
when it is used in a two node master-slave setup.
2016-09-26 11:00:16 +03:00
Johan Wikman
a4aa03a1fb cache: Explicitly handle COM_INIT_DB
With this change, the cache will be aware of which default database
is being used. That will remove the need for the cache parameter
'allowed_references' and thus make the cache easier to configure
and manage.
2016-09-23 19:06:14 +03:00
Johan Wikman
ac43580ce8 cache: storage_args renamed to storage_options 2016-09-22 14:22:45 +03:00
Markus Makela
e484aac6f0 Use CLOCK_MONOTONIC instead of CLOCK_MONOTONIC_RAW
The CLOCK_MONOTONIC_RAW isn't supported on all of the platforms, namely
CentOS 5.
2016-09-22 10:09:25 +03:00
Markus Makela
0a951d5e65 Merge branch 'develop-2.0-merge' into develop 2016-09-22 09:49:10 +03:00
Markus Makela
eaf43ca92d Merge branch '2.0' into develop-2.0-merge 2016-09-21 14:59:30 +03:00
Markus Makela
0882541c80 Make the default directories configurable
The default directories can now be changed at build time. This allows
tarballs to look for libraries in a more sensible place.
2016-09-21 14:21:18 +03:00
Markus Makela
578f21e757 MXS-874: Clear closed state before reconnecting to a server
The backend reference states should be cleared when a reconnection attempt
is made. Should the creation of a new DCB succeed, the backend should no
longer be closed.
2016-09-21 11:15:57 +03:00
Markus Makela
08d980b433 Trim and squeeze whitespace when canonicalizing queries
The canonical form of the query should ignore changes in whitespace as the
semantics of the query stays the same regardless of the amount of
whitespace.
2016-09-21 10:58:24 +03:00
Markus Makela
ca9021b835 Merge branch '2.0' into develop 2016-09-21 02:51:27 +03:00
Markus Makela
62ab834e8a Remove unused variables from readwritesplit
There were some unused variables in readwritesplit that caused builds to
fail.
2016-09-20 21:04:10 +03:00
Markus Makela
dfec3c8552 Install maxbinlogcheck in the right place
Maxbinlogcheck was installed in the wrong place.
2016-09-20 16:32:21 +03:00
counterpoint
923761159f Move MXS-807 refactor read-write split into develop. 2016-09-20 11:55:02 +01:00
Markus Makela
3992135325 Move authentication return codes to gw_authenticator.h
The MYSQL_* authentication return codes are now in gw_authenticator.h so
that all authenticators can use them. Also dropped the MYSQL_ prefix from
the return codes and added AUTH_INCOMPLETE for a generic
authentication-in-progress return code.
2016-09-20 11:44:17 +03:00
Markus Makela
b1b2e5b770 Don't free the shared shard maps
When client sessions are closed, the shared shard maps should not be freed.
2016-09-20 11:33:09 +03:00
Markus Makela
0ab4f04d7b Add authentication plugin name to authenticator API
The authenticators can now declare the authentication plugin name. Right
now this is only relevant for MySQL authentication but for example the
HTTP module could implement both Basic and Digest authentication.
2016-09-20 11:31:13 +03:00
Johan Wikman
4d1eb6fe85 cache: Process response to COM_QUERY
When a query has been sent to a backend, the response is now
processed to the extent that the cache is capable of figuring
out how many rows are being returned, so that the cache setting
`max_resultset_rows` can be processed.

The code is now also written in such a manner that it should be
insensitive to how a package has been split up into a chain of
GWBUFs.
2016-09-20 10:41:02 +03:00
Johan Wikman
7e2a21de9e Cache: Assume we need to handle one packet at a time 2016-09-20 10:41:02 +03:00
Markus Makela
35d4be14d2 Make service and monitor permissions checks optional
MaxScale shouldn't require the service and monitor user checks. It makes
sense to disable the checks to speed up the startup process when the user
knows that the permissions are OK.
2016-09-20 10:30:53 +03:00
Markus Makela
60955ba70d Clean up mysqlmon after pull request merge
The pull request introduced some minor whitespace errors.
2016-09-20 09:23:58 +03:00
TheTuxKeeper
e849297fc8 line length now less than 110 characters 2016-09-20 09:23:58 +03:00
Daniel
48456833da fixed memleak and potential call of mysql_num_rows with NULL 2016-09-20 09:23:58 +03:00
counterpoint
73c974c286 Remove automatic create database - requires too powerful database user. 2016-09-20 09:23:58 +03:00
counterpoint
ec42413db8 Initial changes to implement test before creating maxscale_schema. 2016-09-20 09:23:57 +03:00
Markus Makela
92ef33327e MXS-870: Handle non-contiguous session command responses
Session command responses with multiple packets could be spread across
multiple, non-contiguous buffers. If a buffer contained a complete packet
and some extra data but it wasn't contiguous, the debug assertion in
gwbuf_clone_portion would fail. With release builds, it would cause
eventual out-of-bounds memory access when the response would be sent to
the client.
2016-09-19 09:58:20 +03:00
Markus Makela
89f9f4a42f Lock writeq before inspecting it
Looking at the contents of the writeq should be done under a spinlock
otherwise it is possible that another thread grabs the queue.
2016-09-16 10:05:50 +03:00
Markus Makela
2a4addc298 Clear waiting results flag on client errors
When a backend causes an error and it should be sent to the client, the
backend reference was closed but the waiting results state was not
cleared. This caused a debug assertion to be hit.
2016-09-16 09:58:56 +03:00
Markus Makela
029e6574da MXS-812: Always reset counters when backend is closed
The active operation counters are now closed every time a backend referece
is taken out of use. This should fix a few debug assertions that were hit
in tests.
2016-09-15 08:31:15 +03:00
Markus Makela
ff7634113b Assign Relay Master only to running servers
The Relay Master status was set for servers that were down. The check for
valid master and node IDs was missing which caused a false positive.
2016-09-15 07:13:10 +03:00
Markus Makela
da9c7db231 Fix Travis builds
The travis builds failed due to outdated build scripts. The queuemanager
also failed to build on non-Debug builds.
2016-09-15 07:13:10 +03:00
Johan Wikman
8baba28450 Cleanup modutil_get_next_MySQL_packet
- Single entry/single exit.
- Variables declared as they are needed.
- The GWBUF_EMPTY check removed as it only looks at the first buffer
  in a chain. That is, if there had been a non-empty chain where the
  first buffer is empty, the function would incorrectly have reported
  that the buffer contains no packet.
- Documentation updated.
2016-09-14 15:13:25 +03:00
Johan Wikman
4597cdf7b9 Use gwbuf_split in modutil_get_next_MySQL_packet
Earlier, a copy was made.
2016-09-14 15:13:25 +03:00
Johan Wikman
b138a43ab8 cache: 64KiB is more appropriate default than 64MiB 2016-09-14 14:39:27 +03:00
Johan Wikman
8a12bd6487 Implement parameter handling for cache 2016-09-14 14:11:27 +03:00
Johan Wikman
86909b5067 Add macro for checking buffer contiguousness
A NULL buffer is considered contiguous.
2016-09-14 13:01:34 +03:00
Johan Wikman
65ccc6b8fb CMakeLists.txt cleanup
- maxscale-common no longer linked with libaio as it is not used.
2016-09-14 11:31:01 +03:00
Johan Wikman
9790eaaf20 storage_rocksdb explicity dependent on RocksDB 2016-09-13 11:45:03 +03:00
Johan Wikman
2cb6bdcf67 Fix typo on build variable 2016-09-13 11:45:03 +03:00
Johan Wikman
293b125e12 Define namespace function using explicit scope 2016-09-13 11:45:03 +03:00
Johan Wikman
cf86b80584 Link storage_rocksdb with libraries required by RocksDB 2016-09-13 11:45:03 +03:00
Johan Wikman
7fe981c22f Cache: Handle TTL manually
The RocksDB TTL database only honours the TTL when the database
is compacted. If the database is not compacted, stale values will
be returned until the end of time.

Here we utilize the knowledge that the TTL is stored after the
actual value and use the root database for getting the value,
thereby getting access to the timestamp.

It's still worthwhile using the TTL database as that'll give
us compaction and the removal of stale items.
2016-09-13 11:45:03 +03:00
Johan Wikman
a96d215aa0 Make it possible to include internal RocksDB headers
The RocksDB storage will need to refer to some internal
RocksDB files.
2016-09-13 11:45:03 +03:00
Johan Wikman
a76c05e8db MXS-797 Add initial version of RocksDB storage
RocksDB is cloned from github and version v4.9 (latest at the time of
this writing) is checked out.

RocksDB can only be compiled as C++11, which means that RocksDB and hence
storage_rocksdb can be built only if the GCC version is >= 4.7.

The actual storage implementation is quite straightforward.

- The key is a SHA512 of the entire query. That will be changed so that
  the database/table name is stored in the beginning of the key unhashed
  as that will cause cached items from the same table to be stored
  together. Assumption is that if you access something from a particular
  table, chances are you will access something else as well.
- When the SO is loaded, the initialization function will created a
  subdirectory storage_rocksdb under the MaxScale cache directory.
- For each instance, the RocksDB cache is created into a directory
  whose name is the same as the cache filter name in the configuration
  file, under that directory.
- The storage API's get and put functions are then mapped directly on
  top of RockDB's equivalent functions.
2016-09-13 11:45:03 +03:00
Markus Makela
6dc75d4b9c MXS-860: Detect whether replication is configured
The `detect_stale_slave` functionality used to only work when MaxScale had
the knowledge that a master server has existed and that replication was
working at some point in time. This might be a "safe" way to do it in
regards to staleness of the data but in practice it is preferrable to
always allow slave to be used for reads.

This change adds the missing functionality to the monitor by assigning
slave status to all servers which are configured as replication slaves
when no master can be found.

The new member variable that was added to the SERVER should be removed in
2.1 where the server_info offers the same functionalty without "polluting"
the SERVER type.
2016-09-12 15:59:08 +03:00
Markus Makela
36e243e07d Refactor monitor_mysql_db functions
The different server monitoring functions all did similar work and
combining them into one function makes the whole process of monitoring a
server simpler.
2016-09-12 15:57:27 +03:00
Markus Makela
4cd36161ee Fix stale master detection in multimaster mode
The MySQL monitor now correctly assigns stale status to master servers.
2016-09-12 15:57:27 +03:00
Markus Makela
506ef1b9f6 Assign master status only to root level masters
If a relay master server is found in the replication tree, it should not
get the master status. Previously all master servers were assigned the
master status regardless of their depth in the replication tree.

By comparing the depth value of each potential master, the monitor can
find the right master at the root of the replication tree.
2016-09-12 15:57:27 +03:00