Commit Graph

2905 Commits

Author SHA1 Message Date
9f08e1300f Make Cache/Storage const correct
- From the outside, all actions getting data from the cache/storage
  are const.
- Update documentation.
2016-12-15 10:17:39 +02:00
e5cd9ba47e Refactor testrawstorage slightly
Different storage tests will be added to the same file.
2016-12-15 10:17:39 +02:00
f9a7edc7d2 Fix tee filter
With the addition of filter capabilities, the tee filter should work with
all sorts of routers that require at most the RCAP_TYPE_CONTIGUOUS_INPUT
capability.

Due to a recent discovery of the server's capability to process multiple
requests, the filter can safely send data from one service to another
without waiting for the earlier replies.

This also fixes a minor problem with the cloning of DCBs where the backend
DCBs could end up in the wrong thread's pool.
2016-12-14 23:55:33 +02:00
916320cf67 Clean up maxadmin help output
The detailed output of the new help command was a bit too densely packed
for some commands.

Added missing values for the `alter server` command help output.
2016-12-14 11:23:17 +02:00
7047cdf6c6 MXS-867: Key file Key Management for binlog encryption
Binlog server option ‘encryption_key_file=’ can now use the same key
file the MariaDB 10.1 server might have  in my.cnf:
‘file_key_management_filename=‘

Note: the file content must be in clear, no key encryption.
2016-12-13 15:51:43 +01:00
3872cebd09 MXS-1048: Handle backquoted database names
The schemarouter didn't expect backquoted database names for SHOW TABLE
and USE statements.
2016-12-13 13:51:46 +02:00
5e2cbf3809 Cache: Provide access to head, tail, size, items
In order to be able to test the LRU mechanism properly you need
to be able to access the head and tail from the outside. The same
is regarding the size and items in the cache. In order to be able
to test that the guarantees are upheld, you need to be able to
access those values from the outside.
2016-12-13 12:37:28 +02:00
80c77413ba Cache: Do not use random indexes when testing
It would seem that the likelyhood of different threads accessing
the same items at the same time is greater if each thread continuosly
loops across all items from beginning to finish. That will also ensure
that head and tail surely are accessed. In addition, some function names
have been disambiguated.
2016-12-13 12:37:28 +02:00
75b2895ec2 Cache: If LRUStorage is used, create ST real storage
If the underlying storage does not support max_count and/or max_size
and it accordinly is decorated with LRUStorage, then create the real
storage as single-threaded. Since LRUStorage will do locking it is of
no use to do locking in the real storage as well.
2016-12-13 12:37:28 +02:00
651e1122c2 Fix minor issues in testrawstorage
- Release memory
- Set libdir
- Adjust output
2016-12-13 12:37:28 +02:00
f71be685b6 Cache: Test raw storage
Here we create a number of threads and then randomly start getting
putting and deleting values. The intent is to test that the locking
behaviour of the storage modules is correctly implemented.
2016-12-13 12:37:28 +02:00
3cbfd3f1d0 Cache: Create MT storage if that is asked for 2016-12-13 12:37:28 +02:00
8bf1e50479 Cache: Provide access to raw storage
In order to be able to test the underlying storage separately
direct access to it must be provided (without intermediate LRU
storage).
2016-12-13 12:37:28 +02:00
423b54ef82 Cache: 0 is now default of integer parameters
0 is now the default of all cache configuration parameters and in
all cases the meaning is the same; that is, no limit. Internally
all limits but ttl are now for the sake of consistency 64-bit.
2016-12-13 12:37:28 +02:00
e1a1c8b1cd Cache: Add key generation test
Smoke test for detecting errors in key generation. The input file
is a number of .test-files from the server combined into a single
one. We simply check that a unique key is generated for each
statement.
2016-12-13 12:37:28 +02:00
4239182aa0 Fix diagnostics on filter template
Diagnostics can be required on both instance and session level,
so both cases need to be handled explicitly.

In addition, some reinterpret_casts were changed into static_casts.
Reinterpret_cast needs to be used with the instance, which is a
void** but static_cast is sufficient for the session, which is void*.
2016-12-13 12:28:31 +02:00
7a04259fc0 MXS-756: Retry reads on slave failure
If a slave fails while a non-critical read is being executed, the read is
retried on a different server. This is controlled by the new
`retry_failed_reads` option.

Only selects done that are done outside of a transaction and with
autocommit enabled are retried.
2016-12-13 09:12:58 +02:00
a66c8fb919 Fix crash on NULL client DCB in mysql_backend.c
It is possible that a session is in the dummy state (a transient state)
when a backend write occurs. The check for the client protocol NULL-ness
should extend to the client DCB itself.
2016-12-13 08:28:57 +02:00
ff4d56813e MaxRows filter can handle 16MBytes packets
16 MBytes and larger packet are handled.
Note: a 16MBytes packet is seen as a row by the filter.
2016-12-12 22:00:11 +01:00
259e944b3d Server status changes now happen under a lock
MXS-873 To prevent monitors and MaxAdmin from interfering with each other,
changes to the server status flags now happen under a lock. To avoid
interfering with monitor logic, the monitors now acquire locks to all
of their servers at the start of the monitor loop and release them
before sleeping.
2016-12-12 15:04:05 +02:00
162ae04d20 MXS-867: Binlog server encryption key has same format as MariaDB key file
Default algorithm is now AES_CBC.
The MariaDB 10.1 key file cannot be read as it might contain more than
1 rows.
2016-12-12 14:01:54 +01:00
5ccdfe54bd Hack binlogrouter to work with resultset buffering
The binlogrouter needs to manipulate the protocol structures in order for
the resultset buffering to work correctly. If the state isn't manipulated
for COM_QUERY statements, the resultsets aren't buffered and will be
routed in separate buffers.
2016-12-12 11:56:41 +02:00
03681db687 Fix crash with binlogrouter
If the client protocol isn't set, the command shouldn't be either.
2016-12-12 11:55:49 +02:00
0b44fe1c04 Only buffer result sets for COM_QUERY statements
When a non-COM_QUERY statement is sent, the result set should not be
buffered.
2016-12-12 11:39:54 +02:00
106f482f45 Add result set buffering to MySQLBackend
The backend MySQL protocol module now supports a new routing capability
which allows result sets to be gathered into one buffer before they are
routed onward. This should not be used by modules that expect large
result sets as the result set is buffered in memory.

Adding a limit on how large of a result set could be buffered would allow
relatively safe use of this routing capability without compromising the
stability of the system.
2016-12-12 10:48:53 +02:00
cb218804ef Detect double monitoring of servers
Adding a server to multiple monitors is forbidden. This should be detected
and reported to the end user.

The information provided by the config_runtime system to the client isn't
as detailed as it could be. Some sort of an error message stack should be
added so that client facing interfaces could properly report the reason
for the failure. Currently the only way to detect the reason of the
failure is to parse the log files.
2016-12-12 10:48:53 +02:00
15a8675fca Mark error handler as called for closed sessions
When a DCB error occurs, the handleError entry point of the routers is
called. The caller of this entry point expects that the error handler
marks the DCB as handled. The aforementioned behavior is wrong as the
error handler should not keep track of whether the handler was already
called.
2016-12-11 21:41:38 +02:00
9519be5e60 Cache: Use auto_ptr in storage_inmemory 2016-12-09 13:08:10 +02:00
16fad9c2cd Cache: Make storage_rocksdb exception safe
Now RocksDBStorage is exception safe in its own right, so exception
guards are only needed at the C/C++ boundary.
2016-12-09 13:08:10 +02:00
bf35577941 Cache: Take MXS_EXCEPTION_GUARD into use 2016-12-09 13:08:10 +02:00
ce94170eae Rename maxscale/cpp.hh to maxscale/cppdefs.hh
Just like every c-header must include maxscale/cdefs.h as the first
file, every C++-header must include maxscale/cppdefs.hh as its first
header.
2016-12-09 13:08:10 +02:00
f871664796 Cache: Rename all C++ headers from *.h to *.hh
Also remove some superfluous files and old include guards.
2016-12-09 13:08:10 +02:00
3877d7bc5e Cache: Derive CacheFilter class from maxscale::CacheFilter
Now everything needed for cleanly transfer the control between the
C filter API of MaxScale and a C++ filter implementation is handled
automatically. Nothing of the earlier boiler-plate code is needed.
2016-12-09 13:08:10 +02:00
c421026ce9 Cache: Minor cleanup 2016-12-09 13:08:10 +02:00
e574af7e6e Cache: Derived CacheFilterSession from maxscale::FilterSession 2016-12-09 13:08:10 +02:00
8bbe80df99 Cache: Use MXS_EXCEPTION_GUARD and maxscale::SpinLockGuard 2016-12-09 13:08:10 +02:00
c9bc7a6e76 Cache: Separate C and C++ functionality
cache_storage_api.h contains pure C declarations, while
cache_storage_api.hh contains C++ declarations. Functions dealing
with CACHE_KEY moved to these headers.
2016-12-09 13:08:10 +02:00
528531f3b5 Cache: SessionCache renamed to CacheFilterSession 2016-12-09 13:08:10 +02:00
fcd737a0c9 Cache: It is a hit, also when a stale item is returned
Also add asserts for impossible cases.
2016-12-09 13:08:10 +02:00
d4d40c0b9b MXS-1047: Backport to 2.0
Backported commits 0e50ecb525003b87e8199708008e4a18606c5e54 and
570e12942ba3d2363ec7098dcf1d35c6b68667a1 to 2.0.
2016-12-09 11:13:27 +02:00
570e12942b MXS-1047: Fix batch insert execution
Doing batch inserts though readwritesplit would stall due to the fact that
pending session commands were stored instead of executed immediately.

Session command responses that weren't complete also discarded the partial
event instead of storing it for later use.
2016-12-08 15:09:36 +02:00
0e50ecb525 Detect session command replies with trailing data
When a session command was received, any trailing data was lost even
though an attempt to split is was made.

With this change, each session command reply will be routed individually
and any trailing data is routed separately.
2016-12-08 15:09:36 +02:00
fab7accef3 Cache: Collect RocksDB statistics 2016-12-08 14:58:59 +02:00
90e16d79c5 Cache: Collect storage_inmemory statistics 2016-12-08 14:58:59 +02:00
585c4554c8 Fix binlogrouter regression
Non-replication events were implicitly ignored but this was removed in a
recent change. The code that wasn't previously used didn't break the
replication event handling loop.
2016-12-08 12:24:50 +02:00
66feeb8fbc Fix rwsplit debug assertion
A maximum of zero slaves is an accepted value.
2016-12-08 12:09:06 +02:00
941f0f9001 Cache: Collect LRU statistics 2016-12-08 11:26:46 +02:00
e0bedad46e Cache: Add information facilities
Various components of the cache can now provide information in
the form of a JSON object that then can be further processed.
2016-12-08 11:26:46 +02:00
8f2170c455 Add show command to cache filter
Initial POC implementation that is capable of outputting the
rules of the cache.

In order to do that conveniently, the json object is retained for
the lifetime of the CACHE_RULES object.
2016-12-08 11:26:46 +02:00
d9ced5bbab Fix binlogrouter build failure
The binlogrouter used the wrong variable name.
2016-12-08 10:11:55 +02:00