Commit Graph

744 Commits

Author SHA1 Message Date
6db3cc380b Add cache parameter 'selects' 2017-03-17 09:06:27 +02:00
987a52b398 Further simplify key generation
The original approach was made for RocksDB where it is beneficial
to keep keys of stuff related to each other close to each other.
However, as RocksDB is no longer the primary focus, it just causes
additional cost to dig out the table names.

The key is a 64-bit integer, but crc32 only gives us a 32-bit one.
We create an other 32-bit value by running crc32 over the same SQL,
using the first crc value as adler.

I think that further reduces the chance for clashes:

    uint32_t crc0 = crc32(0, Z_NULL, 0);
    uint32_t crc1;
    uint32_t crc2;

    crc1 = crc32(crc0, "codding", 7) => 1774765869
    crc2 = crc32(crc1, "codding", 7) => 1409592046

    crc1 = crc32(crc0, "gnu", 3)     => 1774765869
    crc2 = crc32(crc1, "gnu", 3)     => 1213798908

Note that the first value is the same, but the second is not.
2017-03-16 18:39:10 +02:00
daf3774329 Use simpler function for calculating hash
Now the key is a 64-bit integer and it is created by the cache
component and can not be unique for the storage.
2017-03-16 16:47:15 +02:00
c235e181ce Null filter no long requires 'capabilities' argument
- If no capabilities are provided, then the filter will have none.
- Now logs its capabilities as startup (useful for debugging).
2017-03-15 10:26:31 +02:00
a58f944f23 Allow query classifier to initialize itself
The process and thread initialization/finalization of the query
classifier plugins is handled using the process and thread
initialization/finalization functions in the module object.

However, the top-level query classifier will also need to perform
process and thread initialization when transaction boundaries are
detected using regular expressions.
2017-03-15 09:35:15 +02:00
d88d02ee95 Fix namedserverfilter source parameter
The `source` parameter was broken by the IPv6 change. Now the filter no
longer uses functions from the core to resolve the network addresses.
2017-03-13 10:45:55 +02:00
37dd561470 Add support for IPv6
Both the listeners and servers now support IPv6 addresses.

The namedserverfilter does not yet use the new structures and needs to be
fixed in a following commit.
2017-03-13 10:45:55 +02:00
b1521e0545 Log the use of prepared statements
If cache decisions should be logged, we log the presence of
prepared statements as they currently are the primary source
for the cache not caching.
2017-03-07 15:45:29 +02:00
09ef292283 Update dbfwfilter build configuration
The filter now uses a more recent syntax to declare the function name
prefixes. This removes a build warning.
2017-03-03 18:44:58 +02:00
da5edadcda MXS-1150: Do not cache non-cachable functions
The resultset of SELECTs that use functions whose result will
always vary or whose result depend upon the user executing the
query should not be cached. The list of functions is the same
as that specified for the query cache of MariaDB:
https://mariadb.com/kb/en/mariadb/query-cache/
2017-03-02 13:13:31 +02:00
923ed9329d Add better cache debugging
Now the statement and the decision is logged. Easier to figure
out why something is or is not cached.
2017-03-02 12:35:53 +02:00
4902ed7cf5 MXS-1137: Do not cache when [user|system] vars are used
If user or system variables are used in a SELECT statement, then
the result will not be cached. That ensures that the wrong result
will not be returned.
2017-03-02 12:35:53 +02:00
aa51528481 Factor out caching decision making 2017-03-02 12:35:53 +02:00
1d48a24d6a MXS-1149: removed send_ok_upstream() routine
send_ok_upstream is no longer in use, just remove it
2017-03-01 14:53:37 +01:00
c9592aa05e MXS-1149: maxrows should return empty resultset instead of OK
maxrows returns empty resultset instead of OK.

The response with coulmndefs is sent then an EOF without flags is added.

Limitations: in case of MULTI_RESULT sets only the first one coulmndefs
is sent with EOF
2017-03-01 14:48:35 +01:00
eefa63be33 Use qc_query_is_type 2017-03-01 14:46:35 +02:00
82049b8ed1 Use cache as long as a trx only performes reads
As before, the cache will be used if there is no ongoing transaction
(includes autocommit being on), or if there is an explicitly read-
only transaction.

In addition, the cache will be used and populated during any other
transaction as long as only pure read statements are executed. After
first non-read statement, the use of the cache is disabled.
2017-03-01 11:08:52 +02:00
f1ea0b03d1 Change order of testing whether caching should be used
First we check the transaction state, then we check whether it is
a SELECT. In the next step, we will widen the cases when caching
is used.
2017-03-01 11:08:52 +02:00
7bf7b1a47a Simplify logic for when deciding whether to cache
The autocommit state is now included in the result returned
from session_trx_is_active(), so it need not be checked separately.

This is preparatory work for MXS-1136 that will change the approach
so that the cache is used and populated in a non readonly transaction
*until* the first non-read statement is encountered.
2017-03-01 11:08:52 +02:00
d617a0b7b9 Move testfilter, router etc to examples
Also changed Plugin devel manual to match. Modified roundrobinrouter
to use static_cast due to the change in router api definitions.
2017-02-28 17:51:48 +02:00
272896b50e Fix luafilter build failure
The filter used the old getCapabilities signature.
2017-02-16 10:16:10 +02:00
49cc2b52e3 Merge branch '2.1.0' into 2.1 2017-02-15 08:44:55 +02:00
5648f708af Update license to BSL 1.1 2017-02-14 21:42:28 +02:00
3a3b31ebb4 NullFilter capabilities are now instance specific
There can now in one configuration be multiple Null filters,
each with its own set of capabilities.
2017-02-14 13:55:50 +02:00
2a49cd6451 getCapabilities now get the instance as argument
Allows the capabilities to be different depending on how the
filter/router has been configured.
2017-02-14 13:55:22 +02:00
b46bc47c87 Add NullFilter filter module
NullFilter is a filter module that does nothing, except reports
capabilities as defined in the configuration file. It's purpose
is only to make it simple to benchmark the performance impact
various routing capabilities have.

Note that since getCapabilities() currently does *not* take an
instance pointer as parameter, all NullFilter instances will
report the same capabilities, the ones specified for the last
filter to have been loaded.
2017-02-09 15:02:34 +02:00
ce5cd69eb3 Remove unused locks and variables
Removed unused spinlocks from DCBs, sessions and the MySQL protocol
structs. They were used in a context where only one thread has access to
the structure.

Removed unused member variables from DCBs.
2017-02-08 15:31:17 +02:00
9f88ee3eec Errors must not be cached
For instance, if bob is returned an error because he does not have
the required grants, then if the error were cached, alice would
receive bob's error reply even if she has the required grants.
2017-02-06 09:41:37 +02:00
4ccfc35b00 Fix regression of MXS-1111
The function that was added to detect the commands that should always be
allowed wasn't used.
2017-02-02 13:50:44 +02:00
13997fc5b9 Fix prepared statements being rejected by dbfwfilter
The merge with 2.0 caused the dbfwfilter to reject all prepared statements
when they are supported in 2.1.
2017-02-02 13:34:57 +02:00
a2ffb0476f Fix insertstream value implicit insert detection
The pointer being at the end of the buffer was not checked.
2017-02-02 12:58:30 +02:00
3a447607a7 Move inserstream into the core
The filter is a part of 2.1 so it should be in the core.

Added a note into the documentation about the experimental status.
2017-02-01 21:08:25 +02:00
4ef4cb490b Cache: Use consistent naming
- snake_case
- member variables prefixed with m_
- static member variables prefixed with s_
- where prefixes are used (z,p,s), the following character
  is capitalized
2017-02-01 10:43:41 +02:00
f2da434b56 Fix merge problem with dbfwfilter
Code was duplicated in the merge and the old function names were used.
2017-02-01 09:58:38 +02:00
e64a641bcd Merge branch '2.0' into develop 2017-02-01 09:35:13 +02:00
00f16e1fa5 Make parameters for command_is_mandatory const
The parameters can be const as they aren't modified.
2017-01-31 10:16:33 +02:00
fdee329198 MXS-1111: Allow COM_PING and other commands to pass the firewall
The firewall filter should allow COM_PING and other similar commands to
pass through as they are mainly used to check the status of the backend
server or to display statistics. The COM_PROCESS_KILL is the exception as
it affects the state of the backend server. This is better controlled with
permissions in the server than in the firewall filter.

Commands that require special grants aren't allowed to pass as they are
mainly for maintenance purposes and these should not be done through the
firewall.
2017-01-30 19:16:33 +02:00
03067b586c Cache: Install storage implementations as well
As the cache is marked as being core, the storage modules needs
to be marked as core as well.
2017-01-30 14:23:54 +02:00
4adc9d705b Build Jansson if library not found
If the Jansson JSON library is not found on the system, it is built from
source.
2017-01-27 12:19:27 +02:00
8b83bf834e Clean up router.h
Nothing moved to core, just rename and cleanup.
2017-01-26 16:14:17 +02:00
aa8851fbe1 Remove listmanager
The file wasn't used, so any traces of listmanager.h or .c have been
erased.
2017-01-25 16:11:12 +02:00
7d51864402 Clean config.h some more
Moved some typedefs to router.h and server.h, changed a few
constants to these enums. Renamed some types in config.h to
remove "Gateway".

There are still some functions in the public header which are
only used in core, but they seem to fit the theme of public functions
so were not moved.
2017-01-25 16:05:51 +02:00
126d9be749 Remove noisy logging 2017-01-23 15:31:01 +02:00
47d21bd519 Masking: Change "rules_file" parameter to "rules"
The firewall filter and the cache have similar rules files and
they use "rules". To be consistent, the masking filter should
use "rules" as well.
2017-01-23 15:17:23 +02:00
ead0d4fe3e No point storing version info in transient storage
As the RocksDB storage is re-created at each startup it does not
make sense to store version information in it.
2017-01-23 15:09:13 +02:00
6a90b0c629 Ok to continue even if storage directory is created at startup
Due to some refactoring that possibility had been lost.
2017-01-23 15:08:57 +02:00
eb05132a5c Cache: Use MXS_MODULE_PARAM_SIZE 2017-01-23 11:01:57 +02:00
50f594fa4e Fix ccrfilter parameters
The filter used the value of `count` for the `time` parameter.

The `options` parameter used the old-style parsing of the options and the
`match` and `exclude` parameters didn't use config_copy_string.
2017-01-22 09:24:00 +02:00
03402a86ed Check that ccrfilter time is enabled before using it
The filter calculated a time difference which was negative if the filter
was not configured to use time intervals.

Added more info level logging to help determine how the filter behaves.
2017-01-22 09:24:00 +02:00
4e6fde8a23 Use query type instead of operation for ccrfilter
The query operation not being a select is too strict and not very
precise. Only queries that actually modify data should trigger the adding
of routing hints.
2017-01-22 09:24:00 +02:00