Commit Graph

757 Commits

Author SHA1 Message Date
0c3f9ffa2e Maxrows: documentation update and use of MXS_MODULE_PARAM_SIZE for max_resultset_size parameter
max_resultset_size parameter now uses MXS_MODULE_PARAM_SIZE and
config_get_size

Documentation change follows the change.
2017-04-04 08:43:55 +02:00
815780aa4a In cache testrules.c ensure there seems to be at least one thread 2017-04-03 14:20:54 +03:00
e6d2c96f5b MXS-1211: maxrows should be configurable to return error when limit has been exceeded
New parameter added to maxsrows filter:
max_resultset_return=empty|error|ok
Default, 'empty' is to return an empty set, as the current
implementation.
'err' will return an ERR reply with the input SQL statement
'ok' will return an OK packet
2017-03-31 14:26:44 +02:00
cbc1e864d9 Use RFC 3986 compliant addresses in log messages
When log messages are written with both address and port information, IPv6
addresses can cause confusion if the normal address:port formatting is
used. The RFC 3986 suggests that all IPv6 addresses are expressed as a
bracket enclosed address optionally followed by the port that is separate
from the address by a colon.

In practice, the "all interfaces" address and port number 3306 can be
written in IPv4 numbers-and-dots notation as 0.0.0.0:3306 and in IPv6
notation as [::]:3306. Using the latter format in log messages keeps the
output consistent with all types of addresses.

The details of the standard can be found at the following addresses:

     https://www.ietf.org/rfc/rfc3986.txt

     https://www.rfc-editor.org/std/std66.txt
2017-03-31 14:12:58 +03:00
4b2d94d0f8 When a transaction has been started, no other checks are needed
If we do check then we will end up turning off the read only flag
that just was set on.
2017-03-28 16:17:13 +03:00
f2734f895e Revert "Update dbfwfilter build configuration"
This reverts commit 09ef292283311483354e6cd628318a5cafd96b2c. The old
syntax is still required on SLES 11.
2017-03-27 10:01:54 +03:00
d0a9571da0 Introduce new qc_parse() prototype
It is now possible to specify what information the caller is interested
in. With this the cost for collecting information during the query parsing
that nobody is interested in can be avoided.
2017-03-22 11:09:36 +02:00
15a22ec295 Do not use shared PCRE2 match data
The match data needs to be unique for each thread, so for the time
being it is created whenever it is needed. A more performant (although
possibly to a negigible amount) solution would be to have a separate
match data for each thread, but that will have to wait for 2.2.
2017-03-21 15:21:13 +02:00
117534d288 Use JIT in regex matching if available 2017-03-21 15:21:13 +02:00
17057ef340 Use thread specific pcre2 match data
For 2.2 a worker/thread object need to be passed around so that
not everyone need to create their own thread id mechanism.
2017-03-21 15:21:13 +02:00
b81c86f228 A rules object may be empty 2017-03-21 10:35:30 +02:00
ed06b6a331 The cache storage storage_inmemory is now the default 2017-03-17 13:45:55 +02:00
9c3aaddda1 Improve cache performance
- Selects are picked out using custom parsing, so if a statement is
  anything else but a SELECT, the cache will never cause the statement
  to be parsed.
- The setting of of the cache parameter `selects` is taken into account.
  If it is `assume_cacheable` then the statement will also not be parsed
  even if it is a SELECT.
2017-03-17 12:22:00 +02:00
6db3cc380b Add cache parameter 'selects' 2017-03-17 09:06:27 +02:00
987a52b398 Further simplify key generation
The original approach was made for RocksDB where it is beneficial
to keep keys of stuff related to each other close to each other.
However, as RocksDB is no longer the primary focus, it just causes
additional cost to dig out the table names.

The key is a 64-bit integer, but crc32 only gives us a 32-bit one.
We create an other 32-bit value by running crc32 over the same SQL,
using the first crc value as adler.

I think that further reduces the chance for clashes:

    uint32_t crc0 = crc32(0, Z_NULL, 0);
    uint32_t crc1;
    uint32_t crc2;

    crc1 = crc32(crc0, "codding", 7) => 1774765869
    crc2 = crc32(crc1, "codding", 7) => 1409592046

    crc1 = crc32(crc0, "gnu", 3)     => 1774765869
    crc2 = crc32(crc1, "gnu", 3)     => 1213798908

Note that the first value is the same, but the second is not.
2017-03-16 18:39:10 +02:00
daf3774329 Use simpler function for calculating hash
Now the key is a 64-bit integer and it is created by the cache
component and can not be unique for the storage.
2017-03-16 16:47:15 +02:00
c235e181ce Null filter no long requires 'capabilities' argument
- If no capabilities are provided, then the filter will have none.
- Now logs its capabilities as startup (useful for debugging).
2017-03-15 10:26:31 +02:00
a58f944f23 Allow query classifier to initialize itself
The process and thread initialization/finalization of the query
classifier plugins is handled using the process and thread
initialization/finalization functions in the module object.

However, the top-level query classifier will also need to perform
process and thread initialization when transaction boundaries are
detected using regular expressions.
2017-03-15 09:35:15 +02:00
d88d02ee95 Fix namedserverfilter source parameter
The `source` parameter was broken by the IPv6 change. Now the filter no
longer uses functions from the core to resolve the network addresses.
2017-03-13 10:45:55 +02:00
37dd561470 Add support for IPv6
Both the listeners and servers now support IPv6 addresses.

The namedserverfilter does not yet use the new structures and needs to be
fixed in a following commit.
2017-03-13 10:45:55 +02:00
b1521e0545 Log the use of prepared statements
If cache decisions should be logged, we log the presence of
prepared statements as they currently are the primary source
for the cache not caching.
2017-03-07 15:45:29 +02:00
09ef292283 Update dbfwfilter build configuration
The filter now uses a more recent syntax to declare the function name
prefixes. This removes a build warning.
2017-03-03 18:44:58 +02:00
da5edadcda MXS-1150: Do not cache non-cachable functions
The resultset of SELECTs that use functions whose result will
always vary or whose result depend upon the user executing the
query should not be cached. The list of functions is the same
as that specified for the query cache of MariaDB:
https://mariadb.com/kb/en/mariadb/query-cache/
2017-03-02 13:13:31 +02:00
923ed9329d Add better cache debugging
Now the statement and the decision is logged. Easier to figure
out why something is or is not cached.
2017-03-02 12:35:53 +02:00
4902ed7cf5 MXS-1137: Do not cache when [user|system] vars are used
If user or system variables are used in a SELECT statement, then
the result will not be cached. That ensures that the wrong result
will not be returned.
2017-03-02 12:35:53 +02:00
aa51528481 Factor out caching decision making 2017-03-02 12:35:53 +02:00
1d48a24d6a MXS-1149: removed send_ok_upstream() routine
send_ok_upstream is no longer in use, just remove it
2017-03-01 14:53:37 +01:00
c9592aa05e MXS-1149: maxrows should return empty resultset instead of OK
maxrows returns empty resultset instead of OK.

The response with coulmndefs is sent then an EOF without flags is added.

Limitations: in case of MULTI_RESULT sets only the first one coulmndefs
is sent with EOF
2017-03-01 14:48:35 +01:00
eefa63be33 Use qc_query_is_type 2017-03-01 14:46:35 +02:00
82049b8ed1 Use cache as long as a trx only performes reads
As before, the cache will be used if there is no ongoing transaction
(includes autocommit being on), or if there is an explicitly read-
only transaction.

In addition, the cache will be used and populated during any other
transaction as long as only pure read statements are executed. After
first non-read statement, the use of the cache is disabled.
2017-03-01 11:08:52 +02:00
f1ea0b03d1 Change order of testing whether caching should be used
First we check the transaction state, then we check whether it is
a SELECT. In the next step, we will widen the cases when caching
is used.
2017-03-01 11:08:52 +02:00
7bf7b1a47a Simplify logic for when deciding whether to cache
The autocommit state is now included in the result returned
from session_trx_is_active(), so it need not be checked separately.

This is preparatory work for MXS-1136 that will change the approach
so that the cache is used and populated in a non readonly transaction
*until* the first non-read statement is encountered.
2017-03-01 11:08:52 +02:00
d617a0b7b9 Move testfilter, router etc to examples
Also changed Plugin devel manual to match. Modified roundrobinrouter
to use static_cast due to the change in router api definitions.
2017-02-28 17:51:48 +02:00
272896b50e Fix luafilter build failure
The filter used the old getCapabilities signature.
2017-02-16 10:16:10 +02:00
49cc2b52e3 Merge branch '2.1.0' into 2.1 2017-02-15 08:44:55 +02:00
5648f708af Update license to BSL 1.1 2017-02-14 21:42:28 +02:00
3a3b31ebb4 NullFilter capabilities are now instance specific
There can now in one configuration be multiple Null filters,
each with its own set of capabilities.
2017-02-14 13:55:50 +02:00
2a49cd6451 getCapabilities now get the instance as argument
Allows the capabilities to be different depending on how the
filter/router has been configured.
2017-02-14 13:55:22 +02:00
b46bc47c87 Add NullFilter filter module
NullFilter is a filter module that does nothing, except reports
capabilities as defined in the configuration file. It's purpose
is only to make it simple to benchmark the performance impact
various routing capabilities have.

Note that since getCapabilities() currently does *not* take an
instance pointer as parameter, all NullFilter instances will
report the same capabilities, the ones specified for the last
filter to have been loaded.
2017-02-09 15:02:34 +02:00
ce5cd69eb3 Remove unused locks and variables
Removed unused spinlocks from DCBs, sessions and the MySQL protocol
structs. They were used in a context where only one thread has access to
the structure.

Removed unused member variables from DCBs.
2017-02-08 15:31:17 +02:00
9f88ee3eec Errors must not be cached
For instance, if bob is returned an error because he does not have
the required grants, then if the error were cached, alice would
receive bob's error reply even if she has the required grants.
2017-02-06 09:41:37 +02:00
4ccfc35b00 Fix regression of MXS-1111
The function that was added to detect the commands that should always be
allowed wasn't used.
2017-02-02 13:50:44 +02:00
13997fc5b9 Fix prepared statements being rejected by dbfwfilter
The merge with 2.0 caused the dbfwfilter to reject all prepared statements
when they are supported in 2.1.
2017-02-02 13:34:57 +02:00
a2ffb0476f Fix insertstream value implicit insert detection
The pointer being at the end of the buffer was not checked.
2017-02-02 12:58:30 +02:00
3a447607a7 Move inserstream into the core
The filter is a part of 2.1 so it should be in the core.

Added a note into the documentation about the experimental status.
2017-02-01 21:08:25 +02:00
4ef4cb490b Cache: Use consistent naming
- snake_case
- member variables prefixed with m_
- static member variables prefixed with s_
- where prefixes are used (z,p,s), the following character
  is capitalized
2017-02-01 10:43:41 +02:00
f2da434b56 Fix merge problem with dbfwfilter
Code was duplicated in the merge and the old function names were used.
2017-02-01 09:58:38 +02:00
e64a641bcd Merge branch '2.0' into develop 2017-02-01 09:35:13 +02:00
00f16e1fa5 Make parameters for command_is_mandatory const
The parameters can be const as they aren't modified.
2017-01-31 10:16:33 +02:00
fdee329198 MXS-1111: Allow COM_PING and other commands to pass the firewall
The firewall filter should allow COM_PING and other similar commands to
pass through as they are mainly used to check the status of the backend
server or to display statistics. The COM_PROCESS_KILL is the exception as
it affects the state of the backend server. This is better controlled with
permissions in the server than in the firewall filter.

Commands that require special grants aren't allowed to pass as they are
mainly for maintenance purposes and these should not be done through the
firewall.
2017-01-30 19:16:33 +02:00