Commit Graph

472 Commits

Author SHA1 Message Date
9d3e5a715b Cache: Remove item if putting fails
If an item cannot be put to the database, it is explicitly removed
to ensure that we cannot have the situation that a stale item is
continuously returned because the updating of the value fails for
whatever reason.
2016-11-23 12:15:58 +02:00
e9029b183e Cache: Allow entries to be explicitly deleted 2016-11-23 12:15:58 +02:00
2ecd5f3340 Prevent stale item from being fetched more than once
If a cached entry becomes stale, then if no extra measures are
taken then every thread hitting that item while it is being fetched
from the server will also refresh it, even though it obviously is
sufficient that one does it.

Now the knowledge that the item is being refreshed is recorded,
so that all other clients are simply returned the stale item. That
way, we'll hit the server once per stale item.
2016-11-23 12:15:58 +02:00
53413a3260 MXS-929: Add reloading of rules to dbfwfilter
The rules and users are stored in thread-local pointers which removes the
need to hold global locks. This allows greater scalability but causes a
slightly larger memory footprint. Usually the increase in memory
consumption is trivial compared to the benefits in scaling.

Using a per-thread rule and user list allows changes to be applied
immediately without locking on the instance level.

The updating of the rules uses the new functinality described in the
debugcmd.h header. The module registers two functions at startup; one for
reloading rules and one for printing them. These custom functions can be
seen in the `maxadmin list functions` output.
2016-11-23 11:14:31 +02:00
79a03049a1 Enable fetching of stale items
It is now possible to specify that a cache item should be returned
if it exists, even if it is stale.
2016-11-22 11:15:26 +02:00
97b039689f Disable write-ahead log
Since it's a cache, we do not need to retain the data from one
MaxScale invocation to the next. Consequently, we can turn off
write ahead logging completely if we simply wipe the RocksDB
database at each startup. In addition, the location of the
cache directory can now be specified explicitly, so it can be
placed, for instance, on a RAM disk.
2016-11-22 08:44:00 +02:00
a018e732eb Make the dbfwfilter rule check a separate utility
The code for the utility is now stored in a separate file. This also
removes the need to include testing headers from other directories.

Also added a function to reload rules that uses the newly modified rule
parsing mechanism. This can be used later on to update the rules at
runtime.
2016-11-18 08:59:52 +02:00
8ae76e3ced Clean up dbfwfilter
Removed unused functions, made function names more consistent.
2016-11-18 08:53:43 +02:00
042491ac07 Rename dbfwfilter objects to more descriptive ones
The object names in the dbfwfilter weren't very descriptive and were not
that easy to comprehend. The same goes for function names which had
somewhat cryptic names.
2016-11-17 12:36:44 +02:00
c4999232ce MXS-935: Implement column matching
With the advent of qc_get_field_info, columns can now be matched.
However, there is still some undeterminism caused by the table
information not containing contextual information (exactly where
is the table used).

Further, suppose table X contains the column A and table Y contains
the column B, then given a statement like

    SELECT a, b from X, Z;

we cannot know whether a is in X or Z, or b in X or Z, without being
aware of the schema, which we currently are not.

Consequently, as long as MaxScale is not aware of the schema, some
heuristics must be applied. For instance, if exactly one table is
referred to, then we can assume that columns that are not explicitly
qualified are from that table.

The rule tests are currently rather rudimentary and need to be
expanded.
2016-11-16 10:44:56 +02:00
2334c663e0 Merge branch 'develop' into MXS-936 2016-11-10 15:42:09 +01:00
6fe9fda46e MXS-935: Add support for column based rule matching
In addition, test program extended to be able to test the
rule matching. Tests to be extended in a subsequent commit.
2016-11-10 11:24:10 +02:00
da152353dd Merge branch 'develop' into MXS-936 2016-11-09 16:21:40 +01:00
4e36e86820 MXS-936: destroyInstance added for filters
MXS-936: destroyInstance added for filters
2016-11-08 17:09:49 +01:00
ef79686a3a Add option to use a unified log file for qlafilter
The qlafilter now has an option to log all messages to a single
file instead of session-specific files. Session ids are printed
to the beginning of the line when using this mode. Documentation
updated to match. Also, added an option to flush after every
write.
2016-11-08 15:31:31 +02:00
7358d932e6 Fix maxrows CMakeLists.txt
There was an add_subdirectory call to a nonexistent `test` directory.
2016-11-08 09:12:47 +02:00
eaef39c008 Merge branch 'develop' into MXS-930 2016-11-07 16:21:06 +01:00
c36e5f6ba4 C_DEBUG removed and added a check for packetlen < MAXROWS_EOF_PACKET_LEN
C_DEBUG removed.
A check is now made for packetlen < MAXROWS_EOF_PACKET_LEN.
An empty result set, OK, is send in such situation
2016-11-07 16:19:38 +01:00
9fa1a0cfec Replace qc_get_affected_fields with qc_get_field_info
qc_get_affected_fields that returned the fields as one string with
fields interspared with spaces, will be removed.
2016-11-07 11:03:22 +02:00
9f01a4612d EOF flags are now read in case 0xfe
EOF flags are now read in case 0xfe and minor improvements
2016-11-07 10:01:05 +01:00
5b7e5aa36f SERVER_MORE_RESULTS_EXIST is now detected
The filter can detect SERVER_MORE_RESULTS_EXIST which means the server
is sending more result sets: example:

DROP PROCEDURE IF EXISTS multi;
DELIMITER $$
CREATE PROCEDURE multi() BEGIN
  SELECT 1;
  SELECT id FROM t2 limit 40;
  set @a=4;
  SELECT 2;
END$$
DELIMITER ;

MySQL> call multi()
2016-11-04 15:48:14 +01:00
094586adbc Merge branch 'develop' into MXS-930 2016-11-04 15:29:40 +01:00
45f463c438 Require GCC 4.8 for RocksDB
GCC 4.8 is the first GCC version that fully implements C++11
2016-11-03 09:27:58 +02:00
02d28a8d8d Always use format strings in dcb_printf
The luafilter didn't use a format string with dcb_printf which can lead to
unexpected results if the returned string contains printf special
characters.
2016-11-03 09:23:36 +02:00
8a6e3fb49f Merge branch 'develop' into MXS-930 2016-11-02 08:57:40 +01:00
5a1af968d2 Take contiguous input capability into use in filters
The filters which require contiguous input now return the
RCAP_TYPE_CONTIGUOUS_INPUT capability flag.
2016-10-31 08:38:05 +02:00
bbade8b736 PCRE2_CODE_UNIT_WIDTH must not be defined. 2016-10-27 15:26:41 +03:00
6cc7a26582 Cache: Add forgotten change 2016-10-27 13:54:22 +03:00
dcfd5b7601 maxrows now handles SHOW statements and SELECT in prepared statements
maxrows now handles SHOW statements and SELECT in prepared statements
2016-10-27 11:34:01 +02:00
0cfb82f146 The MAXROWS_DISCARDING_RESPONSE is handled differently
The MAXROWS_DISCARDING_RESPONSE is handled differently: the OK packet
is sent only after an EOF is seen in a reply even with multiple packet
transmission from the backend
2016-10-27 10:53:07 +02:00
199dd51312 Cache: C_DEBUG removed and debug option extended
The development time C_DEBUG is now removed, and replaced with
a debug option that allows top-level decisions to be logged.
2016-10-27 11:10:36 +03:00
c73c3b390a Addition of MAXROWS_DISCARDING_RESPONSE
Addition of MAXROWS_DISCARDING_RESPONSE

Packets received after sent OK are consumed and not sent
2016-10-27 09:25:08 +02:00
59b3a1fe36 Handle max_resultset_size in maxrows
Handle max_resultset_size in maxrows
2016-10-26 19:20:25 +02:00
7419ddceca Code review update
Code review update
2016-10-26 19:12:56 +02:00
5aa0c32e8f Minor changes
Minor changes
2016-10-26 18:58:07 +02:00
48af3d21b6 max rows filter first implementation
max rows filter first implementation

Resultsets with more than max_resultset_rows

will be skipped and empty result set is returned to client.
Not yet tested with multi_statements
[MaxRows]
type=filter
module=maxrows
max_resultset_rows=10
debug=15
2016-10-26 15:58:06 +02:00
8391579206 Update cache with new capabilities
Now that a filter can express that the transaction state is tracked,
the cache implementation can be simplified. We do not need to cater
for the case that a "too short" or "too long" a packet would be
delivered.

Further, since the autocommit mode and transaction state of the session
are tracked, the filter can cache data when it is safe to do so. In
practice that means when either AUTOCOMMIT is ON and no explicit
transaction is active or when a READ ONLY transaction is active,
irrespective of the autocommit state.

In principle it would be possible to tentatively cache data during
a transaction, and if the transaction is committed successfully
flush the tentatively cached data to the actual cache, but that will
be for another day.
2016-10-25 16:21:31 +03:00
db2cccbd8f Replace getCapabilities() with getCapabilities(void)
The latter means no args, the former means any args.
2016-10-24 11:19:43 +03:00
59a4152d8a Add getCapabilities to filters
Common capabilities are now defined in routing.h. The common
capabilities can be defined using bits 0 - 15.

Router capabilities are defined using bits 16-31 and filter
capabilities (should there ever be such) using bits 32-47.

So, to find out the capabilities of a service you only need to
OR the capabilities of the router and all filters together.

For instance, if a single filter needs statement based routing,
then that is what is done.
2016-10-24 11:19:43 +03:00
348143eb6c Remove too aggressive assert 2016-10-19 11:00:26 +03:00
48b1583830 Add FindJansson.cmake
Added find_package module for CMake which locates the Jansson libraries
and headers. This will make dependency checking easier and prevents build
failures due to missing dependencies.
2016-10-19 09:35:09 +03:00
1333da0712 Remove skygw_utils.h
The general purpose stuff in skygw_utils.h was moved to utils.h
and the corresponding implementation from skygw_utils.cc to utils.c.
Includes updated accordingly.

Skygw_utils.h is now only used by log_manager and by mlist, which
is only used by log_manager. Consequently, skygw_utils.h was moved
to server/maxscale.

Utils.h needs a separate overhaul.
2016-10-14 19:50:54 +03:00
03dbc6df80 Remove dependency on skygw_utils.h
- STRERROR_BUFLEN moved to cdefs.h and renamed to MXS_STRERROR_BUFLEN.
  Better would be to provide a 'const char* mxs_strerror(int errno)'
  that would have a thread specific buffer for the error message.
- MIN and MAX also moved to defs.h as MXS_MIN and MXS_MAX.
- Now only mlist.h of the headers depend upon skygw_utils.h.
2016-10-14 15:42:02 +03:00
bff2cfb7e5 Remove skygw_types.h
Some stuff moved to maxscale/utils.h. Further cleanup still needed.
2016-10-14 14:24:31 +03:00
0800bf4a59 Rename skygw_debug.h to debug.h
Also remove unneded includes in debug.h
2016-10-14 13:48:10 +03:00
1a978be6b6 Cleanup header files
- All now include maxscale/cdefs.h as the very first file.
- MXS_[BEGIN|END]_DECLS added to all C-headers.
  Strictly speaking not necessary for private headers, but
  does not hurt either.
- Include guards moved to the very top of the file.
- #pragma once added.
2016-10-14 11:54:37 +03:00
2fe58ebe5b Move test_utils.h to server/core/test 2016-10-13 22:59:39 +03:00
d04cb54b9f maxscale/maxscale_pcre2.h renamed to maxscale/pcre2.h 2016-10-13 22:59:39 +03:00
76430e060f maxconfig.h renamed to config.h 2016-10-13 22:59:39 +03:00
3a18b64bdd mysql_client_server_protocol.h moved.
Moved to include/maxscale/protocol/mysql.h
2016-10-13 22:59:39 +03:00