851 Commits

Author SHA1 Message Date
Markus Mäkelä
2ef9fbc47a MXS-1346: Add uses_function type rule
The uses_function type rule matches when any of the columns given as
values uses a function. With this, columns can be denied from being used
with a function.
2017-08-24 09:46:02 +03:00
Markus Mäkelä
727b4baa86 Fix application and exemption of masking rules to users
The applies_to and exempted values were only evaluated if both of them
were present.
2017-08-18 10:27:28 +03:00
Johan Wikman
de7004cb95 Merge branch '2.1' into develop 2017-08-15 10:31:30 +03:00
geaaru
3485a32731 Fix compilation errors/warning with gcc-5.4.0 (#137)
Warning are like this:
 error: ‘delimiter’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
2017-08-15 09:56:22 +03:00
Johan Wikman
e9b2a560b8 MXS-1360 Make it possible to specify thread stack size
It is now possible to specify the thread stack size to be used,
when a new thread is created. This will subsequently be used
for allowing the stack size to be specified for worker threads.
2017-08-14 15:24:16 +03:00
Markus Mäkelä
b768d3ca76 MXS-1356: Allow tee filter to be disabled at runtime
Disabling the tee filter at runtime is desirable for cases where the
branched service is not always needed. Migrations and pre-production
setups are one where changes to the branched service are expected and
splitting the queries would result in an error.
2017-08-11 04:09:08 +03:00
Markus Mäkelä
ebe9596966 MXS-1345: Allow an empty list of functions
The function type rule now accepts an empty list of functions, allowing
all functions to be blocked by defining an empty function type rule and
setting action=allow.
2017-08-10 09:51:31 +03:00
Markus Mäkelä
d09e0d03bc MXS-1347: Expose unified log file as a module command
The qlafilter exposes the unified log file as a module command that
outputs an array of JSON strings, one for each line in the file. The
command takes two optional parameters, the start and end lines for the log
file.

Further improvements could be done to allow reading of session specific
log files.

Some sort of an authorization scheme for the REST API should be
implemented to limit the possibility of exposing unwanted parts of the
REST API.
2017-08-09 11:39:25 +03:00
Markus Mäkelä
3eb99139f5 Merge branch '2.1' into develop 2017-07-31 15:57:05 +03:00
Markus Mäkelä
e26ec4f8b9 MXS-1329: Only parse text protocol statements
The query classifier should only be used to parse text protocol
statements. The insertstream filter exploited the fact that any statements
that the filter did not expect would be classified as an unknown
commands. This led to repetitive error messages with binary protocol
statements.
2017-07-27 09:26:27 +03:00
Markus Mäkelä
26bf342e6b Merge branch '2.1' into develop 2017-07-24 12:37:25 +03:00
MassimilianoPinto
dc6bc22237 Masking Filter: removed extra constraints which cause some tests to fail
Masking Filter: removed extra constraints which cause some tests to fail
2017-07-24 11:28:32 +02:00
MassimilianoPinto
2e274d0d8b Added virtual destructor in Rule base class
Added virtual destructor in Rule base class
2017-07-20 15:45:14 +02:00
MassimilianoPinto
e9ad2e5627 Masking filter: added new obfuscation routine
Masking filter: added new obfuscation routine
2017-07-18 11:18:32 +02:00
MassimilianoPinto
62fdda8534 Masking filter: unused code is now removed
Masking filter: unused code is now removed
2017-07-17 14:12:55 +02:00
MassimilianoPinto
1c9f4784dc MXS-1302: pass the rule_type to rule_check_database_options() in order to log the proper rule
rule_check_database_options() can now log the right rule type being
checked
2017-07-17 09:01:53 +02:00
MassimilianoPinto
f1b2257383 MXS-1302: the keyword in the Json is now "match" and MatchRule class substitutes CaptureRule
The keyword in the Json is now "match" and MatchRule class substitutes
CaptureRule

"replace": {
"column": "d_code",
"match": "(?<=aaa).*(?=-12)|(?<=-12).*"
},…
2017-07-14 14:21:40 +02:00
MassimilianoPinto
85d7d67ab8 MXS-1302: capture option in use and pcre2_match matches all the patterns
pcre2_match matches all the patterns
2017-07-14 14:21:40 +02:00
MassimilianoPinto
33e1878fe1 MXS-1302: addition of CaptureRule class
Addition of CaptureRule class, derived from Rule class
2017-07-10 14:10:21 +02:00
MassimilianoPinto
f10cc156a5 MXS-1302: added tests for "obfuscate"
Tests updated
2017-07-10 13:47:29 +02:00
MassimilianoPinto
f3547d14a1 Masking filter: removed create_rule_from_elements() routines
Masking filter cleanup: create_rule_from_elements() removed.
2017-07-10 09:49:20 +02:00
MassimilianoPinto
e70852d10b Code cleanup in ReplaceRule::create_from()
Code cleanup in ReplaceRule::create_from()
2017-07-10 09:49:20 +02:00
MassimilianoPinto
9490af2fb1 MXS-1302: Addition of ObfuscateRule derived class and 'obfuscation' rule
New rule ‘obfuscate’ is being added:
{
"obfuscate": {
"column": “p_name”,
"database": "test",
"table": "masking"
}
},
"applies_to": [ ... ],
"exempted": [ ... ]
2017-07-07 17:31:37 +02:00
MassimilianoPinto
b26e2d8189 MXS-1302: masking filter value replacement is done by new ReplaceRule, derived form base class Rule
The value replacement is now done by new ReplaceRule, derived form base
class Rule
2017-07-07 15:02:59 +02:00
Markus Mäkelä
d47e32966f MXS-1312: Make the permission rule the default type
If a rule is defined with only an optional part, it should be of the
permission type. This type is used to signal that the rule matches if the
optional constraints are fulfilled.

Due to refactoring, the default type was changed from RT_PERMISSION to
RT_UNDEFINED.
2017-07-07 15:44:37 +03:00
Markus Mäkelä
9c4711aac8 MXS-1311: Allow backtick quoted values for function rules
The function rules can now take keywords as arguments if they are quoted
by backticks.
2017-07-07 15:17:57 +03:00
Markus Mäkelä
674b3887c9 MXS-1301: function matches functionless queries in accept mode
A function type rule matches a query if the query uses a function defined
in the rule. This is the desired behavior for blacklist mode operation
with `action=block`.

When in whitelist mode, all queries must match a rule to be allowed
through. For function type rules, this fact is problematic as queries that
don't use functions are blocked. The desired behavior is to allow the use
of certain functions while preventing the use of others.

The allowed set of functions should always contain the "empty set" (no
functions are used) when the filter is in whitelist mode.
2017-06-30 11:07:18 +03:00
Johan Wikman
6cd6ded3d8 Merge branch '2.1-oracle-compat' into develop-new-merge-oracle 2017-06-29 09:39:55 +02:00
Johan Wikman
7f1a949198 Firewall must use its own operator codes
The firewall uses a bitmask for representing what operations a
particular rule should be applied to. Consquently it cannot use
the query classifier qc_query_op_t enumerator values as such,
as they are consecutive numbers.
2017-06-28 21:36:08 +02:00
Johan Wikman
a690b44919 MXS-1278: Sql mode must be specified explicitly
The default sql mode must now be provided explicitly when the query
classifier is setup. This is in preparation for "sql_mode" becoming
a global configuration parameter of MaxScale.
2017-06-28 21:36:08 +02:00
Esa Korhonen
14bb6cf99b Add convenience functions for common tasks with regular expressions
Several filters use a match-or-ignore logic with two regular
expressions when filtering queries. This commit adds a convenience
function for this task. Also adds a convenience function for reading
several regular expression parameters at once, compiling them and
saving the code while checking for errors.

Also, use the new functions in QLA and CCR filters.
2017-06-28 13:26:52 +03:00
Markus Mäkelä
bd7a26d830 Abstract the session default database
The default database can now be manipulated with a set of functions
exposed by the maxscale/protocol/mysql.h header. This removes the need to
handle the structures themselves in the modules and is a step towards
moving the dcb->data contents inside the session.
2017-06-22 10:40:16 +03:00
Markus Mäkelä
77f44ba92b Add missing match/exclude functionality to tee
The tee filter was missing the match/exclude functionality that was in the
previous implementation. The filter was upgraded to use PCRE2 regular
expressions instead of POSIX regular expressions.

Documented that the match/exclude patterns should use PCRE2 syntax.
2017-06-22 10:39:24 +03:00
Markus Mäkelä
46a04773a4 Merge branch '2.1' into develop 2017-06-20 11:01:08 +03:00
Esa Korhonen
b7a6e4b7b0 Fix HintParser line break detection
The HintParser wrongly ignored linebreaks, causing parsing faults
e.g. parsing too far or accepting invalid comments. Now, the parser
detects a line break and terminates comments unless they started with
'/*'. Also, fixed a memory leak when parsing parameter-value-combinations.
2017-06-19 10:49:47 +03:00
Markus Mäkelä
729d0a51d9 Fix qlafilter module entry linkage
The module declaration needs to have C linkage. Added the missing
cppdefs.hh include as the first header included by the module.
2017-06-17 09:38:46 +03:00
Esa Korhonen
7e59254454 Change modutil_get_SQL() to modutil_extract_SQL() + length limit
The modutil_get_SQL()-function allocates storage, while
modutil_extract_SQL() does not. The strings given by the latter
are not 0-terminated so require a length limit when matched using
regexec().

This commit changes the used function in those cases where the
sql-string is not modified nor is the pointer saved for later use.
2017-06-16 10:07:27 +03:00
Esa Korhonen
121284df46 Compile QLA-filter as C++ 2017-06-12 17:38:19 +03:00
Esa Korhonen
509c6d25f9 QLA-filter cleanup
Changed some literal strings to constants + other cleanup.
Removed/shortened some unnecessary comments.
2017-06-12 17:29:10 +03:00
Esa Korhonen
4db6712b66 Use MXS_MODULE_PARAM_REGEX in QLA-filter
The QLA filter now uses the automatically compiled regex parameter type.
Regex matching now prints errors. Some minor refactoring.
2017-06-12 15:40:06 +03:00
Esa Korhonen
bff5e96b1a Query hint to influence Consistent Critical Read Filter
MXS-1067. The CCRFilter will now look at the hints in a write-type
query. The parameter-value combinations "ccr=match" and "ccr=ignore"
are recognized and will override any regex match and ignore settings.

For the hints to work, the HintFilter needs to be present in the filter
chain before the CCRFilter.

Also ran astyle and updated documentation.
2017-06-12 10:51:41 +03:00
Markus Mäkelä
45afbda100 Add descriptions to module commands
Each module command has to describe what it does. This should make module
commands self-documenting to some degree.
2017-06-12 10:08:25 +03:00
Markus Mäkelä
06995837cf MXS-1220: Use JSON output in dbfwfilter module commands
Added a new module command to the dbfwfilter to allow printing of rules
via the REST API. The old module command, `rules`, is now deprecated.
2017-06-12 10:08:25 +03:00
Markus Mäkelä
b3c1e15f22 MXS-1220: Add output for module commands
The module commands can now produce JSON formatted output which is passed
to the caller. The output should conform to the JSON API as closely as
possible.

Currently, the REST API wraps all JSON produced by module commands inside
a meta-object of the following type:

  {
    "meta": <output of module command>
  }

This allows the output to be JSON API conformant without modifying the
modules and allows incremental updates to code.
2017-06-12 10:08:25 +03:00
Markus Mäkelä
457fc80647 Merge branch '2.1' into develop 2017-06-05 13:25:27 +03:00
Markus Mäkelä
539474a28d Disable RocksDB tests in 2.1
The RocksDB based cache storage is not built by default so the tests
should be disabled.
2017-06-05 13:20:15 +03:00
Esa Korhonen
9124de86df Add default value to MaskingFilter "fill" setting
MXS-1261. If "fill" is not defined, an "X" is used as default.
Setting an empty string as "fill" or "value" is disallowed.
2017-06-05 11:30:06 +03:00
Markus Mäkelä
53a2139bb9 MXS-1267: Use filter template in tee
The tee filter now uses the MaxScale filter template.
2017-06-02 13:07:02 +03:00
Markus Mäkelä
e64e24a819 MXS-1267: Fix tee filter test results
The tests should no longer expect client session creation to fail if the
branch session creation is not successful.

The main session is kept alive so that a failure in the branch session
does not cause a failure of the main session. The main session can be
considered more important as it is what the client will use. If the branch
session fails, the failure will be logged so no information is lost.

Also added a missing configuration file and removed some extra-verbose
output in a test script.
2017-06-02 13:07:02 +03:00
Markus Mäkelä
e32937c8f2 MXS-1267: Rename tee structures
The structures now use C++ style names. This is done before the filter is
moved over to the filter template.
2017-06-02 13:07:02 +03:00