hcq/MaxScale
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

MXS-2473 Simplify regular expression settings documentation

Browse Source 
The settings "match", "exclude" and "options" are now explained once
in the general documentation. The individual filter documentation refers
to the general explanation.
This commit is contained in:
Esa Korhonen
2019-05-15 17:04:33 +03:00
parent 96a477ec89
commit bb706394f6
11 changed files with 148 additions and 248 deletions
Download Patch File Download Diff File Expand all files Collapse all files

32
Documentation/Filters/BinlogFilter.md
View File

@ -7,7 +7,7 @@ This filter was introduced in MariaDB MaxScale 2.3.0.
The `binlogfilter` can be combined with a `binlogrouter` service to selectively
replicate the binary log events to slave servers.
The filter uses two parameters, `match` and `exclude`, to decide which events
The filter uses two parameters, *match* and *exclude*, to decide which events
are replicated. If a binlog event does not match or is excluded, the event is
replaced with an empty data event. The empty event is always 35 bytes which
translates to a space reduction in most cases.
@ -18,33 +18,25 @@ that there are no ambiguities in the event filtering.
## Configuration
Both the `match` and `exclude` parameters are optional. If neither of them is
defined, the filter does nothing and all events are replicated.
### `match` and `exclude`
Both the *match* and *exclude* parameters are optional and work mostly as other
[typical regular expression parameters](../Getting-Started/Configuration-Guide.md#standard-regular-expression-settings-for-filters).
If neither of them is defined, the filter does nothing and all events are replicated. This
filter does not accept regular expression options as a separate parameter, such settings
must be defined in the patterns themselves. See the
[PCRE2 api documentation](https://www.pcre.org/current/doc/html/pcre2api.html#SEC20) for
more information.
The two parameters are matched against the database and table name concatenated
with a period. For example, the string the patterns are matched against for the
database `test` and table `t1` is `test.t1`.
For statement based replication, the pattern is matched against all the tables
in the statements. If any of the tables matches the `match` pattern, the event
is replicated. If any of the tables matches the `exclude` pattern, the event is
in the statements. If any of the tables matches the *match* pattern, the event
is replicated. If any of the tables matches the *exclude* pattern, the event is
not replicated.
### `match`
A [PCRE2 regular expression](../Getting-Started/Configuration-Guide.md#regular-expressions)
that is matched against the database and table name. If the pattern matches, the
event is replicated to the slave. If no `match` parameter is defined, all events
are considered to match.
### `exclude`
A [PCRE2 regular expression](../Getting-Started/Configuration-Guide.md#regular-expressions)
that is matched against the database and table name. If the pattern matches, the
event is excluded and is not replicated to the slave. If no `exclude` pattern is
defined, the event filtering is controlled completely by the `match` parameter.
## Example Configuration
With the following configuration, only events belonging to database `customers`

38
Documentation/Filters/CCRFilter.md
View File

@ -35,22 +35,6 @@ comment. The `match`-comment typically has no effect, since write queries by
default trigger the filter anyway. It can be used to override an ignore-type
regular expression that would othewise prevent triggering.
## Filter Options
The CCR filter accepts the following options.
|Option |Description |
|-----------|--------------------------------------------|
|ignorecase |Use case-insensitive matching (default) |
|case |Use case-sensitive matching |
|extended |Use extended regular expression syntax (ERE)|
To use multiple filter options, list them in a comma-separated list.
```
options=case,extended
```
## Filter Parameters
The CCR filter has no mandatory parameters.
@ -81,27 +65,17 @@ the counter reaches zero, the statements are routed normally. If a new data
modifying SQL statement is processed, the counter is reset to the value of
_count_.
### `match`
### `match`, `ignore` and `options`
An optional parameter that can be used to control which statements trigger the
statement re-routing. The parameter value is a regular expression that is used
to match against the SQL text. Only non-SELECT statements are inspected. If this
parameter is defined, *only* matching SQL-queries will trigger the filter
(assuming no ccr hint comments in the query).
These [regular expression settings](../Getting-Started/Configuration-Guide.md#standard-regular-expression-settings-for-filters)
control which statements trigger statement re-routing. Only non-SELECT statements are
inspected. For CCRFilter, the *exclude*-parameter is instead named *ignore*, yet works
similarly.
```
match=.*INSERT.*
```
### `ignore`
An optional parameter that can be used to control which statements don't trigger
the statement re-routing. This does the opposite of the _match_ parameter. The
parameter value is a regular expression that is used to match against the SQL
text. Only non-SELECT statements are inspected.
```
ignore=.*UPDATE.*
options=case,extended
```
## Example Configuration

9
Documentation/Filters/Database-Firewall-Filter.md
View File

@ -318,10 +318,11 @@ rule examplerule match not_function columns ssn
#### `regex`
This rule blocks all queries matching a regex enclosed in single or double
quotes. The regex string expects a PCRE2 syntax regular expression. For more
information about the PCRE2 syntax, read the [PCRE2
documentation](http://www.pcre.org/current/doc/html/pcre2syntax.html).
This rule blocks all queries matching the regular expression. The regex string expects a
PCRE2 syntax regular expression. For more information about PCRE2 syntax, read the
[PCRE2 documentation](http://www.pcre.org/current/doc/html/pcre2syntax.html). Unlike
typical MaxScale regex parameters, the value should be enclosed in single or double
quotes, not in `/.../`. Any compilation options must be included in the pattern itself.
##### Example

45
Documentation/Filters/Named-Server-Filter.md
View File

@ -50,23 +50,29 @@ filters=NamedServerFilter
## Filter Parameters
The NamedServerFilter requires two mandatory parameters.
NamedServerFilter requires at least one *matchXY* - *targetXY* pair.
### `matchXY`
### `matchXY`, `options`
Regular expression the SQL-query is matched against. XY must be a number in the
range 01 - 25. Each *match* setting must have a similarly indexed *target*
setting.
*matchXY* defines a
[PCRE2 regular expression](../Getting-Started/Configuration-Guide.md#regular-expressions)
against which the incoming SQL query is matched. *XY* must be a number in the range
01 - 25. Each *match*-setting pairs with a similarly indexed *target*- setting. If one is
defined, the other must be defined as well. If a query matches the pattern, the filter
attaches a routing hint defined by the *target*-setting to the query. The
*options*-parameter affects how the patterns are compiled as
[usual](../Getting-Started/Configuration-Guide.md#standard-regular-expression-settings-for-filters).
```
match01=^SELECT
options=case,extended
```
### `targetXY`
This is the hint which will be attached to the queries matching the regex. If a
compatible router is used in the service the query will be routed accordingly.
The target can be one of the following:
The hint which is attached to the queries matching the regular expression defined by
*matchXY*. If a compatible router is used in the service the query will be routed
accordingly. The target can be one of the following:
* a server name (adds a `HINT_ROUTE_TO_NAMED_SERVER` hint)
* a list of server names, comma-separated (adds several
@ -115,26 +121,7 @@ names is simply left as is and routed straight through.
user=john
```
## Filter Options
The named server filter accepts the following options.
|Option |Description |
|----------|--------------------------------------------|
|ignorecase|Use case-insensitive matching (default) |
|case |Use case-sensitive matching |
|extended |Ignore white space and # comments |
To use multiple filter options, list them in a comma-separated list.
```
options=case,extended
```
**Note:** The *ignorecase* and *case* options are mutually exclusive and only
one of them should be used.
## Notes
## Additional remarks
The maximum number of accepted *match* - *target* pairs may be higher and can
change if other features are added to the filter. A minimum of 25 is guaranteed
@ -143,7 +130,7 @@ for now.
In the configuration, the indexed match and target settings may be in any order
and may skip numbers. During SQL-query matching, however, the regexes are tested
in ascending order: match01, match02, match03 and so on. As soon as a match is
found for a qiven query, the routing hints are written and the packet is
found for a given query, the routing hints are written and the packet is
forwarded to the next filter or router. Any possibly remaining match regexes are
ignored. This means the *match* - *target* pairs should be indexed in priority
order, or, if priority is not a factor, in order of decreasing match

39
Documentation/Filters/Query-Log-All-Filter.md
View File

@ -23,28 +23,6 @@ password=mypasswd
filters=MyLogFilter
```
## Filter Options
The QLA filter accepts the following options.
Option | Description
-------| -----------
ignorecase | Use case-insensitive matching
case | Use case-sensitive matching
extended | Use extended regular expression syntax (ERE)
To use multiple filter options, list them in a comma-separated list. If no
options are given, default will be used. Multiple options can be enabled
simultaneously.
```
options=case,extended
```
**Note**: older the version of the QLA filter in 0.7 of MariaDB MaxScale used
the `options` to define the location of the log files. This functionality is not
supported anymore and the `filebase` parameter should be used instead.
## Filter Parameters
The QLA filter has one mandatory parameter, `filebase`, and a number of optional
@ -60,25 +38,18 @@ added to the filename for each written session file. For unified log files,
filebase=/tmp/SqlQueryLog
```
The filebase may also be set as the filter option. If both option and parameter
are set, the parameter setting will be used and the filter option ignored.
### `match`, `exclude` and `options`
### `match` and `exclude`
These optional parameters limit logging on a query level. The parameter values
are regular expressions which are matched against the SQL query text. Only SQL
statements that match the regular expression in *match* but do not match the
*exclude* expression are logged.
These
[regular expression settings](../Getting-Started/Configuration-Guide.md#standard-regular-expression-settings-for-filters)
limit which queries are logged.
```
match=select.*from.*customer.*where
exclude=^insert
options=case,extended
```
*match* is checked before *exclude*. If *match* is empty, all queries are
considered matching. If *exclude* is empty, no query is exluded. If both are
empty, all queries are logged.
### `user` and `source`
These optional parameters limit logging on a session level. If `user` is

22
Documentation/Filters/Regex-Filter.md
View File

@ -28,27 +28,29 @@ password=mypasswd
filters=MyRegexfilter
```
## Filter Options
The Regex filter accepts the options ignorecase or case. These define if the pattern text should take the case of the string it is matching against into consideration or not.
## Filter Parameters
The Regex filter requires two mandatory parameters to be defined.
The Regex filter has two mandatory parameters: *match* and *replace*.
### `match`
### `match`, `options`
A parameter that can be used to match text in the SQL statement which should be replaced.
*match* is a
[PCRE2 regular expression](../Getting-Started/Configuration-Guide.md#regular-expressions)
which defines the text in the SQL statements that is replaced.
The *options*-parameter affects how the patterns are compiled as
[usual](../Getting-Started/Configuration-Guide.md#standard-regular-expression-settings-for-filters).
Regex filter does not support the `extended`-option.
```
match=TYPE[ ]*=
options=case
```
If the filter option ignorecase is used all regular expressions are evaluated with the option to ignore the case of the text, therefore a match option of select will match both type, TYPE and any form of the word with upper or lowercase characters.
### `replace`
The replace parameter defines the text that should replace the text in the SQL text which matches the match.
This is the text that should replace the part of the SQL-query matching the pattern
defined in *match*.
```
replace=ENGINE =

38
Documentation/Filters/Tee-Filter.md
View File

@ -36,46 +36,14 @@ filters=DataMartFilter
The tee filter requires a mandatory parameter to define the service to replicate
statements to and accepts a number of optional parameters.
### `match`
### `match`, `exclude` and `options`
An optional parameter used to limit the queries that will be replicated by the
tee filter. The parameter value is a PCRE2 regular expression that is used to
match against the SQL text. Only SQL statements that match the text passed as
the value of this parameter will be sent to the service defined in the filter
section.
These [regular expression settings](../Getting-Started/Configuration-Guide.md#standard-regular-expression-settings-for-filters)
limit the queries replicated by the tee filter.
```
match=/insert.*into.*order*/
```
### `exclude`
An optional parameter used to limit the queries that will be replicated by the
tee filter. The parameter value is a PCRE2 regular expression that is used to
match against the SQL text. Any SQL statements that match the text passed as the
value of this parameter will be excluded from the replication stream.
```
exclude=/select.*from.*t1/
```
If both `match` and `exclude` parameters are defined, `exclude` takes
precedence.
### `options`
The options parameter controls the regular expression options. The following
options are accepted.
|Option |Description |
|----------|--------------------------------------------|
|ignorecase|Use case-insensitive matching |
|case |Use case-sensitive matching |
|extended |Use extended regular expression syntax (ERE)|
To use multiple filter options, list them in a comma-separated list.
```
options=case,extended
```

45
Documentation/Filters/Top-N-Filter.md
View File

@ -9,7 +9,7 @@ Table of Contents
* [Filter Parameters](#filter-parameters)
* [filebase](#filebase)
* [count](#count)
* [match](#match)
* [match, exclude and options](#match-exclude-and-options)
* [exclude](#exclude)
* [source](#source)
* [user](#user)
@ -44,22 +44,6 @@ password=mypasswd
filters=MyLogFilter
```
### Filter Options
The top filter accepts the following options.
|Option |Description |
|----------|--------------------------------------------|
|ignorecase|Use case-insensitive matching |
|case |Use case-sensitive matching |
|extended |Use extended regular expression syntax (ERE)|
To use multiple filter options, list them in a comma-separated list.
```
options=case,extended
```
### Filter Parameters
The top filter has one mandatory parameter, `filebase`, and a number of optional
@ -88,36 +72,17 @@ count=30
The default value for the number of statements recorded is 10.
#### `match`
#### `match`, `exclude` and `options`
An optional parameter that can be used to limit the queries that will be logged
by the top filter. The parameter value is a regular expression that is used to
match against the SQL text. Only SQL statements that matches the text passed as
the value of this parameter will be logged.
These [regular expression settings](../Getting-Started/Configuration-Guide.md#standard-regular-expression-settings-for-filters)
limit the queries logged by the top filter.
```
match=select.*from.*customer.*where
```
All regular expressions are evaluated with the option to ignore the case of the
text, therefore a match option of select will match both select, SELECT and any
form of the word with upper or lowercase characters.
#### `exclude`
An optional parameter that can be used to limit the queries that will be logged
by the top filter. The parameter value is a regular expression that is used to
match against the SQL text. SQL statements that match the text passed as the
value of this parameter will be excluded from the log output.
```
exclude=where
options=case,extended
```
All regular expressions are evaluated with the option to ignore the case of the
text, therefore an exclude option of select will exclude statements that contain
both where, WHERE or any form of the word with upper or lowercase characters.
#### `source`
The optional source parameter defines an address that is used to match against