hcq/MaxScale
hcq/MaxScale
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

MXS-580: Added more options for filters

Browse Source 
The tee, qlafilter, namedserverfilter and topfilter now support the same filter
options: case, ignorecase and extended. The extended filter option enables
the Extended Regular Expression syntax for the filters which is used when
matching queries.
This commit is contained in:
Markus Makela 2016-02-22 15:48:30 +02:00
parent 19356be848
commit aec1310c52
8 changed files with 154 additions and 41 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
Documentation/Filters/Named-Server-Filter.md
View File

@ -26,7 +26,19 @@ filters=NamedServerFilter
## Filter Options
The named server filter accepts the options ignorecase or case. These define if the pattern text should take the case of the string it is matching against into consideration or not.
The named server filter accepts the following options.
|Option |Description |
|----------|--------------------------------------------|
|ignorecase|Use case-insensitive matching |
|case |Use case-sensitive matching |
|extended |Use extended regular expression syntax (ERE)|
To use multiple filter options, list them in a comma-separated list.
```
options=case,extended
```
## Filter Parameters

46
Documentation/Filters/Query-Log-All-Filter.md
View File

@ -2,11 +2,11 @@
## Overview
The Query Log All (QLA) filter is a filter module for MaxScale that is able to log all query content on a per client session basis. Logs are written in a csv format file that lists the time submitted and the SQL statement text.
The Query Log All (QLA) filter is a filter module for MaxScale that is able to log all query content on a per client session basis. Logs are written in a csv format file that lists the time submitted and the SQL statement text.
## Configuration
The configuration block for the QLA filter requires the minimal filter options in it's section within the maxscale.cnf file, stored in /etc/maxscale.cnf.
The configuration block for the QLA filter requires the minimal filter options in it's section within the maxscale.cnf file, stored in /etc/maxscale.cnf.
```
[MyLogFilter]
type=filter
@ -23,53 +23,61 @@ filters=MyLogFilter
## Filter Options
The QLA filter accepts one option value, this is the name that is used for the log files that are written. The file that is created appends the session number to the name given in the options entry. For example:
The QLA filter accepts the following options.
|Option |Description |
|----------|--------------------------------------------|
|ignorecase|Use case-insensitive matching |
|case |Use case-sensitive matching |
|extended |Use extended regular expression syntax (ERE)|
To use multiple filter options, list them in a comma-separated list.
```
options=/tmp/QueryLog
options=case,extended
```
would create log files /tmp/QueryLog.1 etc.
Note, this is included for backward compatibility with the version of the QLA filter that was provided in the initial filters implementation preview in 0.7 of MaxScale. The filebase parameter can now be used and will take precedence over the filter option.
**Note**: older the version of the QLA filter in 0.7 of MaxScale used the `options`
to define the location of the log files. This functionality is not supported
anymore and the `filebase` parameter should be used instead.
## Filter Parameters
The QLA filter accepts a number of optional parameters, these were introduced in the 1.0 release of MaxScale.
The QLA filter accepts a number of optional parameters, these were introduced in the 1.0 release of MaxScale.
### Filebase
The basename of the output file created for each session. A session index is added to the filename for each file written.
The basename of the output file created for each session. A session index is added to the filename for each file written.
```
filebase=/tmp/SqlQueryLog
```
The filebase may also be set as the filter, the mechanism to set the filebase via the filter option is superseded by the parameter. If both are set the parameter setting will be used and the filter option ignored.
The filebase may also be set as the filter, the mechanism to set the filebase via the filter option is superseded by the parameter. If both are set the parameter setting will be used and the filter option ignored.
### Match
An optional parameter that can be used to limit the queries that will be logged by the QLA filter. The parameter value is a regular expression that is used to match against the SQL text. Only SQL statements that matches the text passed as the value of this parameter will be logged.
An optional parameter that can be used to limit the queries that will be logged by the QLA filter. The parameter value is a regular expression that is used to match against the SQL text. Only SQL statements that matches the text passed as the value of this parameter will be logged.
```
match=select.*from.*customer.*where
```
All regular expressions are evaluated with the option to ignore the case of the text, therefore a match option of select will match both select, SELECT and any form of the word with upper or lowercase characters.
All regular expressions are evaluated with the option to ignore the case of the text, therefore a match option of select will match both select, SELECT and any form of the word with upper or lowercase characters.
### Exclude
An optional parameter that can be used to limit the queries that will be logged by the QLA filter. The parameter value is a regular expression that is used to match against the SQL text. SQL statements that match the text passed as the value of this parameter will be excluded from the log output.
An optional parameter that can be used to limit the queries that will be logged by the QLA filter. The parameter value is a regular expression that is used to match against the SQL text. SQL statements that match the text passed as the value of this parameter will be excluded from the log output.
```
exclude=where
```
All regular expressions are evaluated with the option to ignore the case of the text, therefore an exclude option of select will exclude statements that contain both select, SELECT or any form of the word with upper or lowercase characters.
All regular expressions are evaluated with the option to ignore the case of the text, therefore an exclude option of select will exclude statements that contain both select, SELECT or any form of the word with upper or lowercase characters.
### Source
The optional source parameter defines an address that is used to match against the address from which the client connection to MaxScale originates. Only sessions that originate from this address will be logged.
The optional source parameter defines an address that is used to match against the address from which the client connection to MaxScale originates. Only sessions that originate from this address will be logged.
```
source=127.0.0.1
@ -77,7 +85,7 @@ source=127.0.0.1
### User
The optional user parameter defines a user name that is used to match against the user from which the client connection to MaxScale originates. Only sessions that are connected using this username are logged.
The optional user parameter defines a user name that is used to match against the user from which the client connection to MaxScale originates. Only sessions that are connected using this username are logged.
```
user=john
@ -87,13 +95,13 @@ user=john
### Example 1 - Query without primary key
Imagine you have observed an issue with a particular table and you want to determine if there are queries that are accessing that table but not using the primary key of the table. Let's assume the table name is PRODUCTS and the primary key is called PRODUCT_ID. Add a filter with the following definition:
Imagine you have observed an issue with a particular table and you want to determine if there are queries that are accessing that table but not using the primary key of the table. Let's assume the table name is PRODUCTS and the primary key is called PRODUCT_ID. Add a filter with the following definition:
```
[ProductsSelectLogger]
type=filter
module=qlafilter
match=SELECT.*from.*PRODUCTS .*
match=SELECT.*from.*PRODUCTS .*
exclude=WHERE.*PRODUCT_ID.*
filebase=/var/logs/qla/SelectProducts
@ -106,7 +114,7 @@ passwd=mypasswd
filters=ProductsSelectLogger
```
The result of then putting this filter into the service used by the application would be a log file of all select queries that mentioned the table but did not mention the PRODUCT_ID primary key in the predicates for the query.
The result of then putting this filter into the service used by the application would be a log file of all select queries that mentioned the table but did not mention the PRODUCT_ID primary key in the predicates for the query.
Executing `SELECT * FROM PRODUCTS` would log the following into `/var/logs/qla/SelectProducts`:
```
07:12:56.324 7/01/2016, SELECT * FROM PRODUCTS

14
Documentation/Filters/Tee-Filter.md
View File

@ -25,7 +25,19 @@ filters=DataMartFilter
## Filter Options
The tee filter does not support any filter options.
The tee filter accepts the following options.
|Option |Description |
|----------|--------------------------------------------|
|ignorecase|Use case-insensitive matching |
|case |Use case-sensitive matching |
|extended |Use extended regular expression syntax (ERE)|
To use multiple filter options, list them in a comma-separated list.
```
options=case,extended
```
## Filter Parameters

14
Documentation/Filters/Top-N-Filter.md
View File

@ -24,7 +24,19 @@ filters=MyLogFilter
## Filter Options
The top filter does not support any filter options currently.
The top filter accepts the following options.
|Option |Description |
|----------|--------------------------------------------|
|ignorecase|Use case-insensitive matching |
|case |Use case-sensitive matching |
|extended |Use extended regular expression syntax (ERE)|
To use multiple filter options, list them in a comma-separated list.
```
options=case,extended
```
## Filter Parameters

4
server/modules/filter/namedserverfilter.c
View File

@ -191,6 +191,10 @@ createInstance(char **options, FILTER_PARAMETER **params)
{
cflags &= ~REG_ICASE;
}
else if (!strcasecmp(options[i], "extended"))
{
cflags |= REG_EXTENDED;
}
else
{
MXS_ERROR("namedserverfilter: unsupported option '%s'.",

40
server/modules/filter/qlafilter.c
View File

@ -177,14 +177,6 @@ createInstance(char **options, FILTER_PARAMETER **params)
if ((my_instance = calloc(1, sizeof(QLA_INSTANCE))) != NULL)
{
if (options)
{
my_instance->filebase = strdup(options[0]);
}
else
{
my_instance->filebase = strdup("qla");
}
my_instance->source = NULL;
my_instance->userName = NULL;
my_instance->match = NULL;
@ -225,9 +217,36 @@ createInstance(char **options, FILTER_PARAMETER **params)
}
}
}
int cflags = REG_ICASE;
if (options)
{
for (i = 0; options[i]; i++)
{
if (!strcasecmp(options[i], "ignorecase"))
{
cflags |= REG_ICASE;
}
else if (!strcasecmp(options[i], "case"))
{
cflags &= ~REG_ICASE;
}
else if (!strcasecmp(options[i], "extended"))
{
cflags |= REG_EXTENDED;
}
else
{
MXS_ERROR("qlafilter: unsupported option '%s'.",
options[i]);
}
}
}
my_instance->sessions = 0;
if (my_instance->match &&
regcomp(&my_instance->re, my_instance->match, REG_ICASE))
regcomp(&my_instance->re, my_instance->match, cflags))
{
MXS_ERROR("qlafilter: Invalid regular expression '%s'"
" for the match parameter.\n",
@ -242,8 +261,7 @@ createInstance(char **options, FILTER_PARAMETER **params)
return NULL;
}
if (my_instance->nomatch &&
regcomp(&my_instance->nore, my_instance->nomatch,
REG_ICASE))
regcomp(&my_instance->nore, my_instance->nomatch, cflags))
{
MXS_ERROR("qlafilter: Invalid regular expression '%s'"
" for the nomatch paramter.",

32
server/modules/filter/tee.c
View File

@ -407,6 +407,33 @@ createInstance(char **options, FILTER_PARAMETER **params)
}
}
}
int cflags = REG_ICASE;
if (options)
{
for (i = 0; options[i]; i++)
{
if (!strcasecmp(options[i], "ignorecase"))
{
cflags |= REG_ICASE;
}
else if (!strcasecmp(options[i], "case"))
{
cflags &= ~REG_ICASE;
}
else if (!strcasecmp(options[i], "extended"))
{
cflags |= REG_EXTENDED;
}
else
{
MXS_ERROR("tee: unsupported option '%s'.",
options[i]);
}
}
}
if (my_instance->service == NULL)
{
free(my_instance->match);
@ -416,7 +443,7 @@ createInstance(char **options, FILTER_PARAMETER **params)
}
if (my_instance->match &&
regcomp(&my_instance->re, my_instance->match, REG_ICASE))
regcomp(&my_instance->re, my_instance->match, cflags))
{
MXS_ERROR("tee: Invalid regular expression '%s'"
" for the match parameter.",
@ -427,8 +454,7 @@ createInstance(char **options, FILTER_PARAMETER **params)
return NULL;
}
if (my_instance->nomatch &&
regcomp(&my_instance->nore, my_instance->nomatch,
REG_ICASE))
regcomp(&my_instance->nore, my_instance->nomatch, cflags))
{
MXS_ERROR("tee: Invalid regular expression '%s'"
" for the nomatch paramter.\n",

31
server/modules/filter/topfilter.c
View File

@ -233,14 +233,36 @@ createInstance(char **options, FILTER_PARAMETER **params)
params[i]->name);
}
}
int cflags = REG_ICASE;
if (options)
{
MXS_ERROR("topfilter: Options are not supported by this "
" filter. They will be ignored.");
for (i = 0; options[i]; i++)
{
if (!strcasecmp(options[i], "ignorecase"))
{
cflags |= REG_ICASE;
}
else if (!strcasecmp(options[i], "case"))
{
cflags &= ~REG_ICASE;
}
else if (!strcasecmp(options[i], "extended"))
{
cflags |= REG_EXTENDED;
}
else
{
MXS_ERROR("topfilter: unsupported option '%s'.",
options[i]);
}
}
}
my_instance->sessions = 0;
if (my_instance->match &&
regcomp(&my_instance->re, my_instance->match, REG_ICASE))
regcomp(&my_instance->re, my_instance->match, cflags))
{
MXS_ERROR("topfilter: Invalid regular expression '%s'"
" for the match parameter.",
@ -253,8 +275,7 @@ createInstance(char **options, FILTER_PARAMETER **params)
return NULL;
}
if (my_instance->exclude &&
regcomp(&my_instance->exre, my_instance->exclude,
REG_ICASE))
regcomp(&my_instance->exre, my_instance->exclude, cflags))
{
MXS_ERROR("qlafilter: Invalid regular expression '%s'"
" for the nomatch paramter.\n",