hcq/MaxScale
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Reformat and tune QLA filter documentation

Browse Source 
No additions, just tidying up.
This commit is contained in:
Esa Korhonen
2018-01-16 17:10:32 +02:00
parent 23f2c3b980
commit b8e15d2bea
1 changed files with 50 additions and 47 deletions
Download Patch File Download Diff File Expand all files Collapse all files

97
Documentation/Filters/Query-Log-All-Filter.md
View File

@ -2,11 +2,13 @@
## Overview ## Overview
The Query Log All (QLA) filter is a filter module for MariaDB MaxScale that is able to log all query content on a per client session basis. Logs are written in a csv format file that lists the time submitted and the SQL statement text. The Query Log All (QLA) filter logs query content. Logs are written to a file in
CSV format. Log elements are configurable and include the time submitted and the
SQL statement text, among others.
## Configuration ## Configuration
The configuration block for the QLA filter requires the minimal filter options in its section within the maxscale.cnf file, stored in /etc/maxscale.cnf. A minimal configuration is below.
``` ```
[MyLogFilter] [MyLogFilter]
type=filter type=filter
@ -31,72 +33,64 @@ The QLA filter accepts the following options.
case | Use case-sensitive matching case | Use case-sensitive matching
extended | Use extended regular expression syntax (ERE) extended | Use extended regular expression syntax (ERE)
To use multiple filter options, list them in a comma-separated list. If no options are given, default will be used. Multiple options can be enabled simultaneously. To use multiple filter options, list them in a comma-separated list. If no
options are given, default will be used. Multiple options can be enabled
simultaneously.
``` ```
options=case,extended options=case,extended
``` ```
**Note**: older the version of the QLA filter in 0.7 of MariaDB MaxScale used the `options` **Note**: older the version of the QLA filter in 0.7 of MariaDB MaxScale used
to define the location of the log files. This functionality is not supported the `options` to define the location of the log files. This functionality is not
anymore and the `filebase` parameter should be used instead. supported anymore and the `filebase` parameter should be used instead.
## Filter Parameters ## Filter Parameters
The QLA filter has one mandatory parameter, `filebase`, and a number of optional parameters. These were introduced in the 1.0 release of MariaDB MaxScale. The QLA filter has one mandatory parameter, `filebase`, and a number of optional
parameters. These were introduced in the 1.0 release of MariaDB MaxScale.
### `filebase` ### `filebase`
The basename of the output file created for each session. A session index is added to the filename for each file written. This is a mandatory parameter. The basename of the output file created for each session. A session index is
added to the filename for each written session file. For unified log files,
*.unified* is appended. This is a mandatory parameter.
``` ```
filebase=/tmp/SqlQueryLog filebase=/tmp/SqlQueryLog
``` ```
The filebase may also be set as the filter option, the mechanism to set the filebase via the filter option is superseded by the parameter. If both are set the parameter setting will be used and the filter option ignored. The filebase may also be set as the filter option. If both option and parameter
are set, the parameter setting will be used and the filter option ignored.
### `match` ### `match` and `exclude`
An optional parameter that can be used to limit the queries that will be logged by the QLA filter. The parameter value is a regular expression that is used to match against the SQL text. Only SQL statements that matches the text passed as the value of this parameter will be logged. These optional parameters limit logging on a query level. The parameter values
are regular expressions which are matched against the SQL query text. Only SQL
statements that match the regular expression in *match* but do not match the
*exclude* expression are logged.
``` ```
match=select.*from.*customer.*where match=select.*from.*customer.*where
exclude=^insert
``` ```
All regular expressions are evaluated with the option to ignore the case of the text, therefore a match option of select will match both select, SELECT and any form of the word with upper or lowercase characters. *match* is checked before *exclude*. If *match* is empty, all queries are
considered matching. If *exclude* is empty, no query is exluded. If both are
empty, all queries are logged.
### `exclude` ### `user` and `source`
An optional parameter that can be used to limit the queries that will be logged by the QLA filter. The parameter value is a regular expression that is used to match against the SQL text. SQL statements that match the text passed as the value of this parameter will be excluded from the log output. These optional parameters limit logging on a session level. If `user` is
defined, only the sessions with a matching client username are logged. If
``` `source` is defined, only sessions with a matching client source address are
exclude=where logged.
```
All regular expressions are evaluated with the option to ignore the case of the text, therefore an exclude option of select will exclude statements that contain both select, SELECT or any form of the word with upper or lowercase characters.
### `source`
The optional source parameter defines an address that is used to match against the address from which the client connection to MariaDB MaxScale originates. Only sessions that originate from this address will be logged.
```
source=127.0.0.1
```
### `user`
The optional user parameter defines a user name that is used to match against the user from which the client connection to MariaDB MaxScale originates. Only sessions that are connected using this username are logged.
``` ```
user=john user=john
source=127.0.0.1
``` ```
-----------------------------------------------------------
**The following parameters were added in MaxScale 2.1.0**
-----------------------------------------------------------
### `log_type` ### `log_type`
The type of log file to use. The default value is _session_. The type of log file to use. The default value is _session_.
@ -110,11 +104,13 @@ The type of log file to use. The default value is _session_.
log_type=session log_type=session
``` ```
If both logs are required, define `log_type=session,unified`.
### `log_data` ### `log_data`
Type of data to log in the log files. Parameter value is a comma separated list Type of data to log in the log files. The parameter value is a comma separated
of the following values. By default the _date_, _user_ and _query_ options are list of the following elements. By default the _date_, _user_ and _query_
enabled. options are enabled.
| Value | Description | | Value | Description |
| -------- |--------------------------------------------------| | -------- |--------------------------------------------------|
@ -129,8 +125,9 @@ enabled.
log_data=date, user, query log_data=date, user, query
``` ```
If *reply_time* is enabled, the log entry is written when the first reply from server is received. If *reply_time* is enabled, the log entry is written when the first reply from
Otherwise, the entry is written when receiving query from client. server is received. Otherwise, the entry is written when receiving query from
client.
### `flush` ### `flush`
@ -142,7 +139,8 @@ flush=true
### `append` ### `append`
Append new entries to log files instead of overwriting them. The default is false. Append new entries to log files instead of overwriting them. The default is
false.
``` ```
append=true append=true
@ -152,7 +150,10 @@ append=true
### Example 1 - Query without primary key ### Example 1 - Query without primary key
Imagine you have observed an issue with a particular table and you want to determine if there are queries that are accessing that table but not using the primary key of the table. Let's assume the table name is PRODUCTS and the primary key is called PRODUCT_ID. Add a filter with the following definition: Imagine you have observed an issue with a particular table and you want to
determine if there are queries that are accessing that table but not using the
primary key of the table. Let's assume the table name is PRODUCTS and the
primary key is called PRODUCT_ID. Add a filter with the following definition:
``` ```
[ProductsSelectLogger] [ProductsSelectLogger]
@ -171,8 +172,10 @@ passwd=mypasswd
filters=ProductsSelectLogger filters=ProductsSelectLogger
``` ```
The result of then putting this filter into the service used by the application would be a log file of all select queries that mentioned the table but did not mention the PRODUCT_ID primary key in the predicates for the query. The result of using this filter with the service used by the application would
Executing `SELECT * FROM PRODUCTS` would log the following into `/var/logs/qla/SelectProducts`: be a log file of all select queries querying PRODUCTS without using the
PRODUCT_ID primary key in the predicates of the query. Executing `SELECT * FROM
PRODUCTS` would log the following into `/var/logs/qla/SelectProducts`:
``` ```
07:12:56.324 7/01/2016, SELECT * FROM PRODUCTS 07:12:56.324 7/01/2016, SELECT * FROM PRODUCTS
``` ```