hcq/MaxScale
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add table of contents to filter documentation

Browse Source 
Only added the TOC to the longest files.
This commit is contained in:
Esa Korhonen
2018-11-23 17:21:12 +02:00
parent ff22624d3c
commit ab7daa9327
3 changed files with 156 additions and 34 deletions
Download Patch File Download Diff File Expand all files Collapse all files

101
Documentation/Filters/Cache.md
View File

@ -2,6 +2,57 @@
This filter was introduced in MariaDB MaxScale 2.1. This filter was introduced in MariaDB MaxScale 2.1.
Table of Contents
=================
* [Overview](#overview)
* [Limitations](#limitations)
* [Invalidation](#invalidation)
* [Prepared Statements](#prepared-statements)
* [Security](#security)
* [Configuration](#configuration)
* [Filter Parameters](#filter-parameters)
* [storage](#storage)
* [storage_options](#storage_options)
* [hard_ttl](#hard_ttl)
* [soft_ttl](#soft_ttl)
* [max_resultset_rows](#max_resultset_rows)
* [max_resultset_size](#max_resultset_size)
* [max_count](#max_count)
* [max_size](#max_size)
* [rules](#rules)
* [cached_data](#cached_data)
* [selects](#selects)
* [cache_inside_transactions](#cache_inside_transactions)
* [debug](#debug)
* [enabled](#enabled)
* [Runtime Configuration](#runtime-configuration)
* [@maxscale.cache.populate](#maxscalecachepopulate)
* [@maxscale.cache.use](#maxscalecacheuse)
* [@maxscale.cache.soft_ttl](#maxscalecachesoft_ttl)
* [@maxscale.cache.hard_ttl](#maxscalecachehard_ttl)
* [Client Driven Caching](#client-driven-caching)
* [Rules](#rules-1)
* [When to Store](#when-to-store)
* [Qualified Names](#qualified-names)
* [Implication of the <em>default</em> database](#implication-of-the-default-database)
* [Regexp Matching](#regexp-matching)
* [Examples](#examples)
* [When to Use](#when-to-use)
* [Examples](#examples-1)
* [Security](#security-1)
* [Storage](#storage-1)
* [storage_inmemory](#storage_inmemory)
* [storage_rocksdb](#storage_rocksdb)
* [Parameters](#parameters)
* [cache_directory](#cache_directory)
* [collect_statistics](#collect_statistics)
* [Example](#example)
* [Configuration](#configuration-1)
* [cache_rules.json](#cache_rulesjson)
* [Performance](#performance)
* [Summary](#summary)
## Overview ## Overview
From MaxScale version 2.2.11 onwards, the cache filter is no longer From MaxScale version 2.2.11 onwards, the cache filter is no longer
@ -326,9 +377,10 @@ at runtime. Please see
[Runtime Configuration](#runtime-configuation) [Runtime Configuration](#runtime-configuation)
for details. for details.
## Runtime Configuration ### Runtime Configuration
#### `@maxscale.cache.populate`
### `@maxscale.cache.populate`
Using the variable `@maxscale.cache.populate` it is possible to specify at Using the variable `@maxscale.cache.populate` it is possible to specify at
runtime whether the cache should be populated or not. Its initial value is runtime whether the cache should be populated or not. Its initial value is
the value of the configuration parameter `enabled`. That is, by default the the value of the configuration parameter `enabled`. That is, by default the
@ -354,7 +406,8 @@ SELECT @maxscale.cache.populate;
``` ```
but only _after_ it has been explicitly set once. but only _after_ it has been explicitly set once.
### `@maxscale.cache.use` #### `@maxscale.cache.use`
Using the variable `@maxscale.cache.use` it is possible to specify at Using the variable `@maxscale.cache.use` it is possible to specify at
runtime whether the cache should be used or not. Its initial value is runtime whether the cache should be used or not. Its initial value is
the value of the configuration parameter `enabled`. That is, by default the the value of the configuration parameter `enabled`. That is, by default the
@ -387,7 +440,8 @@ SELECT @maxscale.cache.use;
``` ```
but only after it has explicitly been set once. but only after it has explicitly been set once.
### `@maxscale.cache.soft_ttl` #### `@maxscale.cache.soft_ttl`
Using the variable `@maxscale.cache.soft_ttl` it is possible to specify Using the variable `@maxscale.cache.soft_ttl` it is possible to specify
at runtime what _soft ttl_ should be applied. Its initial value is the at runtime what _soft ttl_ should be applied. Its initial value is the
value of the configuration parameter `soft_ttl`. That is, by default the value of the configuration parameter `soft_ttl`. That is, by default the
@ -416,7 +470,8 @@ SELECT @maxscale.cache.soft_ttl;
``` ```
but only after it has explicitly been set once. but only after it has explicitly been set once.
### `@maxscale.cache.hard_ttl` #### `@maxscale.cache.hard_ttl`
Using the variable `@maxscale.cache.hard_ttl` it is possible to specify Using the variable `@maxscale.cache.hard_ttl` it is possible to specify
at runtime what _hard ttl_ should be applied. Its initial value is the at runtime what _hard ttl_ should be applied. Its initial value is the
value of the configuration parameter `hard_ttl`. That is, by default the value of the configuration parameter `hard_ttl`. That is, by default the
@ -442,7 +497,8 @@ SELECT @maxscale.cache.hard_ttl;
``` ```
but only after it has explicitly been set once. but only after it has explicitly been set once.
### Client Driven Caching #### Client Driven Caching
With `@maxscale.cache.populate` and `@maxscale.cache.use` is it possible With `@maxscale.cache.populate` and `@maxscale.cache.use` is it possible
to make the caching completely client driven. to make the caching completely client driven.
@ -488,10 +544,9 @@ SELECT a, b FROM tbl1;
SET @maxscale.cache.populate=false; SET @maxscale.cache.populate=false;
``` ```
# Rules ## Rules
The caching rules are expressed as a JSON object or as an array The caching rules are expressed as a JSON object or as an array of JSON objects.
of JSON objects.
There are two decisions to be made regarding the caching; in what circumstances There are two decisions to be made regarding the caching; in what circumstances
should data be stored to the cache and in what circumstances should the data in should data be stored to the cache and in what circumstances should the data in
@ -526,7 +581,7 @@ matches, the `store` field of each object are evaluated in sequential order
until a match is found. Then, the `use` field of that object is used when until a match is found. Then, the `use` field of that object is used when
deciding whether data in the cache should be used. deciding whether data in the cache should be used.
## When to Store ### When to Store
By default, if no rules file have been provided or if the `store` field is By default, if no rules file have been provided or if the `store` field is
missing from the object, the results of all queries will be stored to the missing from the object, the results of all queries will be stored to the
@ -608,7 +663,7 @@ and so will
select b from tbl where a > 5; select b from tbl where a > 5;
``` ```
### Qualified Names #### Qualified Names
When using `=` or `!=` in the rule object in conjunction with `database`, When using `=` or `!=` in the rule object in conjunction with `database`,
`table` and `column`, the provided string is interpreted as a name, that is, `table` and `column`, the provided string is interpreted as a name, that is,
@ -626,12 +681,12 @@ always be capable of deducing in what table a particular column is. If
that is the case, then a value like `tbl.field` may not necessarily that is the case, then a value like `tbl.field` may not necessarily
be a match even if the field is `field` and the table actually is `tbl`. be a match even if the field is `field` and the table actually is `tbl`.
### Implication of the _default_ database. #### Implication of the _default_ database
If the rules concerns the `database`, then only if the statement refers If the rules concerns the `database`, then only if the statement refers
to *no* specific database, will the default database be considered. to *no* specific database, will the default database be considered.
### Regexp Matching #### Regexp Matching
The string used for matching the regular expression contains as much The string used for matching the regular expression contains as much
information as there is available. For instance, in a situation like information as there is available. For instance, in a situation like
@ -641,7 +696,7 @@ select fld from tbl;
``` ```
the string matched against the regular expression will be `somedb.tbl.fld`. the string matched against the regular expression will be `somedb.tbl.fld`.
### Examples #### Examples
Cache all queries targeting a particular database. Cache all queries targeting a particular database.
``` ```
@ -700,7 +755,7 @@ Cache all queries containing a WHERE clause
Note that that will actually cause all queries that contain WHERE anywhere, Note that that will actually cause all queries that contain WHERE anywhere,
to be cached. to be cached.
## When to Use ### When to Use
By default, if no rules file have been provided or if the `use` field is By default, if no rules file have been provided or if the `use` field is
missing from the object, all users may be returned data from the cache. missing from the object, all users may be returned data from the cache.
@ -764,7 +819,7 @@ Note that `use` is relevant only if the query is subject to caching,
that is, if all queries are cached or if a query matches a particular that is, if all queries are cached or if a query matches a particular
rule in the `store` array. rule in the `store` array.
### Examples #### Examples
Use data from the cache for all users except `admin` (actually `'admin'@'%'`), Use data from the cache for all users except `admin` (actually `'admin'@'%'`),
regardless of what host the `admin` user comes from. regardless of what host the `admin` user comes from.
@ -779,7 +834,7 @@ regardless of what host the `admin` user comes from.
] ]
} }
``` ```
# Security ## Security
As the cache is not aware of grants, unless the cache has been explicitly As the cache is not aware of grants, unless the cache has been explicitly
configured who the caching should apply to, the presence of the cache configured who the caching should apply to, the presence of the cache
@ -859,9 +914,9 @@ should be applied to _alice_ only.
With these rules in place, _bob_ is again denied access, since queries With these rules in place, _bob_ is again denied access, since queries
targeting the table `access` will in his case not be served from the cache. targeting the table `access` will in his case not be served from the cache.
# Storage ## Storage
## `storage_inmemory` ### `storage_inmemory`
This simple storage module uses the standard memory allocator for storing This simple storage module uses the standard memory allocator for storing
the cached data. the cached data.
@ -869,7 +924,7 @@ the cached data.
storage=storage_inmemory storage=storage_inmemory
``` ```
## `storage_rocksdb` ### `storage_rocksdb`
This storage module is not built by default and is not included in the This storage module is not built by default and is not included in the
MariaDB MaxScale packages. MariaDB MaxScale packages.
@ -915,7 +970,7 @@ The value is a boolean and the default is `false`.
storage_options=collect_statistics=true storage_options=collect_statistics=true
``` ```
# Example ## Example
In the following we define a cache _MyCache_ that uses the cache storage module In the following we define a cache _MyCache_ that uses the cache storage module
`storage_inmemory` and whose _soft ttl_ is `30` seconds and whose _hard ttl_ is `storage_inmemory` and whose _soft ttl_ is `30` seconds and whose _hard ttl_ is
@ -956,7 +1011,7 @@ The rules specify that the data of the table `sbtest` should be cached.
} }
``` ```
# Performance ## Performance
Perhaps the most significant factor affecting the performance of the cache is Perhaps the most significant factor affecting the performance of the cache is
whether the statements need to be parsed or not. By default, all statements are whether the statements need to be parsed or not. By default, all statements are
@ -1043,7 +1098,7 @@ high load may be significantly _greater_.
| `verify_cacheable` | _regexp match_ | 58 | | `verify_cacheable` | _regexp match_ | 58 |
| `verify_cacheable` | _exact match_ | 58 | | `verify_cacheable` | _exact match_ | 58 |
## Summary ### Summary
For maximum performance: For maximum performance:

43
Documentation/Filters/Database-Firewall-Filter.md
View File

@ -1,5 +1,48 @@
# Database Firewall filter # Database Firewall filter
Table of Contents
=================
* [Overview](#overview)
* [Configuration](#configuration)
* [Filter Parameters](#filter-parameters)
* [rules](#rules)
* [action](#action)
* [log_match](#log_match)
* [log_no_match](#log_no_match)
* [Rule syntax](#rule-syntax)
* [Mandatory rule parameters](#mandatory-rule-parameters)
* [wildcard](#wildcard)
* [Example](#example)
* [columns](#columns)
* [Example](#example-1)
* [function](#function)
* [Example](#example-2)
* [not_function](#not_function)
* [Example](#example-3)
* [uses_function](#uses_function)
* [Example](#example-4)
* [function and columns](#function-and-columns)
* [Example](#example-5)
* [not_function and columns](#not_function-and-columns)
* [Example](#example-6)
* [regex](#regex)
* [Example](#example-7)
* [limit_queries](#limit_queries)
* [Example](#example-8)
* [no_where_clause](#no_where_clause)
* [Example](#example-9)
* [Optional rule parameters](#optional-rule-parameters)
* [at_times](#at_times)
* [on_queries](#on_queries)
* [Applying rules to users](#applying-rules-to-users)
* [Module commands](#module-commands)
* [dbfwfilter::rules/reload [FILE]](#dbfwfilterrulesreload-file)
* [dbfwfilter::rules](#dbfwfilterrules)
* [Use Cases](#use-cases)
* [Use Case 1 - Prevent rapid execution of specific queries](#use-case-1---prevent-rapid-execution-of-specific-queries)
* [Use Case 2 - Only allow deletes with a where clause](#use-case-2---only-allow-deletes-with-a-where-clause)
## Overview ## Overview
The Database Firewall filter is used to block queries that match a set of The Database Firewall filter is used to block queries that match a set of

46
Documentation/Filters/Masking.md
View File

@ -2,7 +2,32 @@
This filter was introduced in MariaDB MaxScale 2.1. This filter was introduced in MariaDB MaxScale 2.1.
Table of Contents
=================
* [Overview](#overview)
* [Security](#security)
* [Limitations](#limitations)
* [Configuration](#configuration)
* [Filter Parameters](#filter-parameters)
* [rules](#rules)
* [warn_type_mismatch](#warn_type_mismatch)
* [large_payload](#large_payload)
* [prevent_function_usage](#prevent_function_usage)
* [Rules](#rules-1)
* [replace](#replace)
* [obfuscate](#obfuscate)
* [with](#with)
* [applies_to](#applies_to)
* [exempted](#exempted)
* [Module commands](#module-commands)
* [reload](#reload)
* [Example](#example)
* [Configuration](#configuration-1)
* [masking_rules.json](#masking_rulesjson)
## Overview ## Overview
With the _masking_ filter it is possible to obfuscate the returned With the _masking_ filter it is possible to obfuscate the returned
value of a particular column. value of a particular column.
@ -77,7 +102,7 @@ type=service
filters=Mask-SSN filters=Mask-SSN
``` ```
## Filter Parameters ### Filter Parameters
The masking filter has one mandatory parameter - `rules`. The masking filter has one mandatory parameter - `rules`.
@ -145,7 +170,7 @@ prevent_function_usage=false
``` ```
The default value is `true`. The default value is `true`.
# Rules ## Rules
The masking rules are expressed as a JSON object. The masking rules are expressed as a JSON object.
@ -157,8 +182,6 @@ value is an array of rule objects.
} }
``` ```
## Rule
Each rule in the rules array is a JSON object, expected to Each rule in the rules array is a JSON object, expected to
contain the keys `replace`, `with`, `applies_to` and contain the keys `replace`, `with`, `applies_to` and
`exempted`. The two former ones are obligatory and the two `exempted`. The two former ones are obligatory and the two
@ -177,7 +200,7 @@ latter ones optional.
} }
``` ```
#### `replace` ### `replace`
The value of this key is an object that specifies the column The value of this key is an object that specifies the column
whose values should be masked. The object must contain the key whose values should be masked. The object must contain the key
@ -251,7 +274,7 @@ The `match` value must be a valid pcre2 regular expression.
} }
``` ```
#### `obfuscate` ### `obfuscate`
The obfuscate rule allows the obfuscation of the value The obfuscate rule allows the obfuscation of the value
by passing it through an obfuscation algorithm. by passing it through an obfuscation algorithm.
@ -284,7 +307,7 @@ SELECT name from db1.tbl1;`
+------+ +------+
``` ```
#### `with` ### `with`
The value of this key is an object that specifies what the value of the matched The value of this key is an object that specifies what the value of the matched
column should be replaced with for the `replace` rule. Currently, the object column should be replaced with for the `replace` rule. Currently, the object
@ -341,7 +364,7 @@ of the mask value to get the lengths to match.
} }
``` ```
#### `applies_to` ### `applies_to`
With this _optional_ key, whose value must be an array of strings, With this _optional_ key, whose value must be an array of strings,
it can be specified what users the rule is applied to. Each string it can be specified what users the rule is applied to. Each string
@ -363,7 +386,7 @@ should be a MariaDB account string, that is, `%` is a wildcard.
If this key is not specified, then the masking is performed for all If this key is not specified, then the masking is performed for all
users, except the ones exempted using the key `exempted`. users, except the ones exempted using the key `exempted`.
#### `exempted` ### `exempted`
With this _optional_ key, whose value must be an array of strings, With this _optional_ key, whose value must be an array of strings,
it can be specified what users the rule is *not* applied to. Each it can be specified what users the rule is *not* applied to. Each
@ -384,7 +407,8 @@ string should be a MariaDB account string, that is, `%` is a wildcard.
## Module commands ## Module commands
Read [Module Commands](../Reference/Module-Commands.md) documentation for details about module commands. Read [Module Commands](../Reference/Module-Commands.md) documentation for details
about module commands.
The masking filter supports the following module commands. The masking filter supports the following module commands.
@ -398,7 +422,7 @@ MaxScale> call command masking reload MyMaskingFilter
`MyMaskingFilter` refers to a particular filter section in the `MyMaskingFilter` refers to a particular filter section in the
MariaDB MaxScale configuration file. MariaDB MaxScale configuration file.
# Example ## Example
In the following we configure a masking filter _MyMasking_ that should always log a In the following we configure a masking filter _MyMasking_ that should always log a
warning if a masking rule matches a column that is of a type that cannot be masked, warning if a masking rule matches a column that is of a type that cannot be masked,