hcq/MaxScale
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

cache: Update documentation and add rule handling

Browse Source 
The concept of 'allowed_references' was removed from the
documentation and the code. Now that COM_INIT_DB is tracked,
we will always know what the default database is and hence
we can create a cache key that distinguises between identical
queries targeting different default database (that is not
implemented yet in this change).

The rules for the cache is expressed using a JSON object.
There are two decisions to be made; when to store data to the
cache and when to use data from the cache. The latter is
obviously dependent on the former.

In this change, the 'store' handling is implemented; 'use'
handling will be in a subsequent change.
This commit is contained in:
Johan Wikman
2016-09-27 15:56:15 +03:00
parent 20b57b1577
commit 7f24f12cfc
6 changed files with 1246 additions and 163 deletions
Download Patch File Download Diff File Expand all files Collapse all files

211
Documentation/Filters/Cache.md
View File

@ -1,4 +1,4 @@
#Cache
# Cache
## Overview
The cache filter is capable of caching the result of SELECTs, so that subsequent identical
@ -16,6 +16,8 @@ module=cache
ttl=5
storage=...
storage_options=...
rules=...
debug=...
[Cached Routing Service]
type=service
@ -57,36 +59,6 @@ depend upon the specific module. For instance,
storage_options=storage_specific_option1=value1,storage_specific_option2=value2
```
#### `allowed_references`
Specifies whether any or only fully qualified references are allowed in
queries stored to the cache.
```
allowed_references=[qualified|any]
```
The default is `qualified`, which means that only queries where
the database name is included in the table name are subject to caching.
```
select col from db.tbl;
```
If `any` is specified, then also queries where the table name is not
fully qualified are subject to caching.
```
select col from tbl;
```
Care should be excersized before this setting is changed, because, for
instance, the following is likely to produce unexpected results.
```
use db1;
select col from tbl;
...
use db2;
select col from tbl;
```
The setting can be changed to `any`, provided fully qualified names
are always used or if the names of tables in different databases are
different.
#### `max_resultset_rows`
Specifies the maximum number of rows a resultset can have in order to be
@ -119,6 +91,181 @@ If nothing is specified, the default _ttl_ value is 10.
ttl=60
```
#Storage
#### `rules`
Specifies the path of the file where the caching rules are stored. A relative
path is interpreted relative to the _data directory_ of MariaDB MaxScale.
```
rules=/path/to/rules-file
```
#### `debug`
An integer value, using which the level of debug logging made by the cache
can be controlled. The value is actually a bitfield with different bits
denoting different logging.
* `0` (`0b0000`) No logging is made.
* `1` (`0b0001`) A matching rule is logged.
* `2` (`0b0010`) A non-matching rule is logged.
* `4` (`0b0100`) A decision to use data from the cache is logged.
* `8` (`0b1000`) A decision not to use data from the cache is logged.
Default is `0`. To log everything, give `debug` a value of `15`.
```
debug=2
```
# Rules
The caching rules are expressed as a JSON object.
There are two decisions to be made regarding the caching; in what circumstances
should data be stored to the cache and in what circumstances should the data in
the cache be used.
In the JSON object this is visible as follows:
```
{
store: [ ... ],
use: [ ... ]
}
```
The `store` field specifies in what circumstances data should be stored to
the cache and the `use` field specifies in what circumstances the data in
the cache should be used. In both cases, the value is a JSON array containg
objects.
## When to Store
By default, if no rules file have been provided or if the `store` field is
missing from the object, the results of all queries will be stored to the
cache, subject to `max_resultset_rows` and `max_resultset_size` cache filter
parameters.
By providing a `store` field in the JSON object, the decision whether to
store the result of a particular query to the cache can be controlled in
a more detailed manner. The decision to cache the results of a query can
depend upon
* the database,
* the table,
* the column, or
* the query itself.
Each entry in the `store` array is an object containing three fields,
```
{
"attribute": <string>,
"op": <string>
"value": <string>
}
```
where,
* the _attribute_ can be `database`, `table`, `column` or `query`,
* the _op_ can be `=`, `!=`, `like` or `unlike`, and
* the _value_ a string.
If _op_ is `=` or `!=` then _value_ is used verbatim; if it is `like`
or `unlike`, then _value_ is interpreted as a _pcre2_ regular expression.
The objects in the `store` array are processed in order. If the result
of a comparison is _true_, no further processing will be made and the
result of the query in question will be stored to the cache.
If the result of the comparison is _false_, then the next object is
processed. The process continues until the array is exhausted. If there
is no match, then the result of the query is not stored to the cache.
Note that as the query itself is used as the key, although the following
queries
```
select * from db1.tbl
```
and
```
use db1;
select * from tbl
```
target the same table and produce the same results, they will be cached
separately. The same holds for queries like
```
select * from tbl where a = 2 and b = 3;
```
and
```
select * from tbl where b = 3 and a = 2;
```
as well. Although they conceptually are identical, there will be two
cache entries.
### Examples
Cache all queries targeting a particular database.
```
{
"store": [
{
"attribute": "database",
"op": "=",
"value": "db1"
}
]
}
```
Cache all queries _not_ targeting a particular table
```
{
"store": [
{
"attribute": "table",
"op": "!=",
"value": "tbl1"
}
]
}
```
That will exclude queries targeting table _tbl1_ irrespective of which
database it is in. To exclude a table in a particular database, specify
the table name using a qualified name.
```
{
"store": [
{
"attribute": "table",
"op": "!=",
"value": "db1.tbl1"
}
]
}
```
Cache all queries containing a WHERE clause
```
{
"store": [
{
"attribute": "query",
"op": "like",
"value": ".*WHERE.*"
}
]
}
```
Note that that will actually cause all queries that contain WHERE anywhere,
to be cached.
## When to Use
# Storage
## Storage RocksDB

5
server/modules/filter/cache/CMakeLists.txt vendored
View File

@ -1,6 +1,7 @@
add_library(cache SHARED cache.c storage.c)
target_link_libraries(cache maxscale-common)
add_library(cache SHARED cache.c rules.c storage.c)
target_link_libraries(cache maxscale-common jansson)
set_target_properties(cache PROPERTIES VERSION "1.0.0")
set_target_properties(cache PROPERTIES LINK_FLAGS -Wl,-z,defs)
install_module(cache experimental)
add_subdirectory(storage)

232
server/modules/filter/cache/cache.c vendored
View File

@ -14,12 +14,14 @@
#define MXS_MODULE_NAME "cache"
#include <maxscale/alloc.h>
#include <filter.h>
#include <gwdirs.h>
#include <log_manager.h>
#include <modinfo.h>
#include <modutil.h>
#include <mysql_utils.h>
#include <query_classifier.h>
#include "cache.h"
#include "rules.h"
#include "storage.h"
static char VERSION_STRING[] = "V1.0.0";
@ -90,28 +92,31 @@ FILTER_OBJECT *GetModuleObject()
typedef struct cache_config
{
cache_references_t allowed_references;
uint32_t max_resultset_rows;
uint32_t max_resultset_size;
const char* rules;
const char *storage;
const char *storage_options;
uint32_t ttl;
uint32_t debug;
} CACHE_CONFIG;
static const CACHE_CONFIG DEFAULT_CONFIG =
{
CACHE_DEFAULT_ALLOWED_REFERENCES,
CACHE_DEFAULT_MAX_RESULTSET_ROWS,
CACHE_DEFAULT_MAX_RESULTSET_SIZE,
NULL,
NULL,
CACHE_DEFAULT_TTL
NULL,
CACHE_DEFAULT_TTL,
CACHE_DEFAULT_DEBUG
};
typedef struct cache_instance
{
const char *name;
CACHE_CONFIG config;
CACHE_RULES *rules;
CACHE_STORAGE_MODULE *module;
CACHE_STORAGE *storage;
} CACHE_INSTANCE;
@ -167,7 +172,7 @@ static int handle_expecting_response(CACHE_SESSION_DATA *csdata);
static int handle_expecting_rows(CACHE_SESSION_DATA *csdata);
static int handle_expecting_use_response(CACHE_SESSION_DATA *csdata);
static int handle_ignoring_response(CACHE_SESSION_DATA *csdata);
static bool process_params(char **options, FILTER_PARAMETER **params, CACHE_CONFIG* config);
static bool route_using_cache(CACHE_SESSION_DATA *sdata, const GWBUF *key, GWBUF **value);
static int send_upstream(CACHE_SESSION_DATA *csdata);
@ -190,91 +195,23 @@ static void store_result(CACHE_SESSION_DATA *csdata);
*/
static FILTER *createInstance(const char *name, char **options, FILTER_PARAMETER **params)
{
CACHE_INSTANCE *cinstance = NULL;
CACHE_CONFIG config = DEFAULT_CONFIG;
bool error = false;
if (process_params(options, params, &config))
{
CACHE_RULES *rules = NULL;
for (int i = 0; params[i]; ++i)
if (config.rules)
{
const FILTER_PARAMETER *param = params[i];
if (strcmp(param->name, "allowed_references") == 0)
{
if (strcmp(param->value, "qualified") == 0)
{
config.allowed_references = CACHE_REFERENCES_QUALIFIED;
}
else if (strcmp(param->value, "any") == 0)
{
config.allowed_references = CACHE_REFERENCES_ANY;
rules = cache_rules_load(config.rules, config.debug);
}
else
{
MXS_ERROR("Unknown value '%s' for parameter '%s'.", param->value, param->name);
error = true;
}
}
else if (strcmp(param->name, "max_resultset_rows") == 0)
{
int v = atoi(param->value);
if (v > 0)
{
config.max_resultset_rows = v;
}
else
{
config.max_resultset_rows = CACHE_DEFAULT_MAX_RESULTSET_ROWS;
}
}
else if (strcmp(param->name, "max_resultset_size") == 0)
{
int v = atoi(param->value);
if (v > 0)
{
config.max_resultset_size = v * 1024;
}
else
{
MXS_ERROR("The value of the configuration entry '%s' must "
"be an integer larger than 0.", param->name);
error = true;
}
}
else if (strcmp(param->name, "storage_options") == 0)
{
config.storage_options = param->value;
}
else if (strcmp(param->name, "storage") == 0)
{
config.storage = param->value;
}
else if (strcmp(param->name, "ttl") == 0)
{
int v = atoi(param->value);
if (v > 0)
{
config.ttl = v;
}
else
{
MXS_ERROR("The value of the configuration entry '%s' must "
"be an integer larger than 0.", param->name);
error = true;
}
}
else if (!filter_standard_parameter(params[i]->name))
{
MXS_ERROR("Unknown configuration entry '%s'.", param->name);
error = true;
}
rules = cache_rules_create(config.debug);
}
CACHE_INSTANCE *cinstance = NULL;
if (!error)
if (rules)
{
if ((cinstance = MXS_CALLOC(1, sizeof(CACHE_INSTANCE))) != NULL)
{
@ -288,6 +225,7 @@ static FILTER *createInstance(const char *name, char **options, FILTER_PARAMETER
{
cinstance->name = name;
cinstance->config = config;
cinstance->rules = rules;
cinstance->module = module;
cinstance->storage = storage;
@ -296,6 +234,7 @@ static FILTER *createInstance(const char *name, char **options, FILTER_PARAMETER
else
{
MXS_ERROR("Could not create storage instance for %s.", name);
cache_rules_free(rules);
cache_storage_close(module);
MXS_FREE(cinstance);
cinstance = NULL;
@ -304,14 +243,12 @@ static FILTER *createInstance(const char *name, char **options, FILTER_PARAMETER
else
{
MXS_ERROR("Could not load cache storage module %s.", name);
cache_rules_free(rules);
MXS_FREE(cinstance);
cinstance = NULL;
}
}
}
else
{
cinstance = NULL;
}
return (FILTER*)cinstance;
@ -468,6 +405,10 @@ static int routeQuery(FILTER *instance, void *sdata, GWBUF *data)
// possible.
if (qc_get_operation(packet) == QUERY_OP_SELECT)
{
if (cache_rules_should_store(cinstance->rules, csdata->default_db, packet))
{
if (cache_rules_should_use(cinstance->rules, csdata->session))
{
GWBUF *result;
use_default = !route_using_cache(csdata, packet, &result);
@ -489,6 +430,12 @@ static int routeQuery(FILTER *instance, void *sdata, GWBUF *data)
}
}
}
else
{
csdata->state = CACHE_IGNORING_RESPONSE;
}
}
}
}
break;
@ -936,6 +883,125 @@ static int handle_ignoring_response(CACHE_SESSION_DATA *csdata)
return send_upstream(csdata);
}
/**
* Processes the cache params
*
* @param options Options as passed to the filter.
* @param params Parameters as passed to the filter.
* @param config Pointer to config instance where params will be stored.
*
* @return True if all parameters could be processed, false otherwise.
*/
static bool process_params(char **options, FILTER_PARAMETER **params, CACHE_CONFIG* config)
{
bool error = false;
for (int i = 0; params[i]; ++i)
{
const FILTER_PARAMETER *param = params[i];
if (strcmp(param->name, "max_resultset_rows") == 0)
{
int v = atoi(param->value);
if (v > 0)
{
config->max_resultset_rows = v;
}
else
{
config->max_resultset_rows = CACHE_DEFAULT_MAX_RESULTSET_ROWS;
}
}
else if (strcmp(param->name, "max_resultset_size") == 0)
{
int v = atoi(param->value);
if (v > 0)
{
config->max_resultset_size = v * 1024;
}
else
{
MXS_ERROR("The value of the configuration entry '%s' must "
"be an integer larger than 0.", param->name);
error = true;
}
}
else if (strcmp(param->name, "rules") == 0)
{
if (*param->value == '/')
{
config->rules = MXS_STRDUP(param->value);
}
else
{
const char *datadir = get_datadir();
size_t len = strlen(datadir) + 1 + strlen(param->value) + 1;
char *rules = MXS_MALLOC(len);
if (rules)
{
sprintf(rules, "%s/%s", datadir, param->value);
config->rules = rules;
}
}
if (!config->rules)
{
error = true;
}
}
else if (strcmp(param->name, "storage_options") == 0)
{
config->storage_options = param->value;
}
else if (strcmp(param->name, "storage") == 0)
{
config->storage = param->value;
}
else if (strcmp(param->name, "ttl") == 0)
{
int v = atoi(param->value);
if (v > 0)
{
config->ttl = v;
}
else
{
MXS_ERROR("The value of the configuration entry '%s' must "
"be an integer larger than 0.", param->name);
error = true;
}
}
else if (strcmp(param->name, "debug") == 0)
{
int v = atoi(param->value);
if ((v >= CACHE_DEBUG_MIN) && (v <= CACHE_DEBUG_MAX))
{
config->debug = v;
}
else
{
MXS_ERROR("The value of the configuration entry '%s' must "
"be between %d and %d, inclusive.",
param->name, CACHE_DEBUG_MIN, CACHE_DEBUG_MAX);
error = true;
}
}
else if (!filter_standard_parameter(params[i]->name))
{
MXS_ERROR("Unknown configuration entry '%s'.", param->name);
error = true;
}
}
return !error;
}
/**
* Route a query via the cache.
*

17
server/modules/filter/cache/cache.h vendored
View File

@ -15,19 +15,24 @@
#include <limits.h>
#define CACHE_DEBUG_NONE 0
#define CACHE_DEBUG_MATCHING 1
#define CACHE_DEBUG_NON_MATCHING 2
#define CACHE_DEBUG_USE 4
#define CACHE_DEBUG_NON_USE 8
typedef enum cache_references
{
CACHE_REFERENCES_ANY, // select * from tbl;
CACHE_REFERENCES_QUALIFIED // select * from db.tbl;
} cache_references_t;
#define CACHE_DEBUG_RULES (CACHE_DEBUG_MATCHING | CACHE_DEBUG_NON_MATCHING)
#define CACHE_DEBUG_USAGE (CACHE_DEBUG_USE | CACHE_DEBUG_NON_USE)
#define CACHE_DEBUG_MIN CACHE_DEBUG_NONE
#define CACHE_DEBUG_MAX (CACHE_DEBUG_RULES | CACHE_DEBUG_USAGE)
#define CACHE_DEFAULT_ALLOWED_REFERENCES CACHE_REFERENCES_QUALIFIED
// Count
#define CACHE_DEFAULT_MAX_RESULTSET_ROWS UINT_MAX
// Bytes
#define CACHE_DEFAULT_MAX_RESULTSET_SIZE 64 * 1024
// Seconds
#define CACHE_DEFAULT_TTL 10
// Integer value
#define CACHE_DEFAULT_DEBUG 0
#endif

795
server/modules/filter/cache/rules.c vendored Normal file
View File

@ -0,0 +1,795 @@
/*
* Copyright (c) 2016 MariaDB Corporation Ab
*
* Use of this software is governed by the Business Source License included
* in the LICENSE.TXT file and at www.mariadb.com/bsl.
*
* Change Date: 2019-07-01
*
* On the date above, in accordance with the Business Source License, use
* of this software will be governed by version 2 or later of the General
* Public License.
*/
#define MXS_MODULE_NAME "cache"
#include "rules.h"
#include <errno.h>
#include <stdio.h>
#include <maxscale/alloc.h>
#include <modutil.h>
#include <query_classifier.h>
#include <mysql_client_server_protocol.h>
#include "cache.h"
static const char KEY_ATTRIBUTE[] = "attribute";
static const char KEY_COLUMN[] = "column";
static const char KEY_OP[] = "op";
static const char KEY_QUERY[] = "query";
static const char KEY_STORE[] = "store";
static const char KEY_TABLE[] = "table";
static const char KEY_USE[] = "use";
static const char KEY_VALUE[] = "value";
static const char VALUE_ATTRIBUTE_COLUMN[] = "column";
static const char VALUE_ATTRIBUTE_DATABASE[] = "database";
static const char VALUE_ATTRIBUTE_QUERY[] = "query";
static const char VALUE_ATTRIBUTE_TABLE[] = "table";
static const char VALUE_OP_EQ[] = "=";
static const char VALUE_OP_NEQ[] = "!=";
static const char VALUE_OP_LIKE[] = "like";
static const char VALUE_OP_UNLIKE[] = "unlike";
static bool cache_rule_attribute_get(const char *s, cache_rule_attribute_t *attribute);
static const char *cache_rule_attribute_to_string(cache_rule_attribute_t attribute);
static bool cache_rule_op_get(const char *s, cache_rule_op_t *op);
static const char *cache_rule_op_to_string(cache_rule_op_t op);
static bool cache_rule_compare(CACHE_RULE *rule, const char *value);
static CACHE_RULE *cache_rule_create_regexp(cache_rule_attribute_t attribute,
cache_rule_op_t op,
const char *value,
uint32_t debug);
static CACHE_RULE *cache_rule_create_simple(cache_rule_attribute_t attribute,
cache_rule_op_t op,
const char *value,
uint32_t debug);
static CACHE_RULE *cache_rule_create(cache_rule_attribute_t attribute,
cache_rule_op_t op,
const char *value,
uint32_t debug);
static bool cache_rule_compare(CACHE_RULE *rule, const char *value);
static bool cache_rule_matches(CACHE_RULE *rule,
const char *default_db,
const GWBUF *query);
static void cache_rule_free(CACHE_RULE *rule);
static bool cache_rule_matches(CACHE_RULE *rule, const char *default_db, const GWBUF *query);
static void cache_rules_add_store_rule(CACHE_RULES* self, CACHE_RULE* rule);
static bool cache_rules_parse_json(CACHE_RULES* self, json_t* root);
static bool cache_rules_parse_store(CACHE_RULES *self, json_t *store);
static bool cache_rules_parse_store_element(CACHE_RULES *self, json_t *object, size_t index);
/*
* API begin
*/
/**
* Create a default cache rules object.
*
* @param debug The debug level.
*
* @return The rules object or NULL is allocation fails.
*/
CACHE_RULES *cache_rules_create(uint32_t debug)
{
CACHE_RULES *rules = (CACHE_RULES*)MXS_CALLOC(1, sizeof(CACHE_RULES));
if (rules)
{
rules->debug = debug;
}
return rules;
}
/**
* Loads the caching rules from a file and returns corresponding object.
*
* @param path The path of the file containing the rules.
* @param debug The debug level.
*
* @return The corresponding rules object, or NULL in case of error.
*/
CACHE_RULES *cache_rules_load(const char *path, uint32_t debug)
{
CACHE_RULES *rules = NULL;
FILE *fp = fopen(path, "r");
if (fp)
{
json_error_t error;
json_t *root = json_loadf(fp, JSON_DISABLE_EOF_CHECK, &error);
if (root)
{
rules = cache_rules_create(debug);
if (rules)
{
if (!cache_rules_parse_json(rules, root))
{
cache_rules_free(rules);
rules = NULL;
}
}
json_decref(root);
}
else
{
MXS_ERROR("Loading rules file failed: (%s:%d:%d): %s",
path, error.line, error.column, error.text);
}
fclose(fp);
}
else
{
char errbuf[STRERROR_BUFLEN];
MXS_ERROR("Could not open rules file %s for reading: %s",
path, strerror_r(errno, errbuf, sizeof(errbuf)));
}
return rules;
}
/**
* Frees the rules object.
*
* @param path The path of the file containing the rules.
*
* @return The corresponding rules object, or NULL in case of error.
*/
void cache_rules_free(CACHE_RULES *rules)
{
if (rules)
{
cache_rule_free(rules->store_rules);
MXS_FREE(rules);
}
}
/**
* Returns boolean indicating whether the result of the query should be stored.
*
* @param self The CACHE_RULES object.
* @param default_db The current default database, NULL if there is none.
* @param query The query, expected to contain a COM_QUERY.
*
* @return True, if the results should be stored.
*/
bool cache_rules_should_store(CACHE_RULES *self, const char *default_db, const GWBUF* query)
{
bool should_store = false;
CACHE_RULE *rule = self->store_rules;
if (rule)
{
while (rule && !should_store)
{
should_store = cache_rule_matches(rule, default_db, query);
rule = rule->next;
}
}
else
{
should_store = true;
}
return should_store;
}
/**
* Returns boolean indicating whether the cache should be used, that is consulted.
*
* @param self The CACHE_RULES object.
* @param session The current session.
*
* @return True, if the cache should be used.
*/
bool cache_rules_should_use(CACHE_RULES *self, const SESSION *session)
{
// TODO: Also support user.
return true;
}
/*
* API end
*/
/**
* Converts a string to an attribute
*
* @param s A string
* @param attribute On successful return contains the corresponding attribute type.
*
* @return True if the string could be converted, false otherwise.
*/
static bool cache_rule_attribute_get(const char *s, cache_rule_attribute_t *attribute)
{
if (strcmp(s, VALUE_ATTRIBUTE_COLUMN) == 0)
{
*attribute = CACHE_ATTRIBUTE_COLUMN;
return true;
}
if (strcmp(s, VALUE_ATTRIBUTE_DATABASE) == 0)
{
*attribute = CACHE_ATTRIBUTE_DATABASE;
return true;
}
if (strcmp(s, VALUE_ATTRIBUTE_QUERY) == 0)
{
*attribute = CACHE_ATTRIBUTE_QUERY;
return true;
}
if (strcmp(s, VALUE_ATTRIBUTE_TABLE) == 0)
{
*attribute = CACHE_ATTRIBUTE_TABLE;
return true;
}
return false;
}
/**
* Returns a string representation of a attribute.
*
* @param attribute An attribute type.
*
* @return Corresponding string, not to be freed.
*/
static const char *cache_rule_attribute_to_string(cache_rule_attribute_t attribute)
{
switch (attribute)
{
case CACHE_ATTRIBUTE_COLUMN:
return "column";
case CACHE_ATTRIBUTE_DATABASE:
return "database";
case CACHE_ATTRIBUTE_QUERY:
return "query";
case CACHE_ATTRIBUTE_TABLE:
return "table";
default:
ss_dassert(!true);
return "<invalid>";
}
}
/**
* Converts a string to an operator
*
* @param s A string
* @param op On successful return contains the corresponding operator.
*
* @return True if the string could be converted, false otherwise.
*/
static bool cache_rule_op_get(const char *s, cache_rule_op_t *op)
{
if (strcmp(s, VALUE_OP_EQ) == 0)
{
*op = CACHE_OP_EQ;
return true;
}
if (strcmp(s, VALUE_OP_NEQ) == 0)
{
*op = CACHE_OP_NEQ;
return true;
}
if (strcmp(s, VALUE_OP_LIKE) == 0)
{
*op = CACHE_OP_LIKE;
return true;
}
if (strcmp(s, VALUE_OP_UNLIKE) == 0)
{
*op = CACHE_OP_UNLIKE;
return true;
}
return false;
}
/**
* Returns a string representation of an operator.
*
* @param op An operator.
*
* @return Corresponding string, not to be freed.
*/
static const char *cache_rule_op_to_string(cache_rule_op_t op)
{
switch (op)
{
case CACHE_OP_EQ:
return "=";
case CACHE_OP_NEQ:
return "!=";
case CACHE_OP_LIKE:
return "like";
case CACHE_OP_UNLIKE:
return "unlike";
default:
ss_dassert(!true);
return "<invalid>";
}
}
/**
* Creates a CACHE_RULE object doing regexp matching.
*
* @param attribute What attribute this rule applies to.
* @param op An operator, CACHE_OP_LIKE or CACHE_OP_UNLIKE.
* @param value A regular expression.
* @param debug The debug level.
*
* @return A new rule object or NULL in case of failure.
*/
static CACHE_RULE *cache_rule_create_regexp(cache_rule_attribute_t attribute,
cache_rule_op_t op,
const char *cvalue,
uint32_t debug)
{
ss_dassert((op == CACHE_OP_LIKE) || (op == CACHE_OP_UNLIKE));
CACHE_RULE *rule = NULL;
int errcode;
PCRE2_SIZE erroffset;
pcre2_code *code = pcre2_compile((PCRE2_SPTR)cvalue, PCRE2_ZERO_TERMINATED, 0,
&errcode, &erroffset, NULL);
if (code)
{
pcre2_match_data *data = pcre2_match_data_create_from_pattern(code, NULL);
if (data)
{
rule = (CACHE_RULE*)MXS_CALLOC(1, sizeof(CACHE_RULE));
char* value = MXS_STRDUP(cvalue);
if (rule && value)
{
rule->attribute = attribute;
rule->op = op;
rule->value = value;
rule->regexp.code = code;
rule->regexp.data = data;
rule->debug = debug;
}
else
{
MXS_FREE(value);
MXS_FREE(rule);
pcre2_match_data_free(data);
pcre2_code_free(code);
}
}
else
{
MXS_ERROR("PCRE2 match data creation failed. Most likely due to a "
"lack of available memory.");
pcre2_code_free(code);
}
}
else
{
PCRE2_UCHAR errbuf[512];
pcre2_get_error_message(errcode, errbuf, sizeof(errbuf));
MXS_ERROR("Regex compilation failed at %d for regex '%s': %s",
(int)erroffset, cvalue, errbuf);
}
return rule;
}
/**
* Creates a CACHE_RULE object doing simple matching.
*
* @param attribute What attribute this rule applies to.
* @param op An operator, CACHE_OP_EQ or CACHE_OP_NEQ.
* @param value A string.
* @param debug The debug level.
*
* @return A new rule object or NULL in case of failure.
*/
static CACHE_RULE *cache_rule_create_simple(cache_rule_attribute_t attribute,
cache_rule_op_t op,
const char *cvalue,
uint32_t debug)
{
ss_dassert((op == CACHE_OP_EQ) || (op == CACHE_OP_NEQ));
CACHE_RULE *rule = (CACHE_RULE*)MXS_CALLOC(1, sizeof(CACHE_RULE));
char *value = MXS_STRDUP(cvalue);
if (rule && value)
{
rule->attribute = attribute;
rule->op = op;
rule->value = value;
rule->debug = debug;
}
else
{
MXS_FREE(value);
MXS_FREE(rule);
}
return rule;
}
/**
* Creates a CACHE_RULE object.
*
* @param attribute What attribute this rule applies to.
* @param op What operator is used.
* @param value The value.
* @param debug The debug level.
*
* @param rule The rule to be freed.
*/
static CACHE_RULE *cache_rule_create(cache_rule_attribute_t attribute,
cache_rule_op_t op,
const char *value,
uint32_t debug)
{
CACHE_RULE *rule = NULL;
switch (op)
{
case CACHE_OP_EQ:
case CACHE_OP_NEQ:
rule = cache_rule_create_simple(attribute, op, value, debug);
break;
case CACHE_OP_LIKE:
case CACHE_OP_UNLIKE:
rule = cache_rule_create_regexp(attribute, op, value, debug);
break;
default:
ss_dassert(!true);
MXS_ERROR("Internal error.");
break;
}
return rule;
}
/**
* Frees a CACHE_RULE object (and the one it points to).
*
* @param rule The rule to be freed.
*/
static void cache_rule_free(CACHE_RULE* rule)
{
if (rule)
{
if (rule->next)
{
cache_rule_free(rule->next);
}
MXS_FREE(rule->value);
if ((rule->op == CACHE_OP_LIKE) || (rule->op == CACHE_OP_UNLIKE))
{
pcre2_match_data_free(rule->regexp.data);
pcre2_code_free(rule->regexp.code);
}
MXS_FREE(rule);
}
}
/**
* Check whether a value matches a rule.
*
* @param self The rule object.
* @param value The value to check.
*
* @return True if the value matches, false otherwise.
*/
static bool cache_rule_compare(CACHE_RULE *self, const char *value)
{
bool compares = false;
switch (self->op)
{
case CACHE_OP_EQ:
case CACHE_OP_NEQ:
compares = (strcmp(self->value, value) == 0);
break;
case CACHE_OP_LIKE:
case CACHE_OP_UNLIKE:
compares = (pcre2_match(self->regexp.code, (PCRE2_SPTR)value,
PCRE2_ZERO_TERMINATED, 0, 0,
self->regexp.data, NULL) >= 0);
break;
default:
ss_dassert(!true);
}
if ((self->op == CACHE_OP_NEQ) || (self->op == CACHE_OP_UNLIKE))
{
compares = !compares;
}
return compares;
}
/**
* Returns boolean indicating whether the rule matches the query or not.
*
* @param self The CACHE_RULE object.
* @param default_db The current default db.
* @param query The query.
*
* @return True, if the rule matches, false otherwise.
*/
static bool cache_rule_matches(CACHE_RULE *self, const char *default_db, const GWBUF *query)
{
bool matches = false;
switch (self->attribute)
{
case CACHE_ATTRIBUTE_COLUMN:
// TODO: Not implemented yet.
ss_dassert(!true);
break;
case CACHE_ATTRIBUTE_DATABASE:
{
int n;
char **names = qc_get_database_names((GWBUF*)query, &n); // TODO: Make qc const-correct.
if (names)
{
int i = 0;
while (!matches && (i < n))
{
matches = cache_rule_compare(self, names[i]);
++i;
}
for (int i = 0; i < n; ++i)
{
MXS_FREE(names[i]);
}
MXS_FREE(names);
}
if (!matches && default_db)
{
matches = cache_rule_compare(self, default_db);
}
}
break;
case CACHE_ATTRIBUTE_TABLE:
// TODO: Not implemented yet.
ss_dassert(!true);
break;
case CACHE_ATTRIBUTE_QUERY:
// TODO: Not implemented yet.
ss_dassert(!true);
break;
default:
ss_dassert(!true);
}
if ((matches && (self->debug & CACHE_DEBUG_MATCHING)) ||
(!matches && (self->debug & CACHE_DEBUG_NON_MATCHING)))
{
const char *sql = GWBUF_DATA(query) + MYSQL_HEADER_LEN + 1; // Header + command byte.
int sql_len = GWBUF_LENGTH(query) - MYSQL_HEADER_LEN - 1;
const char* text;
if (matches)
{
text = "MATCHES";
}
else
{
text = "does NOT match";
}
MXS_NOTICE("Rule { \"attribute\": \"%s\", \"op\": \"%s\", \"value\": \"%s\" } %s \"%*s\".",
cache_rule_attribute_to_string(self->attribute),
cache_rule_op_to_string(self->op),
self->value,
text,
sql_len, sql);
}
return matches;
}
/**
* Adds a "store" rule to the rules object
*
* @param self Pointer to the CACHE_RULES object that is being built.
* @param rule The rule to be added.
*/
static void cache_rules_add_store_rule(CACHE_RULES* self, CACHE_RULE* rule)
{
if (self->store_rules)
{
CACHE_RULE *r = self->store_rules;
while (r->next)
{
r = r->next;
}
r->next = rule;
}
else
{
self->store_rules = rule;
}
}
/**
* Parses the JSON object used for configuring the rules.
*
* @param self Pointer to the CACHE_RULES object that is being built.
* @param root The root JSON object in the rules file.
*
* @return True, if the object could be parsed, false otherwise.
*/
static bool cache_rules_parse_json(CACHE_RULES *self, json_t *root)
{
bool parsed = false;
json_t *store = json_object_get(root, KEY_STORE);
if (store)
{
if (json_is_array(store))
{
parsed = cache_rules_parse_store(self, store);
}
else
{
MXS_ERROR("The cache rules object contains a `store` key, but it is not an array.");
}
}
// TODO: Parse 'use' as well.
return parsed;
}
/**
* Parses the "store" array.
*
* @param self Pointer to the CACHE_RULES object that is being built.
* @param store The "store" array.
*
* @return True, if the array could be parsed, false otherwise.
*/
static bool cache_rules_parse_store(CACHE_RULES *self, json_t *store)
{
ss_dassert(json_is_array(store));
bool parsed = true;
size_t n = json_array_size(store);
size_t i = 0;
while (parsed && (i < n))
{
json_t *element = json_array_get(store, i);
ss_dassert(element);
if (json_is_object(element))
{
parsed = cache_rules_parse_store_element(self, element, i);
}
else
{
MXS_ERROR("Element %lu of the 'store' array is not an object.", i);
parsed = false;
}
++i;
}
return parsed;
}
/**
* Parses an object in the "store" array.
*
* @param self Pointer to the CACHE_RULES object that is being built.
* @param object An object from the "store" array.
* @param index Index of the object in the array.
*
* @return True, if the object could be parsed, false otherwise.
*/
static bool cache_rules_parse_store_element(CACHE_RULES *self, json_t *object, size_t index)
{
bool parsed = false;
ss_dassert(json_is_object(object));
json_t *a = json_object_get(object, KEY_ATTRIBUTE);
json_t *o = json_object_get(object, KEY_OP);
json_t *v = json_object_get(object, KEY_VALUE);
if (a && o && v && json_is_string(a) && json_is_string(o) && json_is_string(v))
{
cache_rule_attribute_t attribute;
if (cache_rule_attribute_get(json_string_value(a), &attribute))
{
cache_rule_op_t op;
if (cache_rule_op_get(json_string_value(o), &op))
{
CACHE_RULE *rule = cache_rule_create(attribute, op, json_string_value(v), self->debug);
if (rule)
{
cache_rules_add_store_rule(self, rule);
parsed = true;
}
}
else
{
MXS_ERROR("Element %lu in the `store` array has an invalid value "
"\"%s\" for 'op'.", index, json_string_value(o));
}
}
else
{
MXS_ERROR("Element %lu in the `store` array has an invalid value "
"\"%s\" for 'attribute'.", index, json_string_value(a));
}
}
else
{
MXS_ERROR("Element %lu in the `store` array does not contain "
"'attribute', 'op' and/or 'value', or one or all of them "
"is not a string.", index);
}
return parsed;
}

69
server/modules/filter/cache/rules.h vendored Normal file
View File

@ -0,0 +1,69 @@
#ifndef _MAXSCALE_FILTER_CACHE_RULES_H
#define _MAXSCALE_FILTER_CACHE_RULES_H
/*
* Copyright (c) 2016 MariaDB Corporation Ab
*
* Use of this software is governed by the Business Source License included
* in the LICENSE.TXT file and at www.mariadb.com/bsl.
*
* Change Date: 2019-07-01
*
* On the date above, in accordance with the Business Source License, use
* of this software will be governed by version 2 or later of the General
* Public License.
*/
#include <stdbool.h>
#include <jansson.h>
#include <buffer.h>
#include <session.h>
#include <maxscale_pcre2.h>
typedef enum cache_rule_attribute
{
CACHE_ATTRIBUTE_COLUMN,
CACHE_ATTRIBUTE_DATABASE,
CACHE_ATTRIBUTE_QUERY,
CACHE_ATTRIBUTE_TABLE,
} cache_rule_attribute_t;
typedef enum cache_rule_op
{
CACHE_OP_EQ,
CACHE_OP_NEQ,
CACHE_OP_LIKE,
CACHE_OP_UNLIKE
} cache_rule_op_t;
typedef struct cache_rule
{
cache_rule_attribute_t attribute; // What attribute is evalued.
cache_rule_op_t op; // What operator is used.
char *value; // The value from the rule file.
struct
{
pcre2_code* code;
pcre2_match_data* data;
} regexp; // Regexp data, only for CACHE_OP_[LIKE|UNLIKE].
uint32_t debug; // The debug level.
struct cache_rule *next;
} CACHE_RULE;
typedef struct cache_rules
{
uint32_t debug; // The debug level.
CACHE_RULE *store_rules; // The rules for 'store'.
} CACHE_RULES;
CACHE_RULES *cache_rules_create(uint32_t debug);
void cache_rules_free(CACHE_RULES *rules);
CACHE_RULES *cache_rules_load(const char *path, uint32_t debug);
bool cache_rules_should_store(CACHE_RULES *rules, const char *default_db, const GWBUF* query);
bool cache_rules_should_use(CACHE_RULES *rules, const SESSION *session);
#endif