hcq/MaxScale
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

MXS-1475 Add @maxscale.cache.[populate|use]

Browse Source 
The earlier @maxscale.cache.enabled has now been replaced with
@maxscale.cache.populate and @maxscale.cache.use that provide
for more flexibility.

With the former it is possible to control in what circumstances
the cache is populated and with the latter one when it is used.
Together they can be used for having a completely client driven
caching.
This commit is contained in:
Johan Wikman
2018-03-09 13:47:34 +02:00
parent 34dd8a52bb
commit 82b55ff362
6 changed files with 356 additions and 190 deletions
Download Patch File Download Diff File Expand all files Collapse all files

115
Documentation/Filters/Cache.md
View File

@ -311,39 +311,118 @@ enabled=false
```
Default is `true`.
Note that this affects only whether data from the cache is returned; the
cache will be populated normally. Please see
The value affects the initial state of the MaxScale user
variables using which the behaviour of the cache can be modified
at runtime. Please see
[Runtime Configuration](#runtime-configuation)
for how to enable/disable the cache at runtime.
for details.
## Runtime Configuration
Using the variable `@maxscale.cache.enabled` it is possible to specify at
runtime whether the cache should be used or not. Its initial value is the
value of the configuration parameter `enabled`. That is, by default the
### `@maxscale.cache.populate`
Using the variable `@maxscale.cache.populate` it is possible to specify at
runtime whether the cache should be populated or not. Its initial value is
the value of the configuration parameter `enabled`. That is, by default the
value is `true`.
The purpose of this variable is make it possible for an application to decide
statement by statement whether data from the cache can be returned.
statement by statement whether the cache should be populated.
```
set @maxscale.cache.enabled=false;
SET @maxscale.cache.populate=true;
SELECT a, b FROM tbl;
SET @maxscale.cache.populate=false;
SELECT a, b FROM tbl;
set @maxscale.cache.enabled=true;
select c, d FROM tbl;
```
In the example above, the first `SELECT` will always be sent to the
server, while the latter will be served from the cache, unless the data is
stale.
server and the result will be cached, provided the actual cache rules
specifies that it should be. The second `SELECT` may be served from the
cache, depending on the value of `@maxscale.cache.use` (and the cache
rules).
Note that the value of `@maxscale.cache.enabled` has no impact on the
population of the cache; if a `SELECT` matches the rules then the data
returned from the cache will be populated.
The value of `@maxscale.cache.populate` can be queried
```
SELECT @maxscale.cache.populate;
```
but only _after_ it has been explicitly set once.
The value of `@maxscale.cache.enabled` can be queried
### `@maxscale.cache.use`
Using the variable `@maxscale.cache.use` it is possible to specify at
runtime whether the cache should be used or not. Its initial value is
the value of the configuration parameter `enabled`. That is, by default the
value is `true`.
The purpose of this variable is make it possible for an application to decide
statement by statement whether the cache should be used.
```
select @maxscale.cache.enabled;
SET @maxscale.cache.use=true;
SELECT a, b FROM tbl;
SET @maxscale.cache.use=false;
SELECT a, b FROM tbl;
```
The first `SELECT` will be served from the cache, providing the rules
specify that the statement should be cached, the cache indeed contains
the result and the date is not stale (as specified by the _TTL_).
If the data is stale, the `SELECT` will be sent to the server **and**
the cache entry will be updated, irrespective of the value of
`@maxscale.cache.populate`.
If `@maxscale.cache.use` is `true` but the result is not found in the
cache, and the result is subsequently fetched from the server, the
result will **not** be added to the cache, unless
`@maxscale.cache.populate` is also `true`.
The value of `@maxscale.cache.use` can be queried
```
SELECT @maxscale.cache.use;
```
but only after it has explicitly been set once.
### Client Driven Caching
With `@maxscale.cache.populate` and `@maxscale.cache.use` is it possible
to make the caching completely client driven.
Provide no `rules` file, which means that _all_ `SELECT` statements are
subject to caching and that all users receive data from the cache. Set
the startup mode of the cache to _disabled_.
```
[TheCache]
type=filter
module=cache
enabled=false
```
Now, in order to _mark_ statements that should be cached, set
`@maxscale.cache.populate` to `true`, and perform those `SELECT`s.
```
SET @maxscale.cache.populate=true;
SELECT a, b FROM tbl1;
SELECT c, d FROM tbl2;
SELECT e, f FROM tbl3;
SET @maxscale.cache.populate=false;
```
Note that those `SELECT`s must return something in order for the
statement to be _marked_ for caching.
After this, the value of `@maxscale.cache.use` will decide whether
or not the cache is considered.
```
SET @maxscale.cache.use=true;
SELECT a, b FROM tbl1;
SET @maxscale.cache.use=false;
```
With `@maxscale.cache.use` being `true`, the cache is considered
and the result returned from there, if not stale. If it is stale,
the result is fetched from the server and the cached entry is updated.
By setting a very long _TTL_ it is possible to prevent the cache
from ever considering an entry to be stale and instead manually
cause the cache to be updated when needed.
```
UPDATE tbl1 SET a = ...;
SET @maxscale.cache.populate=true;
SELECT a, b FROM tbl1;
SET @maxscale.cache.populate=false;
```
but only after it has been set once.
# Rules

4
include/maxscale/session.h
View File

@ -133,7 +133,9 @@ typedef struct mxs_upstream
* has been removed. The handler must itself parse the value string.
*
* @param context Context provided when handler was registered.
* @param name The variable that is being set.
* @param name The variable that is being set. Note that it
* will always be in all lower-case irrespective
* of the case used when registering.
* @param value_begin The beginning of the value as specified in the
* "set @maxscale.x.y = VALUE" statement.
* @param value_end One past the end of the VALUE.

4
server/core/session.cc
View File

@ -1119,7 +1119,7 @@ bool session_add_variable(MXS_SESSION* session,
{
string key(name);
std::transform(key.begin(), key.end(), key.begin(), toupper);
std::transform(key.begin(), key.end(), key.begin(), tolower);
if (session->variables->find(key) == session->variables->end())
{
@ -1153,7 +1153,7 @@ char* session_set_variable_value(MXS_SESSION* session,
string key(name_begin, name_end - name_begin);
transform(key.begin(), key.end(), key.begin(), toupper);
transform(key.begin(), key.end(), key.begin(), tolower);
SessionVarsByName::iterator i = session->variables->find(key);

156
server/modules/filter/cache/cachefiltersession.cc vendored
View File

@ -38,7 +38,8 @@ inline bool cache_max_resultset_size_exceeded(const CACHE_CONFIG& config, uint64
namespace
{
const char SV_MAXSCALE_CACHE_ENABLED[] = "@maxscale.cache.enabled";
const char SV_MAXSCALE_CACHE_POPULATE[] = "@maxscale.cache.populate";
const char SV_MAXSCALE_CACHE_USE[] = "@maxscale.cache.use";
const char* NON_CACHEABLE_FUNCTIONS[] =
{
@ -187,32 +188,32 @@ CacheFilterSession::CacheFilterSession(MXS_SESSION* pSession, Cache* pCache, cha
, m_zUseDb(NULL)
, m_refreshing(false)
, m_is_read_only(true)
, m_enabled(pCache->config().enabled)
, m_variable_added(false)
, m_use(pCache->config().enabled)
, m_populate(pCache->config().enabled)
{
m_key.data = 0;
reset_response_state();
if (session_add_variable(pSession, SV_MAXSCALE_CACHE_ENABLED,
if (!session_add_variable(pSession, SV_MAXSCALE_CACHE_POPULATE,
&CacheFilterSession::session_variable_handler, this))
{
m_variable_added = true;
MXS_ERROR("Could not add MaxScale user variable '%s', dynamically "
"enabling/disabling the populating of the cache is not possible.",
SV_MAXSCALE_CACHE_POPULATE);
}
else
if (!session_add_variable(pSession, SV_MAXSCALE_CACHE_USE,
&CacheFilterSession::session_variable_handler, this))
{
MXS_ERROR("Could not add MaxScale user variable '%s', dynamically "
"enabling/disabling caching not possible.", SV_MAXSCALE_CACHE_ENABLED);
"enabling/disabling the using of the cache not possible.",
SV_MAXSCALE_CACHE_USE);
}
}
CacheFilterSession::~CacheFilterSession()
{
if (m_variable_added)
{
session_remove_variable(m_pSession, SV_MAXSCALE_CACHE_ENABLED, NULL);
}
MXS_FREE(m_zUseDb);
MXS_FREE(m_zDefaultDb);
}
@ -766,6 +767,8 @@ CacheFilterSession::cache_action_t CacheFilterSession::get_cache_action(GWBUF* p
{
cache_action_t action = CACHE_IGNORE;
if (m_use || m_populate)
{
uint32_t type_mask = qc_get_trx_type_mask(pPacket); // Note, only trx-related type mask
const char* zReason = NULL;
@ -890,11 +893,40 @@ CacheFilterSession::cache_action_t CacheFilterSession::get_cache_action(GWBUF* p
}
}
if (!m_enabled && (action == CACHE_USE_AND_POPULATE))
if (action == CACHE_USE_AND_POPULATE)
{
if (!m_use && !m_populate)
{
action = CACHE_IGNORE;
zReason = "usage and populating disabled";
}
else if (!m_use)
{
action = CACHE_POPULATE;
zReason = "usage disabled";
}
else if (!m_populate)
{
action = CACHE_USE;
zReason = "populating disabled";
}
}
else if (action == CACHE_USE)
{
if (!m_use)
{
action = CACHE_IGNORE;
zReason = "usage disabled";
}
}
else if (action == CACHE_POPULATE)
{
if (!m_populate)
{
action = CACHE_IGNORE;
zReason = "populating disabled";
}
}
if (log_decisions())
{
@ -922,6 +954,14 @@ CacheFilterSession::cache_action_t CacheFilterSession::get_cache_action(GWBUF* p
ss_dassert(zReason);
MXS_NOTICE(zFormat, zDecision, length, pSql, zReason);
}
}
else
{
if (log_decisions())
{
MXS_NOTICE("IGNORE: Both 'use' and 'populate' are disabled.");
}
}
return action;
}
@ -1039,10 +1079,17 @@ CacheFilterSession::routing_action_t CacheFilterSession::route_SELECT(cache_acti
}
if (routing_action == ROUTING_CONTINUE)
{
if (m_populate || m_refreshing)
{
m_state = CACHE_EXPECTING_RESPONSE;
}
else
{
m_state = CACHE_IGNORING_RESPONSE;
}
}
else
{
m_state = CACHE_EXPECTING_NOTHING;
gwbuf_free(pPacket);
@ -1053,10 +1100,8 @@ CacheFilterSession::routing_action_t CacheFilterSession::route_SELECT(cache_acti
dcb->func.write(dcb, pResponse);
}
}
else
else if (should_populate(cache_action))
{
ss_dassert(should_populate(cache_action));
// We will not use any value in the cache, but we will update
// the existing value.
if (log_decisions())
@ -1066,54 +1111,91 @@ CacheFilterSession::routing_action_t CacheFilterSession::route_SELECT(cache_acti
}
m_state = CACHE_EXPECTING_RESPONSE;
}
else
{
// We will not use any value in the cache and we will not
// update the existing value either.
if (log_decisions())
{
MXS_NOTICE("Fetching data from server, without storing to the cache.");
}
m_state = CACHE_IGNORING_RESPONSE;
}
return routing_action;
}
char* CacheFilterSession::handle_session_variable(const char* zName,
const char* pValue_begin,
const char* pValue_end)
namespace
{
bool get_truth_value(const char* begin, const char* end, bool* pValue)
{
bool rv = false;
static const char TRUE[] = "true";
static const char FALSE[] = "false";
static const size_t nTrue = sizeof(TRUE) - 1;
static const size_t nFalse = sizeof(FALSE) - 1;
char* message = NULL;
int len = (end - begin);
int len = (pValue_end - pValue_begin);
if (((len == nTrue) && (strncasecmp(pValue_begin, TRUE, nTrue) == 0)) ||
((len == 1) && (*pValue_begin == '1')))
if (((len == nTrue) && (strncasecmp(begin, TRUE, nTrue) == 0)) ||
((len == 1) && (*begin == '1')))
{
MXS_INFO("Caching enabled.");
m_enabled = true;
*pValue = true;
rv = true;
}
else if (((len == nFalse) && (strncasecmp(pValue_begin, FALSE, nFalse) == 0)) ||
((len == 1) && (*pValue_begin == '0')))
else if (((len == nFalse) && (strncasecmp(begin, FALSE, nFalse) == 0)) ||
((len == 1) && (*begin == '0')))
{
MXS_INFO("Caching disabled.");
m_enabled = false;
*pValue = false;
rv = true;
}
return rv;
}
}
char* CacheFilterSession::handle_session_variable(const char* zName,
const char* pValue_begin,
const char* pValue_end)
{
char* zMessage = NULL;
bool enabled;
if (get_truth_value(pValue_begin, pValue_end, &enabled))
{
if (strcmp(zName, SV_MAXSCALE_CACHE_POPULATE) == 0)
{
m_populate = enabled;
}
else
{
ss_dassert(strcmp(zName, SV_MAXSCALE_CACHE_USE) == 0);
m_use = enabled;
}
}
else
{
static const char FORMAT[] = "The variable %s can only have the values true/false/1/0";
int n = snprintf(NULL, 0, FORMAT, SV_MAXSCALE_CACHE_ENABLED) + 1;
int n = snprintf(NULL, 0, FORMAT, zName) + 1;
message = static_cast<char*>(MXS_MALLOC(n));
zMessage = static_cast<char*>(MXS_MALLOC(n));
if (message)
if (zMessage)
{
sprintf(message, FORMAT, SV_MAXSCALE_CACHE_ENABLED);
sprintf(zMessage, FORMAT, zName);
}
MXS_WARNING("Attempt to set the variable %s to the invalid value \"%.*s\".",
SV_MAXSCALE_CACHE_ENABLED, len, pValue_begin);
zName, n, pValue_begin);
}
return message;
return zMessage;
}
//static

4
server/modules/filter/cache/cachefiltersession.hh vendored
View File

@ -162,7 +162,7 @@ private:
char* m_zUseDb; /**< Pending default database. Needs server response. */
bool m_refreshing; /**< Whether the session is updating a stale cache entry. */
bool m_is_read_only;/**< Whether the current trx has been read-only in pratice. */
bool m_enabled; /**< Whether caching is enabled for this session. */
bool m_variable_added; /*<< Whether the variable was added or not. */
bool m_use; /**< Whether the cache should be used in this session. */
bool m_populate; /**< Whether the cache should be populated in this session. */
};

3
server/modules/filter/test/mock_session.cc
View File

@ -37,11 +37,14 @@ Session::Session(Client* pClient)
strcpy(m_mysql_session.db, "dummy");
pSession->variables = new SessionVarsByName;
m_client_dcb.data = &m_mysql_session;
}
Session::~Session()
{
delete variables;
}
Client& Session::client() const