hcq/MaxScale
hcq/MaxScale
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Further simplify key generation

Browse Source 
The original approach was made for RocksDB where it is beneficial
to keep keys of stuff related to each other close to each other.
However, as RocksDB is no longer the primary focus, it just causes
additional cost to dig out the table names.

The key is a 64-bit integer, but crc32 only gives us a 32-bit one.
We create an other 32-bit value by running crc32 over the same SQL,
using the first crc value as adler.

I think that further reduces the chance for clashes:

    uint32_t crc0 = crc32(0, Z_NULL, 0);
    uint32_t crc1;
    uint32_t crc2;

    crc1 = crc32(crc0, "codding", 7) => 1774765869
    crc2 = crc32(crc1, "codding", 7) => 1409592046

    crc1 = crc32(crc0, "gnu", 3)     => 1774765869
    crc2 = crc32(crc1, "gnu", 3)     => 1213798908

Note that the first value is the same, but the second is not.
This commit is contained in:
Johan Wikman 2017-03-16 18:39:10 +02:00
parent daf3774329
commit 987a52b398
1 changed files with 15 additions and 47 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

62
server/modules/filter/cache/cache.cc vendored
View File

@ -125,59 +125,27 @@ cache_result_t Cache::get_default_key(const char* zDefault_db,
{
ss_dassert(GWBUF_IS_CONTIGUOUS(pQuery));
int n;
bool fullnames = true;
char** pzTables = qc_get_table_names(const_cast<GWBUF*>(pQuery), &n, fullnames);
set<string> dbs; // Elements in set are sorted.
for (int i = 0; i < n; ++i)
{
char *zTable = pzTables[i];
char *zDot = strchr(zTable, '.');
if (zDot)
{
*zDot = 0;
dbs.insert(zTable);
}
else if (zDefault_db)
{
// If zdefault_db is NULL, then there will be a table for which we
// do not know the database. However, that will fail in the server,
// so nothing will be stored.
dbs.insert(zDefault_db);
}
MXS_FREE(zTable);
}
MXS_FREE(pzTables);
// dbs now contain each accessed database in sorted order. Now copy them to a single string.
string tag;
for (set<string>::const_iterator i = dbs.begin(); i != dbs.end(); ++i)
{
tag.append(*i);
}
const unsigned char* pData;
// We hash the databases in the first half of the key. That will ensure that
// identical queries targeting different default databases will not clash.
pData = reinterpret_cast<const unsigned char*>(tag.data());
uint64_t table_crc = crc32(0, Z_NULL, 0);
table_crc = crc32(table_crc, pData, tag.length());
char *pSql;
int length;
modutil_extract_SQL(const_cast<GWBUF*>(pQuery), &pSql, &length);
// Then we hash the query itself in the second half of the key.
pData = reinterpret_cast<const unsigned char*>(pSql);
uint64_t stmt_crc = crc32(0, Z_NULL, 0);
stmt_crc = crc32(stmt_crc, pData, length);
uint64_t crc1 = crc32(0, Z_NULL, 0);
pKey->data = (table_crc << 32) | stmt_crc;
const Bytef* pData;
if (zDefault_db)
{
pData = reinterpret_cast<const Bytef*>(zDefault_db);
crc1 = crc32(crc1, pData, strlen(zDefault_db));
}
pData = reinterpret_cast<const Bytef*>(pSql);
crc1 = crc32(crc1, pData, length);
uint64_t crc2 = crc32(crc1, pData, length);
pKey->data = (crc1 << 32 | crc2);
return CACHE_RESULT_OK;
}