hcq/MaxScale
hcq/MaxScale
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch '2.2' into develop

Browse Source
This commit is contained in:
Markus Mäkelä 2018-06-13 00:25:56 +03:00
commit 8094c67ac2
No known key found for this signature in database
GPG Key ID: 72D48FCE664F7B19
12 changed files with 154 additions and 78 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

51
Documentation/Routers/ReadConnRoute.md
View File

@ -1,20 +1,30 @@
# Readconnroute
This document provides an overview of the **readconnroute** router module and its intended use case scenarios. It also displays all router configuration parameters with their descriptions.
This document provides an overview of the **readconnroute** router module
and its intended use case scenarios. It also displays all router
configuration parameters with their descriptions.
## Overview
The readconnroute router provides simple and lightweight load balancing across a set of servers. The router can also be configured to balance connections based on a weighting parameter defined in the server's section.
The readconnroute router provides simple and lightweight load balancing
across a set of servers. The router can also be configured to balance
connections based on a weighting parameter defined in the server's section.
## Configuration
For more details about the standard service parameters, refer to the [Configuration Guide](../Getting-Started/Configuration-Guide.md).
For more details about the standard service parameters, refer to the
[Configuration Guide](../Getting-Started/Configuration-Guide.md).
### Router Options
**`router_options`** can contain a list of valid server roles. These roles are used as the valid types of servers the router will form connections to when new sessions are created.
**`router_options`** can contain a comma separated list of valid server
roles. These roles are used as the valid types of servers the router will
form connections to when new sessions are created.
Examples:
```
router_options=slave
router_options=slave
router_options=master,slave
```
Here is a list of all possible values for the `router_options`.
@ -26,22 +36,30 @@ synced| A Galera cluster node which is in a synced state with the cluster.
ndb|A MySQL Replication Cluster node
running|A server that is up and running. All servers that MariaDB MaxScale can connect to are labeled as running.
If no `router_options` parameter is configured in the service definition, the router will use the default value of `running`. This means that it will load balance connections across all running servers defined in the `servers` parameter of the service.
If no `router_options` parameter is configured in the service definition,
the router will use the default value of `running`. This means that it will
load balance connections across all running servers defined in the `servers`
parameter of the service.
When a connection is being created and the candidate server is being chosen, the
list of servers is processed in from first entry to last. This means that if two
servers with equal weight and status are found, the one that's listed first in
the _servers_ parameter for the service is chosen.
When a connection is being created and the candidate server is being chosen,
the list of servers is processed in from first entry to last. This means
that if two servers with equal weight and status are found, the one that's
listed first in the _servers_ parameter for the service is chosen.
## Limitations
For a list of readconnroute limitations, please read the [Limitations](../About/Limitations.md) document.
For a list of readconnroute limitations, please read the
[Limitations](../About/Limitations.md) document.
## Examples
The most common use for the readconnroute is to provide either a read or write port for an application. This provides a more lightweight routing solution than the more complex readwritesplit router but requires the application to be able to use distinct write and read ports.
The most common use for the readconnroute is to provide either a read or
write port for an application. This provides a more lightweight routing
solution than the more complex readwritesplit router but requires the
application to be able to use distinct write and read ports.
To configure a read-only service that tolerates master failures, we first need to add a new section in to the configuration file.
To configure a read-only service that tolerates master failures, we first
need to add a new section in to the configuration file.
```
[Read Service]
@ -51,6 +69,9 @@ servers=slave1,slave2,slave3
router_options=slave
```
Here the `router_options` designates slaves as the only valid server type. With this configuration, the queries are load balanced across the slave servers.
Here the `router_options` designates slaves as the only valid server
type. With this configuration, the queries are load balanced across the
slave servers.
For more complex examples of the readconnroute router, take a look at the examples in the [Tutorials](../Tutorials) folder.
For more complex examples of the readconnroute router, take a look at the
examples in the [Tutorials](../Tutorials) folder.

3
include/maxscale/query_classifier.h
View File

@ -84,7 +84,8 @@ typedef enum qc_query_type
QUERY_TYPE_CREATE_TMP_TABLE = 0x080000, /*< Create temporary table:master (could be all) */
QUERY_TYPE_READ_TMP_TABLE = 0x100000, /*< Read temporary table:master (could be any) */
QUERY_TYPE_SHOW_DATABASES = 0x200000, /*< Show list of databases */
QUERY_TYPE_SHOW_TABLES = 0x400000 /*< Show list of tables */
QUERY_TYPE_SHOW_TABLES = 0x400000, /*< Show list of tables */
QUERY_TYPE_DEALLOC_PREPARE = 0x1000000 /*< Dealloc named prepare stmt:all */
} qc_query_type_t;
/**

15
include/maxscale/queryclassifier.hh
View File

@ -217,6 +217,13 @@ public:
*/
void ps_store(GWBUF* buffer, uint32_t id);
/**
* @brief Remove a prepared statement
*
* @param buffer Buffer containing a DEALLOCATE statement or a binary protocol command
*/
void ps_erase(GWBUF* buffer);
/**
* @brief Store a mapping from an external id to the corresponding internal id
*
@ -297,14 +304,6 @@ private:
uint32_t ps_get_type(uint32_t id) const;
uint32_t ps_get_type(std::string id) const;
/**
* @brief Remove a prepared statement
*
* @param id Statement identifier to remove
*/
void ps_erase(std::string id);
void ps_erase(uint32_t id);
/**
* @brief Get the internal ID for the given binary prepared statement
*

22
query_classifier/qc_sqlite/qc_sqlite.cc
View File

@ -2110,15 +2110,25 @@ public:
update_names(zDatabase, table, NULL, NULL);
}
void maxscaleComment()
int maxscaleComment()
{
ss_dassert(this_thread.initialized);
// We are regularily parsing if the thread has been initialized.
// In that case # should be interpreted as the start of a comment,
// otherwise it should not.
int regular_parsing = false;
if (m_status == QC_QUERY_INVALID)
if (this_thread.initialized)
{
m_status = QC_QUERY_PARSED;
m_type_mask = QUERY_TYPE_READ;
regular_parsing = true;
if (m_status == QC_QUERY_INVALID)
{
m_status = QC_QUERY_PARSED;
m_type_mask = QUERY_TYPE_READ;
}
}
return regular_parsing;
}
void maxscaleDeclare(Parse* pParse)
@ -2136,7 +2146,7 @@ public:
ss_dassert(this_thread.initialized);
m_status = QC_QUERY_PARSED;
m_type_mask = QUERY_TYPE_WRITE;
m_type_mask = QUERY_TYPE_DEALLOC_PREPARE;
// If information is collected in several passes, then we may
// this information already.

11
query_classifier/qc_sqlite/sqlite-src-3110100/src/tokenize.c
View File

@ -198,6 +198,7 @@ int sqlite3IsIdChar(u8 c){ return IdChar(c); }
** Store the token type in *tokenType before returning.
*/
#ifdef MAXSCALE
extern int maxscaleComment();
int sqlite3GetToken(Parse* pParse, const unsigned char *z, int *tokenType){
#else
int sqlite3GetToken(const unsigned char *z, int *tokenType){
@ -219,8 +220,7 @@ int sqlite3GetToken(const unsigned char *z, int *tokenType){
case CC_MINUS: {
if( z[1]=='-' ){
#ifdef MAXSCALE
extern void maxscaleComment();
maxscaleComment();
maxscaleComment();
#endif
for(i=2; (c=z[i])!=0 && c!='\n'; i++){}
*tokenType = TK_SPACE; /* IMP: R-22934-25134 */
@ -466,6 +466,13 @@ int sqlite3GetToken(const unsigned char *z, int *tokenType){
testcase( z[0]=='$' ); testcase( z[0]=='@' );
testcase( z[0]==':' ); testcase( z[0]=='#' );
#ifdef MAXSCALE
if (z[0]=='#') {
if (maxscaleComment()) {
for(i=1; (c=z[i])!=0 && c!='\n'; i++){}
*tokenType = TK_SPACE;
return i;
}
}
if (z[0]==':' && z[1]=='=') {
*tokenType = TK_EQ;
return 2;

1
query_classifier/test/expected.sql
View File

@ -23,3 +23,4 @@ QUERY_TYPE_READ|QUERY_TYPE_WRITE
QUERY_TYPE_READ|QUERY_TYPE_WRITE
QUERY_TYPE_READ|QUERY_TYPE_WRITE
QUERY_TYPE_READ|QUERY_TYPE_WRITE
QUERY_TYPE_DEALLOC_PREPARE

1
query_classifier/test/input.sql
View File

@ -23,3 +23,4 @@ SELECT GET_LOCK('lock1',10);
SELECT IS_FREE_LOCK('lock1');
SELECT IS_USED_LOCK('lock1');
SELECT RELEASE_LOCK('lock1');
deallocate prepare select_stmt;

46
query_classifier/test/testreader.cc
View File

@ -37,6 +37,7 @@ enum skip_action_t
typedef std::map<std::string, skip_action_t> KeywordActionMapping;
static KeywordActionMapping mtl_keywords;
static KeywordActionMapping plsql_keywords;
void init_keywords()
{
@ -46,7 +47,7 @@ void init_keywords()
skip_action_t action;
};
static const Keyword KEYWORDS[] =
static const Keyword MTL_KEYWORDS[] =
{
{ "append_file", SKIP_LINE },
{ "cat_file", SKIP_LINE },
@ -91,10 +92,8 @@ void init_keywords()
{ "error", SKIP_NEXT_STATEMENT },
{ "eval", SKIP_STATEMENT },
{ "exec", SKIP_LINE },
{ "exit", SKIP_LINE },
{ "file_exists", SKIP_LINE },
{ "horizontal_results", SKIP_LINE },
{ "if", SKIP_BLOCK },
{ "inc", SKIP_LINE },
{ "let", SKIP_LINE },
{ "let", SKIP_LINE },
@ -138,15 +137,28 @@ void init_keywords()
{ "sync_with_master", SKIP_LINE },
{ "system", SKIP_LINE },
{ "vertical_results", SKIP_LINE },
{ "while", SKIP_BLOCK },
{ "write_file", SKIP_LINE },
};
const size_t N_KEYWORDS = sizeof(KEYWORDS)/sizeof(KEYWORDS[0]);
const size_t N_MTL_KEYWORDS = sizeof(MTL_KEYWORDS)/sizeof(MTL_KEYWORDS[0]);
for (size_t i = 0; i < N_KEYWORDS; ++i)
for (size_t i = 0; i < N_MTL_KEYWORDS; ++i)
{
mtl_keywords[KEYWORDS[i].z_keyword] = KEYWORDS[i].action;
mtl_keywords[MTL_KEYWORDS[i].z_keyword] = MTL_KEYWORDS[i].action;
}
static const Keyword PLSQL_KEYWORDS[] =
{
{ "exit", SKIP_LINE },
{ "if", SKIP_BLOCK },
{ "while", SKIP_BLOCK },
};
const size_t N_PLSQL_KEYWORDS = sizeof(PLSQL_KEYWORDS)/sizeof(PLSQL_KEYWORDS[0]);
for (size_t i = 0; i < N_PLSQL_KEYWORDS; ++i)
{
plsql_keywords[PLSQL_KEYWORDS[i].z_keyword] = PLSQL_KEYWORDS[i].action;
}
}
@ -163,18 +175,24 @@ skip_action_t get_action(const string& keyword, const string& delimiter)
// be handled explicitly.
action = SKIP_DELIMITER;
}
else if (delimiter == ";")
else
{
KeywordActionMapping::iterator i = mtl_keywords.find(key);
if (i != mtl_keywords.end())
{
action = i->second;
}
}
if ((action == SKIP_NOTHING) && (delimiter == ";"))
{
// Some mysqltest keywords, such as "while", "exit" and "if" are also
// PL/SQL keywords. We assume they can only be used in the former role,
// if the delimiter is ";".
string key(keyword);
KeywordActionMapping::iterator i = plsql_keywords.find(key);
std::transform(key.begin(), key.end(), key.begin(), ::tolower);
KeywordActionMapping::iterator i = mtl_keywords.find(key);
if (i != mtl_keywords.end())
if (i != plsql_keywords.end())
{
action = i->second;
}

11
server/core/dcb.cc
View File

@ -2915,10 +2915,15 @@ public:
dcb && atomic_load_int32(&m_more);
dcb = dcb->thread.next)
{
if (!m_func(dcb, m_data))
ss_dassert(dcb->session);
if (dcb->session->state != SESSION_STATE_DUMMY)
{
atomic_store_int32(&m_more, 0);
break;
if (!m_func(dcb, m_data))
{
atomic_store_int32(&m_more, 0);
break;
}
}
}
}

64
server/core/queryclassifier.cc
View File

@ -26,6 +26,22 @@ using namespace maxscale;
const int QC_TRACE_MSG_LEN = 1000;
// Copy of mxs_mysql_extract_ps_id() in modules/protocol/MySQL/mysql_common.cc,
// but we do not want to create a dependency from maxscale-common to that.
uint32_t mysql_extract_ps_id(GWBUF* buffer)
{
uint32_t rval = 0;
uint8_t id[MYSQL_PS_ID_SIZE];
if (gwbuf_copy_data(buffer, MYSQL_PS_ID_OFFSET, sizeof(id), id) == sizeof(id))
{
rval = gw_mysql_get_byte4(id);
}
return rval;
}
// Copied from mysql_common.c
// TODO: The current database should somehow be available in a generic fashion.
const char* qc_mysql_get_current_db(MXS_SESSION* session)
@ -285,6 +301,24 @@ public:
}
}
void erase(GWBUF* buffer)
{
uint8_t cmd = mxs_mysql_get_command(buffer);
if (cmd == MXS_COM_QUERY)
{
erase(get_text_ps_id(buffer));
}
else if (qc_mysql_is_ps_command(cmd))
{
erase(mysql_extract_ps_id(buffer));
}
else
{
ss_info_dassert(!true, "QueryClassifier::PSManager::erase called with invalid query");
}
}
private:
typedef std::tr1::unordered_map<uint32_t, uint32_t> BinaryPSMap;
typedef std::tr1::unordered_map<std::string, uint32_t> TextPSMap;
@ -328,14 +362,9 @@ uint32_t QueryClassifier::ps_get_type(std::string id) const
return m_sPs_manager->get_type(id);
}
void QueryClassifier::ps_erase(std::string id)
void QueryClassifier::ps_erase(GWBUF* buffer)
{
return m_sPs_manager->erase(id);
}
void QueryClassifier::ps_erase(uint32_t id)
{
return m_sPs_manager->erase(id);
return m_sPs_manager->erase(buffer);
}
uint32_t QueryClassifier::get_route_target(uint8_t command, uint32_t qtype, HINT* pHints)
@ -525,27 +554,6 @@ uint32_t QueryClassifier::get_route_target(uint8_t command, uint32_t qtype, HINT
return target;
}
namespace
{
// Copy of mxs_mysql_extract_ps_id() in modules/protocol/MySQL/mysql_common.cc,
// but we do not want to create a dependency from maxscale-common to that.
uint32_t mysql_extract_ps_id(GWBUF* buffer)
{
uint32_t rval = 0;
uint8_t id[MYSQL_PS_ID_SIZE];
if (gwbuf_copy_data(buffer, MYSQL_PS_ID_OFFSET, sizeof(id), id) == sizeof(id))
{
rval = gw_mysql_get_byte4(id);
}
return rval;
}
}
uint32_t QueryClassifier::ps_id_internal_get(GWBUF* pBuffer)
{
uint32_t internal_id = 0;

3
server/modules/protocol/MySQL/mariadbclient/mysql_client.cc
View File

@ -1631,7 +1631,7 @@ static bool reauthenticate_client(MXS_SESSION* session, GWBUF* packetbuf)
*/
static int route_by_statement(MXS_SESSION* session, uint64_t capabilities, GWBUF** p_readbuf)
{
int rc;
int rc = 1;
GWBUF* packetbuf;
do
{
@ -1738,6 +1738,7 @@ static int route_by_statement(MXS_SESSION* session, uint64_t capabilities, GWBUF
// Store the original COM_CHANGE_USER for later
proto->stored_query = packetbuf;
packetbuf = NULL;
rc = 1;
}
else if (proto->changing_user)
{

4
server/modules/routing/readwritesplit/rwsplit_route_stmt.cc
View File

@ -360,6 +360,10 @@ bool RWSplitSession::route_session_write(GWBUF *querybuf, uint8_t command, uint3
{
m_qc.ps_store(querybuf, id);
}
else if (qc_query_is_type(type, QUERY_TYPE_DEALLOC_PREPARE))
{
m_qc.ps_erase(querybuf);
}
MXS_INFO("Session write, routing to all servers.");