hcq/MaxScale
hcq/MaxScale
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch '2.1.7' into develop-2.1-merge

Browse Source
This commit is contained in:
Johan Wikman 2017-09-12 11:08:02 +03:00
commit 02b9e0a01d
28 changed files with 367 additions and 90 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
Documentation/Changelog.md
View File

@ -21,6 +21,7 @@
* MaxScale now supports IPv6
For more details, please refer to:
* [MariaDB MaxScale 2.1.7 Release Notes](Release-Notes/MaxScale-2.1.7-Release-Notes.md)
* [MariaDB MaxScale 2.1.6 Release Notes](Release-Notes/MaxScale-2.1.6-Release-Notes.md)
* [MariaDB MaxScale 2.1.5 Release Notes](Release-Notes/MaxScale-2.1.5-Release-Notes.md)
* [MariaDB MaxScale 2.1.4 Release Notes](Release-Notes/MaxScale-2.1.4-Release-Notes.md)

74
Documentation/Release-Notes/MaxScale-2.1.7-Release-Notes.md Normal file
View File

@ -0,0 +1,74 @@
# MariaDB MaxScale 2.1.7 Release Notes -- 2017-09-11
Release 2.1.7 is a GA release.
This document describes the changes in release 2.1.7, when compared to
release [2.1.6](MaxScale-2.1.6-Release-Notes.md).
If you are upgrading from release 2.0, please also read the following
release notes:
[2.1.6](./MaxScale-2.1.6-Release-Notes.md)
[2.1.5](./MaxScale-2.1.5-Release-Notes.md)
[2.1.4](./MaxScale-2.1.4-Release-Notes.md)
[2.1.3](./MaxScale-2.1.3-Release-Notes.md)
[2.1.2](./MaxScale-2.1.2-Release-Notes.md)
[2.1.1](./MaxScale-2.1.1-Release-Notes.md)
[2.1.0](./MaxScale-2.1.0-Release-Notes.md)
For any problems you encounter, please consider submitting a bug
report at [Jira](https://jira.mariadb.org).
## Changed Features
### Persistent connection statistics
The output of `show servers` now shows the number of times a connection was
taken from a server's pool as well as the ratio of connections taken from the
pool versus newly created connections.
### Logging
When known, the session id will be included in all logged messages. This allows
a range of logged messages related to a particular session (that is, client) to
be bound together, and makes it easier to investigate problems. In practice this
is visible so that if a logged message earlier looked like
```
2017-08-30 12:20:49 warning: [masking] The rule ...
```
it will now look like
```
2017-08-30 12:20:49 warning: (4711) [masking] The rule ...
```
where `4711` is the session id.
## Bug fixes
[Here is a list of bugs fixed in MaxScale 2.1.7.](https://jira.mariadb.org/issues/?jql=project%20%3D%20MXS%20AND%20issuetype%20%3D%20Bug%20AND%20status%20%3D%20Closed%20AND%20fixVersion%20%3D%202.1.7)
* (MXS-1396)[https://jira.mariadb.org/browse/MXS-1396] Persistent connections hang with Percona Server 5.6.37-82.2-log
* (MXS-1395)[https://jira.mariadb.org/browse/MXS-1395] SELECT NAMES FROM TABLE is not parsed completely
* (MXS-1385)[https://jira.mariadb.org/browse/MXS-1385] Monitor script arguments can be truncated
* (MXS-1384)[https://jira.mariadb.org/browse/MXS-1384] maxscale.cnf script field length limitation
* (MXS-1380)[https://jira.mariadb.org/browse/MXS-1380] UNION is partially parsed
* (MXS-1379)[https://jira.mariadb.org/browse/MXS-1379] Undefined outcome on schemarouter query conflict
* (MXS-1375)[https://jira.mariadb.org/browse/MXS-1375] Reused connections get multiple replies to COM_CHANGE_USER
* (MXS-1374)[https://jira.mariadb.org/browse/MXS-1374] Persistent connections can't be altered at runtime
* (MXS-1366)[https://jira.mariadb.org/browse/MXS-1366] Abrupt disconnections with persistent connections
* (MXS-1365)[https://jira.mariadb.org/browse/MXS-1365] Write to invalid memory in avrorouter
* (MXS-1363)[https://jira.mariadb.org/browse/MXS-1363] Servers with zero weight aren't used by readconnroute
* (MXS-1341)[https://jira.mariadb.org/browse/MXS-1341] binlog checksums break avrorouter processing
## Packaging
RPM and Debian packages are provided for the Linux distributions supported
by MariaDB Enterprise.
Packages can be downloaded [here](https://mariadb.com/resources/downloads).
## Source Code
The source code of MaxScale is tagged at GitHub with a tag, which is identical
with the version of MaxScale. For instance, the tag of version X.Y.Z of MaxScale
is maxscale-X.Y.Z.
The source code is available [here](https://github.com/mariadb-corporation/MaxScale).

1
Documentation/Upgrading/Upgrading-To-MaxScale-2.1.md
View File

@ -7,6 +7,7 @@ For more information about MariaDB MaxScale 2.1, please refer to the
[ChangeLog](../Changelog.md).
For a complete list of changes in MaxScale 2.1, refer to the
[MaxScale 2.1.7 Release Notes](../Release-Notes/MaxScale-2.1.7-Release-Notes.md).
[MaxScale 2.1.6 Release Notes](../Release-Notes/MaxScale-2.1.6-Release-Notes.md).
[MaxScale 2.1.5 Release Notes](../Release-Notes/MaxScale-2.1.5-Release-Notes.md).
[MaxScale 2.1.4 Release Notes](../Release-Notes/MaxScale-2.1.4-Release-Notes.md).

2
Documentation/list_fixed_bugs.sh
View File

@ -8,4 +8,4 @@ then
fi
version=$1
curl -s "https://jira.mariadb.org/sr/jira.issueviews:searchrequest-csv-current-fields/temp/SearchRequest.csv?jqlQuery=project+%3D+MXS+AND+status+%3D+Closed+AND+fixVersion+%3D+$version"|cut -f 1,2 -d ,|tail -n+2|sed 's/\(.*\),\(.*\)/* (\2)[https:\/\/jira.mariadb.org\/browse\/\2] \1/'
curl -s "https://jira.mariadb.org/sr/jira.issueviews:searchrequest-csv-current-fields/temp/SearchRequest.csv?jqlQuery=project+%3D+MXS+AND+issuetype+%3D+Bug+AND+status+%3D+Closed+AND+fixVersion+%3D+$version"|cut -f 1,2 -d ,|tail -n+2|sed 's/\(.*\),\(.*\)/* (\2)[https:\/\/jira.mariadb.org\/browse\/\2] \1/'

11
Documentation/list_fixed_issues.sh Executable file
View File

@ -0,0 +1,11 @@
#!/bin/bash
if [ $# -ne 1 ]
then
echo "USAGE: $0 VERSION"
exit 1
fi
version=$1
curl -s "https://jira.mariadb.org/sr/jira.issueviews:searchrequest-csv-current-fields/temp/SearchRequest.csv?jqlQuery=project+%3D+MXS+AND+issuetype+%21%3D+Bug+AND+status+%3D+Closed+AND+fixVersion+%3D+$version"|cut -f 1,2 -d ,|tail -n+2|sed 's/\(.*\),\(.*\)/* (\2)[https:\/\/jira.mariadb.org\/browse\/\2] \1/'

3
include/maxscale/protocol/mysql.h
View File

@ -354,7 +354,8 @@ static inline bool MYSQL_IS_ERROR_PACKET(const uint8_t* header)
static inline bool MYSQL_IS_COM_QUIT(const uint8_t* header)
{
return MYSQL_GET_COMMAND(header) == MYSQL_COM_QUIT;
return MYSQL_GET_COMMAND(header) == MYSQL_COM_QUIT &&
MYSQL_GET_PAYLOAD_LEN(header) == 1;
}
static inline bool MYSQL_IS_COM_INIT_DB(const uint8_t* header)

10
include/maxscale/server.h
View File

@ -57,10 +57,12 @@ typedef struct server_params
*/
typedef struct
{
int n_connections; /**< Number of connections */
int n_current; /**< Current connections */
int n_current_ops; /**< Current active operations */
int n_persistent; /**< Current persistent pool */
int n_connections; /**< Number of connections */
int n_current; /**< Current connections */
int n_current_ops; /**< Current active operations */
int n_persistent; /**< Current persistent pool */
uint64_t n_new_conn; /**< Times the current pool was empty */
uint64_t n_from_pool; /**< Times when a connection was available from the pool */
} SERVER_STATS;
/**

15
include/maxscale/session.h
View File

@ -152,6 +152,7 @@ typedef struct session
GWBUF *buffer; /**< Buffer containing the statement */
const struct server *target; /**< Where the statement was sent */
} stmt; /**< Current statement being executed */
bool qualifies_for_pooling; /**< Whether this session qualifies for the connection pool */
skygw_chk_t ses_chk_tail;
} MXS_SESSION;
@ -441,6 +442,20 @@ json_t* session_to_json(const MXS_SESSION *session, const char* host);
*/
json_t* session_list_to_json(const char* host);
/**
* Qualify the session for connection pooling
*
* @param session Session to qualify
*/
void session_qualify_for_pool(MXS_SESSION* session);
/**
* Check if the session qualifies for connection pooling
*
* @param session
*/
bool session_valid_for_pool(const MXS_SESSION* session);
/**
* @brief Return the session of the dcb currently being processed
* by the calling thread.

1
query_classifier/qc_sqlite/sqlite-src-3110100/src/parse.y
View File

@ -612,6 +612,7 @@ columnid(A) ::= nm(X). {
/*KEY*/
/*LIKE_KW*/
MASTER /*MATCH*/ MERGE
NAMES
NO
OF OFFSET OPEN
QUICK

5
query_classifier/test/maxscale.test
View File

@ -79,3 +79,8 @@ ROLLBACK TO id;
RELEASE SAVEPOINT id;
SELECT her FROM (SELECT @@server_id as her) as t WHERE her REGEXP '.*';
select * from db1.t1 union select * from db2.t2;
# Names is a keyword as well
select names from t;

4
server/core/config.cc
View File

@ -1233,8 +1233,8 @@ MXS_CONFIG_PARAMETER* config_clone_param(const MXS_CONFIG_PARAMETER* param)
if (p2)
{
p2->name = MXS_STRNDUP_A(param->name, MAX_PARAM_LEN);
p2->value = MXS_STRNDUP_A(param->value, MAX_PARAM_LEN);
p2->name = MXS_STRDUP_A(param->name);
p2->value = MXS_STRDUP_A(param->value);
p2->next = NULL;
}

22
server/core/config_runtime.cc
View File

@ -330,6 +330,12 @@ static long get_positive_int(const char *value)
return 0;
}
static inline bool is_valid_integer(const char* value)
{
char* endptr;
return strtol(value, &endptr, 10) >= 0 && *value && *endptr == '\0';
}
bool runtime_alter_server(SERVER *server, const char *key, const char *value)
{
spinlock_acquire(&crt_lock);
@ -360,6 +366,22 @@ bool runtime_alter_server(SERVER *server, const char *key, const char *value)
valid = true;
server_update_credentials(server, server->monuser, value);
}
else if (strcmp(key, CN_PERSISTPOOLMAX) == 0)
{
if (is_valid_integer(value))
{
valid = true;
server->persistpoolmax = atoi(value);
}
}
else if (strcmp(key, CN_PERSISTMAXTIME) == 0)
{
if (is_valid_integer(value))
{
valid = true;
server->persistmaxtime = atoi(value);
}
}
else
{
if (!server_remove_parameter(server, key) && !value[0])

13
server/core/dcb.cc
View File

@ -381,6 +381,7 @@ dcb_connect(SERVER *server, MXS_SESSION *session, const char *protocol)
dcb->persistentstart = 0;
dcb->was_persistent = true;
dcb->last_read = hkheartbeat;
atomic_add_uint64(&server->stats.n_from_pool, 1);
return dcb;
}
else
@ -1230,6 +1231,8 @@ dcb_maybe_add_persistent(DCB *dcb)
&& (dcb->func.established == NULL || dcb->func.established(dcb))
&& strlen(dcb->user)
&& dcb->server
&& dcb->session
&& session_valid_for_pool(dcb->session)
&& dcb->server->persistpoolmax
&& (dcb->server->status & SERVER_RUNNING)
&& !dcb->dcb_errhandle_called
@ -1261,6 +1264,16 @@ dcb_maybe_add_persistent(DCB *dcb)
MXS_FREE(loopcallback);
}
/** Free all buffered data */
gwbuf_free(dcb->fakeq);
gwbuf_free(dcb->readq);
gwbuf_free(dcb->delayq);
gwbuf_free(dcb->writeq);
dcb->fakeq = NULL;
dcb->readq = NULL;
dcb->delayq = NULL;
dcb->writeq = NULL;
dcb->nextpersistent = dcb->server->persistent[dcb->poll.thread.id];
dcb->server->persistent[dcb->poll.thread.id] = dcb;
atomic_add(&dcb->server->stats.n_persistent, 1);

4
server/core/externcmd.cc
View File

@ -216,7 +216,9 @@ bool externcmd_substitute_arg(EXTERNCMD* cmd, const char* match, const char* rep
{
for (int i = 0; cmd->argv[i] && rval; i++)
{
size_t size = strlen(cmd->argv[i]);
size_t size_orig = strlen(cmd->argv[i]);
size_t size_replace = strlen(replace);
size_t size = MXS_MAX(size_orig, size_replace);
char* dest = (char*)MXS_MALLOC(size);
if (dest)
{

2
server/core/log_manager.cc
View File

@ -2966,7 +2966,7 @@ int mxs_log_message(int priority,
message_len -= (buffer_len - MAX_LOGSTRLEN);
buffer_len = MAX_LOGSTRLEN;
ss_dassert(prefix.len + modname_len +
ss_dassert(prefix.len + session_len + modname_len +
augmentation_len + message_len + suppression_len + 1 == buffer_len);
}

21
server/core/modutil.cc
View File

@ -1001,28 +1001,33 @@ GWBUF* modutil_create_query(const char* query)
*/
int modutil_count_statements(GWBUF* buffer)
{
char* ptr = ((char*)(buffer)->start + 5);
char* start = ((char*)(buffer)->start + 5);
char* ptr = start;
char* end = ((char*)(buffer)->end);
int num = 1;
while (ptr < end && (ptr = strnchr_esc(ptr, ';', end - ptr)))
{
num++;
while (*ptr == ';')
while (ptr < end && *ptr == ';')
{
ptr++;
}
}
ptr = end - 1;
while (isspace(*ptr))
{
ptr--;
}
if (*ptr == ';')
if (ptr >= start && ptr < end)
{
num--;
while (ptr > start && isspace(*ptr))
{
ptr--;
}
if (*ptr == ';')
{
num--;
}
}
return num;

9
server/core/mysql_utils.cc
View File

@ -166,6 +166,10 @@ MYSQL *mxs_mysql_real_connect(MYSQL *con, SERVER *server, const char *user, cons
mysql_ssl_set(con, listener->ssl_key, listener->ssl_cert, listener->ssl_ca_cert, NULL, NULL);
}
char yes = 1;
mysql_optionsv(con, MYSQL_OPT_RECONNECT, &yes);
mysql_optionsv(con, MYSQL_INIT_COMMAND, "SET SQL_MODE=''");
MYSQL* mysql = mysql_real_connect(con, server->name, user, passwd, NULL, server->port, NULL, 0);
if (mysql)
@ -174,11 +178,6 @@ MYSQL *mxs_mysql_real_connect(MYSQL *con, SERVER *server, const char *user, cons
MY_CHARSET_INFO cs_info;
mysql_get_character_set_info(mysql, &cs_info);
server->charset = cs_info.number;
if (mysql_query(mysql, "SET SQL_MODE=''"))
{
MXS_ERROR("Failed to change SQL_MODE: %s", mysql_error(mysql));
}
}
return mysql;

2
server/core/secrets.cc
View File

@ -413,7 +413,7 @@ encrypt_password(const char* path, const char *password)
AES_set_encrypt_key(keys->enckey, 8 * MAXSCALE_KEYLEN, &aeskey);
AES_cbc_encrypt(padded_passwd, encrypted, padded_len, &aeskey, keys->initvector, AES_ENCRYPT);
hex_output = (char *) MXS_MALLOC(padded_len * 2);
hex_output = (char *) MXS_MALLOC(padded_len * 2 + 1);
if (hex_output)
{
gw_bin2hex(hex_output, encrypted, padded_len);

3
server/core/server.cc
View File

@ -553,6 +553,9 @@ dprintServer(DCB *dcb, const SERVER *server)
dcb_printf(dcb, "\tPersistent actual size max: %d\n", server->persistmax);
dcb_printf(dcb, "\tPersistent pool size limit: %ld\n", server->persistpoolmax);
dcb_printf(dcb, "\tPersistent max time (secs): %ld\n", server->persistmaxtime);
dcb_printf(dcb, "\tConnections taken from pool: %lu\n", server->stats.n_from_pool);
double d = (double)server->stats.n_from_pool / (double)(server->stats.n_connections + server->stats.n_from_pool + 1);
dcb_printf(dcb, "\tPool availability: %0.2lf%%\n", d * 100.0);
}
if (server->server_ssl)
{

16
server/core/session.cc
View File

@ -171,12 +171,14 @@ static MXS_SESSION* session_alloc_body(SERVICE* service, DCB* client_dcb,
session->stats.connect = time(0);
session->stmt.buffer = NULL;
session->stmt.target = NULL;
session->qualifies_for_pooling = false;
MXS_CONFIG *config = config_get_global_options();
// If MaxScale is running in Oracle mode, then autocommit needs to
// initially be off.
bool autocommit = (config->qc_sql_mode == QC_SQL_MODE_ORACLE) ? false : true;
session_set_autocommit(session, autocommit);
/*<
* Associate the session to the client DCB and set the reference count on
* the session to indicate that there is a single reference to the
@ -497,7 +499,8 @@ printAllSessions()
/** Callback for dprintAllSessions */
bool dprintAllSessions_cb(DCB *dcb, void *data)
{
if (dcb->dcb_role == DCB_ROLE_CLIENT_HANDLER)
if (dcb->dcb_role == DCB_ROLE_CLIENT_HANDLER &&
dcb->session->state != SESSION_STATE_DUMMY)
{
DCB *out_dcb = (DCB*)data;
dprintSession(out_dcb, dcb->session);
@ -1153,6 +1156,17 @@ json_t* session_list_to_json(const char* host)
return mxs_json_resource(host, MXS_JSON_API_SESSIONS, data.json);
}
void session_qualify_for_pool(MXS_SESSION* session)
{
session->qualifies_for_pooling = true;
}
bool session_valid_for_pool(const MXS_SESSION* session)
{
ss_dassert(session->state != SESSION_STATE_DUMMY);
return session->qualifies_for_pooling;
}
MXS_SESSION* session_get_current()
{
DCB* dcb = dcb_get_current();

155
server/modules/protocol/MySQL/MySQLBackend/mysql_backend.c
View File

@ -661,6 +661,27 @@ static inline bool collecting_resultset(MySQLProtocol *proto, uint64_t capabilit
proto->collect_result;
}
/**
* Helpers for checking OK and ERR packets specific to COM_CHANGE_USER
*/
static inline bool not_ok_packet(const GWBUF* buffer)
{
uint8_t* data = GWBUF_DATA(buffer);
return data[4] != MYSQL_REPLY_OK ||
// Should be more than 7 bytes of payload
gw_mysql_get_byte3(data) < MYSQL_OK_PACKET_MIN_LEN - MYSQL_HEADER_LEN ||
// Should have no affected rows
data[5] != 0 ||
// Should not generate an insert ID
data[6] != 0;
}
static inline bool not_err_packet(const GWBUF* buffer)
{
return GWBUF_DATA(buffer)[4] != MYSQL_REPLY_ERR;
}
/**
* @brief With authentication completed, read new data and write to backend
*
@ -701,8 +722,9 @@ gw_read_and_write(DCB *dcb)
/** Ask what type of output the router/filter chain expects */
uint64_t capabilities = service_get_capabilities(session->service);
bool result_collected = false;
MySQLProtocol *proto = (MySQLProtocol *)dcb->protocol;
if (rcap_type_required(capabilities, RCAP_TYPE_STMT_OUTPUT))
if (rcap_type_required(capabilities, RCAP_TYPE_STMT_OUTPUT) || (proto->ignore_replies != 0))
{
GWBUF *tmp = modutil_get_complete_packets(&read_buffer);
/* Put any residue into the read queue */
@ -717,10 +739,9 @@ gw_read_and_write(DCB *dcb)
read_buffer = tmp;
MySQLProtocol *proto = (MySQLProtocol*)dcb->protocol;
if (rcap_type_required(capabilities, RCAP_TYPE_CONTIGUOUS_OUTPUT) ||
proto->collect_result)
proto->collect_result ||
proto->ignore_replies != 0)
{
if ((tmp = gwbuf_make_contiguous(read_buffer)))
{
@ -767,39 +788,67 @@ gw_read_and_write(DCB *dcb)
}
}
MySQLProtocol *proto = (MySQLProtocol *)dcb->protocol;
if (proto->ignore_replies > 0)
{
/**
* The reply to an ignorable command is in the packet. Extract the
* response type and discard the response.
*/
uint8_t result = 0xff;
gwbuf_copy_data(read_buffer, MYSQL_HEADER_LEN, 1, &result);
proto->ignore_replies--;
ss_dassert(proto->ignore_replies >= 0);
gwbuf_free(read_buffer);
int rval = 0;
/** The reply to a COM_CHANGE_USER is in packet */
GWBUF *query = proto->stored_query;
proto->stored_query = NULL;
proto->ignore_replies--;
ss_dassert(proto->ignore_replies >= 0);
GWBUF* reply = modutil_get_next_MySQL_packet(&read_buffer);
while (read_buffer)
{
/** Skip to the last packet if we get more than one */
gwbuf_free(reply);
reply = modutil_get_next_MySQL_packet(&read_buffer);
}
ss_dassert(reply);
ss_dassert(!read_buffer);
uint8_t result = MYSQL_GET_COMMAND(GWBUF_DATA(reply));
int rval = 0;
if (result == MYSQL_REPLY_OK)
{
MXS_INFO("Response to COM_CHANGE_USER is OK, writing stored query");
rval = query ? dcb->func.write(dcb, query) : 1;
}
else if (query)
else
{
/**
* The ignorable command failed when we had a queued query from the
* client. Generate a fake hangup event to close the DCB and send
* an error to the client.
*/
if (result == MYSQL_REPLY_ERR)
{
/** The COM_CHANGE USER failed, generate a fake hangup event to
* close the DCB and send an error to the client. */
log_error_response(dcb, reply);
}
else if (result == MYSQL_REPLY_AUTHSWITCHREQUEST &&
gwbuf_length(reply) > MYSQL_EOF_PACKET_LEN)
{
MXS_ERROR("Received AuthSwitchRequest to '%s' when '%s' was expected",
(char*)GWBUF_DATA(reply) + 5, DEFAULT_MYSQL_AUTH_PLUGIN);
}
else
{
/** This should never happen */
MXS_ERROR("Unknown response to COM_CHANGE_USER (0x%02hhx), "
"ignoring and waiting for correct result", result);
gwbuf_free(reply);
proto->stored_query = query;
proto->ignore_replies++;
return 1;
}
gwbuf_free(query);
poll_fake_hangup_event(dcb);
}
gwbuf_free(reply);
return rval;
}
@ -947,10 +996,28 @@ static int gw_MySQLWrite_backend(DCB *dcb, GWBUF *queue)
CHK_DCB(dcb);
if (dcb->was_persistent && dcb->state == DCB_STATE_POLLING &&
backend_protocol->protocol_auth_state == MXS_AUTH_STATE_COMPLETE)
if (dcb->was_persistent)
{
ss_dassert(!dcb->fakeq);
ss_dassert(!dcb->readq);
ss_dassert(!dcb->delayq);
ss_dassert(!dcb->writeq);
ss_dassert(dcb->persistentstart == 0);
dcb->was_persistent = false;
ss_dassert(backend_protocol->ignore_replies >= 0);
backend_protocol->ignore_replies = 0;
if (dcb->state != DCB_STATE_POLLING ||
backend_protocol->protocol_auth_state != MXS_AUTH_STATE_COMPLETE)
{
MXS_INFO("DCB and protocol state do not qualify for pooling: %s, %s",
STRDCBSTATE(dcb->state),
STRPROTOCOLSTATE(backend_protocol->protocol_auth_state));
gwbuf_free(queue);
return 0;
}
/**
* This is a DCB that was just taken out of the persistent connection pool.
* We need to sent a COM_CHANGE_USER query to the backend to reset the
@ -962,18 +1029,45 @@ static int gw_MySQLWrite_backend(DCB *dcb, GWBUF *queue)
* response is received. */
gwbuf_free(backend_protocol->stored_query);
}
dcb->was_persistent = false;
backend_protocol->ignore_replies++;
ss_dassert(backend_protocol->ignore_replies > 0);
backend_protocol->stored_query = queue;
if (MYSQL_IS_COM_QUIT(GWBUF_DATA(queue)))
{
/** The connection is being closed before the first write to this
* backend was done. The COM_QUIT is ignored and the DCB will be put
* back into the pool once it's closed. */
MXS_INFO("COM_QUIT received as the first write, ignoring and "
"sending the DCB back to the pool.");
gwbuf_free(queue);
return 1;
}
GWBUF *buf = gw_create_change_user_packet(dcb->session->client_dcb->data, dcb->protocol);
return dcb_write(dcb, buf) ? 1 : 0;
int rc = 0;
if (dcb_write(dcb, buf))
{
MXS_INFO("Sent COM_CHANGE_USER");
backend_protocol->ignore_replies++;
backend_protocol->stored_query = queue;
rc = 1;
}
else
{
gwbuf_free(queue);
}
return rc;
}
else if (backend_protocol->ignore_replies > 0)
{
if (MYSQL_IS_COM_QUIT((uint8_t*)GWBUF_DATA(queue)))
{
/** The COM_CHANGE_USER was already sent but the session is already
* closing. We ignore the COM_QUIT in the hopes that the response
* to the COM_CHANGE_USER comes before the DCB is closed. If the
* DCB is closed before the response arrives, the connection will
* not qualify the persistent connection pool. */
MXS_INFO("COM_QUIT received while COM_CHANGE_USER is in progress, ignoring");
gwbuf_free(queue);
}
else
@ -981,8 +1075,10 @@ static int gw_MySQLWrite_backend(DCB *dcb, GWBUF *queue)
/**
* We're still waiting on the reply to the COM_CHANGE_USER, append the
* buffer to the stored query. This is possible if the client sends
* BLOB data on the first command.
* BLOB data on the first command or is sending multiple COM_QUERY
* packets at one time.
*/
MXS_INFO("COM_CHANGE_USER in progress, appending query to queue");
backend_protocol->stored_query = gwbuf_append(backend_protocol->stored_query, queue);
}
return 1;
@ -1210,6 +1306,8 @@ static void backend_set_delayqueue(DCB *dcb, GWBUF *queue)
static int backend_write_delayqueue(DCB *dcb, GWBUF *buffer)
{
ss_dassert(buffer);
ss_dassert(dcb->persistentstart == 0);
ss_dassert(!dcb->was_persistent);
if (MYSQL_IS_CHANGE_USER(((uint8_t *)GWBUF_DATA(buffer))))
{
@ -1958,5 +2056,8 @@ static bool get_ip_string_and_port(struct sockaddr_storage *sa,
static bool gw_connection_established(DCB* dcb)
{
MySQLProtocol *proto = (MySQLProtocol*)dcb->protocol;
return proto->protocol_auth_state == MXS_AUTH_STATE_COMPLETE;
return
proto->protocol_auth_state == MXS_AUTH_STATE_COMPLETE &&
(proto->ignore_replies == 0)
&& !proto->stored_query;
}

13
server/modules/protocol/MySQL/MySQLClient/mysql_client.cc
View File

@ -1038,6 +1038,19 @@ gw_read_finish_processing(DCB *dcb, GWBUF *read_buffer, uint64_t capabilities)
/** Reset error handler when routing of the new query begins */
dcb->dcb_errhandle_called = false;
if (proto->current_command == MYSQL_COM_QUIT)
{
/** The client is closing the connection. We know that this will be the
* last command the client sends so the backend connections are very likely
* to be in an idle state.
*
* If the client is pipelining the queries (i.e. sending N request as
* a batch and then expecting N responses) then it is possible that
* the backend connections are not idle when the COM_QUIT is received.
* In most cases we can assume that the connections are idle. */
session_qualify_for_pool(session);
}
if (rcap_type_required(capabilities, RCAP_TYPE_STMT_INPUT))
{
/**

1
server/modules/protocol/MySQL/mysql_common.c
View File

@ -79,6 +79,7 @@ MySQLProtocol* mysql_protocol_init(DCB* dcb, int fd)
p->protocol_command.scom_nbytes_to_read = 0;
p->stored_query = NULL;
p->extra_capabilities = 0;
p->ignore_replies = 0;
#if defined(SS_DEBUG)
p->protocol_chk_top = CHK_NUM_PROTOCOL;
p->protocol_chk_tail = CHK_NUM_PROTOCOL;

46
server/modules/routing/avrorouter/avro_file.c
View File

@ -635,38 +635,24 @@ avro_binlog_end_t avro_read_all_events(AVRO_INSTANCE *router)
MXS_DEBUG("%s(%x) - %llu", binlog_event_name(hdr.event_type), hdr.event_type, pos);
uint32_t original_size = hdr.event_size;
if (router->binlog_checksum)
{
hdr.event_size -= 4;
}
/* check for FORMAT DESCRIPTION EVENT */
if (hdr.event_type == FORMAT_DESCRIPTION_EVENT)
{
int event_header_length;
int event_header_ntypes;
const int BLRM_FDE_EVENT_TYPES_OFFSET = 2 + 50 + 4 + 1;
const int FDE_EXTRA_BYTES = 5;
int event_header_length = ptr[BLRM_FDE_EVENT_TYPES_OFFSET - 1];
int n_events = hdr.event_size - event_header_length - BLRM_FDE_EVENT_TYPES_OFFSET - FDE_EXTRA_BYTES;
uint8_t* checksum = ptr + hdr.event_size - event_header_length - FDE_EXTRA_BYTES;
/** Extract the event header lengths */
event_header_length = ptr[2 + 50 + 4];
event_header_ntypes = hdr.event_size - event_header_length - (2 + 50 + 4 + 1);
memcpy(router->event_type_hdr_lens, ptr + 2 + 50 + 5, event_header_ntypes);
router->event_types = event_header_ntypes;
switch (event_header_ntypes)
{
case 168: /* mariadb 10 LOG_EVENT_TYPES*/
event_header_ntypes -= 163;
break;
case 165: /* mariadb 5 LOG_EVENT_TYPES*/
event_header_ntypes -= 160;
break;
default: /* mysql 5.6 LOG_EVENT_TYPES = 35 */
event_header_ntypes -= 35;
break;
}
uint8_t *checksum = ptr + hdr.event_size - event_header_length - event_header_ntypes;
if (checksum[0] == 1)
{
found_chksum = true;
}
router->event_types = n_events;
router->binlog_checksum = checksum[0];
}
/* Decode CLOSE/STOP Event */
else if (hdr.event_type == STOP_EVENT)
@ -767,7 +753,7 @@ avro_binlog_end_t avro_read_all_events(AVRO_INSTANCE *router)
break;
}
if (hdr.next_pos > 0 && hdr.next_pos != (pos + hdr.event_size))
if (hdr.next_pos > 0 && hdr.next_pos != (pos + original_size))
{
MXS_INFO("Binlog %s: next pos %u != (pos %llu + event_size %u), truncating to %llu",
router->binlog_name, hdr.next_pos, pos, hdr.event_size, pos);
@ -1013,7 +999,7 @@ void handle_query_event(AVRO_INSTANCE *router, REP_HEADER *hdr, int *pending_tra
{
int dblen = ptr[DBNM_OFF];
int vblklen = ptr[VBLK_OFF];
int len = hdr->event_size - BINLOG_EVENT_HDR_LEN - (PHDR_OFF + vblklen + 1 + dblen) + 1;
int len = hdr->event_size - BINLOG_EVENT_HDR_LEN - (PHDR_OFF + vblklen + 1 + dblen);
char *sql = (char *) ptr + PHDR_OFF + vblklen + 1 + dblen;
char db[dblen + 1];
memcpy(db, (char*) ptr + PHDR_OFF + vblklen, dblen);

12
server/modules/routing/avrorouter/avro_schema.c
View File

@ -1175,9 +1175,9 @@ bool table_create_alter(TABLE_CREATE *create, const char *sql, const char *end)
{
tok = get_tok(tok + len, &len, end);
create->column_names = MXS_REALLOC(create->column_names, sizeof(char*) * create->columns + 1);
create->column_types = MXS_REALLOC(create->column_types, sizeof(char*) * create->columns + 1);
create->column_lengths = MXS_REALLOC(create->column_lengths, sizeof(int) * create->columns + 1);
create->column_names = MXS_REALLOC(create->column_names, sizeof(char*) * (create->columns + 1));
create->column_types = MXS_REALLOC(create->column_types, sizeof(char*) * (create->columns + 1));
create->column_lengths = MXS_REALLOC(create->column_lengths, sizeof(int) * (create->columns + 1));
char avro_token[len + 1];
make_avro_token(avro_token, tok, len);
@ -1208,9 +1208,9 @@ bool table_create_alter(TABLE_CREATE *create, const char *sql, const char *end)
create->column_lengths[i] = create->column_lengths[i + 1];
}
create->column_names = MXS_REALLOC(create->column_names, sizeof(char*) * create->columns - 1);
create->column_types = MXS_REALLOC(create->column_types, sizeof(char*) * create->columns - 1);
create->column_lengths = MXS_REALLOC(create->column_lengths, sizeof(int) * create->columns - 1);
create->column_names = MXS_REALLOC(create->column_names, sizeof(char*) * (create->columns - 1));
create->column_types = MXS_REALLOC(create->column_types, sizeof(char*) * (create->columns - 1));
create->column_lengths = MXS_REALLOC(create->column_lengths, sizeof(int) * (create->columns - 1));
create->columns--;
updates++;
}

1
server/modules/routing/avrorouter/avrorouter.h
View File

@ -283,6 +283,7 @@ typedef struct avro_instance
pcre2_code *alter_table_re;
uint8_t event_types;
uint8_t event_type_hdr_lens[MAX_EVENT_TYPE_END];
uint8_t binlog_checksum;
gtid_pos_t gtid;
TABLE_MAP *active_maps[MAX_MAPPED_TABLES];
HASHTABLE *table_maps;

4
server/modules/routing/debugcli/debugcmd.c
View File

@ -1559,6 +1559,8 @@ struct subcommand alteroptions[] =
"ssl_ca_cert Path to SSL CA certificate\n"
"ssl_version SSL version\n"
"ssl_cert_verify_depth Certificate verification depth\n"
"persistpoolmax Persisted connection pool size\n"
"persistmaxtime Persisted connection maximum idle time\n"
"\n"
"To configure SSL for a newly created server, the 'ssl', 'ssl_cert',\n"
"'ssl_key' and 'ssl_ca_cert' parameters must be given at the same time.\n"
@ -1898,7 +1900,7 @@ execute_cmd(CLI_SESSION *cli)
{
DCB *dcb = cli->session->client_dcb;
int argc, i, j, found = 0;
char *args[MAXARGS + 1];
char *args[MAXARGS + 4];
int in_quotes = 0, escape_next = 0;
char *ptr, *lptr;
bool in_space = false;

6
server/modules/routing/readconnroute/readconnroute.c
View File

@ -312,7 +312,7 @@ newSession(MXS_ROUTER *instance, MXS_SESSION *session)
*/
for (SERVER_REF *ref = inst->service->dbref; ref; ref = ref->next)
{
if (!SERVER_REF_IS_ACTIVE(ref) || SERVER_IN_MAINT(ref->server) || ref->weight == 0)
if (!SERVER_REF_IS_ACTIVE(ref) || SERVER_IN_MAINT(ref->server))
{
continue;
}
@ -370,6 +370,10 @@ newSession(MXS_ROUTER *instance, MXS_SESSION *session)
{
candidate = ref;
}
else if (ref->weight == 0 || candidate->weight == 0)
{
candidate = ref->weight ? ref : candidate;
}
else if (((ref->connections + 1) * 1000) / ref->weight <
((candidate->connections + 1) * 1000) / candidate->weight)
{