hcq/MaxScale
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

All changes 2.0.0...develop

Browse Source 
973b983 Merge branch 'release-2.0.0' into develop
255dd23 Make spinlock functions take const argument
6e23bab Fix bitmask reallocation
338c189 Rename and clean up slavelag filter
3ea8f28 Fix possible NULL pointer dereference
bfe6738 MXS-830: Add module information to logged messages
1fad962 Fix strncat usage
d38997a Adjust log throttling policy
0be4e4b Add hashtable_item_strcasecmp
726100e Take hashtable convenience functions into use
5e7744a Fix typo in maxadmin.md
c5778c8 Merge branch 'release-2.0.0' into develop
b5762af Move from tmpnam to mkstemp
d6f2c71 Add convenience functions to hashtable
359058a MXS-825: Add support for --execdir
636347c Enable runtime reconfiguration of log throttling
ef9fba9 Improve log throttling documentation
aef917a Implement log throttling
e3a5349 Remove shardrouter.c
8051e80 Remove custom qc_sqlite allocation functions
fd34d60 Initial implementation of the learning firewall
a8752a8 Removed "filestem option" from example
1ef2519 Removed "filestem option" from example
0815cc8 Cleanup spinlock.h
ab4dc99 Clean up hashtable.h
ef2c078 Add prototypes for hashtable copy and free functions
fb5cfaf Add 'log_throttling' configuration entry
300d823 Add proper prototypes for hashtable hash and cmp functions
1c649aa qc_mysqlembedded: Include skygw_...h without path.
d276160 Add missing RPM scripts
e70e644 Fix HTTPAuth installation
1b2b389 Combine utils into the server directory
3ff9913 Add missing utils headers to devel package
407efb2 Fix minor packaging problems
99aa6ad Split MaxScale into core, experimental and devel packages
1290386 Merge branch 'develop' of ssh://github.com/mariadb-corporation/maxscale-new into develop
e59f148 Make scripts POSIX sh compatible
7319266 Fixed SHOW SLAVE STATUS in bonlog router
f8d760a Update Binlogrouter.md
0a904ed Update Replication-Proxy-Binlog-Router-Tutorial.md
75d4202 Update Replication-Proxy-Binlog-Router-Tutorial.md
b8651fc Add missing newline in listmanager.h
c7ad047 Add note about user data caches to release notes
70ccc2b Merge branch 'release-2.0.0' into develop
575d1b6 Mistake - dummy session needs list markers set.
8364508 Merge branch 'develop' into binlog_server_semisync
868b902 Update MaxScale limitations
2c8b327 Store listener caches in separate directories
6e183ec Create unique user data caches for each listeners
f643685 Don't free orphaned tee filter sessions
4179afa Allow binlogrouter to be used without a listener
7ad79af Add function for freeing a listener
677a0a2 Move authentication data from services to listeners
4f12af7 Merge remote-tracking branch 'origin/MXS-677' into develop
1419b81 Semi-Sync support to binlog server: code review updtate
0ea0f01 Semi-Sync support to binlog server: added missing routine
4aad909 Semi-Sync support to binlog server
b824e1e Add authenticator support to httpd.c
705a688 Change tabs to spaces
d0c419e Change method of adding list fields to e.g. DCB
25504fc Document the changed routing priority of hints
41666d1 Remove use_ssl_if_enabled global option
a3584e9 Make routing hints have highest priority
34a1d24 Updated document with new binlog router option
01eedc5 Updated documentation with SSL usage
8a4c0f6 Update Replication-Proxy-Binlog-Router-Tutorial.md
4e374aa Update Replication-Proxy-Binlog-Router-Tutorial.md
f3f3c57 Update Replication-Proxy-Binlog-Router-Tutorial.md
617b79f Binlog Server: error messages typo fix
fa8dfae Binlog Server: error messages review
1b8819c Fix freeing of schemarouter session memory
07f49e1 MXS-788: new code review fix
1fd3b09 MXS-788: show services now displays SSL info
6ca2584 MXS-788 code review fix
ae6a7d0 MXS-788 code review
43d3474 Master server SSL connection
90b2377 Use correct variable in listmanager pre-allocation
9a5b238 Fix listmanager pre-allocation
9c78625 Fix a memory leak when backend authentication fails
e59a966 Fix hang in list_find_free
ff30223 Fix freeing of shared data in schemarouter
fc8f9d3 Add missing include in luafilter
ecf7f53 Add missing NULL value to filter parameter array
636d849 Update memory allocation approach
f0d1d38 Add new allocation functions
97d00a0 Fix writing of uninitialized data to logs
e72c9b2 Merge branch 'release-2.0.0' into develop
cf2b712 Merge branch 'release-2.0.0' into develop
8917c5c Change the logic behind valid list entry checks
c10deff Improve documentation about version_string
f59f1f7 Merge branch 'develop' of ssh://github.com/mariadb-corporation/maxscale-new into develop
c88edb3 Backend authentication failure improvement
abd5bee Revert "Backend authentication failure improvement"
5bb3107 Backend authentication failure improvement
b7f434a Add new allocation functions
3f022fa Fix stupid mistake
99c4317 Merge remote-tracking branch 'origin/MXS-677' into develop
3c1ded6 Added connection/authentication failure error reporting in SHOW SLAVE STATUS
0a60f7b Tidy up and deal with review points.
ba103ff blr_slave.c: Update strncpy usage
467331e blr_master.c: Strncpy usage updates
d2b7c0c Merge remote-tracking branch 'origin/develop-nullauth-merge' into develop
5a8c1d0 qc: Measure execution time at the right place.
bccdb93 Merge branch 'NullAuthDeny' into develop
2e6511c Add 5.5.5 prefix to all version strings that lack it
314655a Improve DCB and session initialization and list handling
e1c43f0 MXS-655: Make MaxScale logging logrotate(8) compatible
ce36afd MXS-626: Don't log a header unless maxlog enabled
dcd47a7 blr_file.c: Replace uses of strncpy
6b8f576 bls_slave.c: Replace strncpy with memcpy
68a0039 Add list preallocation, tidy up, simplify init.
cb37d1b Fix copyright etc headers.
11a400d Tidy; comment; fix bad copies and mistakes.
7e36ec4 Add list manager files.
c4794e3 Initial code for list manager.
1b42e25 Merge remote-tracking branch 'origin/MXS-765' into develop
d50f617 Fix problems, extend tests, respond to review.
dcb4a91 Filter test folder removed
0b60dbe Add a couple of comments.
83cdba0 Fix overwriting problem.
ba5d353 Fix overwriting problem.
53671cb Small fixes in response to review.
173d049 blr.c: Review strncpy usage
4ff6ef2 binlog_common.c: Replace strncpy with memcpy
f238e03 maxbinlogcheck.s: Replace strncpy
9807f8d harness: Replace unnecessary use of strncpy
8c7fe6a avro: Modify strncpy usage
9b8008e Small improvements.
b7f784f Fix mistakes in testqueuemanager.c
cc26962 Restore missing poll.c code; add testqueuemanager.c.
2e91806 Format the filter harness
22059e6 Initial implementation connection queueing.
c604dc2 readwritesplit.c: Improve COM_INIT_DB handling
454d920 schemarouter.c: Replace strncpy with strcpy
8e85d66 sharding_common.c: Too long a database name handled explicitly
77f4446 Astyle schemarouter
491f7c2 maxinfo.c: Replace strncpy with memcpy
6b98105 maxinfo: Reformat with astyle
c1dbf08 Handle oversize user and database names
5fa4a0f Merge branch 'develop' of ssh://github.com/mariadb-corporation/maxscale-new into develop
706963b BLR_DBUSERS_TAIL new var in blr.h
d75b9af Tweak comments, remove trailing blanks.
ab2400a Optimise statistics gathering by inline & simpler fns.
fb59ddc Remove unnecessary strncpy/strncat usage in Binlog Server
bdcd551 resultset.c: Change strncpy to memcpy
c6b1c5e Reject rather than cut too long a path
6d8f112 Remove unnecessary strncpy/strncat usage
18bf5ed Remove unnecessary strncpy usage
dc0e2db Make maxpasswd more userfriendly
c9c8695 Fix calculation of padded_len in encryptPassword
2cfd2c6 dbusers.c: Check strncpy usage
7ab9342 Make more thorough checks in secrets_readKeys
be7d593 Format cli.c debugcli.c testroute.c webserver.c
1ee5efb config.c: Check usage of strncpy
3043b12 gq_utils.c: Unnecessary use of strncpy removed
77874ac Add help to maxkeys
38392a3 Update secrets_writeKeys documentation
2d1325c Make SSL optional in MaxScale's own communication
bda00da Fix avro build failures
b2cb31a Add more OOM macros
41ccf17 Fix strdup usage
a48f732 Fix realloc calls
20771f6 Add forgotten extern "C" block
8faf35a Add maxscale allocation functions
bb47890 Add macros for OOM logging
afea388 Fix silly mistakes.
6dafd22 Make deny default for null auth; move code from common to auth.
This commit is contained in:
Johan Wikman
2016-08-17 08:36:01 +03:00
parent b2035745fc
commit dc7f7b6fb4
360 changed files with 11641 additions and 15856 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
server/modules/routing/binlog/CMakeLists.txt
View File

@ -2,12 +2,12 @@ add_library(binlogrouter SHARED blr.c blr_master.c blr_cache.c blr_slave.c blr_f
set_target_properties(binlogrouter PROPERTIES INSTALL_RPATH ${CMAKE_INSTALL_RPATH}:${MAXSCALE_LIBDIR} VERSION "2.0.0")
set_target_properties(binlogrouter PROPERTIES LINK_FLAGS -Wl,-z,defs)
target_link_libraries(binlogrouter maxscale-common ${PCRE_LINK_FLAGS} uuid)
install(TARGETS binlogrouter DESTINATION ${MAXSCALE_LIBDIR})
install_module(binlogrouter core)
add_executable(maxbinlogcheck maxbinlogcheck.c blr_file.c blr_cache.c blr_master.c blr_slave.c blr.c)
target_link_libraries(maxbinlogcheck maxscale-common ${PCRE_LINK_FLAGS} uuid)
install(TARGETS maxbinlogcheck DESTINATION bin)
install_executable(maxbinlogcheck core)
if(BUILD_TESTS)
add_subdirectory(test)

4
server/modules/routing/binlog/binlog_common.c
View File

@ -4,7 +4,7 @@
* Use of this software is governed by the Business Source License included
* in the LICENSE.TXT file and at www.mariadb.com/bsl.
*
* Change Date: 2019-01-01
* Change Date: 2019-07-01
*
* On the date above, in accordance with the Business Source License, use
* of this software will be governed by version 2 or later of the General
@ -80,7 +80,7 @@ bool binlog_next_file_exists(const char* binlogdir, const char* binlog)
char filename[PATH_MAX + 1];
char next_file[BLRM_BINLOG_NAME_STR_LEN + 1];
int offset = sptr - binlog;
strncpy(buf, binlog, offset);
memcpy(buf, binlog, offset);
buf[offset] = '\0';
sprintf(next_file, BINLOG_NAMEFMT, buf, filenum);
snprintf(filename, PATH_MAX, "%s/%s", binlogdir, next_file);

553
server/modules/routing/binlog/blr.c
View File

@ -4,7 +4,7 @@
* Use of this software is governed by the Business Source License included
* in the LICENSE.TXT file and at www.mariadb.com/bsl.
*
* Change Date: 2019-01-01
* Change Date: 2019-07-01
*
* On the date above, in accordance with the Business Source License, use
* of this software will be governed by version 2 or later of the General
@ -46,6 +46,10 @@
* 23/10/2015 Markus Makela Added current_safe_event
* 27/10/2015 Martin Brampton Amend getCapabilities to return RCAP_TYPE_NO_RSESSION
* 19/04/2016 Massimiliano Pinto UUID generation now comes from libuuid
* 05/07/2016 Massimiliano Pinto errorReply now handles error message in SHOW SLAVE STATUS
* for connection error and authentication failure.
* 11/07/2016 Massimiliano Pinto Added SSL backend support
* 22/07/2016 Massimiliano Pinto Added semi_sync replication support
*
* @endverbatim
*/
@ -75,8 +79,10 @@
#include <ini.h>
#include <sys/stat.h>
#include <uuid/uuid.h>
#include <maxscale/alloc.h>
#include <gw.h>
static char *version_str = "V2.0.0";
static char *version_str = "V2.1.0";
/* The router entry points */
static ROUTER *createInstance(SERVICE *service, char **options);
@ -105,6 +111,7 @@ static int blr_load_dbusers(const ROUTER_INSTANCE *router);
static int blr_check_binlog(ROUTER_INSTANCE *router);
int blr_read_events_all_events(ROUTER_INSTANCE *router, int fix, int debug);
void blr_master_close(ROUTER_INSTANCE *);
void blr_free_ssl_data(ROUTER_INSTANCE *inst);
/** The module object definition */
static ROUTER_OBJECT MyObject =
@ -186,8 +193,6 @@ createInstance(SERVICE *service, char **options)
char *value;
int i;
uuid_t defuuid;
char path[PATH_MAX + 1] = "";
char filename[PATH_MAX + 1] = "";
int rc = 0;
char task_name[BLRM_TASK_NAME_LEN + 1] = "";
@ -220,15 +225,12 @@ createInstance(SERVICE *service, char **options)
service->name);
server_free(service->dbref->server);
free(service->dbref);
MXS_FREE(service->dbref);
service->dbref = NULL;
}
if ((inst = (ROUTER_INSTANCE *)calloc(1, sizeof(ROUTER_INSTANCE))) == NULL)
if ((inst = (ROUTER_INSTANCE *)MXS_CALLOC(1, sizeof(ROUTER_INSTANCE))) == NULL)
{
MXS_ERROR("%s: Error: failed to allocate memory for router instance.",
service->name);
return NULL;
}
@ -260,8 +262,8 @@ createInstance(SERVICE *service, char **options)
inst->heartbeat = BLR_HEARTBEAT_DEFAULT_INTERVAL;
inst->mariadb10_compat = false;
inst->user = strdup(service->credentials.name);
inst->password = strdup(service->credentials.authdata);
inst->user = MXS_STRDUP_A(service->credentials.name);
inst->password = MXS_STRDUP_A(service->credentials.authdata);
inst->m_errno = 0;
inst->m_errmsg = NULL;
@ -279,10 +281,22 @@ createInstance(SERVICE *service, char **options)
inst->serverid = 0;
/* SSL replication is disabled by default */
inst->ssl_enabled = 0;
/* SSL config options */
inst->ssl_ca = NULL;
inst->ssl_cert = NULL;
inst->ssl_key = NULL;
inst->ssl_version = NULL;
/* Semi-Sync support */
inst->request_semi_sync = false;
inst->master_semi_sync = 0;
/* Generate UUID for the router instance */
uuid_generate_time(defuuid);
if ((inst->uuid = (char *)calloc(38, 1)) != NULL)
if ((inst->uuid = (char *)MXS_CALLOC(38, 1)) != NULL)
{
sprintf(inst->uuid,
"%02hhx%02hhx%02hhx%02hhx-"
@ -325,8 +339,8 @@ createInstance(SERVICE *service, char **options)
value++;
if (strcmp(options[i], "uuid") == 0)
{
free(inst->uuid);
inst->uuid = strdup(value);
MXS_FREE(inst->uuid);
inst->uuid = MXS_STRDUP_A(value);
}
else if ((strcmp(options[i], "server_id") == 0) || (strcmp(options[i], "server-id") == 0))
{
@ -351,18 +365,18 @@ createInstance(SERVICE *service, char **options)
}
else if (strcmp(options[i], "user") == 0)
{
free(inst->user);
inst->user = strdup(value);
MXS_FREE(inst->user);
inst->user = MXS_STRDUP_A(value);
}
else if (strcmp(options[i], "password") == 0)
{
free(inst->password);
inst->password = strdup(value);
MXS_FREE(inst->password);
inst->password = MXS_STRDUP_A(value);
}
else if (strcmp(options[i], "passwd") == 0)
{
free(inst->password);
inst->password = strdup(value);
MXS_FREE(inst->password);
inst->password = MXS_STRDUP_A(value);
}
else if ((strcmp(options[i], "master_id") == 0) || (strcmp(options[i], "master-id") == 0))
{
@ -370,7 +384,7 @@ createInstance(SERVICE *service, char **options)
if (master_id > 0)
{
inst->masterid = master_id;
inst->set_master_server_id = strdup(value);
inst->set_master_server_id = MXS_STRDUP_A(value);
}
if (strcmp(options[i], "master-id") == 0)
{
@ -382,16 +396,16 @@ createInstance(SERVICE *service, char **options)
}
else if (strcmp(options[i], "master_uuid") == 0)
{
inst->set_master_uuid = strdup(value);
inst->set_master_uuid = MXS_STRDUP_A(value);
inst->master_uuid = inst->set_master_uuid;
}
else if (strcmp(options[i], "master_version") == 0)
{
inst->set_master_version = strdup(value);
inst->set_master_version = MXS_STRDUP_A(value);
}
else if (strcmp(options[i], "master_hostname") == 0)
{
inst->set_master_hostname = strdup(value);
inst->set_master_hostname = MXS_STRDUP_A(value);
}
else if (strcmp(options[i], "mariadb10-compatibility") == 0)
{
@ -399,7 +413,7 @@ createInstance(SERVICE *service, char **options)
}
else if (strcmp(options[i], "filestem") == 0)
{
inst->fileroot = strdup(value);
inst->fileroot = MXS_STRDUP_A(value);
}
else if (strcmp(options[i], "file") == 0)
{
@ -409,6 +423,10 @@ createInstance(SERVICE *service, char **options)
{
inst->trx_safe = config_truth_value(value);
}
else if (strcmp(options[i], "semisync") == 0)
{
inst->request_semi_sync = config_truth_value(value);
}
else if (strcmp(options[i], "lowwater") == 0)
{
inst->low_water = atoi(value);
@ -472,7 +490,21 @@ createInstance(SERVICE *service, char **options)
}
else if (strcmp(options[i], "binlogdir") == 0)
{
inst->binlogdir = strdup(value);
inst->binlogdir = MXS_STRDUP_A(value);
}
else if (strcmp(options[i], "ssl_cert_verification_depth") == 0)
{
int new_depth = atoi(value);
if (new_depth > 0)
{
inst->ssl_cert_verification_depth = new_depth;
}
else
{
MXS_WARNING("Invalid Master ssl_cert_verification_depth %s."
" Setting it to default value %i.",
value, inst->ssl_cert_verification_depth);
}
}
else
{
@ -491,7 +523,7 @@ createInstance(SERVICE *service, char **options)
if (inst->fileroot == NULL)
{
inst->fileroot = strdup(BINLOG_NAME_ROOT);
inst->fileroot = MXS_STRDUP_A(BINLOG_NAME_ROOT);
}
inst->active_logs = 0;
inst->reconnect_pending = 0;
@ -550,38 +582,56 @@ createInstance(SERVICE *service, char **options)
}
/* Allocate dbusers for this router here instead of serviceStartPort() */
if (service->users == NULL)
for (SERV_LISTENER *port = service->ports; port; port = port->next)
{
service->users = (void *)mysql_users_alloc();
if (service->users == NULL)
if ((port->users = mysql_users_alloc()) == NULL)
{
MXS_ERROR("%s: Error allocating dbusers in createInstance",
inst->service->name);
free_instance(inst);
return NULL;
}
}
/* Dynamically allocate master_host server struct, not written in anyfile */
/* Dynamically allocate master_host server struct, not written in any cnf file */
if (service->dbref == NULL)
{
SERVER *server;
SSL_LISTENER *ssl_cfg;
server = server_alloc("_none_", "MySQLBackend", (int)3306);
if (server == NULL)
{
MXS_ERROR("%s: Error for server_alloc in createInstance",
inst->service->name);
if (inst->service->users)
{
users_free(inst->service->users);
inst->service->users = NULL;
}
free_instance(inst);
return NULL;
}
/* Allocate SSL struct for backend connection */
if ((ssl_cfg = MXS_CALLOC(1, sizeof(SSL_LISTENER))) == NULL)
{
MXS_ERROR("%s: Error allocating memory for SSL struct in createInstance",
inst->service->name);
server_free(service->dbref->server);
MXS_FREE(service->dbref);
free_instance(inst);
return NULL;
}
/* Set some SSL defaults */
ssl_cfg->ssl_init_done = false;
ssl_cfg->ssl_method_type = SERVICE_SSL_TLS_MAX;
ssl_cfg->ssl_cert_verify_depth = 9;
/** Set SSL pointer in in server struct */
server->server_ssl = ssl_cfg;
/* Set server unique name */
server_set_unique_name(server, "binlog_router_master_host");
/* Add server to service backend list */
serviceAddBackend(inst->service, server);
}
@ -594,8 +644,9 @@ createInstance(SERVICE *service, char **options)
* automatic master replication start phase
*/
strncpy(path, inst->binlogdir, PATH_MAX);
snprintf(filename, PATH_MAX, "%s/master.ini", path);
static const char MASTER_INI[] = "/master.ini";
char filename[strlen(inst->binlogdir) + sizeof(MASTER_INI)]; // sizeof includes the NULL
sprintf(filename, "%s%s", inst->binlogdir, MASTER_INI);
rc = ini_parse(filename, blr_handler_config, inst);
@ -635,12 +686,60 @@ createInstance(SERVICE *service, char **options)
inst->master_state = BLRM_UNCONNECTED;
/* Try loading dbusers */
blr_load_dbusers(inst);
if (inst->service->ports)
{
blr_load_dbusers(inst);
}
}
/**
*******************************
* Initialise the binlog router
*******************************
*/
/**
* Check first for SSL enabled replication.
* If not remove the SSL struct from server
*/
if (inst->ssl_enabled)
{
if (service->dbref && service->dbref->server && service->dbref->server->server_ssl)
{
/* Initialise SSL: exit on error */
if (listener_init_SSL(service->dbref->server->server_ssl) != 0)
{
MXS_ERROR("%s: Unable to initialize SSL with backend server", service->name);
/* Free SSL struct */
/* Note: SSL struct in server should be freed by server_free() */
blr_free_ssl_data(inst);
server_free(service->dbref->server);
MXS_FREE(service->dbref);
service->dbref = NULL;
free_instance(inst);
return NULL;
}
}
MXS_INFO("%s: Replicating from master with SSL", service->name);
}
else
{
MXS_DEBUG("%s: Replicating from master without SSL", service->name);
/* Free the SSL struct because is not needed if MASTER_SSL = 0
* Provided options, if any, are kept in inst->ssl_* vars
* SHOW SLAVE STATUS can display those values
*/
/* Note: SSL struct in server should be freed by server_free() */
if (service->dbref && service->dbref->server)
{
blr_free_ssl_data(inst);
}
}
if (inst->master_state == BLRM_UNCONNECTED)
{
@ -654,16 +753,14 @@ createInstance(SERVICE *service, char **options)
service->name,
inst->binlogdir);
if (service->users)
{
users_free(service->users);
service->users = NULL;
}
if (service->dbref && service->dbref->server)
{
/* Free SSL data */
blr_free_ssl_data(inst);
server_free(service->dbref->server);
free(service->dbref);
MXS_FREE(service->dbref);
service->dbref = NULL;
}
free_instance(inst);
@ -735,16 +832,27 @@ createInstance(SERVICE *service, char **options)
static void
free_instance(ROUTER_INSTANCE *instance)
{
free(instance->uuid);
free(instance->user);
free(instance->password);
free(instance->set_master_server_id);
free(instance->set_master_uuid);
free(instance->set_master_version);
free(instance->set_master_hostname);
free(instance->fileroot);
free(instance->binlogdir);
free(instance);
for (SERV_LISTENER *port = instance->service->ports; port; port = port->next)
{
users_free(port->users);
}
MXS_FREE(instance->uuid);
MXS_FREE(instance->user);
MXS_FREE(instance->password);
MXS_FREE(instance->set_master_server_id);
MXS_FREE(instance->set_master_uuid);
MXS_FREE(instance->set_master_version);
MXS_FREE(instance->set_master_hostname);
MXS_FREE(instance->fileroot);
MXS_FREE(instance->binlogdir);
/* SSL options */
MXS_FREE(instance->ssl_ca);
MXS_FREE(instance->ssl_cert);
MXS_FREE(instance->ssl_key);
MXS_FREE(instance->ssl_version);
MXS_FREE(instance);
}
/**
@ -770,9 +878,8 @@ newSession(ROUTER *instance, SESSION *session)
session,
inst);
if ((slave = (ROUTER_SLAVE *)calloc(1, sizeof(ROUTER_SLAVE))) == NULL)
if ((slave = (ROUTER_SLAVE *)MXS_CALLOC(1, sizeof(ROUTER_SLAVE))) == NULL)
{
MXS_ERROR("Insufficient memory to create new slave session for binlog router");
return NULL;
}
@ -871,17 +978,17 @@ static void freeSession(ROUTER* router_instance,
if (slave->hostname)
{
free(slave->hostname);
MXS_FREE(slave->hostname);
}
if (slave->user)
{
free(slave->user);
MXS_FREE(slave->user);
}
if (slave->passwd)
{
free(slave->passwd);
MXS_FREE(slave->passwd);
}
free(slave);
MXS_FREE(slave);
}
@ -1086,12 +1193,29 @@ diagnostics(ROUTER *router, DCB *dcb)
if (router_inst->master)
{
dcb_printf(dcb, "\tMaster connection DCB: %p\n",
dcb_printf(dcb, "\tMaster connection DCB: %p\n",
router_inst->master);
}
else
{
dcb_printf(dcb, "\tMaster connection DCB: 0x0\n");
dcb_printf(dcb, "\tMaster connection DCB: 0x0\n");
}
/* SSL options */
if (router_inst->ssl_enabled)
{
dcb_printf(dcb, "\tMaster SSL is ON:\n");
if (router_inst->service->dbref->server && router_inst->service->dbref->server->server_ssl)
{
dcb_printf(dcb, "\t\tMaster SSL CA cert: %s\n",
router_inst->service->dbref->server->server_ssl->ssl_ca_cert);
dcb_printf(dcb, "\t\tMaster SSL Cert: %s\n",
router_inst->service->dbref->server->server_ssl->ssl_cert);
dcb_printf(dcb, "\t\tMaster SSL Key: %s\n",
router_inst->service->dbref->server->server_ssl->ssl_key);
dcb_printf(dcb, "\t\tMaster SSL tls_ver: %s\n",
router_inst->ssl_version ? router_inst->ssl_version : "MAX");
}
}
dcb_printf(dcb, "\tMaster connection state: %s\n",
@ -1305,6 +1429,13 @@ diagnostics(ROUTER *router, DCB *dcb)
dcb_printf(dcb,
"\t\tSlave DCB: %p\n",
session->dcb);
if (session->dcb->ssl)
{
dcb_printf(dcb,
"\t\tSlave connected with SSL: %s\n",
session->dcb->ssl_state == SSL_ESTABLISHED ?
"Established" : "Not connected yet");
}
dcb_printf(dcb,
"\t\tNext Sequence No: %d\n",
session->seqno);
@ -1459,7 +1590,7 @@ extract_message(GWBUF *errpkt)
int len;
len = EXTRACT24(errpkt->start);
if ((rval = (char *)malloc(len)) == NULL)
if ((rval = (char *)MXS_MALLOC(len)) == NULL)
{
return NULL;
}
@ -1502,9 +1633,48 @@ errorReply(ROUTER *instance,
char *errmsg;
unsigned long mysql_errno;
mysql_errno = (unsigned long) extract_field(((uint8_t *)GWBUF_DATA(message) + 5), 16);
errmsg = extract_message(message);
/** Don't handle same error twice on same DCB */
if (backend_dcb->dcb_errhandle_called)
{
/** Check router state and set errno an message */
if (router->master_state < BLRM_BINLOGDUMP || router->master_state != BLRM_SLAVE_STOPPED)
{
/* Authentication failed */
if (router->master_state == BLRM_TIMESTAMP)
{
spinlock_acquire(&router->lock);
/* set io error message */
if (router->m_errmsg)
{
free(router->m_errmsg);
}
router->m_errmsg = mxs_strdup("#28000 Authentication with master server failed");
/* set mysql_errno */
router->m_errno = 1045;
/* Stop replication */
router->master_state = BLRM_SLAVE_STOPPED;
spinlock_release(&router->lock);
/* Force backend DCB close */
dcb_close(backend_dcb);
MXS_ERROR("%s: Master connection error %lu '%s' in state '%s', "
"%s while connecting to master %s:%d",
router->service->name, router->m_errno, router->m_errmsg,
blrm_states[BLRM_TIMESTAMP], msg,
router->service->dbref->server->name,
router->service->dbref->server->port);
}
}
if (errmsg)
{
free(errmsg);
}
/** we optimistically assume that previous call succeed */
*succp = true;
return;
@ -1527,23 +1697,22 @@ errorReply(ROUTER *instance,
strcpy(msg, "");
}
mysql_errno = (unsigned long) extract_field(((uint8_t *)GWBUF_DATA(message) + 5), 16);
errmsg = extract_message(message);
if (router->master_state < BLRM_BINLOGDUMP || router->master_state != BLRM_SLAVE_STOPPED)
{
spinlock_acquire(&router->lock);
/* set mysql_errno */
router->m_errno = mysql_errno;
/* set io error message */
if (router->m_errmsg)
{
free(router->m_errmsg);
MXS_FREE(router->m_errmsg);
}
router->m_errmsg = strdup(errmsg);
router->m_errmsg = MXS_STRDUP_A(errmsg);
spinlock_release(&router->lock);
MXS_ERROR("%s: Master connection error %lu '%s' in state '%s', "
"%sattempting reconnect to master %s:%d",
"%s attempting reconnect to master %s:%d",
router->service->name, mysql_errno, errmsg,
blrm_states[router->master_state], msg,
router->service->dbref->server->name,
@ -1552,7 +1721,7 @@ errorReply(ROUTER *instance,
else
{
MXS_ERROR("%s: Master connection error %lu '%s' in state '%s', "
"%sattempting reconnect to master %s:%d",
"%s attempting reconnect to master %s:%d",
router->service->name, router->m_errno,
router->m_errmsg ? router->m_errmsg : "(memory failure)",
blrm_states[router->master_state], msg,
@ -1562,7 +1731,7 @@ errorReply(ROUTER *instance,
if (errmsg)
{
free(errmsg);
MXS_FREE(errmsg);
}
*succp = true;
dcb_close(backend_dcb);
@ -1696,7 +1865,7 @@ blr_statistics(ROUTER_INSTANCE *router, ROUTER_SLAVE *slave, GWBUF *queue)
*ptr++ = (len & 0xff00) >> 8;
*ptr++ = (len & 0xff0000) >> 16;
*ptr++ = 1;
strncpy(ptr, result, len);
memcpy(ptr, result, len);
return slave->dcb->func.write(slave->dcb, ret);
}
@ -1879,9 +2048,15 @@ static int
blr_handle_config_item(const char *name, const char *value, ROUTER_INSTANCE *inst)
{
SERVICE *service;
char *ssl_cert;
char *ssl_key;
char *ssl_ca_cert;
SERVER *backend_server;
service = inst->service;
backend_server = service->dbref->server;
if (strcmp(name, "master_host") == 0)
{
server_update_address(service->dbref->server, (char *)value);
@ -1892,24 +2067,67 @@ blr_handle_config_item(const char *name, const char *value, ROUTER_INSTANCE *ins
}
else if (strcmp(name, "filestem") == 0)
{
free(inst->fileroot);
inst->fileroot = strdup(value);
MXS_FREE(inst->fileroot);
inst->fileroot = MXS_STRDUP_A(value);
}
else if (strcmp(name, "master_user") == 0)
{
if (inst->user)
{
free(inst->user);
MXS_FREE(inst->user);
}
inst->user = strdup(value);
inst->user = MXS_STRDUP_A(value);
}
else if (strcmp(name, "master_password") == 0)
{
if (inst->password)
{
free(inst->password);
MXS_FREE(inst->password);
}
inst->password = MXS_STRDUP_A(value);
}
/** Checl for SSL options */
else if (strcmp(name, "master_ssl") == 0)
{
inst->ssl_enabled = config_truth_value((char*)value);
}
else if (strcmp(name, "master_ssl_ca") == 0)
{
MXS_FREE(backend_server->server_ssl->ssl_ca_cert);
backend_server->server_ssl->ssl_ca_cert = value ? MXS_STRDUP_A(value) : NULL;
MXS_FREE(inst->ssl_ca);
inst->ssl_ca = value ? MXS_STRDUP_A(value) : NULL;
}
else if (strcmp(name, "master_ssl_cert") == 0)
{
MXS_FREE(backend_server->server_ssl->ssl_cert);
backend_server->server_ssl->ssl_cert = value ? MXS_STRDUP_A(value) : NULL;
MXS_FREE(inst->ssl_cert);
inst->ssl_cert = value ? MXS_STRDUP_A(value) : NULL;
}
else if (strcmp(name, "master_ssl_key") == 0)
{
MXS_FREE(backend_server->server_ssl->ssl_key);
backend_server->server_ssl->ssl_key = value ? MXS_STRDUP_A(value) : NULL;
MXS_FREE(inst->ssl_key);
inst->ssl_key = value ? MXS_STRDUP_A(value) : NULL;
}
else if (strcmp(name, "master_ssl_version") == 0 || strcmp(name, "master_tls_version") == 0)
{
if (value)
{
if (listener_set_ssl_version(backend_server->server_ssl, (char *)value) != 0)
{
MXS_ERROR("Unknown parameter value for 'ssl_version' for"
" service '%s': %s",
inst->service->name,
value);
}
else
{
inst->ssl_version = MXS_STRDUP_A(value);
}
}
inst->password = strdup(value);
}
else
{
@ -1961,18 +2179,22 @@ blr_set_service_mysql_user(SERVICE *service)
MXS_ERROR("create hex_sha1_sha1_password failed for service user %s",
service_user);
free(dpwd);
MXS_FREE(dpwd);
return 1;
}
/* add service user for % and localhost */
(void)add_mysql_users_with_host_ipv4(service->users, service->credentials.name,
"%", newpasswd, "Y", "");
(void)add_mysql_users_with_host_ipv4(service->users, service->credentials.name,
"localhost", newpasswd, "Y", "");
/** Add the service user for % and localhost to all listeners so that
* it can always be used. */
for (SERV_LISTENER *port = service->ports; port; port = port->next)
{
add_mysql_users_with_host_ipv4(port->users, service->credentials.name,
"%", newpasswd, "Y", "");
add_mysql_users_with_host_ipv4(port->users, service->credentials.name,
"localhost", newpasswd, "Y", "");
}
free(newpasswd);
free(dpwd);
MXS_FREE(newpasswd);
MXS_FREE(dpwd);
return 0;
}
@ -1986,57 +2208,58 @@ blr_set_service_mysql_user(SERVICE *service)
static int
blr_load_dbusers(const ROUTER_INSTANCE *router)
{
int loaded = -1;
char path[PATH_MAX + 1] = "";
int loaded_total = 0;
SERVICE *service;
char path[PATH_MAX];
service = router->service;
/* File path for router cached authentication data */
strncpy(path, router->binlogdir, PATH_MAX);
strncat(path, "/cache", PATH_MAX - strlen(path));
strncat(path, "/dbusers", PATH_MAX - strlen(path));
/* Try loading dbusers from configured backends */
loaded = load_mysql_users(service);
if (loaded < 0)
for (SERV_LISTENER *port = service->ports; port; port = port->next)
{
MXS_ERROR("Unable to load users for service %s",
service->name);
sprintf(path, "%s/%s/%s/", router->binlogdir, BLR_DBUSERS_DIR, port->name);
/* Try loading authentication data from file cache */
loaded = dbusers_load(router->service->users, path);
if (loaded != -1)
if (mxs_mkdir_all(path, 0775))
{
MXS_ERROR("Service %s, Using cached credential information file %s.",
service->name,
path);
strcat(path, BLR_DBUSERS_FILE);
}
/* Try loading dbusers from configured backends */
int loaded = load_mysql_users(port);
if (loaded < 0)
{
MXS_ERROR("Unable to load users for service %s", service->name);
/* Try loading authentication data from file cache */
loaded = dbusers_load(port->users, path);
if (loaded != -1)
{
MXS_ERROR("Service %s, Listener %s, Using cached credential information file %s.",
service->name, port->name, path);
}
else
{
MXS_ERROR("Service %s, Listener %s, Unable to read cache credential"
" information from %s. No database user added to service users table.",
service->name, port->name, path);
}
}
else
{
MXS_ERROR("Service %s, Unable to read cache credential information from %s."
" No database user added to service users table.",
service->name,
path);
}
}
else
{
/* don't update cache if no user was loaded */
if (loaded == 0)
{
MXS_ERROR("Service %s: failed to load any user "
"information. Authentication will "
"probably fail as a result.",
service->name);
}
else
{
/* update cached data */
blr_save_dbusers(router);
/* don't update cache if no user was loaded */
if (loaded == 0)
{
MXS_ERROR("Service %s, Listener %s: failed to load any user information."
" Authentication will probably fail as a result.",
service->name, port->name);
}
else
{
/* update cached data */
dbusers_save(port->users, path);
}
}
loaded_total += loaded;
}
/* At service start last update is set to USERS_REFRESH_TIME seconds earlier.
@ -2045,50 +2268,7 @@ blr_load_dbusers(const ROUTER_INSTANCE *router)
service->rate_limit.last = time(NULL) - USERS_REFRESH_TIME;
service->rate_limit.nloads = 1;
return loaded;
}
/**
* Save dbusers to cache file
*
* @param router The router instance
* @return -1 on failure, >= 0 on success
*/
int
blr_save_dbusers(const ROUTER_INSTANCE *router)
{
SERVICE *service;
char path[PATH_MAX + 1] = "";
int mkdir_rval = 0;
service = router->service;
/* File path for router cached authentication data */
strncpy(path, router->binlogdir, PATH_MAX);
strncat(path, "/cache", PATH_MAX - strlen(path));
/* check and create dir */
if (access(path, R_OK) == -1)
{
mkdir_rval = mkdir(path, 0700);
}
if (mkdir_rval == -1)
{
char err_msg[STRERROR_BUFLEN];
MXS_ERROR("Service %s, Failed to create directory '%s': [%d] %s",
service->name,
path,
errno,
strerror_r(errno, err_msg, sizeof(err_msg)));
return -1;
}
/* set cache file name */
strncat(path, "/dbusers", PATH_MAX - strlen(path));
return dbusers_save(service->users, path);
return loaded_total;
}
/**
@ -2153,7 +2333,7 @@ static int blr_check_binlog(ROUTER_INSTANCE *router)
router->m_errno = 2032;
/* set io error message */
router->m_errmsg = strdup(msg_err);
router->m_errmsg = MXS_STRDUP_A(msg_err);
/* set last_safe_pos */
router->last_safe_pos = router->binlog_position;
@ -2248,3 +2428,26 @@ blr_get_event_description(ROUTER_INSTANCE *router, uint8_t event)
return event_desc;
}
/**
* Free SSL struct in server struct
*
* @param inst The router instance
*/
void
blr_free_ssl_data(ROUTER_INSTANCE *inst)
{
SSL_LISTENER *server_ssl;
if (inst->service->dbref->server->server_ssl)
{
server_ssl = inst->service->dbref->server->server_ssl;
/* Free SSL struct */
/* Note: SSL struct in server should be freed by server_free() */
MXS_FREE(server_ssl->ssl_key);
MXS_FREE(server_ssl->ssl_ca_cert);
MXS_FREE(server_ssl->ssl_cert);
MXS_FREE(inst->service->dbref->server->server_ssl);
inst->service->dbref->server->server_ssl = NULL;
}
}

2
server/modules/routing/binlog/blr_cache.c
View File

@ -4,7 +4,7 @@
* Use of this software is governed by the Business Source License included
* in the LICENSE.TXT file and at www.mariadb.com/bsl.
*
* Change Date: 2019-01-01
* Change Date: 2019-07-01
*
* On the date above, in accordance with the Business Source License, use
* of this software will be governed by version 2 or later of the General

175
server/modules/routing/binlog/blr_file.c
View File

@ -4,7 +4,7 @@
* Use of this software is governed by the Business Source License included
* in the LICENSE.TXT file and at www.mariadb.com/bsl.
*
* Change Date: 2019-01-01
* Change Date: 2019-07-01
*
* On the date above, in accordance with the Business Source License, use
* of this software will be governed by version 2 or later of the General
@ -32,8 +32,9 @@
* This is the current supported condition for detecting
* MariaDB 10 transaction start point.
* It's no longer using QUERY_EVENT with BEGIN
* 23/10/2015 Markus Makela Added current_safe_event
* 23/10/2015 Markus Makela Added current_safe_event
* 26/04/2016 Massimiliano Pinto Added MariaDB 10.0 and 10.1 GTID event flags detection
* 11/07/2016 Massimiliano Pinto Added SSL backend support
*
* @endverbatim
*/
@ -58,6 +59,7 @@
#include <skygw_types.h>
#include <skygw_utils.h>
#include <log_manager.h>
#include <maxscale/alloc.h>
static int blr_file_create(ROUTER_INSTANCE *router, char *file);
static void blr_log_header(int priority, char *msg, uint8_t *ptr);
@ -100,22 +102,35 @@ blr_file_init(ROUTER_INSTANCE *router)
if (router->binlogdir == NULL)
{
strncpy(path, get_datadir(), PATH_MAX);
strncat(path, "/", PATH_MAX - strlen(path));
strncat(path, router->service->name, PATH_MAX - strlen(path));
const char* datadir = get_datadir();
size_t len = strlen(datadir) + sizeof('/') + strlen(router->service->name);
if (len > PATH_MAX)
{
MXS_ERROR("The length of %s/%s is more than the maximum length %d.",
datadir, router->service->name, PATH_MAX);
return 0;
}
strcpy(path, datadir);
strcat(path, "/");
strcat(path, router->service->name);
if (access(path, R_OK) == -1)
{
// TODO: Check what kind of error, ENOENT or something else.
mkdir(path, 0700);
// TODO: Check the result of mkdir.
}
router->binlogdir = strdup(path);
router->binlogdir = MXS_STRDUP_A(path);
}
else
{
strncpy(path, router->binlogdir, PATH_MAX);
strcpy(path, router->binlogdir);
}
if (access(router->binlogdir, R_OK) == -1)
if (access(path, R_OK) == -1)
{
MXS_ERROR("%s: Unable to read the binlog directory %s.",
router->service->name, router->binlogdir);
@ -222,6 +237,13 @@ blr_file_add_magic(int fd)
static int
blr_file_create(ROUTER_INSTANCE *router, char *file)
{
if (strlen(file) > BINLOG_FNAMELEN)
{
MXS_ERROR("The binlog filename %s is longer than the maximum allowed length %d.",
file, BINLOG_FNAMELEN);
return 0;
}
int created = 0;
char err_msg[STRERROR_BUFLEN];
@ -239,7 +261,7 @@ blr_file_create(ROUTER_INSTANCE *router, char *file)
{
close(router->binlog_fd);
spinlock_acquire(&router->binlog_lock);
strncpy(router->binlog_name, file, BINLOG_FNAMELEN);
strcpy(router->binlog_name, file);
router->binlog_fd = fd;
router->current_pos = BINLOG_MAGIC_SIZE; /* Initial position after the magic number */
router->binlog_position = BINLOG_MAGIC_SIZE;
@ -390,6 +412,22 @@ blr_file_flush(ROUTER_INSTANCE *router)
BLFILE *
blr_open_binlog(ROUTER_INSTANCE *router, char *binlog)
{
size_t len = strlen(binlog);
if (len > BINLOG_FNAMELEN)
{
MXS_ERROR("The binlog filename %s is longer than the maximum allowed length %d.",
binlog, BINLOG_FNAMELEN);
return NULL;
}
len += (strlen(router->binlogdir) + 1); // +1 for the /.
if (len > PATH_MAX)
{
MXS_ERROR("The length of %s/%s is longer than the maximum allowed length %d.",
router->binlogdir, binlog, PATH_MAX);
return NULL;
}
char path[PATH_MAX + 1] = "";
BLFILE *file;
@ -407,24 +445,24 @@ blr_open_binlog(ROUTER_INSTANCE *router, char *binlog)
return file;
}
if ((file = (BLFILE *)calloc(1, sizeof(BLFILE))) == NULL)
if ((file = (BLFILE *)MXS_CALLOC(1, sizeof(BLFILE))) == NULL)
{
spinlock_release(&router->fileslock);
return NULL;
}
strncpy(file->binlogname, binlog, BINLOG_FNAMELEN);
strcpy(file->binlogname, binlog);
file->refcnt = 1;
file->cache = 0;
spinlock_init(&file->lock);
strncpy(path, router->binlogdir, PATH_MAX);
strncat(path, "/", PATH_MAX - strlen(path));
strncat(path, binlog, PATH_MAX - strlen(path));
strcpy(path, router->binlogdir);
strcat(path, "/");
strcat(path, binlog);
if ((file->fd = open(path, O_RDONLY, 0666)) == -1)
{
MXS_ERROR("Failed to open binlog file %s", path);
free(file);
MXS_FREE(file);
spinlock_release(&router->fileslock);
return NULL;
}
@ -778,7 +816,7 @@ blr_close_binlog(ROUTER_INSTANCE *router, BLFILE *file)
{
close(file->fd);
file->fd = -1;
free(file);
MXS_FREE(file);
}
}
@ -836,25 +874,37 @@ blr_file_size(BLFILE *file)
void
blr_cache_response(ROUTER_INSTANCE *router, char *response, GWBUF *buf)
{
static const char CACHE[] = "/cache";
size_t len = strlen(router->binlogdir) + (sizeof(CACHE) - 1) + sizeof('/') + strlen(response);
if (len > PATH_MAX)
{
MXS_ERROR("The cache path %s%s/%s is longer than the maximum allowed length %d.",
router->binlogdir, CACHE, response, PATH_MAX);
return;
}
char path[PATH_MAX + 1] = "";
int fd;
strncpy(path, router->binlogdir, PATH_MAX);
strncat(path, "/cache", PATH_MAX - strlen(path));
strcpy(path, router->binlogdir);
strcat(path, CACHE);
if (access(path, R_OK) == -1)
{
// TODO: Check error, ENOENT or something else.
mkdir(path, 0700);
// TODO: Check return value.
}
strncat(path, "/", PATH_MAX - strlen(path));
strncat(path, response, PATH_MAX - strlen(path));
strcat(path, "/");
strcat(path, response);
if ((fd = open(path, O_WRONLY | O_CREAT | O_TRUNC, 0666)) == -1)
{
return;
}
write(fd, GWBUF_DATA(buf), GWBUF_LENGTH(buf));
// TODO: Check result.
close(fd);
}
@ -873,15 +923,24 @@ blr_cache_response(ROUTER_INSTANCE *router, char *response, GWBUF *buf)
GWBUF *
blr_cache_read_response(ROUTER_INSTANCE *router, char *response)
{
static const char CACHE[] = "/cache";
size_t len = strlen(router->binlogdir) + (sizeof(CACHE) - 1) + sizeof('/') + strlen(response);
if (len > PATH_MAX)
{
MXS_ERROR("The cache path %s%s/%s is longer than the maximum allowed length %d.",
router->binlogdir, CACHE, response, PATH_MAX);
return NULL;
}
struct stat statb;
char path[PATH_MAX + 1] = "";
int fd;
GWBUF *buf;
strncpy(path, router->binlogdir, PATH_MAX);
strncat(path, "/cache", PATH_MAX - strlen(path));
strncat(path, "/", PATH_MAX - strlen(path));
strncat(path, response, PATH_MAX - strlen(path));
strcpy(path, router->binlogdir);
strcat(path, CACHE);
strcat(path, "/");
strcat(path, response);
if ((fd = open(path, O_RDONLY)) == -1)
{
@ -1521,12 +1580,12 @@ blr_read_events_all_events(ROUTER_INSTANCE *router, int fix, int debug)
BINLOG_EVENT_HDR_LEN -
(4 + 4 + 1 + 2 + 2 + var_block_len + 1 + db_name_len);
statement_sql = calloc(1, statement_len + 1);
statement_sql = MXS_CALLOC(1, statement_len + 1);
if (statement_sql)
{
strncpy(statement_sql,
(char *)ptr + 4 + 4 + 1 + 2 + 2 + var_block_len + 1 + db_name_len,
statement_len);
memcpy(statement_sql,
(char *)ptr + 4 + 4 + 1 + 2 + 2 + var_block_len + 1 + db_name_len,
statement_len);
/* A transaction starts with this event */
if (strncmp(statement_sql, "BEGIN", 5) == 0)
@ -1534,10 +1593,10 @@ blr_read_events_all_events(ROUTER_INSTANCE *router, int fix, int debug)
if (pending_transaction > 0)
{
MXS_ERROR("Transaction cannot be @ pos %llu: "
"Another transaction was opened at %llu",
pos, last_known_commit);
"Another transaction was opened at %llu",
pos, last_known_commit);
free(statement_sql);
MXS_FREE(statement_sql);
gwbuf_free(result);
break;
@ -1551,7 +1610,7 @@ blr_read_events_all_events(ROUTER_INSTANCE *router, int fix, int debug)
if (debug)
{
MXS_DEBUG("> Transaction starts @ pos %llu", pos);
MXS_DEBUG("> Transaction starts @ pos %llu", pos);
}
}
}
@ -1570,7 +1629,7 @@ blr_read_events_all_events(ROUTER_INSTANCE *router, int fix, int debug)
}
}
}
free(statement_sql);
MXS_FREE(statement_sql);
}
else
{
@ -1842,18 +1901,20 @@ blr_file_write_master_config(ROUTER_INSTANCE *router, char *error)
char *section = "binlog_configuration";
FILE *config_file;
int rc;
char path[(PATH_MAX - 15) + 1] = "";
char filename[(PATH_MAX - 4) + 1] = "";
char tmp_file[PATH_MAX + 1] = "";
static const char MASTER_INI[] = "master.ini";
static const char TMP[] = "tmp";
size_t len = strlen(router->binlogdir);
char filename[len + sizeof('/') + sizeof(MASTER_INI)]; // sizeof includes NULL
char tmp_file[len + sizeof('/') + sizeof(MASTER_INI) + sizeof('.') + sizeof(TMP)];
char err_msg[STRERROR_BUFLEN];
char *ssl_ca;
char *ssl_cert;
char *ssl_key;
char *ssl_version;
strncpy(path, router->binlogdir, (PATH_MAX - 15));
snprintf(filename, (PATH_MAX - 4), "%s/master.ini", path);
snprintf(tmp_file, (PATH_MAX - 4), "%s", filename);
strcat(tmp_file, ".tmp");
sprintf(filename, "%s/%s", router->binlogdir, MASTER_INI);
sprintf(tmp_file, "%s/%s.%s", router->binlogdir, MASTER_INI, TMP);
/* open file for writing */
config_file = fopen(tmp_file, "wb");
@ -1881,6 +1942,34 @@ blr_file_write_master_config(ROUTER_INSTANCE *router, char *error)
fprintf(config_file, "master_password=%s\n", router->password);
fprintf(config_file, "filestem=%s\n", router->fileroot);
/* Add SSL options */
if (router->ssl_enabled)
{
/* Use current settings */
ssl_ca = router->service->dbref->server->server_ssl->ssl_ca_cert;
ssl_cert = router->service->dbref->server->server_ssl->ssl_cert;
ssl_key = router->service->dbref->server->server_ssl->ssl_key;
}
else
{
/* Try using previous configuration settings */
ssl_ca = router->ssl_ca;
ssl_cert = router->ssl_cert;
ssl_key = router->ssl_key;
}
ssl_version = router->ssl_version;
if (ssl_key && ssl_cert && ssl_ca)
{
fprintf(config_file, "master_ssl=%d\n", router->ssl_enabled);
fprintf(config_file, "master_ssl_key=%s\n", ssl_key);
fprintf(config_file, "master_ssl_cert=%s\n",ssl_cert);
fprintf(config_file, "master_ssl_ca=%s\n", ssl_ca);
}
if (ssl_version && strlen(ssl_version))
fprintf(config_file, "master_tls_version=%s\n", ssl_version);
fclose(config_file);
/* rename tmp file to right filename */

488
server/modules/routing/binlog/blr_master.c
View File

@ -7,7 +7,7 @@
* Use of this software is governed by the Business Source License included
* in the LICENSE.TXT file and at www.mariadb.com/bsl.
*
* Change Date: 2019-01-01
* Change Date: 2019-07-01
*
* On the date above, in accordance with the Business Source License, use
* of this software will be governed by version 2 or later of the General
@ -47,6 +47,7 @@
* 25/09/2015 Massimiliano Pinto Addition of lastEventReceived for slaves
* 23/10/2015 Markus Makela Added current_safe_event
* 26/04/2016 Massimiliano Pinto Added MariaDB 10.0 and 10.1 GTID event flags detection
* 22/07/2016 Massimiliano Pinto Added semi_sync replication support
*
* @endverbatim
*/
@ -76,6 +77,8 @@
/* Temporary requirement for auth data */
#include <mysql_client_server_protocol.h>
#include <maxscale/alloc.h>
static GWBUF *blr_make_query(char *statement);
@ -99,6 +102,9 @@ extern int blr_check_heartbeat(ROUTER_INSTANCE *router);
static void blr_log_identity(ROUTER_INSTANCE *router);
static void blr_distribute_error_message(ROUTER_INSTANCE *router, char *message, char *state,
unsigned int err_code);
static void blr_extract_header_semisync(uint8_t *pkt, REP_HEADER *hdr);
static int blr_send_semisync_ack (ROUTER_INSTANCE *router, uint64_t pos);
static int blr_get_master_semisync(GWBUF *buf);
int blr_write_data_into_binlog(ROUTER_INSTANCE *router, uint32_t data_len, uint8_t *buf);
void extract_checksum(ROUTER_INSTANCE* router, uint8_t *cksumptr, uint8_t len);
@ -106,6 +112,30 @@ static void blr_terminate_master_replication(ROUTER_INSTANCE *router, uint8_t* p
static int keepalive = 1;
/** Transactio-Safety feature */
typedef enum
{
BLRM_NO_TRANSACTION, /*< No transaction */
BLRM_TRANSACTION_START, /*< A transaction is open*/
BLRM_COMMIT_SEEN, /*< Received COMMIT event in the current transaction */
BLRM_XID_EVENT_SEEN /*< Received XID event of current transaction */
} master_transaction_t;
/** Master Semi-Sync capability */
typedef enum
{
MASTER_SEMISYNC_NOT_AVAILABLE, /*< Semi-Sync replication not available */
MASTER_SEMISYNC_DISABLED, /*< Semi-Sync is disabled */
MASTER_SEMISYNC_ENABLED /*< Semi-Sync is enabled */
} master_semisync_capability_t;
#define MASTER_BYTES_BEFORE_EVENT 5
#define MASTER_BYTES_BEFORE_EVENT_SEMI_SYNC MASTER_BYTES_BEFORE_EVENT + 2
/* Semi-Sync indicator in network packet (byte 6) */
#define BLR_MASTER_SEMI_SYNC_INDICATOR 0xef
/* Semi-Sync flag ACK_REQ in network packet (byte 7) */
#define BLR_MASTER_SEMI_SYNC_ACK_REQ 0x01
/**
* blr_start_master - controls the connection of the binlog router to the
* master MySQL server and triggers the slave registration process for
@ -162,13 +192,14 @@ blr_start_master(void* data)
client->session = router->session;
if ((router->master = dcb_connect(router->service->dbref->server, router->session, BLR_PROTOCOL)) == NULL)
{
char *name;
if ((name = malloc(strlen(router->service->name) + strlen(" Master") + 1)) != NULL)
char *name = MXS_MALLOC(strlen(router->service->name) + strlen(" Master") + 1);
if (name)
{
sprintf(name, "%s Master", router->service->name);
hktask_oneshot(name, blr_start_master, router,
BLR_MASTER_BACKOFF_TIME * router->retry_backoff++);
free(name);
MXS_FREE(name);
}
if (router->retry_backoff > BLR_MAX_BACKOFF)
{
@ -178,7 +209,7 @@ blr_start_master(void* data)
router->service->dbref->server->unique_name);
return;
}
router->master->remote = strdup(router->service->dbref->server->name);
router->master->remote = MXS_STRDUP_A(router->service->dbref->server->name);
MXS_NOTICE("%s: attempting to connect to master server %s:%d, binlog %s, pos %lu",
router->service->name, router->service->dbref->server->name,
@ -225,16 +256,16 @@ blr_restart_master(ROUTER_INSTANCE *router)
spinlock_release(&router->lock);
if (router->master_state < BLRM_BINLOGDUMP)
{
char *name;
router->master_state = BLRM_UNCONNECTED;
if ((name = malloc(strlen(router->service->name) + strlen(" Master") + 1)) != NULL)
char *name = (char*)MXS_MALLOC(strlen(router->service->name) + strlen(" Master") + 1);
if (name)
{
sprintf(name, "%s Master", router->service->name);
hktask_oneshot(name, blr_start_master, router,
BLR_MASTER_BACKOFF_TIME * router->retry_backoff++);
free(name);
MXS_FREE(name);
}
if (router->retry_backoff > BLR_MAX_BACKOFF)
{
@ -315,13 +346,13 @@ blr_master_close(ROUTER_INSTANCE *router)
void
blr_master_delayed_connect(ROUTER_INSTANCE *router)
{
char *name;
char *name = (char*)MXS_MALLOC(strlen(router->service->name) + strlen(" Master Recovery") + 1);
if ((name = malloc(strlen(router->service->name) + strlen(" Master Recovery") + 1)) != NULL)
if (name)
{
sprintf(name, "%s Master Recovery", router->service->name);
hktask_oneshot(name, blr_start_master, router, 60);
free(name);
MXS_FREE(name);
}
}
@ -349,7 +380,7 @@ blr_master_response(ROUTER_INSTANCE *router, GWBUF *buf)
{
MXS_ERROR("Invalid master state machine state (%d) for binlog router.",
router->master_state);
gwbuf_consume(buf, gwbuf_length(buf));
gwbuf_free(buf);
spinlock_acquire(&router->lock);
if (router->reconnect_pending)
@ -388,12 +419,12 @@ blr_master_response(ROUTER_INSTANCE *router, GWBUF *buf)
unsigned long mysql_errno = extract_field(MYSQL_ERROR_CODE(buf), 16);
msg_len = len - 7 - 6; // +7 is where msg starts, 6 is skipped the status message (#42000)
msg_err = (char *)malloc(msg_len + 1);
msg_err = (char *)MXS_MALLOC(msg_len + 1);
if (msg_err)
{
// skip status message only as MYSQL_RESPONSE_ERR(buf) points to GWBUF_DATA(buf) +7
strncpy(msg_err, (char *)(MYSQL_ERROR_MSG(buf) + 6), msg_len);
memcpy(msg_err, (char *)(MYSQL_ERROR_MSG(buf) + 6), msg_len);
/* NULL terminated error string */
*(msg_err + msg_len) = '\0';
@ -405,7 +436,7 @@ blr_master_response(ROUTER_INSTANCE *router, GWBUF *buf)
mysql_errno,
msg_err ? msg_err : "(memory failure)",
blrm_states[router->master_state]);
gwbuf_consume(buf, gwbuf_length(buf));
gwbuf_free(buf);
spinlock_acquire(&router->lock);
@ -415,7 +446,7 @@ blr_master_response(ROUTER_INSTANCE *router, GWBUF *buf)
/* set mysql error message */
if (router->m_errmsg)
{
free(router->m_errmsg);
MXS_FREE(router->m_errmsg);
}
router->m_errmsg = msg_err ? msg_err : "(memory failure)";
@ -435,7 +466,7 @@ blr_master_response(ROUTER_INSTANCE *router, GWBUF *buf)
{
case BLRM_TIMESTAMP:
// Response to a timestamp message, no need to save this.
gwbuf_consume(buf, GWBUF_LENGTH(buf));
gwbuf_free(buf);
buf = blr_make_query("SHOW VARIABLES LIKE 'SERVER_ID'");
router->master_state = BLRM_SERVERID;
router->master->func.write(router->master, buf);
@ -466,7 +497,7 @@ blr_master_response(ROUTER_INSTANCE *router, GWBUF *buf)
}
router->master_state = BLRM_HBPERIOD;
router->master->func.write(router->master, buf);
free(val);
MXS_FREE(val);
break;
}
case BLRM_HBPERIOD:
@ -503,7 +534,7 @@ blr_master_response(ROUTER_INSTANCE *router, GWBUF *buf)
}
if (val)
{
free(val);
MXS_FREE(val);
}
// Response to the master_binlog_checksum, should be stored
if (router->saved_master.chksum2)
@ -562,13 +593,13 @@ blr_master_response(ROUTER_INSTANCE *router, GWBUF *buf)
}
if (key)
{
free(key);
MXS_FREE(key);
}
/* set the master_uuid from master if not set by the option */
if (router->set_master_uuid == NULL)
{
free(router->master_uuid);
MXS_FREE(router->master_uuid);
router->master_uuid = val;
}
else
@ -686,9 +717,107 @@ blr_master_response(ROUTER_INSTANCE *router, GWBUF *buf)
router->master->func.write(router->master, buf);
break;
case BLRM_REGISTER:
// Request a dump of the binlog file
/* discard master reply to COM_REGISTER_SLAVE */
gwbuf_free(buf);
/* if semisync option is set, check for master semi-sync availability */
if (router->request_semi_sync)
{
MXS_NOTICE("%s: checking Semi-Sync replication capability for master server %s:%d",
router->service->name,
router->service->dbref->server->name,
router->service->dbref->server->port);
buf = blr_make_query("SHOW VARIABLES LIKE 'rpl_semi_sync_master_enabled'");
router->master_state = BLRM_CHECK_SEMISYNC;
router->master->func.write(router->master, buf);
break;
}
else
{
/* Continue */
router->master_state = BLRM_REQUEST_BINLOGDUMP;
}
case BLRM_CHECK_SEMISYNC:
{
/**
* This branch could be reached as fallthrough from BLRM_REGISTER
* if request_semi_sync option is false
*/
if (router->master_state == BLRM_CHECK_SEMISYNC)
{
/* Get master semi-sync installed, enabled, disabled */
router->master_semi_sync = blr_get_master_semisync(buf);
/* Discard buffer */
gwbuf_free(buf);
if (router->master_semi_sync == MASTER_SEMISYNC_NOT_AVAILABLE)
{
/* not installed */
MXS_NOTICE("%s: master server %s:%d doesn't have semi_sync capability",
router->service->name,
router->service->dbref->server->name,
router->service->dbref->server->port);
/* Continue */
router->master_state = BLRM_REQUEST_BINLOGDUMP;
}
else
{
if (router->master_semi_sync == MASTER_SEMISYNC_DISABLED)
{
/* Installed but not enabled, right now */
MXS_NOTICE("%s: master server %s:%d doesn't have semi_sync enabled right now, "
"Requesting Semi-Sync Replication",
router->service->name,
router->service->dbref->server->name,
router->service->dbref->server->port);
}
else
{
/* Installed and enabled */
MXS_NOTICE("%s: master server %s:%d has semi_sync enabled, Requesting Semi-Sync Replication",
router->service->name,
router->service->dbref->server->name,
router->service->dbref->server->port);
}
buf = blr_make_query("SET @rpl_semi_sync_slave = 1");
router->master_state = BLRM_REQUEST_SEMISYNC;
router->master->func.write(router->master, buf);
break;
}
}
}
case BLRM_REQUEST_SEMISYNC:
/**
* This branch could be reached as fallthrough from BLRM_REGISTER or BLRM_CHECK_SEMISYNC
* if request_semi_sync option is false or master doesn't support semisync or it's not enabled
*/
if (router->master_state == BLRM_REQUEST_SEMISYNC)
{
/* discard master reply */
gwbuf_free(buf);
/* Continue */
router->master_state = BLRM_REQUEST_BINLOGDUMP;
}
case BLRM_REQUEST_BINLOGDUMP:
/**
* This branch is reached after semi-sync check/request or
* just after sending COM_REGISTER_SLAVE if request_semi_sync option is false
*/
/* Request now a dump of the binlog file */
buf = blr_make_binlog_dump(router);
router->master_state = BLRM_BINLOGDUMP;
router->master->func.write(router->master, buf);
MXS_NOTICE("%s: Request binlog records from %s at "
"position %lu from master server %s:%d",
@ -701,6 +830,7 @@ blr_master_response(ROUTER_INSTANCE *router, GWBUF *buf)
blr_log_identity(router);
break;
case BLRM_BINLOGDUMP:
/**
* Main body, we have received a binlog record from the master
@ -864,6 +994,9 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
int preslen = -1;
int prev_length = -1;
int n_bufs = -1, pn_bufs = -1;
int check_packet_len;
int semisync_bytes;
int semi_sync_send_ack = 0;
/*
* Prepend any residual buffer to the buffer chain we have
@ -924,12 +1057,8 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
int msg_remainder = len;
GWBUF *p = pkt;
if ((msg = malloc(len)) == NULL)
if ((msg = MXS_MALLOC(len)) == NULL)
{
MXS_ERROR("Insufficient memory to buffer event "
"of %d bytes. Binlog %s @ %lu.",
len, router->binlog_name,
router->current_pos);
break;
}
@ -955,7 +1084,7 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
"message as expected. %s @ %lu",
router->binlog_name,
router->current_pos);
free(msg);
MXS_FREE(msg);
/* msg = NULL; Not needed unless msg will be referred to again */
break;
}
@ -985,6 +1114,8 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
n_bufs = 1;
}
semisync_bytes = 0;
/*
* ptr now points at the current message in a contiguous buffer,
* this buffer is either within the GWBUF or in a malloc'd
@ -1015,12 +1146,38 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
router->stats.n_binlogs_ses++;
spinlock_release(&router->lock);
blr_extract_header(ptr, &hdr);
/* Check for semi-sync in event with OK byte[4]:
* move pointer 2 bytes ahead and set check_packet_len accordingly
*/
if (ptr[4] == 0 && router->master_semi_sync != MASTER_SEMISYNC_NOT_AVAILABLE &&
ptr[5] == BLR_MASTER_SEMI_SYNC_INDICATOR)
{
check_packet_len = MASTER_BYTES_BEFORE_EVENT_SEMI_SYNC;
semi_sync_send_ack = ptr[6];
/* Extract header from the semi-sync event */
blr_extract_header_semisync(ptr, &hdr);
/** Remove the semi-sync bytes */
memmove(ptr, ptr + 2, 5);
ptr += 2;
semisync_bytes = 2;
}
else
{
semi_sync_send_ack = 0;
check_packet_len = MASTER_BYTES_BEFORE_EVENT;
/* Extract header from thr event */
blr_extract_header(ptr, &hdr);
}
/* Sanity check */
if (hdr.ok == 0)
{
if (hdr.event_size != len - 5 && (hdr.event_size + 1) < MYSQL_PACKET_LENGTH_MAX)
if (hdr.event_size != len - check_packet_len &&
(hdr.event_size + (check_packet_len - MYSQL_HEADER_LEN)) < MYSQL_PACKET_LENGTH_MAX)
{
MXS_ERROR("Packet length is %d, but event size is %d, "
"binlog file %s position %lu "
@ -1042,13 +1199,13 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
if (msg)
{
free(msg);
MXS_FREE(msg);
/* msg = NULL; Not needed unless msg will be referred to again */
}
break;
}
else if ((hdr.event_size + 1) >= MYSQL_PACKET_LENGTH_MAX)
else if (hdr.event_size + (check_packet_len - MYSQL_HEADER_LEN) >= MYSQL_PACKET_LENGTH_MAX)
{
router->master_event_state = BLR_EVENT_STARTED;
@ -1076,7 +1233,7 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
/* Remove error message */
if (router->m_errmsg)
{
free(router->m_errmsg);
MXS_FREE(router->m_errmsg);
}
router->m_errmsg = NULL;
@ -1118,22 +1275,28 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
extra_bytes = MYSQL_HEADER_LEN + 1;
}
ss_dassert(len - extra_bytes - semisync_bytes > 0);
uint32_t bytes_available = len - extra_bytes - semisync_bytes;
if (router->master_chksum)
{
uint32_t size = (len - extra_bytes) < router->checksum_size ?
len - extra_bytes : router->checksum_size;
uint32_t size = MIN(len - extra_bytes - semisync_bytes,
router->checksum_size);
router->stored_checksum = crc32(router->stored_checksum,
ptr + offset,
size);
router->checksum_size -= size;
if (router->checksum_size == 0 && size < len - offset)
if (router->checksum_size == 0 && size < bytes_available)
{
extract_checksum(router, ptr + offset + size, len - offset - size);
extract_checksum(router, ptr + offset + size,
bytes_available - size);
}
}
if (blr_write_data_into_binlog(router, len - offset, ptr + offset) == 0)
if (blr_write_data_into_binlog(router, bytes_available,
ptr + offset) == 0)
{
/** Failed to write to the binlog file, destroy the buffer
* chain and close the connection with the master */
@ -1157,7 +1320,7 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
*/
if (router->master_chksum)
{
uint32_t pktsum, offset = MYSQL_HEADER_LEN;
uint32_t offset = MYSQL_HEADER_LEN;
uint32_t size = len - MYSQL_HEADER_LEN - MYSQL_CHECKSUM_LEN;
if (router->master_event_state == BLR_EVENT_DONE)
@ -1165,7 +1328,7 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
/** Set the pointer offset to the first byte after
* the header and OK byte */
offset = MYSQL_HEADER_LEN + 1;
size = len - MYSQL_HEADER_LEN - MYSQL_CHECKSUM_LEN - 1;
size = len - (check_packet_len + MYSQL_CHECKSUM_LEN);
}
size = MIN(size, router->checksum_size);
@ -1173,23 +1336,23 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
if (router->checksum_size > 0)
{
router->stored_checksum = crc32(router->stored_checksum,
ptr + offset,
size);
ptr + offset, size);
router->checksum_size -= size;
}
if (router->checksum_size == 0 && size < len - offset)
if(router->checksum_size == 0 && size < (len - offset - semisync_bytes))
{
extract_checksum(router, ptr + offset + size, len - offset - size);
extract_checksum(router, ptr + offset + size,
len - offset - size - semisync_bytes);
}
if (router->partial_checksum_bytes == MYSQL_CHECKSUM_LEN)
{
pktsum = EXTRACT32(router->partial_checksum);
uint32_t pktsum = EXTRACT32(router->partial_checksum);
if (pktsum != router->stored_checksum)
{
router->stats.n_badcrc++;
free(msg);
MXS_FREE(msg);
/* msg = NULL; Not needed unless msg will be referred to again */
MXS_ERROR("%s: Checksum error in event from master, "
"binlog %s @ %lu. Closing master connection.",
@ -1222,7 +1385,7 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
*/
spinlock_acquire(&router->binlog_lock);
if (router->trx_safe == 0 || (router->trx_safe && router->pending_transaction == 0))
if (router->trx_safe == 0 || (router->trx_safe && router->pending_transaction == BLRM_NO_TRANSACTION))
{
/* no pending transaction: set current_pos to binlog_position */
router->binlog_position = router->current_pos;
@ -1271,7 +1434,7 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
// An action should be taken here
}
router->pending_transaction = 1;
router->pending_transaction = BLRM_TRANSACTION_START;
spinlock_release(&router->binlog_lock);
}
@ -1289,18 +1452,19 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
db_name_len = ptr[4 + 20 + 4 + 4];
var_block_len = ptr[4 + 20 + 4 + 4 + 1 + 2];
statement_len = len - (4 + 20 + 4 + 4 + 1 + 2 + 2 + var_block_len + 1 + db_name_len);
statement_sql = calloc(1, statement_len + 1);
strncpy(statement_sql,
(char *)ptr + 4 + 20 + 4 + 4 + 1 + 2 + 2 + var_block_len + 1 + db_name_len,
statement_len);
statement_len = len - (4 + 20 + 4 + 4 + 1 + 2 + 2 + var_block_len + 1 + db_name_len) - semisync_bytes;
statement_sql = MXS_CALLOC(1, statement_len + 1);
MXS_ABORT_IF_NULL(statement_sql);
memcpy(statement_sql,
(char *)ptr + 4 + 20 + 4 + 4 + 1 + 2 + 2 + var_block_len + 1 + db_name_len,
statement_len);
spinlock_acquire(&router->binlog_lock);
/* Check for BEGIN (it comes for START TRANSACTION too) */
if (strncmp(statement_sql, "BEGIN", 5) == 0)
{
if (router->pending_transaction > 0)
if (router->pending_transaction > BLRM_NO_TRANSACTION)
{
MXS_ERROR("A transaction is already open "
"@ %lu and a new one starts @ %lu",
@ -1310,18 +1474,18 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
// An action should be taken here
}
router->pending_transaction = 1;
router->pending_transaction = BLRM_TRANSACTION_START;
}
/* Check for COMMIT in non transactional store engines */
if (strncmp(statement_sql, "COMMIT", 6) == 0)
{
router->pending_transaction = 2;
router->pending_transaction = BLRM_COMMIT_SEEN;
}
spinlock_release(&router->binlog_lock);
free(statement_sql);
MXS_FREE(statement_sql);
}
/* Check for COMMIT in Transactional engines, i.e InnoDB */
@ -1331,7 +1495,7 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
if (router->pending_transaction)
{
router->pending_transaction = 3;
router->pending_transaction = BLRM_XID_EVENT_SEEN;
}
spinlock_release(&router->binlog_lock);
}
@ -1364,27 +1528,18 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
* slaves that attach later.
*/
new_fde_len = hdr.event_size;
new_fde = malloc(hdr.event_size);
new_fde = MXS_MALLOC(hdr.event_size);
if (new_fde)
{
memcpy(new_fde, ptr + 5, hdr.event_size);
if (router->saved_master.fde_event)
{
free(router->saved_master.fde_event);
MXS_FREE(router->saved_master.fde_event);
}
router->saved_master.fde_event = new_fde;
router->saved_master.fde_len = new_fde_len;
}
else
{
MXS_ERROR("%s: Received a format description "
"event that MaxScale was unable to "
"record. Event length is %d.",
router->service->name,
hdr.event_size);
blr_log_packet(LOG_ERR,
"Format Description Event:", ptr, len);
}
}
}
else
@ -1429,9 +1584,8 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
* It is possible for an empty packet to be sent if an
* event is exactly 2^24 bytes long. In this case the
* empty packet should be discarded. */
if (len > MYSQL_HEADER_LEN &&
blr_write_binlog_record(router, &hdr, len - offset, ptr) == 0)
blr_write_binlog_record(router, &hdr, len - offset - semisync_bytes, ptr) == 0)
{
/*
* Failed to write to the
@ -1469,6 +1623,28 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
}
}
/* Handle semi-sync request fom master */
if (router->master_semi_sync != MASTER_SEMISYNC_NOT_AVAILABLE &&
semi_sync_send_ack == BLR_MASTER_SEMI_SYNC_ACK_REQ &&
(router->master_event_state == BLR_EVENT_COMPLETE ||
router->master_event_state == BLR_EVENT_DONE))
{
MXS_DEBUG("%s: binlog record in file %s, pos %lu has "
"SEMI_SYNC_ACK_REQ and needs a Semi-Sync ACK packet to "
"be sent to the master server %s:%d",
router->service->name, router->binlog_name,
router->current_pos,
router->service->dbref->server->name,
router->service->dbref->server->port);
/* Send Semi-Sync ACK packet to master server */
blr_send_semisync_ack(router, hdr.next_pos);
/* Reset ACK sending */
semi_sync_send_ack = 0;
}
/**
* Distributing binlog events to slaves
* may depend on pending transaction
@ -1476,7 +1652,7 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
spinlock_acquire(&router->binlog_lock);
if (router->trx_safe == 0 || (router->trx_safe && router->pending_transaction == 0))
if (router->trx_safe == 0 || (router->trx_safe && router->pending_transaction == BLRM_NO_TRANSACTION))
{
router->binlog_position = router->current_pos;
router->current_safe_event = router->last_event_pos;
@ -1525,7 +1701,7 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
*
*/
if (router->pending_transaction > 1)
if (router->pending_transaction > BLRM_TRANSACTION_START)
{
unsigned long long pos;
unsigned long long end_pos;
@ -1592,8 +1768,9 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
router->binlog_position,
router->current_pos);
strncpy(err_message,
"No transaction events sent", BINLOG_ERROR_MSG_LEN);
static const char MESSAGE[] = "No transaction events sent";
ss_dassert(sizeof(MESSAGE) <= BINLOG_ERROR_MSG_LEN);
strcpy(err_message, MESSAGE);
}
else
{
@ -1606,8 +1783,9 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
pos,
router->current_pos);
strncpy(err_message,
"Incomplete transaction events sent", BINLOG_ERROR_MSG_LEN);
static const char MESSAGE[] = "Incomplete transaction events sent";
ss_dassert(sizeof(MESSAGE) <= BINLOG_ERROR_MSG_LEN);
strcpy(err_message, MESSAGE);
}
/* distribute error message to registered slaves */
@ -1618,7 +1796,7 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
spinlock_acquire(&router->binlog_lock);
router->binlog_position = router->current_pos;
router->pending_transaction = 0;
router->pending_transaction = BLRM_NO_TRANSACTION;
spinlock_release(&router->binlog_lock);
}
@ -1679,7 +1857,7 @@ blr_handle_binlog_record(ROUTER_INSTANCE *router, GWBUF *pkt)
if (msg)
{
free(msg);
MXS_FREE(msg);
msg = NULL;
}
prev_length = len;
@ -1805,16 +1983,31 @@ CreateMySQLAuthData(char *username, char *password, char *database)
if (username == NULL || password == NULL)
{
MXS_ERROR("You must specify both username and password for the binlog router.\n");
MXS_ERROR("You must specify both username and password for the binlog router.");
return NULL;
}
if ((auth_info = calloc(1, sizeof(MYSQL_session))) == NULL)
if (strlen(username) > MYSQL_USER_MAXLEN)
{
MXS_ERROR("Provided user name %s is longer than maximum length %d.",
username, MYSQL_USER_MAXLEN);
return NULL;
}
if (strlen(database) > MYSQL_DATABASE_MAXLEN)
{
MXS_ERROR("Provided database %s is longer than maximum length %d.",
database, MYSQL_DATABASE_MAXLEN);
return NULL;
}
if ((auth_info = MXS_CALLOC(1, sizeof(MYSQL_session))) == NULL)
{
return NULL;
}
strncpy(auth_info->user, username, MYSQL_USER_MAXLEN);
strncpy(auth_info->db, database, MYSQL_DATABASE_MAXLEN);
strcpy(auth_info->user, username);
strcpy(auth_info->db, database);
gw_sha1_str((const uint8_t *)password, strlen(password), auth_info->client_sha1);
return auth_info;
@ -2191,7 +2384,7 @@ blr_extract_column(GWBUF *buf, int col)
ptr += collen;
}
collen = *ptr++;
if ((rval = malloc(collen + 1)) == NULL)
if ((rval = MXS_MALLOC(collen + 1)) == NULL)
{
return NULL;
}
@ -2370,7 +2563,7 @@ blr_stop_start_master(ROUTER_INSTANCE *router)
if (strcmp(router->binlog_name, router->prevbinlog) != 0)
{
strncpy(router->prevbinlog, router->binlog_name, BINLOG_FNAMELEN);
strcpy(router->prevbinlog, router->binlog_name); // Same size
}
if (router->client)
@ -2584,6 +2777,16 @@ blr_distribute_error_message(ROUTER_INSTANCE *router, char *message, char *state
spinlock_release(&router->lock);
}
/**
* @brief Write data into binlogs (incomplete event)
*
* Writes @c data_len bytes of data from @c buf into the current binlog being processed.
*
* @param router Router instance
* @param data_len Number of bytes to write
* @param buf Pointer where the data is read
* @return Number of bytes written or 0 on error
*/
int
blr_write_data_into_binlog(ROUTER_INSTANCE *router, uint32_t data_len, uint8_t *buf)
{
@ -2773,7 +2976,7 @@ bool blr_send_event(blr_thread_role_t role,
void extract_checksum(ROUTER_INSTANCE* router, uint8_t *cksumptr, uint8_t len)
{
uint8_t *ptr = cksumptr;
while (ptr - cksumptr < len)
while (ptr - cksumptr < len && router->partial_checksum_bytes < MYSQL_CHECKSUM_LEN)
{
router->partial_checksum[router->partial_checksum_bytes] = *ptr;
ptr++;
@ -2792,9 +2995,10 @@ static void blr_terminate_master_replication(ROUTER_INSTANCE* router, uint8_t* p
{
unsigned long mysql_errno = extract_field(ptr + 5, 16);
int msg_len = len - 7 - 6; // msg len is decreased by 7 and 6
char *msg_err = (char *)malloc(msg_len + 1);
char *msg_err = (char *)MXS_MALLOC(msg_len + 1);
MXS_ABORT_IF_NULL(msg_err);
strncpy(msg_err, (char *)ptr + 7 + 6, msg_len);
memcpy(msg_err, (char *)ptr + 7 + 6, msg_len);
*(msg_err + msg_len) = '\0';
spinlock_acquire(&router->lock);
@ -2807,8 +3011,110 @@ static void blr_terminate_master_replication(ROUTER_INSTANCE* router, uint8_t* p
spinlock_release(&router->lock);
free(old_errmsg);
MXS_FREE(old_errmsg);
MXS_ERROR("Error packet in binlog stream.%s @ %lu.",
router->binlog_name, router->current_pos);
}
/**
* Populate a header structure for a replication message from a GWBUF structure with semi-sync enabled.
*
* @param pkt The incoming packet in a GWBUF chain
* @param hdr The packet header to populate
*/
static void
blr_extract_header_semisync(register uint8_t *ptr, register REP_HEADER *hdr)
{
hdr->payload_len = EXTRACT24(ptr);
hdr->seqno = ptr[3];
hdr->ok = ptr[4];
/* Data available after 2 bytes (the 2 semisync bytes) */
hdr->timestamp = EXTRACT32(&ptr[5 + 2]);
hdr->event_type = ptr[9 + 2];
hdr->serverid = EXTRACT32(&ptr[10 + 2]);
hdr->event_size = EXTRACT32(&ptr[14 + 2]);
hdr->next_pos = EXTRACT32(&ptr[18 + 2]);
hdr->flags = EXTRACT16(&ptr[22 + 2]);
}
/**
* Send a MySQL Replication Semi-Sync ACK to the master server.
*
* @param router The router instance.
* @param pos The binlog position for the ACK reply.
* @return 1 if the packect is sent, 0 on errors
*/
static int
blr_send_semisync_ack(ROUTER_INSTANCE *router, uint64_t pos)
{
int seqno = 0;
int semi_sync_flag = BLR_MASTER_SEMI_SYNC_INDICATOR;
GWBUF *buf;
int len;
uint8_t *data;
int binlog_file_len = strlen(router->binlog_name);
/* payload is: 1 byte semi-sync indicator + 8 bytes position + binlog name len */
len = 1 + 8 + binlog_file_len;
/* add network header to size */
if ((buf = gwbuf_alloc(len + 4)) == NULL)
{
return 0;
}
data = GWBUF_DATA(buf);
encode_value(&data[0], len, 24); // Payload length
data[3] = 0; // Sequence ID
data[4] = semi_sync_flag; // Semi-sync indicator
/**
* Next Bytes are: 8 bytes log position + len bin_log filename
*/
/* Position */
encode_value(&data[5], pos, 64);
/* Binlog filename */
memcpy((char *)&data[13], router->binlog_name, binlog_file_len);
router->master->func.write(router->master, buf);
return 1;
}
/**
* Check the master semisync capability.
*
* @param buf The GWBUF data with master reply.
* @return Semisync value: non available, enabled, disabled
*/
static int
blr_get_master_semisync(GWBUF *buf)
{
char *key;
char *val = NULL;
int master_semisync = MASTER_SEMISYNC_NOT_AVAILABLE;
key = blr_extract_column(buf, 1);
if (key && strlen(key))
{
val = blr_extract_column(buf, 2);
}
free(key);
if (val) {
if (strncasecmp(val, "ON", 4) == 0)
master_semisync = MASTER_SEMISYNC_ENABLED;
else
master_semisync = MASTER_SEMISYNC_DISABLED;
}
free(val);
return master_semisync;
}

868
server/modules/routing/binlog/blr_slave.c
View File

File diff suppressed because it is too large Load Diff

196
server/modules/routing/binlog/maxbinlogcheck.c
View File

@ -4,7 +4,7 @@
* Use of this software is governed by the Business Source License included
* in the LICENSE.TXT file and at www.mariadb.com/bsl.
*
* Change Date: 2019-01-01
* Change Date: 2019-07-01
*
* On the date above, in accordance with the Business Source License, use
* of this software will be governed by version 2 or later of the General
@ -21,57 +21,37 @@
* @verbatim
* Revision History
*
* Date Who Description
* Date Who Description
* 24/07/2015 Massimiliano Pinto Initial implementation
* 26/08/2015 Massimiliano Pinto Added mariadb10 option
* for MariaDB 10 binlog compatibility
* Currently MariadDB 10 starting transactions
* are detected checking GTID event
* with flags = 0
*
* for MariaDB 10 binlog compatibility
* Currently MariadDB 10 starting transactions
* are detected checking GTID event
* with flags = 0
* @endverbatim
*/
#include <getopt.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <ctype.h>
#include <time.h>
#include <service.h>
#include <server.h>
#include <router.h>
#include <atomic.h>
#include <spinlock.h>
#include <blr.h>
#include <dcb.h>
#include <spinlock.h>
#include <housekeeper.h>
#include <time.h>
#include <skygw_types.h>
#include <skygw_utils.h>
#include <log_manager.h>
#include <mysql_client_server_protocol.h>
#include <ini.h>
#include <sys/stat.h>
#include <getopt.h>
#include <version.h>
#include <gwdirs.h>
#include <maxscale/alloc.h>
#include <log_manager.h>
#include <blr.h>
extern int blr_read_events_all_events(ROUTER_INSTANCE *router, int fix, int debug);
extern uint32_t extract_field(uint8_t *src, int bits);
static void printVersion(const char *progname);
static void printUsage(const char *progname);
static struct option long_options[] =
{
{"debug", no_argument, 0, 'd'},
{"version", no_argument, 0, 'V'},
{"fix", no_argument, 0, 'f'},
{"mariadb10", no_argument, 0, 'M'},
{"help", no_argument, 0, '?'},
{"debug", no_argument, 0, 'd'},
{"version", no_argument, 0, 'V'},
{"fix", no_argument, 0, 'f'},
{"mariadb10", no_argument, 0, 'M'},
{"help", no_argument, 0, '?'},
{0, 0, 0, 0}
};
@ -85,20 +65,12 @@ maxscale_uptime()
int main(int argc, char **argv)
{
ROUTER_INSTANCE *inst;
int fd;
int ret;
char *ptr;
char path[PATH_MAX + 1] = "";
unsigned long filelen = 0;
struct stat statb;
char c;
int option_index = 0;
int num_args = 0;
int debug_out = 0;
int fix_file = 0;
int mariadb10_compat = 0;
char c;
while ((c = getopt_long(argc, argv, "dVfM?", long_options, &option_index)) >= 0)
{
switch (c)
@ -122,71 +94,70 @@ int main(int argc, char **argv)
}
}
num_args = optind;
int num_args = optind;
if (argv[num_args] == NULL)
{
printf("ERROR: No binlog file was specified.\n");
exit(EXIT_FAILURE);
}
size_t len = strlen(argv[num_args]);
if (len > PATH_MAX)
{
printf("ERROR: The length of the provided path exceeds %d characters.\n", PATH_MAX);
exit(EXIT_FAILURE);
}
char path[PATH_MAX + 1];
strcpy(path, argv[num_args]);
char *name = strrchr(path, '/');
if (name)
{
++name;
len = strlen(name);
}
else
{
name = path;
}
if ((len == 0) || (len > BINLOG_FNAMELEN))
{
printf("ERROR: The length of the binlog filename is 0 or exceeds %d characters.\n",
BINLOG_FNAMELEN);
exit(EXIT_FAILURE);
}
ROUTER_INSTANCE *inst = (ROUTER_INSTANCE*)MXS_CALLOC(1, sizeof(ROUTER_INSTANCE));
if (!inst)
{
exit(EXIT_FAILURE);
}
int fd = open(path, fix_file ? O_RDWR : O_RDONLY, 0666);
if (fd == -1)
{
printf("ERROR: Failed to open binlog file %s: %s.\n",
path, strerror(errno));
MXS_FREE(inst);
exit(EXIT_FAILURE);
}
inst->binlog_fd = fd;
inst->mariadb10_compat = mariadb10_compat;
strcpy(inst->binlog_name, name);
// We ignore potential errors.
mxs_log_init(NULL, NULL, MXS_LOG_TARGET_DEFAULT);
mxs_log_set_augmentation(0);
mxs_log_set_priority_enabled(LOG_DEBUG, debug_out);
if ((inst = calloc(1, sizeof(ROUTER_INSTANCE))) == NULL)
{
MXS_ERROR("Memory allocation failed for ROUTER_INSTANCE");
mxs_log_flush_sync();
mxs_log_finish();
return 1;
}
if (argv[num_args] == NULL)
{
printf("ERROR: No binlog file was specified\n");
exit(EXIT_FAILURE);
}
strncpy(path, argv[num_args], PATH_MAX);
if (fix_file)
{
fd = open(path, O_RDWR, 0666);
}
else
{
fd = open(path, O_RDONLY, 0666);
}
if (fd == -1)
{
MXS_ERROR("Failed to open binlog file %s: %s",
path, strerror(errno));
mxs_log_flush_sync();
mxs_log_finish();
free(inst);
return 1;
}
inst->binlog_fd = fd;
if (mariadb10_compat == 1)
{
inst->mariadb10_compat = 1;
}
ptr = strrchr(path, '/');
if (ptr)
{
strncpy(inst->binlog_name, ptr + 1, BINLOG_FNAMELEN);
}
else
{
strncpy(inst->binlog_name, path, BINLOG_FNAMELEN);
}
MXS_NOTICE("maxbinlogcheck %s", binlog_check_version);
unsigned long filelen = 0;
struct stat statb;
if (fstat(inst->binlog_fd, &statb) == 0)
{
filelen = statb.st_size;
@ -195,19 +166,18 @@ int main(int argc, char **argv)
MXS_NOTICE("Checking %s (%s), size %lu bytes", path, inst->binlog_name, filelen);
/* read binary log */
ret = blr_read_events_all_events(inst, fix_file, debug_out);
close(inst->binlog_fd);
int ret = blr_read_events_all_events(inst, fix_file, debug_out);
mxs_log_flush_sync();
MXS_NOTICE("Check retcode: %i, Binlog Pos = %lu", ret, inst->binlog_position);
close(inst->binlog_fd);
MXS_FREE(inst);
mxs_log_flush_sync();
mxs_log_finish();
free(inst);
return 0;
}
@ -230,10 +200,10 @@ printUsage(const char *progname)
printf("The MaxScale binlog check utility.\n\n");
printf("Usage: %s [-f] [-d] [-v] [<binlog file>]\n\n", progname);
printf(" -f|--fix Fix binlog file, require write permissions (truncate)\n");
printf(" -d|--debug Print debug messages\n");
printf(" -M|--mariadb10 MariaDB 10 binlog compatibility\n");
printf(" -V|--version print version information and exit\n");
printf(" -?|--help Print this help text\n");
printf(" -f|--fix Fix binlog file, require write permissions (truncate)\n");
printf(" -d|--debug Print debug messages\n");
printf(" -M|--mariadb10 MariaDB 10 binlog compatibility\n");
printf(" -V|--version print version information and exit\n");
printf(" -?|--help Print this help text\n");
}

25
server/modules/routing/binlog/test/testbinlog.c
View File

@ -4,7 +4,7 @@
* Use of this software is governed by the Business Source License included
* in the LICENSE.TXT file and at www.mariadb.com/bsl.
*
* Change Date: 2019-01-01
* Change Date: 2019-07-01
*
* On the date above, in accordance with the Business Source License, use
* of this software will be governed by version 2 or later of the General
@ -40,6 +40,7 @@
#include <skygw_utils.h>
#include <log_manager.h>
#include <gwdirs.h>
#include <maxscale/alloc.h>
#include <mysql_client_server_protocol.h>
#include <ini.h>
@ -78,7 +79,9 @@ int main(int argc, char **argv) {
char *roptions;
int tests = 1;
roptions = strdup("server-id=3,heartbeat=200,binlogdir=/not_exists/my_dir,transaction_safety=1,master_version=5.6.99-common,master_hostname=common_server,master_uuid=xxx-fff-cccc-fff,master-id=999");
roptions = MXS_STRDUP_A("server-id=3,heartbeat=200,binlogdir=/not_exists/my_dir,"
"transaction_safety=1,master_version=5.6.99-common,"
"master_hostname=common_server,master_uuid=xxx-fff-cccc-fff,master-id=999");
mxs_log_init(NULL, NULL, MXS_LOG_TARGET_DEFAULT);
@ -87,10 +90,10 @@ int main(int argc, char **argv) {
mxs_log_set_priority_enabled(LOG_NOTICE, false);
mxs_log_set_priority_enabled(LOG_ERR, false);
set_libdir(strdup(".."));
set_libdir(MXS_STRDUP_A(".."));
service = service_alloc("test_service", "binlogrouter");
service->credentials.name = strdup("foo");
service->credentials.authdata = strdup("bar");
service->credentials.name = MXS_STRDUP_A("foo");
service->credentials.authdata = MXS_STRDUP_A("bar");
{
char *lasts;
@ -104,10 +107,6 @@ int main(int argc, char **argv) {
server = server_alloc("_none_", "MySQLBackend", (int)3306);
if (server == NULL) {
if (service->users) {
users_free(service->users);
}
return 1;
}
@ -115,9 +114,7 @@ int main(int argc, char **argv) {
serviceAddBackend(service, server);
}
if ((inst = calloc(1, sizeof(ROUTER_INSTANCE))) == NULL) {
MXS_ERROR("Memory allocation FAILED for ROUTER_INSTANCE");
if ((inst = MXS_CALLOC(1, sizeof(ROUTER_INSTANCE))) == NULL) {
mxs_log_flush_sync();
mxs_log_finish();
@ -131,7 +128,7 @@ int main(int argc, char **argv) {
MXS_NOTICE("testbinlog v1.0");
if (inst->fileroot == NULL)
inst->fileroot = strdup(BINLOG_NAME_ROOT);
inst->fileroot = MXS_STRDUP_A(BINLOG_NAME_ROOT);
if (!inst->current_pos)
inst->current_pos = 4;
@ -572,7 +569,7 @@ int main(int argc, char **argv) {
mxs_log_flush_sync();
mxs_log_finish();
free(inst);
MXS_FREE(inst);
return 0;
}