hcq/MaxScale
hcq/MaxScale
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge remote-tracking branch 'origin/develop' into MXS-105

Browse Source
This commit is contained in:
counterpoint 2015-04-24 15:40:07 +01:00
commit 8b7158afda
20 changed files with 566 additions and 93 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

3
CMakeLists.txt
View File

@ -149,7 +149,6 @@ install(FILES ${ERRMSG} DESTINATION mysql)
install(FILES ${CMAKE_SOURCE_DIR}/COPYRIGHT DESTINATION .)
install(FILES ${CMAKE_SOURCE_DIR}/README DESTINATION .)
install(FILES ${CMAKE_SOURCE_DIR}/LICENSE DESTINATION .)
install(DIRECTORY DESTINATION log)
# Install startup scripts and ldconfig files
if(WITH_SCRIPTS)
@ -212,7 +211,7 @@ if(PACKAGE)
set(CPACK_PACKAGE_DESCRIPTION_FILE ${CMAKE_SOURCE_DIR}/etc/DESCRIPTION)
set(CPACK_PACKAGING_INSTALL_PREFIX "${CMAKE_INSTALL_PREFIX}")
set(CPACK_DEBIAN_PACKAGE_CONTROL_EXTRA "${CMAKE_BINARY_DIR}/postinst;{CMAKE_BINARY_DIR}/postrm")
set(CPACK_RPM_PACKAGE_RELEASE 1)
set(CPACK_RPM_PACKAGE_RELEASE ${MAXSCALE_BUILD_NUMBER})
set(CPACK_RPM_POST_INSTALL_SCRIPT_FILE ${CMAKE_BINARY_DIR}/postinst)
set(CPACK_RPM_POST_UNINSTALL_SCRIPT_FILE ${CMAKE_BINARY_DIR}/postrm)
set(CPACK_RPM_PACKAGE_NAME "maxscale")

4
Documentation/About/About-MaxScale.md
View File

@ -13,8 +13,8 @@ A Google Group exists for MaxScale that can be used to discuss ideas, issues and
Send email to [maxscale@googlegroups.com](mailto:maxscale@googlegroups.com)
or use the [forum](http://groups.google.com/forum/#!forum/maxscale) interface
Bugs can be reported in the MariaDB Corporation bugs database
[bug.mariadb.com](http://bugs.mariadb.com)
Bugs can be reported in the MariaDB Jira
[https://mariadb.atlassian.net](https://mariadb.atlassian.net)
## Installing MaxScale
Information about installing MaxScale, either from a repository or by building from source code, is included in the guide [Getting Started with MaxScale](/Documentation/Getting-Started/Getting-Started-With-MaxScale.md).

11
Documentation/Changelog.md
View File

@ -2,6 +2,17 @@
These are the changes introduced in the next MaxScale version. This is not the official change log and the latest changelog can always be found in here: [MaxScale 1.1 Release Notes](Release-Notes/MaxScale-1.1-Release-Notes.md)
## MaxScale 1.1.1
* Schemarouter now also allows for an upper limit to session commans.
* Schemarouter correctly handles SHOW DATABASES responses that span multiple buffers.
* Schemarouter now allows disabling of the session command history.
* Readwritesplit now allows disabling of the session command history.
* Logfiles have been renamed. The log names are now named error.log, messages.log, trace.log and debug.log.
## MaxScale 1.1
**NOTE:** MaxScale default installation directory has changed to `/usr/local/mariadb-maxscale` and the default password for MaxAdmin is now ´mariadb´.
* New modules added

2
Documentation/Getting-Started/Configuration-Guide.md
View File

@ -831,6 +831,8 @@ In above-mentioned case the user-defined variable would only be updated in the m
When a limitation is set, it effectively creates a cap on the session's memory consumption. This might be useful if connection pooling is used and the sessions use large amounts of session commands.
`disable_sescmd_history=true|false` disables the session command history. This way nothing is stored and if a slave server fails and a new one is taken in its stead, the session on that server will be in an inconsistent state compared to the master server. Disabling session command history will allow connection pooling without causing a constant growth in the memory consumption.
An example of Read/Write Split router configuration :
```

6
Documentation/routers/schemarouter/SchemaRouter.md
View File

@ -42,6 +42,12 @@ GRANT SELECT,USAGE ON shard.* TO 'john'@'%';
This would in effect allow the user 'john' to only see the database 'shard' on this server. Take notice that these grants are matched against MaxScale's hostname instead of the client's hostname. Only user authentication uses the client's hostname and all other grants use MaxScale's hostname.
The schemarouter supports the following router options:
|option |parameter |description|
---------------------------------------------
|max_sescmd_hitory |<int> |Set a limit on the number of session modifying commands a session can execute. This sets an effective cap on the memory consupmtion of the session.|
|disable_sescmd_history|<boolean>|Disable the session command history. This will prevent growing memory consumption of a long-running session and allows pooled connections to MaxScale to be used. The drawback of this is the fact that if a server goes down, the session state will not be consistent anymore.|
## Limitations
The schemarouter router currently has some limitations due to the nature of the sharding implementation and the way the session variables are detected and routed. Here is a list of the current limitations.

37
README.md Normal file
View File

@ -0,0 +1,37 @@
# MaxScale by MariaDB Corporation
The MariaDB Corporation MaxScale is an intelligent proxy that allows forwarding of
database statements to one or more database servers using complex rules,
a semantic understanding of the database statements and the roles of
the various servers within the backend cluster of databases.
MaxScale is designed to provide load balancing and high availability
functionality transparently to the applications. In addition it provides
a highly scalable and flexible architecture, with plugin components to
support different protocols and routing decisions.
MaxScale is implemented in C and makes extensive use of the
asynchronous I/O capabilities of the Linux operating system. The epoll
system is used to provide the event driven framework for the input and
output via sockets.
The protocols are implemented as external shared object modules which
can be loaded at runtime. These modules support a fixed interface,
communicating the entries points via a structure consisting of a set of
function pointers. This structure is called the "module object".
The code that routes the queries to the database servers is also loaded
as external shared objects and are referred to as routing modules.
An Google Group exists for MaxScale that can be used to discuss ideas,
issues and communicate with the MaxScale community.
Send email to maxscale@googlegroups.com
or use the [forum](http://groups.google.com/forum/#!forum/maxscale) interface
Bugs can be reported in the MariaDB Corporation bugs database
[https://mariadb.atlassian.net](https://mariadb.atlassian.net) under project MXS.
# Documentation
For information about installing and using MaxScale, please refer to the
[documentation](Documentation/Documentation-Contents.md).

5
client/CMakeLists.txt
View File

@ -1,7 +1,10 @@
add_executable(maxadmin maxadmin.c)
find_library(HIST edit)
if(HIST)
message(STATUS "Building MaxAdmin with editline: ${HIST}")
add_definitions(-DHISTORY)
target_link_libraries(maxadmin ${HIST})
else()
message(STATUS "Could not find editline library. MaxAdmin will be built without it.")
endif()
install(TARGETS maxadmin DESTINATION bin)
install(TARGETS maxadmin DESTINATION bin)

32
client/maxadmin.c
View File

@ -63,7 +63,7 @@ static void DoSource(int so, char *cmd);
static void DoUsage();
static int isquit(char *buf);
static void PrintVersion(const char *progname);
static void read_inifile(char **hostname, char **port, char **user, char **passwd);
static void read_inifile(char **hostname, char **port, char **user, char **passwd,int*);
#ifdef HISTORY
static char *
@ -82,6 +82,7 @@ static struct option long_options[] = {
{"port", required_argument, 0, 'P'},
{"version", no_argument, 0, 'v'},
{"help", no_argument, 0, '?'},
{"emacs", no_argument, 0, 'e'},
{0, 0, 0, 0}
};
@ -94,6 +95,9 @@ static struct option long_options[] = {
int
main(int argc, char **argv)
{
const char* vi = "vi";
const char* emacs = "emacs";
int i, num, rv;
#ifdef HISTORY
char *buf;
@ -109,13 +113,14 @@ char *hostname = "localhost";
char *port = "6603";
char *user = "admin";
char *passwd = NULL;
int use_emacs = 0;
int so;
int option_index = 0;
char c;
read_inifile(&hostname, &port, &user, &passwd);
read_inifile(&hostname, &port, &user, &passwd,&use_emacs);
while ((c = getopt_long(argc, argv, "h:p:P:u:v?",
while ((c = getopt_long(argc, argv, "h:p:P:u:v?e",
long_options, &option_index))
>= 0)
{
@ -135,6 +140,9 @@ char c;
case 'v':
PrintVersion(*argv);
exit(EXIT_SUCCESS);
case 'e':
use_emacs = 1;
break;
case '?':
DoUsage(*argv);
exit(optopt ? EXIT_FAILURE : EXIT_SUCCESS);
@ -224,7 +232,10 @@ char c;
/* Initialize editline */
el = el_init(*argv, stdin, stdout, stderr);
el_set(el, EL_EDITOR, "vi"); /* Default editor is vi */
if(use_emacs)
el_set(el, EL_EDITOR, emacs); /** Editor is emacs */
else
el_set(el, EL_EDITOR, vi); /* Default editor is vi */
el_set(el, EL_SIGNAL, 1); /* Handle signals gracefully */
el_set(el, EL_PROMPT, prompt);/* Set the prompt function */
@ -592,7 +603,7 @@ char *ptr = str + strlen(str);
* @param passwd Pointer to the password to be updated
*/
static void
read_inifile(char **hostname, char **port, char **user, char **passwd)
read_inifile(char **hostname, char **port, char **user, char **passwd, int* editor)
{
char pathname[400];
char *home, *brkt;
@ -624,6 +635,17 @@ char line[400];
*user = strdup(value);
else if (strcmp(name, "passwd") == 0)
*passwd = strdup(value);
else if (strcmp(name, "editor") == 0)
{
if(strcmp(value,"vi") == 0)
*editor = 0;
else if(strcmp(value,"emacs") == 0)
*editor = 1;
else
fprintf(stderr, "WARNING: Unrecognised "
"parameter '%s=%s' in .maxadmin file\n", name, value);
}
else
{
fprintf(stderr, "WARNING: Unrecognised "

6
log_manager/log_manager.cc
View File

@ -1333,9 +1333,9 @@ static bool logfile_set_enabled(
CHK_LOGFILE(lf);
if (val) {
logstr = strdup("---\tLogging is enabled\t--");
logstr = strdup("---\tLogging to file is enabled\t--");
} else {
logstr = strdup("---\tLogging is disabled\t--");
logstr = strdup("---\tLogging to file is disabled\t--");
}
oldval = lf->lf_enabled;
lf->lf_enabled = val;
@ -3146,4 +3146,4 @@ void logmanager_enable_syslog(int val)
void logmanager_enable_maxscalelog(int val)
{
do_maxscalelog = val;
}
}

4
macros.cmake
View File

@ -7,13 +7,15 @@ endfunction()
macro(set_maxscale_version)
#MaxScale version number
# MaxScale version number
set(MAXSCALE_VERSION_MAJOR "1")
set(MAXSCALE_VERSION_MINOR "1")
set(MAXSCALE_VERSION_PATCH "0")
set(MAXSCALE_VERSION_NUMERIC "${MAXSCALE_VERSION_MAJOR}.${MAXSCALE_VERSION_MINOR}.${MAXSCALE_VERSION_PATCH}")
set(MAXSCALE_VERSION "${MAXSCALE_VERSION_MAJOR}.${MAXSCALE_VERSION_MINOR}.${MAXSCALE_VERSION_PATCH}")
# This should be incremented each time a package is rebuilt
set(MAXSCALE_BUILD_NUMBER 2)
endmacro()
macro(set_variables)

6
server/MaxScale_template.cnf
View File

@ -2,10 +2,14 @@
#
# Global parameters
#
# Number of worker threads in MaxScale
# Number of worker threads in MaxScale.
#
# threads=<number of threads>
#
# Directory for the MaxScale log files. Default is /var/log/maxscale/.
#
# logdir=<path to directory>
#
# Enabled logfiles. The message log is enabled by default and
# the error log is always enabled.
#

4
server/core/config.c
View File

@ -908,6 +908,10 @@ int error_count = 0;
/* set monitor interval */
if (interval > 0)
monitorSetInterval(obj->element, interval);
else
skygw_log_write(LOGFILE_ERROR,"Warning: Monitor '%s' "
"missing monitor_interval parameter, "
"default value of 10000 miliseconds.",obj->object);
/* set timeouts */
if (connect_timeout > 0)

163
server/core/gateway.c
View File

@ -74,6 +74,8 @@
#include <execinfo.h>
#include <ini.h>
/** for procname */
#if !defined(_GNU_SOURCE)
# define _GNU_SOURCE
@ -113,6 +115,11 @@ const int num_elements = (sizeof(server_options) / sizeof(char *)) - 1;
const char* default_cnf_fname = "etc/MaxScale.cnf";
const char* default_configdir = "/etc/";
const char* default_logdir = "/var/log/maxscale/";
const char* default_libdir = "/lib64/maxscale/lib/";
const char* default_moddir = "/lib64/maxscale/modules/";
static char* server_groups[] = {
"embedded",
"server",
@ -129,6 +136,10 @@ static char datadir[PATH_MAX+1] = "";
/* The data directory we created for this gateway instance */
static char pidfile[PATH_MAX+1] = "";
static char* configdir = NULL;
static char* logdir = NULL;
static char* libdir = NULL;
static char* moddir = NULL;
/**
* exit flag for log flusher.
*/
@ -150,13 +161,14 @@ static struct option long_options[] = {
{"config", required_argument, 0, 'f'},
{"nodaemon", no_argument, 0, 'd'},
{"log", required_argument, 0, 'l'},
{"logdir", required_argument, 0, 'L'},
{"syslog", required_argument, 0, 's'},
{"maxscalelog", required_argument, 0, 'S'},
{"version", no_argument, 0, 'v'},
{"help", no_argument, 0, '?'},
{0, 0, 0, 0}
};
static int cnf_preparser(void* data, const char* section, const char* name, const char* value);
static void log_flush_shutdown(void);
static void log_flush_cb(void* arg);
static int write_pid_file(char *); /* write MaxScale pidfile */
@ -168,6 +180,7 @@ static void write_footer(void);
static int ntfw_cb(const char*, const struct stat*, int, struct FTW*);
static bool file_is_readable(char* absolute_pathname);
static bool file_is_writable(char* absolute_pathname);
bool handle_path_arg(char** dest, char* path, char* arg, bool rd, bool wr);
static void usage(void);
static char* get_expanded_pathname(
char** abs_path,
@ -186,7 +199,7 @@ static bool resolve_maxscale_conf_fname(
static bool resolve_maxscale_homedir(
char** p_home_dir);
static char* check_dir_access(char* dirname);
static char* check_dir_access(char* dirname,bool,bool);
/**
* Handler for SIGHUP signal. Reload the configuration for the
@ -728,8 +741,9 @@ return_succp:
* read or write is not permitted.
*/
static char* check_dir_access(
char* dirname)
char* dirname, bool rd, bool wr)
{
char errbuf[PATH_MAX*2];
char* errstr = NULL;
if (dirname == NULL)
@ -737,18 +751,27 @@ static char* check_dir_access(
errstr = strdup("Directory argument is NULL");
goto retblock;
}
if (!file_is_readable(dirname))
if(access(dirname,F_OK) != 0)
{
errstr = strdup("MaxScale doesn't have read permission "
"to MAXSCALE_HOME.");
sprintf(errbuf,"Can't access '%s'.",dirname);
errstr = strdup(errbuf);
goto retblock;
}
if (rd && !file_is_readable(dirname))
{
sprintf(errbuf,"MaxScale doesn't have read permission "
"to '%s'.",dirname);
errstr = strdup(errbuf);
goto retblock;
}
if (!file_is_writable(dirname))
if (wr && !file_is_writable(dirname))
{
errstr = strdup("MaxScale doesn't have write permission "
"to MAXSCALE_HOME. Exiting.");
sprintf(errbuf,"MaxScale doesn't have write permission "
"to '%s'.",dirname);
errstr = strdup(errbuf);
goto retblock;
}
@ -998,6 +1021,8 @@ static void usage(void)
" (default: $MAXSCALE_HOME/etc/MaxScale.cnf)\n"
" -l|--log=... log to file or shared memory\n"
" -lfile or -lshm - defaults to shared memory\n"
" -L|--logdir=... path to log file directory\n"
" (default: /var/log/maxscale)\n"
" -s|--syslog= log messages to syslog."
" True or false - defaults to true\n"
" -S|--maxscalelog= log messages to MaxScale log."
@ -1062,6 +1087,8 @@ int main(int argc, char **argv)
char* cnf_file_path = NULL; /*< conf file, to be freed */
char* cnf_file_arg = NULL; /*< conf filename from cmd-line arg */
void* log_flush_thr = NULL;
char* tmp_path;
char* tmp_var;
int option_index;
int logtofile = 0; /* Use shared memory or file */
int syslog_enabled = 1; /** Log to syslog */
@ -1105,7 +1132,8 @@ int main(int argc, char **argv)
goto return_main;
}
}
while ((opt = getopt_long(argc, argv, "dc:f:l:vs:S:?",
while ((opt = getopt_long(argc, argv, "dc:f:l:vs:S:?L:",
long_options, &option_index)) != -1)
{
bool succp = true;
@ -1210,6 +1238,14 @@ int main(int argc, char **argv)
succp = false;
}
break;
case 'L':
if(handle_path_arg(&tmp_path,optarg,NULL,true,false))
{
logdir = tmp_path;
}
break;
case 'S':
if(strstr(optarg,"="))
{
@ -1532,7 +1568,7 @@ int main(int argc, char **argv)
char* log_context = strdup("Home directory command-line argument");
char* errstr;
errstr = check_dir_access(home_dir);
errstr = check_dir_access(home_dir,true,true);
if (errstr != NULL)
{
@ -1566,6 +1602,12 @@ int main(int argc, char **argv)
free(log_context);
}
char pbuf[PATH_MAX];
sprintf(pbuf,"%s/etc/MaxScale.cnf",home_dir);
ini_parse(pbuf,cnf_preparser,NULL);
/**
* Init Log Manager for MaxScale.
* If $MAXSCALE_HOME is set then write the logs into $MAXSCALE_HOME/log.
@ -1577,22 +1619,27 @@ int main(int argc, char **argv)
char buf[1024];
char *argv[8];
bool succp;
/** Set log directory under $MAXSCALE_HOME/log */
sprintf(buf, "%s/log", home_dir);
if(mkdir(buf, 0777) != 0)
/** Use default log directory /var/log/maxscale/ */
if(logdir == NULL)
{
if(errno != EEXIST)
if(access(default_logdir,F_OK) != 0)
{
if(mkdir(logdir,0555) != 0)
{
fprintf(stderr,
"Error: Cannot create log directory: %s\n",
buf);
goto return_main;
fprintf(stderr,
"Error: Cannot create log directory: %s\n",
default_logdir);
goto return_main;
}
}
logdir = strdup(default_logdir);
}
argv[0] = "MaxScale";
argv[1] = "-j";
argv[2] = buf;
argv[2] = logdir;
if(!syslog_enabled)
{
@ -1675,11 +1722,11 @@ int main(int argc, char **argv)
fprintf(stderr,
"Home directory : %s"
"\nConfiguration file : %s"
"\nLog directory : %s/log"
"\nLog directory : %s"
"\nData directory : %s\n\n",
home_dir,
cnf_file_path,
home_dir,
logdir,
datadir);
}
LOGIF(LM, (skygw_log_write_flush(
@ -2003,3 +2050,73 @@ MaxScaleUptime()
{
return time(0) - MaxScaleStarted;
}
bool handle_path_arg(char** dest, char* path, char* arg, bool rd, bool wr)
{
char pathbuffer[PATH_MAX+2];
char* errstr;
bool rval = false;
if(path == NULL && arg == NULL)
return rval;
if(path)
{
snprintf(pathbuffer,PATH_MAX,"%s",path);
if(pathbuffer[strlen(path) - 1] != '/')
{
strcat(pathbuffer,"/");
}
if(arg && strlen(pathbuffer) + strlen(arg) + 1 < PATH_MAX)
strcat(pathbuffer,arg);
if((errstr = check_dir_access(pathbuffer,rd,wr)) == NULL)
{
*dest = strdup(pathbuffer);
rval = true;
}
else
{
fprintf(stderr,"%s\n",errstr);
free(errstr);
errstr = NULL;
}
}
return rval;
}
/**
* Pre-parse the MaxScale.cnf for config, log and module directories.
* @param data Parameter passed by inih
* @param section Section name
* @param name Parameter name
* @param value Parameter value
* @return 1 in all cases
*/
static int cnf_preparser(void* data, const char* section, const char* name, const char* value)
{
char pathbuffer[PATH_MAX];
char* errstr;
if(strcasecmp(section,"maxscale") == 0)
{
if(strcmp(name, "logdir") == 0)
{
/** logdir is only NULL if no command line parameter was given */
if(logdir == NULL)
handle_path_arg(&logdir,(char*)value,NULL,true,true);
}
else if(strcmp(name, "moddir") == 0)
{
handle_path_arg(&moddir,(char*)value,NULL,true,false);
}
else if(strcmp(name, "libdir") == 0)
{
handle_path_arg(&libdir,(char*)value,NULL,true,false);
}
}
return 1;
}

12
server/core/modutil.c
View File

@ -549,7 +549,6 @@ GWBUF* modutil_get_complete_packets(GWBUF** p_readbuf)
packet->next = NULL;
*p_readbuf = packet;
ptr = (uint8_t*)packet->start;
end = (uint8_t*)packet->end;
len = gw_mysql_get_byte3(ptr) + 4;
blen = gwbuf_length(packet);
@ -578,18 +577,13 @@ GWBUF* modutil_get_complete_packets(GWBUF** p_readbuf)
}
/** The next packet is a partial, split into complete and partial packets */
if((buff = gwbuf_alloc(total)) == NULL)
if((buff = gwbuf_clone_portion(packet,0,total)) == NULL)
{
skygw_log_write(LOGFILE_ERROR,
"Error: Failed to allocate new buffer "
" of %d bytes while splitting buffer"
" into complete packets.",
total);
"Error: Failed to partially clone buffer.");
return NULL;
}
buff->next = NULL;
gwbuf_set_type(buff,GWBUF_TYPE_MYSQL);
memcpy(buff->start,packet->start,total);
gwbuf_consume(packet,total);
return buff;
}

3
server/inih/CMakeLists.txt
View File

@ -1 +1,2 @@
add_library(inih ini.c)
add_library(inih ini.c)
target_compile_definitions(inih PUBLIC INI_MAX_LINE=1024)

5
server/modules/include/readwritesplit.h
View File

@ -150,6 +150,7 @@ typedef struct mysql_sescmd_st {
unsigned char reply_cmd; /*< The reply command. One of OK, ERR, RESULTSET or
* LOCAL_INFILE. Slave servers are compared to this
* when they return session command replies.*/
int position; /*< Position of this command */
#if defined(SS_DEBUG)
skygw_chk_t my_sescmd_chk_tail;
#endif
@ -184,6 +185,7 @@ typedef struct sescmd_cursor_st {
rses_property_t** scmd_cur_ptr_property; /*< address of pointer to owner property */
mysql_sescmd_t* scmd_cur_cmd; /*< pointer to current session command */
bool scmd_cur_active; /*< true if command is being executed */
int position; /*< Position of this cursor */
#if defined(SS_DEBUG)
skygw_chk_t scmd_cur_chk_tail;
#endif
@ -248,6 +250,8 @@ typedef struct rwsplit_config_st {
int rw_max_slave_replication_lag;
target_t rw_use_sql_variables_in;
int rw_max_sescmd_history_size;
bool disable_sescmd_hist;
bool disable_slave_recovery;
} rwsplit_config_t;
@ -291,6 +295,7 @@ struct router_client_session {
bool rses_autocommit_enabled;
bool rses_transaction_active;
DCB* client_dcb;
int pos_generator;
#if defined(PREP_STMT_CACHING)
HASHTABLE* rses_prep_stmt[2];
#endif

44
server/modules/include/schemarouter.h
View File

@ -136,6 +136,7 @@ typedef struct mysql_sescmd_st {
GWBUF* my_sescmd_buf; /*< Query buffer */
unsigned char my_sescmd_packet_type;/*< Packet type */
bool my_sescmd_is_replied; /*< Is cmd replied to client */
int position; /*< Position of this command */
#if defined(SS_DEBUG)
skygw_chk_t my_sescmd_chk_tail;
#endif
@ -170,6 +171,7 @@ typedef struct sescmd_cursor_st {
rses_property_t** scmd_cur_ptr_property; /*< address of pointer to owner property */
mysql_sescmd_t* scmd_cur_cmd; /*< pointer to current session command */
bool scmd_cur_active; /*< true if command is being executed */
int position; /*< Position of this cursor */
#if defined(SS_DEBUG)
skygw_chk_t scmd_cur_chk_tail;
#endif
@ -221,6 +223,7 @@ typedef struct backend_ref_st {
DCB* bref_dcb; /*< Backend DCB */
bref_state_t bref_state; /*< State of the backend */
bool bref_mapped; /*< Whether the backend has been mapped */
bool last_sescmd_replied;
int bref_num_result_wait; /*< Number of not yet received results */
sescmd_cursor_t bref_sescmd_cur; /*< Session command cursor */
GWBUF* bref_pending_cmd; /*< For stmt which can't be routed due active sescmd execution */
@ -235,9 +238,25 @@ typedef struct backend_ref_st {
typedef struct schemarouter_config_st {
int rw_max_slave_conn_percent;
int rw_max_slave_conn_count;
target_t rw_use_sql_variables_in;
target_t rw_use_sql_variables_in;
int max_sescmd_hist;
bool disable_sescmd_hist;
} schemarouter_config_t;
/**
* The statistics for this router instance
*/
typedef struct {
int n_queries; /*< Number of queries forwarded */
int n_sescmd; /*< Number of session commands */
int longest_sescmd; /*< Longest chain of stored session commands */
int n_hist_exceeded;/*< Number of sessions that exceeded session
* command history limit */
int sessions;
double ses_longest; /*< Longest session */
double ses_shortest; /*< Shortest session */
double ses_average; /*< Average session length */
} ROUTER_STATS;
/**
* The client session structure used within this router.
@ -268,22 +287,14 @@ struct router_client_session {
GWBUF* queue; /*< Query that was received before the session was ready */
DCB* dcb_route; /*< Internal DCB used to trigger re-routing of buffers */
DCB* dcb_reply; /*< Internal DCB used to send replies to the client */
ROUTER_STATS stats; /*< Statistics for this router */
int n_sescmd;
int pos_generator;
#if defined(SS_DEBUG)
skygw_chk_t rses_chk_tail;
#endif
};
/**
* The statistics for this router instance
*/
typedef struct {
int n_sessions; /*< Number sessions created */
int n_queries; /*< Number of queries forwarded */
int n_master; /*< Number of stmts sent to master */
int n_slave; /*< Number of stmts sent to slave */
int n_all; /*< Number of stmts sent to all */
} ROUTER_STATS;
/**
* The per instance data for the router.
@ -301,15 +312,10 @@ typedef struct router_instance {
ROUTER_STATS stats; /*< Statistics for this router */
struct router_instance* next; /*< Next router on the list */
bool available_slaves; /*< The router has some slaves available */
//HASHTABLE* dbnames_hash; /** Hashtable containing the database names and where to find them */
//char** ignore_list;
} ROUTER_INSTANCE;
#define BACKEND_TYPE(b) (SERVER_IS_MASTER((b)->backend_server) ? BE_MASTER : \
(SERVER_IS_SLAVE((b)->backend_server) ? BE_SLAVE : BE_UNDEFINED));
#if 0
void* dbnames_hash_init(ROUTER_INSTANCE* inst,BACKEND** backends);
bool update_dbnames_hash(ROUTER_INSTANCE* inst,BACKEND** backends, HASHTABLE* hashtable);
#endif
#endif /*< _SCHEMAROUTER_H */

35
server/modules/routing/binlog/blr_file.c
View File

@ -164,6 +164,24 @@ blr_file_rotate(ROUTER_INSTANCE *router, char *file, uint64_t pos)
}
/**
* binlog files need an initial 4 magic bytes at the start. blr_file_add_magic()
* adds them.
*
* @param router The router instance
* @param fd file descriptor to the open binlog file
* @return Nothing
*/
static void
blr_file_add_magic(ROUTER_INSTANCE *router, int fd)
{
unsigned char magic[] = BINLOG_MAGIC;
write(fd, magic, 4);
router->binlog_position = 4; /* Initial position after the magic number */
}
/**
* Create a new binlog file for the router to use.
*
@ -176,7 +194,6 @@ blr_file_create(ROUTER_INSTANCE *router, char *file)
{
char path[1024];
int fd;
unsigned char magic[] = BINLOG_MAGIC;
strcpy(path, router->binlogdir);
strcat(path, "/");
@ -184,7 +201,7 @@ unsigned char magic[] = BINLOG_MAGIC;
if ((fd = open(path, O_RDWR|O_CREAT, 0666)) != -1)
{
write(fd, magic, 4);
blr_file_add_magic(router,fd);
}
else
{
@ -197,7 +214,7 @@ unsigned char magic[] = BINLOG_MAGIC;
close(router->binlog_fd);
spinlock_acquire(&router->binlog_lock);
strncpy(router->binlog_name, file,BINLOG_FNAMELEN);
router->binlog_position = 4; /* Initial position after the magic number */
blr_file_add_magic(router, fd);
spinlock_release(&router->binlog_lock);
router->binlog_fd = fd;
return 1;
@ -232,6 +249,18 @@ int fd;
spinlock_acquire(&router->binlog_lock);
strncpy(router->binlog_name, file,BINLOG_FNAMELEN);
router->binlog_position = lseek(fd, 0L, SEEK_END);
if (router->binlog_position < 4) {
if (router->binlog_position == 0) {
blr_file_add_magic(router, fd);
} else {
/* If for any reason the file's length is between 1 and 3 bytes
* then report an error. */
LOGIF(LE, (skygw_log_write(LOGFILE_ERROR,
"%s: binlog file %s has an invalid length %d.",
router->service->name, path, router->binlog_position)));
return;
}
}
spinlock_release(&router->binlog_lock);
router->binlog_fd = fd;
}

85
server/modules/routing/readwritesplit/readwritesplit.c
View File

@ -628,7 +628,7 @@ createInstance(SERVICE *service, char **options)
* If server weighting has been defined calculate the percentage
* of load that will be sent to each server. This is only used for
* calculating the least connections, either globally or within a
* service, or the numebr of current operations on a server.
* service, or the number of current operations on a server.
*/
if ((weightby = serviceGetWeightingParameter(service)) != NULL)
{
@ -698,6 +698,13 @@ createInstance(SERVICE *service, char **options)
{
rwsplit_process_router_options(router, options);
}
/** These options cancel each other out */
if(router->rwsplit_config.disable_sescmd_hist && router->rwsplit_config.rw_max_sescmd_history_size > 0)
{
router->rwsplit_config.rw_max_sescmd_history_size = 0;
}
/**
* Set default value for max_slave_connections and for slave selection
* criteria. If parameter is set in config file max_slave_connections
@ -807,7 +814,7 @@ static void* newSession(
rwsplit_process_router_options(router, router->service->routerOptions);
}
/** Copy config struct from router instance */
client_rses->rses_config = router->rwsplit_config;
memcpy(&client_rses->rses_config,&router->rwsplit_config,sizeof(rwsplit_config_t));
spinlock_release(&router->lock);
/**
@ -2782,16 +2789,16 @@ static void clientReply (
bool rconn = false;
writebuf = sescmd_cursor_process_replies(writebuf, bref, &rconn);
if(rconn)
if(rconn && !router_inst->rwsplit_config.disable_slave_recovery)
{
select_connect_backend_servers(&router_cli_ses->rses_master_ref,
router_cli_ses->rses_backend_ref,
router_cli_ses->rses_nbackends,
router_cli_ses->rses_config.rw_max_slave_conn_count,
router_cli_ses->rses_config.rw_max_slave_replication_lag,
router_cli_ses->rses_config.rw_slave_select_criteria,
router_cli_ses->rses_master_ref->bref_dcb->session,
router_cli_ses->router);
select_connect_backend_servers(&router_cli_ses->rses_master_ref,
router_cli_ses->rses_backend_ref,
router_cli_ses->rses_nbackends,
router_cli_ses->rses_config.rw_max_slave_conn_count,
router_cli_ses->rses_config.rw_max_slave_replication_lag,
router_cli_ses->rses_config.rw_slave_select_criteria,
router_cli_ses->rses_master_ref->bref_dcb->session,
router_cli_ses->router);
}
}
/**
@ -3667,7 +3674,8 @@ static mysql_sescmd_t* mysql_sescmd_init (
/** Set session command buffer */
sescmd->my_sescmd_buf = sescmd_buf;
sescmd->my_sescmd_packet_type = packet_type;
sescmd->position = atomic_add(&rses->pos_generator,1);
return sescmd;
}
@ -3724,6 +3732,7 @@ static GWBUF* sescmd_cursor_process_replies(
while (scmd != NULL && replybuf != NULL)
{
bref->reply_cmd = *((unsigned char*)replybuf->start + 4);
scur->position = scmd->position;
/** Faster backend has already responded to client : discard */
if (scmd->my_sescmd_is_replied)
{
@ -4361,6 +4370,43 @@ static bool route_session_write(
goto return_succp;
}
if(router_cli_ses->rses_config.disable_sescmd_hist)
{
rses_property_t *prop, *tmp;
backend_ref_t* bref;
bool conflict;
prop = router_cli_ses->rses_properties[RSES_PROP_TYPE_SESCMD];
while(prop)
{
conflict = false;
for(i = 0;i<router_cli_ses->rses_nbackends;i++)
{
bref = &backend_ref[i];
if(BREF_IS_IN_USE(bref))
{
if(bref->bref_sescmd_cur.position <= prop->rses_prop_data.sescmd.position)
{
conflict = true;
break;
}
}
}
if(conflict)
{
break;
}
tmp = prop;
router_cli_ses->rses_properties[RSES_PROP_TYPE_SESCMD] = prop->rses_prop_next;
rses_property_done(tmp);
prop = router_cli_ses->rses_properties[RSES_PROP_TYPE_SESCMD];
}
}
/**
* Additional reference is created to querybuf to
* prevent it from being released before properties
@ -4538,6 +4584,14 @@ static void rwsplit_process_router_options(
{
router->rwsplit_config.rw_max_sescmd_history_size = atoi(value);
}
else if(strcmp(options[i],"disable_sescmd_history") == 0)
{
router->rwsplit_config.disable_sescmd_hist = config_truth_value(value);
}
else if(strcmp(options[i],"disable_slave_recovery") == 0)
{
router->rwsplit_config.disable_slave_recovery = config_truth_value(value);
}
}
} /*< for */
}
@ -4780,6 +4834,12 @@ static bool handle_error_new_connection(
* Try to get replacement slave or at least the minimum
* number of slave connections for router session.
*/
if(inst->rwsplit_config.disable_slave_recovery)
{
succp = have_enough_servers(&rses,1,router_nservers,inst) ? true : false;
}
else
{
succp = select_connect_backend_servers(
&rses->rses_master_ref,
rses->rses_backend_ref,
@ -4789,6 +4849,7 @@ static bool handle_error_new_connection(
rses->rses_config.rw_slave_select_criteria,
ses,
inst);
}
return_succp:
return succp;

192
server/modules/routing/schemarouter/schemarouter.c
View File

@ -716,6 +716,13 @@ createInstance(SERVICE *service, char **options)
return NULL;
}
router->service = service;
router->schemarouter_config.max_sescmd_hist = 0;
router->stats.longest_sescmd = 0;
router->stats.n_hist_exceeded = 0;
router->stats.n_queries = 0;
router->stats.n_sescmd = 0;
router->stats.ses_longest = 0;
router->stats.ses_shortest = (double)((unsigned long)(~0));
spinlock_init(&router->lock);
/** Calculate number of servers */
@ -730,6 +737,45 @@ createInstance(SERVICE *service, char **options)
service->users_from_all = true;
}
bool failure = false;
for(i=0;options && options[i];i++)
{
char* value;
if((value = strchr(options[i],'=')) == NULL)
{
skygw_log_write(LOGFILE_ERROR,"Error: Unknown router options for Schemarouter: %s",options[i]);
failure = true;
break;
}
*value = '\0';
value++;
if(strcmp(options[i],"max_sescmd_history") == 0)
{
router->schemarouter_config.max_sescmd_hist = atoi(value);
}
else if(strcmp(options[i],"disable_sescmd_history") == 0)
{
router->schemarouter_config.disable_sescmd_hist = config_truth_value(value);
}
else
{
skygw_log_write(LOGFILE_ERROR,"Error: Unknown router options for Schemarouter: %s",options[i]);
failure = true;
break;
}
}
/** Setting a limit to the history size is not needed if it is disabled.*/
if(router->schemarouter_config.disable_sescmd_hist && router->schemarouter_config.max_sescmd_hist > 0)
router->schemarouter_config.max_sescmd_hist = 0;
if(failure)
{
free(router);
return NULL;
}
while (server != NULL)
{
nservers++;
@ -759,6 +805,7 @@ createInstance(SERVICE *service, char **options)
router->servers[nservers]->backend_conn_count = 0;
router->servers[nservers]->weight = 1;
router->servers[nservers]->be_valid = false;
router->servers[nservers]->stats.queries = 0;
if(server->server->monuser == NULL && service->credentials.name != NULL)
{
router->servers[nservers]->backend_server->monuser =
@ -887,7 +934,8 @@ static void* newSession(
client_rses->dcb_reply->func.read = internalReply;
client_rses->dcb_reply->state = DCB_STATE_POLLING;
client_rses->dcb_reply->session = session;
memcpy(&client_rses->rses_config,&router->schemarouter_config,sizeof(schemarouter_config_t));
client_rses->n_sescmd = 0;
client_rses->dcb_route = dcb_alloc(DCB_ROLE_REQUEST_HANDLER);
client_rses->dcb_route->func.read = internalRoute;
client_rses->dcb_route->state = DCB_STATE_POLLING;
@ -988,7 +1036,6 @@ static void* newSession(
client_rses->rses_capabilities = RCAP_TYPE_STMT_INPUT;
client_rses->rses_backend_ref = backend_ref;
client_rses->rses_nbackends = router_nservers; /*< # of backend servers */
router->stats.n_sessions += 1;
if (!(succp = rses_begin_locked_router_action(client_rses)))
{
@ -1008,7 +1055,8 @@ static void* newSession(
rses_end_locked_router_action(client_rses);
atomic_add(&router->stats.sessions, 1);
/**
* Version is bigger than zero once initialized.
@ -1047,6 +1095,7 @@ static void closeSession(
void* router_session)
{
ROUTER_CLIENT_SES* router_cli_ses;
ROUTER_INSTANCE* inst;
backend_ref_t* backend_ref;
LOGIF(LD, (skygw_log_write(LOGFILE_DEBUG,
@ -1063,7 +1112,8 @@ static void closeSession(
}
router_cli_ses = (ROUTER_CLIENT_SES *)router_session;
CHK_CLIENT_RSES(router_cli_ses);
inst = router_cli_ses->router;
backend_ref = router_cli_ses->rses_backend_ref;
/**
* Lock router client session for secure read and update.
@ -1118,9 +1168,24 @@ static void closeSession(
router_cli_ses->dcb_route->session = NULL;
dcb_close(router_cli_ses->dcb_reply);
dcb_close(router_cli_ses->dcb_route);
/** Unlock */
rses_end_locked_router_action(router_cli_ses);
rses_end_locked_router_action(router_cli_ses);
spinlock_acquire(&inst->lock);
if(inst->stats.longest_sescmd < router_cli_ses->stats.longest_sescmd)
inst->stats.longest_sescmd = router_cli_ses->stats.longest_sescmd;
double ses_time = difftime(time(NULL),router_cli_ses->rses_client_dcb->session->stats.connect);
if(inst->stats.ses_longest < ses_time)
inst->stats.ses_longest = ses_time;
if(inst->stats.ses_shortest > ses_time)
inst->stats.ses_shortest = ses_time;
inst->stats.ses_average =
(ses_time + ((inst->stats.sessions - 1) * inst->stats.ses_average)) /
(inst->stats.sessions);
spinlock_release(&inst->lock);
}
}
@ -2069,7 +2134,8 @@ static int routeQuery(
if (succp)
{
atomic_add(&inst->stats.n_all, 1);
atomic_add(&inst->stats.n_sescmd, 1);
atomic_add(&inst->stats.n_queries, 1);
ret = 1;
}
goto retblock;
@ -2270,6 +2336,9 @@ diagnostic(ROUTER *instance, DCB *dcb)
ROUTER_INSTANCE *router = (ROUTER_INSTANCE *)instance;
int i = 0;
double sescmd_pct = router->stats.n_sescmd != 0 ?
100.0*((double)router->stats.n_sescmd / (double)router->stats.n_queries) :
0.0;
dcb_printf(dcb,"\33[1;4m%-16s%-16s%-16s\33[0m\n","Server","Queries","State");
for(i=0;router->servers[i];i++)
@ -2282,6 +2351,45 @@ diagnostic(ROUTER *instance, DCB *dcb)
"\33[30;41mDOWN\33[0m");
}
/** Session command statistics */
dcb_printf(dcb,"\n\33[1;4mSession Commands\33[0m\n");
dcb_printf(dcb,"Total number of queries: %d\n",
router->stats.n_queries);
dcb_printf(dcb,"Percentage of session commands: %.2f\n",
sescmd_pct);
dcb_printf(dcb,"Longest chain of stored session commands: %d\n",
router->stats.longest_sescmd);
dcb_printf(dcb,"Session command history limit exceeded: %d times\n",
router->stats.n_hist_exceeded);
if(!router->schemarouter_config.disable_sescmd_hist)
{
dcb_printf(dcb,"Session command history: enabled\n");
if(router->schemarouter_config.max_sescmd_hist == 0)
{
dcb_printf(dcb,"Session command history limit: unlimited\n");
}
else
{
dcb_printf(dcb,"Session command history limit: %d\n",
router->schemarouter_config.max_sescmd_hist);
}
}
else
{
dcb_printf(dcb,"Session command history: disabled\n");
}
/** Session time statistics */
if(router->stats.sessions > 0)
{
dcb_printf(dcb,"\n\33[1;4mSession Time Statistics\33[0m\n");
dcb_printf(dcb,"Longest session: %.2lf seconds\n",router->stats.ses_longest);
dcb_printf(dcb,"Shortest session: %.2lf seconds\n",router->stats.ses_shortest);
dcb_printf(dcb,"Average session length: %.2lf seconds\n",router->stats.ses_average);
}
dcb_printf(dcb,"\n");
}
/**
@ -3126,7 +3234,7 @@ static mysql_sescmd_t* mysql_sescmd_init (
/** Set session command buffer */
sescmd->my_sescmd_buf = sescmd_buf;
sescmd->my_sescmd_packet_type = packet_type;
sescmd->position = atomic_add(&rses->pos_generator,1);
return sescmd;
}
@ -3178,6 +3286,7 @@ static GWBUF* sescmd_cursor_process_replies(
*/
while (scmd != NULL && replybuf != NULL)
{
scur->position = scmd->position;
/** Faster backend has already responded to client : discard */
if (scmd->my_sescmd_is_replied)
{
@ -3729,17 +3838,74 @@ static bool route_session_write(
succp = false;
goto return_succp;
}
/**
if(router_cli_ses->rses_config.max_sescmd_hist > 0 &&
router_cli_ses->n_sescmd >= router_cli_ses->rses_config.max_sescmd_hist)
{
LOGIF(LE, (skygw_log_write(
LOGFILE_ERROR|LOGFILE_TRACE,
"Router session exceeded session command history limit of %d. "
"Closing router session.",
router_cli_ses->rses_config.max_sescmd_hist)));
gwbuf_free(querybuf);
atomic_add(&router_cli_ses->router->stats.n_hist_exceeded,1);
rses_end_locked_router_action(router_cli_ses);
router_cli_ses->rses_client_dcb->func.hangup(router_cli_ses->rses_client_dcb);
goto return_succp;
}
if(router_cli_ses->rses_config.disable_sescmd_hist)
{
rses_property_t *prop, *tmp;
backend_ref_t* bref;
bool conflict;
prop = router_cli_ses->rses_properties[RSES_PROP_TYPE_SESCMD];
while(prop)
{
conflict = false;
for(i = 0;i<router_cli_ses->rses_nbackends;i++)
{
bref = &backend_ref[i];
if(BREF_IS_IN_USE(bref))
{
if(bref->bref_sescmd_cur.position <= prop->rses_prop_data.sescmd.position)
{
conflict = true;
break;
}
}
}
if(conflict)
{
break;
}
tmp = prop;
router_cli_ses->rses_properties[RSES_PROP_TYPE_SESCMD] = prop->rses_prop_next;
rses_property_done(tmp);
prop = router_cli_ses->rses_properties[RSES_PROP_TYPE_SESCMD];
}
}
/**
*
* Additional reference is created to querybuf to
* prevent it from being released before properties
* are cleaned up as a part of router sessionclean-up.
* are cleaned up as a part of router session clean-up.
*/
prop = rses_property_init(RSES_PROP_TYPE_SESCMD);
mysql_sescmd_init(prop, querybuf, packet_type, router_cli_ses);
/** Add sescmd property to router client session */
rses_property_add(router_cli_ses, prop);
atomic_add(&router_cli_ses->stats.longest_sescmd,1);
atomic_add(&router_cli_ses->n_sescmd,1);
for (i=0; i<router_cli_ses->rses_nbackends; i++)
{
if (BREF_IS_IN_USE((&backend_ref[i])))
@ -3794,6 +3960,10 @@ static bool route_session_write(
backend_ref[i].bref_backend->backend_server->name,
backend_ref[i].bref_backend->backend_server->port)));
}
else
{
atomic_add(&backend_ref[i].bref_backend->stats.queries,1);
}
}
}
else