hcq/MaxScale
hcq/MaxScale
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch 'develop' into 1.2.1-binlog_router_trx

Browse Source
This commit is contained in:
Markus Makela 2015-11-02 18:09:59 +02:00
commit 7c582f91ea
24 changed files with 2962 additions and 2258 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
Documentation/Getting-Started/Configuration-Guide.md
View File

@ -50,11 +50,11 @@ Please see the section about [Protocol Modules](#protocol-modules) for more deta
### Global Settings
The global settings, in a section named `[MaxScale]`, allow various parameters that affect MaxScale as a whole to be tuned. Currently the only setting that is supported is the number of threads to use to handle the network traffic. MaxScale will also accept the section name of `[gateway]` for global settings. This is for backward compatibility with versions prior to the naming of MaxScale.
The global settings, in a section named `[MaxScale]`, allow various parameters that affect MaxScale as a whole to be tuned.
#### `threads`
To control the number of threads that poll for network traffic set the parameter threads to a number. It is recommended that you start with a single thread and add more as you find the performance is not satisfactory. MaxScale is implemented to be very thread efficient, so a small number of threads is usually adequate to support reasonably heavy workloads. Adding more threads may not improve performance and can consume resources needlessly.
This parameter controls the number of worker threads that are handling the events coming from the kernel. MaxScale will auto-detect the number of processors of the system unless number of threads is manually configured. It is recommended that you let MaxScale detect how many cores the system has and leave this parameter undefined. The number of used cores will be logged into the message logs and if you are not satisfied with the auto-detected value, you can manually configure it. Increasing the amount of worker threads beyond the number of processor cores does not improve performance and can consume resources needlessly.
```
# Valid options are:

2
Documentation/Reference/Hint-Syntax.md
View File

@ -14,7 +14,7 @@ router_options=master
servers=server1
user=maxuser
passwd=maxpwd
filter=Hint
filters=Hint
[Hint]
type=filter

9
Documentation/Tutorials/MySQL-Replication-Connection-Routing-Tutorial.md
View File

@ -51,11 +51,11 @@ MariaDB [(none)]> grant SELECT on mysql.user to '*username*'@'*maxscalehost*';
Additionally, GRANT SELECT on the mysql.db table and SHOW DATABASES privileges are required in order to load databases name and grants suitable for database name authorization.
```
MariaDB [(none)]> GRANT SELECT ON mysql.db TO 'username'@'maxscalehost';
MariaDB [(none)]> GRANT SELECT ON mysql.db TO '*username*'@'maxscalehost';
**Query OK, 0 rows affected (0.00 sec)**
MariaDB [(none)]> GRANT SHOW DATABASES ON *.* TO 'username'@'maxscalehost';
MariaDB [(none)]> GRANT SHOW DATABASES ON *.* TO '*username*'@'maxscalehost';
**Query OK, 0 rows affected (0.00 sec)**
```
@ -76,7 +76,7 @@ If you wish to use two different usernames for the two different roles of monito
## Creating Your MaxScale Configuration
MaxScale configuration is held in an ini file that is located in the file maxscale.cnf in the directory /etc. This is not created as part of the installation process and must be manually created. A template file does exist in the `/usr/share/maxscale` folder that can be use as a basis for your configuration.
MaxScale reads its configuration from `/etc/maxscale.cnf`. This is not created as part of the installation process and must be manually created. A template file does exist in the `/usr/share/maxscale` folder that can be use as a basis for your configuration.
A global, maxscale, section is included within every MaxScale configuration file; this is used to set the values of various MaxScale wide parameters, perhaps the most important of these is the number of threads that MaxScale will use to execute the code that forwards requests and handles responses for clients.
@ -220,6 +220,7 @@ module=mysqlmon
servers=dbserv1, dbserv2, dbserv3
user=maxscale
passwd=96F99AA1315BDC3604B006F427DD9484
monitor_interval=10000
```
As with the password definition in the server either plain text or encrypted passwords may be used.
@ -255,7 +256,7 @@ or
service maxscale start
```
Check the error log in /var/log/lomaxscale/ to see if any errors are detected in the configuration file and to confirm MaxScale has been started. Also the maxadmin command may be used to confirm that MaxScale is running and the services, listeners etc have been correctly configured.
Check the error log in /var/log/maxscale/ to see if any errors are detected in the configuration file and to confirm MaxScale has been started. Also the maxadmin command may be used to confirm that MaxScale is running and the services, listeners etc have been correctly configured.
```
% maxadmin -pmariadb list services

565
log_manager/log_manager.cc
View File

@ -287,33 +287,13 @@ static bool logmanager_register(bool writep);
static void logmanager_unregister(void);
static bool logmanager_init_nomutex(int argc, char* argv[]);
static void logmanager_done_nomutex(void);
static bool logmanager_is_valid_id(logfile_id_t id);
enum log_spread_down
{
LOG_SPREAD_DOWN_NO = 0,
LOG_SPREAD_DOWN_YES = 1
};
enum log_use_valist
{
LOG_USE_VALIST_NO = 0,
LOG_USE_VALIST_YES = 1
};
enum log_rotate
{
LOG_ROTATE_NO = 0,
LOG_ROTATE_YES = 1
};
static int logmanager_write_log(logfile_id_t id,
enum log_flush flush,
enum log_use_valist use_valist,
enum log_spread_down spread_down,
enum log_rotate rotate,
size_t len,
const char* str,
va_list valist);
static int logmanager_write_log(logfile_id_t id,
enum log_flush flush,
size_t prefix_len,
size_t len,
const char* str);
static blockbuf_t* blockbuf_init(logfile_id_t id);
static void blockbuf_node_done(void* bb_data);
@ -420,10 +400,7 @@ static bool logmanager_init_nomutex(int argc, char* argv[])
lm->lm_enabled_logfiles |= LOGFILE_ERROR;
lm->lm_enabled_logfiles |= LOGFILE_MESSAGE;
#if defined(SS_DEBUG)
lm->lm_enabled_logfiles |= LOGFILE_TRACE;
lm->lm_enabled_logfiles |= LOGFILE_DEBUG;
#endif
fn = &lm->lm_fnames_conf;
fw = &lm->lm_filewriter;
fn->fn_state = UNINIT;
@ -639,6 +616,42 @@ static logfile_t* logmanager_get_logfile(logmanager_t* lmgr, logfile_id_t id)
}
/**
* Returns true if the id log file id is valid.
*
* NOTE: Log manager is assumed to exist.
*
* @param id The id of a log file.
*
*/
static bool logmanager_is_valid_id(logfile_id_t id)
{
bool rval = false;
CHK_LOGMANAGER(lm);
if ((id >= LOGFILE_FIRST) && (id <= LOGFILE_LAST))
{
rval = true;
}
else
{
const char ERRSTR[] = "Invalid logfile id argument.";
int err = logmanager_write_log(LOGFILE_ERROR,
LOG_FLUSH_YES,
0, sizeof(ERRSTR), ERRSTR);
if (err != 0)
{
fprintf(stderr, "Writing to logfile %s failed.\n", STRLOGID(LOGFILE_ERROR));
}
}
return rval;
}
/**
* Finds write position from block buffer for log string and writes there.
*
@ -647,27 +660,20 @@ static logfile_t* logmanager_get_logfile(logmanager_t* lmgr, logfile_id_t id)
* @param id logfile object identifier
* @param flush indicates whether log string must be written to disk
* immediately
* @param use_valist does write involve formatting of the string and use of
* valist argument
* @param spread_down if true, log string is spread to all logs having
* larger id
* @param rotate if set, closes currently open log file and opens a
* new one
* @param str_len length of formatted string
* @param prefix_len length of prefix to be stripped away when syslogging
* @param str_len length of formatted string (including terminating NULL).
* @param str string to be written to log
* @param valist variable-length argument list for formatting the string
*
* @return 0 if succeed, -1 otherwise
*
*/
static int logmanager_write_log(logfile_id_t id,
enum log_flush flush,
enum log_use_valist use_valist,
enum log_spread_down spread_down,
enum log_rotate rotate,
size_t str_len,
const char* str,
va_list valist)
static int logmanager_write_log(logfile_id_t id,
enum log_flush flush,
size_t prefix_len,
size_t str_len,
const char* str)
{
logfile_t* lf;
char* wp;
@ -679,35 +685,18 @@ static int logmanager_write_log(logfile_id_t id,
CHK_LOGMANAGER(lm);
if (id < LOGFILE_FIRST || id > LOGFILE_LAST)
if (!logmanager_is_valid_id(id))
{
const char* errstr = "Invalid logfile id argument.";
/**
* invalid id, since we don't have logfile yet.
*/
err = logmanager_write_log(LOGFILE_ERROR,
LOG_FLUSH_YES,
LOG_USE_VALIST_NO,
LOG_SPREAD_DOWN_NO,
LOG_ROTATE_NO,
strlen(errstr) + 1,
errstr,
valist);
if (err != 0)
{
fprintf(stderr,
"Writing to logfile %s failed.\n",
STRLOGID(LOGFILE_ERROR));
}
err = -1;
ss_dassert(false);
goto return_err;
}
lf = &lm->lm_logfile[id];
// All messages are now logged to the error log file.
lf = &lm->lm_logfile[LOGFILE_ERROR];
CHK_LOGFILE(lf);
/**
* When string pointer is NULL, operation is either flush or rotate.
* When string pointer is NULL, operation is flush.
*/
if (str == NULL)
{
@ -715,10 +704,6 @@ static int logmanager_write_log(logfile_id_t id,
{
logfile_flush(lf); /*< wakes up file writer */
}
else if (rotate)
{
logfile_rotate(lf); /*< wakes up file writer */
}
}
else
{
@ -793,7 +778,8 @@ static int logmanager_write_log(logfile_id_t id,
/** Book space for log string from buffer */
if (do_maxscalelog)
{
wp = blockbuf_get_writepos(&bb, id, safe_str_len, flush);
// All messages are now logged to the error log file.
wp = blockbuf_get_writepos(&bb, LOGFILE_ERROR, safe_str_len, flush);
}
else
{
@ -837,20 +823,10 @@ static int logmanager_write_log(logfile_id_t id,
* Write next string to overwrite terminating null character
* of the timestamp string.
*/
if (use_valist)
{
vsnprintf(wp + timestamp_len + sesid_str_len,
safe_str_len - timestamp_len - sesid_str_len,
str,
valist);
}
else
{
snprintf(wp + timestamp_len + sesid_str_len,
safe_str_len-timestamp_len-sesid_str_len,
"%s",
str);
}
snprintf(wp + timestamp_len + sesid_str_len,
safe_str_len-timestamp_len-sesid_str_len,
"%s",
str);
/** Add an ellipsis to an overflowing message to signal truncation. */
if (overflow && safe_str_len > 4)
@ -860,14 +836,17 @@ static int logmanager_write_log(logfile_id_t id,
/** write to syslog */
if (lf->lf_write_syslog)
{
// Strip away the timestamp and the prefix (e.g. "[Error]: ").
const char *message = wp + timestamp_len + prefix_len;
switch (id)
{
case LOGFILE_ERROR:
syslog(LOG_ERR, "%s", wp + timestamp_len);
syslog(LOG_ERR, "%s", message);
break;
case LOGFILE_MESSAGE:
syslog(LOG_NOTICE, "%s", wp + timestamp_len);
syslog(LOG_NOTICE, "%s", message);
break;
default:
@ -889,61 +868,6 @@ static int logmanager_write_log(logfile_id_t id,
{
free(wp);
}
/**
* disabled because cross-blockbuffer locking either causes deadlock
* or run out of memory blocks.
*/
if (spread_down && false)
{
/**
* Write to target log. If spread_down == true, then
* write also to all logs with greater logfile id.
* LOGFILE_ERROR = 1,
* LOGFILE_MESSAGE = 2,
* LOGFILE_TRACE = 4,
* LOGFILE_DEBUG = 8
*
* So everything written to error log will appear in
* message, trace and debuglog. Messages will be
* written in trace and debug log.
*/
for (i = (id << 1); i <= LOGFILE_LAST; i <<= 1)
{
/** pointer to write buffer of larger-id log */
char* wp_c;
/**< Check if particular log is enabled */
if (!(lm->lm_enabled_logfiles & i))
{
continue;
}
/**
* Seek write position and register to block
* buffer. Then print formatted string to
* write position.
*/
wp_c = blockbuf_get_writepos(&bb_c,
(logfile_id_t)i,
timestamp_len - 1 + str_len,
flush);
/**
* Copy original string from block buffer to
* other logs' block buffers.
*/
snprintf(wp_c, timestamp_len + str_len, "%s", wp);
/** remove double line feed */
if (wp_c[timestamp_len - 1 + str_len - 2] == '\n')
{
wp_c[timestamp_len - 1 + str_len - 2] = ' ';
}
wp_c[timestamp_len - 1 + str_len - 1] = '\n';
/** lock-free unregistration, includes flush if
* bb_state == BB_FULL */
blockbuf_unregister(bb_c);
}
} /* if (spread_down) */
} /* if (str == NULL) */
return_err:
@ -1360,75 +1284,63 @@ return_err:
static bool logfile_set_enabled(logfile_id_t id, bool val)
{
char* logstr;
va_list notused;
bool oldval;
bool succp = false;
int err = 0;
logfile_t* lf;
bool rval = false;
CHK_LOGMANAGER(lm);
if (id < LOGFILE_FIRST || id > LOGFILE_LAST)
if (logmanager_is_valid_id(id))
{
const char* errstr = "Invalid logfile id argument.";
/**
* invalid id, since we don't have logfile yet.
*/
err = logmanager_write_log(LOGFILE_ERROR,
LOG_FLUSH_YES,
LOG_USE_VALIST_NO,
LOG_SPREAD_DOWN_NO,
LOG_ROTATE_NO,
strlen(errstr) + 1,
errstr,
notused);
if (err != 0)
if (use_stdout == 0)
{
fprintf(stderr,
"* Writing to logfile %s failed.\n",
STRLOGID(LOGFILE_ERROR));
}
ss_dassert(false);
goto return_succp;
}
lf = &lm->lm_logfile[id];
CHK_LOGFILE(lf);
if (use_stdout == 0)
{
if (val)
{
logstr = strdup("---\tLogging to file is enabled\t--");
}
else
{
logstr = strdup("---\tLogging to file is disabled\t--");
logfile_t *lf = logmanager_get_logfile(lm, id);
lf->lf_enabled = val;
const char *name;
switch (id)
{
default:
case LOGFILE_ERROR:
name = "LOGFILE_ERROR";
break;
case LOGFILE_MESSAGE:
name = "LOGFILE_MESSAGE";
break;
case LOGFILE_TRACE:
name = "LOGFILE_TRACE";
break;
case LOGFILE_DEBUG:
name = "LOGFILE_DEBUG";
break;
}
const char FORMAT[] = "The logging of %s messages is %s.";
const char *action;
if (val)
{
action = "enabled";
}
else
{
action = "disabled";
}
MXS_NOTICE(FORMAT, name, action);
}
oldval = lf->lf_enabled;
lf->lf_enabled = val;
err = logmanager_write_log(id,
LOG_FLUSH_YES,
LOG_USE_VALIST_NO,
LOG_SPREAD_DOWN_NO,
LOG_ROTATE_NO,
strlen(logstr) + 1,
logstr,
notused);
free(logstr);
rval = true;
}
if (err != 0)
else
{
lf->lf_enabled = oldval;
fprintf(stderr,
"logfile_set_enabled failed. Writing notification to logfile %s "
"failed.\n ",
STRLOGID(id));
goto return_succp;
MXS_ERROR("Invalid logfile id %d.", id);
ss_dassert(!true);
}
succp = true;
return_succp:
return succp;
return rval;
}
void skygw_log_set_augmentation(int bits)
@ -1448,7 +1360,9 @@ int skygw_log_get_augmentation()
* @param file The name of the file where the logging was made.
* @param int The line where the logging was made.
* @param function The function where the logging was made.
* @param str String or printf format string.
* @param prefix_len The length of the text to be stripped away when syslogging.
* @param len Length of str, including terminating NULL.
* @param str String
* @param flush Whether the message should be flushed.
*
* @return 0 if the logging to at least one log succeeded.
@ -1458,6 +1372,7 @@ static int log_write(logfile_id_t id,
const char* file,
int line,
const char* function,
size_t prefix_len,
size_t len,
const char* str,
enum log_flush flush)
@ -1468,50 +1383,9 @@ static int log_write(logfile_id_t id,
{
CHK_LOGMANAGER(lm);
const char* format;
if (log_augmentation == LOG_AUGMENT_WITH_FUNCTION)
{
static const char function_format[] = "%s [%s]";
format = function_format;
len += sizeof(function_format); // A little bit more than needed, but won't hurt.
assert(function);
len += strlen(function);
}
else
{
static const char default_format[] = "%s";
format = default_format;
len += sizeof(default_format); // A little bit more than needed, but won't hurt.
}
len += 1; // For the trailing NULL.
char message[len];
if (log_augmentation == LOG_AUGMENT_WITH_FUNCTION)
{
len = snprintf(message, sizeof(message), format, str, function);
}
else
{
len = snprintf(message, sizeof(message), format, str);
}
assert(len >= 0);
int attempts = 0;
int successes = 0;
/**
* Add one for line feed.
*/
len += sizeof(char);
for (int i = LOGFILE_FIRST; i <= LOGFILE_LAST; i <<= 1)
{
/**
@ -1522,14 +1396,10 @@ static int log_write(logfile_id_t id,
{
++attempts;
va_list valist; // Not used
if (logmanager_write_log((logfile_id_t)i,
flush,
LOG_USE_VALIST_NO,
LOG_SPREAD_DOWN_YES,
LOG_ROTATE_NO,
len, message, valist) == 0)
prefix_len,
len, str) == 0)
{
++successes;
}
@ -1552,6 +1422,55 @@ static int log_write(logfile_id_t id,
return rv;
}
typedef struct log_prefix
{
const char* text; // The prefix, e.g. "[Error]: "
int len; // The length of the prefix without the trailing NULL.
} log_prefix_t;
const char PREFIX_ERROR[] = "[Error] : ";
const char PREFIX_NOTICE[] = "[Notice]: ";
const char PREFIX_INFO[] = "[Info] : ";
const char PREFIX_DEBUG[] = "[Debug] : ";
static log_prefix_t logfile_to_prefix(logfile_id_t id)
{
log_prefix_t prefix;
// The id can be a bitmask, hence we choose the most "severe" one.
if (id & LOGFILE_ERROR)
{
prefix.text = PREFIX_ERROR;
prefix.len = sizeof(PREFIX_ERROR);
}
else if (id & LOGFILE_MESSAGE)
{
prefix.text = PREFIX_NOTICE;
prefix.len = sizeof(PREFIX_NOTICE);
}
else if (id & LOGFILE_TRACE)
{
prefix.text = PREFIX_INFO;
prefix.len = sizeof(PREFIX_INFO);
}
else if (id & LOGFILE_DEBUG)
{
prefix.text = PREFIX_DEBUG;
prefix.len = sizeof(PREFIX_DEBUG);
}
else
{
assert(!true);
prefix.text = PREFIX_ERROR;
prefix.len = sizeof(PREFIX_ERROR);
}
--prefix.len; // Remove trailing NULL.
return prefix;
}
int skygw_log_write_context(logfile_id_t id,
enum log_flush flush,
const char* file,
@ -1567,23 +1486,69 @@ int skygw_log_write_context(logfile_id_t id,
* Find out the length of log string (to be formatted str).
*/
va_start(valist, str);
int len = vsnprintf(NULL, 0, str, valist);
int message_len = vsnprintf(NULL, 0, str, valist);
va_end(valist);
if (len >= 0)
if (message_len >= 0)
{
if (len > MAX_LOGSTRLEN)
log_prefix_t prefix = logfile_to_prefix(id);
static const char FORMAT_FUNCTION[] = "(%s): ";
int augmentation_len = 0;
switch (log_augmentation)
{
len = MAX_LOGSTRLEN;
case LOG_AUGMENT_WITH_FUNCTION:
augmentation_len = sizeof(FORMAT_FUNCTION) - 1; // Remove trailing 0
augmentation_len -= 2; // Remove the %s
augmentation_len += strlen(function);
break;
default:
break;
}
char message[len + 1];
int buffer_len = prefix.len + augmentation_len + message_len + 1; // Trailing NULL
if (buffer_len > MAX_LOGSTRLEN)
{
message_len -= (buffer_len - MAX_LOGSTRLEN);
buffer_len = MAX_LOGSTRLEN;
assert(prefix.len + augmentation_len + message_len + 1 == buffer_len);
}
char buffer[buffer_len];
char *prefix_text = buffer;
char *augmentation_text = buffer + prefix.len;
char *message_text = buffer + prefix.len + augmentation_len;
strcpy(prefix_text, prefix.text);
if (augmentation_len)
{
int len = 0;
switch (log_augmentation)
{
case LOG_AUGMENT_WITH_FUNCTION:
len = sprintf(augmentation_text, FORMAT_FUNCTION, function);
break;
default:
assert(!true);
}
assert(len == augmentation_len);
}
va_start(valist, str);
vsnprintf(message, sizeof(message), str, valist);
vsnprintf(message_text, message_len + 1, str, valist);
va_end(valist);
err = log_write(id, file, line, function, len, message, flush);
err = log_write(id, file, line, function, prefix.len, buffer_len, buffer, flush);
if (err != 0)
{
@ -1597,32 +1562,28 @@ int skygw_log_write_context(logfile_id_t id,
int skygw_log_flush(logfile_id_t id)
{
int err = 0;
va_list valist; /**< Dummy, must be present but it is not processed */
int err = -1;
if (!logmanager_register(false))
if (logmanager_register(false))
{
ss_dfprintf(stderr,
"Can't register to logmanager, nothing to flush\n");
goto return_err;
CHK_LOGMANAGER(lm);
if (logmanager_is_valid_id(id))
{
logfile_t *lf = logmanager_get_logfile(lm, id);
CHK_LOGFILE(lf);
logfile_flush(lf);
err = 0;
}
logmanager_unregister();
}
CHK_LOGMANAGER(lm);
err = logmanager_write_log(id,
LOG_FLUSH_YES,
LOG_USE_VALIST_NO,
LOG_SPREAD_DOWN_NO,
LOG_ROTATE_NO,
0, NULL, valist);
if (err != 0)
else
{
fprintf(stderr, "skygw_log_flush failed.\n");
goto return_unregister;
ss_dfprintf(stderr, "Can't register to logmanager, flushing failed.\n");
}
return_unregister:
logmanager_unregister();
return_err:
return err;
}
@ -1632,48 +1593,30 @@ return_err:
*/
int skygw_log_rotate(logfile_id_t id)
{
int err = 0;
logfile_t* lf;
va_list valist; /**< Dummy, must be present but it is not processed */
int err = -1;
if (!logmanager_register(false))
if (logmanager_register(false))
{
ss_dfprintf(stderr, "Can't register to logmanager, rotating failed\n");
goto return_err;
CHK_LOGMANAGER(lm);
if (logmanager_is_valid_id(id))
{
logfile_t *lf = logmanager_get_logfile(lm, id);
CHK_LOGFILE(lf);
MXS_NOTICE("Log rotation is called for %s.", lf->lf_full_file_name);
logfile_rotate(lf);
err = 0;
}
logmanager_unregister();
}
CHK_LOGMANAGER(lm);
lf = &lm->lm_logfile[id];
LOGIF(LM, (skygw_log_write(LOGFILE_MESSAGE,
"Log rotation is called for %s.",
lf->lf_full_file_name)));
err = logmanager_write_log(id,
LOG_FLUSH_NO,
LOG_USE_VALIST_NO,
LOG_SPREAD_DOWN_NO,
LOG_ROTATE_YES,
0, NULL, valist);
if (err != 0)
else
{
LOGIF(LE, (skygw_log_write(LOGFILE_ERROR,
"Log file rotation failed for file %s.",
lf->lf_full_file_name)));
fprintf(stderr, "skygw_log_rotate failed.\n");
goto return_unregister;
ss_dfprintf(stderr, "Can't register to logmanager, rotating failed.\n");
}
return_unregister:
LOGIF(LM, (skygw_log_write_flush(LOGFILE_MESSAGE,
"File %s use for log writing..",
lf->lf_full_file_name)));
logmanager_unregister();
return_err:
return err;
}

14
log_manager/log_manager.h
View File

@ -185,13 +185,13 @@ const char* get_logpath_default(void);
/**
* Helper, not to be called directly.
*/
#define MAXSCALE_MESSAGE_FLUSH(id, format, ...)\
#define MXS_MESSAGE_FLUSH(id, format, ...)\
do { if (LOG_IS_ENABLED(id)) { skygw_log_write_flush(id, format, ##__VA_ARGS__); } } while (false)
/**
* Helper, not to be called directly.
*/
#define MAXSCALE_MESSAGE(id, format, ...)\
#define MXS_MESSAGE(id, format, ...)\
do { if (LOG_IS_ENABLED(id)) { skygw_log_write(id, format, ##__VA_ARGS__); } } while (false)
/**
@ -200,10 +200,10 @@ const char* get_logpath_default(void);
* @param format The printf format of the message.
* @param ... Arguments, depending on the format.
*/
#define MAXSCALE_ERROR(format, ...) MAXSCALE_MESSAGE_FLUSH(LOGFILE_ERROR, format, ##__VA_ARGS__)
#define MAXSCALE_WARNING(format, ...) MAXSCALE_MESSAGE(LOGFILE_ERROR, format, ##__VA_ARGS__)
#define MAXSCALE_NOTICE(format, ...) MAXSCALE_MESSAGE(LOGFILE_MESSAGE, format, ##__VA_ARGS__)
#define MAXSCALE_INFO(format, ...) MAXSCALE_MESSAGE(LOGFILE_TRACE, format, ##__VA_ARGS__)
#define MAXSCALE_DEBUG(format, ...) MAXSCALE_MESSAGE(LOGFILE_DEBUG, format, ##__VA_ARGS__)
#define MXS_ERROR(format, ...) MXS_MESSAGE_FLUSH(LOGFILE_ERROR, format, ##__VA_ARGS__)
#define MXS_WARNING(format, ...) MXS_MESSAGE(LOGFILE_ERROR, format, ##__VA_ARGS__)
#define MXS_NOTICE(format, ...) MXS_MESSAGE(LOGFILE_MESSAGE, format, ##__VA_ARGS__)
#define MXS_INFO(format, ...) MXS_MESSAGE(LOGFILE_TRACE, format, ##__VA_ARGS__)
#define MXS_DEBUG(format, ...) MXS_MESSAGE(LOGFILE_DEBUG, format, ##__VA_ARGS__)
#endif /** LOG_MANAGER_H */

4
server/core/CMakeLists.txt
View File

@ -1,5 +1,5 @@
if(BUILD_TESTS OR BUILD_TOOLS)
add_library(fullcore STATIC adminusers.c atomic.c config.c buffer.c dbusers.c dcb.c filter.c gwbitmask.c gw_utils.c hashtable.c hint.c housekeeper.c load_utils.c memlog.c modutil.c monitor.c poll.c resultset.c secrets.c server.c service.c session.c spinlock.c thread.c users.c utils.c gwdirs.c externcmd.c)
add_library(fullcore STATIC adminusers.c atomic.c config.c buffer.c dbusers.c dcb.c filter.c gwbitmask.c gw_utils.c hashtable.c hint.c housekeeper.c load_utils.c memlog.c modutil.c monitor.c poll.c resultset.c secrets.c server.c service.c session.c spinlock.c thread.c users.c utils.c gwdirs.c externcmd.c maxscale_pcre2.c)
if(WITH_JEMALLOC)
target_link_libraries(fullcore ${JEMALLOC_LIBRARIES})
elseif(WITH_TCMALLOC)
@ -12,7 +12,7 @@ add_executable(maxscale atomic.c buffer.c spinlock.c gateway.c
gw_utils.c utils.c dcb.c load_utils.c session.c service.c server.c
poll.c config.c users.c hashtable.c dbusers.c thread.c gwbitmask.c
monitor.c adminusers.c secrets.c filter.c modutil.c hint.c
housekeeper.c memlog.c resultset.c gwdirs.c externcmd.c)
housekeeper.c memlog.c resultset.c gwdirs.c externcmd.c maxscale_pcre2.c)
if(WITH_JEMALLOC)
target_link_libraries(maxscale ${JEMALLOC_LIBRARIES})

161
server/core/config.c
View File

@ -75,6 +75,9 @@
#include <sys/utsname.h>
#include <pcre.h>
#include <dbusers.h>
#include <gw.h>
#define PCRE2_CODE_UNIT_WIDTH 8
#include <pcre2.h>
/** According to the PCRE manual, this should be a multiple of 3 */
#define MAXSCALE_PCRE_BUFSZ 24
@ -95,12 +98,14 @@ static int handle_feedback_item(const char *, const char *);
static void global_defaults();
static void feedback_defaults();
static void check_config_objects(CONFIG_CONTEXT *context);
static int maxscale_getline(char** dest, int* size, FILE* file);
int config_truth_value(char *str);
bool isInternalService(char *router);
int config_get_ifaddr(unsigned char *output);
int config_get_release_string(char* release);
FEEDBACK_CONF * config_get_feedback_data();
void config_add_param(CONFIG_CONTEXT*,char*,char*);
bool config_has_duplicate_sections(const char* config);
static char *config_file = NULL;
static GATEWAY_CONF gateway;
static FEEDBACK_CONF feedback;
@ -274,6 +279,10 @@ config_load(char *file)
CONFIG_CONTEXT config;
int rval, ini_rval;
if (config_has_duplicate_sections(file))
{
return 0;
}
MYSQL *conn;
conn = mysql_init(NULL);
if (conn) {
@ -356,6 +365,11 @@ int rval;
if (!config_file)
return 0;
if (config_has_duplicate_sections(config_file))
{
return 0;
}
if (gateway.version_string)
free(gateway.version_string);
@ -1541,7 +1555,16 @@ handle_global_item(const char *name, const char *value)
int i;
if (strcmp(name, "threads") == 0)
{
gateway.n_threads = atoi(value);
int thrcount = atoi(value);
if (thrcount > 0)
{
gateway.n_threads = thrcount;
}
else
{
skygw_log_write(LE, "Warning: Invalid value for 'threads': %s.", value);
return 0;
}
}
else if (strcmp(name, "non_blocking_polls") == 0)
{
@ -1644,7 +1667,7 @@ global_defaults()
{
uint8_t mac_addr[6]="";
struct utsname uname_data;
gateway.n_threads = 1;
gateway.n_threads = get_processor_count();
gateway.n_nbpoll = DEFAULT_NBPOLLS;
gateway.pollsleep = DEFAULT_POLLSLEEP;
gateway.auth_conn_timeout = DEFAULT_AUTH_CONNECT_TIMEOUT;
@ -2629,3 +2652,137 @@ GATEWAY_CONF* config_get_global_options()
{
return &gateway;
}
/**
* Check if sections are defined multiple times in the configuration file.
* @param config Path to the configuration file
* @return True if duplicate sections were found or an error occurred
*/
bool config_has_duplicate_sections(const char* config)
{
bool rval = false;
const int table_size = 10;
int errcode;
PCRE2_SIZE erroffset;
HASHTABLE *hash = hashtable_alloc(table_size, simple_str_hash, strcmp);
pcre2_code *re = pcre2_compile((PCRE2_SPTR) "^\\s*\\[(.+)\\]\\s*$", PCRE2_ZERO_TERMINATED,
0, &errcode, &erroffset, NULL);
pcre2_match_data *mdata;
int size = 1024;
char *buffer = malloc(size * sizeof(char));
if (buffer && hash && re &&
(mdata = pcre2_match_data_create_from_pattern(re, NULL)))
{
hashtable_memory_fns(hash, (HASHMEMORYFN) strdup, NULL,
(HASHMEMORYFN) free, NULL);
FILE* file = fopen(config, "r");
if (file)
{
while (maxscale_getline(&buffer, &size, file) > 0)
{
if (pcre2_match(re, (PCRE2_SPTR) buffer,
PCRE2_ZERO_TERMINATED, 0, 0,
mdata, NULL) > 0)
{
/**
* Neither of the PCRE2 calls will fail since we know the pattern
* beforehand and we allocate enough memory from the stack
*/
PCRE2_SIZE len;
pcre2_substring_length_bynumber(mdata, 1, &len);
len += 1; /** one for the null terminator */
PCRE2_UCHAR section[len];
pcre2_substring_copy_bynumber(mdata, 1, section, &len);
if (hashtable_add(hash, section, "") == 0)
{
skygw_log_write(LE, "Error: Duplicate section found: %s",
section);
rval = true;
}
}
}
fclose(file);
}
else
{
char errbuf[STRERROR_BUFLEN];
skygw_log_write(LE, "Error: Failed to open file '%s': %s", config,
strerror_r(errno, errbuf, sizeof(errbuf)));
rval = true;
}
}
else
{
skygw_log_write(LE, "Error: Failed to allocate enough memory when checking"
" for duplicate sections in configuration file.");
rval = true;
}
hashtable_free(hash);
pcre2_code_free(re);
pcre2_match_data_free(mdata);
free(buffer);
return rval;
}
/**
* Read from a FILE pointer until a newline character or the end of the file is found.
* The provided buffer will be reallocated if it is too small to store the whole
* line. The size after the reallocation will be stored in @c size. The read line
* will be stored in @c dest and it will always be null terminated. The newline
* character will not be copied into the buffer.
* @param dest Pointer to a buffer of at least @c size bytes
* @param size Size of the buffer
* @param file A valid file stream
* @return When a complete line was successfully read the function returns 1. If
* the end of the file was reached before any characters were read the return value
* will be 0. If the provided buffer could not be reallocated to store the complete
* line the original size will be retained, everything read up to this point
* will be stored in it as a null terminated string and -1 will be returned.
*/
int maxscale_getline(char** dest, int* size, FILE* file)
{
char* destptr = *dest;
int offset = 0;
if (feof(file))
{
return 0;
}
while (true)
{
if (*size <= offset)
{
char* tmp = (char*) realloc(destptr, *size * 2);
if (tmp)
{
destptr = tmp;
*size *= 2;
}
else
{
skygw_log_write(LE, "Error: Failed to reallocate memory from %d"
" bytes to %d bytes when reading from file.",
*size, *size * 2);
destptr[offset - 1] = '\0';
*dest = destptr;
return -1;
}
}
if ((destptr[offset] = fgetc(file)) == '\n' || feof(file))
{
destptr[offset] = '\0';
break;
}
offset++;
}
*dest = destptr;
return 1;
}

58
server/core/dcb.c
View File

@ -80,6 +80,8 @@
#include <hashtable.h>
#include <hk_heartbeat.h>
#define SSL_ERRBUF_LEN 140
/** Defined in log_manager.cc */
extern int lm_enabled_logfiles_bitmask;
extern size_t log_ses_count[];
@ -1480,8 +1482,8 @@ dcb_write_SSL_error_report (DCB *dcb, int ret, int ssl_errno)
}
do
{
char errbuf[140];
ERR_error_string_n(ssl_errno,errbuf,140);
char errbuf[SSL_ERRBUF_LEN];
ERR_error_string_n(ssl_errno,errbuf, sizeof(errbuf));
skygw_log_write(LE,"%d:%s",ssl_errno,errbuf);
} while((ssl_errno = ERR_get_error()) != 0);
}
@ -1490,8 +1492,8 @@ dcb_write_SSL_error_report (DCB *dcb, int ret, int ssl_errno)
{
do
{
char errbuf[140];
ERR_error_string_n(ssl_errno,errbuf,140);
char errbuf[SSL_ERRBUF_LEN];
ERR_error_string_n(ssl_errno,errbuf,sizeof(errbuf));
skygw_log_write(LE,"%d:%s",ssl_errno,errbuf);
} while((ssl_errno = ERR_get_error()) != 0);
}
@ -1646,8 +1648,8 @@ dcb_drain_writeq_SSL(DCB *dcb)
case SSL_ERROR_SYSCALL:
while((ssl_errno = ERR_get_error()) != 0)
{
char errbuf[140];
ERR_error_string_n(ssl_errno,errbuf,140);
char errbuf[SSL_ERRBUF_LEN];
ERR_error_string_n(ssl_errno,errbuf,sizeof(errbuf));
skygw_log_write(LE,"%s",errbuf);
}
if(errno != 0)
@ -2865,8 +2867,9 @@ int dcb_create_SSL(DCB* dcb)
*/
int dcb_accept_SSL(DCB* dcb)
{
int rval = 0,ssl_rval,errnum = 0,fd,b = 0,pending;
char errbuf[140];
int rval = 0,ssl_rval,ssl_errnum = 0,fd,b = 0,pending;
int err_errnum;
char errbuf[SSL_ERRBUF_LEN];
fd = dcb->fd;
do
@ -2874,22 +2877,20 @@ int dcb_accept_SSL(DCB* dcb)
ssl_rval = SSL_accept(dcb->ssl);
LOGIF(LD,(skygw_log_write_flush(LD,"[dcb_accept_SSL] SSL_accept %d, error %d",
ssl_rval,errnum)));
ssl_rval,ssl_errnum)));
switch(ssl_rval)
{
case 0:
errnum = SSL_get_error(dcb->ssl,ssl_rval);
ssl_errnum = SSL_get_error(dcb->ssl,ssl_rval);
skygw_log_write(LE,"Error: SSL authentication failed (SSL error %d):",
dcb,
dcb->remote,
errnum);
ssl_errnum);
if(errnum == SSL_ERROR_SSL ||
errnum == SSL_ERROR_SYSCALL)
if(ssl_errnum == SSL_ERROR_SSL ||
ssl_errnum == SSL_ERROR_SYSCALL)
{
while((errnum = ERR_get_error()) != 0)
while((err_errnum = ERR_get_error()) != 0)
{
ERR_error_string_n(errnum,errbuf,140);
ERR_error_string_n(err_errnum,errbuf,sizeof(errbuf));
skygw_log_write(LE,"%s",errbuf);
}
}
@ -2903,9 +2904,9 @@ int dcb_accept_SSL(DCB* dcb)
case -1:
errnum = SSL_get_error(dcb->ssl,ssl_rval);
ssl_errnum = SSL_get_error(dcb->ssl,ssl_rval);
if(errnum == SSL_ERROR_WANT_READ || errnum == SSL_ERROR_WANT_WRITE)
if(ssl_errnum == SSL_ERROR_WANT_READ || ssl_errnum == SSL_ERROR_WANT_WRITE)
{
/** Not all of the data has been read. Go back to the poll
queue and wait for more.*/
@ -2921,17 +2922,22 @@ int dcb_accept_SSL(DCB* dcb)
"Error: Fatal error in SSL_accept for %s: (SSL version: %s SSL error code: %d)",
dcb->remote,
SSL_get_version(dcb->ssl),
errnum);
if(errnum == SSL_ERROR_SSL ||
errnum == SSL_ERROR_SYSCALL)
ssl_errnum);
if(ssl_errnum == SSL_ERROR_SSL ||
ssl_errnum == SSL_ERROR_SYSCALL)
{
while((errnum = ERR_get_error()) != 0)
while((err_errnum = ERR_get_error()) != 0)
{
ERR_error_string_n(errnum,errbuf,140);
ERR_error_string_n(err_errnum,errbuf,sizeof(errbuf));
skygw_log_write(LE,
"%s",
errbuf);
}
if(errno)
{
skygw_log_write(LE, "Error: SSL authentication failed due to system"
" error %d: %s", errno, strerror_r(errno, errbuf, sizeof(errbuf)));
}
}
}
break;
@ -2965,7 +2971,7 @@ int dcb_accept_SSL(DCB* dcb)
int dcb_connect_SSL(DCB* dcb)
{
int rval,errnum;
char errbuf[140];
char errbuf[SSL_ERRBUF_LEN];
rval = SSL_connect(dcb->ssl);
switch(rval)
@ -3000,7 +3006,7 @@ int dcb_connect_SSL(DCB* dcb)
else
{
rval = -1;
ERR_error_string_n(errnum,errbuf,140);
ERR_error_string_n(errnum,errbuf,sizeof(errbuf));
skygw_log_write_flush(LE,
"Error: Fatal error in SSL_accept for %s@%s: (SSL error code: %d) %s",
dcb->user,

3713
server/core/gateway.c
View File

File diff suppressed because it is too large Load Diff

20
server/core/gw_utils.c
View File

@ -226,3 +226,23 @@ struct hostent *hp;
addr->sin_port = htons(pnum);
return 1;
}
/**
* Return the number of processors available.
* @return Number of processors or 1 if the required definition of _SC_NPROCESSORS_CONF
* is not found
*/
long get_processor_count()
{
long processors = 1;
#ifdef _SC_NPROCESSORS_ONLN
if ((processors = sysconf(_SC_NPROCESSORS_ONLN)) <= 0)
{
skygw_log_write(LE, "Unable to establish the number of available cores. Defaulting to 4.");
processors = 4;
}
#else
#error _SC_NPROCESSORS_ONLN not available.
#endif
return processors;
}

139
server/core/maxscale_pcre2.c Normal file
View File

@ -0,0 +1,139 @@
/*
* This file is distributed as part of the MariaDB Corporation MaxScale. It is free
* software: you can redistribute it and/or modify it under the terms of the
* GNU General Public License as published by the Free Software Foundation,
* version 2.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
* details.
*
* You should have received a copy of the GNU General Public License along with
* this program; if not, write to the Free Software Foundation, Inc., 51
* Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
*
* Copyright MariaDB Corporation Ab 2015
*
*/
/**
* @file maxscale_pcre2.c - Utility functions for regular expression matching
* with the bundled PCRE2 library.
*
* @verbatim
* Revision History
*
* Date Who Description
* 30-10-2015 Markus Makela Initial implementation
* @endverbatim
*/
#include <maxscale_pcre2.h>
/**
* Utility wrapper for PCRE2 library function call pcre2_substitute.
*
* This function replaces all occurences of a pattern with the provided replacement
* and places the end result into @c dest. This buffer must be allocated by the caller.
* If the size of @c dest is not large enough it will be reallocated to a larger size.
* The size of @c dest is stored in @c size if any reallocation takes place.
*
* @param re Compiled pattern to use
* @param subject Subject string
* @param replace Replacement string
* @param dest Destination buffer
* @param size Size of the desination buffer
* @return MXS_PCRE2_MATCH if replacements were made, MXS_PCRE2_NOMATCH if nothing
* was replaced or MXS_PCRE2_ERROR if memory reallocation failed
*/
mxs_pcre2_result_t mxs_pcre2_substitute(pcre2_code *re, const char *subject, const char *replace,
char** dest, size_t* size)
{
int rc;
mxs_pcre2_result_t rval = MXS_PCRE2_ERROR;
pcre2_match_data *mdata = pcre2_match_data_create_from_pattern(re, NULL);
if (mdata)
{
while ((rc = pcre2_substitute(re, (PCRE2_SPTR) subject, PCRE2_ZERO_TERMINATED, 0,
PCRE2_SUBSTITUTE_GLOBAL, mdata, NULL,
(PCRE2_SPTR) replace, PCRE2_ZERO_TERMINATED,
(PCRE2_UCHAR*) *dest, size)) == PCRE2_ERROR_NOMEMORY)
{
char *tmp = realloc(*dest, *size * 2);
if (tmp == NULL)
{
break;
}
*dest = tmp;
*size *= 2;
}
if (rc > 0)
{
rval = MXS_PCRE2_MATCH;
}
else if (rc == 0)
{
rval = MXS_PCRE2_NOMATCH;
}
pcre2_match_data_free(mdata);
}
return rval;
}
/**
* Do a simple matching of a pattern to a string.
*
* This function compiles the given pattern and checks if the subject string matches
* it.
* @param pattern Pattern used for matching
* @param subject Subject string to match
* @param options PCRE2 compilation options
* @param error The PCRE2 error code is stored here if one is available
* @return MXS_PCRE2_MATCH if @c subject matches @c pattern, MXS_PCRE2_NOMATCH if
* they do not match and MXS_PCRE2_ERROR if an error occurred. If an error occurred
* within the PCRE2 library, @c error will contain the error code. Otherwise it is
* set to 0.
*/
mxs_pcre2_result_t mxs_pcre2_simple_match(const char* pattern, const char* subject,
int options, int *error)
{
int err;
size_t erroff;
mxs_pcre2_result_t rval = MXS_PCRE2_ERROR;
pcre2_code *re = pcre2_compile((PCRE2_SPTR) pattern, PCRE2_ZERO_TERMINATED,
options, &err, &erroff, NULL);
if (re)
{
pcre2_match_data *mdata = pcre2_match_data_create_from_pattern(re, NULL);
if (mdata)
{
int rc = pcre2_match(re, (PCRE2_SPTR) subject, PCRE2_ZERO_TERMINATED,
0, 0, mdata, NULL);
if (rc == PCRE2_ERROR_NOMATCH)
{
rval = MXS_PCRE2_NOMATCH;
}
else if (rc > 0)
{
/** Since we used the pattern to create the matching data,
* pcre2_match will never return 0 */
rval = MXS_PCRE2_MATCH;
}
pcre2_match_data_free(mdata);
}
else
{
*error = 0;
}
pcre2_code_free(re);
}
else
{
*error = err;
}
return rval;
}

127
server/core/modutil.c
View File

@ -38,6 +38,19 @@ extern int lm_enabled_logfiles_bitmask;
extern size_t log_ses_count[];
extern __thread log_info_t tls_log_info;
/** These are used when converting MySQL wildcards to regular expressions */
static SPINLOCK re_lock = SPINLOCK_INIT;
static bool pattern_init = false;
static pcre2_code *re_percent = NULL;
static pcre2_code *re_single = NULL;
static pcre2_code *re_escape = NULL;
static const PCRE2_SPTR pattern_percent = (PCRE2_SPTR) "%";
static const PCRE2_SPTR pattern_single = (PCRE2_SPTR) "([^\\\\]|^)_";
static const PCRE2_SPTR pattern_escape = (PCRE2_SPTR) "[.]";
static const char* sub_percent = ".*";
static const char* sub_single = "$1.";
static const char* sub_escape = "\\.";
static void modutil_reply_routing_error(
DCB* backend_dcb,
int error,
@ -842,3 +855,117 @@ int modutil_count_statements(GWBUF* buffer)
return num;
}
/**
* Initialize the PCRE2 patterns used when converting MySQL wildcards to PCRE syntax.
*/
void prepare_pcre2_patterns()
{
spinlock_acquire(&re_lock);
if (!pattern_init)
{
int err;
size_t erroff;
PCRE2_UCHAR errbuf[STRERROR_BUFLEN];
if ((re_percent = pcre2_compile(pattern_percent, PCRE2_ZERO_TERMINATED,
0, &err, &erroff, NULL)) &&
(re_single = pcre2_compile(pattern_single, PCRE2_ZERO_TERMINATED,
0, &err, &erroff, NULL)) &&
(re_escape = pcre2_compile(pattern_escape, PCRE2_ZERO_TERMINATED,
0, &err, &erroff, NULL)))
{
assert(!pattern_init);
pattern_init = true;
}
else
{
pcre2_get_error_message(err, errbuf, sizeof(errbuf));
skygw_log_write(LE, "Error: Failed to compile PCRE2 pattern: %s",
errbuf);
}
if (!pattern_init)
{
pcre2_code_free(re_percent);
pcre2_code_free(re_single);
pcre2_code_free(re_escape);
re_percent = NULL;
re_single = NULL;
re_escape = NULL;
}
}
spinlock_release(&re_lock);
}
/**
* Check if @c string matches @c pattern according to the MySQL wildcard rules.
* The wildcard character @c '%%' is replaced with @c '.*' and @c '_' is replaced
* with @c '.'. All Regular expression special characters are escaped before
* matching is made.
* @param pattern Wildcard pattern
* @param string String to match
* @return MXS_PCRE2_MATCH if the pattern matches, MXS_PCRE2_NOMATCH if it does
* not match and MXS_PCRE2_ERROR if an error occurred
* @see maxscale_pcre2.h
*/
mxs_pcre2_result_t modutil_mysql_wildcard_match(const char* pattern, const char* string)
{
prepare_pcre2_patterns();
mxs_pcre2_result_t rval = MXS_PCRE2_ERROR;
bool err = false;
PCRE2_SIZE matchsize = strlen(string) + 1;
PCRE2_SIZE tempsize = matchsize;
char* matchstr = (char*) malloc(matchsize);
char* tempstr = (char*) malloc(tempsize);
pcre2_match_data *mdata_percent = pcre2_match_data_create_from_pattern(re_percent, NULL);
pcre2_match_data *mdata_single = pcre2_match_data_create_from_pattern(re_single, NULL);
pcre2_match_data *mdata_escape = pcre2_match_data_create_from_pattern(re_escape, NULL);
if (matchstr && tempstr && mdata_percent && mdata_single && mdata_escape)
{
if (mxs_pcre2_substitute(re_escape, pattern, sub_escape,
&matchstr, &matchsize) == MXS_PCRE2_ERROR ||
mxs_pcre2_substitute(re_single, matchstr, sub_single,
&tempstr, &tempsize) == MXS_PCRE2_ERROR ||
mxs_pcre2_substitute(re_percent, tempstr, sub_percent,
&matchstr, &matchsize) == MXS_PCRE2_ERROR)
{
err = true;
}
if (!err)
{
int errcode;
rval = mxs_pcre2_simple_match(matchstr, string, PCRE2_CASELESS, &errcode);
if (rval == MXS_PCRE2_ERROR)
{
if (errcode != 0)
{
PCRE2_UCHAR errbuf[STRERROR_BUFLEN];
pcre2_get_error_message(errcode, errbuf, sizeof(errbuf));
skygw_log_write(LE, "Error: Failed to match pattern: %s",
errbuf);
}
err = true;
}
}
}
else
{
err = true;
}
if (err)
{
skygw_log_write(LE, "Error: Fatal error when matching wildcard patterns.");
}
pcre2_match_data_free(mdata_percent);
pcre2_match_data_free(mdata_single);
pcre2_match_data_free(mdata_escape);
free(matchstr);
free(tempstr);
return rval;
}

23
server/core/service.c
View File

@ -471,7 +471,8 @@ serviceStart(SERVICE *service)
if (check_service_permissions(service))
{
if (service->ssl_mode == SSL_DISABLED || (service->ssl_mode != SSL_DISABLED && serviceInitSSL(service) != 0))
if (service->ssl_mode == SSL_DISABLED ||
(service->ssl_mode != SSL_DISABLED && serviceInitSSL(service) == 0))
{
if ((service->router_instance = service->router->createInstance(
service,service->routerOptions)))
@ -1999,11 +2000,11 @@ int *data;
}
/**
* Initialize the servce's SSL context. This sets up the generated RSA
* Initialize the service's SSL context. This sets up the generated RSA
* encryption keys, chooses the server encryption level and configures the server
* certificate, private key and certificate authority file.
* @param service
* @return
* @param service Service to initialize
* @return 0 on success, -1 on error
*/
int serviceInitSSL(SERVICE* service)
{
@ -2043,7 +2044,11 @@ int serviceInitSSL(SERVICE* service)
break;
}
service->ctx = SSL_CTX_new(service->method);
if((service->ctx = SSL_CTX_new(service->method)) == NULL)
{
skygw_log_write(LE, "Error: SSL context initialization failed.");
return -1;
}
/** Enable all OpenSSL bug fixes */
SSL_CTX_set_options(service->ctx,SSL_OP_ALL);
@ -2053,13 +2058,19 @@ int serviceInitSSL(SERVICE* service)
{
rsa_512 = RSA_generate_key(512,RSA_F4,NULL,NULL);
if (rsa_512 == NULL)
skygw_log_write(LE,"Error: 512-bit RSA key generation failed.");
{
skygw_log_write(LE,"Error: 512-bit RSA key generation failed.");
return -1;
}
}
if(rsa_1024 == NULL)
{
rsa_1024 = RSA_generate_key(1024,RSA_F4,NULL,NULL);
if (rsa_1024 == NULL)
{
skygw_log_write(LE,"Error: 1024-bit RSA key generation failed.");
return -1;
}
}
if(rsa_512 != NULL && rsa_1024 != NULL)

1
server/include/gw.h
View File

@ -88,4 +88,5 @@ int gw_getsockerrno(int fd);
int parse_bindconfig(char *, unsigned short, struct sockaddr_in *);
int setipaddress(struct in_addr *, char *);
char* get_libdir();
long get_processor_count();
#endif

1
server/include/maxconfig.h
View File

@ -20,6 +20,7 @@
#include <skygw_utils.h>
#include <stdint.h>
#include <openssl/sha.h>
#include <spinlock.h>
/**
* @file config.h The configuration handling elements
*

52
server/include/maxscale_pcre2.h Normal file
View File

@ -0,0 +1,52 @@
#ifndef _MAXSCALE_PCRE2_H
#define _MAXSCALE_PCRE2_H
/*
* This file is distributed as part of the MariaDB Corporation MaxScale. It is free
* software: you can redistribute it and/or modify it under the terms of the
* GNU General Public License as published by the Free Software Foundation,
* version 2.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
* FOR A PARTICULAR PURPOSE. See the GNU General Public License for more
* details.
*
* You should have received a copy of the GNU General Public License along with
* this program; if not, write to the Free Software Foundation, Inc., 51
* Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
*
* Copyright MariaDB Corporation Ab 2015
*
*/
#ifndef PCRE2_CODE_UNIT_WIDTH
#define PCRE2_CODE_UNIT_WIDTH 8
#endif
#include <pcre2.h>
/**
* @file maxscale_pcre2.h - Utility functions for regular expression matching
* with the bundled PCRE2 library.
*
* @verbatim
* Revision History
*
* Date Who Description
* 30-10-2015 Markus Makela Initial implementation
* @endverbatim
*/
typedef enum
{
MXS_PCRE2_MATCH,
MXS_PCRE2_NOMATCH,
MXS_PCRE2_ERROR
} mxs_pcre2_result_t;
mxs_pcre2_result_t mxs_pcre2_substitute(pcre2_code *re, const char *subject,
const char *replace, char** dest, size_t* size);
mxs_pcre2_result_t mxs_pcre2_simple_match(const char* pattern, const char* subject,
int options, int* error);
#endif

2
server/include/modutil.h
View File

@ -34,6 +34,7 @@
#include <buffer.h>
#include <dcb.h>
#include <string.h>
#include <maxscale_pcre2.h>
#define PTR_IS_RESULTSET(b) (b[0] == 0x01 && b[1] == 0x0 && b[2] == 0x0 && b[3] == 0x01)
#define PTR_IS_EOF(b) (b[0] == 0x05 && b[1] == 0x0 && b[2] == 0x0 && b[4] == 0xfe)
@ -68,4 +69,5 @@ GWBUF *modutil_create_mysql_err_msg(
const char *msg);
int modutil_count_signal_packets(GWBUF*,int,int,int*);
mxs_pcre2_result_t modutil_mysql_wildcard_match(const char* pattern, const char* string);
#endif

4
server/maxscale_template.cnf
View File

@ -3,12 +3,12 @@
# Global parameters
#
# Set the number of threads to a value equal to the number of CPU cores.
# Number of threads is autodetected, uncomment for manual configuration
# Complete list of configuration options:
# https://github.com/mariadb-corporation/MaxScale/blob/master/Documentation/Getting-Started/Configuration-Guide.md
[maxscale]
threads=4
#threads=8
# Server definitions
#

8
server/modules/include/schemarouter.h
View File

@ -28,11 +28,14 @@
*
* @endverbatim
*/
#ifndef PCRE2_CODE_UNIT_WIDTH
#define PCRE2_CODE_UNIT_WIDTH 8
#endif
#include <dcb.h>
#include <hashtable.h>
#include <mysql_client_server_protocol.h>
#include <pcre.h>
#include <pcre2.h>
/**
* Bitmask values for the router session's initialization. These values are used
* to prevent responses from internal commands being forwarded to the client.
@ -330,9 +333,10 @@ typedef struct router_instance {
HASHTABLE* ignored_dbs; /*< List of databases to ignore when the
* database mapping finds multiple servers
* with the same database */
pcre* ignore_regex; /*< Databases matching this regex will
pcre2_code* ignore_regex; /*< Databases matching this regex will
* not cause the session to be terminated
* if they are found on more than one server. */
pcre2_match_data* ignore_match_data;
} ROUTER_INSTANCE;

234
server/modules/monitor/mysql_mon.c
View File

@ -51,8 +51,8 @@
* @endverbatim
*/
#include <mysqlmon.h>
#include <modutil.h>
/** Defined in log_manager.cc */
extern int lm_enabled_logfiles_bitmask;
@ -81,7 +81,10 @@ static void set_master_heartbeat(MYSQL_MONITOR *, MONITOR_SERVERS *);
static void set_slave_heartbeat(MONITOR *, MONITOR_SERVERS *);
static int add_slave_to_master(long *, int, long);
bool isMySQLEvent(monitor_event_t event);
void check_maxscale_schema_replication(MONITOR *monitor);
static bool report_version_err = true;
static const char* hb_table_name = "maxscale_schema.replication_heartbeat";
static MONITOR_OBJECT MyObject = {
startMonitor,
stopMonitor,
@ -219,6 +222,7 @@ startMonitor(void *arg, void* opt)
{
memset(handle->events,true,sizeof(handle->events));
}
handle->tid = (THREAD)thread_start(monitorMain, monitor);
return handle;
}
@ -734,6 +738,7 @@ int num_servers=0;
MONITOR_SERVERS *root_master = NULL;
size_t nrounds = 0;
int log_no_master = 1;
bool heartbeat_checked = false;
spinlock_acquire(&mon->lock);
handle = (MYSQL_MONITOR *)mon->handle;
@ -762,6 +767,13 @@ detect_stale_master = handle->detectStaleMaster;
}
/** Wait base interval */
thread_millisleep(MON_BASE_INTERVAL_MS);
if (handle->replicationHeartbeat && !heartbeat_checked)
{
check_maxscale_schema_replication(mon);
heartbeat_checked = true;
}
/**
* Calculate how far away the monitor interval is from its full
* cycle and if monitor interval time further than the base
@ -1465,3 +1477,223 @@ bool isMySQLEvent(monitor_event_t event)
}
return false;
}
/**
* Check if replicate_ignore_table is defined and if maxscale_schema.replication_hearbeat
* table is in the list.
* @param database Server to check
* @return False if the table is not replicated or an error occurred when querying
* the server
*/
bool check_replicate_ignore_table(MONITOR_SERVERS* database)
{
MYSQL_RES *result;
bool rval = true;
if (mysql_query(database->con,
"show variables like 'replicate_ignore_table'") == 0 &&
(result = mysql_store_result(database->con)) &&
mysql_num_fields(result) > 1)
{
MYSQL_ROW row;
while ((row = mysql_fetch_row(result)))
{
if (strlen(row[1]) > 0 &&
strcasestr(row[1], hb_table_name))
{
skygw_log_write(LE, "Warning: 'replicate_ignore_table' is "
"defined on server '%s' and '%s' was found in it. ",
database->server->unique_name, hb_table_name);
rval = false;
}
}
mysql_free_result(result);
}
else
{
skygw_log_write(LE, "Error: Failed to query server %s for "
"'replicate_ignore_table': %s",
database->server->unique_name,
mysql_error(database->con));
rval = false;
}
return rval;
}
/**
* Check if replicate_do_table is defined and if maxscale_schema.replication_hearbeat
* table is not in the list.
* @param database Server to check
* @return False if the table is not replicated or an error occurred when querying
* the server
*/
bool check_replicate_do_table(MONITOR_SERVERS* database)
{
MYSQL_RES *result;
bool rval = true;
if (mysql_query(database->con,
"show variables like 'replicate_do_table'") == 0 &&
(result = mysql_store_result(database->con)) &&
mysql_num_fields(result) > 1)
{
MYSQL_ROW row;
while ((row = mysql_fetch_row(result)))
{
if (strlen(row[1]) > 0 &&
strcasestr(row[1], hb_table_name) == NULL)
{
skygw_log_write(LE, "Warning: 'replicate_do_table' is "
"defined on server '%s' and '%s' was not found in it. ",
database->server->unique_name, hb_table_name);
rval = false;
}
}
mysql_free_result(result);
}
else
{
skygw_log_write(LE, "Error: Failed to query server %s for "
"'replicate_do_table': %s",
database->server->unique_name,
mysql_error(database->con));
rval = false;
}
return rval;
}
/**
* Check if replicate_wild_do_table is defined and if it doesn't match
* maxscale_schema.replication_heartbeat.
* @param database Database server
* @return False if the table is not replicated or an error occurred when trying to
* query the server.
*/
bool check_replicate_wild_do_table(MONITOR_SERVERS* database)
{
MYSQL_RES *result;
bool rval = true;
if (mysql_query(database->con,
"show variables like 'replicate_wild_do_table'") == 0 &&
(result = mysql_store_result(database->con)) &&
mysql_num_fields(result) > 1)
{
MYSQL_ROW row;
while ((row = mysql_fetch_row(result)))
{
if (strlen(row[1]) > 0)
{
mxs_pcre2_result_t rc = modutil_mysql_wildcard_match(row[1], hb_table_name);
if (rc == MXS_PCRE2_NOMATCH)
{
skygw_log_write(LE, "Warning: 'replicate_wild_do_table' is "
"defined on server '%s' and '%s' does not match it. ",
database->server->unique_name,
hb_table_name);
rval = false;
}
}
}
mysql_free_result(result);
}
else
{
skygw_log_write(LE, "Error: Failed to query server %s for "
"'replicate_wild_do_table': %s",
database->server->unique_name,
mysql_error(database->con));
rval = false;
}
return rval;
}
/**
* Check if replicate_wild_ignore_table is defined and if it matches
* maxscale_schema.replication_heartbeat.
* @param database Database server
* @return False if the table is not replicated or an error occurred when trying to
* query the server.
*/
bool check_replicate_wild_ignore_table(MONITOR_SERVERS* database)
{
MYSQL_RES *result;
bool rval = true;
if (mysql_query(database->con,
"show variables like 'replicate_wild_ignore_table'") == 0 &&
(result = mysql_store_result(database->con)) &&
mysql_num_fields(result) > 1)
{
MYSQL_ROW row;
while ((row = mysql_fetch_row(result)))
{
if (strlen(row[1]) > 0)
{
mxs_pcre2_result_t rc = modutil_mysql_wildcard_match(row[1], hb_table_name);
if (rc == MXS_PCRE2_MATCH)
{
skygw_log_write(LE, "Warning: 'replicate_wild_ignore_table' is "
"defined on server '%s' and '%s' matches it. ",
database->server->unique_name,
hb_table_name);
rval = false;
}
}
}
mysql_free_result(result);
}
else
{
skygw_log_write(LE, "Error: Failed to query server %s for "
"'replicate_wild_do_table': %s",
database->server->unique_name,
mysql_error(database->con));
rval = false;
}
return rval;
}
/**
* Check if the maxscale_schema.replication_heartbeat table is replicated on all
* servers and log a warning if problems were found.
* @param monitor Monitor structure
*/
void check_maxscale_schema_replication(MONITOR *monitor)
{
MONITOR_SERVERS* database = monitor->databases;
bool err = false;
while (database)
{
connect_result_t rval = mon_connect_to_db(monitor, database);
if (rval == MONITOR_CONN_OK)
{
if (!check_replicate_ignore_table(database) ||
!check_replicate_do_table(database) ||
!check_replicate_wild_do_table(database) ||
!check_replicate_wild_ignore_table(database))
{
err = true;
}
}
else
{
mon_log_connect_error(database, rval);
}
database = database->next;
}
if (err)
{
skygw_log_write(LE, "Warning: Problems were encountered when "
"checking if '%s' is replicated. Make sure that the table is "
"replicated to all slaves.", hb_table_name);
}
}

32
server/modules/protocol/mysql_client.c
View File

@ -590,29 +590,29 @@ static int gw_mysql_do_authentication(DCB *dcb, GWBUF **buf) {
sizeof(protocol->scramble),
username,
stage1_hash);
}
else
{
LOGIF(LM, (skygw_log_write(LOGFILE_MESSAGE,
"%s: login attempt for user '%s', user not "
"found.",
dcb->service->name, username)));
}
}
/* Do again the database check */
auth_ret = check_db_name_after_auth(dcb, database, auth_ret);
/* Do again the database check */
auth_ret = check_db_name_after_auth(dcb, database, auth_ret);
}
}
/* on succesful auth set user into dcb field */
if (auth_ret == 0) {
dcb->user = strdup(client_data->user);
}
else
{
skygw_log_write(LOGFILE_ERROR,
"%s: login attempt for user '%s', authentication failed.",
dcb->service->name, username);
}
{
skygw_log_write(LM, "%s: login attempt for user '%s', authentication failed.",
dcb->service->name, username);
if (dcb->ipv4.sin_addr.s_addr == 0x0100007F &&
!dcb->service->localhost_match_wildcard_host)
{
skygw_log_write_flush(LM, "If you have a wildcard grant that covers"
" this address, try adding "
"'localhost_match_wildcard_host=true' for "
"service '%s'. ", dcb->service->name);
}
}
/* let's free the auth_token now */
if (auth_token) {

10
server/modules/protocol/mysql_common.c
View File

@ -1489,16 +1489,6 @@ int gw_find_mysql_user_password_sha1(char *username, uint8_t *gateway_password,
!dcb->service->localhost_match_wildcard_host)
{
/* Skip the wildcard check and return 1 */
LOGIF(LE,
(skygw_log_write_flush(
LOGFILE_ERROR,
"Error : user %s@%s not found, try set "
"'localhost_match_wildcard_host=1' in "
"service definition of the configuration "
"file.",
key.user,
dcb->remote)));
break;
}

36
server/modules/routing/schemarouter/schemarouter.c
View File

@ -41,9 +41,6 @@
/** Size of the hashtable used to store ignored databases */
#define SCHEMAROUTER_HASHSIZE 100
/** Size of the offset vector used for regex matching */
#define SCHEMA_OVEC_SIZE 24
MODULE_INFO info = {
MODULE_API_ROUTER,
MODULE_BETA_RELEASE,
@ -380,13 +377,11 @@ showdb_response_t parse_showdb_response(ROUTER_CLIENT_SES* rses, backend_ref_t*
}
else
{
const int ovec_count = SCHEMA_OVEC_SIZE;
int ovector[ovec_count];
if (!(hashtable_fetch(rses->router->ignored_dbs, data) ||
(rses->router->ignore_regex &&
pcre_exec(rses->router->ignore_regex, NULL, data,
strlen(data), 0, 0, ovector, ovec_count) >= 0)))
pcre2_match(rses->router->ignore_regex, (PCRE2_SPTR)data,
PCRE2_ZERO_TERMINATED, 0, 0,
rses->router->ignore_match_data, NULL) >= 0)))
{
duplicate_found = true;
skygw_log_write(LE, "Error: Database '%s' found on servers '%s' and '%s' for user %s@%s.",
@ -801,19 +796,36 @@ createInstance(SERVICE *service, char **options)
if((param = config_get_param(conf,"ignore_databases_regex")))
{
const char* errptr;
int erroffset;
pcre* re = pcre_compile(param->value, 0, &errptr, &erroffset, NULL);
int errcode;
PCRE2_SIZE erroffset;
pcre2_code* re = pcre2_compile((PCRE2_SPTR)param->value, PCRE2_ZERO_TERMINATED, 0,
&errcode, &erroffset, NULL);
if(re == NULL)
{
PCRE2_UCHAR errbuf[512];
pcre2_get_error_message(errcode, errbuf, sizeof(errbuf));
skygw_log_write(LE, "Error: Regex compilation failed at %d for regex '%s': %s",
erroffset, param->value, errptr);
erroffset, param->value, errbuf);
hashtable_free(router->ignored_dbs);
free(router);
return NULL;
}
pcre2_match_data* match_data = pcre2_match_data_create_from_pattern(re, NULL);
if (match_data == NULL)
{
skygw_log_write(LE, "Error: PCRE2 match data creation failed. This"
" is most likely caused by a lack of available memory.");
pcre2_code_free(re);
hashtable_free(router->ignored_dbs);
free(router);
return NULL;
}
router->ignore_regex = re;
router->ignore_match_data = match_data;
}
if((param = config_get_param(conf,"ignore_databases")))

1
utils/skygw_utils.h
View File

@ -282,7 +282,6 @@ bool is_valid_posix_path(char* path);
bool strip_escape_chars(char*);
int simple_str_hash(char* key);
EXTERN_C_BLOCK_END
#endif /* SKYGW_UTILS_H */