hcq/MaxScale
hcq/MaxScale
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch '2.1' into develop

Browse Source
This commit is contained in:
Markus Mäkelä 2017-02-08 09:32:18 +02:00
commit 5cd29c4220
16 changed files with 120 additions and 179 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
Documentation/About/Limitations.md
View File

@ -231,5 +231,7 @@ executed in separate parts.
query will be routed to the first available server. This possibly returns an
error about database rights instead of a missing database.
* As text protocol prepared statements are relatively rare, prepared statements
are not supported in schemarouter.
* The preparation of a prepared statement is routed to all servers. The
execution of a prepared statement is routed to the first available server or to
the server pointed by a routing hint attached to the query. In practice this
means that prepared statements aren't supported by the schemarouter.

17
Documentation/Changelog.md
View File

@ -1,5 +1,22 @@
#Changelog
## MariaDB MaxScale 2.1
* Hierarchical configuration files are now supported.
* Logging is now performed in a way compatible with logrotate(8).
* Persistent connections are reset upon resuse.
* Galera monitor now consistently chooses the same node as master.
* Galera Monitor can set the preferred donor nodes list.
* The configuration can now be altered dynamically and the changes are persisted.
* There is now a monitor for Amazon Aurora clusters.
* MySQL Monitor now has a multi-master mode.
* MySQL Monitor now has a failover mode.
* Named Server Filter now supports wildcards for source option.
* Binlog Server can now be configured to encrypt binlog files.
* New filters, _cache, _ccrfilter_, _masking_, and _maxrows_ are introduced.
For more details, please refer to:
* [MariaDB MaxScale 2.1.0 Release Notes](Release-Notes/MaxScale-2.1.0-Release-Notes.md)
## MariaDB MaxScale 2.0
* MariaDB MaxScale is licensed under MariaDB BSL.
* SSL can be used in the communication between MariaDB MaxScale and the backend servers.

36
Documentation/Getting-Started/Configuration-Guide.md
View File

@ -589,42 +589,6 @@ filters=counter | QLA
The requests pass through the filters from left to right in the order defined in
the configuration parameter.
#### `monitor`
The monitor parameter tells the service which cluster of servers to use. The
value of the option is the name of a monitor object. This removes the redundant
definition of servers in both services and monitors.
If both the `monitor` and `servers` parameters are defined, only the `monitor`
parameter will be used.
Only one cluster can be defined for a service. Multi-cluster setups should use
the `servers` parameter to list the servers that the service should use.
Here is an excerpt from an example configuration with a service that defines the
cluster parameter.
```
[db-cluster-1-monitor]
type=monitor
module=mysqlmon
servers=server1,server2,server3
user=maxuser
passwd=maxpwd
monitor_interval=1000
[my-readwrite-service]
type=service
router=readwritesplit
monitor=db-cluster-1-monitor
user=maxuser
passwd=maxpwd
```
The _db-cluster-1-monitor_ cluster consists of three servers monitored by a
_mysqlmon_ monitor module. The _my-readwrite-service_ uses this monitor as the
source for servers.
#### `servers`
The servers parameter in a service definition provides a comma separated list of

2
Documentation/Release-Notes/MaxScale-2.0.4-Release-Notes.md
View File

@ -1,4 +1,4 @@
# MariaDB MaxScale 2.0.4 Release Notes
# MariaDB MaxScale 2.0.4 Release Notes -- 2017-02-01
Release 2.0.4 is a GA release.

62
Documentation/Release-Notes/MaxScale-2.1.0-Release-Notes.md
View File

@ -3,7 +3,7 @@
Release 2.1.0 is a Beta release.
This document describes the changes in release 2.1.0, when compared to
release 2.0.X.
release 2.0.4.
For any problems you encounter, please consider submitting a bug
report at [Jira](https://jira.mariadb.org).
@ -71,8 +71,8 @@ MaxScale to rotate the the log file.
This behaviour is now compatible with logrotate(8).
Further, if MaxScale is configured to use shared memory for the log file,
the file is created into the directory "/dev/shm/maxscale". Earlier the
log file was created into the directory "/dev/shm/maxscale.PID", where PID
the file is created into the directory `/dev/shm/maxscale`. Earlier the
log file was created into the directory `/dev/shm/maxscale.PID`, where PID
was the pid of the MaxScale process.
In addition, there is now a mechanism that prevents the flooding of the log, in
@ -132,12 +132,11 @@ For more details, read the [Galeramon documentation](../Monitors/Galera-Monitor.
MaxAdmin now defaults to Emacs editing mode instead of VIM. To activate
with VIM-mode start MaxAdmin with option -i.
### Support for text wildcards in hostnames
### Named Server Filter
The source option can now handle wildcards such as:
192.168.%.%
Wildcards in text-form hostnames in the mysql.user-table of the backend
database are now supported. For matching, the client IP-address must be
converted to text-form, which requires a reverse DNS lookup. Please see
[Limitations](../About/Limitations.md) for more information.
For more details, read the [Named Server Filter documentation](../Filters/Named-Server-Filter.md).
## New Features
@ -189,15 +188,6 @@ allowing runtime tuning of parameters.
- `destroy monitor`: Destroy a created monitor
- `alter monitor`: Alter monitor parameters
### Monitors as server sources for services
Services now accept a `monitor` parameter which can be used to point a service
to a cluster of servers that are monitored by a monitor. The value of the
`monitor` parameter should be the name of a monitor in the configuration name.
For more details, refer to the [Configuration
Guide](../Getting-Started/Configuration-Guide.md#cluster) section on clusters.
### Module commands
Introduced in MaxScale 2.1, the module commands are special, module-specific
@ -281,10 +271,46 @@ to large sets of data with a single query.
For more information, refer to the [Maxrows](../Filters/Maxrows.md)
documentation.
### Galeramon Monitor new option
The `set_donor_nodes` option allows the setting of _global variable_ _wsrep_sst_donor_ with a list the preferred donor nodes (among slave ones).
For more details, read the [Galeramon documentation](../Monitors/Galera-Monitor.md).
### Binlog Server encrypted binlogs
The binlog server can optionally encrypt the events received from the master server: the setup requires MariaDB 10.1 master (with Encryption active) and the `mariadb10-compatibility=1` option set.
For more details, read the [Binlogrouter documentation](../Routers/Binlogrouter.md).
## Bug fixes
[Here is a list of bugs fixed since the release of MaxScale 2.0.X.](https://jira.mariadb.org/browse/MXS-739?jql=project%20%3D%20MXS%20AND%20issuetype%20%3D%20Bug%20AND%20resolution%20in%20(Fixed%2C%20Done)%20AND%20fixVersion%20%3D%202.0.0)
[Here is a list of bugs fixed since the release of MaxScale 2.0.4.](https://jira.mariadb.org/browse/MXS-951?jql=project%20%3D%20MXS%20AND%20issuetype%20%3D%20Bug%20AND%20resolution%20in%20(Fixed%2C%20Done)%20AND%20fixVersion%20%3D%202.1.0%20AND%20fixVersion%20!%3D%202.0.1%20AND%20fixVersion%20!%3D%202.0.2%20AND%20fixVersion%20!%3D%202.0.3%20AND%20fixVersion%20!%3D%202.0.4)
* [MXS-1025](https://jira.mariadb.org/browse/MXS-1025) qc_sqlite always reports " Statement was parsed, but not classified"
* [MXS-977](https://jira.mariadb.org/browse/MXS-977) MaxAdmin show monitor output missing formatting
* [MXS-951](https://jira.mariadb.org/browse/MXS-951) Using utf8mb4 on galera hosts stops maxscale connections
* [MXS-889](https://jira.mariadb.org/browse/MXS-889) "Ungrade Test" Jenkins job fails with CeentOS/RHEL 5 and SLES11
* [MXS-887](https://jira.mariadb.org/browse/MXS-887) create_env Jenkin job fails
* [MXS-873](https://jira.mariadb.org/browse/MXS-873) Changing server status via maxadmin is not atomic
* [MXS-832](https://jira.mariadb.org/browse/MXS-832) Problem with Regex filter as readconnroute doesn't wait for complete packets
* [MXS-831](https://jira.mariadb.org/browse/MXS-831) new_master event not triggered by galeramon
* [MXS-828](https://jira.mariadb.org/browse/MXS-828) Remove "Syslog logging is disabled." to stdout when starting without syslog
* [MXS-825](https://jira.mariadb.org/browse/MXS-825) --execdir option does not work
* [MXS-805](https://jira.mariadb.org/browse/MXS-805) Server weights don't work with LEAST_BEHIND_MASTER
* [MXS-804](https://jira.mariadb.org/browse/MXS-804) Grants for user@IP/Netmask doesn't work
* [MXS-799](https://jira.mariadb.org/browse/MXS-799) fatal signal 11 when socket could not be opened
* [MXS-769](https://jira.mariadb.org/browse/MXS-769) Malloc return value must be checked.
* [MXS-711](https://jira.mariadb.org/browse/MXS-711) All service ports use the same user data
* [MXS-650](https://jira.mariadb.org/browse/MXS-650) Connection attempt w/o SSL to SSL service gives confusing error
* [MXS-626](https://jira.mariadb.org/browse/MXS-626) Don't log anything to maxlog until it is known whether that is wanted.
* [MXS-590](https://jira.mariadb.org/browse/MXS-590) MaxScale doesn't log an error when .secrets file is not owned by current user
* [MXS-586](https://jira.mariadb.org/browse/MXS-586) Tee filter hangs when using range
* [MXS-576](https://jira.mariadb.org/browse/MXS-576) Maxscale does not generate warning/error if incorrect values is set for persistpoolmax
* [MXS-397](https://jira.mariadb.org/browse/MXS-397) Unsafe handling of dcb_readqueue
* [MXS-390](https://jira.mariadb.org/browse/MXS-390) Lack of checks of dynamic memory allocation
* [MXS-350](https://jira.mariadb.org/browse/MXS-350) Return value of realloc must not be assigned to provided pointer.
* [MXS-348](https://jira.mariadb.org/browse/MXS-348) Incorrect use of strncat
* [MXS-253](https://jira.mariadb.org/browse/MXS-253) Use of strncpy is dangerous
* [MXS-126](https://jira.mariadb.org/browse/MXS-126) debug assert in TEE filter test
## Known Issues and Limitations

4
include/maxscale/service.h
View File

@ -90,7 +90,7 @@ typedef struct server_ref_t
#define SERVICE_MAX_RETRY_INTERVAL 3600 /*< The maximum interval between service start retries */
/** Value of service timeout if timeout checks are disabled */
#define SERVICE_NO_SESSION_TIMEOUT LONG_MAX
#define SERVICE_NO_SESSION_TIMEOUT 0
/**
* Parameters that are automatically detected but can also be configured by the
@ -145,7 +145,7 @@ typedef struct service
SERVICE_REFRESH_RATE rate_limit; /**< The refresh rate limit for users table */
MXS_FILTER_DEF **filters; /**< Ordered list of filters */
int n_filters; /**< Number of filters */
long conn_idle_timeout; /**< Session timeout in seconds */
uint64_t conn_idle_timeout; /**< Session timeout in seconds */
char *weightby; /**< Service weighting parameter name */
struct service *next; /**< The next service in the linked list */
bool retry_start; /**< If starting of the service should be retried later */

89
server/core/dcb.c
View File

@ -1915,14 +1915,11 @@ dprintOneDCB(DCB *pdcb, DCB *dcb)
}
}
/*
* @brief Print DCB list statistics
*
* @param pdcb DCB to print results to
*/
void
dprintDCBList(DCB *pdcb)
static bool dprint_all_dcbs_cb(DCB *dcb, void *data)
{
DCB *pdcb = (DCB*)data;
dprintOneDCB(pdcb, dcb);
return true;
}
/**
@ -1930,23 +1927,19 @@ dprintDCBList(DCB *pdcb)
*
* @param pdcb DCB to print results to
*/
void
dprintAllDCBs(DCB *pdcb)
void dprintAllDCBs(DCB *pdcb)
{
dcb_foreach(dprint_all_dcbs_cb, pdcb);
}
int nthr = config_threadcount();
for (int i = 0; i < nthr; i++)
{
spinlock_acquire(&all_dcbs_lock[i]);
for (DCB *dcb = all_dcbs[i]; dcb; dcb = dcb->thread.next)
{
dprintOneDCB(pdcb, dcb);
}
spinlock_release(&all_dcbs_lock[i]);
}
static bool dlist_dcbs_cb(DCB *dcb, void *data)
{
DCB *pdcb = (DCB*)data;
dcb_printf(pdcb, " %-16p | %-26s | %-18s | %s\n",
dcb, gw_dcb_state2string(dcb->state),
((dcb->session && dcb->session->service) ? dcb->session->service->name : ""),
(dcb->remote ? dcb->remote : ""));
return true;
}
/**
@ -1962,24 +1955,24 @@ dListDCBs(DCB *pdcb)
dcb_printf(pdcb, " %-16s | %-26s | %-18s | %s\n",
"DCB", "State", "Service", "Remote");
dcb_printf(pdcb, "------------------+----------------------------+--------------------+----------\n");
dcb_foreach(dlist_dcbs_cb, pdcb);
dcb_printf(pdcb, "------------------+----------------------------+--------------------+----------\n\n");
}
int nthr = config_threadcount();
static bool dlist_clients_cb(DCB *dcb, void *data)
{
DCB *pdcb = (DCB*)data;
for (int i = 0; i < nthr; i++)
if (dcb->dcb_role == DCB_ROLE_CLIENT_HANDLER)
{
spinlock_acquire(&all_dcbs_lock[i]);
for (DCB *dcb = all_dcbs[i]; dcb; dcb = dcb->thread.next)
{
dcb_printf(pdcb, " %-16p | %-26s | %-18s | %s\n",
dcb, gw_dcb_state2string(dcb->state),
((dcb->session && dcb->session->service) ? dcb->session->service->name : ""),
(dcb->remote ? dcb->remote : ""));
}
spinlock_release(&all_dcbs_lock[i]);
dcb_printf(pdcb, " %-15s | %16p | %-20s | %10p\n",
(dcb->remote ? dcb->remote : ""),
dcb, (dcb->session->service ?
dcb->session->service->name : ""),
dcb->session);
}
dcb_printf(pdcb, "------------------+----------------------------+--------------------+----------\n\n");
return true;
}
/**
@ -1995,27 +1988,7 @@ dListClients(DCB *pdcb)
dcb_printf(pdcb, " %-15s | %-16s | %-20s | %s\n",
"Client", "DCB", "Service", "Session");
dcb_printf(pdcb, "-----------------+------------------+----------------------+------------\n");
int nthr = config_threadcount();
for (int i = 0; i < nthr; i++)
{
spinlock_acquire(&all_dcbs_lock[i]);
for (DCB *dcb = all_dcbs[i]; dcb; dcb = dcb->thread.next)
{
if (dcb->dcb_role == DCB_ROLE_CLIENT_HANDLER)
{
dcb_printf(pdcb, " %-15s | %16p | %-20s | %10p\n",
(dcb->remote ? dcb->remote : ""),
dcb, (dcb->session->service ?
dcb->session->service->name : ""),
dcb->session);
}
}
spinlock_release(&all_dcbs_lock[i]);
}
dcb_foreach(dlist_clients_cb, pdcb);
dcb_printf(pdcb, "-----------------+------------------+----------------------+------------\n\n");
}
@ -3512,7 +3485,9 @@ void dcb_process_idle_sessions(int thr)
{
MXS_SESSION *session = dcb->session;
if (session->service && session->client_dcb && session->client_dcb->state == DCB_STATE_POLLING &&
if (session->service && session->client_dcb &&
session->client_dcb->state == DCB_STATE_POLLING &&
session->service->conn_idle_timeout &&
hkheartbeat - session->client_dcb->last_read > session->service->conn_idle_timeout * 10)
{
poll_fake_hangup_event(dcb);

11
server/core/poll.c
View File

@ -1522,17 +1522,6 @@ void poll_fake_hangup_event(DCB *dcb)
poll_add_event_to_dcb(dcb, NULL, ev);
}
/**
* Print the event queue contents
*
* @param pdcb The DCB to print the event queue to
*/
void
dShowEventQ(DCB *pdcb)
{
}
/**
* Print the event queue statistics
*

19
server/core/server.c
View File

@ -597,7 +597,7 @@ spin_reporter(void *dcb, char *desc, int value)
}
/**
* Diagnostic to print all DCBs in persistent pool for a server
* Diagnostic to print number of DCBs in persistent pool for a server
*
* @param pdcb DCB to print results to
* @param server SERVER for which DCBs are to be printed
@ -605,22 +605,7 @@ spin_reporter(void *dcb, char *desc, int value)
void
dprintPersistentDCBs(DCB *pdcb, const SERVER *server)
{
DCB *dcb;
int nthr = config_threadcount();
for (int i = 0; i < nthr; i++)
{
#if SPINLOCK_PROFILE
dcb_printf(pdcb, "DCB List Spinlock Statistics:\n");
spinlock_stats(&server->persistlock, spin_reporter, pdcb);
#endif
dcb = server->persistent[i];
while (dcb)
{
dprintOneDCB(pdcb, dcb);
dcb = dcb->nextpersistent;
}
}
dcb_printf(pdcb, "Number of persistent DCBs: %d\n", server->stats.n_persistent);
}
/**

10
server/core/session.c
View File

@ -460,16 +460,6 @@ printAllSessions()
dcb_foreach(printAllSessions_cb, NULL);
}
/*
* @brief Print session list statistics
*
* @param pdcb DCB to print results to
*/
void
dprintSessionList(DCB *pdcb)
{
}
/** Callback for dprintAllSessions */
bool dprintAllSessions_cb(DCB *dcb, void *data)
{

3
server/modules/filter/cache/cachefiltersession.cc vendored
View File

@ -404,9 +404,8 @@ int CacheFilterSession::handle_expecting_response()
switch ((int)MYSQL_GET_COMMAND(header))
{
case MYSQL_REPLY_OK:
case MYSQL_REPLY_ERR:
store_result();
case MYSQL_REPLY_ERR:
rv = send_upstream();
m_state = CACHE_IGNORING_RESPONSE;
break;

2
server/modules/routing/avrorouter/avro.c
View File

@ -1074,7 +1074,7 @@ void converter_func(void* data)
/** We reached end of file, flush unwritten records to disk */
if (router->task_delay == 1)
{
avro_flush_all_tables(router, true);
avro_flush_all_tables(router, AVROROUTER_FLUSH);
avro_save_conversion_state(router);
}

9
server/modules/routing/avrorouter/avro_file.c
View File

@ -68,7 +68,12 @@ bool avro_open_binlog(const char *binlogdir, const char *file, int *dest)
if ((fd = open(path, O_RDONLY)) == -1)
{
MXS_ERROR("Failed to open binlog file %s.", path);
if (errno != ENOENT)
{
char err[MXS_STRERROR_BUFLEN];
MXS_ERROR("Failed to open binlog file %s: %d, %s", path, errno,
strerror_r(errno, err, sizeof(err)));
}
return false;
}
@ -745,7 +750,7 @@ avro_binlog_end_t avro_read_all_events(AVRO_INSTANCE *router)
router->trx_count >= router->trx_target)
{
update_used_tables(router);
avro_flush_all_tables(router, false);
avro_flush_all_tables(router, AVROROUTER_SYNC);
avro_save_conversion_state(router);
notify_all_clients(router);
total_rows += router->row_count;

1
server/modules/routing/avrorouter/avro_rbr.c
View File

@ -272,7 +272,6 @@ bool handle_row_event(AVRO_INSTANCE *router, REP_HEADER *hdr, uint8_t *ptr)
/** There should always be a table map event prior to a row event.
* TODO: Make the active_maps dynamic */
TABLE_MAP *map = router->active_maps[table_id % sizeof(router->active_maps)];
ss_dassert(map);
if (map)
{

10
server/modules/routing/avrorouter/avro_schema.c
View File

@ -479,7 +479,15 @@ static const char *extract_field_name(const char* ptr, char* dest, size_t size)
if (ptr > start)
{
/** Valid identifier */
snprintf(dest, size, "%.*s", (int)(ptr - start), start);
size_t bytes = ptr - start;
if (bt)
{
bytes--;
}
memcpy(dest, start, bytes);
dest[bytes] = '\0';
make_valid_avro_identifier(dest);
ptr = next_field_definition(ptr);

18
server/modules/routing/debugcli/debugcmd.c
View File

@ -134,12 +134,6 @@ struct subcommand showoptions[] =
{0, 0, 0}
},
#endif
{
"dcblist", 0, 0, dprintDCBList,
"Show DCB statistics",
"Show statistics for the list of all DCBs(descriptor control blocks)",
{0}
},
{
"dcbs", 0, 0, dprintAllDCBs,
"Show all DCBs",
@ -166,12 +160,6 @@ struct subcommand showoptions[] =
"Show the epoll polling system statistics",
{0, 0, 0}
},
{
"eventq", 0, 0, dShowEventQ,
"Show event queue",
"Show the queue of events waiting to be processed",
{0, 0, 0}
},
{
"eventstats", 0, 0, dShowEventStats,
"Show event queue statistics",
@ -262,12 +250,6 @@ struct subcommand showoptions[] =
"Show a single session in MaxScale, e.g. show session 5",
{ARG_TYPE_SESSION, 0, 0}
},
{
"sessionlist", 0, 0, dprintSessionList,
"Show session list statistics",
"Show statistics for the list of all sessions",
{0, 0, 0}
},
{
"sessions", 0, 0, dprintAllSessions,
"Show all sessions",