10988 Commits

Author SHA1 Message Date
Markus Mäkelä
616fb30818
Add basic router alteration test
The test checks that alterations to routers work and are persisted.
2018-07-11 14:08:49 +03:00
Markus Mäkelä
1e68261bce
Add missing handling of max_slave_connections
The parameter accepts both counts and percentages which requires special
handling in the router. This needs to be done when the configuration is
updated.
2018-07-11 14:08:49 +03:00
Markus Mäkelä
30d3dcfd4e
Persist router parameters
Router parameters are now correctly persisted after they have been
modified.
2018-07-11 14:08:48 +03:00
Markus Mäkelä
861b0857cd
Use correct type for transaction_replay_max_size
transaction_replay_max_size is a size, not a count.
2018-07-11 14:08:48 +03:00
Markus Mäkelä
881c49c57d
Fix mxs729_maxadmin
The results returned appear to be inverted with C++11 version of an
unordered map.
2018-07-11 14:08:47 +03:00
Markus Mäkelä
86cdb14286
Don't process queued commands when replaying transaction
If a transaction is replayed, queued commands must not be processed. The
exception to this rule is when pending session commands are executed
before the first statement in the replayed transaction is executed.
2018-07-11 14:08:47 +03:00
Markus Mäkelä
0614ff4c9d
Fix handling of transactions with large results
If transaction replaying was enabled and a result was returned in more
than one call to clientReply, a NULL value would be added to the statement
which in turn would trigger a debug assertion.

Similarly any following statements in the transaction would be executed
regardless of whether the result was complete.

Renamed the statement execution function to better describe what it does.

Extended the basic functional test case to cover this.
2018-07-11 14:08:47 +03:00
Markus Mäkelä
77a1417479
Replace TR1 headers with standard headers
Now that the C++11 standard is the default one, we can remove the TR1
headers and classes.
2018-07-11 14:08:46 +03:00
Markus Mäkelä
28ceb1bad8
Take router reconfiguration into use
The router parameters can now be reconfigured at runtime if the router
supports it. Both MaxAdmin and the REST API support it.
2018-07-11 14:08:46 +03:00
Markus Mäkelä
a5e384fd29
Add reconfiguration to router API
Added a new router API entry point that allows configuration changes after
the instance has been created. This makes alterations to most service
parameters at runtime possible.

An option to reconfiguration would have been the creation of a new
service and the eventual destruction of the old one. This would be a more
complicated and costly method but from an architectural point of view it
is interesting.

The actual implementation of the configuration change is left to the
router. Currently, only readwritesplit performs reconfiguration as
implementing it with versioned configurations is very easy.

Versioned configurations can be considered an adequate first step but it
is not an optimal solution as it causes a bottleneck in the reference
counting of the shared configuration. Thread-specific configuration
definitions would make for a more efficient solution but the
implementation is more complex.
2018-07-11 14:08:45 +03:00
Markus Mäkelä
bd4be3a97b
Use shared configurations in readwritesplit
By using a shared pointer instead of a plain object, we can replace the
router configuration without it affecting existing sessions. This is a
change that is required to enable runtime reconfiguration of
readwritesplit.
2018-07-11 14:08:45 +03:00
Marko
13763ace32 MXS-1820 Make routing of queries in schemarouter case insensitive 2018-07-09 16:18:18 +03:00
Marko
0fe5eb2743 MXS-1820 Show original case in schemarouter SHOW DATABASES queries
Original case is now used in schemarouter this means that database and
table names are now case sensitive as well.
2018-07-09 16:17:38 +03:00
Markus Mäkelä
5b9da16518
Fix order of parameters in mxs1507_migrate_trx
The switchover parameters were given in the wrong order.
2018-07-09 12:10:36 +03:00
Markus Mäkelä
b4174cba44
Wait for monitor in mxs1507_migrate_trx
The five second sleep was not adequate and replacing it with monitor
waiting should improve stability.
2018-07-09 12:10:36 +03:00
Markus Mäkelä
5c4926b899
Use C++11 RNG in mysqlmon tests
The glib re-entrant random number generation functions crashe on CentOS 7
for no apparent reason. As the C++11 random number library provides a more
versatile, widely supported and simple method of acquiring random numbers,
it should be used instead.
2018-07-09 12:10:36 +03:00
Markus Mäkelä
e6cf20ea29
Fix mmmon hang
The iteration of servers would never exit.
2018-07-09 12:10:36 +03:00
Markus Mäkelä
9d94230237
Assign status bits only for running servers
In previously the status bits were assigned only for running servers. Due
to the changes done in the monitoring algorithm, the slave and master
status bits are assigned to servers that are down. This change broke a
number of tests and deviates from previous behavior.

To keep the old behavior and to fix the test, the status bits are not
assigned to servers that are down.
2018-07-09 12:10:36 +03:00
Markus Mäkelä
f1905d33b1
Fix MaxCtrl list servers test
The test did not expect GTIDs to be displayed.
2018-07-09 12:10:36 +03:00
Markus Mäkelä
a11d83679e
Fix REST API regression
The threads resource was not pass the schema validation test; no ID field
was defined.
2018-07-09 12:10:36 +03:00
Markus Mäkelä
8470da9313
Fix failing tests
Changed mxs1719 to expect a failure with the query and added monitor waits
to mxs359_read_only.
2018-07-09 12:06:48 +03:00
Marko
40d076b4ce MXS-1849 Fix broken test to work with new schemarouter 2018-07-09 12:00:40 +03:00
Marko
89e83fab4b MXS-1849 Add basic test case 2018-07-09 11:55:54 +03:00
Marko
293279366f MXS-1849 Fix handling of show tables queries in schemarouter
There was a bug in handling of the "show tables from" queries that
caused multiple results to the client. These queries are now handled
correctly.
2018-07-09 11:34:59 +03:00
Markus Mäkelä
bfd3d2975d
Merge branch '2.2' into develop 2018-07-06 14:56:19 +03:00
Johan Wikman
f2b8487577 MXS-1624 Add configuration parameter
With the global configuration parameter 'query_classifier_cache'
the query classification cache can be turned on. At the moment it
does not matter what value it has; its presence simply enables the
caching.

Eventually you will be able to specify how much memory the cache
is allowed to consume.
2018-07-06 12:12:31 +03:00
Johan Wikman
e7913cc022 MXS-1624 Update qc_setup() prototype
Now takes a structure that, if present, enables the query
classification caching and specifies the properties of the
cache.

For the time being no actual properties are yet available.
2018-07-06 12:12:31 +03:00
Johan Wikman
b7c5abf73b MXS-1624 Discard QC result if the sql mode has changed
The sql mode affects the result of the query classification.
Consequently, we cannot use the cached result if it was generated
when the sql mode was something else than what it currently is.
2018-07-06 12:12:31 +03:00
Johan Wikman
1be9528227 MXS-1624 Provide initial implemenation of QC caching
The mapping from a canonical statement to the query classification
result is maintained by the class QCInfoCache of which there exist
an instance per thread. That way no locking is needed but the
information will be cached multiple times (but that is a smaller
price to pay). Currently the information is stored in a regular
std::unordered_map, which means that the consumed amount of
memory will just keep on growing unless the number of canonical
statements used by clients happens to have an upper bound.

The LRU cache (that provides means for putting a bound on the
amount of memory used and number of items) used in the cache filter
will be generalized and be taken into use here as well.

The key is now the canonical statement itself, which means that
a fair amount of memory will be used. To preserve memory it might
make sense to use a hashed value instead, although that at least
in principle opens up the possibility for unintended collisions.

This feature will also be made configurable.
2018-07-06 12:12:31 +03:00
Johan Wikman
a044929661 MXS-1624 Allow the closing of an info object
If an info object can be dupped, it must also be possible to
close it.
2018-07-06 12:12:31 +03:00
Johan Wikman
295bae5673 MXS-1624 Implement qc_dup
Only for qc_sqlite.

After a second look qc_mysqlembedded will not support dupping
the statement information. Without additional changes, simply stashing
an info object away, parsing another new GWBUF, deleting that and
then using the stashed away info object will not work; the THD object
will be corrupted. As qc_mysqlembedded is _only_ used for verifying the
sqlite-based parser this is not important anyway.
2018-07-06 12:12:31 +03:00
Johan Wikman
4e168f36f7 MXS-1624 Make the QC info object explicit
The query classifier stores information about the statement carried
by a GWBUF in the GWBUF itself. We need to be able to store that
object out side the lifetime of the GWBUF. So, we require that a
query classifier is capable of duplicating references to that object.
2018-07-06 12:12:31 +03:00
Johan Wikman
d01db42679 MXS-1624 Clean up query_classifier.cc
Minor cleanup:
- Unit variables places in anonymous namespace.
- Unit variables accessed via this_unit struct.

This is the first commit of many where the mapping from canonical
statement to query classification result is introduced.

The mapping is placed above the actual query classifier, so that all
query classifiers will benefit from it.

The mapping will be maintained by thread so that there will not be
any synchronization issues. Further, initially a simple map without
upper boundary will be used; if this is found to provide measurable
benefits, the map will then be replaced with an LRU mechanism so
that it becomes possible to specify just how much memory the mapping
may use.
2018-07-06 12:12:31 +03:00
Johan Wikman
c89bdb9626 MXS-1823 Replace meaningless eventq info with meaningful
The evq_length file held the returned number of descriptors from
the last epoll_wait() call. As such it is highly temporal and not
particularly meaningful.

That has now been removed and the instead the average number of
returned descriptors is maintained. That information changes slowly
and thus carries some meaning.
2018-07-06 11:22:42 +03:00
Johan Wikman
d339b89990 MXS-1774 Make rejection of functions optional
It is now possible to prevent the masking filter from rejecting
statements using functions in conjunction with fields to be
masked. So now it is possible to not use the blanket rejection
of the masking filter and replace it with more detailed firewall
rules.
2018-07-06 11:20:31 +03:00
Johan Wikman
43cc6ca3cd MXS-1774 Reject query with masked columns + functions
The masking filter works only on the result-set. However, if
functions are used, the column names will not be available in
the result-set, and hence masking will not take place.

Now, the statement is checked and if functions are used in
conjunction with columns that should be masked, the statement
is rejected. Thus, functions can no longer be used for bypassing
the masking. That was possible earlier as well, but required
manually setting up the firewall filter.
2018-07-06 11:20:31 +03:00
Markus Mäkelä
cb05199bac
Stop MaxScale before fixing replication
Before the state of the backend servers is checked, MaxScale needs to be
stopped to prevent the automated failover from interfering in the start-up
process.
2018-07-05 21:03:57 +03:00
Markus Mäkelä
702f8aaed4
Merge branch '2.2' into develop 2018-07-05 13:53:14 +03:00
Markus Mäkelä
5c610503ba
MXS-1949: Add test case
Added a test case that verifies the bug is fixed.
2018-07-05 13:47:08 +03:00
Markus Mäkelä
f44d305a46
MXS-1949: Fix user injection warning
If a service has no active servers and users are injected, a warning would
be logged. This is a misleading warning if the service has no servers and
should only be logged if the failure to load any users is an unexpected
situation.
2018-07-05 12:49:46 +03:00
Markus Mäkelä
33fa9b26fe
Make version counter in mlist_t volatile
The log manager is the only one that uses the mlist_t versioned list. The
counter that keeps track of the version number was not modified using
atomic operations meaning that the compiler is free to optimize away parts
of the lock-free versioning mechanism that uses it.

To prevent this optimization, the variable is declared volatile. A rewrite
is direly needed but it cannot be done in 2.2.
2018-07-05 12:47:28 +03:00
Markus Mäkelä
f5110209f7
MXS-1942: Use MaxScale version in MaxCtrl
Displaying the MaxScale version helps identify which package the
executable was bundled with. As the MaxCtrl source is a part of MaxScale,
there's no need for separate versioning.
2018-07-05 12:47:28 +03:00
Markus Mäkelä
9e039ed024
Fix typo in DEB install scripts
The package suffix was wrong.
2018-07-05 12:01:38 +03:00
Markus Mäkelä
7954de0da6
Always log commit ID
There's no good reason to not include the commit ID in release build
startup messages.
2018-07-05 12:01:38 +03:00
Markus Mäkelä
bbf3909296
MXS-1958: Add test case
The test case verifies that the problem exists.
2018-07-05 12:01:38 +03:00
Markus Mäkelä
d301109b1a
Update release document
Use more precise git commands for pushing a single tag.
2018-07-05 12:01:38 +03:00
Johan Wikman
152eed7f63 Merge branch '2.2' into develop 2018-07-05 09:45:18 +03:00
Johan Wikman
f1d7303919 Update 2.2 version number 2018-07-05 09:44:13 +03:00
Johan Wikman
4f634c5010 Merge branch '2.2' into develop 2018-07-05 09:41:00 +03:00
Johan Wikman
7765aff74b Update 2.2.11 release date 2018-07-05 09:38:09 +03:00