8152 Commits

Author SHA1 Message Date
Markus Mäkelä
aabbf64991 MXS-1300: Add server state manipulation test
The test sets and clears server states and verifies it via the REST
API. It also checks that only correct arguments are accepted.
2017-07-18 11:37:18 +03:00
Markus Mäkelä
18f05578c3 MXS-1300: Disable all output with --quiet
As the option is currently only used for testing, it's better to have it
produce no output at all.
2017-07-18 11:37:18 +03:00
Markus Mäkelä
738dc48df7 MXS-1300: Add basic maxctrl tests
Added basic tests for maxctrl. They simply test that the diagnostic
commands work.
2017-07-18 11:37:18 +03:00
Markus Mäkelä
d938dcc701 MXS-1300: Make output optional
Minor refactoring to the core library to allow multiple calls from within
the same program.

Added --quiet option to silence output so that tests aren't so
verbose. Currently this only works on UNIX based systems.
2017-07-18 11:37:18 +03:00
Markus Mäkelä
ec5b0fea39 MXS-1300: Use native promises for requests
The request-promise-native package adds support for request with native
promises. This is a convenient way to synchronize multiple HTTP requests.

The functions are changed to use promises to make testing easier. With
promises, the testing code can use the chai-as-promised library.

There are a few cases where doRequest is called with a callback that again
calls doAsyncRequest. With multiple hosts, this causes each command to be
propagated to all servers. This is a design flaw of the current multi-host
mode and needs to be changed.

Changed maxctrl.js to wait on the promises to silence some warnings if the
promise rejections are ignored.
2017-07-18 11:37:18 +03:00
Markus Mäkelä
b54e94ce95 MXS-1300: Add TSV output to MaxCtrl
TSV output is a lot easier to parse compared to the pretty-printed
Unicode.
2017-07-18 11:37:18 +03:00
Markus Mäkelä
816e317485 MXS-1300: Make MaxCtrl more like a library
All invocations into the common helper functions are now done through the
`maxctrl` function. This allows the program arguments to be passed to the
core as a parameter instead of reading them from global variables.

This helps with the implicit initialization that was done when the yargs
library was required which caused duplicated output.

Refactored the core functions so that they only process the argument
vector. The parsing of the argument vector is done in maxctrl.js where it
is more appropriate.
2017-07-18 11:37:18 +03:00
Markus Mäkelä
67b2654f50 Move REST API test scripts into a common directory
Moved and renamed the starting and stopping scripts from the REST API
tests to a common directory. This way the MaxCtrl tests can use the same
scripts to start and stop MaxScale.

Also moved the test configuration file into the `test/` directory and
changed some of the default directory locations.
2017-07-18 11:37:18 +03:00
Markus Mäkelä
173a97ae70 Refactor REST API test script into a npm test framework
The script can now be used to run an arbitrary set of Node.js tests as
long as they define the `test` npm target. Refactored REST API tests to
fit into this framework.
2017-07-18 11:37:18 +03:00
Markus Mäkelä
01fa1a827a MXS-1300: Move the REST API tests back into the core
As the REST API is a part of the core, it is more appropriate for the
tests to reside there as well. Further refactoring of the testing needs to
be done to allow multiple components to use the same framework but with
different tests.
2017-07-18 11:37:18 +03:00
Markus Mäkelä
a082871726 MXS-1300: Refactor the way maxctrl is invoked
Making the invocation explicit should allow testing without actually
starting the process fromt the command line. Also returning a promise
allows chaining of commands for testing and verification.
2017-07-18 11:37:18 +03:00
Markus Mäkelä
6c601955d6 MXS-1300: Build MaxCtrl if npm was found
If npm is installed on the local system, try to build MaxCtrl as a part of
the core.
2017-07-18 11:37:18 +03:00
Markus Mäkelä
d3f45f0e0e MXS-1300: Speed up REST API testing
The MariaDB servers are now started first. This will give them time to
prepare while other parts of the tests are being prepared.
2017-07-18 11:37:18 +03:00
Markus Mäkelä
4dcefed974 MXS-1300: Fix test server creation
Before creating the users, the master will do a RESET MASTER to remove the
automatically generated binlogs. The servers should now start a lot faster
since they no longer need to replicate the large amounts of data that were
in these generated binlogs.
2017-07-18 11:37:18 +03:00
Markus Mäkelä
16ed95d865 MXS-1220: Fix REST API tests
The service and server tests were deleting the relationships object and
posting a body without it instead of posting an empty relationships object
to delete relationships.
2017-07-18 11:37:18 +03:00
Markus Mäkelä
6479788283 MXS-1300: Add docker based cluster for REST API testing
The REST API test suite now starts a four node MariaDB cluster with
docker-compose. This allows quick and easy local testing.
2017-07-18 11:37:18 +03:00
Markus Mäkelä
d59d3ab638 MXS-1300: Combine REST API and MaxCtrl tests
The REST API tests are now located under the maxctrl directory. This
allows both tests to use the same framework for testing.
2017-07-18 11:37:18 +03:00
Markus Mäkelä
f83d793704 MXS-1300: Split the code into core and cli parts
The cli part of the code just passes the process argument vector the the
core part. The core part contains the actual functional parts of the
code. This split is done to make testing easier.
2017-07-18 11:37:18 +03:00
Markus Mäkelä
2670ca36a0 MXS-1300: Send requests to multiple hosts
The host and port options were replace with one hosts option which
combines the two. This allows a single command to be executed on multiple
maxscale instances. Added accompanying code for checking responsiveness of
all maxscale instances and handling synchronization of the check
results. The code uses ECMAScript 2016 native promises to do this.

Added a configurable timeout to all requests. This should allow users to
interact with MaxScale over slow and laggy networks.

Before each executed command, all hosts given as parameters are pinged to
make sure they are alive. This should reduce the possibility of partial
execution of commands due to failed MaxScales.

The results of requests are grouped by the values of the hosts
option. This allows users to compare results of multiple servers with
relative ease.

Fixed reversion of the default credentials from mariadb:admin to
admin:mariadb.
2017-07-18 11:37:18 +03:00
Markus Mäkelä
11c8493167 MXS-1300: Only use single quotes
Fixed mixed use of single and double quotes.
2017-07-18 11:37:18 +03:00
Markus Mäkelä
c7a45466fb Use Backend::write for all writes in schemarouter
The schemarouter should always use Backend::write for communication with
the backend servers. This keeps the backends in the correct state.

This fix solves some of the random test failures in the `sharding` test.
2017-07-18 11:36:54 +03:00
MassimilianoPinto
62fdda8534 Masking filter: unused code is now removed
Masking filter: unused code is now removed
2017-07-17 14:12:55 +02:00
MassimilianoPinto
ac142a1b71 Documentation update with 'obfuscate' rule and 'match' option in 'replace' rule
Documentation update with 'obfuscate' rule and 'match' option in
'replace' rule
2017-07-17 12:16:43 +02:00
MassimilianoPinto
1c9f4784dc MXS-1302: pass the rule_type to rule_check_database_options() in order to log the proper rule
rule_check_database_options() can now log the right rule type being
checked
2017-07-17 09:01:53 +02:00
MassimilianoPinto
f1b2257383 MXS-1302: the keyword in the Json is now "match" and MatchRule class substitutes CaptureRule
The keyword in the Json is now "match" and MatchRule class substitutes
CaptureRule

"replace": {
"column": "d_code",
"match": "(?<=aaa).*(?=-12)|(?<=-12).*"
},…
2017-07-14 14:21:40 +02:00
MassimilianoPinto
85d7d67ab8 MXS-1302: capture option in use and pcre2_match matches all the patterns
pcre2_match matches all the patterns
2017-07-14 14:21:40 +02:00
Markus Mäkelä
2410c23e66 Assume that the test database exists
The tests should not drop the test database and then recreate it. This
adds an unnecessary burden on replication which will cause false positives
when replication is not fast enough.
2017-07-11 22:34:01 +03:00
Markus Mäkelä
a77406e134 MXS-1300: Add calling of module commands to MaxCtrl
Module commands can now be called with MaxCtrl.
2017-07-10 22:23:49 +03:00
Markus Mäkelä
e14aa729ab MXS-1300: Add module commands to MaxCtrl
Added inspection of module commands to MaxCtrl. The commands can't be
called yet.
2017-07-10 22:23:49 +03:00
Markus Mäkelä
1514ede4e1 MXS-1300: Add logging commands
Added commands for altering logging parameters and for rotating logs.
2017-07-10 22:23:48 +03:00
Markus Mäkelä
78a95def7d MXS-1300: Add build script for MaxCtrl
The build uses the pkg library to package a Node.js app and all of its
dependenices.
2017-07-10 22:23:48 +03:00
Markus Mäkelä
ebfa631bae MXS-1330: Add alter command
Servers, monitors and services can now be altered with MaxCtrl.
2017-07-10 22:23:48 +03:00
Markus Mäkelä
46339d620a MXS-1220: Interpret no relationships as old relationships
When a PATCH request to a resource is made with no relationships defined,
it should be interpreted to be the same as the old relationships. Removal
of relationships can still be done by defining an empty relationships
object.
2017-07-10 22:23:48 +03:00
Markus Mäkelä
731d19ba70 MXS-1220: Add test for monitor alteration regression
Altering monitor parameters without defining the relationships was
interpreted as an attempt to remove all relationships. This test
reproduces the problem.
2017-07-10 22:23:48 +03:00
Markus Mäkelä
4913cf1ec7 MXS-1300: Add maxscale to show commands
The command shows general information about MaxScale.
2017-07-10 22:23:48 +03:00
Markus Mäkelä
f4dacff27c MXS-1300: Correct clear entry point documentation
Corrected the clear entry point documentation by changing the method from
POST to PUT. Fixed MaxCtrl to use the correct methods for set/clear.
2017-07-10 22:23:48 +03:00
Markus Mäkelä
252022534c MXS-1300: Add start/stop commands
Added start and stop commands for starting and stopping services and
monitors.
2017-07-10 22:23:48 +03:00
Markus Mäkelä
a0e7a97729 MXS-1300: Add create/destroy/list commands for users
Users can now be listed, created and destroyed via MaxCtrl.
2017-07-10 22:23:48 +03:00
Markus Mäkelä
b0f2d02787 MXS-1300: Add link/unlink command to MaxCtrl
The link/unlink commands are similar to the add/remove commands in
MaxAdmin but they link services and monitors to a list of servers instead
of linking servers to a list of either services or monitors. This can
reduce the number of required calls to alter the configuration while
making the process easier for MaxCtrl to do via the REST API.

Also fixed a false negative where a service alteration which results in a
no-op would be counted as an error.
2017-07-10 22:23:47 +03:00
Markus Mäkelä
7db1f4c768 MXS-1300: Add create/destroy commands
Servers, monitors and listeners can now be created and destroyed with
MaxCtrl.

Exported lodash as a part of the common.js module so that commands can use
it to construct objects. If there are enough common uses of this library,
some of the object construction could be moved to the common module.
2017-07-10 22:23:46 +03:00
Markus Mäkelä
35745ee925 MXS-1220: Add deletion of listeners
The listeners can now be deleted via the REST API. Documented the added
endpoint in the REST API documentation.
2017-07-10 22:23:46 +03:00
Markus Mäkelä
7e40c93cc4 MXS-1220: Add listener creation documentation
The documentation for creating listeners was not added.
2017-07-10 22:23:45 +03:00
Markus Mäkelä
e0076ca8a5 MXS-1220: Fix request methods for user creation
The creation of network users and enabling of UNIX accounts documented PUT
as the method for enabling them when the commands actually used POST.
2017-07-10 22:23:45 +03:00
Markus Mäkelä
bb5b83bf7b MXS-1300: Add enable/disable command
The enable/disable commands allow logging options and UNIX admin accounts
to be enabled and disabled.
2017-07-10 22:23:45 +03:00
Markus Mäkelä
cfa0fd9803 MXS-1300: Create more helper functions
The common error logging function prints and colors the output in an
uniform way. The updateValue function allows easy PATCH updates to a
single value of a resource.
2017-07-10 22:23:45 +03:00
Markus Mäkelä
c7b9b7ac4a MXS-1220: Unify resource member naming
All resoures now use the `state` member to describe their internal
state. This includes servers, services and monitors. This means that the
`status` keyword can be reserved for something else and it can be removed
until it is needed again.

Changed the module maturity field to `maturity` to better describe its
purpose.
2017-07-10 22:23:45 +03:00
Markus Mäkelä
a8bfdac49c MXS-1220: Use PUT to set/clear server
The correct method for set/clear endpoints is PUT, not POST, as the
resource always ends up in the desired state, regardless of its current
state.
2017-07-10 22:23:45 +03:00
Markus Mäkelä
30a40a090b MXS-1300: Make global option descriptions more brief
The option descriptions now fit on one line with their parameter types and
default values.
2017-07-10 22:23:44 +03:00
Markus Mäkelä
f62e899b79 MXS-1220: Add missing start/stop documentation for service resources
The service resource start and stop endpoints weren't documented.
2017-07-10 22:23:44 +03:00
Markus Mäkelä
6955be901f MXS-1300: Use single quotes in Node.js code
The Node.js code should use single quotes as it appears to be a prevalent
style choice for Node.js applications.
2017-07-10 22:23:44 +03:00