hcq/MaxScale
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge branch '2.1' into 2.2

Browse Source
This commit is contained in:
Johan Wikman
2018-02-12 14:00:08 +02:00
parent 3310af1dd9 cd1604f915
commit 5c10d0ae92
7 changed files with 361 additions and 97 deletions
Download Patch File Download Diff File Expand all files Collapse all files

86
BUILD/mdbci/README.md Normal file
View File

@ -0,0 +1,86 @@
# Building Maxscale on Virtual Machine with MDBCI
MDBCI is a tool to manage virtual machines (VM). VM can be described in
the simple JSON format.
JSON templates can be found in [BUILD/mdbci/templates](templates)
MDBCI_VM_PATH have to be set before executing any MDBCI commands.
This variable points to the directory to store 'Vagrantfile's
for all VMs as well as all additional files (e.g. *network_config)
libvirt/qemu and AWS VMs are supported.
See [MDBCI README](https://github.com/mariadb-corporation/mdbci#mariadb-continuous-integration-infrastructure-mdbci) for details.
Installation instructions: [PREPARATION_FOR_MDBCI](https://github.com/mariadb-corporation/mdbci/blob/integration/PREPARATION_FOR_MDBCI.md)
## Build script
[BUILD/mdbci/build.sh](build.sh) prepares VM, executes Maxscale build and creates binary RPM or DEB repository.
Build options can be defined in the environmental variables. For variables descriptions and default values see
comments in the [BUILD/mdbci/build.sh](build.sh) and [BUILD/mdbci/set_build_variables.sh](set_build_variables.sh)
Script does not store build log, please use ```stdout``` and ```stderr``` redirection to store log.
## Default build
[BUILD/mdbci/build.sh](build.sh) can be executed without defining any variable.
Prerequirements:
* MDBCI installed
* Vagrant and vagrant-libvirt plugin installed
* $HOME/maxscale_gpg_keys/ directory contains one public and one corresponding private key (files with .public and .private extensions)
* $HOME/team_keys file contains public keys to be installed to VM
By default VM will not stay alive after the build. Please use ```export do_not_destroy_vm=yes```
to prevent VM from being destroyed. In this case VM have to be destroyed manually by ```mdbci destroy```
command.
VM will be created in ```$HOME/vms directory```, binary repository will be created the in
```$HOME/repository/``` directory.
The name of binary repository sub-directory is generated based on source default
branch name and current date and time.
## Upgrade test
If `run_upgrade_test` variable set to `yes` the upgrade test will be executed after the build.
Upgrade test includes installation of old version of Maxscale from [production repository]
(https://downloads.mariadb.com/MaxScale/) and upgrading it to recently built version.
The old version can be defined in `old_target` variable.
Upgrade test starts Maxscale with only one service - `CLI` and tries to execute 'maxadmin' command.
If this command exits with success ('0') upgrade test reports PASSED.
## Build with AWS VM
To build using AWS VM it is necessary to configure AWS credentials and install `vagrant-aws` plugin.
AWS credentials have to be configured in the $HOME/.aws directory as described in the
[AWS CLI documentation](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-welcome.html)
## Example of build
```bash
# see https://github.com/mariadb-corporation/mdbci/tree/integration/BOXES
# for available boxes
export box="ubuntu_xenial_libvirt"
# 'source'can be any reference to the Maxscale repo
# examples:
# source="refs/tags/2.1.1" - tag "2.1.1"
# source="refs/heads/develop" - "develop" branch
# source="2.1" - can be branch or tag "2.1"
# (use refs/tags and refs/heads if there are
# both tag "2.1" and branch "2.1")
# source="816983691af8fbded1b2b06a8c19e02587a54f32" - commit ID
export source="2.2"
# the name of binary repository
export target="2.1-2018Jan10"
cd BUILD/mdbci
./build.sh
```

8
maxscale-system-test/README.md
View File

@ -8,17 +8,15 @@ System level tests for MaxScale
- >= 4 machines for Master/Slave
- >= 4 machines for Galera cluster
- environmental variables contains all information about backend: IPs, user names, passwords, paths to tools, etc
- backed can be created with help of [MDBCI tool](https://github.com/OSLL/mdbci)
- configuring of Master/Slave and Galera can be done with help of [build scripts package](https://github.com/mariadb-corporation/build-scripts-vagrant)
- backed can be created with help of [MDBCI tool](https://github.com/mariadb-corporation/mdbci)
- configuring of Master/Slave and Galera can be done with help of [maxscale-system-test/mdbci/run_test.sh script](mdbci/run_test.sh)
## Manuals
[How to run tests](https://github.com/mariadb-corporation/build-scripts-vagrant/blob/master/RUN_TEST.md)
[MDBCI and running tests basics](mdbci/README.md)
[Hints: How to write a test](HOW_TO_WRITE_TEST.md)
[Build and test environment setup (if you want to play with MDBCI and Vagrant on your local machine](ENV_SETUP.md)
[Jenkins instructions](JENKINS.md)
## Environmental variables

180
maxscale-system-test/mdbci/README.md Normal file
View File

@ -0,0 +1,180 @@
# Running Maxscale system tests on Virtual Machines with MDBCI
MDBCI is a tool to manage virtual machines (VMs).
VMs can be described in the simple JSON format.
JSON templates for test configuration can be found in
[maxscale-system-test/mdbci/templates/](templates/)
'MDBCI_VM_PATH' have to be set before executing any MDBCI commands.
This variable points to the directory to store 'Vagrantfile's
for all VMs as well as all additional files (e.g. *network_config)
libvirt/qemu and AWS VMs are supported.
See [MDBCI README](https://github.com/mariadb-corporation/mdbci#mariadb-continuous-integration-infrastructure-mdbci) for details.
Installation instructions: [PREPARATION_FOR_MDBCI](https://github.com/mariadb-corporation/mdbci/blob/integration/PREPARATION_FOR_MDBCI.md)
## Basics of test setup
Test setup is described in template. Templates are stored in
[maxscale-system-test/mdbci/templates/](templates/)
Own template have to be put to the same directory.
Default environment for tests consists of:
* one VM for Maxscale
* 4 VMs for master/slave setup
* 4 VMs for Galera cluster
Template for this configuration is
[maxscale-system-test/mdbci/templates/default.json.template](templates/default.json.template)
Another templates:
Template name|Description
---|---
```nogalera``` |only 1 VM for Maxscale and 4 for Master/Slaves|
```twomaxscales``` |2 VMs for Maxscale and 4 for Master/Slaves|
```big``` |1 VM for Maxscale, 8 for Master/Slaves and 4 for Galera|
```big15``` |1 VM for Maxscale, 15 for Master/Slaves and 4 for Galera|
```box``` for ```big``` and ```big15``` is hard-coded as ```centos_7_aws_large```
Template can contain references to any environmental variables - they all
will be replaced with values before VMs starting
The [maxscale-system-test/mdbci/run_test.sh](run_test.sh) script
brings test VMs configuration up and tries to execute
```maxscale-system-test``` using 'ctest'.
Script can be executed without any parameters and without defining any
environmental variables.
In this case, tests will be executed for CentOS 7, MariaDB 10.2 and
Maxscale from current 'develop' repository
[http://max-tst-01.mariadb.com/ci-repository/develop/mariadb-maxscale/](http://max-tst-01.mariadb.com/ci-repository/develop/mariadb-maxscale/)
VMs will not be destroyed after the tests.
The name of test run (and name of VMs set) is generated based on ```box``` parameter
and current date/time
### Basic run_test.sh parameters
Variable name|Meaning
---|---
```target``` |name of binary repository to install Maxscale from|
```box``` |Vagrant box to be used to create VMs |
```test_set``` |Set of test to be executed in the 'ctest' format|
```version```|Version of DB server in Master/Slave backend|
```galera_version```|Version of DB server in Galera backend|
```product```|Type of backend - 'mariadb' or 'mysql'|
```template```|Name of *.json.template file with VMs descriptions in MDBCI format|
```team_keys```|Path to the file with public ssh keys - this file is loaded to VMs|
```do_not_destroy_vm```|if 'yes' VMs stay alive after test|
```name```|The name of test run - any string to identify VMs set|
For complete list of environmental variables see comments in
[maxscale-system-test/mdbci/run_test.sh](run_test.sh)
and file [maxscale-system-test/mdbci/set_run_test_variables.sh](set_run_test_variables.sh)
See [https://github.com/mariadb-corporation/mdbci/tree/integration/BOXES](https://github.com/mariadb-corporation/mdbci/tree/integration/BOXES)
for available boxes.
```test_set``` examples
test_set|Meaning
---|---
```-I 4,4```|Run single test number 4|
```-I 4,4,1,25,30```|Run tests number 4, 25 and 30 (the number '1' is a 'Stride' parameter)|
```-I 1,10```|Run tests from 1 to 10|
```-L REPL_BACKED```|Run all tests with 'REPL_BACKEND' label|
```-LE UNSTABLE```|Run all tests EXCEPT tests with 'UNSTABLE' label|
If ```galera_version``` is not defined the value of ```version``` is used also for Galera backend
### Test execution
After execution of 'run_test.sh` by default VMs stay alive and other tests can be executed.
Test use environmental variables to get all infio about test setup (about VMs).
The script [maxscale-system-test/mdbci/set_env.sh](set_env.sh)
loads all needed values (IPs, paths to ssh keyfiles,
user names, etc) into environmental variables. Script uses
data from ```${MDBCI_VM_PATH}/${name}_network_config``` file
and also calls MDBCI commands.
Script have to be sourced:
```bash
source ./mdbci/set_env.sh $name
```
or
```bash
. ./mdbci/set_env.sh $name
```
After it, any 'maxscale-system-test ' can be executed, e.g.:
```bash
./sql_queries
```
### Basic MDBCI and Vagrant operations
#### Restore ${name}.config_file
```bash
${mdbci_dir}/mdbci show network_config $name
```
#### Suspend VMs
Before rebooting computer it is recommended to suspend
Vagrant-controlled VMs
```bash
cd ${MDBCI_VM_PATH}/$name
vagrant suspend
```
#### Resume suspended VMs
```bash
cd ${MDBCI_VM_PATH}/$name
vagrant resume
```
#### Destroying VMs
```bash
${mdbci_dir}/mdbci destroy $name
```
#### Restoring backend
Every test before any actions checks backend and tries to restore it if its broken.
To restore backend separatelly and for intial backend setup check_backend' can be used:
```bash
. ./mdbci/set_env.sh $name
./check_backend
```
#### Restoring VMs from snapshot
'run_test.sh' makes snapshot of all VMs before tests. The name of snapshot is 'clean'.
In case of problem, after 'snapshot revert' it is recommended to re-create
${name}_network_config file, re-load environmental variables and run
'check_backend'
```bash
${mdbci_dir}/mdbci snapshot revert --path-to-nodes $name --snapshot-name clean
${mdbci_dir}/mdbci show network_config $name
. ./mdbci/set_env.sh $name
./check_backend
```

64
maxscale-system-test/mdbci/templates/big.json.template
View File

@ -4,10 +4,10 @@
"hostname" : "node000",
"box" : "centos_7_aws_large",
"product" : {
"name": "###product###",
"version": "###version###",
"name": "${product}",
"version": "${version}",
"cnf_template" : "server1.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
@ -17,10 +17,10 @@
"hostname" : "node001",
"box" : "centos_7_aws_large",
"product" : {
"name": "###product###",
"version": "###version###",
"name": "${product}",
"version": "${version}",
"cnf_template" : "server2.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
@ -29,10 +29,10 @@
"hostname" : "node002",
"box" : "centos_7_aws_large",
"product" : {
"name": "###product###",
"version": "###version###",
"name": "${product}",
"version": "${version}",
"cnf_template" : "server3.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
@ -41,10 +41,10 @@
"hostname" : "node003",
"box" : "centos_7_aws_large",
"product" : {
"name": "###product###",
"version": "###version###",
"name": "${product}",
"version": "${version}",
"cnf_template" : "server4.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
"node_004" :
@ -52,10 +52,10 @@
"hostname" : "node004",
"box" : "centos_7_aws_large",
"product" : {
"name": "###product###",
"version": "###version###",
"name": "${product}",
"version": "${version}",
"cnf_template" : "server5.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
"node_005" :
@ -63,10 +63,10 @@
"hostname" : "node005",
"box" : "centos_7_aws_large",
"product" : {
"name": "###product###",
"version": "###version###",
"name": "${product}",
"version": "${version}",
"cnf_template" : "server6.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
"node_006" :
@ -74,10 +74,10 @@
"hostname" : "node006",
"box" : "centos_7_aws_large",
"product" : {
"name": "###product###",
"version": "###version###",
"name": "${product}",
"version": "${version}",
"cnf_template" : "server7.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
"node_007" :
@ -85,10 +85,10 @@
"hostname" : "node007",
"box" : "centos_7_aws_large",
"product" : {
"name": "###product###",
"version": "###version###",
"name": "${product}",
"version": "${version}",
"cnf_template" : "server8.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
@ -98,9 +98,9 @@
"box" : "centos_7_aws",
"product" : {
"name": "galera",
"version": "###galera_version###",
"version": "${galera_version}",
"cnf_template" : "galera_server1.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
@ -110,9 +110,9 @@
"box" : "centos_7_aws",
"product" : {
"name": "galera",
"version": "###galera_version###",
"version": "${galera_version}",
"cnf_template" : "galera_server2.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
@ -122,9 +122,9 @@
"box" : "centos_7_aws",
"product" : {
"name": "galera",
"version": "###galera_version###",
"version": "${galera_version}",
"cnf_template" : "galera_server3.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
@ -134,9 +134,9 @@
"box" : "centos_7_aws",
"product" : {
"name": "galera",
"version": "###galera_version###",
"version": "${galera_version}",
"cnf_template" : "galera_server4.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},

116
maxscale-system-test/mdbci/templates/big15.json.template
View File

@ -4,10 +4,10 @@
"hostname" : "node_000",
"box" : "centos_7_aws_large",
"product" : {
"name": "###product###",
"version": "###version###",
"name": "${product}",
"version": "${version}",
"cnf_template" : "server1.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
@ -17,10 +17,10 @@
"hostname" : "node_001",
"box" : "centos_7_aws_large",
"product" : {
"name": "###product###",
"version": "###version###",
"name": "${product}",
"version": "${version}",
"cnf_template" : "server2.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
@ -29,10 +29,10 @@
"hostname" : "node_002",
"box" : "centos_7_aws_large",
"product" : {
"name": "###product###",
"version": "###version###",
"name": "${product}",
"version": "${version}",
"cnf_template" : "server3.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
@ -41,10 +41,10 @@
"hostname" : "node_003",
"box" : "centos_7_aws_large",
"product" : {
"name": "###product###",
"version": "###version###",
"name": "${product}",
"version": "${version}",
"cnf_template" : "server4.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
"node_004" :
@ -52,10 +52,10 @@
"hostname" : "node_004",
"box" : "centos_7_aws_large",
"product" : {
"name": "###product###",
"version": "###version###",
"name": "${product}",
"version": "${version}",
"cnf_template" : "server5.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
"node_005" :
@ -63,10 +63,10 @@
"hostname" : "node_005",
"box" : "centos_7_aws_large",
"product" : {
"name": "###product###",
"version": "###version###",
"name": "${product}",
"version": "${version}",
"cnf_template" : "server6.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
"node_006" :
@ -74,10 +74,10 @@
"hostname" : "node_006",
"box" : "centos_7_aws_large",
"product" : {
"name": "###product###",
"version": "###version###",
"name": "${product}",
"version": "${version}",
"cnf_template" : "server7.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
"node_007" :
@ -85,10 +85,10 @@
"hostname" : "node_007",
"box" : "centos_7_aws_large",
"product" : {
"name": "###product###",
"version": "###version###",
"name": "${product}",
"version": "${version}",
"cnf_template" : "server8.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
"node_008" :
@ -96,10 +96,10 @@
"hostname" : "node_008",
"box" : "centos_7_aws_large",
"product" : {
"name": "###product###",
"version": "###version###",
"name": "${product}",
"version": "${version}",
"cnf_template" : "server9.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
"node_009" :
@ -107,65 +107,65 @@
"hostname" : "node_009",
"box" : "centos_7_aws_large",
"product" : {
"name": "###product###",
"version": "###version###",
"name": "${product}",
"version": "${version}",
"cnf_template" : "server10.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
"node_0010" :
"node_010" :
{
"hostname" : "node_0010",
"box" : "centos_7_aws_large",
"product" : {
"name": "###product###",
"version": "###version###",
"name": "${product}",
"version": "${version}",
"cnf_template" : "server11.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
"node_0011" :
"node_011" :
{
"hostname" : "node_0011",
"box" : "centos_7_aws_large",
"product" : {
"name": "###product###",
"version": "###version###",
"name": "${product}",
"version": "${version}",
"cnf_template" : "server12.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
"node_0012" :
"node_012" :
{
"hostname" : "node_0012",
"box" : "centos_7_aws_large",
"product" : {
"name": "###product###",
"version": "###version###",
"name": "${product}",
"version": "${version}",
"cnf_template" : "server13.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
"node_0013" :
"node_013" :
{
"hostname" : "node_0013",
"box" : "centos_7_aws_large",
"product" : {
"name": "###product###",
"version": "###version###",
"name": "${product}",
"version": "${version}",
"cnf_template" : "server14.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
"node_0014" :
"node_014" :
{
"hostname" : "node_0014",
"box" : "centos_7_aws_large",
"product" : {
"name": "###product###",
"version": "###version###",
"name": "${product}",
"version": "${version}",
"cnf_template" : "server15.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
@ -175,9 +175,9 @@
"box" : "centos_7_aws",
"product" : {
"name": "galera",
"version": "###galera_version###",
"version": "${galera_version}",
"cnf_template" : "galera_server1.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
@ -187,9 +187,9 @@
"box" : "centos_7_aws",
"product" : {
"name": "galera",
"version": "###galera_version###",
"version": "${galera_version}",
"cnf_template" : "galera_server2.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
@ -199,9 +199,9 @@
"box" : "centos_7_aws",
"product" : {
"name": "galera",
"version": "###galera_version###",
"version": "${galera_version}",
"cnf_template" : "galera_server3.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},
@ -211,9 +211,9 @@
"box" : "centos_7_aws",
"product" : {
"name": "galera",
"version": "###galera_version###",
"version": "${galera_version}",
"cnf_template" : "galera_server4.cnf",
"cnf_template_path": "~/build-scripts/test-setup-scripts/cnf"
"cnf_template_path": "${cnf_path}"
}
},

2
query_classifier/qc_mysqlembedded/qc_mysqlembedded.cc
View File

@ -3525,7 +3525,7 @@ extern "C"
static MXS_MODULE info =
{
MXS_MODULE_API_QUERY_CLASSIFIER,
MXS_MODULE_IN_DEVELOPMENT,
MXS_MODULE_GA,
MXS_QUERY_CLASSIFIER_VERSION,
"Query classifier based upon MySQL Embedded",
"V1.0.0",

2
query_classifier/qc_sqlite/qc_sqlite.cc
View File

@ -5036,7 +5036,7 @@ MXS_MODULE* MXS_CREATE_MODULE()
static MXS_MODULE info =
{
MXS_MODULE_API_QUERY_CLASSIFIER,
MXS_MODULE_BETA_RELEASE,
MXS_MODULE_GA,
MXS_QUERY_CLASSIFIER_VERSION,
"Query classifier using sqlite.",
"V1.0.0",