MaxScale/maxscale-system-test/JENKINS.md

Jenkins

List of Jenkins installations

URL	Description
max-tst-01.mariadb.com:8089	AWS, qemu; Regular testing for different MariaDB versions, different Linux distributions, Developers testing
maxscale-jenkins.mariadb.com:8089/	AWS, VBox; Regular builds for all distributions, build for Coverity, regular test VBox+CentOS6+MariaDB5.5
maxscale-jenkins.mariadb.com:8090	MDBCI testing and debugging, Jenkins experiments

Basic Jenkins jobs

max-tst-01.mariadb.com:8089

Job	Description
build_and_test	Build Maxscale and run systems tests
run_test	Run system tests, Maxscale package should be in the repository
build	Build Maxscale, create repository and publish it to http://max-tst-01.mariadb.com/ci-repository/
run_test_no_env_rebuild	Run system tests without creating a new set of VMs
create_env	Create VMs, install build environment to Maxscale machine, build Maxscale on Maxscale machine
destroy	Destroy VMs created by run_test or create_env
remove_lock	Remove Vagrant lock set by run_test or create_env

Every test run should have unique name (parameter 'name'). This name is used as a name of MDBCI configuration. If parameter 'do_not_destroy' is set to 'yes' virtual machines (VM) are not destroyed after tests execution and can be laters used for debugging or new test runs (see run_test_no_env_rebuild) VMs can be accessed from vagrant@max-tst-01.mariadb.com machine using 'mdbci ssh' or 'vagrant ssh' as well as direct ssh access using environmental variables provided by set_env_vagrant.sh script.

Parameter 'box' defines type of VM and Linux distribution to be used for tests.

Test results go to CDash, logs and core dumps are stored here.

create_env job allows to create a set of VMs (for backend and Maxscale) and does Maxscale build on the Maxscale VM. After execution this job Maxscale machine contains Maxscale source and binaries. NOTE: to properly configure Maxscale init scripts it is necessary to use rpm/dpkg tool to install Maxscale package (package can be found in the Maxscale build directory).

run_test and create_env jobs create Vagrant lock which prevents running two Vagrant instances in parallel (such parallel execution can cause Vagrant of VM provider failures). In case of job crash or interruption by user Vagrant lock stays in locked state and prevents any new VM creation. To remove lock job remove_lock should be used.

Process examples

Running regression test against a branch

Execute build_and_test

Recommendations regarding parameters:

• 'name' - unique name: it can be any text string, but as a good practice rule: 'name' should refer to branch, Linux distribution, date/time of testing, MariaDB version
• 'box' - most recommended boxes are 'centos_7.0_libvirt' (QEMU box) and 'centos7' (Amazon Web Services box)
• 'source' - which type of source to use. BRANCH for git branch, TAG for a git tag and COMMIT for a commit ID.
• 'value' - name of the branch (if 'source' is BRANCH), name of the GIT tag (if 'source' is TAG) or commint ID (if 'source' is COMMIT)

Build MaxScale

Execute build job.

Parameter 'target' means a name of repository to put packages: e.g. if 'target' is 'develop' packages are going to http://max-tst-01.mariadb.com/ci-repository/develop/

NOTE: building is executed only for selected distribution ('box' parameter). Be careful with other distributions: if build is not executed for same distribution old version can be in the repository (from some old builds). Later tests have to be executed against the same distribution otherwise they can be run against old version of MaxScale. It is recommended to use unique name for 'target'.

To debug failed build:

• set 'do_not_destroy_vm' parameter to 'yes'
• after the build:

ssh -i vagrant.pem vagrant@max-tst-01.mariadb.com
cd ~/mdbci/build-<box>-<date><time>
vagrant ssh

For example:

ssh -i vagrant.pem vagrant@max-tst-01.mariadb.com
cd ~/mdbci/build_centos6-20160119-0935
vagrant ssh

Create set of Master/Slave and Galera nodes and setup build environment for Maxscale on one more node

Execute create_env job.

Login to Maxscale machine (see environment documentation). MaxScale source code, binaries and packages can be found in the ~/workspace/ directory. All build tools are installed. GIT can be used to go trough source code. It is not recommended to commit anything from virtual machine to GitHub.

Please use 'rpm' or 'dpkg' to properly install Maxscale package (/etc/init.d/maxscale script will not be installed without execution of 'rpm' or 'dpkg')

Running test agains exiting version of Maxscale

Execute run_test job.

Be sure Maxscale binary repository is present on the http://max-tst-01.mariadb.com/ci-repository/ server. Please check:

• there is a directory with the name equal to 'target' parameter
• there is sub-directory for selected distribution ('box' parameter)

e.g. if 'target' is 'develop' and distribution is CentOS7 (boxes 'centos7' or 'centos_7.0_libvirt') the directory http://max-tst-01.mariadb.com/ci-repository/develop/mariadb-maxscale/centos/7/x86_64/ have to contain Maxscale RPM packages.

If parameter 'do_not_destroy' set to 'yes' after the test virtual machine will not be destroyed and can be used for debugging. See environment documentation to get know how to access virtual machines.

Maintenance operations

If test run was executed with parameter 'do_not_destroy' set yo 'yes' please do not forget to execute destroy against your 'target'

This job also have to be executed if test run job crashed or it was interrupted.

In case of build or test job crash, interruption, Jenkins crash during Vagrant operation it is possible that Vagrant lock stays in locked state and no other job can progress (job can be started, but it is waiting for Vagrant lock - '/home/vagrant/vagrant_lock' can be seen in the job log). In this case lock can be removed by remove_lock job.