From 87c3b1d25b590525dd3ca13e84da53182b975d03 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Markus=20M=C3=A4kel=C3=A4?= Date: Mon, 31 Dec 2018 17:06:38 +0200 Subject: [PATCH 1/7] MXS-2234: Extend persisted config file log message The log message should now contain enough information to more easily debug any configuration issues that result from updating the main config file after runtime changes have been done. Also fixed the theoretical race condition in detection of persisted configuration files by only checking for their existence once. --- server/core/config.cc | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/server/core/config.cc b/server/core/config.cc index f0d44e12d..ac05083b9 100644 --- a/server/core/config.cc +++ b/server/core/config.cc @@ -917,6 +917,7 @@ config_load_and_process(const char* filename, bool (*process_config)(CONFIG_CONT { bool rval = false; DUPLICATE_CONTEXT dcontext; + bool have_persisted_configs = false; if (duplicate_context_init(&dcontext)) { @@ -950,8 +951,12 @@ config_load_and_process(const char* filename, bool (*process_config)(CONFIG_CONT * TODO: Figure out a cleaner way to do this */ is_persisted_config = true; + have_persisted_configs = true; - MXS_NOTICE("Loading generated configuration files from '%s'", persist_cnf); + MXS_NOTICE("Runtime configuration changes have been done to MaxScale. Loading persisted " + "configuration files and applying them on top of the main configuration file. " + "These changes can override the values of the main configuration file: " + "To revert them, remove all the files in '%s'.", persist_cnf); DUPLICATE_CONTEXT p_dcontext; /** * We need to initialize a second duplicate context for the @@ -977,12 +982,12 @@ config_load_and_process(const char* filename, bool (*process_config)(CONFIG_CONT if (!check_config_objects(config_context.next) || !process_config(config_context.next)) { rval = false; - if (contains_cnf_files(persist_cnf)) + if (have_persisted_configs) { - MXS_WARNING("One or more generated configurations were found at '%s'. " - "If the error relates to any of the files located there, " - "remove the offending configurations from this directory.", - persist_cnf); + MXS_WARNING("Persisted configuration files generated by runtime configuration " + "changes were found at '%s' and at least one configuration error was " + "encountered. If the errors relate to any of the persisted configuration " + "files, remove the offending files and restart MaxScale.", persist_cnf); } } } From 1b0b32cbe8b769de045825af1763c94a0556aad3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Markus=20M=C3=A4kel=C3=A4?= Date: Thu, 3 Jan 2019 13:07:05 +0200 Subject: [PATCH 2/7] Update release documentation Updated the instructions on how releases are made. --- maxscale-system-test/Documentation/RELEASE.md | 113 ++++++++---------- 1 file changed, 49 insertions(+), 64 deletions(-) diff --git a/maxscale-system-test/Documentation/RELEASE.md b/maxscale-system-test/Documentation/RELEASE.md index 0412a90f3..d25b7d333 100644 --- a/maxscale-system-test/Documentation/RELEASE.md +++ b/maxscale-system-test/Documentation/RELEASE.md @@ -2,58 +2,78 @@ ## Pre-release Checklist -* Create new release notes and add all fixed bugs, use a previous one as a template +* Make sure all bugs that have been fixed are also closed on Jira and have the + correct fixVersion + +For major releases: + +* Create new release notes and add all fixed bugs, use a previous one as a + template + +For bug fix releases: + +* Run the `Documentation/Release-Notes/generate_release_notes.sh` script to + auto-generate release notes + +Finally: + * Add link to release notes and document major changes in Changelog.md ## 1. Tag -Release builds are always made using a tag. However, the used -tag is a _tentative_ tag, to ensure that there never is a need -to _move_ any tag, should the release have to be modified after -it has been tagged. All that is needed is to create a new -tentative tag. +Release builds are always made using a tag and a separate branch. However, the +used tag is a _tentative_ tag, to ensure that there never is a need to _move_ +any tag, should the release have to be modified after it has been tagged. All +that is needed is to create a new tentative tag. -The source for release `x.y.z` is tagged with `maxscale-x.y.z-ttN` -where `N` is 1 for the first attempt and incremented in case the -`x.y.z` source still needs to be modified and the release rebuilt. +The source for release `x.y.z` is tagged with `maxscale-x.y.z-ttN` where `N` is +1 for the first attempt and incremented in case the `x.y.z` source still needs +to be modified and the release rebuilt. -The final tag `maxscale-x.y.z` is created _after_ the packages have -been published and we are certain they will not be rebuilt, without -also the version number being changed. +The final tag `maxscale-x.y.z` is created _after_ the packages have been +published and we are certain they will not be rebuilt, without also the version +number being changed. -So, ensure that the `maxscale-x.y.z-ttN` has been created and pushed -to the repository. +To create the tag and the branch from the main _x.y_ branch: ``` +git checkout x.y +git checkout -b x.y.z +git tag maxscale-x.y.z-ttN +git push -u origin x.y.z git push origin refs/tags/maxscale-x.y.z-ttN ``` +**A note on fixing bugs while doing a release:** + +A separate branch is used to guarantee that no commits are added once the +release proceedings have started. If any fixes to code or documentation need to +be done, do them on the _x.y.z_ branch. If a fix has been made, create a new tag +by incrementing the `-ttN` suffix by one and push both the branch and the new +tag to the repo. + **NOTE** The tentative suffix - `-ttN` - is **only** used when specifying the tag for the build, it must **not** be present in any other names or paths. ## 2. Build and upgrade test -The Jenkins -[build_for_release](http://127.0.0.1:8089/job/build_for_release/) -job should be used for building the packages. - -Note that the above will not work unless you have set up an -ssh tunnel to Jenkins: -``` -$ ssh -f -N -L 8089:127.0.0.1:8089 vagrant@max-tst-01.mariadb.com -``` +The BuildBot [build_for_release](https://maxscale-ci.mariadb.com/#/builders/38) +job should be used for building the packages. Use your GitHub account to log in +to actually see the job. Click the blue _Build for release_ button in the top +right corner to start it. ### Parameters to define -#### `scm_source` +#### `branch` This is the tag that is used to build the release. ``` refs/tags/maxscale-x.y.z-ttN ``` -#### `version_number` + +#### `The version number of this release in x.y.z format` The version number of this release in x.y.z format. This will create two packages; maxscale-x.y.z-release and maxscale-x.y.z-debug. @@ -61,39 +81,16 @@ The version number of this release in x.y.z format. This will create two package x.y.z ``` -#### `old_target` +#### `Old target` -The previous released version, used by upgrade tests. +The previous released version, used by upgrade tests. Set it to the previous +release e.g. for 2.2.19 set it to 2.2.18. For GA releases, set it to the latest +release of the previous major release e.g. for 2.3.0 set it to 2.2.19. ``` x.y.z ``` -### 1.4.x build - -Use the [build_all](http://127.0.0.1:8089/job/build_all/) job. - -For `1.4` builds the default values of the following parameters -should be changed: - -#### use_mariadbd - -``` -yes -``` - -#### cnf_file - -``` -maxscale.cnf.minimum.1.4.4 -``` - -#### maxadmin_command - -``` -maxadmin -pmariadb show services -``` - ## 3. Copying to code.mariadb.com ssh to `code.mariadb.com` with your LDAP credentials. @@ -165,18 +162,6 @@ git tag -d maxscale-x.y.z-ttN git push origin :refs/tags/maxscale-x.y.z-ttN ``` -## 6. Create the branch - -Release `x.y.z` is typically developed in the branch `x.y`. -Once `x.y.z` has been released, the branch `x.y.z` also needs -to be created. - -```bash -git checkout maxscale-x.y.z -git checkout -b x.y.z -git push origin x.y.z -``` - ## 7. Update the release date Once the branch `x.y.z` has been created and the actual release From c8d4052aecb4f745182e7890c2f65f479f68da83 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Markus=20M=C3=A4kel=C3=A4?= Date: Thu, 3 Jan 2019 13:34:10 +0200 Subject: [PATCH 3/7] Improve release note generation script The script can now be run from outside of the directory it is contained in. --- Documentation/Release-Notes/generate_release_notes.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/Documentation/Release-Notes/generate_release_notes.sh b/Documentation/Release-Notes/generate_release_notes.sh index 905e90476..a2b14160b 100755 --- a/Documentation/Release-Notes/generate_release_notes.sh +++ b/Documentation/Release-Notes/generate_release_notes.sh @@ -1,5 +1,7 @@ #!/bin/bash +cd $(dirname $(realpath $0)) + major="`cd ../../ && cmake -P ./VERSION.cmake -L|grep 'MAXSCALE_VERSION_MAJOR'|sed 's/.*=//'`" minor="`cd ../../ && cmake -P ./VERSION.cmake -L|grep 'MAXSCALE_VERSION_MINOR'|sed 's/.*=//'`" patch="`cd ../../ && cmake -P ./VERSION.cmake -L|grep 'MAXSCALE_VERSION_PATCH'|sed 's/.*=//'`" From 7b6ae4bc13ba3ce8c89eef8325f5e5f74b7fa551 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Markus=20M=C3=A4kel=C3=A4?= Date: Thu, 3 Jan 2019 13:36:07 +0200 Subject: [PATCH 4/7] Add 2.2.19 release notes Generated 2.2.9 release notes and added them to the changelog. --- Documentation/Changelog.md | 1 + .../MaxScale-2.2.19-Release-Notes.md | 36 +++++++++++++++++++ 2 files changed, 37 insertions(+) create mode 100644 Documentation/Release-Notes/MaxScale-2.2.19-Release-Notes.md diff --git a/Documentation/Changelog.md b/Documentation/Changelog.md index baea690a5..31b1e0824 100644 --- a/Documentation/Changelog.md +++ b/Documentation/Changelog.md @@ -28,6 +28,7 @@ the master. There is also limited capability for rejoining nodes. For more details, please refer to: +* [MariaDB MaxScale 2.2.19 Release Notes](Release-Notes/MaxScale-2.2.19-Release-Notes.md) * [MariaDB MaxScale 2.2.18 Release Notes](Release-Notes/MaxScale-2.2.18-Release-Notes.md) * [MariaDB MaxScale 2.2.17 Release Notes](Release-Notes/MaxScale-2.2.17-Release-Notes.md) * [MariaDB MaxScale 2.2.16 Release Notes](Release-Notes/MaxScale-2.2.16-Release-Notes.md) diff --git a/Documentation/Release-Notes/MaxScale-2.2.19-Release-Notes.md b/Documentation/Release-Notes/MaxScale-2.2.19-Release-Notes.md new file mode 100644 index 000000000..42f0cfa26 --- /dev/null +++ b/Documentation/Release-Notes/MaxScale-2.2.19-Release-Notes.md @@ -0,0 +1,36 @@ +# MariaDB MaxScale 2.2.19 Release Notes + +Release 2.2.19 is a GA release. + +This document describes the changes in release 2.2.19, when compared to the +previous release in the same series. + +For any problems you encounter, please consider submitting a bug +report on [our Jira](https://jira.mariadb.org/projects/MXS). + +## Bug fixes + +* [MXS-2238](https://jira.mariadb.org/browse/MXS-2238) MaxScale fails to send large CDC schemas +* [MXS-2234](https://jira.mariadb.org/browse/MXS-2234) Add extra info to log when MaxScale loads persisted configuration files +* [MXS-2232](https://jira.mariadb.org/browse/MXS-2232) version_string prefix 5.5.5- is always added +* [MXS-2231](https://jira.mariadb.org/browse/MXS-2231) Kerberos together with ssl doesn't work + +## Known Issues and Limitations + +There are some limitations and known issues within this version of MaxScale. +For more information, please refer to the [Limitations](../About/Limitations.md) document. + +## Packaging + +RPM and Debian packages are provided for supported the Linux distributions. + +Packages can be downloaded [here](https://mariadb.com/downloads/mariadb-tx/maxscale). + +## Source Code + +The source code of MaxScale is tagged at GitHub with a tag, which is identical +with the version of MaxScale. For instance, the tag of version X.Y.Z of MaxScale +is `maxscale-X.Y.Z`. Further, the default branch is always the latest GA version +of MaxScale. + +The source code is available [here](https://github.com/mariadb-corporation/MaxScale). From 46bea87ff66e55585bb1c7d1356e5e1332ec7a9b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Markus=20M=C3=A4kel=C3=A4?= Date: Fri, 4 Jan 2019 10:20:30 +0200 Subject: [PATCH 5/7] MXS-2238: Fix reading of large Avro schemas The schemas were read incorrectly which resulted in large schemas having multiple newlines in them. --- server/modules/routing/avrorouter/avro_client.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/server/modules/routing/avrorouter/avro_client.c b/server/modules/routing/avrorouter/avro_client.c index 05b98f747..296b43fdd 100644 --- a/server/modules/routing/avrorouter/avro_client.c +++ b/server/modules/routing/avrorouter/avro_client.c @@ -853,9 +853,7 @@ GWBUF* read_avro_json_schema(const char *avrofile, const char* dir) nread--; } - buffer[nread++] = '\n'; - - GWBUF * newbuf = gwbuf_alloc_and_load(nread, buffer); + GWBUF* newbuf = gwbuf_alloc_and_load(nread, buffer); if (newbuf) { @@ -863,6 +861,8 @@ GWBUF* read_avro_json_schema(const char *avrofile, const char* dir) } } + rval = gwbuf_append(rval, gwbuf_alloc_and_load(1, "\n")); + fclose(file); } else From bc346422fbbd7c1f5031675ba2a060217b73a386 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Markus=20M=C3=A4kel=C3=A4?= Date: Fri, 4 Jan 2019 10:43:43 +0200 Subject: [PATCH 6/7] Add minimal CDC client Added a minimal CDC client application that uses the CDC connector library. This is mainly intended for manual testing so it is built using the accompanying Makefile instead of being a part of the CMake system. --- connectors/cdc-connector/cdc_connector.h | 2 +- connectors/cdc-connector/examples/Makefile | 5 +++++ connectors/cdc-connector/examples/main.cpp | 2 +- 3 files changed, 7 insertions(+), 2 deletions(-) create mode 100644 connectors/cdc-connector/examples/Makefile diff --git a/connectors/cdc-connector/cdc_connector.h b/connectors/cdc-connector/cdc_connector.h index d7455c8ac..b44408d4a 100644 --- a/connectors/cdc-connector/cdc_connector.h +++ b/connectors/cdc-connector/cdc_connector.h @@ -32,7 +32,7 @@ namespace CDC // The error strings returned by the getError library. These can be used to // check for the most common errors (which right now is only the timeout). -const char* TIMEOUT = "Request timed out"; +static const char* TIMEOUT = "Request timed out"; // The typedef for the Row type class Row; diff --git a/connectors/cdc-connector/examples/Makefile b/connectors/cdc-connector/examples/Makefile new file mode 100644 index 000000000..27da805ed --- /dev/null +++ b/connectors/cdc-connector/examples/Makefile @@ -0,0 +1,5 @@ +all: + c++ -I ../ ../cdc_connector.cpp main.cpp -ljansson -lcrypto -o cdc + +clean: + rm -rf cdc diff --git a/connectors/cdc-connector/examples/main.cpp b/connectors/cdc-connector/examples/main.cpp index a51082d81..f626ea27d 100644 --- a/connectors/cdc-connector/examples/main.cpp +++ b/connectors/cdc-connector/examples/main.cpp @@ -26,7 +26,7 @@ int main(int argc, char** argv) if (conn.connect(argv[5])) { - CDC::Row row; + CDC::SRow row; while ((row = conn.read())) { From 2172c4ca85e855feadfc21344bcd00204fdec513 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Markus=20M=C3=A4kel=C3=A4?= Date: Sun, 6 Jan 2019 13:12:38 +0200 Subject: [PATCH 7/7] MXS-2048: Fix dbfwfilter module command documentation Fixed the command names. --- Documentation/Filters/Database-Firewall-Filter.md | 12 +++++++----- .../Release-Notes/MaxScale-2.2.19-Release-Notes.md | 1 + 2 files changed, 8 insertions(+), 5 deletions(-) diff --git a/Documentation/Filters/Database-Firewall-Filter.md b/Documentation/Filters/Database-Firewall-Filter.md index 7c3401919..26b800ee7 100644 --- a/Documentation/Filters/Database-Firewall-Filter.md +++ b/Documentation/Filters/Database-Firewall-Filter.md @@ -374,16 +374,18 @@ details about module commands. The dbfwfilter supports the following module commands. -### `dbfwfilter::rules/reload [FILE]` +### `rules/reload FILTER [FILE]` Load a new rule file or reload the current rules. New rules are only taken into use if they are successfully loaded and in cases where loading of the rules -fail, the old rules remain in use. The _FILE_ argument is an optional path to a -rule file and if it is not defined, the current rule file is used. +fail, the old rules remain in use. The _FILTER_ parameter is the filter instance +whose rules are reloaded. The _FILE_ argument is an optional path to a rule file +and if it is not defined, the current rule file is used. -### `dbfwfilter::rules` +### `rules FILTER` -Shows the current statistics of the rules. +Shows the current statistics of the rules. The _FILTER_ parameter is the filter +instance to inspect. ## Use Cases diff --git a/Documentation/Release-Notes/MaxScale-2.2.19-Release-Notes.md b/Documentation/Release-Notes/MaxScale-2.2.19-Release-Notes.md index 42f0cfa26..543b613f3 100644 --- a/Documentation/Release-Notes/MaxScale-2.2.19-Release-Notes.md +++ b/Documentation/Release-Notes/MaxScale-2.2.19-Release-Notes.md @@ -14,6 +14,7 @@ report on [our Jira](https://jira.mariadb.org/projects/MXS). * [MXS-2234](https://jira.mariadb.org/browse/MXS-2234) Add extra info to log when MaxScale loads persisted configuration files * [MXS-2232](https://jira.mariadb.org/browse/MXS-2232) version_string prefix 5.5.5- is always added * [MXS-2231](https://jira.mariadb.org/browse/MXS-2231) Kerberos together with ssl doesn't work +* [MXS-2048](https://jira.mariadb.org/browse/MXS-2048) Rewrite dbfwfilter module command documentation ## Known Issues and Limitations