8307 Commits

Author SHA1 Message Date
Markus Mäkelä
826654b9de MXS-1300: Build MaxCtrl with experimental packages
The build scripts now download Node.js 6.11.2 for building and packaging
MaxCtrl. The client package is also created when the extra packages are
built.
2017-08-09 11:39:25 +03:00
Markus Mäkelä
7c38a56ea2 MXS-1300: Add client as a valid package name
The MaxCtrl component, `client`, needs to be in the list of valid
component names.
2017-08-09 11:39:25 +03:00
Markus Mäkelä
c2a2688b93 Fix duplicate listener checks
Only the protocol, port and address of the listener were used to check if
a listener exists. The check should also use the name of the listener to
be sure that each name is unique.

Expanded tests to check that the creation of duplicate listeners is
detected. Did minor improvements to related test code.
2017-08-09 11:39:25 +03:00
Markus Mäkelä
c593a1bf32 MXS-1300: Test cluster diff functionality
The `cluster diff` command is now tested by doing add/alter/remove
operations on servers. As the same methods are used for all objects, they
should detect all sorts of modifications between two Maxscales.
2017-08-09 11:39:25 +03:00
Markus Mäkelä
70ae93a9ab MXS-1300: Move service checks in cluster to a function
The check that the two MaxScales have the same services is now in a common
function. Added more details into the error message as to why the command
failed. The hosts and their extra services are logged for both the source
and target of the command.

The `cluster diff` command could display the extra services in the diff
but this could be interpreted as a sign that MaxScale could act on that
part of the diff. Returning an error instead of silently ignoring the
extra services should allow for a more consistent user experience.
2017-08-09 11:39:25 +03:00
Markus Mäkelä
f422e92496 MXS-1300: Return output instead of printing it
The `execute` command now returns the output of the command instead of
printint it. This allows the tests to actually test the output of the
commands instead manually verifying that it is correct. It also allows the
library part to be used as an actual library that only returns data.
2017-08-09 11:39:25 +03:00
Markus Mäkelä
9a04dd0311 MXS-1300: Resolved the promise for help commands
The promise returned by the `execute` MaxCtrl library function is now
properly resolved also for `help` type commands.
2017-08-09 11:39:24 +03:00
Markus Mäkelä
527ada770c MXS-1300: Fix double maxscale startup script and configuration
The configuration wrongly spelled `secondary` as `secodary` causing a
failure to start the secondary MaxScale. The startup script didn't create
the log directory and removed the wrong maxadmin socket file.
2017-08-09 11:39:24 +03:00
Markus Mäkelä
1308e7a304 Fix resource collection modification times
The resource collection modification times weren't updated when an
individual resource was updated. By tracking back the path of a resource
and updating the modification times, all nodes in the path will get the
correct modification time.
2017-08-09 11:39:24 +03:00
Markus Mäkelä
7e18e49eec MXS-1220: Move request body checks to a higher level
The checks whether a request body is present are now done at a higher
level. This removes the need to do the checks at the resource handler
callback, removing duplicated code.

The checks are done by adding constraints to resources that must be
fulfilled by each request.

Added debug assertions to make sure that the core logic of the REST API
resource system works.
2017-08-09 11:39:24 +03:00
Markus Mäkelä
1b65bd2a9d Improve runtime listener creation error messages
The error messages for listener creation are now more detailed. In
addition to this, they are also conveyed to the runtime error system.
2017-08-09 11:39:24 +03:00
Markus Mäkelä
0ab7df88dc MXS-1300: Fix MaxCtrl cluster sync test for listeners
The test is not very useful for a locally hosted setup as both MaxScales
can't bind to the same port.
2017-08-09 11:39:24 +03:00
Markus Mäkelä
2f7251a769 Reuse same MariaDB cluster for testing
The test created a cluster for each test. Reusing the same servers allows
faster testing by reusing the already running servers.
2017-08-09 11:39:24 +03:00
Markus Mäkelä
cb066fd09a Fix memory leaks in REST API
The call to MHD_basic_auth_get_username_password allocates memory for both
the password and the username.

mxs_json_add_relation leaked a reference to a JSON object by using
json_array_append instead of json_array_append_new.
2017-08-09 11:39:24 +03:00
Markus Mäkelä
db531fb2e2 MXS-1300: Improve MaxCtrl HTTP error messages
The messages now show where the request failed and what was
requested. This should help resolve both develper and end-user problems.

Also fixed the missing logging of the output string in the `parse`
callback of the main function and cleaned up the POSTed server body.
2017-08-09 11:39:24 +03:00
Markus Mäkelä
8f7be8a4f3 MXS-1300: Fix MaxCtrl cluster tests
Fixed tests to use correct functions, extended tests of problematic
cases. Added a secondary directory for the second MaxScale instance.
2017-08-09 11:39:24 +03:00
Markus Mäkelä
d2543c4841 Fix refactoring errors
The monitor deletion would mistakenly label all monitors as not being
created at runtime. Due to this, the deletion of monitors would fail.
2017-08-09 11:39:24 +03:00
Markus Mäkelä
05d185fc02 Fix monitor repurposing
The monitor active state is now modified under the same lock. This should
make creation and destruction of monitors deterministic.
2017-08-09 11:39:24 +03:00
Markus Mäkelä
1743f4c1b7 Handle server repurposing under a lock
By moving the repurposing of the servers under the global server lock, the
repurposing of a server and allocation of a new server behave in the same
way.

Also fixed the wrong error message on server creation failure referring to
invalid server relationships.
2017-08-09 11:39:24 +03:00
Markus Mäkelä
e133e758a6 MXS-1300: Fix deletion of monitors
The monitors should only be reused if they have the same name and they use
the same module. This way the only difference is in configuration.

Fixed MaxCtrl detection of bad options and altered monitor creation test
to expect correct results. Also improved some of the error messages.
2017-08-09 11:39:24 +03:00
Markus Mäkelä
af847c29c3 Improve runtime configuration error messages
The error messages are now more specific and describe in detail what is
wrong in the request.
2017-08-09 11:39:24 +03:00
Markus Mäkelä
161776d576 MXS-1300: Add more MaxCtrl cluster tests
Added scripts that start two MaxScale on the same machine. This allows the
cluster synchronization commands to be tested locally.
2017-08-09 11:39:24 +03:00
Markus Mäkelä
3bd13cded6 MXS-1300: Fix minor MaxCtrl bugs
Fixed unresolved promises, wrong resources being requested and enabled
strict mode so unknown arguments are not ignored.
2017-08-09 11:39:24 +03:00
Markus Mäkelä
512c3c018d Add recycling of destroyed monitors
If a destroyed monitor is created again, it will be reused. This should
prevent excessive memory growth when the same monitor is created and
destroyed again.
2017-08-09 11:39:24 +03:00
Markus Mäkelä
a3e9d5c327 MXS-1300: Add creation/deletion of listeners to cluster sync
New listeners are created and old ones removed. As listeners cannot yet be
altered via the REST API, differing listeners are not modified.
2017-08-09 11:39:24 +03:00
Markus Mäkelä
0dc53df32e MXS-1300: Add listener diffs
The diff command now shows new, removed and changed listeners.
2017-08-09 11:39:24 +03:00
Markus Mäkelä
10bfa072d9 MXS-1300: Add test for cluster command internals
The test tests the internals of the cluster diff generation
algorithms. This should guarantee that the diff calculation and the
resulting synchronization will work as expected without having to test it
on a live cluster.
2017-08-09 11:39:24 +03:00
Markus Mäkelä
8fb4532468 MXS-1300: Add cluster sync command
The `cluster sync` command synchronizes a MaxScale cluster with one
server. This allows new MaxScale instances to be synchronized with an old
instance by using the same configuration file and performing a cluster
sync.
2017-08-09 11:39:23 +03:00
Markus Mäkelä
fad1b4e3e0 MXS-1300: Add simple diff calculation
The `cluster diff` command prints out the difference between the source
and target MaxScale. This allows users to detect changes that have
happened on one MaxScale but not on the other.
2017-08-09 11:39:23 +03:00
Markus Mäkelä
2c92a86fd6 MXS-1300: Move MaxCtrl into a separate package
Moved MaxCtrl into a separate package to simplify packaging and
distribution process. Having the client as a separate component allows one
set of packages to be used on multiple platforms.
2017-08-09 11:39:23 +03:00
Markus Mäkelä
666568e1a0 Fix rule files for masking tests
The tests used invalid JSON in the rule files.
2017-08-09 11:38:09 +03:00
Johan Wikman
c11ffd91c5 MXS-1349 Access right field when picking out preparable stmt
The preparable statement is stored in two fields; in one in full
and in another at most 255 chars.
2017-08-08 15:46:01 +03:00
Johan Wikman
fdb588b866 MXS-1307 Tweak cte tests
The "parser" of compare cannot handle the following:

  prepare stmt1 from "
  with t as (select a from t1 where b >= 'c')
    select * from t2,t where t2.c=t.a;
  ";

That is, that a line *not* terminating the full statement (the
line 'select * from t2,t where t2.c=t.a;' above) ends with a ';'.
Thus, it needs to be changed into e.g.

  prepare stmt1 from "
  with t as (select a from t1 where b >= 'c')
    select * from t2,t where t2.c=t.a;";
2017-08-08 15:46:01 +03:00
Johan Wikman
a02bc3d40b MXS-1348 Treat usage differences as non-fatal
In the context of subselects and CTEs it is somewhat difficult to
make qc_sqlite and qc_mysqlembedded to agree upon in what context
a particular field is used. As that information is not used anywhere,
a usage discrepancy is for now treated as a warning.
2017-08-08 15:46:01 +03:00
Johan Wikman
1d693640c7 MXS-1337 All callbacks now exception safe
No exceptions can now ever escape from the C++ side of qc_sqlite
to sqlite3 (where they would cause havoc).
2017-08-08 15:46:01 +03:00
Johan Wikman
e084c0a464 MXS-1337 Thread initialization is checked by explicit callbacks
When any of the QcSqliteInfo member functions are called, they
can assume that thread initialization has been performed.
2017-08-08 15:46:01 +03:00
Johan Wikman
83a5614d2a MXS-1337 Initialization status moved to thread specific data
The initialization state is thread specific, not QcSqliteInfo
instance specific.
2017-08-08 15:46:01 +03:00
Johan Wikman
5c68494044 MXS-1337: More functions moved into QcSqliteInfo
All callbacks called by sqlite now only access the thread specific
QcSqliteInfo and call the corresponding function on that instance.
This is the first step in making qc_sqlite exception safe from the
point of view of sqlite3.

As the diff is confusing, basically the ONLY thing that has been
done is:

BEFORE:
-------
class QcSqliteInfo
{
    ...
};

static void some_helper(...) { ... }

void mxs_someCallback(...)
{
   QC_TRACE();
   QcSqliteInfo* info = this_thread.pInfo;
   ss_dassert(info);

   info->m_status = ...;
   some_helper(info, ...);
}

AFTER:
------
class QcSqliteInfo
{
    ...
    void some_helper(...) { ... }

    void mxs_someCallback(...)
    {
        m_status = ...;
        some_helper(this, ...);
    }
};

void mxs_someCallback(...)
{
   QC_TRACE();
   QcSqliteInfo* pInfo = this_thread.pInfo;
   ss_dassert(pInfo);

   pInfo->mxs_someCallback(...);
}
2017-08-08 15:46:01 +03:00
Johan Wikman
4f4151bca9 MXS-1340 Report true table and not alias name
With this change, for a statement like

    SELECT t2.a FROM t1 t2;

the affected field is reported as t1.a and not as t2.a, as it
was before.

For a statement like

    SELECT t.f FROM d.t;

qc_mysqlembedded will now return "d.t.f" as the affected field,
while qc_sqlite will still return "t.f" as both implementations did
before. In qc_mysqlembedded's case that is a side-effect of the
alias handling. To get qc_sqlite to return the same (which would
be good), the table names would have to be collected in a smarter
way than they are now.
2017-08-08 15:46:01 +03:00
MassimilianoPinto
b9588a89ac MXS-770: Removed support for PURGE BINARY LOGS without TO option.
PURGE BINARY LOGS doesn’t exist in MySQL/ MariaDB without options.

MaxScale binlog router supports only PURGE BINARY LOGS TO ‘file’
2017-08-08 10:02:48 +02:00
MassimilianoPinto
f9db51eb35 MXS 770 Documentation Update
Binlog router documentation update with PURGE BINARY LOGS TO 'file'
2017-08-08 08:56:43 +02:00
Esa Korhonen
2708a4d7ac Add PAM authenticator documentation
Also fixed some small errors in general authenticator documentation.
2017-08-07 12:22:59 +03:00
Esa Korhonen
ed05d24a9a Move SSL-code in mysql_auth.c and pam_client_session.cc to
a separate function in ssl.cc

Removes some duplicate code.
2017-08-07 12:22:59 +03:00
Esa Korhonen
8ef8ee6600 Add client-to-MaxScale SSL support to PAM authenticator
Only client-side SSL is supported for now.
2017-08-07 12:22:59 +03:00
Esa Korhonen
7488129afc PAM code cleanup & refactor
Divided functionality into classes, fixed comments +
various other cleanup. BackenAuth no longer increments
sequence on sending password. SQLite busy timeout shortened
to 1 second.
2017-08-07 12:22:59 +03:00
Esa Korhonen
f916b74c2e Add first version of PAM authenticator plugin
This includes the client and backend authenticators. Currently,
only a simple password-based scheme with the SQL-client "dialog" plugin
is supported. In this mode, the server sends the first PAM message
with the AuthSwitchRequest packet and the client responds with the
password. No further authentication messages are supported. If the
connection is not encrypted, the password is sent in plaintext. The
client password is used as is for logging in to backends.
2017-08-07 12:22:59 +03:00
MassimilianoPinto
a9b0eb791b MXS-770: PURGE BINARY LOGS, minor improvements.
PURGE BINARY LOGS, minor improvements.
2017-08-07 10:26:28 +02:00
MassimilianoPinto
113d2ad87a MXS-770 Addition of PURGE BINARY LOGS feature
PURGE BINARY LOGS; deletes all files in binlogdir and GTID maps repo
but keeps current binlog file.

PURGE BINARY LOGS TO ‘file’; deletes all files in binlogdir and GTID
maps repo up to specified file.

mariadb10_slave_gtid=On option is needed in order to keep the list of
binlog files.
2017-08-07 10:26:28 +02:00
Johan Wikman
a438ff7c86 MXS-1340 is_sequence_related... functions moved to QcSqliteInfo 2017-08-07 08:31:41 +03:00
Johan Wikman
c29e8b2b9e MXS-1340 Move update_field_info to QcSqliteInfo 2017-08-07 08:31:41 +03:00