8283 Commits

Author SHA1 Message Date
Markus Mäkelä
a3e9d5c327 MXS-1300: Add creation/deletion of listeners to cluster sync
New listeners are created and old ones removed. As listeners cannot yet be
altered via the REST API, differing listeners are not modified.
2017-08-09 11:39:24 +03:00
Markus Mäkelä
0dc53df32e MXS-1300: Add listener diffs
The diff command now shows new, removed and changed listeners.
2017-08-09 11:39:24 +03:00
Markus Mäkelä
10bfa072d9 MXS-1300: Add test for cluster command internals
The test tests the internals of the cluster diff generation
algorithms. This should guarantee that the diff calculation and the
resulting synchronization will work as expected without having to test it
on a live cluster.
2017-08-09 11:39:24 +03:00
Markus Mäkelä
8fb4532468 MXS-1300: Add cluster sync command
The `cluster sync` command synchronizes a MaxScale cluster with one
server. This allows new MaxScale instances to be synchronized with an old
instance by using the same configuration file and performing a cluster
sync.
2017-08-09 11:39:23 +03:00
Markus Mäkelä
fad1b4e3e0 MXS-1300: Add simple diff calculation
The `cluster diff` command prints out the difference between the source
and target MaxScale. This allows users to detect changes that have
happened on one MaxScale but not on the other.
2017-08-09 11:39:23 +03:00
Markus Mäkelä
2c92a86fd6 MXS-1300: Move MaxCtrl into a separate package
Moved MaxCtrl into a separate package to simplify packaging and
distribution process. Having the client as a separate component allows one
set of packages to be used on multiple platforms.
2017-08-09 11:39:23 +03:00
Markus Mäkelä
666568e1a0 Fix rule files for masking tests
The tests used invalid JSON in the rule files.
2017-08-09 11:38:09 +03:00
Johan Wikman
c11ffd91c5 MXS-1349 Access right field when picking out preparable stmt
The preparable statement is stored in two fields; in one in full
and in another at most 255 chars.
2017-08-08 15:46:01 +03:00
Johan Wikman
fdb588b866 MXS-1307 Tweak cte tests
The "parser" of compare cannot handle the following:

  prepare stmt1 from "
  with t as (select a from t1 where b >= 'c')
    select * from t2,t where t2.c=t.a;
  ";

That is, that a line *not* terminating the full statement (the
line 'select * from t2,t where t2.c=t.a;' above) ends with a ';'.
Thus, it needs to be changed into e.g.

  prepare stmt1 from "
  with t as (select a from t1 where b >= 'c')
    select * from t2,t where t2.c=t.a;";
2017-08-08 15:46:01 +03:00
Johan Wikman
a02bc3d40b MXS-1348 Treat usage differences as non-fatal
In the context of subselects and CTEs it is somewhat difficult to
make qc_sqlite and qc_mysqlembedded to agree upon in what context
a particular field is used. As that information is not used anywhere,
a usage discrepancy is for now treated as a warning.
2017-08-08 15:46:01 +03:00
Johan Wikman
1d693640c7 MXS-1337 All callbacks now exception safe
No exceptions can now ever escape from the C++ side of qc_sqlite
to sqlite3 (where they would cause havoc).
2017-08-08 15:46:01 +03:00
Johan Wikman
e084c0a464 MXS-1337 Thread initialization is checked by explicit callbacks
When any of the QcSqliteInfo member functions are called, they
can assume that thread initialization has been performed.
2017-08-08 15:46:01 +03:00
Johan Wikman
83a5614d2a MXS-1337 Initialization status moved to thread specific data
The initialization state is thread specific, not QcSqliteInfo
instance specific.
2017-08-08 15:46:01 +03:00
Johan Wikman
5c68494044 MXS-1337: More functions moved into QcSqliteInfo
All callbacks called by sqlite now only access the thread specific
QcSqliteInfo and call the corresponding function on that instance.
This is the first step in making qc_sqlite exception safe from the
point of view of sqlite3.

As the diff is confusing, basically the ONLY thing that has been
done is:

BEFORE:
-------
class QcSqliteInfo
{
    ...
};

static void some_helper(...) { ... }

void mxs_someCallback(...)
{
   QC_TRACE();
   QcSqliteInfo* info = this_thread.pInfo;
   ss_dassert(info);

   info->m_status = ...;
   some_helper(info, ...);
}

AFTER:
------
class QcSqliteInfo
{
    ...
    void some_helper(...) { ... }

    void mxs_someCallback(...)
    {
        m_status = ...;
        some_helper(this, ...);
    }
};

void mxs_someCallback(...)
{
   QC_TRACE();
   QcSqliteInfo* pInfo = this_thread.pInfo;
   ss_dassert(pInfo);

   pInfo->mxs_someCallback(...);
}
2017-08-08 15:46:01 +03:00
Johan Wikman
4f4151bca9 MXS-1340 Report true table and not alias name
With this change, for a statement like

    SELECT t2.a FROM t1 t2;

the affected field is reported as t1.a and not as t2.a, as it
was before.

For a statement like

    SELECT t.f FROM d.t;

qc_mysqlembedded will now return "d.t.f" as the affected field,
while qc_sqlite will still return "t.f" as both implementations did
before. In qc_mysqlembedded's case that is a side-effect of the
alias handling. To get qc_sqlite to return the same (which would
be good), the table names would have to be collected in a smarter
way than they are now.
2017-08-08 15:46:01 +03:00
MassimilianoPinto
b9588a89ac MXS-770: Removed support for PURGE BINARY LOGS without TO option.
PURGE BINARY LOGS doesn’t exist in MySQL/ MariaDB without options.

MaxScale binlog router supports only PURGE BINARY LOGS TO ‘file’
2017-08-08 10:02:48 +02:00
MassimilianoPinto
f9db51eb35 MXS 770 Documentation Update
Binlog router documentation update with PURGE BINARY LOGS TO 'file'
2017-08-08 08:56:43 +02:00
Esa Korhonen
2708a4d7ac Add PAM authenticator documentation
Also fixed some small errors in general authenticator documentation.
2017-08-07 12:22:59 +03:00
Esa Korhonen
ed05d24a9a Move SSL-code in mysql_auth.c and pam_client_session.cc to
a separate function in ssl.cc

Removes some duplicate code.
2017-08-07 12:22:59 +03:00
Esa Korhonen
8ef8ee6600 Add client-to-MaxScale SSL support to PAM authenticator
Only client-side SSL is supported for now.
2017-08-07 12:22:59 +03:00
Esa Korhonen
7488129afc PAM code cleanup & refactor
Divided functionality into classes, fixed comments +
various other cleanup. BackenAuth no longer increments
sequence on sending password. SQLite busy timeout shortened
to 1 second.
2017-08-07 12:22:59 +03:00
Esa Korhonen
f916b74c2e Add first version of PAM authenticator plugin
This includes the client and backend authenticators. Currently,
only a simple password-based scheme with the SQL-client "dialog" plugin
is supported. In this mode, the server sends the first PAM message
with the AuthSwitchRequest packet and the client responds with the
password. No further authentication messages are supported. If the
connection is not encrypted, the password is sent in plaintext. The
client password is used as is for logging in to backends.
2017-08-07 12:22:59 +03:00
MassimilianoPinto
a9b0eb791b MXS-770: PURGE BINARY LOGS, minor improvements.
PURGE BINARY LOGS, minor improvements.
2017-08-07 10:26:28 +02:00
MassimilianoPinto
113d2ad87a MXS-770 Addition of PURGE BINARY LOGS feature
PURGE BINARY LOGS; deletes all files in binlogdir and GTID maps repo
but keeps current binlog file.

PURGE BINARY LOGS TO ‘file’; deletes all files in binlogdir and GTID
maps repo up to specified file.

mariadb10_slave_gtid=On option is needed in order to keep the list of
binlog files.
2017-08-07 10:26:28 +02:00
Johan Wikman
a438ff7c86 MXS-1340 is_sequence_related... functions moved to QcSqliteInfo 2017-08-07 08:31:41 +03:00
Johan Wikman
c29e8b2b9e MXS-1340 Move update_field_info to QcSqliteInfo 2017-08-07 08:31:41 +03:00
Johan Wikman
eecd7a03c5 MXS-1340 Collect alias names
Alias names for tables are now collected, so that the true
table name of a referred to field can be reported. That modification
will be made in a subsequent commit.
2017-08-07 08:31:41 +03:00
MassimilianoPinto
5933062666 Binlog router documentation update.
Removed SHOW FULL BINARY LOGS and added details about SHOW BINARY LOGS
output when option binlog_structure=tree
2017-08-04 11:20:07 +02:00
Esa Korhonen
8e9c943d48 Proxy protocol setting error detection + documentation
The setting parsing is now similar to the other server settings.
The header is printed if log_info is on.
Changed the setting name to simply "proxy_protocol".
Updated documentation.
2017-08-04 11:43:29 +03:00
MassimilianoPinto
75b17151f3 Removed FULL option from SHOW BINARY LOGS
If option ‘binlog_structure’ is set to ‘tree’ then SHOW BINARY LOGS
displays the tree details of the binlog files.

MySQL [(none)]> SHOW BINARY LOGS;
+--------------------------+-----------+
| Log_name                 | File_size |
+--------------------------+-----------+
| 0/10122/mysql-bin.000117 |      1167 |
| 0/10122/mysql-bin.000118 |       652 |
| 0/10124/foo-bin.000016   |      5082 |
| 0/10124/foo-bin.000017   |       491 |
+--------------------------+-----------+

With option set to ‘flat’ (which is the default) the output contains
only
names:

MySQL [(none)]> SHOW BINARY LOGS;
+------------------+-----------+
| Log_name         | File_size |
+------------------+-----------+
| mysql-bin.000110 |       425 |
| mysql-bin.000111 |     10409 |
| mysql-bin.000112 |      9659 |
+------------------+-----------+
2017-08-04 10:08:11 +02:00
Markus Mäkelä
1f4856cdc5 Merge branch '2.1' into develop 2017-08-03 19:01:32 +03:00
Markus Mäkelä
f7b8744460 Add more error messages to monitors
When the execution of a query fails, the error reported by the Connector-C
and the server where the query was executed is logged.
2017-08-03 15:42:40 +03:00
Timofey Turenko
ab82c7455c remove manual flex installation 2017-08-03 15:38:19 +03:00
MassimilianoPinto
ae5eb004fe SHOW [FULL] BINARY LOGS is now able to report the same log file in use with different server_ids
SHOW [FULL] BINARY LOGS is now able to report the same filename in use
with different server_ids: this can happen with binlog_structure=tree

example from SHOW FULL BINARY LOGS

0/10122/mysql-bin.000113
…
0/10122/mysql-bin.000116
…
0/5306/mysql-bin.000113

SHOW BINARY LOGS shows the same file twice:

mysql-bin.000113
…
mysql-bin.000116
…
mysql-bin.000113
2017-08-03 11:11:37 +02:00
Johan Wikman
72c5d1844a MXS-1337 Add variable prefixes
Member variables of QcSqliteInfo now have m_, p, z and n prefixes
as appropriate.
2017-08-03 08:36:25 +03:00
Johan Wikman
7a5e3ede21 MXS-1339 QC: Report each table just once
If a particular table appears in a statement multiple times,
qc_get_table_names will report it as many times as it appears.
Each name should be reported just once. Same applies for
database names.
2017-08-03 08:34:22 +03:00
MassimilianoPinto
d5de149e32 Update GTID repository while receiving ROTATE events and check the current file exists.
Update GTID repository while receiving ROTATE events and check the
current file exists.
2017-08-02 17:32:07 +02:00
Markus Mäkelä
bda1c5d55d Fix use-after-free in buffer.c
The fix to MXS-1338 added a bug where the buffer objects were freed after
the shared buffer was freed.
2017-08-02 17:55:35 +03:00
Markus Mäkelä
1ec7d9a712 Fix FindNodeJS.cmake
Only `node` executable was searched.
2017-08-02 16:04:29 +03:00
Markus Mäkelä
92bc3f046e Fix build failures on CentOS 6
Removed superfluous thread_local storage specifier from adminusers.cc and
fixed signed to unsigned integer comparisons in qc_sqlite.cc.
2017-08-02 15:44:45 +03:00
MassimilianoPinto
7d8407ff22 Binlog Server doesn't start without last found GTID in the repo only if “mariadb10_master_gtid” option is On
Binlog Server doesn't start without last found GTID in the repo only if
“mariadb10_master_gtid” option is On
2017-08-02 11:56:32 +02:00
Markus Mäkelä
854c4a1ed3 Add support for non-glibc systems
MaxScale can now be built on systems that use an alternative libc
implementation e.g. musl.
2017-08-02 11:51:55 +03:00
Markus Mäkelä
782b8db2aa Fix readwritesplit handling of unexpected responses
The backend server can send a response even if the client hasn't sent a
request. One case where this occurs is when the server is shutting
down. The internal logic of readwritesplit can't handle unexpected states
gracefully so the safest thing to do is to just ignore them and send the
responses to the client.
2017-08-02 11:51:26 +03:00
Markus Mäkelä
63b16243e7 MXS-1338: Fix memory leak of after buffer cloning
When a buffer is cloned and then the original buffer parsed and freed, the
freeing of the cloned buffer will not release the memory that was
allocated when the original buffer is parsed.

This is a side-effect of how the buffer objects are stored in the buffer
and not in the shared memory buffer. The creation of a buffer object after
cloning will cause the buffer object to be lost as the cloned buffer
didn't have a pointer to the buffer object that was created later.

By moving the buffer objects into the shared memory buffer, the memory
leak is fixed.
2017-08-02 11:09:49 +03:00
Johan Wikman
542f3b69db MXS-1337 update_names(...) moved to QcSqliteInfo
The update_names() will have to be updated to take a possible alias
name as well. That needs to be tracked inside QcSqliteInfo, so that
when there is a statement like "select t2.a from t1 t2" we report
the field as t1.a and not as t2.a.
2017-08-02 09:11:35 +03:00
Johan Wikman
cfb5a315cf MXS-1337 Some variables renamed 2017-08-02 09:11:35 +03:00
Johan Wikman
19d36d425d MXS-1337 More functionality moved into QcSqliteInfo 2017-08-02 09:11:35 +03:00
Johan Wikman
cb15d90da9 MXS-1337 qc_sqlite info structure now a C++ class
The structure for storing statement information is now a C++ class.
At least initially it will not be turned into a full-fledged class
that would contain all functionality, but the code will be a mishmash
of C and C++.
2017-08-02 09:11:35 +03:00
MassimilianoPinto
0597b625d3 SHOW FULL BINARY LOGS reports domain_id/server_id prefix only if binlog_structure is TREE
SHOW FULL BINARY LOGS reports domain_id/server_id prefix only if
binlog_structure is TREE
2017-08-01 15:49:16 +02:00
Markus Mäkelä
0dbf64b1ab Fix broken link in 2.1.5 release notes
The link to the list of issues was broken.
2017-08-01 10:50:11 +03:00