Commit Graph

13103 Commits

Author SHA1 Message Date
de3f4fe5a8 MXS-2346 Ttl should be stored as milliseconds
cacheconfig.hh had not been pushed earlier.
2019-04-12 15:48:28 +03:00
0c7a3240bd MXS-2346 Exclude core parameters
Core parameters are not handled by the module but by the core and
must hence be ignored when validating and configuring.
2019-04-12 15:03:02 +03:00
c381aefefc MXS-2346 Add config::ParamInteger and config::Integer 2019-04-12 15:03:02 +03:00
17aa494c87 MXS-2346 Update cache filter
As a proof of concept, the cache filter has been updated to
use the new configuration mechanism.
2019-04-12 15:03:02 +03:00
09702ab0a0 MXS-2346 Provide new configuration mechanism
The configuration mechanism consists of the following concepts:

Specification
  Specifies the available configuration parameters of a module,
  their names and their types.
Param
  Specifies a parameter, its name and its type.
Type
  Specifies the type of a configuration parameters; Bool, Size,
  Count, etc.
Configuration
  Specifies the configuration values of a particular instance of
  the module. Configuration walks hand in hand with Specification,
  the latter specifies what the former should contain.

A Specification is capable of configuring a Configuration from a
MXS_CONFIG_PARAMETER, checking in the process that all parameters
are of the correct type and that the required parameters are present.

A Specification is capable of persisting itself so that it later
can be read back.

The mechanism is closed for modification but open for extension in
the sense that if a module requires a custom parameter, all it needs
to do is to derive one class from Param and another from Type.

The canonical way for using this mechanism is as follows. Consider
a module xyx that has three parameters; a parameter called
"enabled" that is of boolean type, a parameter called "period"
that is of duration type, and a parameter "cache" that is of
size type. That would be declared as follows:

    // xyz.hh
    class XYZSession;

    class XYZ : public maxscale::Filter<XYZ, XYZSession>
    {
    public:
        static XYZ* create(const char* zName, MXS_CONFIG_PARAMETER* pParams);

    private:
        XYZ();

        static config::Specification                       s_specification;
        static config::ParamBool                           s_enabled;
        static config::ParamDuration<std::chrono::seconds> s_period;
        static config::ParamSize                           s_cache;

        config::Configuration                              m_configuration;
        config::Bool                                       m_enabled;
        config::Duration<std::chrono::seconds>             m_period;
        config::Size                                       m_cache;
    };

    // xyz.cc

    config::Specification XYZ::s_specification(MXS_MODULE_NAME);

    config::ParamBool XYZ::s_enabled(
        &s_specification,
        "enabled",
        "Specifies whether ... should be enabled or not."
        );
    config::ParamDuration<std::chrono::seconds> XYZ::s_period(
        &s_specification,
        "period",
        "Specifies the period. Rounded to the nearest second."
        );
    config::ParamSize XYZ::s_cache(
        &s_specification,
        "cache",
        "Specifies the size of the internal cache."
        );

    XYZ::XYZ()
        : m_configuration(&s_specification)
        , m_enabled(&m_configuration, &s_enabled)
        , m_period(&m_configuration, &s_period)
        , m_cache(&m_configuration, &s_cache)
    {
    }

    XYZ* XYZ::create(const char* zName, MXS_CONFIG_PARAMETER* pParams)
    {
        XYZ* pXyz = new XYZ;

        if (!s_specification.configure(pXyz->m_configuration, pParams))
        {
            delete pXyz;
            pXyz = nullptr;
        }

        return pXyz;
    }
2019-04-12 15:03:02 +03:00
d2c71472b0 MXS-2432 Recognize RESET
RESET QUERY CACHE is reported to be a session command, which will
cause it to be sent to all servers. RESET [MASTER|SLAVE] are
classified as write, which will cause them to be sent to the master.

It could be argued that RESET [MASTER|SLAVE] should cause an error
to be sent to the client.
2019-04-12 14:51:01 +03:00
3127aa85c5 MXS-2432 Add test that reveals issue
RESET is neither parsed not tokenized, so RESET statements ends
up being sent to the master. That is fine for all but RESET QUERY
CACHE that should be sent to all servers.
2019-04-12 14:51:01 +03:00
c643f9bc8d Merge branch '2.3' into develop 2019-04-12 13:23:49 +03:00
382d485c2e Add history to maxctrl cli 2019-04-12 13:02:02 +03:00
961cb80521 MXS-2313: Add hint test case
The test verifies that hints override priority.
2019-04-12 13:02:02 +03:00
1c87e2facf Add changes to admin user hashing to release notes 2019-04-12 13:02:01 +03:00
4131f09c16 MXS-2431 Recognize the XA keyword
Recognize the XA keyword and classify the statement as write.
Needs to be dealt with explicitly as sqlite3 assumes there are
no keywords starting with the letter X.
2019-04-12 11:05:15 +03:00
62f2a86a5f MXS-2431 Add test that reveals the problem 2019-04-12 10:30:36 +03:00
1652b18a7b Fix whitespace in canonicalized queries
Trailing whitespace was not removed and whitespace wasn't normalized to
spaces.
2019-04-12 09:18:07 +03:00
d2ecaa83a6 Move result start handling into separate function
The largest part of the code deals with the start of a response. Moving
this into a subfunction makes the function clearer as the switch statement
inside a switch statement is removed.
2019-04-12 09:18:07 +03:00
746bd53668 Simplify RWBackend result handling
By processing the packets one at a time, the reply state is updated
correctly regardless of how many packets are received. This removes the
need for the clunky code that used modutil_count_signal_packets to detect
the end of the result set.
2019-04-12 09:18:05 +03:00
e6526dd9ea Add extra info logging to readwritesplit
Added logging into RWBackend reply state processing code to know more.
2019-04-12 09:17:48 +03:00
139651c092 MXS-2253 Runtime ttl changes are made in seconds
Internally durations are stored in milliseconds but runtime changes
using SQL are made in seconds. Consequently, the provided value must
be multiplied by 1000 before being stored.
2019-04-11 15:53:45 +03:00
74634abc80 MXS-1662 Move PAM authentication function into maxbase
The same code can be used for REST-API authentication.
2019-04-09 14:41:40 +03:00
bc5f9da6c4 Add classification test case
Also removed the dead code that was never used to get coverage to 100%.
2019-04-09 10:56:38 +03:00
9a5b60a071 Add forced maintenance mode tests
Tested that the force option works and is accepted.
2019-04-09 10:00:50 +03:00
0932d10169 Do node checks in parallel
The checking of MariaDB and Galera nodes is now done asynchronously while
the MaxScale check is done which leads to faster testing. Rough
measurements show that doing all the work in parallel reduces test startup
time by two seconds. Most of the time appears to still be in the MaxScale
startup which takes on average three to four seconds per test.
2019-04-09 09:43:19 +03:00
5e3af05d48 Speed up test startup
The VM connectivity and log truncation is now done in parallel.
2019-04-09 09:43:19 +03:00
55bb3e9250 Make tests less verbose
The numeric values of the labels aren't of value when inspecting test
results. For this reason, it makes sense to put them behind the verbose
flag to make test framework debugging happen purpose.
2019-04-09 09:43:19 +03:00
a102efa01f Backport: Document the force option for set
Added documentation for the new option and mentioned it in the release
notes.
2019-04-09 09:43:19 +03:00
0cb15976e8 Backport: Add force option to set endpoint
The new `force=yes` option closes all connections to the server that is
being put into maintenance mode. This will immediately close all open
connections to the server without allowing results to return.
2019-04-09 09:43:18 +03:00
7fb840ac9e Sort CN_ definitions 2019-04-09 09:43:18 +03:00
ffd2d80ea0 Link with rt
On CentOS 6 (glibc 2.12) you must explicitly link with rt for
clock_gettime.
2019-04-08 11:24:30 +03:00
05515cca16 MXS-2259: Limit size of client reads
Given the assumption that queries are rarely 16MB long and that
realistically the only time that happens is during a large dump of data,
we can limit the size of a single read to at most one MariaDB/MySQL packet
at a time. This change allows the network throttling to engage a lot
sooner and reduces the maximum overshoot of throtting to 16MB.
2019-04-05 22:48:16 +03:00
aad29404c6 Fix parameter value error
The argumets were given in the wrong order.
2019-04-05 13:33:16 +03:00
b54e67223f MXS-2423: Add missing parameters to maxscale endpoint
Also updated the REST API documentation to include the newer output
(automating this update would be valuable).
2019-04-05 13:33:16 +03:00
9722c0887a Log connection ID when reading server handshake
By logging the connection ID for each created connection, failures can be
traced back from the backend server all the way up to the client
application.
2019-04-05 13:33:16 +03:00
ec890b33cd Prevent checksum mismatch on second trx replay
If a transaction replay has to be executed twice due to a failure of the
original candidate master, the query queue could contain replayed
queries. The replayed queries would be placed into the queue if a new
connection needs to be created before the transaction replay can start.
2019-04-05 13:33:16 +03:00
6421af1bb4 Backport query queue changes to 2.3
Backported the changes that convert the query queue in readwritesplit into
a proper queue. This changes combines both
5e3198f8313b7bb33df386eb35986bfae1db94a3 and
6042a53cb31046b1100743723567906c5d8208e2 into one commit.
2019-04-05 13:33:16 +03:00
31c93cfe1c Use SHA2-512 for admin users
MD5 is not secure enough with the modern hardware. Upgrading to SHA2-512
helps move the problem to the future.
2019-04-05 01:00:47 +03:00
adba581a4d Fix addition of admin users
The user passwords were stored in plaintext format.
2019-04-05 01:00:44 +03:00
daf5f52c64 Pass raw password to users_auth
By passing the raw password deeper into the authentication code, it can be
used to verify the user can access some systems. Right now, this is not
required by the simple salted password comparison done in MaxScale.
2019-04-05 00:42:00 +03:00
40d73948a9 MXS-1662: Move mxs_crypt into utils
Moved the mxs_crypt function into utils and renamed to mxs::crypt (no C
code used it).
2019-04-05 00:42:00 +03:00
d2f31aab0a MXS-2420 Add debug function for decoding response packet
When debugging you occasionally want to find out what a packet
contains (e.g. delivered to clientReply). Manually looking into
the packet works, but is tedious. With this function you when
the execution has been stopped in GDB examine a protocol packet.

E.g.

Thread 3 "maxscale" hit Breakpoint 6, RWSplitSession::clientReply (this=0x7fffe401ed20, writebuf=0x7fffe401e910, backend_dcb=0x7fffe401dbe0) at /home/wikman/MariaDB/MaxScale/server/modules/routing/readwritesplit/rwsplitsession.cc:567
567	    DCB* client_dcb = backend_dcb->session->client_dcb;
(gdb) p dbg_decode_response(writebuf)
$30 = 0x7ffff0d40d54 "Packet no: 1, Payload len: 44, Command : ERR, Code: 1146, Message : Table 'test.blahasdf' doesn't exist"
2019-04-04 16:05:16 +03:00
2aa3515fc8 Merge commit '09cb4a885f88d30b5108d215dcdaa5163229a230' into develop 2019-04-04 14:34:17 +03:00
f812302d62 Document readwritesplit behavior with server states 2019-04-04 13:00:35 +03:00
556c83f83a MXS-2417: Add test case 2019-04-04 08:59:28 +03:00
b08d4e37b5 MXS-2416: Pass deleter to std::shared_ptr<GWBUF>
As shared_ptr doesn't automatically use std::default_delete<T>, it needs
to be explicitly passed to the constructor.
2019-04-03 12:57:06 +03:00
09cb4a885f Fix readconnroute name in examples 2019-04-03 12:57:05 +03:00
e3e66f8e90 MXS-2417: Add option to ignore persisted configs
The load_persisted_configs parameter now controls whether persisted
runtime changes are loaded on startup. The changes are still generated as
it persists the current state of MaxScale making problem analysis easier.
2019-04-03 12:57:05 +03:00
a217dde1f0 MXS-2419: Queue queries executed during trx replay
By storing the queries in the query queue and routing it once the
transaction replay is done, we prevent two problems:

* Multiple transaction replays would overwrite the m_interrupted_query
  buffer that was used to store any queries executed during the
  transaction replay.

* Incorrect ordering of queries when the query queue is not empty and a
  new query is executed during transaction replay.
2019-04-03 12:57:05 +03:00
2dfd7d35ac MXS-2418: Crash on trx replay when log_info is enabled
If the session starts with no master but later one becomes available, when
a transaction is started the code would unconditionally use the master's
name in a log message.
2019-04-03 12:57:05 +03:00
cc1b8b2d56 Use std::begin and std::end
This makes it clear that they are standard functions.
2019-04-02 14:21:54 +03:00
5242cd5ebf Readwritesplit: Graceful maintenance mode
By allowing transactions to the master to end even if the server is in
maintenance mode makes it possible to terminate connections at a known
point. This helps prevent interrupted transactions which can help reduce
errors that are visible to the clients.
2019-04-02 14:21:54 +03:00
319d9fa1c5 Document the force option for set
Added documentation for the new option and mentioned it in the release
notes.
2019-04-02 14:21:54 +03:00