Commit Graph

1323 Commits

Author SHA1 Message Date
510cae2fe0 Allocate DCB on owning thread
The DCB is now fully allocated on the thread that owns it. This guarantees
that the owner is always correct when it is used.

The code in poll_add_dcb still manipulates which worker the DCB is
allocated. This needs to be removed and the detection of special needs
(maxadmin, maxinfo) must be moved into the listener.
2019-04-26 11:40:30 +03:00
4947081377 The std::ostringstream was incorrectly cleared in the LOG macro. 2019-04-25 15:24:10 +03:00
fadbc0b1ae Separate Monitor management to its own file
Allows better separation of file local data. Also allows moving monitor-
related code from config_runtime.cc.
2019-04-25 12:32:41 +03:00
6aedcc085f Remove references to NDB server state 2019-04-24 14:15:56 +03:00
75c0ac5323 Move items from MonitorWorker to MonitorWorkerSimple
MonitorWorker only enforces the use of a worker thread but otherwise
does not define how the monitor is implemented.
2019-04-24 11:27:11 +03:00
4186f39616 Merge branch '2.3' into develop 2019-04-23 12:11:09 +03:00
24fc82e160 Move large query processing inside RWBackend
The knowledge of which function to call can be internal to RWBackend. This
make the use of the class easier as one can simply write to the backend.
2019-04-18 13:58:34 +03:00
b9e6384a41 Adding streaming log macros. 2019-04-18 10:42:39 +03:00
d3bfe0a712 Fix handling of large result packets
The RWBackend didn't know to skip the tail end of large packets.
2019-04-17 11:16:16 +03:00
993334b9fd MXS-2349: Add socket parameter
Servers now accept the `socket` parameter in the configuration as well as
in the REST API.
2019-04-16 11:52:37 +03:00
969ef5f9f7 MXS-1662 Add PAM authentication option for admin users
If normal authentication fails and a PAM service is defined, PAM authentication
is attempted. Separate services can be set for read-only users and admin-level
users.
2019-04-15 13:28:44 +03:00
0c7a3240bd MXS-2346 Exclude core parameters
Core parameters are not handled by the module but by the core and
must hence be ignored when validating and configuring.
2019-04-12 15:03:02 +03:00
c381aefefc MXS-2346 Add config::ParamInteger and config::Integer 2019-04-12 15:03:02 +03:00
09702ab0a0 MXS-2346 Provide new configuration mechanism
The configuration mechanism consists of the following concepts:

Specification
  Specifies the available configuration parameters of a module,
  their names and their types.
Param
  Specifies a parameter, its name and its type.
Type
  Specifies the type of a configuration parameters; Bool, Size,
  Count, etc.
Configuration
  Specifies the configuration values of a particular instance of
  the module. Configuration walks hand in hand with Specification,
  the latter specifies what the former should contain.

A Specification is capable of configuring a Configuration from a
MXS_CONFIG_PARAMETER, checking in the process that all parameters
are of the correct type and that the required parameters are present.

A Specification is capable of persisting itself so that it later
can be read back.

The mechanism is closed for modification but open for extension in
the sense that if a module requires a custom parameter, all it needs
to do is to derive one class from Param and another from Type.

The canonical way for using this mechanism is as follows. Consider
a module xyx that has three parameters; a parameter called
"enabled" that is of boolean type, a parameter called "period"
that is of duration type, and a parameter "cache" that is of
size type. That would be declared as follows:

    // xyz.hh
    class XYZSession;

    class XYZ : public maxscale::Filter<XYZ, XYZSession>
    {
    public:
        static XYZ* create(const char* zName, MXS_CONFIG_PARAMETER* pParams);

    private:
        XYZ();

        static config::Specification                       s_specification;
        static config::ParamBool                           s_enabled;
        static config::ParamDuration<std::chrono::seconds> s_period;
        static config::ParamSize                           s_cache;

        config::Configuration                              m_configuration;
        config::Bool                                       m_enabled;
        config::Duration<std::chrono::seconds>             m_period;
        config::Size                                       m_cache;
    };

    // xyz.cc

    config::Specification XYZ::s_specification(MXS_MODULE_NAME);

    config::ParamBool XYZ::s_enabled(
        &s_specification,
        "enabled",
        "Specifies whether ... should be enabled or not."
        );
    config::ParamDuration<std::chrono::seconds> XYZ::s_period(
        &s_specification,
        "period",
        "Specifies the period. Rounded to the nearest second."
        );
    config::ParamSize XYZ::s_cache(
        &s_specification,
        "cache",
        "Specifies the size of the internal cache."
        );

    XYZ::XYZ()
        : m_configuration(&s_specification)
        , m_enabled(&m_configuration, &s_enabled)
        , m_period(&m_configuration, &s_period)
        , m_cache(&m_configuration, &s_cache)
    {
    }

    XYZ* XYZ::create(const char* zName, MXS_CONFIG_PARAMETER* pParams)
    {
        XYZ* pXyz = new XYZ;

        if (!s_specification.configure(pXyz->m_configuration, pParams))
        {
            delete pXyz;
            pXyz = nullptr;
        }

        return pXyz;
    }
2019-04-12 15:03:02 +03:00
c643f9bc8d Merge branch '2.3' into develop 2019-04-12 13:23:49 +03:00
d2ecaa83a6 Move result start handling into separate function
The largest part of the code deals with the start of a response. Moving
this into a subfunction makes the function clearer as the switch statement
inside a switch statement is removed.
2019-04-12 09:18:07 +03:00
746bd53668 Simplify RWBackend result handling
By processing the packets one at a time, the reply state is updated
correctly regardless of how many packets are received. This removes the
need for the clunky code that used modutil_count_signal_packets to detect
the end of the result set.
2019-04-12 09:18:05 +03:00
e6526dd9ea Add extra info logging to readwritesplit
Added logging into RWBackend reply state processing code to know more.
2019-04-12 09:17:48 +03:00
0cb15976e8 Backport: Add force option to set endpoint
The new `force=yes` option closes all connections to the server that is
being put into maintenance mode. This will immediately close all open
connections to the server without allowing results to return.
2019-04-09 09:43:18 +03:00
7fb840ac9e Sort CN_ definitions 2019-04-09 09:43:18 +03:00
05515cca16 MXS-2259: Limit size of client reads
Given the assumption that queries are rarely 16MB long and that
realistically the only time that happens is during a large dump of data,
we can limit the size of a single read to at most one MariaDB/MySQL packet
at a time. This change allows the network throttling to engage a lot
sooner and reduces the maximum overshoot of throtting to 16MB.
2019-04-05 22:48:16 +03:00
b54e67223f MXS-2423: Add missing parameters to maxscale endpoint
Also updated the REST API documentation to include the newer output
(automating this update would be valuable).
2019-04-05 13:33:16 +03:00
31c93cfe1c Use SHA2-512 for admin users
MD5 is not secure enough with the modern hardware. Upgrading to SHA2-512
helps move the problem to the future.
2019-04-05 01:00:47 +03:00
daf5f52c64 Pass raw password to users_auth
By passing the raw password deeper into the authentication code, it can be
used to verify the user can access some systems. Right now, this is not
required by the simple salted password comparison done in MaxScale.
2019-04-05 00:42:00 +03:00
40d73948a9 MXS-1662: Move mxs_crypt into utils
Moved the mxs_crypt function into utils and renamed to mxs::crypt (no C
code used it).
2019-04-05 00:42:00 +03:00
2aa3515fc8 Merge commit '09cb4a885f88d30b5108d215dcdaa5163229a230' into develop 2019-04-04 14:34:17 +03:00
e3e66f8e90 MXS-2417: Add option to ignore persisted configs
The load_persisted_configs parameter now controls whether persisted
runtime changes are loaded on startup. The changes are still generated as
it persists the current state of MaxScale making problem analysis easier.
2019-04-03 12:57:05 +03:00
ebd2162024 Add force option to set endpoint
The new `force=yes` option closes all connections to the server that is
being put into maintenance mode. This will immediately close all open
connections to the server without allowing results to return.
2019-04-02 14:21:54 +03:00
f4c6b648d8 MXS-2271 Remove monitor->m_active field
Deactivated monitors are now moved to a separate, inaccessible list.
2019-04-02 13:08:38 +03:00
d89f0c062b MXS-2271 Change Monitor->m_name to std::string
Also, monitor address is no longer printed.
2019-04-02 13:08:38 +03:00
5f5d2ef183 MXS-2363 Implement /v1/maxscale/query_classifier/cache
That URL will now return information about the statements in
the query classifier cache. The information is collected using
the same map in a serial manner from all routing workers (that
each have their own cache). Since all caches will contains the
same statements, collecting the information in a serial manner
means that the overall memory consumption will be lower than
what it would be if the information was collected in parallel.
2019-03-29 11:31:35 +02:00
64a0327ada MXS-2363 Add string constants needed by new QC functionality 2019-03-29 11:31:35 +02:00
56ec56f7a0 MXS-2363 Enable the serial execution of function objects
Make it possible to run function objects serially on all
routing workers. Sometimes that is exactly what you want.
2019-03-29 11:31:35 +02:00
9ec82932cf MXS-2363 Skeleton of /v1/maxscale/query_classifier/cache
URL routing in place, callback exists, but no actual information.
2019-03-29 11:31:35 +02:00
388e952c5f MXS-1951: Fix port conflict detection
With the addition of SO_REUSEPORT support, it is no longer possible to
rely on the network stack to prevent multiple listeners from listening on
the same port. Without explicitly checking for the ports it would be
possible for two listeners from two different services to listen on the
same port in which case the service would be almost randomly chosen.
2019-03-28 16:34:12 +02:00
df3b501563 MXS-1951: Add SO_REUSEPORT support
If SO_REUSEPORT is available and the kernel supports it, listeners will
now listen on separate file descriptors. This removes the need for
cross-worker communication when in normal operation which should make
MaxScale scale better.
2019-03-28 16:34:12 +02:00
cb957200c9 MXS-1951: Detect SO_REUSEPORT support
The support for SO_REUSEPORT is now detected at runtime.
2019-03-28 16:34:12 +02:00
e745d1cffb MXS-2312: Use default network buffer sizes
Removed the hard-coded network buffer sizes. This effectively makes them
configurable via the kernel.
2019-03-28 16:34:12 +02:00
5c7846da3f MXS-1951: Make listener fd worker-local
By storing the file descriptor inside a worker-local variable, it is
possible to handle both unique file descriptors (created with
SO_REUSEPORT) and shared file descriptors with the same code. The way in
which the file descriptor is stored in the rworker_local object determines
the way the listener behaves.
2019-03-28 16:34:12 +02:00
27cc247e47 MXS-1951: Clean up listener creation
The old values that were encoded into a string wasn't necessary as the
data is already present in a processed form in the member variables.
2019-03-28 16:34:12 +02:00
841bdf4d0f Merge branch '2.3' into develop 2019-03-28 15:33:09 +02:00
116efb2409 MXS-2379: Fix maxinfo HTTP interface
Added the missing HTTP request handlers and a new JSON conversion
function.
2019-03-28 14:03:56 +02:00
6bc2c54081 Merge commit 'a78f0fbe2537542dc7f3f0dd8b19b93ac8d9d7f8' into develop 2019-03-28 13:53:40 +02:00
858327acf7 Rename Being Drained to Draining
With this, the words are unique and can be searched for more easily. This
does not fix the test failure of mxs2273_being_drained.
2019-03-28 13:21:24 +02:00
ac0a3d1d47 Extend string helper functions
Moved mxs::strtok into mxb namespace and added mxb::join for joining
objects into strings.
2019-03-28 13:21:23 +02:00
d40e29d5f6 A little houskeeping.
Increasing counter sizes from int to long for averages.
Rename random functions to end with _co instead of _exclusive to
indicate range [close, open[, and to allow future suffixes oc, cc and oo.
2019-03-27 13:15:14 +02:00
a78f0fbe25 Merge branch '2.2' into 2.3 2019-03-27 12:16:54 +02:00
04f70879d2 MXS-2410: Uniquely identify DCBs
The address alone doesn't guarantee validity of a DCB.
2019-03-27 11:58:31 +02:00
0915487329 Pass std::string to add/remove task
By passing strings instead of const char pointers to the task control
functions, we can safely make copies of them knowing that the contents
won't disappear.
2019-03-26 13:44:04 +02:00
f37340e9fd MXS-2397 Provide context information for fields
The query classifier now returns contextual information for a fields;
does it appear in the (right hand side) of a UNION or in a SUBQUERY.
2019-03-25 15:24:35 +02:00