Commit Graph

2861 Commits

Author SHA1 Message Date
433528aa59 MXS-1731: Treat empty parameters as errors
If a parameter is defined without a value, it is now treated as an error.
2018-03-21 14:58:09 +02:00
e4efc29297 MXS-1702: Process comments when canonicalizing
The canonicalization process now strips non-executable comments from the
SQL and replaces all constants in executable comments.

Enabled the comment test and updated expected output of the select and
alter tests.
2018-03-20 13:53:55 +02:00
9fe25c8003 MXS-1702: Update tests
Updated tests with new expected output. Also took new function into use
and removed the old one.

Since the comment removal isn't added yet, one of the tests is expected to
fail and it is temporarily disabled.
2018-03-20 13:53:55 +02:00
a627e342f3 MXS-1702: Fix minor canonicalization bugs
Allow hexadecimal digits if a 0x prefix has been seen. Squash repeating
whitespace and remove quotes from quoted strings.
2018-03-20 13:53:55 +02:00
8d3f02b8a4 MXS-1702: Canonicalize negative numbers
The new canonicalization function now processes negative numbers
correctly. It uses a look-behind operation to detect whether the operation
is a negation of a number or a subtraction from another value.
2018-03-20 13:53:55 +02:00
75fadd711a MXS-1702: Add new canonicalization function
The function uses a hand-written parser that picks out the values and
replaces them with question marks. This function is not yet able to
process negative values; a `-?` token will be generated for all negative
numbers.

The new function will canonicalize the query in such a way that string and
number constants cannot be distinguished which means that it won't pass
the current test cases.
2018-03-20 13:53:55 +02:00
a03e8d46f0 MXS-1702: Rename tests
The test naming now follows the common convention.
2018-03-20 13:53:55 +02:00
e90c29cce2 MXS-1702: Clean up and convert test to C++
Cleaned up and updated the test; the code was written with a pre-C99
standard in mind.

Added a get() method into mxs::Buffer to make it easier to use with
non-C++ functions.
2018-03-20 13:53:54 +02:00
cb170eb88e MXS-1702: Move canonicalization test into core
Moved the test into the core where it belongs.
2018-03-20 13:53:54 +02:00
0fde6e501d Don't treat EINTR as an error
When the epoll_wait call returns with an error and errno is set to EINTR,
no warning should be logged as this is correct behavior.
2018-03-20 13:42:25 +02:00
f525822472 Merge branch '2.2' into develop 2018-03-20 13:14:54 +02:00
dcf9d7f152 Fix calls to diagnostics_json
Add missing listener JSON diagnostics call. Check that the
diagnostics_json function exists before calling it.

As the protocol modules don't have diagnostics functions, they aren't
called.

Replace hard-coded strings with constant parameters. This makes it
slightly cleaner.
2018-03-20 13:07:27 +02:00
28e4133669 Add missing jansson dependency on maxscale-common
The Jansson library must be built or must exist before MaxScale.
2018-03-20 12:35:06 +02:00
82b55ff362 MXS-1475 Add @maxscale.cache.[populate|use]
The earlier @maxscale.cache.enabled has now been replaced with
@maxscale.cache.populate and @maxscale.cache.use that provide
for more flexibility.

With the former it is possible to control in what circumstances
the cache is populated and with the latter one when it is used.
Together they can be used for having a completely client driven
caching.
2018-03-16 14:34:04 +02:00
71c8a327b2 MXS-1475 Introduce variable @maxscale.cache.enabled
With the variable @maxscale.cache.enabled the caching can be
enabled/disabled at runtime.

In a subsequent commit, the variable will actually be used.
2018-03-16 14:34:04 +02:00
2434482dc6 MXS-1475 Attempt to set unknown MXS user variable causes error
By causing an error if an unknown MaxScale user variable is set,
the user will become aware of typos etc.
2018-03-16 14:34:04 +02:00
872a51a376 MXS-1475 Enable MaxScale specific user variables
With the changes in this commit it is possible to add and remove
MaxScale specific user variables. A MaxScale specific user variable
is a user variable that is interpreted by MaxScale and that
potentially changes the behaviour of MaxScale.

MaxScale specific user variables are of the format "@maxscale.x.y"
where "@maxscale" is a mandatory prefix, x a scope identifying the
component that handles the variable and y the component specific
variable. So, a variable might be called e.g. "@maxscale.cache.enabled".
The scope "core" is reserved (although not enforced yet) to MaxScale
itself.

The idea is that although MaxScale catches these, they are passed
through to the server. The benefit of this is that we do not need to
detect e.g. "SELECT @maxscale.cache.enabled", but can let the result
be returned from the server.

The interpretation of a provided value is handled by the component that
adds the variable. In a subsequent commit, it will be possible for a
component to reject a value, which will then cause an error to be
returned to the client.

There are 3 new functions:

- session_add_variable() using which a variable is added,
- session_remove_variable() using which a variable is removed, and
- session_set_variable_value().

The two former ones are to be called by components, the last one by
the protocol that catches the "set @maxscale..." statements.
2018-03-16 14:34:04 +02:00
3af469a074 Merge branch '2.2' into develop 2018-03-16 12:54:09 +02:00
6afd57122d Merge branch '2.2' into develop 2018-03-16 12:39:55 +02:00
391ec78a0b MXS-1721 Destroy a filter instance only once
If two services referred to the same filter instance, it would
cause the filter to deleted twice at MaxScale shutdown with a
crash as the result.

Now when the services are deleted we just collect the unique
filter instances and then delete them after all services have
been deleted.
2018-03-16 12:00:18 +02:00
07cca088c9 MXS-1717: Fix test regressions
Due to the changes done for MXS-1717, the bug673 test had to be adjusted
and a newline has to be printed after users_diagnostic is called.
2018-03-15 23:23:15 +02:00
dad6a4f9bf Merge branch '2.2' into develop 2018-03-13 11:26:41 +02:00
633b08ed0d MXS-1717 Show which listener users are coming from
Earlier, if a service had multiple listeners you would have had

   MaxScale> show dbusers MyService
   User names: alice@% ...
   User names: bob@% ...

That is, no indication of which listener is reporting what. With
this commit the result will be

   User names (MyListener1): alice@% ...
   User names (MyListener2): bob@% ...

Further, the diagnostics function of an authenticator is now expected
to write the list of users to the provided DCB, without performing any
other formatting. The formatting (printing "User names" and appending
a line-feed) is now handled by the handler for the MaxAdmin command
"show dbusers".
2018-03-13 10:25:42 +02:00
69383c0943 Merge branch '2.2' into develop 2018-03-12 14:38:37 +02:00
aea9c36498 Merge branch '2.1' into 2.2 2018-03-12 14:38:13 +02:00
c5345d34ca MXS-1714 Use local_address also with MaxScale connections
If local address has been specified, then all connections created
using mxs_mysql_real_connect() will use that same local address as
well.

A system test has not been created as our VMs do not have more than
one usable IP-address. Locally it has been verified to work as
expected.
2018-03-12 11:35:46 +02:00
885d0af50f Merge branch '2.2' into develop 2018-03-09 21:00:16 +02:00
3e8f51bbf3 MXS-1705 Make note of workers only after creation
The variable containing the number of workers must be updated
only after the workers have been successfully created.

Failure to do this led to crash in Worker::shutdown_all() if a
terminating signal was received after the worker initialization
had failed.
2018-03-08 15:00:28 +02:00
010c1b2625 MXS-1705 Add explanatory comment 2018-03-08 15:00:28 +02:00
bf9767e54e MXS-1705 Try with O_DIRECT, if fails try without
Instead of trying to figure out whether the kernel supports O_DIRECT
in conjunction with pipes, let's just use it and if it fails, try
without O_DIRECT.

Case in point, based on circumstantial evidence it seems that in a
container context, it may appear as if the kernel supports O_DIRECT
when it in reality does not. So better to use brute-force.
2018-03-08 15:00:28 +02:00
5f328bc017 MXS-1689 Add better error messages
If a listener section specifies both a 'socket' and a 'port' the
creation will fail with a clear error message.

If 'address' and 'socket' is specified, there will be a warning that
the address is meaningless.
2018-03-07 15:25:43 +02:00
7ae3931511 MXS-1689 Properly check for duplicate port/socket of service
It is now impossible to create two listeners for a service that
would listen on the same port/socket (as before), but the error
message is now sensible and provides detailed information to the
user.
2018-03-07 15:25:43 +02:00
d17c3b8ab7 Cast to uint before comparing
Fixes compilation error.
2018-03-07 13:32:42 +02:00
8ef1385d1a MXS-1703: Turn MariaDB Monitor struct to class with public fields
Allows using std::string for strings. Also, cleanup.
2018-03-07 13:25:52 +02:00
82bb624981 MXS-1690: Need Backpressure mechanism (#169)
* implication of backpressure

* fix typo and refactor

* misc refactor

* misc fix

* add function session_unlink_backend_dcb and more comments

* misc fix

* refactor, move all throttling logic into dcb.cc

* misc fix

* misc fix
2018-03-07 12:34:47 +02:00
d6b8147119 Merge branch '2.2' into develop 2018-03-06 17:02:03 +02:00
55276be6f2 MXS-1699: Log progress messages at startup
When MaxScale is starting, the loading of the listeners can take a while
if there are a large number of services and users to load. To signal this
to the user, progress messages should be logged after every service is
started.
2018-03-06 15:56:07 +02:00
86eae02366 Log message on failed worker message
When a worker message fails, an error message should be logged to know why
it failed.
2018-03-06 13:35:15 +02:00
b619fb0707 MXS-1699: Log progress messages at startup
When MaxScale is starting, the loading of the listeners can take a while
if there are a large number of services and users to load. To signal this
to the user, progress messages should be logged after every service is
started.
2018-03-06 12:47:20 +02:00
ea83420620 Merge branch '2.2' into develop 2018-03-01 18:08:45 +02:00
a197e6c859 Remove unnecessary code
A descriptor is always added to the global epoll instance or
to a specific worker, never just to _any_ worker.
2018-02-28 20:11:27 +02:00
236e906d88 Revert "Turn MariaDB Monitor struct to class with public fields"
This reverts commit cb6f70119d9857b277306e9af5881fe29c574a32.
2018-02-24 15:37:50 +02:00
e385e6fcd4 Merge branch '2.2' into develop 2018-02-22 10:13:55 +02:00
6e9e83ccaf MXS-1674 Change load granularity to 1 second
With a granularity of 1 second, the load will from a human
perspective reflect the current situation. That also means
that the maxadmin output shows "natural" steps; 1s, 1m and 1h.
2018-02-21 13:05:58 +02:00
cb6f70119d Turn MariaDB Monitor struct to class with public fields
Allows using std::string for strings. Also, cleanup.
2018-02-21 11:00:42 +02:00
fd4fd4eead MXS-1674 Add worker load calculation
By definition, the load is calculated using the following formula:

  L = 100 * ((T - t) / T)

where T is a time period and t the time of that period that the worker
spends in epoll_wait(). So, if there is so much work that epoll_wait()
always returns immediately, then the load is 100 and if the thread
spends the entire period in epoll_wait(), then the load is 0.

The basic idea is that the timeout given to epoll_wait() is adjusted
so that epoll_wait() will always return roughly at 10 seconds interval.
By making a note of when we are about to enter epoll_wait() and when we
return from it, we have all the information we need for calculating the
load.

Due to the nature of things, we will not be able to calculate the load
at exact 10-second boundaries, but it will be pretty close. And the load
is always calculated using the true length of the period.

We will then calculate 1 minute load by averaging the load value for 6
consecutive 10-second periods and the 1 hour load by averaging the load
value of 60 consecutive 1 minute loads.

So, while the 10-second load represents the load of the most recently
measured 10-second period (and not the load of the most recent 10
seconds), the 1 minute load and the 1 hour load represents the load of
the most recent minute and hour respectively.
2018-02-20 09:18:43 +02:00
350eaf0e90 Add complete set of atomit_store-operations 2018-02-16 15:06:24 +02:00
1042b861bb MXS-1669: Fix load average tracking
The output of `show threads` could have a negative historic thread load
average that could be explained by the overflow of the signed 32-bit
integer used to count the number of samples.

The time that each thread started to process an event for a DCB used an
old value that is no longer used. Updating this to DCB::last_read retains
the 2.0 behavior.
2018-02-15 11:18:22 +02:00
f388e2f838 Merge branch '2.2' into develop 2018-02-12 14:00:40 +02:00
b8d3da4968 Add error tolerance to "servers_no_promotion"
Previously, if the list contained servers that were not monitored by
the monitor yet were valid servers, an error value would be returned
and the monitor failed to start.

With this update, the non-monitored servers are simply ignored when
forming the final list.

Also, added printing of the list to diagnostics.
2018-02-12 10:49:28 +02:00