If the output buffer given to pcre2_substitute is too small, an error
value is written to the last parameter (output length). That value
should not be used for calculations. This patch gives a copy as
parameter instead.
Coincidentally, this commit fixes the crashes of query classifier tests.
Also, increase buffer growth rate in utils.c.
If a client connects from an IPv4 address, but the listener listens on an
IPv6 address, the client IP will be a IPv6 mapped IPv4 address
e.g. ::ffff:127.0.0.1. A grant for an IPv4 address should still match an
IPv6 mapped IPv4 address.
When users were loaded, the permissions for the service user were
checked. The conditional that makes sure the check is executed only at
startup was checking the listener's users instead of the SQLite handle
which caused all reloads of users to check the permissions.
This reverts commit f3c83770903151a0a3b53593c3e05fa0af94cd5f. The
functionality was used implicitly by modules that declare the
RCAP_TYPE_CONTIGUOUS_OUTPUT capability.
The help messages are now more descriptive and have usage information in
them. This should help users use the commands without relying on the
online documentation.
The RCAP_TYPE_STMT_OUTPUT is not used in its previous form. It can be
altered to route only complete packets back to the client. This allows
routers to do safer parsing on the results.
New parameter added to maxsrows filter:
max_resultset_return=empty|error|ok
Default, 'empty' is to return an empty set, as the current
implementation.
'err' will return an ERR reply with the input SQL statement
'ok' will return an OK packet
When log messages are written with both address and port information, IPv6
addresses can cause confusion if the normal address:port formatting is
used. The RFC 3986 suggests that all IPv6 addresses are expressed as a
bracket enclosed address optionally followed by the port that is separate
from the address by a colon.
In practice, the "all interfaces" address and port number 3306 can be
written in IPv4 numbers-and-dots notation as 0.0.0.0:3306 and in IPv6
notation as [::]:3306. Using the latter format in log messages keeps the
output consistent with all types of addresses.
The details of the standard can be found at the following addresses:
https://www.ietf.org/rfc/rfc3986.txthttps://www.rfc-editor.org/std/std66.txt
It is now possible to specify what information the caller is interested
in. With this the cost for collecting information during the query parsing
that nobody is interested in can be avoided.
The match data needs to be unique for each thread, so for the time
being it is created whenever it is needed. A more performant (although
possibly to a negigible amount) solution would be to have a separate
match data for each thread, but that will have to wait for 2.2.
When MaxScale is being started and the users are loaded, the MySQL
authenticator should not load the database users for internal services
abstracted as servers.
The loading of users at startup for internal services is avoided because
the startup is done in a single thread context and the internal services
have not yet been started.
The delayed loading of users will cause the authentication to fail when
the first client connect. This triggers the reloading of the users and the
second attempt at authentication will succeed. All of this is hidden from
the end user.
If a server points to a local MaxScale listener, the permission checks for
that server are skipped. This allows permission checks to be used with a
mix of external servers and internal services.
- Selects are picked out using custom parsing, so if a statement is
anything else but a SELECT, the cache will never cause the statement
to be parsed.
- The setting of of the cache parameter `selects` is taken into account.
If it is `assume_cacheable` then the statement will also not be parsed
even if it is a SELECT.
The original approach was made for RocksDB where it is beneficial
to keep keys of stuff related to each other close to each other.
However, as RocksDB is no longer the primary focus, it just causes
additional cost to dig out the table names.
The key is a 64-bit integer, but crc32 only gives us a 32-bit one.
We create an other 32-bit value by running crc32 over the same SQL,
using the first crc value as adler.
I think that further reduces the chance for clashes:
uint32_t crc0 = crc32(0, Z_NULL, 0);
uint32_t crc1;
uint32_t crc2;
crc1 = crc32(crc0, "codding", 7) => 1774765869
crc2 = crc32(crc1, "codding", 7) => 1409592046
crc1 = crc32(crc0, "gnu", 3) => 1774765869
crc2 = crc32(crc1, "gnu", 3) => 1213798908
Note that the first value is the same, but the second is not.
When a standalone master server is detected, it should receive the stale
status to prevent it from losing the master status if another server is
started and allow_cluster_recovery is enabled.
The json_stringn function should be used instead of the json_string to
allow null characters as well as non-null terminated strings to be
embedded in the JSON values.
The CDC example Python programs now decode the data as UTF-8 instead of
ASCII.
Transaction boundaries can now be detected using regexes.
All else being equal, it gives a 10% performance improvement
compared to qc-based detection.
In a subsequent change, mysql_client.c will be modified to use
qc_get_trx_type_mask() instead of qc_get_type_mask().
Currently the use of regex matching is turned on using an
environment variable. That will change.
The process and thread initialization/finalization of the query
classifier plugins is handled using the process and thread
initialization/finalization functions in the module object.
However, the top-level query classifier will also need to perform
process and thread initialization when transaction boundaries are
detected using regular expressions.
The connector plugin directory can now be controlled with the
`connector_plugindir` argument and configuration option. This should allow
the connector to use the system plugins if the versions are binary
compatible.
Replaced calls to mysql_options to mysql_optionsv as the former is
deprecated in Connector-C 3.0 and the latter is supported in Connector-C
2.3.
The client protocol module can resolve whether a password was used based
on the information the authenticators gather before authentication is
done. It uses the authentication token length as the basis on which it
makes the decision.
The users were deleted before each individual server was queried. This
caused authentication to fail if the authentication data was loaded from
multiple servers.
The client connection and the server listener sockets used largely similar
code. Combining them allows for simpler protocol code.
Cleaned up parts of the DCB listener creation and moved the parsing of the
network binding configuration to a higher level.
The socket creation code in mysql_backend.c wasn't MySQL specific and it
could be used for all non-blocking network connections. Thus, it makes
sense to move it to a common file where other protocol modules can use
it.
The address resolution code now uses `getaddrinfo` to resolve all
addresses instead of manually handling wildcard hosts. This allows the
same code to be used for all addresses.
Both the listeners and servers now support IPv6 addresses.
The namedserverfilter does not yet use the new structures and needs to be
fixed in a following commit.
The SQLite database is now always created on disk. This will remove the
need to dump the database users from the in-memory database to the
persisted on-disk database.
This change will also make the authentication compatible with older SQLite
implementations which lack the URI-based database strings found in newer
versions.
The authenticators should have a similar way to print diagnostic
information as filter and routers do. This allows the authenticators to
print the users in their own format.
In the future, all the diagnostic entry points should be changed so that
they return a structure that contains the information in a standard
form. This information can then be formatted in different ways by other
modules.
