The response handling logic did not always take the last packet for
inspection when a COM_CHANGE_USER was executed. The OK packet will always
be the last one since the COM_CHANGE_USER is the last command that was
sent.
When an unexpected response to a COM_CHANGE_USER is received, it is now
processes and discarded instead of treated as an error. This will allow
further analysis of the situation in addition to possibly solving some of
the problems that the persistent connections have.
Added extra info level logging to relevant parts of the code that deal
with the COM_CHANGE_USER reply processing. This information should allow
tracking of the response state for debugging purposes.
The response to the COM_CHANGE_USER should always be turned into a
contiguous buffer of complete packets. This guarantees that the code that
processes it functions properly.
As COM_QUIT would terminate the connection, there's no need to initiate
the session reset process. Also make sure all buffers are empty before
putting the DCB into the pool.
Added extra debug assertions for parts of the code that are related to the
COM_CHANGE_USER processing.
When a persistent connection is reused, a COM_CHANGE_USER command is
executed to reset the session state. If the reused connection was closed
before the response to the COM_CHANGE_USER was received and taken into use
by another connection, another COM_CHANGE_USER would be sent to, again,
reset the session state. Due to the fact that the first response is still
on its way, it will appear as if two responses are generated for a single
COM_CHANGE_USER.
The way to fix this is to avoid putting connections that haven't been
successfully reset into the connection pool.
When a session is being closed in a controlled manner, i.e. a COM_QUIT is
received from the client, it is possible to deduce from this fact that the
backend connections are very likely to be idle. This can be used as an
additional qualification that must be met by all connections before they
can be candidates for connection pooling.
This assumption will not hold with batched and asynchronous queries. In
this case it is possible that the COM_QUIT is received from the client
before even the first result from the backend is read. For this to work,
the protocol module would need to track the number and state of expected
responses.
If a connection has not been fully established (i.e. authentication has
been completed) then it should not be considered as a connection pool
candidate.
There's no need to use Python 3 for the script and using it introduces
problems due to the poor availability of MySQL Connector/Python for
Python3.
ENUM, SET and DECIMAL values should have a length of -1 as the length is
meaningless for these types.
The schema generator program needs to add the real_type and length fields
if the data types define them.
Also fixed a bug where the real_type and length fields were checked for
generated fields.
This reverts commit f3c83770903151a0a3b53593c3e05fa0af94cd5f. The
functionality was used implicitly by modules that declare the
RCAP_TYPE_CONTIGUOUS_OUTPUT capability.
The RCAP_TYPE_STMT_OUTPUT is not used in its previous form. It can be
altered to route only complete packets back to the client. This allows
routers to do safer parsing on the results.
When log messages are written with both address and port information, IPv6
addresses can cause confusion if the normal address:port formatting is
used. The RFC 3986 suggests that all IPv6 addresses are expressed as a
bracket enclosed address optionally followed by the port that is separate
from the address by a colon.
In practice, the "all interfaces" address and port number 3306 can be
written in IPv4 numbers-and-dots notation as 0.0.0.0:3306 and in IPv6
notation as [::]:3306. Using the latter format in log messages keeps the
output consistent with all types of addresses.
The details of the standard can be found at the following addresses:
https://www.ietf.org/rfc/rfc3986.txthttps://www.rfc-editor.org/std/std66.txt
Printing all output as raw bytes allows MaxScale to control the formatting
process. This also removes the need to convert the bytes to Python strings
and the need to parse the JSON.
The json_stringn function should be used instead of the json_string to
allow null characters as well as non-null terminated strings to be
embedded in the JSON values.
The CDC example Python programs now decode the data as UTF-8 instead of
ASCII.
Transaction boundaries can now be detected using regexes.
All else being equal, it gives a 10% performance improvement
compared to qc-based detection.
In a subsequent change, mysql_client.c will be modified to use
qc_get_trx_type_mask() instead of qc_get_type_mask().
Currently the use of regex matching is turned on using an
environment variable. That will change.
The client protocol module can resolve whether a password was used based
on the information the authenticators gather before authentication is
done. It uses the authentication token length as the basis on which it
makes the decision.
The client connection and the server listener sockets used largely similar
code. Combining them allows for simpler protocol code.
Cleaned up parts of the DCB listener creation and moved the parsing of the
network binding configuration to a higher level.
The socket creation code in mysql_backend.c wasn't MySQL specific and it
could be used for all non-blocking network connections. Thus, it makes
sense to move it to a common file where other protocol modules can use
it.
The address resolution code now uses `getaddrinfo` to resolve all
addresses instead of manually handling wildcard hosts. This allows the
same code to be used for all addresses.
Both the listeners and servers now support IPv6 addresses.
The namedserverfilter does not yet use the new structures and needs to be
fixed in a following commit.
Currently the only situation where a user needs to be authenticated after
the initial authentication is when a COM_CHANGE_USER is being
executed. This was previously handled by directly calling a function in
the MySQLAuth authenticator.
The new entry in the API of the authenticators is very specific to MySQL
and should be reviewed once other protocols are added.
Removed unused spinlocks from DCBs, sessions and the MySQL protocol
structs. They were used in a context where only one thread has access to
the structure.
Removed unused member variables from DCBs.
As the cdc_kafka_producer script is an example, it should flush the
producer after every new record. This should make it easier to see that
events from MaxScale are sent to Kafka.
There's no need to process the JSON twice as the Kafka producer is
expected to be used with the Python CDC client which already splits the
JSON with newlines.
The transaction tracking now flags the transaction as open even when a
COMMIT/ROLLBACK is received. The next statement after the transaction is
closed resets the transaction state to inactive. This means that the end
of the transaction belongs in the transaction.
This allows all statements that belong to a transaction to be detected
with the transaction tracking functions provided by the core. With this,
the routing modules should have an easier time handling transactions.
When the transaction ends, it's good to know what type of a transaction
just ended. Currently, this will be used by readwritesplit to detect when
a read-only transaction ends.
