As the cdc_kafka_producer script is an example, it should flush the
producer after every new record. This should make it easier to see that
events from MaxScale are sent to Kafka.
There's no need to process the JSON twice as the Kafka producer is
expected to be used with the Python CDC client which already splits the
JSON with newlines.
The transaction tracking now flags the transaction as open even when a
COMMIT/ROLLBACK is received. The next statement after the transaction is
closed resets the transaction state to inactive. This means that the end
of the transaction belongs in the transaction.
This allows all statements that belong to a transaction to be detected
with the transaction tracking functions provided by the core. With this,
the routing modules should have an easier time handling transactions.
When the transaction ends, it's good to know what type of a transaction
just ended. Currently, this will be used by readwritesplit to detect when
a read-only transaction ends.
All modules now declare a name for the module. This is name is added as a
prefix to all messages logged by a module. The prefix should help
determine which part of the system logs a message.
The MXS_MODULDE object now contains optinal pointers for functions
to be called att process and thread startup and shutdown. Since the
functions were added to the end, strictly speaking, all structures
would not have needed to have been modified, but better to be
explicit. In a subsequent change, these will be called.
C++ does not support flexible arrays, so for the time being C++
modules are restricted to 10 parameters. Better approach is to
factor out the parameters to a separate array and then just store
a pointer to that array in MXS_MODULE.
The MODULE_INFO is now the main object which is used by modules to convey
information to the MaxScale core. The MXS_MODULE name is more apt as it
now contains the actual module definition.
The old MODULES structure was moved into load_utils.c as an internal
implementation and was renamed so that it is not confused with the new
MODULE structure.
The modules are now declared with a common macro. This allows future
additions to the module loading process while also making the loaded
symbol name a constant.
This allows modules to only expose one entry point with a consistent
signature. In the future, this could be used to implement declarations of
module parameters.
It is possible that a session is in the dummy state (a transient state)
when a backend write occurs. The check for the client protocol NULL-ness
should extend to the client DCB itself.
The backend MySQL protocol module now supports a new routing capability
which allows result sets to be gathered into one buffer before they are
routed onward. This should not be used by modules that expect large
result sets as the result set is buffered in memory.
Adding a limit on how large of a result set could be buffered would allow
relatively safe use of this routing capability without compromising the
stability of the system.
Doing batch inserts though readwritesplit would stall due to the fact that
pending session commands were stored instead of executed immediately.
Session command responses that weren't complete also discarded the partial
event instead of storing it for later use.
When a session command was received, any trailing data was lost even
though an attempt to split is was made.
With this change, each session command reply will be routed individually
and any trailing data is routed separately.
The routing capabilities now define the type of output the reply
processing chain expects. Currently, this only consists of two
capabilities; complete packet output and contiguous buffer output. The
latter implies the former.
The MySQLBackend protocol now only checks for complete packets if the
service requires statement based routing. This should remove unnecessary
processing when data is only streamed from the backend to the client.
JSON does not have a concept of streams and a common way to stream JSON is
to separate each JSON object with a newline. Adding a newline makes it
easier to parse as JSON values do not natively contain newlines.
The packet was generated with the wrong number of elements due to usage of
sizeof on an integer where the correct type was an uint8_t.
This only fixes the malformed packets but does not fix the root cause of
the problem. The affected rows and last insert ID are length encoded
integers which should be handled. The current code treats them as one byte
fields.
