Commit Graph

29 Commits

Author SHA1 Message Date
3eb99139f5 Merge branch '2.1' into develop 2017-07-31 15:57:05 +03:00
fc81a8097f Add missing length and real_type fields to cdc_schema.go
The Go version of the CDC schema generator was missing the new `length`
and `real_type` fields.
2017-07-25 21:08:50 +03:00
26bf342e6b Merge branch '2.1' into develop 2017-07-24 12:37:25 +03:00
00f046da1b MXS-1316: Read the raw input in cdc_kafka_producer.py
The input can be read as raw bytes since kafka expects the data to be of
type bytes instead of str.
2017-07-13 13:27:51 +03:00
cb57e10761 Develop merge
Develop merge
2017-06-29 15:34:22 +02:00
1df8fda93f Merge branch '2.0' into 2.1 2017-06-27 20:52:58 +03:00
bd1da82c96 Fix cdc_schema.py
There's no need to use Python 3 for the script and using it introduces
problems due to the poor availability of MySQL Connector/Python for
Python3.

ENUM, SET and DECIMAL values should have a length of -1 as the length is
meaningless for these types.
2017-06-23 11:10:41 +03:00
46a04773a4 Merge branch '2.1' into develop 2017-06-20 11:01:08 +03:00
469ed1de1b MXS-1271: Remove unnecessary encode call
The trailing newline can be removed from the string without encoding it
first.
2017-06-20 10:40:22 +03:00
17c0bf7dec Merge branch '2.1' into develop 2017-06-14 13:59:18 +03:00
4c9fc1a39e Merge branch '2.0' into 2.1 2017-06-14 13:36:40 +03:00
c3ffc0ba9c Improve cdc.py error handling
When the CDC service reponds with an error, the program prints the error
and exits.
2017-06-05 20:40:29 +03:00
457fc80647 Merge branch '2.1' into develop 2017-06-05 13:25:27 +03:00
a37a665bd2 MXS-1271: Remove redundant JSON parsing
As the cdc.py script outputs newline delimited JSON, there's no need to
parse the JSON in the cdc_kafka_producer.py script.
2017-06-05 13:17:12 +03:00
f546a17e77 Update change date of 2.2 2017-06-01 10:24:20 +03:00
a3cfabf37b MXS-1271: Remove redundant JSON parsing
As the cdc.py script outputs newline delimited JSON, there's no need to
parse the JSON in the cdc_kafka_producer.py script.
2017-05-30 15:42:09 +03:00
b5d1504e6a Merge branch '2.0' into 2.1 2017-05-26 15:22:43 +03:00
ad109408b5 MXS-1216: Add real_type and length to cdc_schema.py
The schema generator program needs to add the real_type and length fields
if the data types define them.

Also fixed a bug where the real_type and length fields were checked for
generated fields.
2017-05-16 13:27:46 +03:00
8e2c7fd952 Merge branch '2.0' into 2.1 2017-05-10 09:12:42 +03:00
c67894b011 Add Avro schema generator Python script
The script is the Python version of the cdc_schema.go application.
2017-04-04 16:42:39 +03:00
7d7d8a0560 Print all output as raw bytes
Printing all output as raw bytes allows MaxScale to control the formatting
process. This also removes the need to convert the bytes to Python strings
and the need to parse the JSON.
2017-03-20 11:10:28 +02:00
3396741c21 Improve UTF-8 handling in avrorouter
The json_stringn function should be used instead of the json_string to
allow null characters as well as non-null terminated strings to be
embedded in the JSON values.

The CDC example Python programs now decode the data as UTF-8 instead of
ASCII.
2017-03-15 17:37:25 +02:00
5648f708af Update license to BSL 1.1 2017-02-14 21:42:28 +02:00
e64a641bcd Merge branch '2.0' into develop 2017-02-01 09:35:13 +02:00
6cd16d26b8 Flush the producer after every new record
As the cdc_kafka_producer script is an example, it should flush the
producer after every new record. This should make it easier to see that
events from MaxScale are sent to Kafka.
2017-01-30 20:14:27 +02:00
78545b2c5c Read complete lines with Kafka producer
There's no need to process the JSON twice as the Kafka producer is
expected to be used with the Python CDC client which already splits the
JSON with newlines.
2017-01-27 13:06:25 +02:00
ae49e0e580 Add custom usage for CDC schema generator
The usage prints the required arguments and describes what this tool is
intended for.
2016-12-22 10:03:30 +02:00
d7f79942be Merge branch '2.0' into develop 2016-09-09 15:12:58 +03:00
4e3de4c56d Rename and relocate CDC Python examples
Moved the CDC example scripts into the protocol directory and added the .py
suffix. Fixed all references to these scripts.
2016-09-05 10:32:37 +03:00