Add insertstream documentation

The documentation explains how the filter works.
This commit is contained in:
Markus Mäkelä 2017-01-13 11:15:58 +02:00
parent 1911430e96
commit a3a9edd3b6
2 changed files with 85 additions and 0 deletions

View File

@ -87,6 +87,7 @@ Here are detailed documents about the filters MariaDB MaxScale offers. They cont
- [RabbitMQ Filter](Filters/RabbitMQ-Filter.md)
- [Named Server Filter](Filters/Named-Server-Filter.md)
- [Luafilter](Filters/Luafilter.md)
- [Insert Stream Filter](Filters/Insert-Stream-Filter.md)
## Monitors

View File

@ -0,0 +1,84 @@
# Insert Stream Filter
The _insertstream_ filter converts bulk inserts into CSV data streams that are
consumed by the backend server via the LOAD DATA LOCAL INFILE mechanism. This
leverages the speed advantage of LOAD DATA LOCAL INFILE over regular inserts
while also reducing the overall network traffic by condensing the inserted
values into CSV.
## Filter Parameters
This filter has no parameters.
## Details of Operation
The filter translates all INSERT statements done inside an explicit transaction
into LOAD DATA LOCAL INFILE streams. The file name used in the request will
always be _maxscale.data_.
The following example is translated into a LOAD DATA LOCAL INFILE request
followed by two CSV rows.
```
BEGIN;
INSERT INTO test.t1 VALUES (1, "hello"), (2, "world");
COMMIT;
```
Multiple inserts to the same table are combined into a single stream. This
allows for efficient bulk loading with simple insert statements.
The following example will use only one LOAD DATA LOCAL INFILE request followed
by four CSV rows.
```
BEGIN;
INSERT INTO test.t1 VALUES (1, "hello"), (2, "world");
INSERT INTO test.t1 VALUES (3, "foo"), (4, "bar");
COMMIT;
```
Non-INSERT statements executed inside the transaction will close the streaming
of the data. Avoid interleaving SELECT statements with INSERT statements inside
transactions.
The following example has to use two LOAD DATA LOCAL INFILE requests, each
followed by two CSV rows.
**Note:** Avoid doing this!
```
BEGIN;
INSERT INTO test.t1 VALUES (1, "hello"), (2, "world");
SELECT * FROM test.t1;
INSERT INTO test.t1 VALUES (3, "foo"), (4, "bar");
COMMIT;
```
### Estimating Network Bandwidth Reduction
The more inserts that are streamed, the more efficient this filter is. The
saving in network bandwidth in bytes can be estimated with the following
formula:
```
((20 + t) * n) + (n * (m * 2)) - 108 - t = x
n = Number of INSERT statements
m = Number of values in each insert statement
t = Length of table name
x = Number of bytes saved
```
Positive values indicate savings in network bandwidth usage.
## Example Configuration
The filter has no parameters so it is extremely simple to configure. The
following example shows the required filter configuration.
```
[Insert-Stream]
type=filter
module=insertstream
```