Add insertstream documentation
The documentation explains how the filter works.
This commit is contained in:
@ -87,6 +87,7 @@ Here are detailed documents about the filters MariaDB MaxScale offers. They cont
|
|||||||
- [RabbitMQ Filter](Filters/RabbitMQ-Filter.md)
|
- [RabbitMQ Filter](Filters/RabbitMQ-Filter.md)
|
||||||
- [Named Server Filter](Filters/Named-Server-Filter.md)
|
- [Named Server Filter](Filters/Named-Server-Filter.md)
|
||||||
- [Luafilter](Filters/Luafilter.md)
|
- [Luafilter](Filters/Luafilter.md)
|
||||||
|
- [Insert Stream Filter](Filters/Insert-Stream-Filter.md)
|
||||||
|
|
||||||
## Monitors
|
## Monitors
|
||||||
|
|
||||||
|
84
Documentation/Filters/Insert-Stream-Filter.md
Normal file
84
Documentation/Filters/Insert-Stream-Filter.md
Normal file
@ -0,0 +1,84 @@
|
|||||||
|
# Insert Stream Filter
|
||||||
|
|
||||||
|
The _insertstream_ filter converts bulk inserts into CSV data streams that are
|
||||||
|
consumed by the backend server via the LOAD DATA LOCAL INFILE mechanism. This
|
||||||
|
leverages the speed advantage of LOAD DATA LOCAL INFILE over regular inserts
|
||||||
|
while also reducing the overall network traffic by condensing the inserted
|
||||||
|
values into CSV.
|
||||||
|
|
||||||
|
## Filter Parameters
|
||||||
|
|
||||||
|
This filter has no parameters.
|
||||||
|
|
||||||
|
## Details of Operation
|
||||||
|
|
||||||
|
The filter translates all INSERT statements done inside an explicit transaction
|
||||||
|
into LOAD DATA LOCAL INFILE streams. The file name used in the request will
|
||||||
|
always be _maxscale.data_.
|
||||||
|
|
||||||
|
The following example is translated into a LOAD DATA LOCAL INFILE request
|
||||||
|
followed by two CSV rows.
|
||||||
|
|
||||||
|
```
|
||||||
|
BEGIN;
|
||||||
|
INSERT INTO test.t1 VALUES (1, "hello"), (2, "world");
|
||||||
|
COMMIT;
|
||||||
|
```
|
||||||
|
|
||||||
|
Multiple inserts to the same table are combined into a single stream. This
|
||||||
|
allows for efficient bulk loading with simple insert statements.
|
||||||
|
|
||||||
|
The following example will use only one LOAD DATA LOCAL INFILE request followed
|
||||||
|
by four CSV rows.
|
||||||
|
|
||||||
|
```
|
||||||
|
BEGIN;
|
||||||
|
INSERT INTO test.t1 VALUES (1, "hello"), (2, "world");
|
||||||
|
INSERT INTO test.t1 VALUES (3, "foo"), (4, "bar");
|
||||||
|
COMMIT;
|
||||||
|
```
|
||||||
|
|
||||||
|
Non-INSERT statements executed inside the transaction will close the streaming
|
||||||
|
of the data. Avoid interleaving SELECT statements with INSERT statements inside
|
||||||
|
transactions.
|
||||||
|
|
||||||
|
The following example has to use two LOAD DATA LOCAL INFILE requests, each
|
||||||
|
followed by two CSV rows.
|
||||||
|
|
||||||
|
**Note:** Avoid doing this!
|
||||||
|
|
||||||
|
```
|
||||||
|
BEGIN;
|
||||||
|
INSERT INTO test.t1 VALUES (1, "hello"), (2, "world");
|
||||||
|
SELECT * FROM test.t1;
|
||||||
|
INSERT INTO test.t1 VALUES (3, "foo"), (4, "bar");
|
||||||
|
COMMIT;
|
||||||
|
```
|
||||||
|
|
||||||
|
### Estimating Network Bandwidth Reduction
|
||||||
|
|
||||||
|
The more inserts that are streamed, the more efficient this filter is. The
|
||||||
|
saving in network bandwidth in bytes can be estimated with the following
|
||||||
|
formula:
|
||||||
|
|
||||||
|
```
|
||||||
|
((20 + t) * n) + (n * (m * 2)) - 108 - t = x
|
||||||
|
|
||||||
|
n = Number of INSERT statements
|
||||||
|
m = Number of values in each insert statement
|
||||||
|
t = Length of table name
|
||||||
|
x = Number of bytes saved
|
||||||
|
```
|
||||||
|
|
||||||
|
Positive values indicate savings in network bandwidth usage.
|
||||||
|
|
||||||
|
## Example Configuration
|
||||||
|
|
||||||
|
The filter has no parameters so it is extremely simple to configure. The
|
||||||
|
following example shows the required filter configuration.
|
||||||
|
|
||||||
|
```
|
||||||
|
[Insert-Stream]
|
||||||
|
type=filter
|
||||||
|
module=insertstream
|
||||||
|
```
|
Reference in New Issue
Block a user