f84af95ac4
### Two main changes:
- 1、add minidump replay
- 2、change minidump serialization of statistic messages and some interface between main logic of nereids optimizer and minidump
### Use of nereids ut:
- 1、save minidump files:
Execute command by mysql-client:
```
set enable_nereids_planner=true;
set enable_minidump=true;
```
Execute sql in mysql-client
- 2、use nereids-ut script to execute directory:
```
cp -r ${DORIS_HOME}/minidump ${DORIS_HOME}/output/fe && cd ${DORIS_HOME}/output/fe
./nereids_ut --d ${directory_of_minidump_files}
```
### Refactor of minidump
- move statistics used serialization to serialization of input and serialize with catalogs
- generating minidump file only when enable_minidump flag is set, minidump module interactive with main optimizer only by :
serializeInputsToDumpFile(catalog, statistics, query) && serializeOutputsToDumpFile(outputplan).
39 lines
1.5 KiB
Markdown
39 lines
1.5 KiB
Markdown
# Tutorial of Nereids minidump
|
|
Minidump is a basic dump file for Nereids optimizer which arms
|
|
to replay problems and used as unit test tool for developers.
|
|
It include minimal necessary informations for Nereids optimizer
|
|
to replay a scenario in Optimizer.
|
|
|
|
## Save minidump file
|
|
In two cases fe would save minidump file
|
|
- when query throw an exception
|
|
- when use 'set enable_minidump=true;' switch by User
|
|
|
|
## File structure of minidump file
|
|
Minidump files would save in $DORIS_HOME/log/minidump path
|
|
and named by queryId like: c9bf82b70ce34c28-8d281e31412f3ee6.json
|
|
File structure of minidump file includes:
|
|
| key | Value |
|
|
|:----|:----|
|
|
|Sql | original sql string |
|
|
|SessionVariables | sessionVariables different than default |
|
|
|DatabaseName | database name|
|
|
|Tables | schemas in table, partition info and some table properties |
|
|
|ColocateTableIndex | table distribute informations |
|
|
|ColumnStatistics | all ColumnStatistics in table |
|
|
|Histogram | all histogram information in table|
|
|
|ParsedPlan | plan after parsed in json format|
|
|
|ResultPlan | plan after all optimized in json format|
|
|
|
|
## Use of nereids_ut.sh
|
|
When we want to use this shell, we need to go to output/fe/minidump path
|
|
Now only one parameter is allowed for this shell which is directory
|
|
of minidump files like $DORIS_HOME/log/minidump
|
|
```shell
|
|
cd output/fe/minidump
|
|
./nereids_ut.sh --d $minidump_files_path
|
|
```
|
|
The running reports can be found in output/fe/minidump/log
|
|
which is minidump.out
|
|
|