1. Optimize the error msg of Tablet scheduler.
2. Missing helper nodes info when modify Frontends.
3. Fix bug that olap tablet's header lock is not released.
Previously we do not allow repair tablet if the table it belongs
to is under ALTER process. But it will possibly let the alter job
failed due to some replica's failure of load.
* Remove build rows counter in PartitionHashJoinNode
* Fix unit test fail in RuntimeProfileTest
* Add check for result type length in cast_to_string_val
1. Fix bug that failed to query restored table after schema change.
2. Fix bug that failed to add rollup to restored table.
3. Optimize the info of SHOW ALTER TABLE stmt.
4. Optimize the info of some PROCs.
5. Optimize the tablet checker to avoid adding too much task to scheduler.
* Add param of specified thirdparty path
1. The thirdparth path can be specify on build.sh: ./build.sh --thirdparty /specified/path/to/thirdparty
2. If there are only thirdparty param of build.sh, it will build both fe and be
3. Add unit test of routine load stmt
4. Remove source code in docker image
* Add DORIS_THIRDPARTY env in docker image
1. Set DORIS_THIRDPARTY env in docker image. The build.sh will use /var/local/thirdparty instead of /source/code/thirdparty
2. remove --thirdparty param of build.sh
* Change image workdir to /root
There is unnegligible cost to covnert VectorRowBatch to RowBatch,
When we seek block, we only read one row from engine to minimize
this convert cost.
This patch can optimize some query's time from 5s to 2s
There are A, B, C replicas of one tablet.
A has 0 - 10 version.
B has 0 - 5, 6, 7, 9, 10 version.
1. B has missed versions, so it clones 0 - 10 from A, and remove overlapped versions in its header.
2. Coincidentally, 6 is a version for delete predicate (delete where day = 20181221).
When removing overlapped versions, version 6 is removed but delete predicate is not be removed.
3. Unfortunately, 0-10 cloned from A has data indicated at 20181221.
4. B performs compaction, and data generated by 20181221 is be removed falsely.
There are A, B, C replicas of one tablet.
A has 0 - 10 version.
B has 0 - 5, 6, 7, 9, 10 version.
1. B has missed versions, so it clones 0 - 10 from A, and remove overlapped versions in its header.
2. Coincidentally, 6 is a version for delete predicate (delete where day = 20181221).
When removing overlapped versions, version 6 is removed but delete predicate is not be removed.
3. Unfortunately, 0-10 cloned from A has data indicated at 20181221.
4. B performs compaction, and data generated by 20181221 is be removed falsely.
Record query consumption into fe audit log. Its basic mode of work is as follows, one of instance of parent plan is responsible for accumulating sub plan's consumption and send to it's parent, BE coordinator will get total consumption because it's a single instance.
Help to locate big query when system overload, by checking consumptions of running parts of current all queries or specified one query. Its basic mode of work is as follows: firstly trigger BE to report RuntimeProfiles, and wait a moment. secondly caculate consumptions with RuntimeProfiles reported by BE. The consumptions supported by it are the cost of running ExecNode in query when call it.
1. Add sql parser and sql scanner for routine load stmt such as KW_ROUTINE(routine), KW_PAUSE.
2. Create routine load statment like
CREATE ROUTINE LOAD name ON database.table
(properties of routine load)
[PROPERTIES (key1=value1, )]
FROM [KAFKA](type of routine load)
(properties of this type)
properties of routine load:
The load property of CreateRoutineLoadStmt is disordered: Both 'LoadColumnsInfo, PartitionNames xxx' and 'PartitionNames, ColumnsInfo xxx' is right.
[COLUMNS TERMINATED BY separator ]
[(col1, ...)]
[SET (k1=f1(xx), k2=f2(xx))]
WHERE
[PARTITION (p1, p2)]
type of routine load:
KAFKA
different type has different properties
properties of this type:
k1 = v1
k2 = v2
3. Pause/Resume/Stop routine load statment like
PAUSE/RESUME/STOP ROUTINE LOAD jobName
4. Ddlexecutor support CreateRoutineLoadStmt, Pause/Resume/StopRoutineLoadStmt
5. Pause/Stop routine load will clear all of task which belong to job immediately
The task which has been not committed will be abort.
6. Resume routine load will change job state to need scheduler
The RoutineLoadJobScheduler will scheduler it later.
7. Show routine load statment like
SHOW ROUTINE LOAD jobName
8. All of load property can implement LoadProperty such as LoadColumnsInfo, PartitionsNames etc
9. The sql of LoadColumnsInfo is Columns (c1, c2, c3) set (c1, c2, c3=c1+c2)
10. Add check of routineLoadName, db.routineLoadName is unique in database when job state is not final state.
