Before changing default insert operation to streaming load, if the select result
of a insert stmt is empty, a label will still be returned to the user, and user
can use this label to check the insert load job's status.
After changing the insert operation, if the select result is empty, a exception
will be thrown to user client directly without any label.
This new usage pattern is not friendly to already existed users, which is forcing
them to change their way of using insert operation.
So I add a new FE config 'using_old_load_usage_pattern', default is false.
If set to true, a label will be returned to user even if the select result is empty.
The issue is following:
Request1:
BE aborts the txn.
Attachment of txn is set.
Attachment of txn is set to null without lock of txn because the task has been aborted by FE.
Request2:
BE aborts the txn again.
Attachment of txn is set again.
Request1:
The attachment is not null so the job wants to find the task and commit it.
The job could not find the task so it is paused. (NullPointer Exception)
In this commit, The commit request will check if task exists instead of checking txn attachment
is not null.
Without dbId parameter, the backend report version can not be
updated when publish task report to FE, which may cause incorrect
order of report.
Related commit: 5c1b4f6
If one child of a binary predicate is column, and the other is a constant expr,
set the compatible type to column's type.
eg:
k1(int):
... WHERE k1 = "123" --> k1 = cast("123" as int);
k2(varchar):
... WHERE 123 = k2 --> cast(123 as varchar) = k2
This optimization is for the case that some users may use a int column to save date, eg: 20190703,
but query with predicate: col = "20190703".
If not casting "20190703" to int, query optimizer can not do partition prune correctly.
When a rollup table contains value columns of REPLACE aggregation type,
all key columns of base table should be included in this rollup.
Without all key columns, the order of rows is undefined.
For example, table schema k1,k2,k3,v1(REPLACE)
1,2,3,A
1,2,4,B
1,2,5,C
A rollup with column(k1,k2,v1):
1,2,A
1,2,B
1,2,C
No matter A or B or C be the last winner, it is meanless to user.
Also fix a bug that set password for current user on non-master FE
will throw NullPointerException.
This commit change the idToTable to concurrent hashmap in database. It don't need to lock the database before getTable.
The database will be locked in GlobalTxnManager. The load job will be locked after that.
So the lock sequence is: database, load manager, load job.
When performing storage_medium_migration, the header may be already droped.
In this scenario, header returned will be null pointer.
Saving data to a null pointer will cause core dump.
The previous setting of timeout of a publish version task is mess.
I change it to a configurable time, default it 30 seconds.
And when the table is under rollup or schema change, I double this timeout.
This a kind of best-effort-optimization. Because with a short timeout,
a replica's publish version task is more likely to fail. And if quorum replicas
of a tablet fail to publish, the alter job will fail.
If the table is not under rollup or schema change, the failure of a replica's
publish version task has a minor effect because the replica can be repaired
by tablet repair process very soon. But the tablet repair process will not
repair rollup replicas.
* This commit has brought contribution to streaming mini load
The operation of streaming mini load is sames as previous. Also, user can check the load by frontend.
The difference is that streaming mini load finish the task before reply of REST API while the non-streaming only register a load.
* When updating doris
Updating fe or be firstly are also supported. After fe and be are updated, the streaming mini load will take effect.
* For multi mini load
The non-streaming mini load still has been used by multi mini load. The behavior of multi mini load has not been changed.
* Add a interface named isSupportedFunction
This function is used to protect the correctness of new feature which consists of be and fe during updaing.
