Commit Graph

17380 Commits

Author SHA1 Message Date
686cbffd8a [fix](multi-catalog) fix iceberg defalut credentials proviers (#31704) 2024-03-07 16:16:05 +08:00
Pxl
dc9de4b6b5 [Bug](load) fix wrong data in mv when routine load with function mapping (#31787) 2024-03-07 16:16:05 +08:00
c7d86c062a [fix](Nereids) json_object can't be odd parameters, need even parameters (#31767) 2024-03-07 16:15:34 +08:00
b978365a7b [fix](cloud) Fix FE cannot work when restarting after schema change (#31842)
```
2024-03-05 21:16:43,300 INFO (stateListener|110) [RollupJobV2.replayPendingJob():749] replay waiting txn rollup job: 30405
2024-03-05 21:16:43,300 INFO (stateListener|110) [MaterializedViewHandler.replayAlterJobV2():1114] set table's state to ROLLUP, table id: 29056, job id: 30405
2024-03-05 21:16:43,300 DEBUG (stateListener|110) [Env.replayJournal():2751] journal 21241 replayed.
2024-03-05 21:16:43,300 DEBUG (stateListener|110) [JournalObservable.notifyObservers():95] notify observers: journal: 21241, pos: 0, size: 0, obs: []
2024-03-05 21:16:43,300 DEBUG (stateListener|110) [JournalEntity.readFields():193] get opcode: 45
2024-03-05 21:16:43,300 DEBUG (stateListener|110) [EditLog.loadJournal():170] replay journal op code: 45
2024-03-05 21:16:43,301 DEBUG (stateListener|110) [InternalCatalog.unprotectUpdateReplica():1048] replay update a replica table id: 29056 partition id: 29050 index id: 30406 tablet id: 30505 backend id: 10002 replica id: 30506 version: 2 schema hash: -1 data size: 0 row count: 0 last failed version: -1 last success version: 2
2024-03-05 21:16:43,301 DEBUG (stateListener|110) [CloudReplica.getBackendId():143] connect context is null in getBackendId
2024-03-05 21:16:43,301 ERROR (stateListener|110) [EditLog.loadJournal():1226] Operation Type 45
java.lang.NullPointerException: table id: 29056 partition id: 29050 index id: 30406 tablet id: 30505 backend id: 10002 replica id: 30506 version: 2 schema hash: -1 data size: 0 row count: 0 last failed version: -1 last success version: 2
        at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:921) ~[guava-32.1.2-jre.jar:?]
        at org.apache.doris.datasource.InternalCatalog.unprotectUpdateReplica(InternalCatalog.java:1054) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.datasource.InternalCatalog.replayUpdateReplica(InternalCatalog.java:1075) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env.replayUpdateReplica(Env.java:3706) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.persist.EditLog.loadJournal(EditLog.java:398) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env.replayJournal(Env.java:2747) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env.transferToMaster(Env.java:1459) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env.access$1400(Env.java:319) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env$5.runOneCycle(Env.java:2638) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.common.util.Daemon.run(Daemon.java:116) ~[doris-fe.jar:1.2-SNAPSHOT]
```
2024-03-07 16:13:36 +08:00
d0847a8953 [feat](Nereids): add struct info map in group (#31800) 2024-03-07 16:13:36 +08:00
2e4201e406 [opt](session variable) max_msg_size_of_result_receiver #31809 2024-03-07 16:13:19 +08:00
f897db666c [docs](flink-connector) Fix Flink Doris Connector typo (#31830) 2024-03-07 16:12:23 +08:00
ff8c679851 [fix](group commit) group commit may heap-use-after-free if execute plan failed (#31839) 2024-03-07 16:12:23 +08:00
28f0b7eb32 [Improvement](profile)Add tvf active_be_tasks() #31815 2024-03-07 16:12:23 +08:00
538032a75c [fix](partition) add log when tablet partition id eq 0 (#31796) 2024-03-07 16:11:25 +08:00
2e94512e07 [fix](build)Fix index_tool build error (#31834)
Co-authored-by: Luennng <luennng@gmail.com>
2024-03-07 16:11:25 +08:00
f5991d8b02 [docs](jdbc catalog) add sqlserver timestamp type mapping doc (#31832) 2024-03-07 16:11:23 +08:00
Pxl
2913a111f6 [Bug](top-n) init query_ctx runtime predicate before _build_pipelines #31895 2024-03-07 10:19:42 +08:00
Pxl
b4dbb087c0 [Bug](top-n) init query_ctx runtime predicate before operators prepare #31876 2024-03-06 19:09:10 +08:00
6ea5218ee8 Revert "[Enhencement](env) Checking Master branch must use JDK17 (#31587)"
This reverts commit fa499cc200344eaaf837fd52211820dc7b7b9296.
2024-03-06 13:13:49 +08:00
649f04e17c Revert "[feature](doris compose) doris compose use jdk 17 image (#31775)"
This reverts commit 9d2e5f1c00bd04277b492df979b367e1184c09de.
2024-03-06 13:13:26 +08:00
7e4cc1d177 [fix](storage medium) Fix show partition storage medium not right whe… (#30160) (#31642) 2024-03-06 13:11:57 +08:00
1dcb99519d [fix](move-memtable) use tuple desc from schema param for delta writer (#31747) 2024-03-06 13:08:30 +08:00
4f174c4fb9 [feature](function) Support for aggregate function foreach combiner (#31526) 2024-03-06 13:08:30 +08:00
7f3a666fac [fix](planner) fix ifnull and nvl function with one parameters exception message anbugious (#31808)
When ifnull or nvl funtion have only one parameter, nereids planner would throw an exception and go back to
original planner/ Original planner get secend parameter directly without check, so it return unexpected error message
2024-03-06 13:08:30 +08:00
9af64d848f [fix](pipelineX) fix error distribution in DistinctStreamingAggOperatorX (#31804) 2024-03-06 13:08:30 +08:00
3e1e6c8d76 [fix](cloud) Fix version_p0 run in cloud mode (#31810) 2024-03-06 13:08:04 +08:00
0f1cbcc86a [refactor](jdbc catalog) split postgresql jdbc Executor (#31730) 2024-03-06 13:08:04 +08:00
2e9bd268cd [improvement](jdbc catalog) support sqlserver timestamp type read (#31805) 2024-03-06 13:08:04 +08:00
aba58b0f7b [test](Nereids) add grouping sets test (#31675)
Co-authored-by: feiniaofeiafei <moailing@selectdb.com>
2024-03-06 13:08:04 +08:00
1d2d0bd411 [fix](update) Update set value should consider sequence column (#31626)
When using update command to set column value, if the column is sequence column, the column 'DORIS_SEQUENCE_COL' should also be set to the same value.
2024-03-06 13:08:04 +08:00
679c7657f6 [fix](cloud) Fix regression case test_dynamic_partition run in cloud mode (#31803) 2024-03-06 13:08:04 +08:00
d94d2c65f6 [fix](Nereids) let OrToIn rewritten result have stable order (#31731) 2024-03-06 13:07:59 +08:00
c43bc8349f [fix](move-memtable) free resources when fail to open stream (#31762) 2024-03-06 13:07:59 +08:00
1434d3983b [enhancement](test) Test the unique model by modify a key type from TINYINT to other type (#31713) 2024-03-06 13:07:59 +08:00
2d6e975d5a [fix](cast) fix wrong result while cast string to float (#31781)
Issue Number: close #31518
2024-03-06 13:07:59 +08:00
eea9b56f69 [fix](group commit) handle group commit create plan error (#31757) 2024-03-06 13:07:59 +08:00
7b51c4aaca [chore](fe) Speed up fe-core checkstyle (#31785)
by excluding checkstyle-cachefile from auto clean
2024-03-06 13:07:59 +08:00
cc8112273e [fix](mtmv) Avoiding the occurrence of null pointers in logs due to the deletion of MTMV (#31722) 2024-03-06 13:07:49 +08:00
808563470f [pipelineX](debug) Refactor code and complete debug string (#31733) 2024-03-06 13:07:49 +08:00
300f25c392 [chore](ci) remove required of pipelinex (#31806) 2024-03-06 13:07:49 +08:00
c98e93edc3 [chore](ci) add back pipelinex in master (#31799) 2024-03-06 13:07:49 +08:00
97640ee0e8 [test](leading) add leading tpc-ds regression test cases (#31681)
Co-authored-by: libinfeng <libinfeng@selectdb.com>
2024-03-06 13:07:49 +08:00
f2bece23d8 [chore](ci) add back required to test (#31795) 2024-03-06 13:07:49 +08:00
a909bc36ee [fix](jdbc catalog) Change Druid Pool dependency to version 1.2.5 (#31783) 2024-03-06 13:07:49 +08:00
d9943d0374 [chore](Nereids) change consumer rules name (#31777) 2024-03-06 13:07:49 +08:00
ad3308c8ab [fix](hive) support partition prune for _HIVE_DEFAULT_PARTITION_ (#31736)
This PR #23026 support the partition prune for hive table with `_HIVE_DEFAULT_PARTITION`,
but it will always select partition with `_HIVE_DEFAULT_PARTITION`.

This PR #31613 support null partition for olap table's list partition, so we can treat `_HIVE_DEFAULT_PARTITION`
as null partition of hive table.

So this PR change the partition prune logic
2024-03-06 13:07:49 +08:00
9541fb2602 [chore](ci) remove required of pipelinex (#31791) 2024-03-06 13:07:49 +08:00
9d2e5f1c00 [feature](doris compose) doris compose use jdk 17 image (#31775) 2024-03-06 13:07:49 +08:00
11903d29a1 [fix](jdbc catalog) fix close abort in sqlserver (#31718) 2024-03-06 13:07:49 +08:00
ca59579bf6 fix](Nereids): support max_value in range partition (#31721) 2024-03-06 13:07:08 +08:00
7998da4691 [fix](cast) wrong result while cast const to double then to string (#31657)
Issue Number: close #31514
2024-03-06 13:06:27 +08:00
5ed3c7eeef [opt](doc) update release verify (#31773) 2024-03-06 13:06:27 +08:00
2c26a308d1 [fix](broker-load) fix broker load statement type conversion failure (#31746)
Co-authored-by: Luwei <814383175@qq.com>
2024-03-06 13:06:27 +08:00
7c30cb20fd [Fix](partial update) Fix partial update load false when schema includes auto increment column (#31725)
Problem:
When partially updating columns without specifying the auto-increment column, and the imported data contains new keys, an error stating the auto-increment column could not be found occurs.

Reason:
The logic for partial column updates does not account for new keys in auto-increment columns. Since auto-increment columns can be generated by the system, it's possible to omit this column data during import. However, partial column updates treat this as a regular column, expecting it to be nullable or have a default value for automatic filling, overlooking the fact that auto-increment columns can also be auto-filled. This oversight leads to the error.

Solution:
Incorporate a check for auto-increment columns into the partial column update logic, and include the logic for generating auto-increment column values in the process of completing partial updates.
2024-03-06 13:06:27 +08:00