Commit Graph

7782 Commits

Author SHA1 Message Date
e91d16854b [fix](function) fix date_format function execution error on fe (#31645) 2024-03-07 16:53:19 +08:00
a3c24b84e3 [fix](mtmv) support null value in partition for updating (#31843) 2024-03-07 16:53:19 +08:00
da5a40077f [fix](http stream) http stream support memtable_on_sink_node header (#31866) 2024-03-07 16:53:19 +08:00
9d5c51f5c3 [enhancement](nereids) show nullaware semi join in plan (#31738) 2024-03-07 16:53:19 +08:00
5905ffa1da [enhancement](nereids) allow reorder mark join (#30644) 2024-03-07 16:53:19 +08:00
5b00f4fbeb [improvement](jdbc catalog) opt get db2 schema list & xml type mapping (#31856)
1. Trim Schema Names: Adapted the system to remove trailing spaces from DB2 schema names, ensuring compatibility without affecting query operations.
2. XML Mapping: Implemented a feature to directly map XML types to String.
2024-03-07 16:53:19 +08:00
80c80b6719 [enhancement](Nereids): support more condition Date/DateTime Literal (#31858) 2024-03-07 16:16:05 +08:00
561709451c [fix](Nereids) fix group_concat(distinct) failed (#31873) 2024-03-07 16:16:05 +08:00
ad2f7fc316 [bugfix](coordinator) should use fragment id not profile fragment id to cancel fragment (#31852) 2024-03-07 16:16:05 +08:00
Pxl
3716f8a171 [Bug](partition) fix npe when prune partition with not exist partition column in mv #31860 2024-03-07 16:16:05 +08:00
5c1c73e5bb [Fix](Nereids) fix missing comment when creating table (#31869) 2024-03-07 16:16:05 +08:00
686cbffd8a [fix](multi-catalog) fix iceberg defalut credentials proviers (#31704) 2024-03-07 16:16:05 +08:00
Pxl
dc9de4b6b5 [Bug](load) fix wrong data in mv when routine load with function mapping (#31787) 2024-03-07 16:16:05 +08:00
c7d86c062a [fix](Nereids) json_object can't be odd parameters, need even parameters (#31767) 2024-03-07 16:15:34 +08:00
b978365a7b [fix](cloud) Fix FE cannot work when restarting after schema change (#31842)
```
2024-03-05 21:16:43,300 INFO (stateListener|110) [RollupJobV2.replayPendingJob():749] replay waiting txn rollup job: 30405
2024-03-05 21:16:43,300 INFO (stateListener|110) [MaterializedViewHandler.replayAlterJobV2():1114] set table's state to ROLLUP, table id: 29056, job id: 30405
2024-03-05 21:16:43,300 DEBUG (stateListener|110) [Env.replayJournal():2751] journal 21241 replayed.
2024-03-05 21:16:43,300 DEBUG (stateListener|110) [JournalObservable.notifyObservers():95] notify observers: journal: 21241, pos: 0, size: 0, obs: []
2024-03-05 21:16:43,300 DEBUG (stateListener|110) [JournalEntity.readFields():193] get opcode: 45
2024-03-05 21:16:43,300 DEBUG (stateListener|110) [EditLog.loadJournal():170] replay journal op code: 45
2024-03-05 21:16:43,301 DEBUG (stateListener|110) [InternalCatalog.unprotectUpdateReplica():1048] replay update a replica table id: 29056 partition id: 29050 index id: 30406 tablet id: 30505 backend id: 10002 replica id: 30506 version: 2 schema hash: -1 data size: 0 row count: 0 last failed version: -1 last success version: 2
2024-03-05 21:16:43,301 DEBUG (stateListener|110) [CloudReplica.getBackendId():143] connect context is null in getBackendId
2024-03-05 21:16:43,301 ERROR (stateListener|110) [EditLog.loadJournal():1226] Operation Type 45
java.lang.NullPointerException: table id: 29056 partition id: 29050 index id: 30406 tablet id: 30505 backend id: 10002 replica id: 30506 version: 2 schema hash: -1 data size: 0 row count: 0 last failed version: -1 last success version: 2
        at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:921) ~[guava-32.1.2-jre.jar:?]
        at org.apache.doris.datasource.InternalCatalog.unprotectUpdateReplica(InternalCatalog.java:1054) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.datasource.InternalCatalog.replayUpdateReplica(InternalCatalog.java:1075) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env.replayUpdateReplica(Env.java:3706) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.persist.EditLog.loadJournal(EditLog.java:398) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env.replayJournal(Env.java:2747) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env.transferToMaster(Env.java:1459) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env.access$1400(Env.java:319) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env$5.runOneCycle(Env.java:2638) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.common.util.Daemon.run(Daemon.java:116) ~[doris-fe.jar:1.2-SNAPSHOT]
```
2024-03-07 16:13:36 +08:00
d0847a8953 [feat](Nereids): add struct info map in group (#31800) 2024-03-07 16:13:36 +08:00
2e4201e406 [opt](session variable) max_msg_size_of_result_receiver #31809 2024-03-07 16:13:19 +08:00
28f0b7eb32 [Improvement](profile)Add tvf active_be_tasks() #31815 2024-03-07 16:12:23 +08:00
7e4cc1d177 [fix](storage medium) Fix show partition storage medium not right whe… (#30160) (#31642) 2024-03-06 13:11:57 +08:00
4f174c4fb9 [feature](function) Support for aggregate function foreach combiner (#31526) 2024-03-06 13:08:30 +08:00
7f3a666fac [fix](planner) fix ifnull and nvl function with one parameters exception message anbugious (#31808)
When ifnull or nvl funtion have only one parameter, nereids planner would throw an exception and go back to
original planner/ Original planner get secend parameter directly without check, so it return unexpected error message
2024-03-06 13:08:30 +08:00
0f1cbcc86a [refactor](jdbc catalog) split postgresql jdbc Executor (#31730) 2024-03-06 13:08:04 +08:00
2e9bd268cd [improvement](jdbc catalog) support sqlserver timestamp type read (#31805) 2024-03-06 13:08:04 +08:00
1d2d0bd411 [fix](update) Update set value should consider sequence column (#31626)
When using update command to set column value, if the column is sequence column, the column 'DORIS_SEQUENCE_COL' should also be set to the same value.
2024-03-06 13:08:04 +08:00
d94d2c65f6 [fix](Nereids) let OrToIn rewritten result have stable order (#31731) 2024-03-06 13:07:59 +08:00
2d6e975d5a [fix](cast) fix wrong result while cast string to float (#31781)
Issue Number: close #31518
2024-03-06 13:07:59 +08:00
eea9b56f69 [fix](group commit) handle group commit create plan error (#31757) 2024-03-06 13:07:59 +08:00
7b51c4aaca [chore](fe) Speed up fe-core checkstyle (#31785)
by excluding checkstyle-cachefile from auto clean
2024-03-06 13:07:59 +08:00
cc8112273e [fix](mtmv) Avoiding the occurrence of null pointers in logs due to the deletion of MTMV (#31722) 2024-03-06 13:07:49 +08:00
97640ee0e8 [test](leading) add leading tpc-ds regression test cases (#31681)
Co-authored-by: libinfeng <libinfeng@selectdb.com>
2024-03-06 13:07:49 +08:00
a909bc36ee [fix](jdbc catalog) Change Druid Pool dependency to version 1.2.5 (#31783) 2024-03-06 13:07:49 +08:00
d9943d0374 [chore](Nereids) change consumer rules name (#31777) 2024-03-06 13:07:49 +08:00
ad3308c8ab [fix](hive) support partition prune for _HIVE_DEFAULT_PARTITION_ (#31736)
This PR #23026 support the partition prune for hive table with `_HIVE_DEFAULT_PARTITION`,
but it will always select partition with `_HIVE_DEFAULT_PARTITION`.

This PR #31613 support null partition for olap table's list partition, so we can treat `_HIVE_DEFAULT_PARTITION`
as null partition of hive table.

So this PR change the partition prune logic
2024-03-06 13:07:49 +08:00
11903d29a1 [fix](jdbc catalog) fix close abort in sqlserver (#31718) 2024-03-06 13:07:49 +08:00
ca59579bf6 fix](Nereids): support max_value in range partition (#31721) 2024-03-06 13:07:08 +08:00
7998da4691 [fix](cast) wrong result while cast const to double then to string (#31657)
Issue Number: close #31514
2024-03-06 13:06:27 +08:00
2c26a308d1 [fix](broker-load) fix broker load statement type conversion failure (#31746)
Co-authored-by: Luwei <814383175@qq.com>
2024-03-06 13:06:27 +08:00
7c30cb20fd [Fix](partial update) Fix partial update load false when schema includes auto increment column (#31725)
Problem:
When partially updating columns without specifying the auto-increment column, and the imported data contains new keys, an error stating the auto-increment column could not be found occurs.

Reason:
The logic for partial column updates does not account for new keys in auto-increment columns. Since auto-increment columns can be generated by the system, it's possible to omit this column data during import. However, partial column updates treat this as a regular column, expecting it to be nullable or have a default value for automatic filling, overlooking the fact that auto-increment columns can also be auto-filled. This oversight leads to the error.

Solution:
Incorporate a check for auto-increment columns into the partial column update logic, and include the logic for generating auto-increment column values in the process of completing partial updates.
2024-03-06 13:06:27 +08:00
294cb1729d [fix](feut) fix TabletRepairAndBalanceTest (#31653) 2024-03-06 13:05:31 +08:00
d7d8b4ee4a [fix](fe) Fix NPE in FrontendServiceImpl.loadTxnCommit if table is dropped (#31715) 2024-03-06 13:05:31 +08:00
a26b1eeee7 [fix](connections) fix connection hang after too many connections (#31594)
Issue Number: close #31569

Fix fe connection hang after too high qps

After fix, the third SQL will return error instead of hang:
ERROR 1203 (HY000): #42000Too many connections
2024-03-06 13:05:22 +08:00
Pxl
0c71b666d7 [Bug](resource) fix npe on Resource read from json (#31723)
fix npe on Resource read from json
2024-03-06 13:05:22 +08:00
3777ffb43f [enhancement](nereids)support null partition for list partition (#31613) 2024-03-06 13:05:22 +08:00
33c356e6d8 fix in stats estimation (#31684)
the valid in option should

! (op.min > expr.max or op.max < expr.min) 
=============> 
op.min <= expr.max and op.max >= expr.min
2024-03-06 13:04:54 +08:00
e2ebf9d566 [feature](Nereids) parallel output file (#31623)
legacy planner impl PR: #6539
2024-03-06 13:04:30 +08:00
a5b9127656 [refactor](jdbc catalog) split sqlserver jdbc executor (#31679) 2024-03-06 13:04:29 +08:00
8198a31563 [fix](testcases) make all auto partition P2 cases nonConcurrent (#31708) (#31821)
make all auto partition P2 cases nonConcurrent
2024-03-06 09:12:02 +08:00
bc898e3cdc [pipelineX](bug) Fix incorrect join operator judgement (#31690)
* [pipelineX](bug) Fix incorrect join operator judgement

* update
2024-03-03 19:22:41 +08:00
344e80fbbc [fix](jdbc catalog) Change Druid Pool dependency to version 1.2.11 2024-03-03 17:58:54 +08:00
47d330c74d [enhancement](Nereids) New optimizer support check column privileges (#31700)
* [enhancement](Nereids) New optimizer support check column privileges (#31494)

(cherry picked from commit accfcfcae5a3343a9e2b7241453e56738255de1b)

* column privilege

* column privilege
2024-03-03 13:48:31 +08:00