Commit Graph

21085 Commits

Author SHA1 Message Date
bceebfcf38 branch-2.1: [test](jdbc catalog) add some jdbc query tvf test (#45991)
cherry-pick #45938
2024-12-26 15:05:50 +08:00
7addefbfe8 branch-2.1: [fix](mem leak) fe non_heap mem leak while use jdbc catalog #45806 (#45980)
Cherry-picked from #45806

Co-authored-by: camby <cambyzhu@tencent.com>
2024-12-26 15:05:28 +08:00
1396f5a950 [2.1][improvement](jdbc catalog) Optimize JdbcCatalog case mapping stability (#43256)
cherry-pick #41510
2024-12-26 14:57:56 +08:00
879bb79655 [regression-test](fix) fix two case bug (#45974) 2024-12-26 10:25:32 +08:00
289d621faa [improvement](information_schema)Show view definition in information_schema.views. (#45857) (#45930)
backport: https://github.com/apache/doris/pull/45857
2024-12-26 10:11:13 +08:00
1cf6986cea [pick](branch-2.1) pick #44092 (#45836) 2024-12-25 23:11:19 +08:00
a65adcaa49 branch-2.1: [fix](Outfile) optimize the error message of SELECT INTO OUTFILE #45859 (#45951)
Cherry-picked from #45859

Co-authored-by: Tiewei Fang <fangtiewei@selectdb.com>
2024-12-25 23:10:43 +08:00
bccfb04436 branch-2.1: [fix](parser) Syntax error for add partition with null null #45865 (#45913)
Cherry-picked from #45865

Co-authored-by: Uniqueyou <wangyixuan@selectdb.com>
2024-12-25 22:15:20 +08:00
c94ac6c9f8 [branch-2.1](ORC) fix predicate filter failed when use hive 1.x version (#45809)
### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #43185 

Pick the pr to branch-2.1 to fix predicate filter failed when use hive
1.x version

Co-authored-by: fantasy12345zsq <1575033031@qq.com>
2024-12-25 18:56:57 +08:00
a3ec8621a4 branch-2.1: [improvement](log)Reduce stats log. #45900 (#45916)
Cherry-picked from #45900

Co-authored-by: James <lijibing@selectdb.com>
2024-12-25 16:35:08 +08:00
20c49c3638 [regression-test](fix) fix global var (#45917) 2024-12-25 15:29:40 +08:00
303f6bd849 [test](regression-test) use unified trigger_and_wait_compaction metho… (#45908) 2024-12-25 13:59:51 +08:00
a6e4497811 [opt](inverted index) Add troubleshooting logs (#44182) (#45777)
https://github.com/apache/doris/pull/44182
2024-12-25 13:58:01 +08:00
e8be023b35 [typo](docker) Adjust the indentation format of the init_be and entry_point scripts, as well as the duration of loop execution(Merge-2.1). (#45860)
### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Master PR: #45308 

Problem Summary:

Adjust the indentation format of the `init_be` and `entry_point`
scripts, as well as the duration of loop execution.
Adjust the smallest unit of all indentations to a single tab character,
and modify the loop duration when checking the BE startup status,
changing both from 300 seconds to 30 seconds to speed up the overall
Docker startup time.
2024-12-25 12:16:42 +08:00
980f28f9e2 [feat](docker)Modify the init_be and start_be scripts to meet the requirements for rapid Docker startup(Merge 2.1). (#45858)
### What problem does this PR solve?

Issue Number: close #xxx

Related PR: #45267

Master PR: #45269

Problem Summary:

To meet the needs of rapid Docker startup, I have made adjustments to
two related scripts in the Docker startup process. First, I added a env
`SKIP_CHECK_ULIMIT` to the `start_be.sh` script, which will skip the
size checks for `swap`, `ulimit`, and `max_map_count`. At the same time,
I used `--console` to start the process and print logs. The reason why I
did not use the `--daemon` daemon command to execute is that starting
with a foreground log printing method in a Docker container is the
correct and reliable approach.

At the same time, I added a check logic for a `be.conf` configuration
item in the `init_be.sh` script: if it is the first time starting,
append the export `SKIP_CHECK_ULIMIT=true` to skip the `ulimit` value
check in the BE process. In summary, these adjustments can meet the
basic requirements for rapid Docker startup usage.
2024-12-25 12:11:14 +08:00
547e88b1ee branch-2.1: [fix](csv reader) fix core dump when parsing csv with enclose #45485 (#45889)
Cherry-picked from #45485

Co-authored-by: hui lai <laihui@selectdb.com>
2024-12-25 12:09:20 +08:00
fdb65197e7 [feat](mtmv)mtmv support paimon partition refresh(#43959,#44911) (#45878)
pick: #44911 #43959

only pick code about paimon, not pick some code about MTMV REFRESH
2024-12-25 12:08:27 +08:00
6c2107788c [fix](nereids) support one phase DeferMaterializeTopN #45693-branch-2.1 (#45845) 2024-12-25 12:00:58 +08:00
4c2abc7ba8 branch-2.1: [opt](pick_pr) add branch name to title for manual pick pr tool #45902 (#45911)
Cherry-picked from #45902

Co-authored-by: yujun <yujun@selectdb.com>
2024-12-25 11:53:43 +08:00
64195d79ee [refactor](metrics) Remove IntAtomicCounter & CoreLocal #45742 (#45870)
cherry pick from #45742
2024-12-24 23:13:48 +08:00
6d6473efae branch-2.1: [fix](tabletScheduler) Fix addTablet dead lock in tabletScheduler #45298 (#45769)
Cherry-picked from #45298

Co-authored-by: deardeng <dengxin@selectdb.com>
2024-12-24 21:44:30 +08:00
1314a2b942 branch-2.1: [test](nereids) add test simplify comparison predicate #44886 (#45804)
cherry pick from #44886
2024-12-24 21:43:15 +08:00
f0031d9954 [cherry-pick](branch-21)support posexplode table function (#43221) (#45783)
cherry-pick from master #43221
2024-12-24 21:42:30 +08:00
0b41c927e3 branch-2.1: [case](auth)Add case for auth #45478 (#45819)
Cherry-picked from #45478

Co-authored-by: zhangdong <zhangdong@selectdb.com>
2024-12-24 21:41:48 +08:00
2bfc72e173 [fix](nereids) fix illegal import of guava's Throwables (#45876)
fix illegal import of guava's Throwables
2024-12-24 21:36:52 +08:00
0a64cea3bf [fix](mtmv) Fix mv is deleted in nested mv causing query err and fix some test (#45744) (#45862)
pr: 
https://github.com/apache/doris/pull/45744

commitId: 067643df
2024-12-24 20:03:24 +08:00
bcc2c155df [test] fix partition prune unstable cases (#44976) (#45846)
cherry-pick #44976 to branch-2.1
2024-12-24 19:06:26 +08:00
956138da5c [enhancement](blacklist) ignore shutdown message to avoid add the backend to blacklist (#45875)
### What problem does this PR solve?

1. If one request failed to send fragment to BE then it will call
removeProxy;
2. removeProxy will call shutdown and shutdown will cancel all requests
related with this client.
3. Other request will get exception when receive data from be and will
add this be to blacklist.

Should avoid this scenario.

None
2024-12-24 19:03:51 +08:00
c9d3f1122b [regression-test](framework) disable defining global variable in test… (#45840) 2024-12-24 17:21:27 +08:00
8b35b0e477 branch-2.1: [fix](function) fixed some nested type func's param type which is not suitable and make result wrong #44923 (#45798)
Cherry-picked from #44923
2024-12-24 14:57:33 +08:00
69704df447 [enhancement](blacklist) print the detail stack that the client is shutdown (#45850) 2024-12-24 14:52:58 +08:00
9c7280bd58 [test](case) add partition prune cases (#44722) (#45823)
cherry-pick #44722 to branch-2.1
2024-12-24 14:25:58 +08:00
4514652aa2 [fix](profile) Fix clean profile (#45838)
### What problem does this PR solve?
Clean all profile command does not clean runtime profile. Makes runtime
profile leak, and memory can not be released.

Problem Summary:

![img_v3_02hs_62c4ed81-fe1f-423f-ba92-a1762af5162g](https://github.com/user-attachments/assets/4cec62a5-9077-4e36-91a8-29e5fedd4560)
After clean all profile, summary profile is clear, but runtime profile
and counter still exists.

![img_v3_02hs_4de10ca2-b878-4722-9b6e-9738270e6a8g](https://github.com/user-attachments/assets/b30754f2-a401-4127-aa5f-ed053d9f5c75)

After fix:

![img_v3_02hs_750960cf-9ebd-41e1-8f13-b41e5bc094dg](https://github.com/user-attachments/assets/5a5edb0b-bd7d-439e-92c2-0ed69589a7b5)

![img_v3_02hs_7e111a74-bacb-43e0-8acb-ec406ce68d8g](https://github.com/user-attachments/assets/556af476-7ea0-4924-880e-183f2cf9e870)
2024-12-24 14:25:03 +08:00
02f15a8ef0 [fix](inverted index) Fix Null Pointer Exception in function match(#45456)(#45774)
pick: https://github.com/apache/doris/pull/45456
2024-12-24 11:27:13 +08:00
1c47f5cf57 [regression-test](fix) fix case with invalid grammer (#45211) (#45791)
Problem Summary:
Defining global variables in a test suite can easily introduce errors,
such as variable conflicts in concurrent scenarios. In #44576 , a method
for detecting global variables was introduced.

That approach conflicts with the function calling like
connect(user=user, password="pwd"). Furthermore, the connect(user=user,
password="pwd") method cannot specify assignments to parameters with
default values; values still need to be assigned according to the order
of parameters. Therefore, modifying these calling method.
2024-12-24 11:10:28 +08:00
aa728aef66 [Improvement](tools) refine tools schema (#45825)
pick from master #45778
2024-12-24 10:27:08 +08:00
536ca0bfe6 [fix](auditlog)Fix auditlog NPE (#45772) (#45808)
backport: https://github.com/apache/doris/pull/45772
2024-12-23 21:22:41 +08:00
3de3c6ff44 [fix](group commit) fix group commit session (#45800)
if connect to observer, enable group commit and do insert overwrite,
will get:
```
ERROR 1105 (HY000): errCode = 2, detailMessage = errCode = 2, detailMessage = Nereids does not support group commit insert
```
2024-12-23 19:09:54 +08:00
901b93264c branch-2.1: [opt](log) add more info in nereids timeout log #45705 (#45793)
cherry-pick #45705
2024-12-23 17:57:38 +08:00
6454ee6589 branch-2.1: [Fix](ccr) Incorrectly generated sql with NONE aggregate type #45452 (#45489)
Cherry-picked from #45452

Co-authored-by: Uniqueyou <wangyixuan@selectdb.com>
2024-12-23 16:48:29 +08:00
412355b749 branch-2.1: [fix](nereids) fix comparison with date like #45735 (#45781)
Cherry-picked from #45735

Co-authored-by: yujun <yujun@selectdb.com>
2024-12-23 15:21:08 +08:00
3006eea5cc branch-2.1: [Improve](Variant) pick random backend as coordinator #45754 (#45771)
Cherry-picked from #45754

Co-authored-by: lihangyu <lihangyu@selectdb.com>
2024-12-23 14:02:18 +08:00
Pxl
d70c17bdc0 [Improvement](scan) use loop to instead recursion on Level1Iterator::_normal_next #38005 (#45767)
pick from  #38005
2024-12-23 13:54:12 +08:00
fe3632aa02 branch-2.1: [Fix](ccr) Wrong result of tosql for add column with comment #45319 (#45727)
pick: https://github.com/apache/doris/pull/45319
2024-12-23 11:20:05 +08:00
b845813e6f [opt](fs)add local file support for 2.1 (#45632) (#45702)
bp: #45632
2024-12-23 10:25:34 +08:00
818fd88a10 [test](p0) force replica = 1 in p0 regression tests (#45684)
### What problem does this PR solve?

Some tests in p0 does not set replica = 1, and will fail when BE number
is not enough.

This PR fixed this problem by set
`force_olap_table_replication_allocation` in p0 fe.conf
2024-12-23 10:23:50 +08:00
4a7e2911c7 branch-2.1: [fix](catelog) Unifies partition items string #45669 (#45700)
Cherry-picked from #45669

---------

Co-authored-by: walter <maochuan@selectdb.com>
2024-12-23 10:04:33 +08:00
ca9f0ade25 [Fix](ShortCircuit) fix prepared statement with partial arguments prepared (#45371) (#45732)
(#45371)
2024-12-23 10:03:34 +08:00
608000b8ce branch-2.1: [enhance](mtmv)Change the way to verify the existence of partition names when refreshing MTMV #45290 (#45725)
Cherry-picked from #45290

Co-authored-by: zhangdong <zhangdong@selectdb.com>
2024-12-23 10:01:02 +08:00
35bdd0019e branch-2.1: [fix](routine load) make routine load delay eof schedule work #45528 (#45724)
Cherry-picked from #45528

Co-authored-by: hui lai <laihui@selectdb.com>
2024-12-23 09:59:03 +08:00