Commit Graph

18263 Commits

Author SHA1 Message Date
d72d5c9b5d [fix](inverted index) normal process query for null condition when index is missing (#33663) 2024-04-17 23:42:14 +08:00
520f0f51d1 [improvement](spill) avoid occuping too much memory while spill build block during the hash join build phase (#33747) 2024-04-17 23:42:14 +08:00
b518b9dd15 [shuffle](minor) Log error status if exchange is shutdown early (#33748) 2024-04-17 23:42:14 +08:00
4460d23cd9 [chore](variable) update nereids timeout second default value to 30 (#33749) 2024-04-17 23:42:14 +08:00
b5640ae763 [fix](restore) add indexes as part of table signature (#33650) 2024-04-17 23:42:14 +08:00
ffa0e57122 [enhancement](auditlog) ignore any errors in write audit log (#33750) 2024-04-17 23:42:14 +08:00
5555cc175f [feature](windows function)Improve error handling for window functions (#33673) 2024-04-17 23:42:14 +08:00
cb255f688f [Fix](inverted index) fix memory leak when inverted index writer not close (#33724) 2024-04-17 23:42:14 +08:00
f48a4680a8 [fix](test) remove array_intersect case since its result is not stable (#33742) 2024-04-17 23:42:14 +08:00
c6d1d75ff2 [Fix](Json type) forbit schema change adding JSON columns with none null default value (#33686) 2024-04-17 23:42:14 +08:00
2ca4278c42 [refactor](pipelineX) Reduce prepare overhead (PART III) (#33689) 2024-04-17 23:42:14 +08:00
2648a92594 [FIX](load)fix load with split-by-string (#33713) 2024-04-17 23:42:14 +08:00
Pxl
9bcb23351c [Improvement](runtime-filter) make sync rf size work when need_local_merge (#33717)
make sync rf size work when need_local_merge
2024-04-17 23:42:14 +08:00
6a33067daf [fix](chore) update some CHECK to CHECK_EQ, to print more details while CHECK failed (#33709) 2024-04-17 23:42:13 +08:00
bb33375dba [test](xor) add test for xor #33731 2024-04-17 23:42:13 +08:00
89c4fa5a75 [fix](move-memtable) close wait on all sinks (#33710) 2024-04-17 23:42:13 +08:00
f52067415b Improve analyze mv/mtmv wait row count report logic. (#33695) 2024-04-17 23:42:13 +08:00
ff8cb3cc43 [Fix](executor)Fix routine load failed when can not find group (#33596) 2024-04-17 23:42:13 +08:00
b44fed8dc2 [fix](restore) Reset index id for restore (#33648) 2024-04-17 23:42:13 +08:00
11f7d11a96 [branch-2.1](regression-test) fix test_chema_change_fail (#33753) #33788 2024-04-17 23:42:13 +08:00
5734e2bd30 [opt](meta-cache) refine the meta cache (#33449) (#33754)
bp #33449
2024-04-17 23:42:13 +08:00
6976d019a3 [opt](inverted index) topn opt reads only limit number of records (#33665) 2024-04-17 23:42:13 +08:00
2854048eb5 fix compile 2024-04-17 23:42:13 +08:00
5053ac2b59 [improve](CI)Core modules require maintainer review (#32468) 2024-04-17 23:42:13 +08:00
43974a2334 (Fix)(nereids) modify create view privilege check error message (#33669) 2024-04-17 23:42:13 +08:00
db846709d2 [opt](Nereids) auto fallback when meet udf override (#33708) 2024-04-17 23:42:13 +08:00
81f7c53bad [fix](Nereids) could not query variant that not from table (#33704) 2024-04-17 23:42:13 +08:00
1c025c0488 [docker](hive) add hive3 docker compose and modify scripts (#33115)
add hive3 docker compose from:
big-data-europe/docker-hive#56
2024-04-17 23:42:13 +08:00
22a6b1d3f5 [feature](function) support hll functions hll_from_base64, hll_to_base64 (#32089)
Issue Number: #31320 

Support two hll functions:

- hll_from_base64
Convert a base64 string(result of function hll_to_base64) into a hll.
- hll_to_base64
Convert an input hll to a base64 string.
2024-04-17 23:42:13 +08:00
3096150d1b [feature](agg) support aggregate function group_array_intersect (#33265) 2024-04-17 23:42:13 +08:00
07a8f44443 [improvement](spill) improve config and fix spill bugs (#33519) 2024-04-17 23:42:13 +08:00
3f267e36d1 [fix](nereids)InSubquery's withChildren method lost typeCoercionExpr (#33692) 2024-04-17 23:42:13 +08:00
b07e0a2f06 [FIX](cast)fix full/right out join for cast array (#33475)
in some case, we has code
```
        if (_join_op == TJoinOp::RIGHT_OUTER_JOIN || _join_op == TJoinOp::FULL_OUTER_JOIN) {
            _probe_column_convert_to_null = _convert_block_to_null(*input_block);
        }
```
then do next function like cast , but in function cast we assume block column is same with from_type.which will make status error
2024-04-17 23:42:13 +08:00
2890f6c3cf [opt](Nereids) date literal support basic format with timezone (#33662) 2024-04-17 23:42:13 +08:00
11266dd9b8 [minor](Nereids): remove useless override (#33651) 2024-04-17 23:42:13 +08:00
16e9eb3b05 [fix](analyze) avoid java.util.ConcurrentModificationException (#33674)
```
java.util.ConcurrentModificationException: null
        at java.util.TreeMap$ValueSpliterator.forEachRemaining(TreeMap.java:3226) ~[?:?]
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921) ~[?:?]
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
        at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682) ~[?:?]
        at org.apache.doris.statistics.AnalysisManager.findShowAnalyzeResult(AnalysisManager.java:552) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.statistics.AnalysisManager.showAnalysisJob(AnalysisManager.java:533) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ShowExecutor.handleShowAnalyze(ShowExecutor.java:2772) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ShowExecutor.execute(ShowExecutor.java:447) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.handleShow(StmtExecutor.java:2738) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.executeByLegacy(StmtExecutor.java:1010) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:624) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:526) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:333) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:228) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:176) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:205) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:258) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
        at java.lang.Thread.run(Thread.java:840) ~[?:?]
```

Due to the `Collections.synchronizedNavigableMap`'s java doc:

```
In order to guarantee serial access, it is critical that all access to the backing navigable map is accomplished through the returned navigable map (or its views).
It is imperative that the user manually synchronize on the returned navigable map when traversing any of its collection views, or the collections views of any of its subMap, headMap or tailMap views, via Iterator, Spliterator or Stream
```
2024-04-17 23:42:13 +08:00
ca728a2405 [feature](proc)Add table's indexes info in show proc interface (#33438)
1. Add show proc `/dbs/db_id/table_id/indexes` impl
2. Remove index_id in `show index from table`
3. Add test cases

---------

Co-authored-by: Luennng <luennng@gmail.com>
2024-04-17 23:42:13 +08:00
dac2829194 [fix](routine-load) fix data lost when FE leader change (#33678) 2024-04-17 23:42:13 +08:00
53a3d6c154 [fix](testcase) fix miss used global variables in index testcases (#33293) 2024-04-17 23:42:13 +08:00
775022c204 [refactor](pipelineX) Reduce prepare overhead (PART II) (#33681) 2024-04-17 23:42:13 +08:00
59de97be5e [improvement](mow) Add profile for delete_bitmap get_agg function (#33576) 2024-04-17 23:42:13 +08:00
d6a63b9789 [Update](cloud) skip show data size assertion in cloud mode (#33677) 2024-04-17 23:42:13 +08:00
d15981abd2 [Enhencement](Nereids) add rule of agg(case when) to agg(filter) (#33598) 2024-04-17 23:42:13 +08:00
1fba73eea4 [fix](fe) Fix finalizeCommand sendAndFlush NullPointerException (#33420) 2024-04-17 23:42:13 +08:00
fe3b6824b3 [case](regression) Add backup temp partition case (#33646) 2024-04-17 23:42:13 +08:00
8e38549a92 [fix](nereids) Use correct PREAGGREGATION in agg(filter(scan)) (#33454)
1. set `PreAggStatus` to `ON` when agg key column by max or min;
2. #28747 may change `PreAggStatus` of scan, inherit it from the previous one.
2024-04-17 23:42:13 +08:00
d18f5e2544 [refactor](refresh-catalog) refactor the refresh catalog code (#33653)
To unify the code.
In previous, we do catalog refresh in `CatalogMgr`, but do
database and table refresh in `RefreshMgr`, which is very confusing.

This PR move all `refresh` related code from CatalogMgr to RefreshMgr.

No logic is changed in this PR.
2024-04-17 23:42:12 +08:00
4863167f90 [refactor](pipelineX) Reduce prepare overhead (PART I) (#33550) 2024-04-17 23:42:12 +08:00
e0ec2da29b [fix](routine-load) fix get kafka offset timeout may too long (#33502) 2024-04-17 23:42:12 +08:00
Pxl
341cb40693 [Chore](log) adjust output order on PrintInstanceStandardInfo and reduce warning log when rpc finished (#33652)
adjust output order on PrintInstanceStandardInfo and reduce warning log when rpc finished
2024-04-17 23:42:12 +08:00