Commit Graph

8173 Commits

Author SHA1 Message Date
ff8cb3cc43 [Fix](executor)Fix routine load failed when can not find group (#33596) 2024-04-17 23:42:13 +08:00
b44fed8dc2 [fix](restore) Reset index id for restore (#33648) 2024-04-17 23:42:13 +08:00
5734e2bd30 [opt](meta-cache) refine the meta cache (#33449) (#33754)
bp #33449
2024-04-17 23:42:13 +08:00
2854048eb5 fix compile 2024-04-17 23:42:13 +08:00
43974a2334 (Fix)(nereids) modify create view privilege check error message (#33669) 2024-04-17 23:42:13 +08:00
db846709d2 [opt](Nereids) auto fallback when meet udf override (#33708) 2024-04-17 23:42:13 +08:00
81f7c53bad [fix](Nereids) could not query variant that not from table (#33704) 2024-04-17 23:42:13 +08:00
22a6b1d3f5 [feature](function) support hll functions hll_from_base64, hll_to_base64 (#32089)
Issue Number: #31320 

Support two hll functions:

- hll_from_base64
Convert a base64 string(result of function hll_to_base64) into a hll.
- hll_to_base64
Convert an input hll to a base64 string.
2024-04-17 23:42:13 +08:00
3096150d1b [feature](agg) support aggregate function group_array_intersect (#33265) 2024-04-17 23:42:13 +08:00
07a8f44443 [improvement](spill) improve config and fix spill bugs (#33519) 2024-04-17 23:42:13 +08:00
3f267e36d1 [fix](nereids)InSubquery's withChildren method lost typeCoercionExpr (#33692) 2024-04-17 23:42:13 +08:00
2890f6c3cf [opt](Nereids) date literal support basic format with timezone (#33662) 2024-04-17 23:42:13 +08:00
11266dd9b8 [minor](Nereids): remove useless override (#33651) 2024-04-17 23:42:13 +08:00
16e9eb3b05 [fix](analyze) avoid java.util.ConcurrentModificationException (#33674)
```
java.util.ConcurrentModificationException: null
        at java.util.TreeMap$ValueSpliterator.forEachRemaining(TreeMap.java:3226) ~[?:?]
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921) ~[?:?]
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
        at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682) ~[?:?]
        at org.apache.doris.statistics.AnalysisManager.findShowAnalyzeResult(AnalysisManager.java:552) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.statistics.AnalysisManager.showAnalysisJob(AnalysisManager.java:533) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ShowExecutor.handleShowAnalyze(ShowExecutor.java:2772) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ShowExecutor.execute(ShowExecutor.java:447) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.handleShow(StmtExecutor.java:2738) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.executeByLegacy(StmtExecutor.java:1010) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:624) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:526) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:333) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:228) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:176) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:205) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:258) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
        at java.lang.Thread.run(Thread.java:840) ~[?:?]
```

Due to the `Collections.synchronizedNavigableMap`'s java doc:

```
In order to guarantee serial access, it is critical that all access to the backing navigable map is accomplished through the returned navigable map (or its views).
It is imperative that the user manually synchronize on the returned navigable map when traversing any of its collection views, or the collections views of any of its subMap, headMap or tailMap views, via Iterator, Spliterator or Stream
```
2024-04-17 23:42:13 +08:00
ca728a2405 [feature](proc)Add table's indexes info in show proc interface (#33438)
1. Add show proc `/dbs/db_id/table_id/indexes` impl
2. Remove index_id in `show index from table`
3. Add test cases

---------

Co-authored-by: Luennng <luennng@gmail.com>
2024-04-17 23:42:13 +08:00
dac2829194 [fix](routine-load) fix data lost when FE leader change (#33678) 2024-04-17 23:42:13 +08:00
d15981abd2 [Enhencement](Nereids) add rule of agg(case when) to agg(filter) (#33598) 2024-04-17 23:42:13 +08:00
1fba73eea4 [fix](fe) Fix finalizeCommand sendAndFlush NullPointerException (#33420) 2024-04-17 23:42:13 +08:00
8e38549a92 [fix](nereids) Use correct PREAGGREGATION in agg(filter(scan)) (#33454)
1. set `PreAggStatus` to `ON` when agg key column by max or min;
2. #28747 may change `PreAggStatus` of scan, inherit it from the previous one.
2024-04-17 23:42:13 +08:00
d18f5e2544 [refactor](refresh-catalog) refactor the refresh catalog code (#33653)
To unify the code.
In previous, we do catalog refresh in `CatalogMgr`, but do
database and table refresh in `RefreshMgr`, which is very confusing.

This PR move all `refresh` related code from CatalogMgr to RefreshMgr.

No logic is changed in this PR.
2024-04-17 23:42:12 +08:00
4863167f90 [refactor](pipelineX) Reduce prepare overhead (PART I) (#33550) 2024-04-17 23:42:12 +08:00
e0ec2da29b [fix](routine-load) fix get kafka offset timeout may too long (#33502) 2024-04-17 23:42:12 +08:00
466b9f35d5 [fix](nereids)EliminateGroupBy should keep the output's datatype same as old ones (#33585) 2024-04-17 23:42:12 +08:00
f6af79c0ed [fix](catalog) Remove unexpected cleanup when reading jdbc data (#33529) 2024-04-17 23:42:12 +08:00
7659b1aa67 [opt](Nereids) prefer slot type to support delete task better (#33559) 2024-04-17 23:42:12 +08:00
e53a76d75b [fix](planner) fix bug of InlineViewRef's tableNameToSql method (#33575) 2024-04-17 23:42:12 +08:00
d000658a9b [fix](nereids) ExtractAndNormalizeWindowExpression should only normalize alias in output (#33527) 2024-04-17 23:42:12 +08:00
b2face0d20 [feature](Nereids): date literal suppose Zone (#33534)
support
```
'2022-05-01 01:02:55+02:30
'2022-05-01 01:02:55Asia/Shanghai
```
2024-04-17 23:42:12 +08:00
2cd4012541 [opt](scan) read scan ranges in the order of partitions (#33515) (#33657)
backport: #33515
2024-04-17 23:42:12 +08:00
55be6c6309 [fix](schema change) follow fe set sc fail replicas as bad (#33569) 2024-04-17 23:42:12 +08:00
75d0d8f2cc [fix](create table) Fix create table exception without cleaning the e… (#33574) 2024-04-17 23:42:12 +08:00
bb817e419c [fix](Nereids) create view should forward to master (#33626)
fix: create view should forward to master
2024-04-17 23:42:12 +08:00
4740b22481 [fix](test) fix some p2 external table test cases (#33624)
bp #33621
Also fix a merge bug from #33245
2024-04-17 23:42:12 +08:00
b710095d41 fix compile 2024-04-17 23:42:12 +08:00
face7c42fd [enhancement](plsql) Support select * from routines (#32866)
Support show of plsql procedure using select * from routines.
2024-04-17 23:42:12 +08:00
1be753ed75 [enhancement](mysql compatible) add user and procs_priv tables to mysql db in all catalogs (#33058)
Issue Number: close #xxx

This PR aims to enhance the compatibility of BI tools (such as Dbeaver, DataGrip) when using the mysql connector to connect to Doris, because some BI tools query some tables in the mysql database. In our tests, the user and procs_priv tables were mainly queried. This PR adds these two tables and adds actual data to the user table. However, please note that most of the fields in the user table are in Doris' own format rather than mysql format, so it can only ensure that the BI tool is querying No error is reported when accessing these tables, which does not guarantee that the data is completely displayed, and the tables under Doris's mysql database do not support data modification.
Thanks to @liujiwen-up for assisting in testing
2024-04-17 23:42:12 +08:00
e88bd3ca4c [bugfix](feoom) add timeout to send be request to avoid too long to occupy memory (#33614)
Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-04-17 23:42:11 +08:00
56b7839447 [feature](backup) ignore table that not support type when backup, and… (#33158)
* [feature](backup) ignore table that not support type when backup, and not report exception

Signed-off-by: nextdreamblue <zxw520blue1@163.com>

* fix

Signed-off-by: nextdreamblue <zxw520blue1@163.com>

---------

Signed-off-by: nextdreamblue <zxw520blue1@163.com>
2024-04-17 23:42:11 +08:00
348a043b76 [neride][foldconstant] disable fold constant by be 2024-04-17 23:42:00 +08:00
a545a34eec [fix](feservice) remove connect context (#33583) 2024-04-17 23:42:00 +08:00
379c2e0762 [Fix](Nereids) fix leading hint should have all tables in one query block (#33517) 2024-04-17 23:42:00 +08:00
7d1b66e5ed [Fix] fix compile problem (#33578)
Co-authored-by: feiniaofeiafei <moailing@selectdb.com>
2024-04-17 23:42:00 +08:00
80c6531cfc [fix](timeout) query timeout was not correctly set (#33444) 2024-04-17 23:42:00 +08:00
979e091e0a [opt](Nereids) support select async mv partition (#33560) 2024-04-17 23:42:00 +08:00
ae4a7c93ac [feat](nereids) support create view in nereids (#32743) 2024-04-17 23:42:00 +08:00
7bc82cab73 [chore](Nereids) better way to return error (#33507) 2024-04-17 23:42:00 +08:00
01f333086d [pipelineX](fix) Fix data pooling judgement for bucket join (#33533) 2024-04-17 23:42:00 +08:00
7f27461f4b [Chore](code comment) add comment for #32999 (#33548) 2024-04-17 23:41:59 +08:00
b2b385a4ff [improve](fold) support complex type for constant folding (#32867) 2024-04-17 23:41:59 +08:00
Pxl
883f0a96c4 [Improvementation](stream-load) improve streamLoadPut log warning detail (#33535)
improve streamLoadPut log warning detail
2024-04-17 23:41:59 +08:00