Commit Graph

18429 Commits

Author SHA1 Message Date
11266dd9b8 [minor](Nereids): remove useless override (#33651) 2024-04-17 23:42:13 +08:00
16e9eb3b05 [fix](analyze) avoid java.util.ConcurrentModificationException (#33674)
```
java.util.ConcurrentModificationException: null
        at java.util.TreeMap$ValueSpliterator.forEachRemaining(TreeMap.java:3226) ~[?:?]
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509) ~[?:?]
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499) ~[?:?]
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921) ~[?:?]
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
        at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682) ~[?:?]
        at org.apache.doris.statistics.AnalysisManager.findShowAnalyzeResult(AnalysisManager.java:552) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.statistics.AnalysisManager.showAnalysisJob(AnalysisManager.java:533) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ShowExecutor.handleShowAnalyze(ShowExecutor.java:2772) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ShowExecutor.execute(ShowExecutor.java:447) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.handleShow(StmtExecutor.java:2738) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.executeByLegacy(StmtExecutor.java:1010) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:624) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:526) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:333) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:228) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:176) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:205) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:258) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
        at java.lang.Thread.run(Thread.java:840) ~[?:?]
```

Due to the `Collections.synchronizedNavigableMap`'s java doc:

```
In order to guarantee serial access, it is critical that all access to the backing navigable map is accomplished through the returned navigable map (or its views).
It is imperative that the user manually synchronize on the returned navigable map when traversing any of its collection views, or the collections views of any of its subMap, headMap or tailMap views, via Iterator, Spliterator or Stream
```
2024-04-17 23:42:13 +08:00
ca728a2405 [feature](proc)Add table's indexes info in show proc interface (#33438)
1. Add show proc `/dbs/db_id/table_id/indexes` impl
2. Remove index_id in `show index from table`
3. Add test cases

---------

Co-authored-by: Luennng <luennng@gmail.com>
2024-04-17 23:42:13 +08:00
dac2829194 [fix](routine-load) fix data lost when FE leader change (#33678) 2024-04-17 23:42:13 +08:00
53a3d6c154 [fix](testcase) fix miss used global variables in index testcases (#33293) 2024-04-17 23:42:13 +08:00
775022c204 [refactor](pipelineX) Reduce prepare overhead (PART II) (#33681) 2024-04-17 23:42:13 +08:00
59de97be5e [improvement](mow) Add profile for delete_bitmap get_agg function (#33576) 2024-04-17 23:42:13 +08:00
d6a63b9789 [Update](cloud) skip show data size assertion in cloud mode (#33677) 2024-04-17 23:42:13 +08:00
d15981abd2 [Enhencement](Nereids) add rule of agg(case when) to agg(filter) (#33598) 2024-04-17 23:42:13 +08:00
1fba73eea4 [fix](fe) Fix finalizeCommand sendAndFlush NullPointerException (#33420) 2024-04-17 23:42:13 +08:00
fe3b6824b3 [case](regression) Add backup temp partition case (#33646) 2024-04-17 23:42:13 +08:00
8e38549a92 [fix](nereids) Use correct PREAGGREGATION in agg(filter(scan)) (#33454)
1. set `PreAggStatus` to `ON` when agg key column by max or min;
2. #28747 may change `PreAggStatus` of scan, inherit it from the previous one.
2024-04-17 23:42:13 +08:00
d18f5e2544 [refactor](refresh-catalog) refactor the refresh catalog code (#33653)
To unify the code.
In previous, we do catalog refresh in `CatalogMgr`, but do
database and table refresh in `RefreshMgr`, which is very confusing.

This PR move all `refresh` related code from CatalogMgr to RefreshMgr.

No logic is changed in this PR.
2024-04-17 23:42:12 +08:00
4863167f90 [refactor](pipelineX) Reduce prepare overhead (PART I) (#33550) 2024-04-17 23:42:12 +08:00
e0ec2da29b [fix](routine-load) fix get kafka offset timeout may too long (#33502) 2024-04-17 23:42:12 +08:00
Pxl
341cb40693 [Chore](log) adjust output order on PrintInstanceStandardInfo and reduce warning log when rpc finished (#33652)
adjust output order on PrintInstanceStandardInfo and reduce warning log when rpc finished
2024-04-17 23:42:12 +08:00
466b9f35d5 [fix](nereids)EliminateGroupBy should keep the output's datatype same as old ones (#33585) 2024-04-17 23:42:12 +08:00
c15ac3ffca [testcases](auto-partition) fix data sync (#33635)
fix data sync
2024-04-17 23:42:12 +08:00
7b16cb5a4c [feature](inverted index) add slop functionality to match_phrase (#33225)
https://github.com/apache/doris-website/pull/553 doc
2024-04-17 23:42:12 +08:00
f6af79c0ed [fix](catalog) Remove unexpected cleanup when reading jdbc data (#33529) 2024-04-17 23:42:12 +08:00
7659b1aa67 [opt](Nereids) prefer slot type to support delete task better (#33559) 2024-04-17 23:42:12 +08:00
5b616da543 [refine](Operator) When _stop_emplace_flag is not set to true, perform batch processing on the block. (#33173) 2024-04-17 23:42:12 +08:00
e53a76d75b [fix](planner) fix bug of InlineViewRef's tableNameToSql method (#33575) 2024-04-17 23:42:12 +08:00
d000658a9b [fix](nereids) ExtractAndNormalizeWindowExpression should only normalize alias in output (#33527) 2024-04-17 23:42:12 +08:00
87e6c94851 [docker](script)add --grace to be_prestop.sh (#33599) 2024-04-17 23:42:12 +08:00
b2face0d20 [feature](Nereids): date literal suppose Zone (#33534)
support
```
'2022-05-01 01:02:55+02:30
'2022-05-01 01:02:55Asia/Shanghai
```
2024-04-17 23:42:12 +08:00
3df8f0cad8 [improve](move-memtable) add more info in LoadStreamStub errors (#33618) 2024-04-17 23:42:12 +08:00
46a258dc85 [improvement](binlog)Support inverted index format v2 in CCR (#33415) 2024-04-17 23:42:12 +08:00
690bf54346 [runtime filter](fix) Fix wrong results caused by IN_OR_BLOOM filter (#33701) (#33715) 2024-04-17 23:42:12 +08:00
Pxl
918f7225b4 [Bug](runtime-filter) make need_local_merge unrelated with broadcast and support merge on bitmap_filter (#33664) 2024-04-17 23:42:12 +08:00
f8acb1ee2e [fix](merge-on-write) schema change may cause mow duplicate key (#33536) (#33660) 2024-04-17 23:42:12 +08:00
06a155abb0 [branch-2.1](cherry-pick) Pick some partial-update PR from master (#33639)
* [Fix](partial-update) Fix partial update fail when the datetime default value is 'current_time' (#32926)

* Problem: When importing data that includes datetime with a default value of current time for partial column updates, the import fails.
Reason: Partial column updates do not handle the logic for datetime default values.
Solution: During partial column updates, when the default value is set to current time, read the current time from the runtime state and write it into the data.

* [Enhancement](partial update)Add timezone case for partial update timestamp #33177

* [fix](partial update) Support partial update when the date default value is 'current_date'. This PR is a extension of PR #32926. (#33394)
2024-04-17 23:42:12 +08:00
e8ebc02e0b [enhancement](merge-on-write) compaction should not check correctness if tablet is not ready (#33435) (#33658) 2024-04-17 23:42:12 +08:00
2cd4012541 [opt](scan) read scan ranges in the order of partitions (#33515) (#33657)
backport: #33515
2024-04-17 23:42:12 +08:00
d0394b7f89 [fix](test) fix some unstable p2 test cases (#33637) (#33655)
bp #33637
2024-04-17 23:42:12 +08:00
ad82ed324e remove is cloud mode 2024-04-17 23:42:12 +08:00
Pxl
e85a2c8866 [Chore](status) change unknow filter error to internal error (#33633) 2024-04-17 23:42:12 +08:00
e7209d9a85 [fix](merge-iterator) Fix mem leak when get next batch failed (#33627) 2024-04-17 23:42:12 +08:00
55be6c6309 [fix](schema change) follow fe set sc fail replicas as bad (#33569) 2024-04-17 23:42:12 +08:00
75d0d8f2cc [fix](create table) Fix create table exception without cleaning the e… (#33574) 2024-04-17 23:42:12 +08:00
bb817e419c [fix](Nereids) create view should forward to master (#33626)
fix: create view should forward to master
2024-04-17 23:42:12 +08:00
4740b22481 [fix](test) fix some p2 external table test cases (#33624)
bp #33621
Also fix a merge bug from #33245
2024-04-17 23:42:12 +08:00
b710095d41 fix compile 2024-04-17 23:42:12 +08:00
a5db73070e [test](mtmv)Add column name case sensitive test (#33538) 2024-04-17 23:42:12 +08:00
face7c42fd [enhancement](plsql) Support select * from routines (#32866)
Support show of plsql procedure using select * from routines.
2024-04-17 23:42:12 +08:00
1be753ed75 [enhancement](mysql compatible) add user and procs_priv tables to mysql db in all catalogs (#33058)
Issue Number: close #xxx

This PR aims to enhance the compatibility of BI tools (such as Dbeaver, DataGrip) when using the mysql connector to connect to Doris, because some BI tools query some tables in the mysql database. In our tests, the user and procs_priv tables were mainly queried. This PR adds these two tables and adds actual data to the user table. However, please note that most of the fields in the user table are in Doris' own format rather than mysql format, so it can only ensure that the BI tool is querying No error is reported when accessing these tables, which does not guarantee that the data is completely displayed, and the tables under Doris's mysql database do not support data modification.
Thanks to @liujiwen-up for assisting in testing
2024-04-17 23:42:12 +08:00
e88bd3ca4c [bugfix](feoom) add timeout to send be request to avoid too long to occupy memory (#33614)
Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-04-17 23:42:11 +08:00
3decd850e2 [case](regression) Add backup restore test for hdfs repo (#33581) 2024-04-17 23:42:11 +08:00
8ee8de7857 [Fix](executor)reset remote scan thread num #33579 2024-04-17 23:42:11 +08:00
56b7839447 [feature](backup) ignore table that not support type when backup, and… (#33158)
* [feature](backup) ignore table that not support type when backup, and not report exception

Signed-off-by: nextdreamblue <zxw520blue1@163.com>

* fix

Signed-off-by: nextdreamblue <zxw520blue1@163.com>

---------

Signed-off-by: nextdreamblue <zxw520blue1@163.com>
2024-04-17 23:42:11 +08:00