Commit Graph

14973 Commits

Author SHA1 Message Date
e06e976a8b [test](case) delete duplicate pipelineX cases (#27381) 2023-11-22 12:58:30 +08:00
5442e8d1fc [pipelineX](dependency) split different dependencies (#27366) 2023-11-22 12:50:39 +08:00
604842526b [improvement](expr) improve if expr performance (#27296) 2023-11-22 12:48:06 +08:00
1441387e5b [improvement](doris compose) improve start be container speed (#27368) 2023-11-22 11:04:04 +08:00
Pxl
b541de7a03 do not push down agg on aggregate column (#27356)
do not push down agg on aggregate column
2023-11-22 10:53:29 +08:00
b821672f8b [test](regression) add 'sync' for som stream load (#27357) 2023-11-22 10:52:34 +08:00
eaa1ca7143 [fix](fe) Fix show frontends npt in some situations (#27295)
```
java.lang.NullPointerException: null
    at com.sleepycat.je.rep.util.ReplicationGroupAdmin.getMasterSocket(ReplicationGroupAdmin.java:191)
    at com.sleepycat.je.rep.util.ReplicationGroupAdmin.doMessageExchange(ReplicationGroupAdmin.java:607)
    at com.sleepycat.je.rep.util.ReplicationGroupAdmin.getGroup(ReplicationGroupAdmin.java:406)
    at org.apache.doris.ha.BDBHA.getElectableNodes(BDBHA.java:132)
    at org.apache.doris.common.proc.FrontendsProcNode.getFrontendsInfo(FrontendsProcNode.java:84)
    at org.apache.doris.qe.ShowExecutor.handleShowFrontends(ShowExecutor.java:1923)
    at org.apache.doris.qe.ShowExecutor.execute(ShowExecutor.java:355)
    at org.apache.doris.qe.StmtExecutor.handleShow(StmtExecutor.java:2113)
    ...
```
2023-11-22 10:46:59 +08:00
c332811721 [fix](ci) fix bug of new trigger (#27379)
* [fix](ci) fix bug of new trigger
Co-authored-by: stephen <hello-stephen@qq.com>
2023-11-22 10:35:39 +08:00
e7a2b7b862 [comm](github) add submodule create-or-update-comment (#27301) 2023-11-22 10:24:38 +08:00
4fbcad9c7c [minor](show_export) make result of file url usable (#27209)
* [minor](show_export) make result of file url usable

* update regression-test

* update regression-test
2023-11-22 10:14:45 +08:00
34f500ac2a [github](action) optimize trigger teamcity pipeline (#27257)
optimize triiger teamcity pipeline
2023-11-22 09:34:47 +08:00
9b59bc14b5 [test](Export) add show export regression testes (#27140) 2023-11-22 00:13:30 +08:00
6e86bf5b1b [test](decimalv2) add some regression cases about decimalv2 (#27352)
All cases' results are tested and passed with decimalv3
Cases about:
Calculation ( +, - , *, /)
Kinds of predicates(<, >, =, <>, in, not in, is null, is not null)
Load test(from csv and select into)
Runtime filter
Delete conditions
Key columns(agg/duplicate/uniq model, distributed/partition, bitmap index...)
2023-11-21 21:36:20 +08:00
c9b959d2d8 [opt](Nereids) AssertNumRows node should triger runtime filter pruning #27279
1. optimize rf prune when col stats are not avaliable
2. add regression case to check plan and rf for tpcds_sf100 with stats
3. add regression case to check plan and rf for tpcds_sf100 without stats
2023-11-21 21:00:41 +08:00
1cd1c58eee [Feature](group commit) move group_commit_interval_ms from be.conf to table property (#27116) 2023-11-21 20:50:02 +08:00
c1435c0589 [regression test](routine test) add case for send_batch_parallelism (#27333) 2023-11-21 20:43:20 +08:00
a0fab03289 [case](regression-test) Add backup & restore regression test with auto Bucket and dynamic partition (#27033)
* (Case)Add backup & restore regression test
1. (case)Table with auto Bucket
2. Table with dynamic partition
(case)set "reserve_dynamic_partition_enable = true" on restore properties
(case)set "reserve_dynamic_partition_enable = false" on restore properties

Co-authored-by: Bears0haunt <bearshaunt0@gamil.com>
2023-11-21 20:32:45 +08:00
2f04873da9 [fix](clone) Fix engine_clone file exist (#27361)
Signed-off-by: Jack Drogon <jack.xsuperman@gmail.com>
2023-11-21 20:30:27 +08:00
d541424936 do not check invisible column stats (#27201)
1. forbid_unknown_col_stats check ignore invisible columns
2. a better error message when meet unknown col stats
2023-11-21 19:46:16 +08:00
ffd4face00 [refactor](callonce) remove callonce usage in pipeline task to get full stack trace when core (#27331)
---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
2023-11-21 19:42:39 +08:00
dea40e7095 [fix](Nereids): NullSafeEqual should be in HashJoinCondition (#27127)
Originally, we just put `EqualTo` in `HashJoinCondition`, we also need to allow `NullSafeEqual`
2023-11-21 19:08:14 +08:00
445484270b [fix](Nereids): fill miss slot in having subquery (#27177)
fill miss slot in having subquery.

such as 
```
select * from t group by k having max(k) in (select k from t2)
```

the max(k) should be push down aggregate
2023-11-21 18:54:58 +08:00
6fb57c2e1a [fix](docs) fix auto partition version label #27358 2023-11-21 18:38:48 +08:00
1ebb54afdc [fix](null equal) fix coredump of pushing eq_for_null (#27341) 2023-11-21 18:36:33 +08:00
f7a1c3edac [FIX](jsonb)fix jsonb is not in predict column #27325 2023-11-21 18:35:12 +08:00
e58983e59c [improvement](publish version) reduce publish version not exist log (#27332) 2023-11-21 18:00:01 +08:00
7e707f5d64 [fix](fe ut) Fix OlapQueryCacheTest failed (#27305)
1. 
```
java.lang.NullPointerException: null
        at org.apache.doris.catalog.Env.getCurrentSystemInfo(Env.java:793) ~[classes/:?]
        at org.apache.doris.qe.SimpleScheduler$UpdateBlacklistThread.run(SimpleScheduler.java:206) ~[classes/:?]
        at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_382]

java.lang.NullPointerException
        at org.apache.doris.qe.OlapQueryCacheTest.setUp(OlapQueryCacheTest.java:226)
```

2.
```
[ERROR] testSqlCacheKeyWithNestedViewForNereids  Time elapsed: 1.962 s  <<< FAILURE!
java.lang.AssertionError: SELECT command denied to user 'testCluster:testUser'@'192.168.1.1' for table 'internal: testCluster:testDb: appevent'
	at org.apache.doris.qe.OlapQueryCacheTest.parseSqlByNereids(OlapQueryCacheTest.java:579)
	at org.apache.doris.qe.OlapQueryCacheTest.testSqlCacheKeyWithNestedViewForNereids(OlapQueryCacheTest.java:1338)
```

3.
```
[ERROR] Tests run: 28, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 113.63 s <<< FAILURE! - in org.apache.doris.qe.OlapQueryCacheTest
[ERROR] testCacheModeTable  Time elapsed: 1.657 s  <<< ERROR!
java.lang.IllegalArgumentException: Value of type org.apache.doris.qe.QueryState incompatible with return type org.apache.doris.system.SystemInfoService of org.apache.doris.catalog.Env#getCurrentSystemInfo()
        at org.apache.doris.qe.OlapQueryCacheTest.setUp(OlapQueryCacheTest.java:156)
```
2023-11-21 16:47:57 +08:00
721282242b [pipelineX](bug) Fix potential memory access problem (#27326) 2023-11-21 14:36:25 +08:00
016dccb4b7 [fix](null_equal) fix wrong result and coredump of operator <=> (#27312)
* [fix](null_equal) fix wrong result and coredump of operator <=>

* fix
2023-11-21 14:32:03 +08:00
402095b85e [doc](fix) fix date trunc doc (#27317) 2023-11-21 14:14:37 +08:00
fcf7691e2f [pipeline](close) change the close order in pipeline engine (#27290) 2023-11-21 14:10:21 +08:00
b63f00987f [regression](invered index) add test null index case and fix fault injection index case (#27265) 2023-11-21 14:04:57 +08:00
dd2e9f682f [Fix](statistics)Fix analyze sql including key word bug (#27321)
Fix analyze sql including key word bug. Need use `` to wrap column names.
2023-11-21 13:15:37 +08:00
Pxl
cee8cc44e2 [Bug](insert)fix insert wrong data on mv when stmt have multiple values (#27297)
fix insert wrong data on mv when stmt have multiple values
2023-11-21 12:55:30 +08:00
d809bee46e [refactor](profilev2) add BlocksProduced RowsProduced counter #27291 2023-11-21 12:01:11 +08:00
3e8177bbbd Revert "[improvement](routine-load) add routine load rows check (#25818)" (#27304)
This reverts commit 5e3277e8fbc92f08344bc86e9b2c74033a8bbc01.
2023-11-21 11:28:05 +08:00
2548e27c97 [fix](memory) Fix work load group meaningless GC #27307 2023-11-21 09:59:21 +08:00
87a7de7660 [pipelineX](bug) Fix 'No broadcast buffer left' (#27298) 2023-11-20 22:37:21 +08:00
c0f22e8feb [FIX](complextype)fix struct nested complex collection type and and regresstest (#26973) 2023-11-20 22:29:12 +08:00
840f3b6439 [pipelineX](dependency) Wake by task by read dependency (#27260) 2023-11-20 22:00:59 +08:00
0b459e50fb [fix](partial update) keep case insensitivity and use the columns' origin names in partialUpdateCols in origin planner (#27223)
close: #27161
2023-11-20 21:16:28 +08:00
fec94b7278 [feature](Nereids): use session variable to enable rule (#27036) 2023-11-20 20:23:24 +08:00
20d7ab061b [PipelineX](complex type pushdown) do not push down complex type to storage layer based on PR#25007 (#27278) 2023-11-20 18:47:59 +08:00
19b1d5365c [minor](stats) rename stats related session variable name #26936 2023-11-20 18:13:12 +08:00
273cbfc36c [fix](inverted index) reset fs_writer to nullptr before throw exception (#27202) 2023-11-20 17:40:56 +08:00
7164b7cebb [case](regression) add read json by line test case (#26670)
Co-authored-by: qinhao <qinhao@newland.com.cn>
2023-11-20 17:33:40 +08:00
80c75b6da4 [fix](schema change) fix bug of query failure after rename column (#26300) 2023-11-20 16:54:40 +08:00
bd2e8239f8 [fix](broker) not print broker request detail default (#27126)
if broker detail contails ak/sk or some sensitive information, not print these info default in log
---------
Signed-off-by: nextdreamblue <zxw520blue1@163.com>
2023-11-20 16:41:47 +08:00
34c3cde0de Revert "[feature-wip](catalog) support deltalake catalog step1-metadata (#22493)" (#27095)
This reverts commit 5b641ebd40fff71e632ee9be4ede58b744b602b9.

Currently, Deltalake Catalog is not a usable feature. We will continue to implement it in the datalake plug-in system in the future, so we will delete it from the FE code for now.
2023-11-20 16:10:33 +08:00
d939903753 [improvement](statistics)Use count as ndv for unique/agg olap table single key column (#27186)
Single key column of unique/agg olap table has the same value of count and ndv, for this kind of column,
don't need to calculate ndv, simply use count as ndv.
2023-11-20 15:49:08 +08:00