database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
19,567 Commits 1 Branch 0 Tags
232ee74566af255d1b10d7064bc1401ac69c5201
Commit Graph

19567 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
airborne12
232ee74566 [Fix](inverted index) fix memory leak for index compaction (#38586) 
Pick from (#36209)
 2024-07-31 19:19:38 +08:00
airborne12
aed0cc8ba0 [Fix](inverted index) remove duplicate stats of inverted_index_query_cache_miss #36707 (#38580) 
cherry pick from #36707
 2024-07-31 19:18:58 +08:00
airborne12
7357d7bd3b [Update](inverted index) Add column name to debug point for "no need to read data" optimization #37649 (#38579) 
cherry pick from #37649
 2024-07-31 19:17:46 +08:00
Dongyang Li
c011060e4f [chore](ci) adjust thirdparty docker image source for easy management… (#38558) 
… (#37307)



pick from master #37307

Co-authored-by: stephen <hello-stephen@qq.com>
 2024-07-31 14:47:16 +08:00
谢健
e26f2441bd [opt](Nereids): disable PRUNE_EMPTY_PARTITION rule in SqlTestBase.java #38246 (#38455) 
## Proposed changes

#38246

Co-authored-by: zclllhhjj <zhaochangle@selectdb.com>
 2024-07-31 14:34:35 +08:00
HappenLee
3b234cfab6 [performance](exec) Performance problem create too many scanner task (#38460) 
## Proposed changes

cherry pick the pr: #38430

<!--Describe your changes.-->
 2024-07-31 14:34:01 +08:00
lihangyu
aa9bdd76d0 [Pick](Variant) pick some fix #38413 #38364 (#38512) 2024-07-31 11:03:31 +08:00
camby
9d8b2e85ae [fix](partial-update) insert only without auto_inc column should not use partial update (#38229) (#38504) 
cherry-pick #38229 to branch-2.1


## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-31 11:01:08 +08:00
starocean999
9e696d72f1 [fix](nereids)check functionBuilders is not null before using it (#38535) 
## Proposed changes

pick from master https://github.com/apache/doris/pull/38457

<!--Describe your changes.-->
 2024-07-31 11:00:39 +08:00
feiniaofeiafei
94111da2a9 [Fix](nereids) fix normalize repeat alias rewrite (#38166) (#38454) 
cherry-pick #38166 to branch-2.1
 2024-07-31 10:59:15 +08:00
walter
182bf4d323 [chore](fe) Returns dropped tables in GetMeta request (#38541) 
Cherry-pick #38019
 2024-07-31 10:57:00 +08:00
zhangdong
319933d0a6 [enhance](mtmv)mtmv date trunc support hour (#37678) (#38422) 
pick: https://github.com/apache/doris/pull/37678
 2024-07-31 10:46:08 +08:00
Jibing-Li
c78e04ac31 [improvement](statistics)Support get index row count and table delta rows. (#38520) 
backport: https://github.com/apache/doris/pull/38492
 2024-07-31 10:39:21 +08:00
zhangdong
d9d7f8de95 [fix](auth) fix create mv not need alter priv of table (#38011) (#38423) 
pick: https://github.com/apache/doris/pull/38011
 2024-07-31 10:35:03 +08:00
Mryange
017dad8c54 [fix](type)support runtime predicate for time type (#38258) (#38465) 
## Proposed changes
https://github.com/apache/doris/pull/38258
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-31 10:27:36 +08:00
LiBinfeng
742b98185c [test](Nereids) Add all hint tpcds tests (#38081) (#38368) 
cherry-pick:#38081
add all cases control by leading hint of tpcds
some cases are not supported because:
scalar/in/exists subquery: 6, 9, 28, 35, 41, 45, 58, 69, 95 set
operation to joins: 14, 23, 49, 56, 60, 66, 74, 75, 76, 77, 80, 87

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-31 10:24:10 +08:00
924060929
a328e01d97 [fix](nereids) fix insert stmt throw MultiCastDataSink cannot be cast to DataStreamSink (#38526) (#38547) 
fix `insert ... with ... select ...`, which not use some cte, and throw an exception:
```
errCode = 2, detailMessage = class org.apache.doris.planner.MultiCastDataSink cannot be cast to class org.apache.doris.planner.DataStreamSink (org.apache.doris.planner.MultiCastDataSink and org.apache.doris.planner.DataStreamSink are in unnamed module of loader 'app')
```
 2024-07-31 00:47:30 +08:00
924060929
1ba1e343bd [fix](nereids) fix The children format needs to be [WhenClause+, DefaultValue?] (#38491) (#38546) 
fix The children format needs to be [WhenClause+, DefaultValue?]
 2024-07-30 23:51:18 +08:00
morrySnow
e25eb733c1 [opt](compatibility) fill schema and table name in protocol column def (#38126) (#38522) 
pick from master #38126

according to mysql text protocol Protocol::ColumnDefinition41, should
fill schema name, table name into it if column from table.
 2024-07-30 17:41:07 +08:00
feiniaofeiafei
43ec98a30b [Feat](nereids) add pull up literal when infer predicates (#37314) (#38156) 
cherry-pick from master #37314
 2024-07-30 17:19:18 +08:00
minghong
377d4615c7 [feat](FE) add rest api to get last profile (#38268) (#38451) 
## Proposed changes
pick from #38268

if you want to get the profile of the query just executed, it not
neccessary to find the query_id now.
this is useful to get profiles in shell script, for example, we want to
get all profiles of tpcds queries (103 query profile) in a shell script.

before:
curl

http://root:@127.0.0.1:8030/api/profile/text?query_id=990cc3b19e464e03-a3af05a997eb4091
> profile
after:
curl http://root:@127.0.0.1:8030/api/profile/text > profilex

Issue Number: close #xxx

<!--Describe your changes.-->

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-30 13:36:43 +08:00
camby
715bcd13f1 [opt](mow) opt mow lookup with sequence column (#38287) (#38406) 2024-07-30 09:46:09 +08:00
airborne12
cefee4dbc0 [Pick 2.1](clucene) update clucene version (#38496) 
## Proposed changes

backport #38482
 2024-07-30 09:40:04 +08:00
Dongyang Li
d17b196459 [regression](s3) add default conf for s3 releated cases (#37952) (#38472) 
replace COS with OSS in the TeamCity pipeline to improve stability

pick from master #37952

---------

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-29 18:01:27 +08:00
hui lai
17d351af80 [fix](csv reader) fix csv parser incorrect if enclosing line_delimiter (#38347) (#38445) 
Csv reader parse data incorrect when data enclosing line_delimiter, for
example, line_delimiter is \n and enclose is ', data as follows:
```
'aaaaaaaaaaaa
bbbb'
```
it will be parsed as two columns: `'aaaaaaaaaaaa` and `bbbb',` rather
than one column
```
'aaaaaaaaaaaa
bbbb'
```

The reason why this happened is csv reader will not reset result when
not match enclose in this `output_buf_read`, causing incorrect
truncation was made.

Co-authored-by: Xin Liao <liaoxinbit@126.com>
 2024-07-29 14:55:45 +08:00
zclllhhjj
dab0138567 [branch-2.1](insert-overwrite) Fix insert overwrite auto detect transaction safe (#38103) (#38442) 
pick https://github.com/apache/doris/pull/38103
 2024-07-29 10:21:03 +08:00
Jerry Hu
87cf2d1fb4 [fix](spill) Duplicate calls to Dependency::set_ready() in hash join(#37461) (#38399) 
## Proposed changes

pick #37461
Duplicate calling the function `Dependency::set_ready()` will cause
pipeline tasks to be scheduled incorrectly.
 2024-07-29 09:44:48 +08:00
Xin Liao
e9f12fac47 [fix](load) fix no error url for stream load #38325 (#38417) 
cherry pick from #38325
 2024-07-28 19:06:57 +08:00
Xin Liao
d8744cd3d0 [Opt](load) don't print stack when some errors occur for stream load #38332 (#38418) 
cherry pick from #38332
 2024-07-28 19:04:24 +08:00
Jibing-Li
2b82bec406 [fix](statistics)Add row count to AnalysisJob and update tableStats using this row count after analyze. (#38428) 
Add row count to AnalysisJob and update tableStats using this row count
after analyze.
Take row count to consideration in stats health calculation.
 2024-07-27 11:32:29 +08:00
924060929
ee65195ba5 [enhancement](nereids) Speedup partition pruner (#38191) (#38405) 
1. fast return when partition predicate is true/false/null
2. fast compute table's hash code
3. fast merge two ranges when equals
 2024-07-26 15:19:32 +08:00
LiBinfeng
2f6b2dbdc4 [opt](Nereids) add where Null rule to create empty relation as where false (#38135) (#38361) 
pick from master #38135 

explain shape plan select * from table2 where Null; explain shape plan
select * from table2 where false; in this case, null literal can be
regard as same as false literal
 2024-07-26 14:50:06 +08:00
walter
df96db3f96 [fix](backup) Read BackupMeta with the specified meta version (#38396) 
Cherry-pick #38370
 2024-07-26 13:59:48 +08:00
Gabriel
c93f3bd24e [Improvement](bloom filter) Forbid small bloom filter (#38349) (#38392) 
Bloom filter has a expected filter ratio when data is enough. This PR
forbid too small bloom filter which has a big bias for filter ratio.

pick #38349
 2024-07-26 10:11:31 +08:00
Pxl
b4e82d2322 [Improvement](rpc) set grpc channel's keepAliveTime and remove proxy … (#38381) 
…on InterruptedExcep… (#37304)

## Proposed changes
1. set grpc channel's keepAliveTime
2. remove proxy on InterruptedException/TimeoutException to avoid
channel unavailable
pick from #37304
## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-25 22:11:23 +08:00
walter
a8da7fcdc6 [chore](backup) Log create restore job failed (#38341) 
Cherry-pick #38267
 2024-07-25 20:45:13 +08:00
Kaijie Chen
0152a4e86f [config](be) add be config migration_lock_timeout_ms (#38000) (#38337) 
backport #38000
 2024-07-25 17:36:34 +08:00
meiyi
91f0301b43 [fix](group commit) Pick some group commit pr (#38320) 
Pick https://github.com/apache/doris/pull/38292,
https://github.com/apache/doris/pull/34021,
https://github.com/apache/doris/pull/38228, some modify of
https://github.com/apache/doris/pull/37260, some modify of
https://github.com/apache/doris/pull/37595
 2024-07-25 17:32:44 +08:00
Kaijie Chen
54787f51c7 [log](load) print reason when LoadPathMgr failed to delete file (#38235) (#38340) 
backport #38235
 2024-07-25 17:28:09 +08:00
walter
3d6689d32c [fix](restore) Reset next version for restored partitions (#38343) 
Cherry-pick #38321
 2024-07-25 16:07:54 +08:00
abmdocrt
32530113cf [cherry-pick](branch-2.1) Fix some group commit forward to master problems (#38345) 
## Proposed changes

<!--Describe your changes.-->

Pick #38228 #38265
 2024-07-25 14:48:02 +08:00
Jerry Hu
4288f09e8c [fix](debug_point) Add debug point:reached_limit_early & return_empty_block (#38328) 
## Proposed changes

pick #38127 and #37629
 2024-07-25 14:46:40 +08:00
zfr95
d0a9e5656d [fix](auth)Add some log in auth case (#38289) 
pick from master #37362 

## Proposed changes

[fix](auth)Add some log in auth case
 2024-07-25 14:46:01 +08:00
Gabriel
a8139d9f39 [Improvement](profile) Provide more info for schedule time (#38290) (#38338) 
## Proposed changes

pick #38290

<!--Describe your changes.-->
 2024-07-25 14:43:52 +08:00
zzzxl
e2bb86e7f8 [fix](inverted index) fixed in_list condition not indexed on pipelinex (#38178) 
## Proposed changes

https://github.com/apache/doris/pull/36565
https://github.com/apache/doris/pull/37842
https://github.com/apache/doris/pull/37921
https://github.com/apache/doris/pull/37386

<!--Describe your changes.-->
 2024-07-25 14:42:34 +08:00
Qi Chen
a751372e76 [Feature](multi-catalog) Add memory tracker for orc reader/writer and arrow parquet writer。 (#37257) 
## Proposed changes

backport #37234
 2024-07-25 13:51:59 +08:00
lihangyu
73fc55b203 [Pick](Variant) fix some issue by RQG (#38336) 
#38318 
#38291
 2024-07-25 12:19:07 +08:00
AlexYue
e396f853a0 Pick "[enhance](Cooldown) Use config to control whether use cooldown replica for scanning first" (#38322) 
## Proposed changes

<!--Describe your changes.-->

Same as master #37492
 2024-07-25 12:17:38 +08:00
zhangstar333
70cde39fe0 [cherry-pick](branch-21) fix conv function get wrong result as parse overflow (#38001) (#38309) 
## Proposed changes

cherry-pick from https://github.com/apache/doris/pull/38001

<!--Describe your changes.-->
 2024-07-25 12:06:46 +08:00
zhangstar333
e9052e2180 [cherry-pick](branch-21) fix mod function cause core dump (#37999) (#38308) 
## Proposed changes
cherry-pick from master https://github.com/apache/doris/pull/37999

<!--Describe your changes.-->

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-25 12:06:21 +08:00