database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
19,577 Commits 1 Branch 0 Tags
46dfb65b6b7df2b8df7a2575c6a1379135c89ed5
Commit Graph

7327 Commits

Author SHA1 Message Date
zhangdong
46dfb65b6b [fix](auth)fix show routine load db is null (#38574) 
pick: https://github.com/apache/doris/pull/38365
 2024-07-31 22:34:35 +08:00
Lijia Liu
96413e679d [branch-2.1](mtmv) Support read sync materialized view in async materialized view (#38462) 
## Proposed changes

pick #37396

<!--Describe your changes.-->

---------

Co-authored-by: liutang123 <liulijia@gmail.com>
 2024-07-31 22:32:28 +08:00
Jibing-Li
002232b2da [improvement](statistics)Analyze all columns when partition first loaded. (#38606) 
backport: https://github.com/apache/doris/pull/38540
 2024-07-31 19:59:41 +08:00
airborne12
a75511ae08 [Feature](inverted index) add no need read data optimize config (#38584) 
pick from #36686
 2024-07-31 19:39:17 +08:00
camby
9d8b2e85ae [fix](partial-update) insert only without auto_inc column should not use partial update (#38229) (#38504) 
cherry-pick #38229 to branch-2.1


## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-31 11:01:08 +08:00
starocean999
9e696d72f1 [fix](nereids)check functionBuilders is not null before using it (#38535) 
## Proposed changes

pick from master https://github.com/apache/doris/pull/38457

<!--Describe your changes.-->
 2024-07-31 11:00:39 +08:00
feiniaofeiafei
94111da2a9 [Fix](nereids) fix normalize repeat alias rewrite (#38166) (#38454) 
cherry-pick #38166 to branch-2.1
 2024-07-31 10:59:15 +08:00
walter
182bf4d323 [chore](fe) Returns dropped tables in GetMeta request (#38541) 
Cherry-pick #38019
 2024-07-31 10:57:00 +08:00
zhangdong
319933d0a6 [enhance](mtmv)mtmv date trunc support hour (#37678) (#38422) 
pick: https://github.com/apache/doris/pull/37678
 2024-07-31 10:46:08 +08:00
Jibing-Li
c78e04ac31 [improvement](statistics)Support get index row count and table delta rows. (#38520) 
backport: https://github.com/apache/doris/pull/38492
 2024-07-31 10:39:21 +08:00
zhangdong
d9d7f8de95 [fix](auth) fix create mv not need alter priv of table (#38011) (#38423) 
pick: https://github.com/apache/doris/pull/38011
 2024-07-31 10:35:03 +08:00
924060929
a328e01d97 [fix](nereids) fix insert stmt throw MultiCastDataSink cannot be cast to DataStreamSink (#38526) (#38547) 
fix `insert ... with ... select ...`, which not use some cte, and throw an exception:
```
errCode = 2, detailMessage = class org.apache.doris.planner.MultiCastDataSink cannot be cast to class org.apache.doris.planner.DataStreamSink (org.apache.doris.planner.MultiCastDataSink and org.apache.doris.planner.DataStreamSink are in unnamed module of loader 'app')
```
 2024-07-31 00:47:30 +08:00
924060929
1ba1e343bd [fix](nereids) fix The children format needs to be [WhenClause+, DefaultValue?] (#38491) (#38546) 
fix The children format needs to be [WhenClause+, DefaultValue?]
 2024-07-30 23:51:18 +08:00
morrySnow
e25eb733c1 [opt](compatibility) fill schema and table name in protocol column def (#38126) (#38522) 
pick from master #38126

according to mysql text protocol Protocol::ColumnDefinition41, should
fill schema name, table name into it if column from table.
 2024-07-30 17:41:07 +08:00
feiniaofeiafei
43ec98a30b [Feat](nereids) add pull up literal when infer predicates (#37314) (#38156) 
cherry-pick from master #37314
 2024-07-30 17:19:18 +08:00
minghong
377d4615c7 [feat](FE) add rest api to get last profile (#38268) (#38451) 
## Proposed changes
pick from #38268

if you want to get the profile of the query just executed, it not
neccessary to find the query_id now.
this is useful to get profiles in shell script, for example, we want to
get all profiles of tpcds queries (103 query profile) in a shell script.

before:
curl

http://root:@127.0.0.1:8030/api/profile/text?query_id=990cc3b19e464e03-a3af05a997eb4091
> profile
after:
curl http://root:@127.0.0.1:8030/api/profile/text > profilex

Issue Number: close #xxx

<!--Describe your changes.-->

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-30 13:36:43 +08:00
zclllhhjj
dab0138567 [branch-2.1](insert-overwrite) Fix insert overwrite auto detect transaction safe (#38103) (#38442) 
pick https://github.com/apache/doris/pull/38103
 2024-07-29 10:21:03 +08:00
Jibing-Li
2b82bec406 [fix](statistics)Add row count to AnalysisJob and update tableStats using this row count after analyze. (#38428) 
Add row count to AnalysisJob and update tableStats using this row count
after analyze.
Take row count to consideration in stats health calculation.
 2024-07-27 11:32:29 +08:00
924060929
ee65195ba5 [enhancement](nereids) Speedup partition pruner (#38191) (#38405) 
1. fast return when partition predicate is true/false/null
2. fast compute table's hash code
3. fast merge two ranges when equals
 2024-07-26 15:19:32 +08:00
LiBinfeng
2f6b2dbdc4 [opt](Nereids) add where Null rule to create empty relation as where false (#38135) (#38361) 
pick from master #38135 

explain shape plan select * from table2 where Null; explain shape plan
select * from table2 where false; in this case, null literal can be
regard as same as false literal
 2024-07-26 14:50:06 +08:00
walter
df96db3f96 [fix](backup) Read BackupMeta with the specified meta version (#38396) 
Cherry-pick #38370
 2024-07-26 13:59:48 +08:00
Gabriel
c93f3bd24e [Improvement](bloom filter) Forbid small bloom filter (#38349) (#38392) 
Bloom filter has a expected filter ratio when data is enough. This PR
forbid too small bloom filter which has a big bias for filter ratio.

pick #38349
 2024-07-26 10:11:31 +08:00
Pxl
b4e82d2322 [Improvement](rpc) set grpc channel's keepAliveTime and remove proxy … (#38381) 
…on InterruptedExcep… (#37304)

## Proposed changes
1. set grpc channel's keepAliveTime
2. remove proxy on InterruptedException/TimeoutException to avoid
channel unavailable
pick from #37304
## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-25 22:11:23 +08:00
walter
a8da7fcdc6 [chore](backup) Log create restore job failed (#38341) 
Cherry-pick #38267
 2024-07-25 20:45:13 +08:00
meiyi
91f0301b43 [fix](group commit) Pick some group commit pr (#38320) 
Pick https://github.com/apache/doris/pull/38292,
https://github.com/apache/doris/pull/34021,
https://github.com/apache/doris/pull/38228, some modify of
https://github.com/apache/doris/pull/37260, some modify of
https://github.com/apache/doris/pull/37595
 2024-07-25 17:32:44 +08:00
walter
3d6689d32c [fix](restore) Reset next version for restored partitions (#38343) 
Cherry-pick #38321
 2024-07-25 16:07:54 +08:00
abmdocrt
32530113cf [cherry-pick](branch-2.1) Fix some group commit forward to master problems (#38345) 
## Proposed changes

<!--Describe your changes.-->

Pick #38228 #38265
 2024-07-25 14:48:02 +08:00
Gabriel
a8139d9f39 [Improvement](profile) Provide more info for schedule time (#38290) (#38338) 
## Proposed changes

pick #38290

<!--Describe your changes.-->
 2024-07-25 14:43:52 +08:00
AlexYue
e396f853a0 Pick "[enhance](Cooldown) Use config to control whether use cooldown replica for scanning first" (#38322) 
## Proposed changes

<!--Describe your changes.-->

Same as master #37492
 2024-07-25 12:17:38 +08:00
Yulei-Yang
c23228e331 [Improvement](profile) add catalog info in profile (#38302) 
bp #38283
 2024-07-25 11:57:42 +08:00
zclllhhjj
bc7fc4106d [branch-2.1](function) fix FE impl of some time functions (#37746) (#38316) 
pick https://github.com/apache/doris/pull/37746

before:
```sql
mysql> select date_ceil("2020-12-12 12:12:12.123", interval 2 second);
+-----------------------+
| '2020-12-12 12:12:12' |
+-----------------------+
| 2020-12-12 12:12:12   |
+-----------------------+
1 row in set (0.10 sec)

mysql> select CONVERT_TZ('9999-12-31 23:59:59.999999', 'Pacific/Galapagos', 'Pacific/Galapagos');
+------+
| NULL |
+------+
| NULL |
+------+
1 row in set (0.09 sec)

mysql [(none)]>select CONVERT_TZ('9999-12-31 23:59:59.999999', 'Pacific/Galapagos', 'Pacific/GalapaGoS');
+-----------------------------------------------------------------------------------------------------------+
| convert_tz(cast('9999-12-31 23:59:59.999999' as DATETIMEV2(6)), 'Pacific/Galapagos', 'Pacific/GalapaGoS') |
+-----------------------------------------------------------------------------------------------------------+
| 9999-12-31 23:59:59.999999                                                                                |
+-----------------------------------------------------------------------------------------------------------+
1 row in set (0.08 sec) --- gone to BE
```
after:
```sql
mysql> select date_ceil("2020-12-12 12:12:12.123", interval 2 second);
+------------------------------+
| '2020-12-12 12:12:14.000000' |
+------------------------------+
| 2020-12-12 12:12:14          |
+------------------------------+
1 row in set (0.11 sec)

mysql> select CONVERT_TZ('9999-12-31 23:59:59.999999', 'Pacific/Galapagos', 'Pacific/Galapagos');
+-----------------------------------------------------------------------------------------------------------+
| convert_tz(cast('9999-12-31 23:59:59.999999' as DATETIMEV2(6)), 'Pacific/Galapagos', 'Pacific/Galapagos') |
+-----------------------------------------------------------------------------------------------------------+
| 9999-12-31 23:59:59.999999                                                                                |
+-----------------------------------------------------------------------------------------------------------+
1 row in set (0.23 sec)

mysql> select CONVERT_TZ('9999-12-31 23:59:59.999999', 'Pacific/Galapagos', 'Pacific/GalapaGoS');
+------------------------------+
| '9999-12-31 23:59:59.999999' |
+------------------------------+
| 9999-12-31 23:59:59.999999   |
+------------------------------+
1 row in set (0.11 sec) --- finished in FE
```
 2024-07-25 11:38:27 +08:00
xzj7019
8ad4390edb [fix](nereids) refine row count estimation for mark join (#38270) (#38297) 
pick from master #38270
 2024-07-25 10:19:13 +08:00
meiyi
e23c1339a8 [fix](group commit) Fix prepare stmt setNull return too many filtered rows error (#38262) (#38276) 
Pick https://github.com/apache/doris/pull/38262
 2024-07-24 19:02:59 +08:00
morrySnow
17c33665e5 [fix](agg_state) adjust nullable should apply on agg_state inner type too (#37489) (#38281) 
pick from master #37489

after adjust nullable, some children nullable has changed. so, we need
to update agg_state type inner type nullable too.
 2024-07-24 13:55:09 +08:00
abmdocrt
792bd7c74a [cherry-pick](branch-2.1) Pick "[Enhancement](group commit)Optimize be select for group commit #35558" (#37830) 
Pick #35558
 2024-07-24 09:21:07 +08:00
yujun
798b8573c9 [fix](load data) decommission replica don't load data when it misses versions #38198 (#38256) 
cherry pick from #38198
 2024-07-23 20:28:05 +08:00
meiyi
b591837e13 [fix](insert) Pick Insert init plan error may forget to abort txn (#34220) (#38260) 
Pick https://github.com/apache/doris/pull/34220
 2024-07-23 20:21:38 +08:00
Jensen
ca7d8325af [Fix](Nereids) TopN should forbid two-phase topN if child is DistributionSpecGather (#36877) (#38164) 
pick from master #36877
 2024-07-23 16:27:55 +08:00
Siyang Tang
ba5d23aea8 [enhancement](log) Changed log of waiting previous transactions to info level for debug convenience (#38133) (#38230) 
## Proposed changes

As title.
 2024-07-23 14:16:04 +08:00
Gabriel
33e04b12ff [conf](parallel) Reduce parallel tasks for large cluster (#38196) (#38233) 
For large cluster, too many parallel tasks will cause performance issue.
So this PR limit the max parallel tasks in Doris.

pick #38196

<!--Describe your changes.-->
 2024-07-23 13:42:31 +08:00
walter
fdc8b454d2 [chore](backup) Remove delete_if_exists properties for creating repository (#38192) 
delete_if_exists is a temporary solution introduced in #25847, to avoid
concurrent testing conflicts.

Cherry-pick #38190
 2024-07-23 11:09:02 +08:00
Yulei-Yang
41b355721f [fix](multicatalog) make lastdbofcatalog a session variable (#38114) 
bp #37828
 2024-07-23 10:34:40 +08:00
daidai
193be20c86 [feature](csv)Supports reading CSV data using LF and CRLF as line separators. (#37687) (#38099) 
bp #37687
 2024-07-22 22:53:04 +08:00
Calvin Kirs
87306858a1 [Fix](JobManager)Release the lock immediately after modifying job metadata to avoid holding the lock for an extended period.#38162 (#38163) 
## Proposed changes
#38162

<!--Describe your changes.-->
 2024-07-20 10:52:21 +08:00
zy-kkk
b24774c340 [branch-2.1][fix](compatibility) Enhance SSL and Long Flag Handling in MySQL Handshake (#38140) 
pick(#38086)
Issue Number: close #38065

This PR introduces a fix to the SSL and CLIENT_LONG_FLAG handling in the
MySQL handshake process. It ensures that the flags are correctly set
according to the protocol requirements and resolves the issues related
to decimal data fetching when SSL is enabled.
 2024-07-19 22:50:51 +08:00
qiye
dfc82a4011 [fix](inverted index)Support rename column with inverted index (#38079) (#38121) 
## Proposed changes

backport #38079
 2024-07-19 17:58:49 +08:00
Mingyu Chen
27d7461644 [Opt](Iceberg) Simplify the code of getting time travel snapshotId (#34299) (#38101) 
bp #34299

Co-authored-by: Butao Zhang <zhangbutao@cmss.chinamobile.com>
 2024-07-19 09:45:56 +08:00
Mingyu Chen
bb2b7774df [feature](iceberg) iceberg write support insert overwrite and optimize hive write transaction statistics and (#37191) (#38097) 
bp #37191

Co-authored-by: kang <35803862+ghkang98@users.noreply.github.com>
Co-authored-by: lik40 <lik40@chinatelecom.cn>
 2024-07-19 09:45:41 +08:00
feiniaofeiafei
301ff6af22 [Fix](nereids) add backquote and qualifier to udf, column, column alias and table alias when create view (#37237) (#37984) 
cherry-pick #37237 to branch-2.1

---------

Co-authored-by: feiniaofeiafei <moailing@selectdb.com>
 2024-07-19 00:56:26 +08:00
zhangdong
5d17ff9d95 [fix](mtmv)Mtmv support force replica 2.1 (#38090) 
master do not have bug, because has call
`PropertyAnalyzer.getInstance().rewriteOlapProperties`

2.1 not have this method ,should call
`PropertyAnalyzer.rewriteReplicaAllocationProperties`
 2024-07-18 23:52:34 +08:00