database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
19,601 Commits 1 Branch 0 Tags
a4e793752fccef8e49d98d6992df3ed352eb0942
Commit Graph

8877 Commits

Author SHA1 Message Date
wuwenchi
a4e793752f [bugfix](iceberg)revert count(*) directly returned by fe for 2.1 (#38566) (#38655) 
bp: #38566
 2024-08-01 18:56:19 +08:00
meiyi
e8690b62ee [fix](group commit) Pick add debug log show why group commit not work; delete wal when replay success (#38611) (#38659) 
Pick https://github.com/apache/doris/pull/38611
 2024-08-01 16:59:54 +08:00
Siyang Tang
cafcf7acc1 [cherry-pick](SSL) Fix ssl connection close 2.1 (#38587) (#38677) 
## Proposed changes

Issue Number: close #38590 

If SSL connection closed, a specified packet will sent to indicate the
closing of connection. The SSL engine will be shut down and output an
empty unwrapped result.

Therefore, handle this case correctly to avoid buffer overflow by
breaking the reading flow and do the cleanup stuff initiatively.
 2024-08-01 16:06:30 +08:00
slothever
b0943064e0 [fix](kerberos)fix and refactor ugi login for kerberos and simple authentication (#38607) 
pick from  (#37301)
 2024-08-01 14:01:32 +08:00
wuwenchi
c310b92727 [bugfix](external)Modify the default value of pushdowncount for 2.1 (#37754) (#38591) 
bp: #37754
 2024-08-01 12:10:54 +08:00
wuwenchi
057ee1905f [bugfix](hudi)add timetravel for nereids for 2.1 (#38324) (#38582) 
## Proposed changes

bp #38324
 2024-08-01 11:37:57 +08:00
wuwenchi
41fa7bc9fd [bugfix](paimon)Fixed the reading of timestamp with time zone type data for 2.1 (#37716) (#38592) 
bp: #37716
 2024-08-01 10:23:06 +08:00
abmdocrt
9f1e41c623 [Cherry-pick](branch-2.1) Pick "[Enhancement](audit log) Add print audit log sesssion variable #38419" (#38624) 
## Proposed changes

<!--Describe your changes.-->

Pick #38419
 2024-08-01 10:19:19 +08:00
amory
6bd93b119f [pick](cast)Feature cast complexttype2 json (#38632) 
## Proposed changes
backport: https://github.com/apache/doris/pull/36548
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-01 09:18:15 +08:00
zhangdong
098ac67a75 [fix](catalog)fix db name may be null in NotificationEvent (#38421) (#38596) 
pick: https://github.com/apache/doris/pull/38421
 2024-07-31 22:56:44 +08:00
Tiewei Fang
b21b906306 [Fix](outfile) FE check the hdfs URI of outfile (#38602) 
bp: #38203

1. Previously, if the root path of the HDFS URI started with two
slashes, the outfile would be successfully exported without errors, but
the exported path would not be the expected path.
Currently, we will delete repeated '/' which specified by users in FE.

2. move the test case for outfile HDFS from p2 to p0.
 2024-07-31 22:46:37 +08:00
wuwenchi
ef8a1918c3 [case][fix](iceberg)move rest cases from p2 to p0 and fix iceberg version issue for 2.1 (#37898) (#38589) 
bp: #37898
 2024-07-31 22:41:56 +08:00
wuwenchi
1c176db010 [bugfix](paimon)add support for 'in' and 'not in' for 2.1 #38390 (#38576) 
## Proposed changes

bp: #38390
 2024-07-31 22:38:27 +08:00
zhangdong
46dfb65b6b [fix](auth)fix show routine load db is null (#38574) 
pick: https://github.com/apache/doris/pull/38365
 2024-07-31 22:34:35 +08:00
Lijia Liu
96413e679d [branch-2.1](mtmv) Support read sync materialized view in async materialized view (#38462) 
## Proposed changes

pick #37396

<!--Describe your changes.-->

---------

Co-authored-by: liutang123 <liulijia@gmail.com>
 2024-07-31 22:32:28 +08:00
Jibing-Li
002232b2da [improvement](statistics)Analyze all columns when partition first loaded. (#38606) 
backport: https://github.com/apache/doris/pull/38540
 2024-07-31 19:59:41 +08:00
airborne12
a75511ae08 [Feature](inverted index) add no need read data optimize config (#38584) 
pick from #36686
 2024-07-31 19:39:17 +08:00
谢健
e26f2441bd [opt](Nereids): disable PRUNE_EMPTY_PARTITION rule in SqlTestBase.java #38246 (#38455) 
## Proposed changes

#38246

Co-authored-by: zclllhhjj <zhaochangle@selectdb.com>
 2024-07-31 14:34:35 +08:00
camby
9d8b2e85ae [fix](partial-update) insert only without auto_inc column should not use partial update (#38229) (#38504) 
cherry-pick #38229 to branch-2.1


## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-31 11:01:08 +08:00
starocean999
9e696d72f1 [fix](nereids)check functionBuilders is not null before using it (#38535) 
## Proposed changes

pick from master https://github.com/apache/doris/pull/38457

<!--Describe your changes.-->
 2024-07-31 11:00:39 +08:00
feiniaofeiafei
94111da2a9 [Fix](nereids) fix normalize repeat alias rewrite (#38166) (#38454) 
cherry-pick #38166 to branch-2.1
 2024-07-31 10:59:15 +08:00
walter
182bf4d323 [chore](fe) Returns dropped tables in GetMeta request (#38541) 
Cherry-pick #38019
 2024-07-31 10:57:00 +08:00
zhangdong
319933d0a6 [enhance](mtmv)mtmv date trunc support hour (#37678) (#38422) 
pick: https://github.com/apache/doris/pull/37678
 2024-07-31 10:46:08 +08:00
Jibing-Li
c78e04ac31 [improvement](statistics)Support get index row count and table delta rows. (#38520) 
backport: https://github.com/apache/doris/pull/38492
 2024-07-31 10:39:21 +08:00
zhangdong
d9d7f8de95 [fix](auth) fix create mv not need alter priv of table (#38011) (#38423) 
pick: https://github.com/apache/doris/pull/38011
 2024-07-31 10:35:03 +08:00
924060929
a328e01d97 [fix](nereids) fix insert stmt throw MultiCastDataSink cannot be cast to DataStreamSink (#38526) (#38547) 
fix `insert ... with ... select ...`, which not use some cte, and throw an exception:
```
errCode = 2, detailMessage = class org.apache.doris.planner.MultiCastDataSink cannot be cast to class org.apache.doris.planner.DataStreamSink (org.apache.doris.planner.MultiCastDataSink and org.apache.doris.planner.DataStreamSink are in unnamed module of loader 'app')
```
 2024-07-31 00:47:30 +08:00
924060929
1ba1e343bd [fix](nereids) fix The children format needs to be [WhenClause+, DefaultValue?] (#38491) (#38546) 
fix The children format needs to be [WhenClause+, DefaultValue?]
 2024-07-30 23:51:18 +08:00
morrySnow
e25eb733c1 [opt](compatibility) fill schema and table name in protocol column def (#38126) (#38522) 
pick from master #38126

according to mysql text protocol Protocol::ColumnDefinition41, should
fill schema name, table name into it if column from table.
 2024-07-30 17:41:07 +08:00
feiniaofeiafei
43ec98a30b [Feat](nereids) add pull up literal when infer predicates (#37314) (#38156) 
cherry-pick from master #37314
 2024-07-30 17:19:18 +08:00
minghong
377d4615c7 [feat](FE) add rest api to get last profile (#38268) (#38451) 
## Proposed changes
pick from #38268

if you want to get the profile of the query just executed, it not
neccessary to find the query_id now.
this is useful to get profiles in shell script, for example, we want to
get all profiles of tpcds queries (103 query profile) in a shell script.

before:
curl

http://root:@127.0.0.1:8030/api/profile/text?query_id=990cc3b19e464e03-a3af05a997eb4091
> profile
after:
curl http://root:@127.0.0.1:8030/api/profile/text > profilex

Issue Number: close #xxx

<!--Describe your changes.-->

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-30 13:36:43 +08:00
zclllhhjj
dab0138567 [branch-2.1](insert-overwrite) Fix insert overwrite auto detect transaction safe (#38103) (#38442) 
pick https://github.com/apache/doris/pull/38103
 2024-07-29 10:21:03 +08:00
Jibing-Li
2b82bec406 [fix](statistics)Add row count to AnalysisJob and update tableStats using this row count after analyze. (#38428) 
Add row count to AnalysisJob and update tableStats using this row count
after analyze.
Take row count to consideration in stats health calculation.
 2024-07-27 11:32:29 +08:00
924060929
ee65195ba5 [enhancement](nereids) Speedup partition pruner (#38191) (#38405) 
1. fast return when partition predicate is true/false/null
2. fast compute table's hash code
3. fast merge two ranges when equals
 2024-07-26 15:19:32 +08:00
LiBinfeng
2f6b2dbdc4 [opt](Nereids) add where Null rule to create empty relation as where false (#38135) (#38361) 
pick from master #38135 

explain shape plan select * from table2 where Null; explain shape plan
select * from table2 where false; in this case, null literal can be
regard as same as false literal
 2024-07-26 14:50:06 +08:00
walter
df96db3f96 [fix](backup) Read BackupMeta with the specified meta version (#38396) 
Cherry-pick #38370
 2024-07-26 13:59:48 +08:00
Gabriel
c93f3bd24e [Improvement](bloom filter) Forbid small bloom filter (#38349) (#38392) 
Bloom filter has a expected filter ratio when data is enough. This PR
forbid too small bloom filter which has a big bias for filter ratio.

pick #38349
 2024-07-26 10:11:31 +08:00
Pxl
b4e82d2322 [Improvement](rpc) set grpc channel's keepAliveTime and remove proxy … (#38381) 
…on InterruptedExcep… (#37304)

## Proposed changes
1. set grpc channel's keepAliveTime
2. remove proxy on InterruptedException/TimeoutException to avoid
channel unavailable
pick from #37304
## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-25 22:11:23 +08:00
walter
a8da7fcdc6 [chore](backup) Log create restore job failed (#38341) 
Cherry-pick #38267
 2024-07-25 20:45:13 +08:00
meiyi
91f0301b43 [fix](group commit) Pick some group commit pr (#38320) 
Pick https://github.com/apache/doris/pull/38292,
https://github.com/apache/doris/pull/34021,
https://github.com/apache/doris/pull/38228, some modify of
https://github.com/apache/doris/pull/37260, some modify of
https://github.com/apache/doris/pull/37595
 2024-07-25 17:32:44 +08:00
walter
3d6689d32c [fix](restore) Reset next version for restored partitions (#38343) 
Cherry-pick #38321
 2024-07-25 16:07:54 +08:00
abmdocrt
32530113cf [cherry-pick](branch-2.1) Fix some group commit forward to master problems (#38345) 
## Proposed changes

<!--Describe your changes.-->

Pick #38228 #38265
 2024-07-25 14:48:02 +08:00
Gabriel
a8139d9f39 [Improvement](profile) Provide more info for schedule time (#38290) (#38338) 
## Proposed changes

pick #38290

<!--Describe your changes.-->
 2024-07-25 14:43:52 +08:00
AlexYue
e396f853a0 Pick "[enhance](Cooldown) Use config to control whether use cooldown replica for scanning first" (#38322) 
## Proposed changes

<!--Describe your changes.-->

Same as master #37492
 2024-07-25 12:17:38 +08:00
Yulei-Yang
c23228e331 [Improvement](profile) add catalog info in profile (#38302) 
bp #38283
 2024-07-25 11:57:42 +08:00
zclllhhjj
bc7fc4106d [branch-2.1](function) fix FE impl of some time functions (#37746) (#38316) 
pick https://github.com/apache/doris/pull/37746

before:
```sql
mysql> select date_ceil("2020-12-12 12:12:12.123", interval 2 second);
+-----------------------+
| '2020-12-12 12:12:12' |
+-----------------------+
| 2020-12-12 12:12:12   |
+-----------------------+
1 row in set (0.10 sec)

mysql> select CONVERT_TZ('9999-12-31 23:59:59.999999', 'Pacific/Galapagos', 'Pacific/Galapagos');
+------+
| NULL |
+------+
| NULL |
+------+
1 row in set (0.09 sec)

mysql [(none)]>select CONVERT_TZ('9999-12-31 23:59:59.999999', 'Pacific/Galapagos', 'Pacific/GalapaGoS');
+-----------------------------------------------------------------------------------------------------------+
| convert_tz(cast('9999-12-31 23:59:59.999999' as DATETIMEV2(6)), 'Pacific/Galapagos', 'Pacific/GalapaGoS') |
+-----------------------------------------------------------------------------------------------------------+
| 9999-12-31 23:59:59.999999                                                                                |
+-----------------------------------------------------------------------------------------------------------+
1 row in set (0.08 sec) --- gone to BE
```
after:
```sql
mysql> select date_ceil("2020-12-12 12:12:12.123", interval 2 second);
+------------------------------+
| '2020-12-12 12:12:14.000000' |
+------------------------------+
| 2020-12-12 12:12:14          |
+------------------------------+
1 row in set (0.11 sec)

mysql> select CONVERT_TZ('9999-12-31 23:59:59.999999', 'Pacific/Galapagos', 'Pacific/Galapagos');
+-----------------------------------------------------------------------------------------------------------+
| convert_tz(cast('9999-12-31 23:59:59.999999' as DATETIMEV2(6)), 'Pacific/Galapagos', 'Pacific/Galapagos') |
+-----------------------------------------------------------------------------------------------------------+
| 9999-12-31 23:59:59.999999                                                                                |
+-----------------------------------------------------------------------------------------------------------+
1 row in set (0.23 sec)

mysql> select CONVERT_TZ('9999-12-31 23:59:59.999999', 'Pacific/Galapagos', 'Pacific/GalapaGoS');
+------------------------------+
| '9999-12-31 23:59:59.999999' |
+------------------------------+
| 9999-12-31 23:59:59.999999   |
+------------------------------+
1 row in set (0.11 sec) --- finished in FE
```
 2024-07-25 11:38:27 +08:00
xzj7019
8ad4390edb [fix](nereids) refine row count estimation for mark join (#38270) (#38297) 
pick from master #38270
 2024-07-25 10:19:13 +08:00
meiyi
e23c1339a8 [fix](group commit) Fix prepare stmt setNull return too many filtered rows error (#38262) (#38276) 
Pick https://github.com/apache/doris/pull/38262
 2024-07-24 19:02:59 +08:00
morrySnow
17c33665e5 [fix](agg_state) adjust nullable should apply on agg_state inner type too (#37489) (#38281) 
pick from master #37489

after adjust nullable, some children nullable has changed. so, we need
to update agg_state type inner type nullable too.
 2024-07-24 13:55:09 +08:00
abmdocrt
792bd7c74a [cherry-pick](branch-2.1) Pick "[Enhancement](group commit)Optimize be select for group commit #35558" (#37830) 
Pick #35558
 2024-07-24 09:21:07 +08:00
yujun
798b8573c9 [fix](load data) decommission replica don't load data when it misses versions #38198 (#38256) 
cherry pick from #38198
 2024-07-23 20:28:05 +08:00