database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
19,601 Commits 1 Branch 0 Tags
a4e793752fccef8e49d98d6992df3ed352eb0942
Commit Graph

3155 Commits

Author SHA1 Message Date
HappenLee
28998300d4 [Bug](fix) fix ubsan use int32_t pointer access bool value (#38621) 
## Proposed changes

Issue Number: close #38617

<!--Describe your changes.-->
 2024-08-01 13:52:12 +08:00
wuwenchi
057ee1905f [bugfix](hudi)add timetravel for nereids for 2.1 (#38324) (#38582) 
## Proposed changes

bp #38324
 2024-08-01 11:37:57 +08:00
amory
338fa32303 [pick](simdjson) fix simdjson with object array when jsonroot is not empty (#38633) 
## Proposed changes
backport: https://github.com/apache/doris/pull/38490
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-01 11:04:54 +08:00
wuwenchi
41fa7bc9fd [bugfix](paimon)Fixed the reading of timestamp with time zone type data for 2.1 (#37716) (#38592) 
bp: #37716
 2024-08-01 10:23:06 +08:00
amory
184b8cbbe4 [pick](json)fix jsonb deseriaze (#38630) 
## Proposed changes
backport: https://github.com/apache/doris/pull/37251
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-01 10:18:27 +08:00
amory
6bd93b119f [pick](cast)Feature cast complexttype2 json (#38632) 
## Proposed changes
backport: https://github.com/apache/doris/pull/36548
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-01 09:18:15 +08:00
feiniaofeiafei
4c330e3fc6 [Fix](test) fix pull up literal predicate regression (#38564) 
## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-31 22:59:08 +08:00
Tiewei Fang
b21b906306 [Fix](outfile) FE check the hdfs URI of outfile (#38602) 
bp: #38203

1. Previously, if the root path of the HDFS URI started with two
slashes, the outfile would be successfully exported without errors, but
the exported path would not be the expected path.
Currently, we will delete repeated '/' which specified by users in FE.

2. move the test case for outfile HDFS from p2 to p0.
 2024-07-31 22:46:37 +08:00
wuwenchi
ef8a1918c3 [case][fix](iceberg)move rest cases from p2 to p0 and fix iceberg version issue for 2.1 (#37898) (#38589) 
bp: #37898
 2024-07-31 22:41:56 +08:00
Lijia Liu
96413e679d [branch-2.1](mtmv) Support read sync materialized view in async materialized view (#38462) 
## Proposed changes

pick #37396

<!--Describe your changes.-->

---------

Co-authored-by: liutang123 <liulijia@gmail.com>
 2024-07-31 22:32:28 +08:00
airborne12
66ebf709ba [Fix](inverted index) fix fast execute for not_in expr #37745 (#38594) 
cherry pick from #37745
 2024-07-31 19:58:12 +08:00
airborne12
7730aa2170 [Fix](inverted index) fix wrong no need read data when same column in inverted index and like function #36687 (#38581) 
cherry pick from #36687
 2024-07-31 19:41:39 +08:00
airborne12
7357d7bd3b [Update](inverted index) Add column name to debug point for "no need to read data" optimization #37649 (#38579) 
cherry pick from #37649
 2024-07-31 19:17:46 +08:00
lihangyu
aa9bdd76d0 [Pick](Variant) pick some fix #38413 #38364 (#38512) 2024-07-31 11:03:31 +08:00
camby
9d8b2e85ae [fix](partial-update) insert only without auto_inc column should not use partial update (#38229) (#38504) 
cherry-pick #38229 to branch-2.1


## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-31 11:01:08 +08:00
starocean999
9e696d72f1 [fix](nereids)check functionBuilders is not null before using it (#38535) 
## Proposed changes

pick from master https://github.com/apache/doris/pull/38457

<!--Describe your changes.-->
 2024-07-31 11:00:39 +08:00
feiniaofeiafei
94111da2a9 [Fix](nereids) fix normalize repeat alias rewrite (#38166) (#38454) 
cherry-pick #38166 to branch-2.1
 2024-07-31 10:59:15 +08:00
Mryange
017dad8c54 [fix](type)support runtime predicate for time type (#38258) (#38465) 
## Proposed changes
https://github.com/apache/doris/pull/38258
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-31 10:27:36 +08:00
LiBinfeng
742b98185c [test](Nereids) Add all hint tpcds tests (#38081) (#38368) 
cherry-pick:#38081
add all cases control by leading hint of tpcds
some cases are not supported because:
scalar/in/exists subquery: 6, 9, 28, 35, 41, 45, 58, 69, 95 set
operation to joins: 14, 23, 49, 56, 60, 66, 74, 75, 76, 77, 80, 87

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-31 10:24:10 +08:00
feiniaofeiafei
43ec98a30b [Feat](nereids) add pull up literal when infer predicates (#37314) (#38156) 
cherry-pick from master #37314
 2024-07-30 17:19:18 +08:00
LiBinfeng
2f6b2dbdc4 [opt](Nereids) add where Null rule to create empty relation as where false (#38135) (#38361) 
pick from master #38135 

explain shape plan select * from table2 where Null; explain shape plan
select * from table2 where false; in this case, null literal can be
regard as same as false literal
 2024-07-26 14:50:06 +08:00
zzzxl
e2bb86e7f8 [fix](inverted index) fixed in_list condition not indexed on pipelinex (#38178) 
## Proposed changes

https://github.com/apache/doris/pull/36565
https://github.com/apache/doris/pull/37842
https://github.com/apache/doris/pull/37921
https://github.com/apache/doris/pull/37386

<!--Describe your changes.-->
 2024-07-25 14:42:34 +08:00
lihangyu
73fc55b203 [Pick](Variant) fix some issue by RQG (#38336) 
#38318 
#38291
 2024-07-25 12:19:07 +08:00
zhangstar333
70cde39fe0 [cherry-pick](branch-21) fix conv function get wrong result as parse overflow (#38001) (#38309) 
## Proposed changes

cherry-pick from https://github.com/apache/doris/pull/38001

<!--Describe your changes.-->
 2024-07-25 12:06:46 +08:00
zhangstar333
e9052e2180 [cherry-pick](branch-21) fix mod function cause core dump (#37999) (#38308) 
## Proposed changes
cherry-pick from master https://github.com/apache/doris/pull/37999

<!--Describe your changes.-->

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-25 12:06:21 +08:00
zhangstar333
57864e8554 [cherry-pick](branch-21) fix collect_set function core dump without arena pool (#38234) (#38307) 
## Proposed changes

cherry-pick from master #38234

<!--Describe your changes.-->
 2024-07-25 12:05:52 +08:00
zclllhhjj
bc7fc4106d [branch-2.1](function) fix FE impl of some time functions (#37746) (#38316) 
pick https://github.com/apache/doris/pull/37746

before:
```sql
mysql> select date_ceil("2020-12-12 12:12:12.123", interval 2 second);
+-----------------------+
| '2020-12-12 12:12:12' |
+-----------------------+
| 2020-12-12 12:12:12   |
+-----------------------+
1 row in set (0.10 sec)

mysql> select CONVERT_TZ('9999-12-31 23:59:59.999999', 'Pacific/Galapagos', 'Pacific/Galapagos');
+------+
| NULL |
+------+
| NULL |
+------+
1 row in set (0.09 sec)

mysql [(none)]>select CONVERT_TZ('9999-12-31 23:59:59.999999', 'Pacific/Galapagos', 'Pacific/GalapaGoS');
+-----------------------------------------------------------------------------------------------------------+
| convert_tz(cast('9999-12-31 23:59:59.999999' as DATETIMEV2(6)), 'Pacific/Galapagos', 'Pacific/GalapaGoS') |
+-----------------------------------------------------------------------------------------------------------+
| 9999-12-31 23:59:59.999999                                                                                |
+-----------------------------------------------------------------------------------------------------------+
1 row in set (0.08 sec) --- gone to BE
```
after:
```sql
mysql> select date_ceil("2020-12-12 12:12:12.123", interval 2 second);
+------------------------------+
| '2020-12-12 12:12:14.000000' |
+------------------------------+
| 2020-12-12 12:12:14          |
+------------------------------+
1 row in set (0.11 sec)

mysql> select CONVERT_TZ('9999-12-31 23:59:59.999999', 'Pacific/Galapagos', 'Pacific/Galapagos');
+-----------------------------------------------------------------------------------------------------------+
| convert_tz(cast('9999-12-31 23:59:59.999999' as DATETIMEV2(6)), 'Pacific/Galapagos', 'Pacific/Galapagos') |
+-----------------------------------------------------------------------------------------------------------+
| 9999-12-31 23:59:59.999999                                                                                |
+-----------------------------------------------------------------------------------------------------------+
1 row in set (0.23 sec)

mysql> select CONVERT_TZ('9999-12-31 23:59:59.999999', 'Pacific/Galapagos', 'Pacific/GalapaGoS');
+------------------------------+
| '9999-12-31 23:59:59.999999' |
+------------------------------+
| 9999-12-31 23:59:59.999999   |
+------------------------------+
1 row in set (0.11 sec) --- finished in FE
```
 2024-07-25 11:38:27 +08:00
zclllhhjj
21b3fc3d1e [branch-2.1](function) fix coredump for MULTI_MATCH_ANY (#37959) (#38314) 
pick https://github.com/apache/doris/pull/37959

[INVALID_ARGUMENT][E33] Compile regexp expression failed. got Embedded
start anchors not supported.. some expressions may be illegal
 2024-07-25 11:34:22 +08:00
zclllhhjj
79a6496bb6 [branch-2.1](function) fix wrong result when convert_tz is out of bound (#37358) (#38313) 
## Proposed changes

pick https://github.com/apache/doris/pull/37358

before:
```sql
mysql> select CONVERT_TZ(cast('0000-01-01 00:00:00.00001'  as DATETIMEV1), cast('Asia/Shanghai' as VARCHAR(65533)), cast('America/Los_Angeles' as VARCHAR(65533)));
+---------------------------------------------------------------------------------------------------------------------------------------------------+
| convert_tz(cast('0000-01-01 00:00:00.00001' as DATETIME), cast('Asia/Shanghai' as VARCHAR(65533)), cast('America/Los_Angeles' as VARCHAR(65533))) |
+---------------------------------------------------------------------------------------------------------------------------------------------------+
| q535-12-31 08:01:19                                                                                                                               |
+---------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.12 sec)
```
now:
```sql
mysql> select CONVERT_TZ(cast('0000-01-01 00:00:00.00001'  as DATETIMEV1), cast('Asia/Shanghai' as VARCHAR(65533)), cast('America/Los_Angeles' as VARCHAR(65533)));
+---------------------------------------------------------------------------------------------------------------------------------------------------+
| convert_tz(cast('0000-01-01 00:00:00.00001' as DATETIME), cast('Asia/Shanghai' as VARCHAR(65533)), cast('America/Los_Angeles' as VARCHAR(65533))) |
+---------------------------------------------------------------------------------------------------------------------------------------------------+
| NULL                                                                                                                                              |
+---------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.09 sec)
```
 2024-07-25 11:32:44 +08:00
xzj7019
8ad4390edb [fix](nereids) refine row count estimation for mark join (#38270) (#38297) 
pick from master #38270
 2024-07-25 10:19:13 +08:00
meiyi
e23c1339a8 [fix](group commit) Fix prepare stmt setNull return too many filtered rows error (#38262) (#38276) 
Pick https://github.com/apache/doris/pull/38262
 2024-07-24 19:02:59 +08:00
morrySnow
17c33665e5 [fix](agg_state) adjust nullable should apply on agg_state inner type too (#37489) (#38281) 
pick from master #37489

after adjust nullable, some children nullable has changed. so, we need
to update agg_state type inner type nullable too.
 2024-07-24 13:55:09 +08:00
Jensen
ca7d8325af [Fix](Nereids) TopN should forbid two-phase topN if child is DistributionSpecGather (#36877) (#38164) 
pick from master #36877
 2024-07-23 16:27:55 +08:00
daidai
193be20c86 [feature](csv)Supports reading CSV data using LF and CRLF as line separators. (#37687) (#38099) 
bp #37687
 2024-07-22 22:53:04 +08:00
lihangyu
d9fd419e47 [Fix](JsonReader) fix json with duplicate key entry may result out of bound exception (#38147) 
#38146
 2024-07-19 22:53:02 +08:00
amory
22b9cc6eb2 [fix](mysql)fix mysql row buf (#38145) 
## Proposed changes
backport: https://github.com/apache/doris/pull/37936
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-19 22:18:42 +08:00
Mingyu Chen
bb2b7774df [feature](iceberg) iceberg write support insert overwrite and optimize hive write transaction statistics and (#37191) (#38097) 
bp #37191

Co-authored-by: kang <35803862+ghkang98@users.noreply.github.com>
Co-authored-by: lik40 <lik40@chinatelecom.cn>
 2024-07-19 09:45:41 +08:00
feiniaofeiafei
301ff6af22 [Fix](nereids) add backquote and qualifier to udf, column, column alias and table alias when create view (#37237) (#37984) 
cherry-pick #37237 to branch-2.1

---------

Co-authored-by: feiniaofeiafei <moailing@selectdb.com>
 2024-07-19 00:56:26 +08:00
camby
de2272ce48 [fix](round) fix round decimal128 overflow (#37733) (#37963) 
cherry-pick #37733 to branch-2.1
 2024-07-18 23:50:23 +08:00
lihangyu
8c6ff22e04 [Pick](Variant) fix heap use after free and optimize cases #37991 #37976 (#38037) 2024-07-18 16:53:09 +08:00
bobhan1
c30c1d2436 [branch-2.1] Picks "[opt](delete) Delete job should retry for failure that is not DELETE_INVALID_XXX #37834" (#38032) 
## Proposed changes

picks https://github.com/apache/doris/pull/37834 and
https://github.com/apache/doris/pull/38043
 2024-07-18 14:50:30 +08:00
starocean999
ebc178af23 [fix](nereids)acos function should return null literal instead of NaN value (#37996) 
pick from master https://github.com/apache/doris/pull/37932

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-18 09:28:56 +08:00
morrySnow
78d51ca320 [fix](Nereids) tablet prune wrong when decimal value scale is nagtive (#37889) (#38013) 
pick from master #37889

we use unscaled value of BigDecimal in tablet prune. So we need to
ensure BigDecimal's precision and scale is same with the literal
contains it.
 2024-07-17 20:12:14 +08:00
starocean999
b2a4cff51b [fix](nereids)fix nullable property of ForEachCombinator (#37980) 
## Proposed changes

pick from master https://github.com/apache/doris/pull/37796

<!--Describe your changes.-->
 2024-07-17 13:48:21 +08:00
Mingyu Chen
ba66ff5768 [fix](multi-catalog)fix paimon meta properties convert (#37249) (#37958) 
bp #37249

Co-authored-by: slothever <18522955+wsjz@users.noreply.github.com>
 2024-07-17 01:04:33 +08:00
HappenLee
b6e5281a1c [Fix](bug) fix the divide zero in local shuffle: (#37948) 
## Proposed changes

cherry pick #37906 

<!--Describe your changes.-->
 2024-07-17 01:03:53 +08:00
amory
21c6b854f7 [fix](explode-json-object)fix explode json object (#37956) 
## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-17 01:03:07 +08:00
苏小刚
f7068b5658 [cherry-pick](branch-2.1) Make doris read hive text table parameters and behavior consistent with hive (#37840) 
## Proposed changes

pick from master https://github.com/apache/doris/pull/37638

<!--Describe your changes.-->
 2024-07-16 22:24:50 +08:00
lihangyu
b15ccdbe98 [Pick](Variant) pick some fix (#37922) 
#37674
#37839
#37883 
#37857 
#37794
 2024-07-16 21:38:47 +08:00
caiconghui
9ff129b630 [fix](stream_load) fix stream load may failed caused by column name with keyword (#35822) (#37890) 
#35938 #35822 
let 
KW_SQL,
KW_CACHE,
KW_COLOCATE,
KW_COMPRESS_TYPE,
KW_DORIS_INTERNAL_TABLE_ID,
KW_HOTSPOT,
KW_PRIVILEGES,
KW_RECENT,
KW_STAGES,
KW_WARM,
KW_UP,
KW_CONVERT_LSC
be as non-reserved

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->

---------

Co-authored-by: caiconghui1 <caiconghui1@jd.com>
 2024-07-16 20:20:34 +08:00