database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
19,346 Commits 1 Branch 0 Tags
00a571854195b89a4dff12e5260dd8a2d8cb0b8e
Commit Graph

3080 Commits

Author SHA1 Message Date
Mingyu Chen
56a207c3f0 [case](paimon/iceberg)move cases from p2 to p0 (#37276) (#37738) 
bp #37276

Co-authored-by: wuwenchi <wuwenchihdu@hotmail.com>
 2024-07-13 10:01:05 +08:00
slothever
f2556ba182 [feature](insert)support external hive truncate table DDL (#37659) 
pick: #36801
 2024-07-12 22:37:47 +08:00
seawinde
ffa9e49bc7 [feature](mtmv) pick some mtmv pr from master (#37651) 
cherry-pick from master
pr: #36318
commitId: c1999479

pr: #36111
commitId: 35ebef62

pr: #36175
commitId: 4c8e66b4

pr: #36414
commitId: 5e009b5a

pr: #36770
commitId: 19e2126c

pr: #36567
commitId: 3da83514
 2024-07-12 10:35:54 +08:00
feiniaofeiafei
6214d6421f [Fix](planner) fix bug of char(255) toSql (#37340) (#37671) 
cherry-pick #37340 from master
 2024-07-12 10:33:24 +08:00
Sun Chenyang
4dc933bb28 [cherry-pick] (branch-2.1) fix query errors caused by ignore_above (#37685) 
## Proposed changes
pick from master #37679
 2024-07-12 09:31:45 +08:00
Sun Chenyang
79a208259e [cherry-pick] (branch-2.1) Remove the check for inverted index file exists #36945 (#37423) 2024-07-11 21:35:52 +08:00
lihangyu
217eac790b [pick](Variant) pick some refactor and fix #34925 #36317 #36201 #36793 (#37526) 2024-07-11 21:25:34 +08:00
Xin Liao
d7cae940d2 [fix](test) fix case conflict between test_tvf_based_broker_load and test_broker_load #37622 (#37631) 
cherry pick from #37622
 2024-07-11 17:52:21 +08:00
seawinde
1eb04cf538 [feature](mtmv) Support query rewrite by materialized view when query is aggregate and materialized view has no aggregate (#36278) (#37497) 
cherry-pick from master
pr: #36278
commitId: 649f9bc6
 2024-07-11 10:54:50 +08:00
lihangyu
e6b8ebc847 [Fix](Short Circuit) fix no project list in OlapScanNode (#37121) (#37504) 
pick from #37121
 2024-07-11 10:04:28 +08:00
lihangyu
e1cb568d11 [Optimize] Add session variable `max_fetch_remote_schema_tablet_count… (#37505) 
pick from #37217
 2024-07-11 10:04:20 +08:00
Xin Liao
0a95757a4d [opt](test) Optimize execution time of test_s3_load case #37562 (#37612) 
cherry pick from #37562
 2024-07-10 19:09:46 +08:00
abmdocrt
a4aaace481 [Fix](branch-2.1) Fix partial update regression test (#37560) 
Version 2.1 does not support partial update for variant type data, so we
delete this case.
 2024-07-10 10:44:24 +08:00
morrySnow
db4d061a68 [fix](Nereids) null type result with alias name should keep alias name (#37457) (#37524) 
pick from master #37457
 2024-07-09 20:46:51 +08:00
feiniaofeiafei
8ef83259ff [fix](planner) fix bug of select stmt toSql(#37274) (#37344) 
cherry-pick from master #37274
 2024-07-09 20:34:57 +08:00
minghong
9b075bc873 [fix](nereids) derive column stats for 'expr and A is not null' (#37235) (#37498) 
pick from #37235 
the algorithm for computing stats for "expr1 and expr2" predicate is as
following:
1. compute output stats of expr1 based on input stats. the result stats
is denoted by leftStats
2. compute stats of expr2 based on leftStats after step1, leftStats
should be normalized to avoid abnormal cases, such as ndv > rowCount or
numNulls > rowCount

Issue Number: close #xxx

<!--Describe your changes.-->

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-09 17:46:57 +08:00
abmdocrt
2d5eb55310 [cherry-pick](branch-2.1) Pick "[Enhancement](partial update) Add some partial update case (#36385)" (#37382) 
#36385
 2024-07-09 10:52:04 +08:00
Mingyu Chen
ca0e44f83f [fix](case) fix struct format out files (#37350) (#37499) 
bp #37350
 2024-07-09 10:11:50 +08:00
amory
0a103aa11f [improve](json)improve json support empty keys #36762 (#37351) 2024-07-08 19:04:51 +08:00
seawinde
fbc954e8be [feat](mtmv) Support grouping_sets rewrite when query rewrite by materialized view (#36056) (#37436) 
cherry pick from master
pr: #36056
commitId: 569c9772
 2024-07-08 15:06:16 +08:00
seawinde
779a51570e [opt](mtmv) Set query rewrite by materialized view default enable (#35897) (#36949) 
cherry pick from master
pr: #35897
commitId: 603fa82f
 2024-07-08 14:29:38 +08:00
hui lai
c66df8d9e6 [branch-2.1](load) fix no error url if no partition can be found (#36831) (#37401) 
## Proposed changes

pick #36831

before
```
Stream load result: {
    "TxnId": 2014,
    "Label": "83ba46bd-280c-4e22-b581-4eb126fd49cf",
    "Comment": "",
    "TwoPhaseCommit": "false",
    "Status": "Fail",
    "Message": "[DATA_QUALITY_ERROR]Encountered unqualified data, stop processing",
    "NumberTotalRows": 1,
    "NumberLoadedRows": 1,
    "NumberFilteredRows": 0,
    "NumberUnselectedRows": 0,
    "LoadBytes": 1669,
    "LoadTimeMs": 58,
    "BeginTxnTimeMs": 0,
    "StreamLoadPutTimeMs": 10,
    "ReadDataTimeMs": 0,
    "WriteDataTimeMs": 47,
    "CommitAndPublishTimeMs": 0
}
```

after
```
Stream load result: {
    "TxnId": 2014,
    "Label": "83ba46bd-280c-4e22-b581-4eb126fd49cf",
    "Comment": "",
    "TwoPhaseCommit": "false",
    "Status": "Fail",
    "Message": "[DATA_QUALITY_ERROR]too many filtered rows",
    "NumberTotalRows": 1,
    "NumberLoadedRows": 0,
    "NumberFilteredRows": 1,
    "NumberUnselectedRows": 0,
    "LoadBytes": 1669,
    "LoadTimeMs": 58,
    "BeginTxnTimeMs": 0,
    "StreamLoadPutTimeMs": 10,
    "ReadDataTimeMs": 0,
    "WriteDataTimeMs": 47,
    "CommitAndPublishTimeMs": 0,
    "ErrorURL": "http://XXXX:8040/api/_load_error_log?file=__shard_4/error_log_insert_stmt_c6461270125a615b-2873833fb48d56a3_c6461270125a615b_2873833fb48d56a3"
}
```

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-08 10:41:33 +08:00
abmdocrt
32529ecda2 [cherry-pick](branch-2.1) Pick "[Enhancement](partial update) Add partial update mix cases (#37113)" (#37384) 
#37113
 2024-07-07 18:26:46 +08:00
zzzxl
c399a0e216 [opt](inverted index) reduce generation of the rowid_result if not necessary #35357 (#36569) 2024-07-06 21:33:03 +08:00
bobhan1
ef59af8df0 [branch-2.1] Picks "[fix](regression) Fix p0 case test_modify_reorder_column #37256" (#37332) 
## Proposed changes

picks https://github.com/apache/doris/pull/37256
 2024-07-05 22:20:14 +08:00
bobhan1
5de6aa74c0 [branch-2.1] Picks "[opt](autoinc) Forbid some schema change when the table has auto-increment column #37186" (#37331) 
## Proposed changes

picks https://github.com/apache/doris/pull/37186
 2024-07-05 21:59:30 +08:00
seawinde
8a0d05d9b0 [opt](mtmv) Materialized view partition track supports date_trunc and optimize the fail reason (#35562) (#36947) 
cherry pick from master #35562
commitId: 43d0f191
 2024-07-05 15:12:43 +08:00
qiye
f8cee439b6 [feature](ES Catalog) map nested/object type in ES to JSON type in Doris (#37101) (#37182) 
backport #37101
 2024-07-05 10:48:32 +08:00
morrySnow
256221a574 [fix](Nereids) normalize aggregate should not push down lambda's param (#37109) (#37285) 
pick from master #37109

ArrayItemSlot should not be inputSlot
 2024-07-05 09:33:57 +08:00
morrySnow
8373610281 [opt](ctas) add a variable to control varchar length in ctas (#37069) (#37284) 
pick from master #37069

add a new session variable: use_max_length_of_varchar_in_ctas

In CTAS (Create Table As Select), if CHAR/VARCHAR columns do not
originate from the source table, whether to set the length of such a
column to MAX, which is 65533. The default is true.
 2024-07-04 22:09:41 +08:00
feiniaofeiafei
4e4f3d204e [feat](Nereids) push down predicates with multi columns through LogicalWindow and LogicalPartitionTopN (#36828) (#36981) 
cherry-pick #36828 to branch-2.1

The requirement for predicate pushdown through the window operator is
that the partition by slots of the window contains all slots in the
predicate. The original implementation of doris only allows predicate
pushdown with one slot. This PR relaxes this restriction and allows for
predicate pushdown with multiple slots. The same applies to the
predicate pushdown of the LogicalPartitionTopN operator. The following
sql is an example.

select
	*
from
	(
	select
		row_number() over(partition by id, value1 order by value1) as num,
		id,
		value1
	from
		push_down_multi_column_predicate_through_window_t ) t
where
	abs(id + value1)<4
	and num <= 2;


Co-authored-by: feiniaofeiafei <moailing@selectdb.com>
 2024-07-04 21:00:08 +08:00
Mingyu Chen
55636e8035 [test](migrate) move 3 cases from p2 to p0 (#36957) (#37264) 
bp #36957

Co-authored-by: slothever <18522955+wsjz@users.noreply.github.com>
 2024-07-04 20:09:59 +08:00
morrySnow
c7ad1f3d21 [fix](Nereids) simplify window expression should inherit data type (#37061) (#37283) 
pick from master #37061

after window expression rewritten by literal.
literal's data type should same with original window expression.
 2024-07-04 19:19:05 +08:00
morrySnow
e4fb506c20 [fix](Nereids) null type in result set will be cast to tinyint (#37019) (#37281) 
pick from master #37019
 2024-07-04 19:18:35 +08:00
Mingyu Chen
ceef9ee123 [feature](serde) support presto compatible output format (#37039) (#37253) 
bp #37039
 2024-07-04 13:56:05 +08:00
Mingyu Chen
3613413a54 [fix](hive) support find serde info from both tbl properties and serde properties (#37043) (#37188) 
bp #37043
 2024-07-04 13:55:38 +08:00
Jibing-Li
bf3ea1839c [test]Mv external p2 test case to p0. (#37070) (#37140) 
backport: https://github.com/apache/doris/pull/37070
 2024-07-04 11:19:31 +08:00
minghong
26be313d40 [mv](nereids) mv cost related PRs (#35652 #35701 #35864 #36368 #36789 #34970) (#37097) 
## Proposed changes
pick from #35652 #35701 #35864 #36368 #36789 #34970

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-04 09:42:11 +08:00
bobhan1
69aebc2d25 [branch-2.1] Picks "[Fix](schema change) Fix can't do reorder column schema change for MOW table and duplicate key table #37067" (#37226) 
## Proposed changes

picks https://github.com/apache/doris/pull/37067
 2024-07-03 22:42:51 +08:00
zhannngchen
97945af947 [fix](merge-on-write) when full clone failed, duplicate key might occur (#37001) (#37229) 
cherry-pick #37001
 2024-07-03 19:48:10 +08:00
zy-kkk
a9f9113c48 [branch-2.1][test](external)move hive cases from p2 to p0 (#37149) 
pk (#36855)
test_hive_same_db_table_name
test_hive_special_char_partition
test_complex_types
test_wide_table
 2024-07-03 19:44:52 +08:00
Mingyu Chen
e5695e058f [test](migrate) move 2 cases from p2 to p0 (#36935) (#37200) 
bp #36935

Co-authored-by: zhangdong <493738387@qq.com>
 2024-07-03 17:29:01 +08:00
minghong
b3f2bd20e3 [feat](nereids) support explain delete from clause #36782 (#37100) 
## Proposed changes
pick from  #36782

support explain like:
explain delete from T where A=1

Issue Number: close #xxx

<!--Describe your changes.-->

(cherry picked from commit dc369cd13096dbb90700f7fbf8f35a9059d9906f)

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-03 15:08:08 +08:00
Qi Chen
e857680661 [Migrate-Test](multi-catalog) Migrate p2 tests from p2 to p0. (#37175) 
Backport #36989.
 2024-07-03 11:08:49 +08:00
Tiewei Fang
4761090848 [fix](tvf) Partition columns in CTAS need to be compatible with the STRING type of external tables/TVF (#37161) 
bp: #35489
 2024-07-03 10:58:08 +08:00
Tiewei Fang
0aeb768bf9 [Fix](export/outfile) Support compression when exporting data to Parquet / ORC. (#37167) 
bp: #36490
 2024-07-03 10:53:57 +08:00
zhangdong
6804ae5827 [enhance](mtmv)reduce the behavior of triggering the mtmv state to change to schema_change (#36513) (#37122) 
pick from https://github.com/apache/doris/pull/36513
 2024-07-03 10:42:03 +08:00
zhangdong
5bb6642b86 [fix](mtmv)mtmv support default key (#36221) (#36601) 
pick : https://github.com/apache/doris/pull/36221
 2024-07-02 23:08:38 +08:00
zhangdong
65375b48fc [enhance](mtmv)support replace materialized view (#36749) (#37147) 
pick: https://github.com/apache/doris/pull/36749
 2024-07-02 23:00:55 +08:00
wuwenchi
e7e1e967cf [test](migrate) move 2 cases from p2 to p0 for 2.1 (#37139) 
pick #37004
 2024-07-02 22:50:53 +08:00