database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
19,699 Commits 1 Branch 0 Tags
fcdd0cb4e33abaebc2dfbbe608ea31a84313b78d
Commit Graph

5106 Commits

Author SHA1 Message Date
zzzxl
fcdd0cb4e3 [test](inverted index) Change the duplicate name test (#39017) 
https://github.com/apache/doris/pull/39013
 2024-08-07 22:10:44 +08:00
meiyi
749c9f7b56 [fix](group commit) fix repaly wal check label status (#38883) (#38997) 
pick https://github.com/apache/doris/pull/38883
 2024-08-07 22:06:59 +08:00
lihangyu
773008d6fa [Fix](Json) fix some cast issue (#38683) (#39025) 
#38683
 2024-08-07 22:05:43 +08:00
amory
931fa5bd1b [pick](cases)add array_contains cases from mysql_fulltext (#38709) 
## Proposed changes
backport: https://github.com/apache/doris/pull/34848
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-07 17:13:27 +08:00
amory
6f37e483f8 [improve](config)del useless creation config for inverted index (#39005) 
## Proposed changes
delete useless config : enable_create_inverted_index_for_array
backport: https://github.com/apache/doris/pull/39006
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-07 17:13:05 +08:00
walter
7e95d7cbec [bugfix](backup)(cooldown) cancel backup properly when be backup failed (#38724) (#38993) 
Co-authored-by: zhangyuan <ayuanzhang@tencent.com>
 2024-08-07 15:58:11 +08:00
feiniaofeiafei
f9788b4ca5 [Fix](nereids) fix partition_prune or expression evaluate wrongly (#38897) (#38998) 
cherry-pick #38897 to branch-2.1
 2024-08-07 13:49:42 +08:00
Sun Chenyang
b856530b09 [fix](inverted index) disable range query in StringTypeInvertedIndexReader (#38218) (#38926) 
## Proposed changes

pick from master #38218

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-07 10:44:02 +08:00
seawinde
2b1aa05370 pick some pr from to branch21 #38115 #38008 #37929 (#38940) 
## Proposed changes

pr: https://github.com/apache/doris/pull/38115
commitId: 2b29288c

pr: https://github.com/apache/doris/pull/38008
commitId: c6b924da

pr: https://github.com/apache/doris/pull/37929
commitId: d44fcdc5
 2024-08-07 10:19:41 +08:00
Xr Ling
2543b569bb [Optimize](Row store) pick #37145, #38236 (#38932) 2024-08-07 09:55:42 +08:00
meiyi
3abb222064 [fix](group commit) Fix test_group_commit_async_wal_msg_fault_injection case (#35313) (#38911) 
pick https://github.com/apache/doris/pull/35313
 2024-08-06 17:57:22 +08:00
924060929
fcb4483ed1 [chore](sql) Forbid show hidden columns and create table with hidden column (#38796) (#38924) 
Forbid show hidden columns and create table with hidden column
 2024-08-06 14:24:41 +08:00
924060929
75fe929dc4 [enhancement](nereids) Support eliminate outer join by match expression (#38537) (#38925) 
enable run match expression outer of filter plan, e.g join conjunct
support eliminate outer join by match expression, if any arguments of match expression is null literal
 2024-08-06 13:16:57 +08:00
zzzxl
ff6fa33021 [opt](inverted index) mow supports index optimization #(#38180) 
## Proposed changes

https://github.com/apache/doris/pull/37428
https://github.com/apache/doris/pull/37429

<!--Describe your changes.-->
 2024-08-06 11:18:13 +08:00
feiniaofeiafei
ab3057b2d4 [Feat](nereids) support date function in partition prune (#38743) (#38898) 
cherry-pick #38743 to branch-2.1
 2024-08-06 09:13:13 +08:00
zzzxl
bcea54147c [feature](inverted index) String type inverted index match function c… (#38872) 
https://github.com/apache/doris/pull/38170
 2024-08-06 09:06:05 +08:00
zhiqiang
c7b59b38ef [fix](hist) Fix unstable result of aggregrate function hist #38608 (#38893) 
cherry pick from #38608
 2024-08-06 08:52:03 +08:00
feiniaofeiafei
0f0b0e9b37 [Feat](nereids) Support date_trunc function in partition prune (#38025) (#38849) 
cherry-pick #38025 to branch-2.1
 2024-08-05 18:29:10 +08:00
starocean999
40567b5d69 [fix](nereids)support group_concat with distinct and order by (#38871) 
## Proposed changes

pick from master https://github.com/apache/doris/pull/38080

<!--Describe your changes.-->
 2024-08-05 18:23:55 +08:00
meiyi
7d4ff34d1f [fix](regression) fix test_primary_key_simple_case (#38798) (#38844) 
pick https://github.com/apache/doris/pull/38798
 2024-08-05 16:32:41 +08:00
Uniqueyou
65154f8abe [branch-2.1] (doris-future) Support auto partition name function (#38853) 
cherry-pick https://github.com/apache/doris/pull/34258 to branch-2.1
 2024-08-05 16:04:24 +08:00
Kang
808397e0d2 [fix](testcase) add order by to fix unstable output of passwordLeaked #38813 (#38855) 
cherry pick from #38813
 2024-08-05 13:51:54 +08:00
meiyi
aaee1d9bbd [fix](regression) fix prepare_insert when execute prepare stmt in observer fe (#38545) (#38850) 
pick https://github.com/apache/doris/pull/38545
 2024-08-05 13:45:13 +08:00
feiniaofeiafei
de9b9d6a39 [Fix](nereids) change char(0) to char(1), varchar(0) to varchar(65533) when create table (#38427) (#38530) 
cherry-pick #38427 to branch-2.1

---------

Co-authored-by: morrySnow <101034200+morrySnow@users.noreply.github.com>
 2024-08-05 09:18:18 +08:00
daidai
607c0b82a9 [opt](serde)Optimize the filling of fixed values ​​into block columns without repeated deserialization. (#37377) (#38245) (#38810) 
## Proposed changes
pick pr: #38575  and fix this pr bug :  #38245
 2024-08-05 09:13:08 +08:00
amory
2653087843 [pick](array-funcs)fix array with empty arg in be behavior (#38708) 
## Proposed changes
backport: https://github.com/apache/doris/pull/36845
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-05 09:08:28 +08:00
zhangstar333
1b3d4b4d31 [cherry-pick](branch-21)fix operator do_projections should use local_state intermediate_projections (#38612) (#38765) 
## Proposed changes

cherry-pick from master https://github.com/apache/doris/pull/38612

<!--Describe your changes.-->
 2024-08-05 09:07:16 +08:00
daidai
5d02c48715 [feature](hive)Support reading renamed Parquet Hive and Orc Hive tables. (#38432) (#38809) 
bp #38432 

## Proposed changes
Add `hive_parquet_use_column_names` and `hive_orc_use_column_names`
session variables to read the table after rename column in `Hive`.

These two session variables are referenced from
`parquet_use_column_names` and `orc_use_column_names` of `Trino` hive
connector.

By default, these two session variables are true. When they are set to
false, reading orc/parquet will access the columns according to the
ordinal position in the Hive table definition.

For example:
```mysql
in Hive :
hive> create table tmp (a int , b string) stored as parquet;
hive> insert into table tmp values(1,"2");
hive> alter table tmp  change column  a new_a int;
hive> insert into table tmp values(2,"4");

in Doris :
mysql> set hive_parquet_use_column_names=true;
Query OK, 0 rows affected (0.00 sec)

mysql> select  * from tmp;
+-------+------+
| new_a | b    |
+-------+------+
|  NULL | 2    |
|     2 | 4    |
+-------+------+
2 rows in set (0.02 sec)

mysql> set hive_parquet_use_column_names=false;
Query OK, 0 rows affected (0.00 sec)

mysql> select  * from tmp;
+-------+------+
| new_a | b    |
+-------+------+
|     1 | 2    |
|     2 | 4    |
+-------+------+
2 rows in set (0.02 sec)
```

You can use `set
parquet.column.index.access/orc.force.positional.evolution = true/false`
in hive 3 to control the results of reading the table like these two
session variables. However, for the rename struct inside column parquet
table, the effects of hive and doris are different.
 2024-08-05 09:06:49 +08:00
hui lai
f76397277e [fix](routine load) fix show routine load task result incorrect (#38523) (#38826) 
pick (#38523)

Create a job:
```
CREATE ROUTINE LOAD testShow ON test_show_routine_load
COLUMNS TERMINATED BY ","
PROPERTIES
(
"max_batch_interval" = "5",
"max_batch_rows" = "300000",
"max_batch_size" = "209715200"
)
FROM KAFKA
(
"kafka_broker_list" = "127.0.0.1:19092",
"kafka_topic" = "test_show_routine_load",
"property.kafka_default_offsets" = "OFFSET_BEGINNING"
);
```
show routine load task:
```
SHOW ROUTINE LOAD TASK WHERE JobName = "testShow";
```
result:
```
ERROR 1105 (HY000): errCode = 2, detailMessage = The job named testshowdoes not exists or job state is stopped or cancelled
```

Do not  use `toLowerCase` method;
 2024-08-04 22:18:25 +08:00
hui lai
79b07d0b8a [fix](routine load) fix enclose and escape can not set in routine load job (#38402) (#38825) 
pick (#38402)
 2024-08-04 22:17:12 +08:00
hui lai
8e4fad99a1 [test](routine load) add routine load case with timestamp as offset(#38567) (#38822) 
pick (#38567)
 2024-08-04 22:05:19 +08:00
hui lai
eef8c87fb5 [chore](test) disable fault injection to make pipeline task check happy (#38665) (#38821) 
pick (#38665)

test_delta_writer_v2_back_pressure_fault_injection would make pipeline
task can not finish, disable it temporarily to make pipeline task check
happy.
 2024-08-04 11:18:56 +08:00
Luwei
0603ec1d9d [enhancement](compaction) optimizing memory usage for compaction (#37099) (#37486) 2024-08-04 10:49:18 +08:00
HappenLee
7bdc508ac7 [Bug](fix) fix coredump case in (not null, null) execpt (not null, not null) case (#38756) 
## Proposed changes

Issue Number: close #38612

<!--Describe your changes.-->
 2024-08-04 10:44:10 +08:00
qiye
fe3e3d0fab [fix](test)Fix build index fault test (#38736) (#38762) 
## Proposed changes

backport #38736
 2024-08-03 23:48:29 +08:00
qiye
74908c123a [fix](test)Fix unstable test drop index fault #38768 (#38772) 
## Proposed changes

bp #38768

<!--Describe your changes.-->
 2024-08-03 23:47:55 +08:00
bobhan1
64b69ed1ba [branch-2.1] Picks "[opt](merge-on-write) Skip the alignment process of some rowsets in partial update #38487" (#38682) 
## Proposed changes

picks https://github.com/apache/doris/pull/38487
 2024-08-02 20:05:31 +08:00
amory
556f0fc784 [pick](json-keys) support json_keys function (#38631) 
## Proposed changes
backport: https://github.com/apache/doris/pull/36411
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-02 19:10:00 +08:00
morrySnow
f24d55fc94 [fix](syntax) multi statements must delim with semicolon (#38670) (#38753) 
pick from master #38670
 2024-08-02 14:49:51 +08:00
Mingyu Chen
da7b2cf578 [refactor](catalog) set "use_meta_cache" default to true (#38244)(#38352)(#38619) (#38355) 
bp #38244 #38352 #38619

---------

Co-authored-by: Yulei-Yang <yulei.yang0699@gmail.com>
 2024-08-02 14:13:38 +08:00
amory
9b07cd2069 [pick](json-serde)pick jsonb string deserialize with spec char (#38711) 
## Proposed changes
backport: https://github.com/apache/doris/pull/37176
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-02 13:37:41 +08:00
amory
7bcda89881 [pick](case) fix one_nested_types cases (#38723) 
## Proposed changes
backport: https://github.com/apache/doris/pull/38410
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-02 12:07:14 +08:00
qiye
f21d7e3833 [test](inverted index)Add cases for inverted index format v2 (#38132)(#38443) (#38222) 
## Proposed changes

backport #38132 #38443
 2024-08-02 12:04:26 +08:00
amory
1d982ada45 [pick](array-funcs)pick array func array_enumerate_uniq bugfix (#38721) 
## Proposed changes
backport: https://github.com/apache/doris/pull/38384
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-02 11:25:17 +08:00
amory
f5bc65989c [pick](array-range)improve array_range func for large param (#38707) 
## Proposed changes
backport: https://github.com/apache/doris/pull/38284
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-02 11:22:46 +08:00
amory
84d9b2fcf4 [pick](nestedtypes) support nested type with agg replace_if_not_null (#38719) 
## Proposed changes
backport: https://github.com/apache/doris/pull/38304
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-02 11:18:33 +08:00
zzzxl
0da388ade5 [fix](inverted index) fix match_phrase_ edge query result error #38327 (#38740) 2024-08-01 23:17:53 +08:00
qiye
4d980b8235 [feature](http action)Add http action to show nested inverted index file (#38272) (#38672) 
backport #38272
 2024-08-01 19:30:59 +08:00
qiye
555dccb1a4 [fix](bloom filter)Fix rename column with bloom filter (#38431) (#38662) 
backport #38431
 2024-08-01 19:01:14 +08:00
qiye
e59a866d6a [fix](test)Fix unstable index compaction fault injection case (#38501) (#38673) 
## Proposed changes

bp #38501
 2024-08-01 18:58:51 +08:00