database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
20,269 Commits 1 Branch 0 Tags
386c55f00aa9cd2f178eefb1d41f6e082e0715cd
Commit Graph

5403 Commits

Author SHA1 Message Date
Mingyu Chen
5b3b2cec80 [feat](metatable) support table$partitions for hive table (#40774) (#41230) 
bp #40774
and pick part of #34552, add `isPartitionedTable()` interface in `TableIf`
 2024-09-25 09:52:07 +08:00
Mingyu Chen
fc0b4b9ced [fix](explain) fix NPE when explain verbose with partition batch mode (#40969) (#41231) 
bp #40969
 2024-09-24 21:10:13 +08:00
Sun Chenyang
8bb57bcc3e [fix] (inverted index) fix the error in the query result when using count on index (#41200) 
## Proposed changes

Introduced by  #39473

<!--Describe your changes.-->
 2024-09-24 19:47:18 +08:00
zhangdong
3ad9dce067 [case](mtmv) Failed to fix the case of concurrent insert overwrite ex… (#41163) 
…ecution (#41125)

pick: https://github.com/apache/doris/pull/41125
 2024-09-24 16:04:30 +08:00
Calvin Kirs
336a658f9a [Fix](test)Fixing Kerberos Principal Not Mapping Correctly to a Local User (#40905) (#41128) 
## Proposed changes
error log


http://43.132.222.7:8111/buildConfiguration/Doris_External_Regression/529960?buildTab=tests&status=muted&focusLine=NaN&suite=&package=external_table_p0.kerberos&class=test_single_hive_kerberos&expandedTest=build%3A%28id%3A529960%29%2Cid%3A2000000060

## Changes
- If there are multiple realms, it's best to configure
hadoop.security.auth_to_local to correctly map Kerberos Principals to
local users.
- Thrift exceptions should use those from hive-shade, but to avoid
future changes in hive-shade, I omitted part of the package name prefix.

(cherry picked from commit 902cf1e913f1b01dccb5978e9c9d521a2e8c9350)

#40905
 2024-09-24 16:02:51 +08:00
walter
c81182aed8 [fix](restore) Fix clean restore with view #40620 (#41186) 
cherry pick from #40620
 2024-09-24 14:10:27 +08:00
Mingyu Chen
ff6f17c22c [fix](external-p2) ignore external p2 cases(#41148) (#41179) 
bp #41148
 2024-09-24 09:58:50 +08:00
daidai
2b427c316a [feature](functions) impl scalar functions normal_cdf,to_iso8601,from_iso8601_date (#40695) (#41049) 
bp #40695
 2024-09-24 09:52:39 +08:00
Kaijie Chen
8494496dc9 [fix](p2) fix assertion in test_broker_load_p2 (#41116) (#41117) 
cherry-pick #41116
 2024-09-24 09:46:07 +08:00
walter
0d38a9a36d [feature](restore) support atomic restore (#41107) 
Cherry-pick #40353, #40734, #40817, #40876, #40921, #41017, #41083
 2024-09-24 09:41:41 +08:00
airborne12
48e60f3ff3 [Fix](inverted index) fix wrong opt for count_on_index #41127 (#41154) 
cherry pick from #41127
 2024-09-23 22:45:52 +08:00
minghong
5bcea1983d [opt](nereids) enable runtime filter use cte as target #40815 (2.1) (#41090) 
## Proposed changes
pick #40815
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-09-23 22:34:03 +08:00
walter
a6ef7e00e4 [fix](restore) Fix view signature #41120 (#41150) 
cherry pick from #41120
 2024-09-23 22:04:20 +08:00
Mingyu Chen
51bb4e9331 [fix](test-case) fix unstable test case about table_options sys table(#41006) (#41121) 
bp #41006
 2024-09-23 17:00:51 +08:00
lihangyu
76d62c42db [pick](Variant) check enable light_schema_change when create table with variant type (#41102) 
#40908
 2024-09-23 14:30:16 +08:00
Socrates
f6917acd6a [cherry-pick](branch2.1) Impl translate and url encode 2.1 (#41051) 
## Proposed changes

pick https://github.com/apache/doris/pull/40567

some code about const folding should wait the pr picked:
https://github.com/apache/doris/pull/40441
 2024-09-23 14:26:27 +08:00
airborne12
e175c63d41 [Fix](inverted index) Fix wrong need read data opt when enable_common_expr_pushdown is disabled #40689 (#41101) 
cherry pick from #40689
 2024-09-23 14:21:30 +08:00
lihangyu
9af2de5e6a [regression-test](point query) add enable_mow_light_delete=false to related cases (#41103) 
#40520
 2024-09-23 12:32:29 +08:00
Mryange
0e5c4281dc [fix](function) fix Substring/SubReplace error result with input utf8… (#40954) 
… string (#40929)
https://github.com/apache/doris/pull/40929
```

mysql [(none)]>select sub_replace("你好世界","a",1);
+-------------------------------------+
| sub_replace('你好世界', 'a', 1)     |
+-------------------------------------+
| �a�好世界                             |
+-------------------------------------+

mysql [(none)]>select SUBSTRING('中文测试',5);
+------------------------------------------+
| substring('中文测试', 5, 2147483647)     |
+------------------------------------------+
| 中文测试                                 |
+------------------------------------------+
1 row in set (0.04 sec)

now
mysql [(none)]>select sub_replace("你好世界","a",1);
+-------------------------------------+
| sub_replace('你好世界', 'a', 1)     |
+-------------------------------------+
| 你a世界                             |
+-------------------------------------+
1 row in set (0.05 sec)

mysql [(none)]>select SUBSTRING('中文测试',5);
+------------------------------------------+
| substring('中文测试', 5, 2147483647)     |
+------------------------------------------+
|                                          |
+------------------------------------------+
1 row in set (0.13 sec)
```

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-09-23 10:44:03 +08:00
bobhan1
7d64c8cbc6 [branch-2.1] Picks "[opt](autoinc) Remove some restrictions on schema change on table that has auto-increment column #40280" (#41096) 
picks https://github.com/apache/doris/pull/40280
 2024-09-23 09:30:15 +08:00
minghong
9dc55f90eb [opt](nereids) set lower bound for range-selectivity（2.1） (#41061) 
## Proposed changes
pick #40089
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-09-22 07:32:22 +08:00
Siyang Tang
d1d52ae68c [feature](compaction) Add an http action for visibility of compaction score on each tablet (#38489) (#40826) 
pick: #38489 

Usage:
1. `curl http://be_ip:be_host/api/compaction_score?top_n=10` Returns a
json object contains compaction score for top n, n=top_n.
```
[
    {
        "compaction_score": "5",
        "tablet_id": "42595"
    },
    {
        "compaction_score": "5",
        "tablet_id": "42587"
    },
    {
        "compaction_score": "5",
        "tablet_id": "42593"
    },
    {
        "compaction_score": "5",
        "tablet_id": "42597"
    },
    {
        "compaction_score": "5",
        "tablet_id": "42589"
    },
    {
        "compaction_score": "5",
        "tablet_id": "42599"
    },
    {
        "compaction_score": "5",
        "tablet_id": "42601"
    },
    {
        "compaction_score": "5",
        "tablet_id": "42591"
    },
    {
        "compaction_score": "5",
        "tablet_id": "42585"
    },
    {
        "compaction_score": "4",
        "tablet_id": "10034"
    }
]
```
If top_n is not specified, return all compaction score for all tablets.
If top_n is illegal, raise an error.
```
invalid argument: top_n=wrong
```

2. `curl http://be_ip:be_host/api/compaction_score?sync_meta=true`
`sync_meta` is only available on cloud mode, will sync meta from meta
service. It can cooperate with top_n.
If add param `sync_meta` on non-cloud mode, will raise an error.
```
sync meta is only available for cloud mode
```

3. In the future, this endpoint may extend other utility, like fetching
tablet compaction score by table id, etc.

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-09-21 20:35:55 +08:00
Mryange
9877a08834 [feature](function) support ngram_search function #38226 (#40893) 
https://github.com/apache/doris/pull/38226 
mysql [test]>select ngram_search('123456789' , '12345' , 3);
+---------------------------------------+
| ngram_search('123456789', '12345', 3) |
+---------------------------------------+
|                                   0.6 |
+---------------------------------------+
1 row in set (0.01 sec)

mysql [test]>select ngram_search("abababab","babababa",2);
+-----------------------------------------+
| ngram_search('abababab', 'babababa', 2) |
+-----------------------------------------+
|                                       1 |
+-----------------------------------------+
1 row in set (0.01 sec)
```

doc https://github.com/apache/doris-website/pull/899

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-09-21 20:34:44 +08:00
lihangyu
d5115a21b5 [pick](ShortCircuit) Conjuncts outof key columns's order should be handled (#41071) 
#37900
 2024-09-21 20:34:05 +08:00
seawinde
4967635e54 [test](mtmv) Fix regression test unstable (#40871) (#41082) 
## Proposed changes

commitId: da6ac0c5
pr: https://github.com/apache/doris/pull/40871
 2024-09-20 23:49:23 +08:00
wuwenchi
1686650a56 [bugfix](test)Set the enable_fallback_to_original_planner behavior for 2.1 (#40913)(#40989) (#41065) 
bp:  #40913 #40989
 2024-09-20 20:08:38 +08:00
starocean999
dbd72deaa5 [opt](nereids)show user friendly error msg when window function contains order by expression (#40937) (#41036) 
pick from master #40937
 2024-09-20 18:50:55 +08:00
daidai
c744eb87c5 [fix](regression)fix some regression test (#40928) (#41046) 
bp #40928
 2024-09-20 18:17:44 +08:00
zhiqiang
1259fe2bd5 [fix](covar) Fix covar nullable on branch-2.1 (#40841) 
covar should not be always nullable.

This fix on branch-2.1 makes covar same with master on FE.
 2024-09-20 17:35:27 +08:00
Mingyu Chen
8e860a26a7 [fix](systable) fix unstable case for partitions table (#40553) (#41043) 
bp #40553
 2024-09-20 17:13:30 +08:00
Vallish Pai
d7f60c5ef1 [Cherry-pick](branch 2.1)[Enhancement] support last_query_id function (#41025) 
## Proposed changes

Issue Number: close #xxx

backport #40739
 2024-09-20 14:07:39 +08:00
TengJianPing
6539c8fd35 [fix](decimal) throw overflow exception if result is NaN of Infinit when converting from decimal to float (#40290) (#41007) 
## Proposed changes

Issue Number: close #xxx

BP #40290
 2024-09-20 14:05:51 +08:00
Socrates
e0fac66223 [branch-2.1](fix) fix snappy decompressor bug (#40862) 
## Proposed changes
Hadoop snappycodec source :

https://github.com/apache/hadoop/blob/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/native/src/codec/SnappyCodec.cc
Example:
OriginData(The original data will be divided into several large data
block.) :
     large data block1 | large data block2 | large data block3 | ....
The large data block will be divided into several small data block.
Suppose a large data block is divided into three small blocks:
large data block1: | small block1 | small block2 | small block3 |
CompressData: <A [B1 compress(small block1) ] [B2 compress(small block1)
] [B3 compress(small block1)]>

A : original length of the current block of large data block.
sizeof(A) = 4 bytes.
A = length(small block1) + length(small block2) + length(small block3)
Bx : length of  small data block bx.
sizeof(Bx) = 4 bytes.
Bx = length(compress(small blockx))
 2024-09-20 11:57:14 +08:00
Xin Liao
3249931fa6 [Fix](test) fix flake case in both test_insert_strict_fail_url and test_insert_partition_fail_url #40998 (#41024) 
cherry pick from #40998
 2024-09-20 09:52:09 +08:00
bobhan1
64880a10d6 [branch-2.1] Picks "[Fix](partial update) Fix partial update failed when merge_type=MERGE #40730" (#40951) 
picks https://github.com/apache/doris/pull/40730
 2024-09-20 00:02:17 +08:00
Kaijie Chen
f32d540ac5 [test](move-memtable) add more fault injection cases for writer v2 (#39169) (#40986) 
backport #39169
 2024-09-19 22:31:12 +08:00
zhangdong
7a24719f48 [enhance](mtmv) Refuse to execute insert overwrite on the same table … (#40992) 
…(#40558)

pick: https://github.com/apache/doris/pull/40558
 2024-09-19 22:18:46 +08:00
zy-kkk
5f583fa329 [branch-2.1][test](jdbc catalog) add oceanbase ce jdbc catalog test (#40978) 
pick #34972)
 2024-09-19 22:11:24 +08:00
Kaijie Chen
e7d41eb143 [test](p2) print error message when assert fails (#40699) (#40977) 
cherry-pick #40699
 2024-09-19 22:01:40 +08:00
Jerry Hu
b8bc9b699c [fix](scan) Incorrect scan keys lead to wrong query results. (#40814) (#40971) 
## Proposed changes
pick #40814
```
mysql [doris_14555]>select * from table_9436528_3;
+------+------+------+------+------------------------+--------------------+------+
| col1 | col2 | col3 | col5 | col4                   | col6               | col7 |
+------+------+------+------+------------------------+--------------------+------+
| -100 |    1 |  -82 |    1 | 2024-02-16 04:37:37.00 | -1299962421.904282 | NULL |
| -100 |    1 |   92 |    1 | 2024-02-16 04:37:37.00 |   23423423.0324234 | NULL |
| -100 |    0 |  -82 |    0 | 2023-11-11 10:49:43.00 |   840968969.872149 | NULL |
```
wrong result:
```
mysql [doris_14555]>select * from table_9436528_3 where col1 <= -100 and col2 in (true, false) and col3 = -82;
+------+------+------+------+------------------------+--------------------+------+
| col1 | col2 | col3 | col5 | col4                   | col6               | col7 |
+------+------+------+------+------------------------+--------------------+------+
| -100 |    1 |  -82 |    1 | 2024-02-16 04:37:37.00 | -1299962421.904282 | NULL |
| -100 |    1 |   92 |    1 | 2024-02-16 04:37:37.00 |   23423423.0324234 | NULL |
+------+------+------+------+------------------------+--------------------+------+
```

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-09-19 22:01:02 +08:00
feiniaofeiafei
8302261dd2 [Fix](nereids) set all nullable aggregate function to alwaysnullable in window expression (#40693) (#40809) 
cherry-pick from master #40693
 2024-09-19 15:19:06 +08:00
amory
f483a7605c [fix](nestedtypes) fix nested type with is_exclusive (#40434) 
is_exclusive in column_array/map/struct has wrong semantic , we should
make sure it's nested column is also is_exclusive which can make
behavior right in some operator like join.

## Proposed changes
backport: https://github.com/apache/doris/pull/40398
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-09-19 12:28:51 +08:00
camby
84f0b1fbfe [feature](view) support create or replace view stmt (#40715) (#40915) 
pick #40715 to branch-2.1
 2024-09-19 01:10:43 +08:00
daidai
774efe78e6 [fix](regression)fix maxcompute p0 case (#40933) 
## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-09-19 01:09:53 +08:00
Siyang Tang
7847dad4b8 [enhancement](show) Move storage policy info from show create table to show partitions (#39923) (#40824) 
## Proposed changes

pick: #39923
 2024-09-18 23:35:19 +08:00
morrySnow
3336e6acbb [fix](Nereids) avoid bad cast when compute scale for round (#40776) (#40904) 
pick from master #40776

for pass test case, also fix errors in computeResultInFe

computeResultInFe will generate wrong result set if output does not
match between final result and the node executing computeResultInFe
 2024-09-18 13:48:01 +08:00
Jibing-Li
c176c9feb0 [fix](statistics)Fix empty table keep auto analyze bug. (#40811) (#40896) 
backport: https://github.com/apache/doris/pull/40811
 2024-09-18 12:51:44 +08:00
Vallish Pai
4511d3e900 [cherry-pick](branch2.1) fix unstable case of partitions (#40886) 
## Proposed changes

Issue Number: close #xxx
backport #40861
 2024-09-18 09:49:24 +08:00
zhangdong
782973ee77 [fix](auth)Http api check auth (#40688) (#40865) 
pick: https://github.com/apache/doris/pull/40688
 2024-09-15 23:50:54 +08:00
seawinde
fe9f4fdd95 [pick](mtmv)Pick some pr to 21 #39998 #40332 #40485 (#40843) 
## Proposed changes
pr: https://github.com/apache/doris/pull/40485
commitId: 18a374f6

pr: https://github.com/apache/doris/pull/40332
commitId: cd2062ae

pr: https://github.com/apache/doris/pull/39998
commitId: 948fb2ab
 2024-09-15 23:48:33 +08:00