database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
20,269 Commits 1 Branch 0 Tags
386c55f00aa9cd2f178eefb1d41f6e082e0715cd
Commit Graph

9229 Commits

Author SHA1 Message Date
meiyi
386c55f00a [fix](transaction) commit txn check txn status (#40064) (#41227) 
pick https://github.com/apache/doris/pull/40064/
 2024-09-25 10:56:10 +08:00
Mingyu Chen
5b3b2cec80 [feat](metatable) support table$partitions for hive table (#40774) (#41230) 
bp #40774
and pick part of #34552, add `isPartitionedTable()` interface in `TableIf`
 2024-09-25 09:52:07 +08:00
Mingyu Chen
fc0b4b9ced [fix](explain) fix NPE when explain verbose with partition batch mode (#40969) (#41231) 
bp #40969
 2024-09-24 21:10:13 +08:00
Jibing-Li
5c5251405a [improvement](statistics)Remove analyze retry logic. (#41220) 
Remove analyze retry logic when task failed. Because usually retry would
fail again and retry would bring a long time of sleep, which cause the
analyze job running too slow.
Master pr: https://github.com/apache/doris/pull/33703
 2024-09-24 20:26:01 +08:00
hui lai
a1429efc41 [fix](routine load) write edit log when rescheduled job (#40728) (#41134) 
pick (#40728)
```
2024-09-11 20:00:53,079 ERROR (replayer|105) [RoutineLoadManager.replayChangeRoutineLoadJob():836] should not happened
org.apache.doris.common.DdlException: errCode = 2, detailMessage = Could not transform PAUSED to PAUSED
	at org.apache.doris.load.routineload.RoutineLoadJob.checkStateTransform(RoutineLoadJob.java:855) ~[doris-fe.jar:1.2-SNAPSHOT]
	at org.apache.doris.load.routineload.RoutineLoadJob.unprotectUpdateState(RoutineLoadJob.java:1407) ~[doris-fe.jar:1.2-SNAPSHOT]
	at org.apache.doris.load.routineload.RoutineLoadJob.updateState(RoutineLoadJob.java:1394) ~[doris-fe.jar:1.2-SNAPSHOT]
	at org.apache.doris.load.routineload.RoutineLoadManager.replayChangeRoutineLoadJob(RoutineLoadManager.java:834) ~[doris-fe.jar:1.2-SNAPSHOT]
	at org.apache.doris.persist.EditLog.loadJournal(EditLog.java:717) ~[doris-fe.jar:1.2-SNAPSHOT]
	at org.apache.doris.catalog.Env.replayJournal(Env.java:2913) ~[doris-fe.jar:1.2-SNAPSHOT]
	at org.apache.doris.catalog.Env$4.runOneCycle(Env.java:2675) ~[doris-fe.jar:1.2-SNAPSHOT]
	at org.apache.doris.common.util.Daemon.run(Daemon.java:116) ~[doris-fe.jar:1.2-SNAPSHOT]
```

`unprotectNeedReschedule()` will change job state to
`JobState.NEED_SCHEDULE` without `logOpRoutineLoadJob`.If job is paused
then rescheduled and paused finally, the record of two consecutive edit
logs will be 'PAUSED', the correct
replay sequence should be: `PAUSED` -> `NEED_SCHEDULE` ->` PAUSED`. 

Therefore, it is need to write edit log when rescheduled job.
 2024-09-24 16:03:55 +08:00
walter
c81182aed8 [fix](restore) Fix clean restore with view #40620 (#41186) 
cherry pick from #40620
 2024-09-24 14:10:27 +08:00
daidai
2b427c316a [feature](functions) impl scalar functions normal_cdf,to_iso8601,from_iso8601_date (#40695) (#41049) 
bp #40695
 2024-09-24 09:52:39 +08:00
walter
0d38a9a36d [feature](restore) support atomic restore (#41107) 
Cherry-pick #40353, #40734, #40817, #40876, #40921, #41017, #41083
 2024-09-24 09:41:41 +08:00
airborne12
48e60f3ff3 [Fix](inverted index) fix wrong opt for count_on_index #41127 (#41154) 
cherry pick from #41127
 2024-09-23 22:45:52 +08:00
minghong
5bcea1983d [opt](nereids) enable runtime filter use cte as target #40815 (2.1) (#41090) 
## Proposed changes
pick #40815
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-09-23 22:34:03 +08:00
walter
a6ef7e00e4 [fix](restore) Fix view signature #41120 (#41150) 
cherry pick from #41120
 2024-09-23 22:04:20 +08:00
Calvin Kirs
d529f00dc0 [branch-2.1][Fix](hdfs-fs)The cache expiration should explicitly release the held fs (#38610) (#41108) 
bp #38610
 2024-09-23 18:55:08 +08:00
Calvin Kirs
fdd7ffc0bd [branch-2.1][Fix](http)Ignore exceptions when getting system information errors #39671 (#41106) 
bp #39671
 2024-09-23 18:35:35 +08:00
Calvin Kirs
e1057ac26d [branch-2.1][fix](metadata)Add FE metadata-related file checks #40546 (#41113) 
## Proposed changes

#40546
 2024-09-23 17:13:35 +08:00
lihangyu
76d62c42db [pick](Variant) check enable light_schema_change when create table with variant type (#41102) 
#40908
 2024-09-23 14:30:16 +08:00
Socrates
f6917acd6a [cherry-pick](branch2.1) Impl translate and url encode 2.1 (#41051) 
## Proposed changes

pick https://github.com/apache/doris/pull/40567

some code about const folding should wait the pr picked:
https://github.com/apache/doris/pull/40441
 2024-09-23 14:26:27 +08:00
walter
059f822a3a [fix](restore) Remove the sqlMode from the view signature #40615 (#41110) 
cherry pick from #40615
 2024-09-23 14:07:52 +08:00
zclllhhjj
d44ee1cddb [branch-2.1](session-variable) Add a debug variable of constant fold (#35584) (#41032) 
pick https://github.com/apache/doris/pull/35584
 2024-09-23 10:01:05 +08:00
bobhan1
7d64c8cbc6 [branch-2.1] Picks "[opt](autoinc) Remove some restrictions on schema change on table that has auto-increment column #40280" (#41096) 
picks https://github.com/apache/doris/pull/40280
 2024-09-23 09:30:15 +08:00
minghong
9dc55f90eb [opt](nereids) set lower bound for range-selectivity（2.1） (#41061) 
## Proposed changes
pick #40089
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-09-22 07:32:22 +08:00
Mryange
9877a08834 [feature](function) support ngram_search function #38226 (#40893) 
https://github.com/apache/doris/pull/38226 
mysql [test]>select ngram_search('123456789' , '12345' , 3);
+---------------------------------------+
| ngram_search('123456789', '12345', 3) |
+---------------------------------------+
|                                   0.6 |
+---------------------------------------+
1 row in set (0.01 sec)

mysql [test]>select ngram_search("abababab","babababa",2);
+-----------------------------------------+
| ngram_search('abababab', 'babababa', 2) |
+-----------------------------------------+
|                                       1 |
+-----------------------------------------+
1 row in set (0.01 sec)
```

doc https://github.com/apache/doris-website/pull/899

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-09-21 20:34:44 +08:00
lihangyu
d5115a21b5 [pick](ShortCircuit) Conjuncts outof key columns's order should be handled (#41071) 
#37900
 2024-09-21 20:34:05 +08:00
wuwenchi
e568ba3b4c [bugfix](hive)Handle exceptions when submitting tasks fails to prevent dead loop for 2.1 (#40708) (#41052) 
bp: #40708
 2024-09-21 08:47:31 +08:00
zhiqiang
63a1e0dccb [fix](profile) Fix npe when updating profile for internal load task. #40990 (#41077) 
cherry pick from #40990
 2024-09-20 23:46:43 +08:00
Vallish Pai
ff22059d5d [cherry-pick](branch-2.1)[fix](sys_table) fix potential NPE when query table_properties and partitions sys table (#41081) 
## Proposed changes

Issue Number: close #xxx

bp #40972
 2024-09-20 23:43:33 +08:00
wuwenchi
f4b06e83eb [bugfix](hive)Fix cache inconsistency issue for 2.1 (#40729) (#41050) 
## Proposed changes
bp: #40729
 2024-09-20 20:28:55 +08:00
starocean999
f60a881425 [fix](nereids)adjust conjunct's nullable info in LogicalExternalRelation (#41031) 
## Proposed changes

pick from master https://github.com/apache/doris/pull/41014
<!--Describe your changes.-->
 2024-09-20 20:20:12 +08:00
wuwenchi
2bb83302e4 [bugfix](hive)Remove redundant split operations for 2.1 (#40683) (#41055) 
## Proposed changes
bp: #40683
 2024-09-20 20:09:11 +08:00
starocean999
dbd72deaa5 [opt](nereids)show user friendly error msg when window function contains order by expression (#40937) (#41036) 
pick from master #40937
 2024-09-20 18:50:55 +08:00
zhiqiang
1259fe2bd5 [fix](covar) Fix covar nullable on branch-2.1 (#40841) 
covar should not be always nullable.

This fix on branch-2.1 makes covar same with master on FE.
 2024-09-20 17:35:27 +08:00
Mingyu Chen
8e860a26a7 [fix](systable) fix unstable case for partitions table (#40553) (#41043) 
bp #40553
 2024-09-20 17:13:30 +08:00
Mingyu Chen
3484e7b2ff [fix](table_options) fix potential NPE when quering table_options systable (#40944) (#41040) 
bp #40944
 2024-09-20 16:57:02 +08:00
Vallish Pai
d7f60c5ef1 [Cherry-pick](branch 2.1)[Enhancement] support last_query_id function (#41025) 
## Proposed changes

Issue Number: close #xxx

backport #40739
 2024-09-20 14:07:39 +08:00
bobhan1
64880a10d6 [branch-2.1] Picks "[Fix](partial update) Fix partial update failed when merge_type=MERGE #40730" (#40951) 
picks https://github.com/apache/doris/pull/40730
 2024-09-20 00:02:17 +08:00
zy-kkk
8b1906be81 [branch-2.1][fix](jdbc catalog) Fixed FE memory leak by enabling weak references in HikariCP (#40923) 
pick  (#40773)

When FE collects statistics, its ThreadLocal will hold the ThreadLocal
of HikariCP in JdbcClient, making it difficult for HikariCP to be gc, so
this problem can be alleviated by enabling weak references of HikariCP.
 2024-09-19 23:59:14 +08:00
Kaijie Chen
549bc3e288 [fix](pipelinex) fix fragment instance progress reports (#40325) (#40987) 
backport #40325
 2024-09-19 23:58:38 +08:00
bobhan1
49dec9f39d [branch-2.1] Picks "[opt](merge-on-write) Reduce the version not continuous logs for merge-on-write table #40946" (#40996) 
picks https://github.com/apache/doris/pull/40946
 2024-09-19 23:58:05 +08:00
zhangdong
f81d087bab [enhance](mtmv)Improve the speed of obtaining table snapshots (#40717) (#40995) 
pick: https://github.com/apache/doris/pull/40717
 2024-09-19 22:47:50 +08:00
zhangdong
fd5423145d [enhance](mtmv)tasks tvf not getLock (#40816) (#40994) 
pick: https://github.com/apache/doris/pull/40816
 2024-09-19 22:40:17 +08:00
zhangdong
7ecf088a61 [enhance](mtmv)when drop temp partition, not trigger materialized vie… (#40993) 
…w refresh (#40875)

pick: https://github.com/apache/doris/pull/40875
 2024-09-19 22:35:31 +08:00
zhangdong
7a24719f48 [enhance](mtmv) Refuse to execute insert overwrite on the same table … (#40992) 
…(#40558)

pick: https://github.com/apache/doris/pull/40558
 2024-09-19 22:18:46 +08:00
feiniaofeiafei
8302261dd2 [Fix](nereids) set all nullable aggregate function to alwaysnullable in window expression (#40693) (#40809) 
cherry-pick from master #40693
 2024-09-19 15:19:06 +08:00
camby
84f0b1fbfe [feature](view) support create or replace view stmt (#40715) (#40915) 
pick #40715 to branch-2.1
 2024-09-19 01:10:43 +08:00
Siyang Tang
7847dad4b8 [enhancement](show) Move storage policy info from show create table to show partitions (#39923) (#40824) 
## Proposed changes

pick: #39923
 2024-09-18 23:35:19 +08:00
morrySnow
5c70fe35ea [fix](Nereids) sink distribute is wrong when table is unpartitioned (#40541) (#40901) 
pick from master #40541
 2024-09-18 15:18:41 +08:00
lihangyu
956b56c664 [pick](short circuit) avoid set cacheId when non prepared execute (#40909) 
#40525
 2024-09-18 14:50:50 +08:00
zy-kkk
f3a42faf40 [branch-2.1][improvement](jdbc catalog) support jdbc external catalog insert stmt in nereids (#40902) 
pick  (#39813)
 2024-09-18 14:02:20 +08:00
morrySnow
7d23a7c587 [fix](planner) throw NPE when insert overwrite partition(*) (#40656) (#40903) 
pick from master #40656
 2024-09-18 13:48:48 +08:00
morrySnow
3336e6acbb [fix](Nereids) avoid bad cast when compute scale for round (#40776) (#40904) 
pick from master #40776

for pass test case, also fix errors in computeResultInFe

computeResultInFe will generate wrong result set if output does not
match between final result and the node executing computeResultInFe
 2024-09-18 13:48:01 +08:00
Jibing-Li
8a575d0891 [improvement](statistics)User admin instead of root to run stats sql. (#40867) (#40898) 
backport: https://github.com/apache/doris/pull/40867
 2024-09-18 13:36:50 +08:00