Commit Graph

18263 Commits

Author SHA1 Message Date
f3204ec18c [fix](memory) Fix thread_context check failed: thread_mem_tracker()->label() == "Orphan" #27468 2023-11-24 20:57:41 +08:00
14527c4930 [chore](case) adjust timeout of broker load case (#27539)
when running p2, this case may fail because of timeout.
2023-11-24 20:55:26 +08:00
a0b1cb48a1 [Improve](regresscases) update cases for three-level nested types #27529 2023-11-24 20:53:28 +08:00
e4258dd868 [chore](fe plugin) Upgrade dependency to doris 2.0-SNAPSHOT #27522 2023-11-24 19:30:52 +08:00
d98d2d491f [chore](tablet scheduler) update sched ctx err (#27514) 2023-11-24 18:09:49 +08:00
d769fb69dd NEED (#27531) 2023-11-24 18:04:36 +08:00
70bbaa4e56 [test](regression) add cases about datev1/datatimev1 (#27543)
All cases' results are tested and passed with datetime/date v2
Cases about:
Calculation ( +, -
Kinds of predicates(<, >, =, <>, in, not in, is null, is not null)
Load test(from csv and select into)
Runtime filter
Delete conditions
Key columns(agg/duplicate/uniq model, distributed/partition, bitmap index...)
2023-11-24 17:58:32 +08:00
096ce9d75b [typo](doc)Add flink cdc partial column update column (#26402) 2023-11-24 17:10:33 +08:00
553e4a8903 [feature-wip](merge-on-write) MOW table support different primary keys and sort keys (#24788) 2023-11-24 16:37:30 +08:00
2152d5a5dd [regression test](control) add some regression case for control (#27346)
* [regression test](control) add some regression case for control

```
1. CANCEL DECOMMISSION BACKEND
2. SET REPLICA STATUS
3. CLEAN TRASH
4. SET REPLICA VERSION
5. BAD REPLICA
```
2023-11-24 16:13:12 +08:00
fdcb401835 [doc](fix) json load supported format (#27415)
Co-authored-by: qinhao <qinhao@newland.com.cn>
2023-11-24 16:06:44 +08:00
89d5efac46 [pipelineX](dependency) Use dependency to replace ready_to_execute (#27528) 2023-11-24 16:03:14 +08:00
Pxl
33b540641e [Bug](materialized-view) add limitation for duplicate expr on materialized view (#27523)
add limitation for duplicate expr on materialized view
2023-11-24 15:47:14 +08:00
126714ace4 [fix](pipelineX) fix nullptr in loca exchange dependency (#27488)
fix nullptr in loca exchange dependency
2023-11-24 15:12:25 +08:00
ce99de3c7a [regression-test](fix)add ctas test cases. (#24278) 2023-11-24 14:44:11 +08:00
674dd98fb2 [fix](Nereids): CustomRewriteJob don't handle condition rule return null (#27516) 2023-11-24 14:38:37 +08:00
540132f656 [fix](profilev2) fix merge profile min is zero #27524 2023-11-24 14:11:11 +08:00
dbff9d7a89 [chore](fuzzy) topn_opt_limit_threshold (#27496) 2023-11-24 14:08:27 +08:00
dbbab63623 [fix](nereids)keep cast operator if cast a varchar to another longer varchar in LogicalSetOperator (#27393) 2023-11-24 14:07:04 +08:00
dfe3a2dd01 [feature](mtmv)(3)Implementing multi table materialized views (#26146)
Introduction to Main Classes:
- MTMVService:MTMV services for other modules to call
- MTMVHookService:All operations that affect the MTMV
  - MTMVJobManager:All operations that affect the MTMV job
  - MTMVCacheManager:All operations that affect the MTMV Cache
- MTMVTask&MTMVJob:Inherit from job framework
2023-11-24 12:34:38 +08:00
da71fde066 [fix](build index) Fix inverted index hardlink leak and missing problem (#26903) 2023-11-24 10:30:21 +08:00
d5908b0c8b [regression](compaction) Add case to test base_compaction with dup_key_max_file_size_limit (#27088) 2023-11-24 10:27:51 +08:00
c24a33c857 [enhancement](audit) hide password and other sensitive information in log and audit log (#27115)
Signed-off-by: nextdreamblue <zxw520blue1@163.com>
2023-11-24 10:27:30 +08:00
0a8e3d2199 [enhance](PrefetchReader) Make the prefetch timeout one config (#27371) 2023-11-24 10:24:15 +08:00
9a92323221 [fix](fs) prefetch buffer check status before verifing buf size (#27388) 2023-11-24 10:19:37 +08:00
160adbaa69 [regression test](routine test) add case for num_as_string (#27436) 2023-11-24 10:15:47 +08:00
75c9f00137 [Bug](bitmap) Fix heap-use-after-free in the bitmap functions (#27411) 2023-11-24 10:09:31 +08:00
17ca75f834 [chore](Nereids): add eager aggregate into rules (#27505)
Add `Eager Aggregate` rules into Rewrite rules.
2023-11-24 10:06:04 +08:00
5a82a5746e [pipelineX](bug) Fix core dump if cancelled (#27449) 2023-11-24 09:21:17 +08:00
df628e1538 [chore](merge-on-write) disable rowid conversion check for mow table by default (#27482) 2023-11-23 23:39:01 +08:00
39a5229027 [fix](regression)Fix hive p2 case (#27466) 2023-11-23 23:32:21 +08:00
b580ee91ce [fix](compile) fix macOS compile and format code (#27494) 2023-11-23 23:24:10 +08:00
8e74470db9 [fix](statistics)Fix auto analyze remove finished job bug (#27486)
Finished job must be removed from the job list, otherwise the next batch of jobs will not be scheduled.
2023-11-23 23:22:02 +08:00
eb878ad0d2 [fix](Export) add feut for Cancel Export (#27178) 2023-11-23 23:18:30 +08:00
5adbe47d3a [test](regression) add stream load tvf properties regression test (#27467) 2023-11-23 23:04:10 +08:00
5d31bc99b8 [Fix](Group_commit) Fix group commit regression test failure (#27475) 2023-11-23 23:03:38 +08:00
540bce4d1b [typo](log) Let env lock msg more distinct (#27493) 2023-11-23 23:03:06 +08:00
aa766a7b4a [bugfix](pipeline core) lock fragment context during task close to avoid concurrent delete (#27484) 2023-11-23 21:43:39 +08:00
d73b945535 [chore](Nereids): rename pushdown to push_down (#27473) 2023-11-23 21:04:40 +08:00
d04a2de3cc [fix](hms) fix compatibility issue of hive metastore client (#27327)
For hive version lower than 2.3.7, there is no enum ClientCapability.INSERT_ONLY_TABLES.
So if we send this enum to the server side, the server side will get a null,
and this will cause some undefined behavior, eg, failed to get tables infos from hms.
2023-11-23 19:42:46 +08:00
78203c8b7a [chore](docker cases): support for specifying output from the command line (#27340) 2023-11-23 19:36:43 +08:00
dd65cc1d14 [opt](MergedIO) no need to merge large columns (#27315)
1. Fix a profile bug of `MergeRangeFileReader`, and add a profile `ApplyBytes` to show the total bytes  of ranges.
2. There's no need to merge large columns, because `MergeRangeFileReader` will increase the copy time.
2023-11-23 19:15:47 +08:00
2ea33518b0 [Opt](load) use batching to optimize auto partition (#26915)
use batching to optimize auto partition
2023-11-23 19:12:28 +08:00
511eedb4ff [fix](nereids)select base index if mv's data type is different from base table (#27387)
normally, mv column's data type should be same as base table. This pr plays as a fail-safe, if mv column's data type is different from base table accidentally, fall back to select base table to make the query works.
2023-11-23 18:41:59 +08:00
d9f6e51884 [fix](planner)output slot should be materialized as intermediate slot in agg node (#27282) 2023-11-23 18:41:08 +08:00
1555b11035 [fix](nereids)remove literal partition by and order by expression in window function (#26899) 2023-11-23 18:40:51 +08:00
2ec3395087 [fix](planner)the data type should be the same between input slot and sort slot (#27137) 2023-11-23 18:40:02 +08:00
ab739a6cdc [Chore](workflow)Fix Pr comment not worker (#27400) 2023-11-23 18:17:31 +08:00
6fdaf2da6f [fix](ci) 1. if skip compile then skip p0 p1 external pipelinex_p0 also, (#27469)
1. if skip compile then skip p0 p1 external pipelinex_p0 also
2. if change files too many, export all_files may cause problem of 'Argument list too long', change to use file
Co-authored-by: stephen <hello-stephen@qq.com>
2023-11-23 18:11:29 +08:00
772f181e94 [fix](stats) Fix thread leaks when doing checkpoint (#27334) 2023-11-23 03:18:19 -06:00