b34a4779e5
[feat](optimizer) calculate stats health based on real updated rows count ( #24599 )
...
Use actual load rows since last analyze rather than delta of total row count.
2023-09-22 18:12:13 +08:00
c346f4d638
[Improvement](Storage) Lazy init mini_download dir ( #24649 )
2023-09-22 16:23:26 +08:00
263506f8ab
[refactor](pipelineX) add MultiCast operator ( #24656 )
2023-09-22 15:41:14 +08:00
85a1fbd5d3
[Improve](stats)Use Log4j class library instead of Quartz ( #24732 )
...
Quartz new version not support java 8
2023-09-22 15:23:58 +08:00
74bba4bdaf
[enhancement](regression-test) Add routine load case ( #24536 )
2023-09-22 14:55:01 +08:00
4edba083c0
[feat](Nereids) Support tablesample syntax ( #23717 )
...
Add table sample support like such SQL:
```sql
select * from test_table_sample_tbl tablesample(4 rows);
select * from test_table_sample_tbl t tablesample(20 percent);
select * from test_table_sample_tbl t tablesample(20 percent) repeatable 2;
```
This function has already been implemented in legacy planner, more detailed description could be found here: https://github.com/apache/doris/pull/10170
2023-09-22 14:24:07 +08:00
3c99743bf2
[enhancement](csv_reader)Optimize the reading efficiency of nullable (string) columns. ( #24698 )
...
Optimize the performance of stream load tsv by reducing virtual function calls .
(Optimize read performance of nullable (string) columns by reducing virtual function calls.)
before : 600+ s
after : 560+ s
2023-09-22 13:44:37 +08:00
320fc1481a
[fix](Nereids) some expression not cast in in predicate ( #24680 )
...
1. should use castIfNotSameType in InPredicate and CaseWhen
2. StringLikeLiteral should override equals to ignore type
2023-09-22 12:58:33 +08:00
034582bb64
[pipelineX](fix) Fix broadcast dependency hanging ( #24740 )
2023-09-22 12:24:32 +08:00
d82b603b53
[bug](jsonb) fix non simd int128 overflow #24750
2023-09-22 12:23:43 +08:00
22616d125d
[function](bitmap) add function alias bitmap_andnot and bitmap_andnot_count ( #24771 )
2023-09-22 12:18:31 +08:00
c8655d1dae
[enhancement](compaction) print correct error messsage & avoid unnecessary stacktrace ( #24758 )
2023-09-22 11:58:11 +08:00
b5f6ace204
[fix](planner) ctas should not change any meta of column in source table ( #24767 )
...
if previous PR #22770 . we try to fix incorrect nullable in target table.
However we changed nullable info of column in source table unexpectly
2023-09-22 11:49:38 +08:00
8a85a75b8b
[chore](scanner) check columns' nullable with schema ( #24724 )
...
Add a validation to prevent potential schema inconsistency issues.
2023-09-22 11:34:53 +08:00
6f961ba0e9
[Enhance](external)add prepare hive data in case ( #24703 )
2023-09-22 11:19:46 +08:00
37a7532a49
[fix](pipelineX) get sink local state in distinctstreamingagg ( #24752 )
2023-09-22 11:16:46 +08:00
181a218280
Revert "[Chore](ci)Temporarily cancel the mandatory restrictions of shellCheck ( #24765 )" ( #24770 )
...
This reverts commit 016dd2a45ff278bd4776018f6d608a6ce5e5bb21.
2023-09-22 11:14:46 +08:00
f5c07ad6da
[enhance](external)add tools for generate orc and parquet file ( #24713 )
2023-09-22 11:13:58 +08:00
79fe9d58b0
[bug](expr) forget write date_liter expr code info ( #24687 )
...
[bug](expr) forget write date_liter expr code info
2023-09-22 11:08:41 +08:00
e9ef6c7da7
[chore](workflow) Fix security issues in Code Checks ( #24761 )
...
The workflow `Code Checks` needs write permissions granted by the event `pull_request_target` to comment on pull requests. However, if the workflow ran users' code, the malicious code would do some dangerous actions on our repository.
The following changes are made in this PR:
1. Instead of applying patches, we use `sed` to modify the `entrypoint.sh` in action-sh-checker explicitly in the workflow.
2. Revoke the write permissions when generating `compile_commands.json` which is produced by executing the build script `build.sh`.
2023-09-22 10:39:39 +08:00
016dd2a45f
[Chore](ci)Temporarily cancel the mandatory restrictions of shellCheck ( #24765 )
2023-09-22 10:20:13 +08:00
a718f1f6cd
[fix](lock): do not use instance lock to protect static data ( #24611 )
2023-09-22 10:16:54 +08:00
c832e018d0
[Dependence](Fe)Upgrade Fe dependencies ( #24606 )
...
* be scanner
- Upgrade avro to 1.11.2
fe
- Upgrade quartz to 2.5.0-rc1
- Upgrade maxcompute to 0.45-2-publish
- Binding avro-ipc to 1.11.2
* Binding hbase version to 2.5.5
binding nimbusds version to 9.35
2023-09-22 10:14:42 +08:00
cec3fcdf78
[typo](doc) modifying old parameters ( #24565 )
...
* modifying old parameters
* Supplementary demo
2023-09-22 09:09:06 +08:00
269b9eaf8b
[typo](docs)add FAQ for broker load error: Cannot locate default realm ( #24693 )
2023-09-22 09:06:36 +08:00
090be20ca4
[cases](regresstests) add negative case for agg table and fix agg table support replace typ… #24715
...
add negative case for agg table
fix agg table support replace agg type for complex type , and Now We only support complex type with agg state for replace only
fix test output
2023-09-22 09:05:20 +08:00
e4c55cc5aa
[fix](web): field in Controller is singleton, which would introduce thread safe problem, so remove it ( #24667 )
...
Co-authored-by: yiguolei <676222867@qq.com >
2023-09-22 08:58:01 +08:00
a9576df74a
[typo](comments) Change the Chinese comments to English #24659
2023-09-22 08:56:56 +08:00
09e03247ec
[chore](readability) Better readability of ExecNode.cpp #24733
2023-09-22 08:54:57 +08:00
665e878811
[fix](Compile) fix BE complie error #24756
2023-09-22 08:53:30 +08:00
6685875e72
[Improvement](statistics)Mark unfinished analysis job failed after master reboot
...
Before, the Pending/Running analysis tasks/jobs will stay in the unfinished status for ever after FE reboot, which is misleading.
In this pr, jobs/tasks are only logged to editlog when they finish. So the unfinished tasks/jobs are abandoned after reboot.
Also return without retry when analyze table cancelled by user in with sync mode.
2023-09-21 23:11:50 +08:00
48f3614ce8
[fix](nereids) a bug in estimation of intersect ndv ( #24664 )
2023-09-21 23:06:31 +08:00
a17034b7ff
[fix](planner) do not support UDF without paramter ( #24730 )
...
for example:
CREATE ALIAS FUNCTION f() WITH PARAMETERS() AS now();
2023-09-21 22:23:21 +08:00
c9b2f4cb92
[workload](pipeline) Add cgroup cpu controller ( #24052 )
2023-09-21 21:49:33 +08:00
94907ca0f2
[improvement](brpc) change parking slot to 32 ( #24567 )
2023-09-21 21:35:46 +08:00
b8d8cfadfe
[enhancement](delete) avoid null txn state after delete job committed ( #24741 )
2023-09-21 21:21:06 +08:00
58ab25ccaa
Revert "[Feature](merge-on-write)Support ignore mode for merge-on-write unique table ( #21773 )" ( #24731 )
...
This reverts commit 3ee89aea35726197cb7e94bb4f2c36bc9d50da84.
2023-09-21 21:01:28 +08:00
ce14183c3a
[fix](compile) fix GHA mac ut compile failure ( #24738 )
2023-09-21 21:00:23 +08:00
3bd341a1be
[enhancement](schema) Add schema consistency check when add partition ( #24707 )
2023-09-21 20:35:45 +08:00
232226e75a
[Improve](count on index) improve performance for count on index other than match ( #24678 )
2023-09-21 20:30:27 +08:00
6c19e106ad
[fix](rest catalog)support set region for s3 ( #24566 )
...
Use REST Catalog to access S3 and support setting up regions：
```
CREATE CATALOG iceberg_rest_s3 PROPERTIES (
"type"="iceberg",
"iceberg.catalog.type"="rest",
"uri" = "http://127.0.0.1:8181 ",
"s3.endpoint" = "http://127.0.0.1:8010 ",
"s3.access_key" = "admin",
"s3.secret_key" = "password",
"s3.region" = "us-east-1"
);
```
2023-09-21 20:19:43 +08:00
a48b19ceb6
[feature](Outfile) select into outfile supports to export struct/map/array type data to orc file format ( #24350 )
...
We do not support nested complex type in this pr.
2023-09-21 20:15:18 +08:00
8a226bbd63
[fix](start_be) ignore output from command -v ( #24739 )
2023-09-21 19:57:43 +08:00
7630fe7b7b
[bug](node)fix dense_rank function in partition sort node return wrong rows ( #24727 )
2023-09-21 19:13:30 +08:00
8eb14eec7c
[enhancement](baddisk) record bad disk in be_custom.conf to handle ( #24639 )
2023-09-21 18:31:58 +08:00
ec987b1b7d
[fix](index compaction)ignore doc which dose not exist in destination segment ( #24729 )
2023-09-21 18:27:08 +08:00
cbf2e913e8
[test](config) change desired_max_waiting_jobs in regression pipeline ( #24710 )
2023-09-21 17:27:52 +08:00
85fb46bb71
[refactor](cache) Refactor preloaded timezone global cache ( #24694 )
...
Refactor preloaded timezone global cache
2023-09-21 17:26:41 +08:00
b9997d69fa
[refactor](pipelineX) return error when local_state can not find id ( #24360 )
2023-09-21 16:04:08 +08:00
36c9366a8b
[improve](tablet schema) add config to modify tablet schema recycle interval ( #24602 )
2023-09-21 15:57:11 +08:00
