Commit Graph

18263 Commits

Author SHA1 Message Date
Pxl
e196a4fd8f [Chore](case) log out show alter table info on no_await (#27926)
log out show alter table info on no_await
2023-12-04 14:30:16 +08:00
Pxl
2b715924c5 [Chore](function) set normal function use_default_implementation_for_constants to default (#27891)
set normal function use_default_implementation_for_constants to default
2023-12-04 14:19:25 +08:00
e19af1b2ed [regression](Nereids) add rule test for push down limit + sort test (#26642) 2023-12-04 14:18:55 +08:00
Pxl
45a49ac059 Bug](column) support insert default for ColumnFixedLengthObject #27927 2023-12-04 12:52:50 +08:00
e62d19d90d [improve](partition) support auto list partition with more columns (#27817)
before the partition by column only have one column.
now remove those limit, could have more columns.
2023-12-04 11:33:18 +08:00
80f528bf26 [enhancement](backup-restore) add config for upload/download task num per be (#27772)
set upload/download task num per be, and improve the overall speed of upload/download, enhance the performance of backup and recovery.

---------

Signed-off-by: nextdreamblue <zxw520blue1@163.com>
2023-12-04 11:19:45 +08:00
Pxl
e3d2425d47 [Improvement](join) remove insert_indices_from_join and special judge for -1 (#27779)
remove insert_indices_from_join and special judge for -1
2023-12-04 11:03:22 +08:00
f8bdbf67b4 [fix](deploy) K8s deploy manager cannot get group host info by endpoint (#27813)
K8s deploy manager cannot get group host info by endpoint. If we get group host info by endpoint, there is no need to init statefulset.
2023-12-04 10:50:43 +08:00
d2a99aa03b [refactor](scan) change scan reschedule into scan context (#27766)
* [refactor](scan) change scan reschedule into scan context
2023-12-04 10:25:52 +08:00
e9827f443f [typo](docs)add some tips for mc and add dlf.catalog.id info to dlf doc (#27754)
add doc for max compute and dlf
2023-12-04 10:03:21 +08:00
a62ab4049e [fix](nereids)add HllUnion and BitmapUnion for pre agg match (#27548) 2023-12-04 09:48:53 +08:00
f2cfc87aca [fix](nereids) temporary partition is selected only if user manually specified (#27893)
q1: "select * from ut_p temporary partitions(tp1) where val > 0"
in q1, temporary partition tp1 is scaned

q2: "select * from ut_p where val > 0"
in q2, temporary partition tp1 is not scaned.
2023-12-04 09:44:27 +08:00
dfad778d51 [Opt](Coordinator) Don't shuffle scan range locations when performing scan range assignments. (#27268)
Currently same query will scan different bytes in each instance in a single node environment. Don't shuffle scan range locations when performing scan range assignments to avoid it.
2023-12-04 09:26:11 +08:00
ba893a4e60 [log](table) add table lock failed log (#27659) 2023-12-03 23:34:21 +08:00
16fa6a1615 [refact](bdbje) Refact BDBEnvironment and BDBJEJournal (#27778)
* Add more ut about "org.apache.doris.journal.bdbje"
* Make tiny refactor about "org.apache.doris.journal.bdbje"
2023-12-03 23:10:07 +08:00
e2d5532e3f [improvement](disk migration) disk migration skip cooldown tablets (#27807) 2023-12-03 23:05:52 +08:00
a64656748b [Enhancenment](wal) disable group commit when streamload size is too large (#27781) 2023-12-03 23:05:11 +08:00
18d3f660d3 [fix](env) state listener avoid endless waiting (#27881) 2023-12-03 23:00:32 +08:00
97d36b4f38 [fix](csv_reader) fix trim_double_quotes behavior change (#27882) 2023-12-03 22:57:55 +08:00
3ddc8211d1 [FIX](array )fix array<null> literal in fe (#27750) 2023-12-03 13:19:22 +08:00
43f2966889 [case](regression) using load_parallelism when load csv and json from s3 (#27525)
Co-authored-by: qinhao <qinhao@newland.com.cn>
2023-12-03 09:56:47 +08:00
80d2c7ab41 [feature](parquet)support read parquet lzo compress. (#27706) 2023-12-03 09:55:52 +08:00
fc8b32be7a [Opt](multi-catalog) Opt parquet orc reader numeric copy by memcpy() and memset(). (#27545)
Opt parquet orc reader null map decoding by memset().
2023-12-03 09:55:05 +08:00
72b2a336e9 [chore](Nereids): use debug to show phase of nereids (#27903) 2023-12-02 23:19:49 +08:00
be30bd1e40 [improvement](spinlock) remove some potential bad spinlock usage (#27904)
* [improvement](spinlock) remove some potential spinlock usage

---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
2023-12-02 20:33:54 +08:00
66cfcc67cb [Fix](exectuor)Fix Follower Fe query queue may not work when exec alter #27831 2023-12-02 20:19:50 +08:00
421ab56c3e [pipelineX](improvement) Support local shuffle for join and agg (#27852) 2023-12-02 20:17:18 +08:00
654984206b [feature](profile) add avg/min/max info in uint counter #27883 2023-12-02 17:32:35 +08:00
934f37ee4f [fix](Nereids): fix bugs in struct info (#27890) 2023-12-02 15:15:20 +08:00
10483ea12c [fix](profile) fix error set with peak_memory_usage in pipeline #27749 2023-12-02 14:12:38 +08:00
2e1ce758f1 [feature](function) support ip function ipv6numtostring(alias inet6_ntoa) (#27342) 2023-12-02 11:48:19 +08:00
54b5d04ff9 [improve](csv_reader) handle csv reader error (#27892) 2023-12-02 10:05:02 +08:00
Pxl
f65103e2a6 [Chore](runtime-filter) unify interfaces of bloom filter and remove some unused code (#27822)
* unify interfaces of bloom filter and remove some unused code
2023-12-02 07:42:55 +08:00
a1a75fcfbd [fix](runtime filter) Fix extremely high CPU usage caused by rf merge #27894 2023-12-02 07:40:52 +08:00
d9bbeca431 [improve](env) Improve catalog not ready tips (#27715) 2023-12-01 22:52:43 +08:00
b74388c3b1 [case](regression) Add backup restore test with specified partition (#27694) 2023-12-01 22:31:59 +08:00
1706699e7e [fix](multi-catalog)support the max compute partition prune (#27154)
1. max compute partition prune,
we just support filter mc partitions by '=',it can filter just one partition
to support multiple partition filter and range operator('>','<', '>='..), the partition prune should be supported.

2. add max compute row count cache and partitionValues cache

3. add max compute regression case
2023-12-01 22:28:26 +08:00
f4afcae452 [case](regression) Stream load 2pc exceptions (#27804)
Co-authored-by: qinhao <qinhao@newland.com.cn>
2023-12-01 22:27:40 +08:00
68525fc112 [feature](profile) add RuntimeFilterInfo in merge profile #27869 2023-12-01 21:42:25 +08:00
fcfd0aa8e0 [fix](doc) spell error (#27079)
fixed Spelling errors in metadata-operation and cold-hot-separation
2023-12-01 21:30:50 +08:00
d80bfc19c9 [fix](doc) spell error fixes for FE & BE Config documents (#27619) 2023-12-01 20:53:26 +08:00
327035f2b0 [fix](doc) chinese translation replaced and case fix (#27611) 2023-12-01 20:53:07 +08:00
8749e5208f [fix](jdbc catalog) fix insert into jdbc table column order (#27855) 2023-12-01 20:46:48 +08:00
7e3d6bc9f1 [Fix](Variant) Implement ColumnObject::update_hash_with_value (#27873) 2023-12-01 20:14:47 +08:00
1451a835b7 [fix](stats) Don't save colToPartitions anymore to save mem (#27879) 2023-12-01 19:54:30 +08:00
19281e3590 [tpcds] remove useless tpcds tools config (#27867)
Co-authored-by: zhongjian.xzj <zhongjian.xzj@192.168.2.31>
2023-12-01 18:47:12 +08:00
3f20cf1456 [fix](nereids)set operation's result type is wrong if decimal overflows (#27870) 2023-12-01 18:40:06 +08:00
c93e5d9e89 [doc](flink-connector) update flink doc and options (#27875)
---------

Co-authored-by: wudi <>
2023-12-01 17:40:08 +08:00
007506ce42 [fix](like_func) incorrect result of like with 'NO_BACKSLASH_ESCAPES' mode (#27842) 2023-12-01 17:32:46 +08:00
26e81b6573 [fix](stats)min and max return NaN when table is empty (#27862)
fix analyze empty table and min/max null value bug:
1. Skip empty analyze task for sample analyze task. (Full analyze task already skipped).
2. Check sample rows is not 0 before calculate the scale factor.
3. Remove ' in sql template after remove base64 encoding for min/max value.
2023-12-01 17:00:56 +08:00