Commit Graph

15270 Commits

Author SHA1 Message Date
86c2b93e5b [improvement](fixreplica) move to healthy replica when fix replica bad (#27934)
---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
2023-12-04 16:04:27 +08:00
4d1aa131ee [Feature](datatype) add be ut codes for IPv4/v6 (#26534)
Add unit test codes for IP types
2023-12-04 15:25:02 +08:00
2eb8e0d66a [minor](testcase) Add auto partition test cases (#27921)
Add auto partition test cases
2023-12-04 15:10:42 +08:00
a6a6892f90 [chore](status code) avoid print stack for DATA_QUALITY_ERROR (#27935)
issue introduced by #27065
2023-12-04 15:04:27 +08:00
e80526ee3a [opt](nereids)remove partition & histogram from col stats to reduce memory usage #27885 2023-12-04 14:52:05 +08:00
48935c14e2 [Improvement](variant) limit the column size on tablet schema (#27399) (#27785)
1. limit the column count to default 2048
2. fix get_inverted_index return nullptr when variant's unique id is -1, using it's parent unique id instead
3. avoid add same path subcolumn duplicately in tablet schema
4. make extracted column unique id -1
2023-12-04 14:47:36 +08:00
27e1b2ac2d [docs](fix) fix document error (#27612) 2023-12-04 14:30:41 +08:00
Pxl
e196a4fd8f [Chore](case) log out show alter table info on no_await (#27926)
log out show alter table info on no_await
2023-12-04 14:30:16 +08:00
Pxl
2b715924c5 [Chore](function) set normal function use_default_implementation_for_constants to default (#27891)
set normal function use_default_implementation_for_constants to default
2023-12-04 14:19:25 +08:00
e19af1b2ed [regression](Nereids) add rule test for push down limit + sort test (#26642) 2023-12-04 14:18:55 +08:00
Pxl
45a49ac059 Bug](column) support insert default for ColumnFixedLengthObject #27927 2023-12-04 12:52:50 +08:00
e62d19d90d [improve](partition) support auto list partition with more columns (#27817)
before the partition by column only have one column.
now remove those limit, could have more columns.
2023-12-04 11:33:18 +08:00
80f528bf26 [enhancement](backup-restore) add config for upload/download task num per be (#27772)
set upload/download task num per be, and improve the overall speed of upload/download, enhance the performance of backup and recovery.

---------

Signed-off-by: nextdreamblue <zxw520blue1@163.com>
2023-12-04 11:19:45 +08:00
Pxl
e3d2425d47 [Improvement](join) remove insert_indices_from_join and special judge for -1 (#27779)
remove insert_indices_from_join and special judge for -1
2023-12-04 11:03:22 +08:00
f8bdbf67b4 [fix](deploy) K8s deploy manager cannot get group host info by endpoint (#27813)
K8s deploy manager cannot get group host info by endpoint. If we get group host info by endpoint, there is no need to init statefulset.
2023-12-04 10:50:43 +08:00
d2a99aa03b [refactor](scan) change scan reschedule into scan context (#27766)
* [refactor](scan) change scan reschedule into scan context
2023-12-04 10:25:52 +08:00
e9827f443f [typo](docs)add some tips for mc and add dlf.catalog.id info to dlf doc (#27754)
add doc for max compute and dlf
2023-12-04 10:03:21 +08:00
a62ab4049e [fix](nereids)add HllUnion and BitmapUnion for pre agg match (#27548) 2023-12-04 09:48:53 +08:00
f2cfc87aca [fix](nereids) temporary partition is selected only if user manually specified (#27893)
q1: "select * from ut_p temporary partitions(tp1) where val > 0"
in q1, temporary partition tp1 is scaned

q2: "select * from ut_p where val > 0"
in q2, temporary partition tp1 is not scaned.
2023-12-04 09:44:27 +08:00
dfad778d51 [Opt](Coordinator) Don't shuffle scan range locations when performing scan range assignments. (#27268)
Currently same query will scan different bytes in each instance in a single node environment. Don't shuffle scan range locations when performing scan range assignments to avoid it.
2023-12-04 09:26:11 +08:00
ba893a4e60 [log](table) add table lock failed log (#27659) 2023-12-03 23:34:21 +08:00
16fa6a1615 [refact](bdbje) Refact BDBEnvironment and BDBJEJournal (#27778)
* Add more ut about "org.apache.doris.journal.bdbje"
* Make tiny refactor about "org.apache.doris.journal.bdbje"
2023-12-03 23:10:07 +08:00
e2d5532e3f [improvement](disk migration) disk migration skip cooldown tablets (#27807) 2023-12-03 23:05:52 +08:00
a64656748b [Enhancenment](wal) disable group commit when streamload size is too large (#27781) 2023-12-03 23:05:11 +08:00
18d3f660d3 [fix](env) state listener avoid endless waiting (#27881) 2023-12-03 23:00:32 +08:00
97d36b4f38 [fix](csv_reader) fix trim_double_quotes behavior change (#27882) 2023-12-03 22:57:55 +08:00
3ddc8211d1 [FIX](array )fix array<null> literal in fe (#27750) 2023-12-03 13:19:22 +08:00
43f2966889 [case](regression) using load_parallelism when load csv and json from s3 (#27525)
Co-authored-by: qinhao <qinhao@newland.com.cn>
2023-12-03 09:56:47 +08:00
80d2c7ab41 [feature](parquet)support read parquet lzo compress. (#27706) 2023-12-03 09:55:52 +08:00
fc8b32be7a [Opt](multi-catalog) Opt parquet orc reader numeric copy by memcpy() and memset(). (#27545)
Opt parquet orc reader null map decoding by memset().
2023-12-03 09:55:05 +08:00
72b2a336e9 [chore](Nereids): use debug to show phase of nereids (#27903) 2023-12-02 23:19:49 +08:00
be30bd1e40 [improvement](spinlock) remove some potential bad spinlock usage (#27904)
* [improvement](spinlock) remove some potential spinlock usage

---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
2023-12-02 20:33:54 +08:00
66cfcc67cb [Fix](exectuor)Fix Follower Fe query queue may not work when exec alter #27831 2023-12-02 20:19:50 +08:00
421ab56c3e [pipelineX](improvement) Support local shuffle for join and agg (#27852) 2023-12-02 20:17:18 +08:00
654984206b [feature](profile) add avg/min/max info in uint counter #27883 2023-12-02 17:32:35 +08:00
934f37ee4f [fix](Nereids): fix bugs in struct info (#27890) 2023-12-02 15:15:20 +08:00
10483ea12c [fix](profile) fix error set with peak_memory_usage in pipeline #27749 2023-12-02 14:12:38 +08:00
2e1ce758f1 [feature](function) support ip function ipv6numtostring(alias inet6_ntoa) (#27342) 2023-12-02 11:48:19 +08:00
54b5d04ff9 [improve](csv_reader) handle csv reader error (#27892) 2023-12-02 10:05:02 +08:00
Pxl
f65103e2a6 [Chore](runtime-filter) unify interfaces of bloom filter and remove some unused code (#27822)
* unify interfaces of bloom filter and remove some unused code
2023-12-02 07:42:55 +08:00
a1a75fcfbd [fix](runtime filter) Fix extremely high CPU usage caused by rf merge #27894 2023-12-02 07:40:52 +08:00
d9bbeca431 [improve](env) Improve catalog not ready tips (#27715) 2023-12-01 22:52:43 +08:00
b74388c3b1 [case](regression) Add backup restore test with specified partition (#27694) 2023-12-01 22:31:59 +08:00
1706699e7e [fix](multi-catalog)support the max compute partition prune (#27154)
1. max compute partition prune,
we just support filter mc partitions by '=',it can filter just one partition
to support multiple partition filter and range operator('>','<', '>='..), the partition prune should be supported.

2. add max compute row count cache and partitionValues cache

3. add max compute regression case
2023-12-01 22:28:26 +08:00
f4afcae452 [case](regression) Stream load 2pc exceptions (#27804)
Co-authored-by: qinhao <qinhao@newland.com.cn>
2023-12-01 22:27:40 +08:00
68525fc112 [feature](profile) add RuntimeFilterInfo in merge profile #27869 2023-12-01 21:42:25 +08:00
fcfd0aa8e0 [fix](doc) spell error (#27079)
fixed Spelling errors in metadata-operation and cold-hot-separation
2023-12-01 21:30:50 +08:00
d80bfc19c9 [fix](doc) spell error fixes for FE & BE Config documents (#27619) 2023-12-01 20:53:26 +08:00
327035f2b0 [fix](doc) chinese translation replaced and case fix (#27611) 2023-12-01 20:53:07 +08:00
8749e5208f [fix](jdbc catalog) fix insert into jdbc table column order (#27855) 2023-12-01 20:46:48 +08:00