d2a99aa03b
[refactor](scan) change scan reschedule into scan context ( #27766 )
...
* [refactor](scan) change scan reschedule into scan context
2023-12-04 10:25:52 +08:00
e9827f443f
[typo](docs)add some tips for mc and add dlf.catalog.id info to dlf doc ( #27754 )
...
add doc for max compute and dlf
2023-12-04 10:03:21 +08:00
a62ab4049e
[fix](nereids)add HllUnion and BitmapUnion for pre agg match ( #27548 )
2023-12-04 09:48:53 +08:00
f2cfc87aca
[fix](nereids) temporary partition is selected only if user manually specified ( #27893 )
...
q1: "select * from ut_p temporary partitions(tp1) where val > 0"
in q1, temporary partition tp1 is scaned
q2: "select * from ut_p where val > 0"
in q2, temporary partition tp1 is not scaned.
2023-12-04 09:44:27 +08:00
dfad778d51
[Opt](Coordinator) Don't shuffle scan range locations when performing scan range assignments. ( #27268 )
...
Currently same query will scan different bytes in each instance in a single node environment. Don't shuffle scan range locations when performing scan range assignments to avoid it.
2023-12-04 09:26:11 +08:00
ba893a4e60
[log](table) add table lock failed log ( #27659 )
2023-12-03 23:34:21 +08:00
16fa6a1615
[refact](bdbje) Refact BDBEnvironment and BDBJEJournal ( #27778 )
...
* Add more ut about "org.apache.doris.journal.bdbje"
* Make tiny refactor about "org.apache.doris.journal.bdbje"
2023-12-03 23:10:07 +08:00
e2d5532e3f
[improvement](disk migration) disk migration skip cooldown tablets ( #27807 )
2023-12-03 23:05:52 +08:00
a64656748b
[Enhancenment](wal) disable group commit when streamload size is too large ( #27781 )
2023-12-03 23:05:11 +08:00
18d3f660d3
[fix](env) state listener avoid endless waiting ( #27881 )
2023-12-03 23:00:32 +08:00
97d36b4f38
[fix](csv_reader) fix trim_double_quotes behavior change ( #27882 )
2023-12-03 22:57:55 +08:00
3ddc8211d1
[FIX](array )fix array<null> literal in fe ( #27750 )
2023-12-03 13:19:22 +08:00
43f2966889
[case](regression) using load_parallelism when load csv and json from s3 ( #27525 )
...
Co-authored-by: qinhao <qinhao@newland.com.cn >
2023-12-03 09:56:47 +08:00
80d2c7ab41
[feature](parquet)support read parquet lzo compress. ( #27706 )
2023-12-03 09:55:52 +08:00
fc8b32be7a
[Opt](multi-catalog) Opt parquet orc reader numeric copy by memcpy() and memset(). ( #27545 )
...
Opt parquet orc reader null map decoding by memset().
2023-12-03 09:55:05 +08:00
72b2a336e9
[chore](Nereids): use debug to show phase of nereids ( #27903 )
2023-12-02 23:19:49 +08:00
be30bd1e40
[improvement](spinlock) remove some potential bad spinlock usage ( #27904 )
...
* [improvement](spinlock) remove some potential spinlock usage
---------
Co-authored-by: yiguolei <yiguolei@gmail.com >
2023-12-02 20:33:54 +08:00
66cfcc67cb
[Fix](exectuor)Fix Follower Fe query queue may not work when exec alter #27831
2023-12-02 20:19:50 +08:00
421ab56c3e
[pipelineX](improvement) Support local shuffle for join and agg ( #27852 )
2023-12-02 20:17:18 +08:00
654984206b
[feature](profile) add avg/min/max info in uint counter #27883
2023-12-02 17:32:35 +08:00
934f37ee4f
[fix](Nereids): fix bugs in struct info ( #27890 )
2023-12-02 15:15:20 +08:00
10483ea12c
[fix](profile) fix error set with peak_memory_usage in pipeline #27749
2023-12-02 14:12:38 +08:00
2e1ce758f1
[feature](function) support ip function ipv6numtostring(alias inet6_ntoa) ( #27342 )
2023-12-02 11:48:19 +08:00
54b5d04ff9
[improve](csv_reader) handle csv reader error ( #27892 )
2023-12-02 10:05:02 +08:00
f65103e2a6
[Chore](runtime-filter) unify interfaces of bloom filter and remove some unused code ( #27822 )
...
* unify interfaces of bloom filter and remove some unused code
2023-12-02 07:42:55 +08:00
a1a75fcfbd
[fix](runtime filter) Fix extremely high CPU usage caused by rf merge #27894
2023-12-02 07:40:52 +08:00
d9bbeca431
[improve](env) Improve catalog not ready tips ( #27715 )
2023-12-01 22:52:43 +08:00
b74388c3b1
[case](regression) Add backup restore test with specified partition ( #27694 )
2023-12-01 22:31:59 +08:00
1706699e7e
[fix](multi-catalog)support the max compute partition prune ( #27154 )
...
1. max compute partition prune,
we just support filter mc partitions by '='，it can filter just one partition
to support multiple partition filter and range operator('>','<', '>='..), the partition prune should be supported.
2. add max compute row count cache and partitionValues cache
3. add max compute regression case
2023-12-01 22:28:26 +08:00
f4afcae452
[case](regression) Stream load 2pc exceptions ( #27804 )
...
Co-authored-by: qinhao <qinhao@newland.com.cn >
2023-12-01 22:27:40 +08:00
68525fc112
[feature](profile) add RuntimeFilterInfo in merge profile #27869
2023-12-01 21:42:25 +08:00
fcfd0aa8e0
[fix](doc) spell error ( #27079 )
...
fixed Spelling errors in metadata-operation and cold-hot-separation
2023-12-01 21:30:50 +08:00
d80bfc19c9
[fix](doc) spell error fixes for FE & BE Config documents ( #27619 )
2023-12-01 20:53:26 +08:00
327035f2b0
[fix](doc) chinese translation replaced and case fix ( #27611 )
2023-12-01 20:53:07 +08:00
8749e5208f
[fix](jdbc catalog) fix insert into jdbc table column order ( #27855 )
2023-12-01 20:46:48 +08:00
7e3d6bc9f1
[Fix](Variant) Implement ColumnObject::update_hash_with_value ( #27873 )
2023-12-01 20:14:47 +08:00
1451a835b7
[fix](stats) Don't save colToPartitions anymore to save mem ( #27879 )
2023-12-01 19:54:30 +08:00
19281e3590
[tpcds] remove useless tpcds tools config ( #27867 )
...
Co-authored-by: zhongjian.xzj <zhongjian.xzj@192.168 .2.31>
2023-12-01 18:47:12 +08:00
3f20cf1456
[fix](nereids)set operation's result type is wrong if decimal overflows ( #27870 )
2023-12-01 18:40:06 +08:00
c93e5d9e89
[doc](flink-connector) update flink doc and options ( #27875 )
...
---------
Co-authored-by: wudi <>
2023-12-01 17:40:08 +08:00
007506ce42
[fix](like_func) incorrect result of like with 'NO_BACKSLASH_ESCAPES' mode ( #27842 )
2023-12-01 17:32:46 +08:00
26e81b6573
[fix](stats)min and max return NaN when table is empty ( #27862 )
...
fix analyze empty table and min/max null value bug:
1. Skip empty analyze task for sample analyze task. (Full analyze task already skipped).
2. Check sample rows is not 0 before calculate the scale factor.
3. Remove ' in sql template after remove base64 encoding for min/max value.
2023-12-01 17:00:56 +08:00
18338a33b6
[bugfix](mergeprofile) ignore null profile to avoid bug ( #27860 )
...
---------
Co-authored-by: yiguolei <yiguolei@gmail.com >
2023-12-01 16:56:29 +08:00
34c85c962f
[opt](Nereids) improve semi/anti join estimation when column stats are unavailable #27793
...
this change improves performance of tpch q20. on sf500, improved from 6.3sec to 1.1 sec
this change has no impaction on tpcds
when column stats is unknown,
the basic algorithm to estimate left semi join output row count is its left child output row count.
q1: "A left semi join B on A.x=B.x"
the output row is estimated as A.rowCount.
But the basic algorithm is not good to following pattern:
q2: "A left semi join filter(B) on A.x=B.x"
Because there is a filter on B, usually this left semi join also reduce the row count of A, and we estimate
the output of q2 as A.rowCount * Filter.rowCount/B.rowCount
2023-12-01 15:48:33 +08:00
137f94eac9
[Bug](func) coredump in equal for null in function ( #27844 )
2023-12-01 15:48:01 +08:00
94b75515e5
[minor](stats) Throw error when sync analyze failed ( #27845 )
2023-12-01 15:44:27 +08:00
64fad89eb1
[Chore](case) add case of join with big hashtable ( #27825 )
...
add case of join with big hashtable
2023-12-01 15:32:23 +08:00
39692266d3
[minor](stats) Update olap table row count after analyze ( #27814 )
2023-12-01 13:51:42 +08:00
e868c990ff
[feature](Nereids) support add constraint on table ( #27627 )
...
support add constraint on the table including
- primary key constraint
- unique constrain
- foreign key constraint
2023-12-01 13:28:48 +08:00
48d7df205f
[chore](log) Add more detail msg for waitRPC exception #27771
2023-12-01 11:59:47 +08:00
