375770f2b4
[fix](hudi) move wrong members in HMSExternalTable ( #36187 )
...
Previously, there are 2 members: TableScanParams and IncrementalRelation
in HMSExternalTable.
These 2 members are for Hudi's incremental query, so their lifecycle
should be with query task,
should not be saved in HMSExternalTable.
This PR mainly changes:
- Add LogicalHudiScan and PhysicalHudiScan, extends from LogicalFileScan
and PhysicalFileScan.
- Move TableScanParams and IncrementalRelation from HMSExternalTable to
XXXHudiScan.
- Add or modify related Nereids rules
2024-06-13 11:50:40 +08:00
c84b56140c
[Fix](outfile) Add a configuration for exporting data in Parquet format using select into outfile ( #36143 )
...
backport: #36142
2024-06-13 11:49:46 +08:00
226775f059
[Feature](Point Query) fully support in nereids #35823 ( #36205 )
2024-06-13 08:37:31 +08:00
3a3c8cd9ee
[cherry-pick](branch-2.1) fix inverted index format is lost during a schema change #36059 ( #36100 )
2024-06-12 23:06:51 +08:00
cc7ab2b9fe
[fix](inverted index)Delete tmp dirs when BE starts to avoid tmp files left by last crash #35951 ( #36190 )
...
When BE crashes, there may be tmp files left in the tmp dir, so we
remove and rebuild the tmp dir every time we start BE to prevent rubbish
data from occupying the disk.
2024-06-12 23:05:44 +08:00
04e62d9c42
[fix](invert index) ensure that the pred result sign of the inlist is in order #36085 ( #36191 )
2024-06-12 23:04:31 +08:00
6d54527395
[fix](dynamic partition) fix dynamic partition thread met uncatch exception #35778 ( #36166 )
...
cherry pick from #35778
2024-06-12 22:16:51 +08:00
f1e83f5656
[opt](inverted index) performance optimization for need_read_data in compound #35346 ( #36192 )
2024-06-12 20:02:00 +08:00
e1694e3d91
[Pick 2.1](inverted index) fix memory leak in inverted index writer for array values #36144 ( #36165 )
2024-06-12 19:59:57 +08:00
205bf73d4e
[Pick 2.1](inverted index) low level log for fulltext query info #35820 ( #36183 )
2024-06-12 19:59:22 +08:00
9708ca8fcb
[Feature](Prepared Statment) Implement in nereids planner ( #35318 ) ( #36172 )
2024-06-12 19:54:17 +08:00
0b28420e1c
[pick](Variant) make remote schema fetch rpc timeout configurable ( #35296 ) ( #36174 )
2024-06-12 19:51:53 +08:00
d1eb917076
[fix](rpc) fix transfer large data and enable transfer_large_data_by_brpc by default #35770 ( #36169 )
...
cherry pick from #35770
2024-06-12 19:39:07 +08:00
ff517ab677
[opt](load) use notify to replace polling for FlushToken #35796 ( #36170 )
...
cherry pick from #35796
2024-06-12 19:37:27 +08:00
14ece32b87
[Pick 2.1](inverted index) add inverted index reader memory size into searcher cache ( #36160 )
...
Pick from #35149
2024-06-12 14:40:20 +08:00
73eda9bdb7
[fix](ci) external pipeline use regression-test/pipeline/external/conf/be.conf ( #36139 )
...
external pipeline use regression-test/pipeline/external/conf/be.conf instead of regression-test/pipeline/p0/conf/be.conf
relate to master #36132
Co-authored-by: stephen <hello-stephen@qq.com >
2024-06-12 11:40:16 +08:00
b75533e72b
[branch-2.1](beut) fix BE UT ( #36147 )
...
only for branch-2.1
2024-06-12 08:21:38 +08:00
c78c7f6b45
[branch-2.1](test) fix some tests in external p0 ( #36127 )
...
Also move the analysis exception of "Not support insert with partition
spec in hive catalog."
from create sink phase to bind sink phase.
So that when `set enable_fallback_to_original_planner=false;`, the
return error will be correct.
2024-06-11 22:15:28 +08:00
acbfcf7ad9
[fix](Nereids) fix four phase aggregation compute wrong result ( #36131 )
...
cherry pick from #36128
2024-06-11 20:40:18 +08:00
d2a6911791
[opt](split) close the batch mode of file split in default ( #36109 )
...
bp: #36108
2024-06-11 19:19:09 +08:00
596a9a16d3
[chore](Compile) Fix segment cache ut's compile error due to miss cherry-pick ( #36099 )
2024-06-11 17:12:42 +08:00
3b23eee37c
Revert "[fix](auto-partition) fix auto partition load lost data in multi sender ( #35287 )" ( #36098 )
...
Reverts apache/doris#35630 because it brought some more damaging bugs.
we will fix it and merge in next version
2024-06-11 17:11:42 +08:00
fce09ae2f6
[fix](third-party) enable keepalive on socket created by libevent ( #36088 )
...
pick #35805 #36026
2024-06-11 14:18:22 +08:00
e46ed37530
[fix](snappy) avoid potential buffer overflow ( #35537 ) ( #36094 )
...
pick #35537
If skip more than once when available is zero, then a buffer overflow
occurs.
2024-06-11 14:17:59 +08:00
0dccc4e6e4
[cherry-pick](branch-2.1)fix http error when downloading varaint inverted index file #35668 ( #36061 )
...
pick from master[#35668 ](https://github.com/apache/doris/pull/35668 )
2024-06-11 14:09:05 +08:00
4a277affdc
[fix](scan) In-predicate should not be pushed down for non-key column( #35913 ) ( #35968 )
...
pick #35913
2024-06-11 11:13:34 +08:00
afe2c57e05
[Fix](explain) fix tablet showing problem ( #35830 ) ( #36028 )
...
cherry-pick: #35830
2024-06-11 10:55:35 +08:00
75a6f28f2e
[cherry-pick]Add query type when report ( #35918 )
...
pick #34978
2024-06-11 10:51:59 +08:00
8e06f4ad34
[improvement](decimal) improve overflow error message ( #34689 ) ( #36078 )
...
bp #34689
Co-authored-by: TengJianPing <18241664+jacktengg@users.noreply.github.com >
2024-06-09 20:29:38 +08:00
1916891725
[fix](regression): fix nereids_hint_tpcds_p0 query64 shape ( #35906 )
...
only for 2.1
2024-06-09 14:20:34 +08:00
a0f3c1cd1e
[chore](Compile) Fix S3 file writer ut's compile error due to miss cherry-pick ( #36037 )
...
The S3 File Writer's ut can't pass ut compile, this pr tries to fix it.
2024-06-08 22:21:20 +08:00
936bf65622
[fix](nereids)decimal and datetime literal comparison should compare datatype too ( #36064 )
...
pick from master #36055
2024-06-08 22:01:37 +08:00
9e972cb0b9
[bugfix](iceberg)Fix the datafile path error issue for 2.1 ( #36066 )
...
bp: #35957
2024-06-08 21:51:46 +08:00
075481faf1
[opt](Nereids) use date signature for date arithmetic as far as possible ( #36060 )
...
pick from master #35863
2024-06-08 09:05:34 +08:00
16fcdcd4b7
[fix](Nereids) not do distinct when aggregate with distinct project ( #36057 )
...
pick from master #35899
2024-06-08 09:04:56 +08:00
240d8938f8
[bugfix](iceberg)Fixed missing type of iceberg table for timetravel for 2.1 ( #36048 )
...
bp: #36047
2024-06-07 21:13:56 +08:00
bd6b913e00
[bugfix](paimon)paimon's field length judgment error for 2.1 ( #36049 )
...
bp #35981
2024-06-07 21:13:08 +08:00
57704a3435
[branch-2.1](testcase) fix nereids check on test_mask_function ( #36052 )
...
prohibit to fallback to original planner in this case
2024-06-07 21:12:31 +08:00
5148c3908e
[fix](mtmv)fix mtmv show partition error when base table dropped ( #35729 ) ( #36051 )
...
bp #35729
2024-06-07 21:09:41 +08:00
67f4d88988
[enhancement](Nereids) support 4 phases distinct aggregate with full distribution ( #36016 )
...
cherry pick from #35871
2024-06-07 21:08:33 +08:00
1715bae26f
[opt](parquet-writer) Specify the row group size when writing data to Parquet files. ( #35081 ) ( #36042 )
...
bp #35081
Co-authored-by: Tiewei Fang <43782773+BePPPower@users.noreply.github.com >
2024-06-07 17:57:11 +08:00
19bc98a11a
[pick 2.1 ][fix ut][fix](inverted index) cloud mode supports lowercase ( #32841 ) ( #36034 )
...
pick from master #32841
Co-authored-by: zzzxl <33418555+zzzxl1993@users.noreply.github.com >
2024-06-07 17:08:29 +08:00
a518915626
[fix](pipeline) Do not push data in local exchange if eos ( #35972 ) ( #36010 )
...
pick #35972 and #34536
2024-06-07 15:40:55 +08:00
4030164270
[feature](tools) embed async profiler to help user generate flame graph for frontend ( #36018 )
...
cherry pick from #35010
2024-06-07 15:38:01 +08:00
4fc3d0ce2c
[Chore](pipeline) set PipelineFragmentContext::_timeout and adjust dump_pipeline_tasks infomation display ( #36023 )
...
## Proposed changes
pick from #35328
2024-06-07 15:36:29 +08:00
89a553264f
[fix](regression) fix some analyzer error check failed ( #36020 )
...
Co-authored-by: moailing <moailing@selectdb.com >
2024-06-07 15:34:24 +08:00
db5048b5b5
[feature](Nereids) support select ALL ( #36029 )
...
pick from master #35935
2024-06-07 15:29:56 +08:00
9f3fe3e57c
[fix](DDL) not set table type as default comment when create table ( #36025 )
...
pick from master #35855
2024-06-07 15:29:10 +08:00
d1318a7d08
[branch-2.1](jvm) disable BE's jvm metrics ( #36009 )
...
disable BE's jvm metrics on external p0, because there is some issue
with ASAN when BE exit.
2024-06-07 15:28:02 +08:00
2358ebe338
[Fix](feut) Fix distribute hint test with reorder cross join ( #35987 )
...
when checking plan shape of distribute hint test, the shape of it does
not changed but projection has changed. So it needed to be updated
2024-06-07 14:47:45 +08:00
