Commit Graph

16613 Commits

Author SHA1 Message Date
b60a272be0 [Bug](pipeline) Fix pipeline load lose data use wrong var check order (#30341) 2024-01-25 13:24:52 +08:00
6614d40dad [bugfix](core) fix core due to send rpc and request is deconstructed (#30344)
Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-01-25 13:24:52 +08:00
xy
b1722231c4 [Fix](doc) Modify some default values in the documentation (#30304)
Co-authored-by: xingying01 <xingying01@corp.netease.com>
2024-01-25 13:24:52 +08:00
01c394acc2 [fix](inverted index)support merge null_bitmap during index compaction (#30326)
`null_bitmap` file is not considered in index compaction process. This will lead wrong query result when doc is contain `NULL` values.
2024-01-25 13:24:52 +08:00
081da1299a [fix](fe) Fix FrontendHbResponse serialization compatibility problem (#30337)
* Between branch-1.2-lts and branch-2.1, `FrontendHbResponse` has upgrade
  compatiblity problem, because `arrowFlightSqlPort` field, only metaVerserion
  less than FeMetaVersion.VERSION_121 will call `FrontendHbResponse.readField`

* Introduced by https://github.com/apache/doris/pull/24314
2024-01-25 13:24:52 +08:00
ff47c69bf9 [doc](fix) K8s doc modify and add pvc network docs (#30307) 2024-01-25 13:24:52 +08:00
61bab1f1c0 [Fix](inverted index) fix data size when drop inverted index (#30327) 2024-01-25 13:24:52 +08:00
Pxl
1d7d7ee9b6 [Chore](join) split out join hash map from hash map (#30280)
split out join hash map from hash map
2024-01-25 13:24:52 +08:00
Pxl
5b462194d1 [Feature](materialized-view) support rewrite case when to if on legacy planner to make mv work (#30320)
support rewrite case when to if on legacy planner to make mv work
2024-01-25 13:24:52 +08:00
5ba9987d3a [Fix](executor)Release resource correctly when drop workload group (#30279) 2024-01-25 13:24:52 +08:00
3d22f9cfc8 [feature](replica) Add drop replica safely on backend (#30303) 2024-01-25 13:24:52 +08:00
49f879f8fd [regression test](framework) add waitFor action (#30289) 2024-01-25 13:24:52 +08:00
bee6ae73c7 [minor](Nereids): enable PushDownTopNDistinctThroughJoin (#30275) 2024-01-25 13:24:52 +08:00
15728756e2 [improve](streamload) do not print stack trace when get data from pipe meet cancel (#30297) 2024-01-25 13:24:52 +08:00
2e6a00690f [Fix](smooth-upgrade) fix unnecessary high version of smooth upgrade (#30283)
fix unnecessary high version of smooth upgrade
2024-01-25 13:24:09 +08:00
825fe1470f [Docs](arm) update arm compilation and auto partition (#30311)
update arm compilation and auto partition
2024-01-25 13:24:09 +08:00
1322c813d3 [Fix](executor)Fix return stack to fe #30316 2024-01-25 13:24:09 +08:00
0287651114 [fix](char) fix char column type not check lenght (#30266) 2024-01-25 13:24:09 +08:00
86d7a8be44 [improvement](statistics nereids)Nereids support select mv. (#30267) 2024-01-25 13:24:09 +08:00
67051318a2 [docs](fix)fix max_filter_ratio default value #30318 2024-01-25 13:24:09 +08:00
63cf2777f9 [improvement](statistics)Catch load column stats exception, avoid print too much stack info to fe.out #30315 2024-01-25 13:24:09 +08:00
83ea486b15 [fix](Nereids): Except just can merge with left deep shape (#30270) 2024-01-25 13:24:09 +08:00
1aa006c80f [fix](ci) add single quote to the value of the session variables when setting it (#30295)
Co-authored-by: stephen <hello-stephen@qq.com>
2024-01-25 13:24:09 +08:00
72026778fe [optimize](inverted index) Use inverted index filter null field (#30237) 2024-01-25 13:24:09 +08:00
Pxl
7e60369ba2 [Feature](materialized-view) support create mv with count(*) (#30313)
support create mv with count(*)
2024-01-25 13:24:09 +08:00
101b2593fc [regression test](schema change) add case for tinyint/smallint/int/bigint/float/double type in agg (#30193) 2024-01-25 13:24:09 +08:00
df504df475 [regression test](schema change) add case for partition (#30195) 2024-01-25 13:24:09 +08:00
e94683fffb [pipelineX](refactor) refine relationship between shared state and dependency (#30294) 2024-01-25 13:24:09 +08:00
2951f9a2c7 [fix](Nereids) relation in constraints should be multi parts (#30293) 2024-01-25 13:24:09 +08:00
7e1a986fa1 [feat](Nereids): eliminate left outer join by unique when comparing mv (#30228) 2024-01-25 13:24:09 +08:00
cd70f45ce2 [test](ut) added UT cases for show build index (#29561)
Added UT case for show build index flow
2024-01-25 13:24:09 +08:00
ca5a314765 [fix](function) make STRLEFT and STRRIGHT and SUBSTR function DEPEND_ON_ARGUMENT (#28352)
make STRLEFT and STRRIGHT function DEPEND_ON_ARGUMENT
2024-01-25 13:23:59 +08:00
88fdb2677d [fix](catalog) fix Uninitialized connection pool parameters in hmsjdbcclient (#30262) 2024-01-25 13:23:59 +08:00
2f68aac885 [Improvement](Nereids) Support to query rewrite by materialized view when join input has aggregate (#30230)
Support to query rewrite by materialized view when join input has aggregate, the aggregate should be simple
For example as following:
The materialized view def is 
>            select
>              l_linenumber,
>              count(distinct l_orderkey),
>              sum(case when l_orderkey in (1,2,3) then l_suppkey * l_linenumber else 0 end),
>              max(case when l_orderkey in (4, 5) then (l_quantity *2 + part_supp_a.qty_max) * 0.88 else 100 end),
>              avg(case when l_partkey in (2, 3, 4) then l_discount + o_totalprice + part_supp_a.qty_sum else 50 end)
>            from lineitem
>            left join orders on l_orderkey = o_orderkey
>            left join 
>              (select ps_partkey, ps_suppkey, sum(ps_availqty) qty_sum, max(ps_availqty) qty_max,
>                min(ps_availqty) qty_min,
>                avg(ps_supplycost) cost_avg
>                from partsupp
>                group by ps_partkey,ps_suppkey) part_supp_a
>              on l_partkey = part_supp_a.ps_partkey
>                and l_suppkey = part_supp_a.ps_suppkey
>            group by l_linenumber;

when query is like following, it can be rewritten by mv above
>            select
>              l_linenumber,
>              sum(case when l_orderkey in (1,2,3) then l_suppkey * l_linenumber else 0 end),
>              avg(case when l_partkey in (2, 3, 4) then l_discount + o_totalprice + part_supp_a.qty_sum else 50 end)
>            from lineitem
>            left join orders on l_orderkey = o_orderkey
>            left join 
>              (select ps_partkey, ps_suppkey, sum(ps_availqty) qty_sum, max(ps_availqty) qty_max,
>                min(ps_availqty) qty_min,
>                avg(ps_supplycost) cost_avg
>                from partsupp
>                group by ps_partkey,ps_suppkey) part_supp_a
>              on l_partkey = part_supp_a.ps_partkey
>                and l_suppkey = part_supp_a.ps_suppkey
>            group by l_linenumber;
2024-01-25 13:23:59 +08:00
f85b04c2c6 [fix](datatype) fixed decimal type implicit cast handling in BinaryPredicate (#30181) 2024-01-25 13:23:12 +08:00
79d178ca54 [test](ut) added UT cases for show create materialized view (#29563) 2024-01-25 13:23:12 +08:00
c7360fd014 [feature](function) support ip function named ipv4_cidr_to_range(addr, cidr) (#29819)
* support ip function ipv4_cidr_to_range

* fix ipv4_cidr_to_range function only support ipv4 type
2024-01-24 10:02:03 +08:00
dde5ed5231 [fix](fe-memory) Fix fe schema change high memory usage (#30231)
Signed-off-by: Jack Drogon <jack.xsuperman@gmail.com>
2024-01-24 10:02:03 +08:00
5213f941dd [improvement](cooldown) print the cooldown version when follow cooldown version (#30239) 2024-01-24 10:02:03 +08:00
040ace126c [fix](Nereids) group by key should be required (#30273) 2024-01-24 10:01:14 +08:00
2b3e7589b7 [fix](nereids)group by expr may be lost in EliminateGroupByConstant rule (#30274) 2024-01-24 10:01:14 +08:00
4af3fd2a2e [fix](Nereids) fix bug in case-when/if stats estimation (#30265) 2024-01-24 10:00:25 +08:00
1a51d04cb8 [fix](move-memtable) fix schema use-after-free in delta writer v2 (#30254) 2024-01-24 10:00:25 +08:00
5e66e2519d [improve](column) support append_data_by_selector function in const column (#29996)
support append_data_by_selector function in const column
2024-01-24 09:59:45 +08:00
Pxl
02a27a587a remove some unused member function of IFunctionBase (#30260) 2024-01-24 09:59:45 +08:00
72f4e7e2d1 [security] Don't print token (#30227) 2024-01-24 09:59:45 +08:00
b98acf2d90 [fix](mtmv)mtmv default open enableNereidsDML #30235 2024-01-24 09:59:45 +08:00
4cbacb5b39 [enhancement](recover) Support skipping bad tablet in select by session variable (#30241)
In some scenarios, user has a huge amount of data and only a single replica was specified when creating the table, if one of the tablet is damaged, the table will not be able to be select. If the user does not care about the integrity of the data, they can use this variable to temporarily skip the bad tablet for querying and load the remaining data into a new table.
2024-01-24 09:59:43 +08:00
1b9f1f6483 [feature](Planner): Push down TopNDistinct through Join (#30216)
Push down TopNDistinct through Outer/Cross Join
2024-01-24 09:59:13 +08:00
f4a10c3fbc [enhancement](Nereids): add builder for hyper graph (#30061) 2024-01-24 09:58:31 +08:00