Commit Graph

2848 Commits

Author SHA1 Message Date
a8ba933947 [Fix](nereids) fix bind order by expression logic (#33843) 2024-04-19 15:02:49 +08:00
2675e94a93 [feature](variable) add read_only and super_read_only (#33795) 2024-04-19 15:02:21 +08:00
5abc84af71 [fix](txn insert) Fix txn insert commit failed when schema change (#33706) 2024-04-19 15:01:57 +08:00
8f6f4cf0eb [Pick](Variant) pick #33734 #33766 #33707 to branch-2.1 (#33848)
* [Fix](Variant Type) forbit distribution info contains variant columns (#33707)

* [Fix](Variant) VariantRootColumnIterator::read_by_rowids with wrong null map size (#33734)

insert_range_from should start from `size` with `count` elements for null map

* [Fix](Variant) check column index validation for extracted columns (#33766)
2024-04-18 19:42:44 +08:00
34a97d5e8b [fix](Nereids)fix unstable plan shape in limit_push_down case 2024-04-18 19:05:24 +08:00
e106d34190 [enhancement](plsql) regression for routine select and show create procedure (#33608)
add regression for routines and show create procedure
Issue Number: close #31297

add regression for routines and show create procedure
2024-04-18 19:04:03 +08:00
04e30c91a0 [Fix](Variant) VariantRootColumnIterator::read_by_rowids with wrong null map size (#33734)
insert_range_from should start from `size` with `count` elements for null map
2024-04-18 19:02:58 +08:00
Pxl
8c535c51b5 [Improvement](materialized-view) support multiple agg function have same base table slot (#33774)
support multiple agg function have same base table slot
2024-04-18 19:02:49 +08:00
a57e0d3500 [Pick](nerids) pick #33010 #32982 #33531 to branch 2.1 (#33829) 2024-04-18 18:40:36 +08:00
d72d5c9b5d [fix](inverted index) normal process query for null condition when index is missing (#33663) 2024-04-17 23:42:14 +08:00
2648a92594 [FIX](load)fix load with split-by-string (#33713) 2024-04-17 23:42:14 +08:00
bb33375dba [test](xor) add test for xor #33731 2024-04-17 23:42:13 +08:00
81f7c53bad [fix](Nereids) could not query variant that not from table (#33704) 2024-04-17 23:42:13 +08:00
1c025c0488 [docker](hive) add hive3 docker compose and modify scripts (#33115)
add hive3 docker compose from:
big-data-europe/docker-hive#56
2024-04-17 23:42:13 +08:00
22a6b1d3f5 [feature](function) support hll functions hll_from_base64, hll_to_base64 (#32089)
Issue Number: #31320 

Support two hll functions:

- hll_from_base64
Convert a base64 string(result of function hll_to_base64) into a hll.
- hll_to_base64
Convert an input hll to a base64 string.
2024-04-17 23:42:13 +08:00
3096150d1b [feature](agg) support aggregate function group_array_intersect (#33265) 2024-04-17 23:42:13 +08:00
b07e0a2f06 [FIX](cast)fix full/right out join for cast array (#33475)
in some case, we has code
```
        if (_join_op == TJoinOp::RIGHT_OUTER_JOIN || _join_op == TJoinOp::FULL_OUTER_JOIN) {
            _probe_column_convert_to_null = _convert_block_to_null(*input_block);
        }
```
then do next function like cast , but in function cast we assume block column is same with from_type.which will make status error
2024-04-17 23:42:13 +08:00
d15981abd2 [Enhencement](Nereids) add rule of agg(case when) to agg(filter) (#33598) 2024-04-17 23:42:13 +08:00
8e38549a92 [fix](nereids) Use correct PREAGGREGATION in agg(filter(scan)) (#33454)
1. set `PreAggStatus` to `ON` when agg key column by max or min;
2. #28747 may change `PreAggStatus` of scan, inherit it from the previous one.
2024-04-17 23:42:13 +08:00
7b16cb5a4c [feature](inverted index) add slop functionality to match_phrase (#33225)
https://github.com/apache/doris-website/pull/553 doc
2024-04-17 23:42:12 +08:00
06a155abb0 [branch-2.1](cherry-pick) Pick some partial-update PR from master (#33639)
* [Fix](partial-update) Fix partial update fail when the datetime default value is 'current_time' (#32926)

* Problem: When importing data that includes datetime with a default value of current time for partial column updates, the import fails.
Reason: Partial column updates do not handle the logic for datetime default values.
Solution: During partial column updates, when the default value is set to current time, read the current time from the runtime state and write it into the data.

* [Enhancement](partial update)Add timezone case for partial update timestamp #33177

* [fix](partial update) Support partial update when the date default value is 'current_date'. This PR is a extension of PR #32926. (#33394)
2024-04-17 23:42:12 +08:00
4740b22481 [fix](test) fix some p2 external table test cases (#33624)
bp #33621
Also fix a merge bug from #33245
2024-04-17 23:42:12 +08:00
1be753ed75 [enhancement](mysql compatible) add user and procs_priv tables to mysql db in all catalogs (#33058)
Issue Number: close #xxx

This PR aims to enhance the compatibility of BI tools (such as Dbeaver, DataGrip) when using the mysql connector to connect to Doris, because some BI tools query some tables in the mysql database. In our tests, the user and procs_priv tables were mainly queried. This PR adds these two tables and adds actual data to the user table. However, please note that most of the fields in the user table are in Doris' own format rather than mysql format, so it can only ensure that the BI tool is querying No error is reported when accessing these tables, which does not guarantee that the data is completely displayed, and the tables under Doris's mysql database do not support data modification.
Thanks to @liujiwen-up for assisting in testing
2024-04-17 23:42:12 +08:00
48880c3e1a [Fix](timezone) fix miss of expected rounding of Date type with timezone #33553 2024-04-17 23:42:11 +08:00
1f5116f3c1 [testcases](auto-partition) Add and fix testcases in P0 #33588 2024-04-17 23:42:11 +08:00
379c2e0762 [Fix](Nereids) fix leading hint should have all tables in one query block (#33517) 2024-04-17 23:42:00 +08:00
ae4a7c93ac [feat](nereids) support create view in nereids (#32743) 2024-04-17 23:42:00 +08:00
249a9c9875 [Feature](Variant) support aggregation model for Variant type (#33493)
refactor use `insert_from` to replace `replace_column_data` for variable lengths columns
2024-04-17 23:42:00 +08:00
6bcf24b1f6 [bug](not in) if not in (null) could eos early (#33482)
* [bug](not in) if not in (null) could eos early
2024-04-17 23:41:59 +08:00
9dfc9665fb [Fix](inverted index) fix build index error status when batch_next (#33532) 2024-04-17 23:41:59 +08:00
09fb30c989 (Chore)[regression-test] fix unstable output variant case (#33520) 2024-04-17 23:41:59 +08:00
82d2bde3c7 [fix](nereids) do not transpose semi join agg when mark join (#32475) 2024-04-17 23:41:59 +08:00
272269f9c1 [Fix](inverted index) fix fast execute problem when need read data opt enabled (#33526) 2024-04-17 23:41:59 +08:00
e26a53d8a6 [fix](nereids) SemiJoinSemiJoinTransposeProject rule didn't handle mark join correctly (#33401) 2024-04-12 15:09:25 +08:00
78b81d4150 [fix](test) remove distribute node of shape in some regression test (#33463) 2024-04-12 15:09:25 +08:00
d2f84229ec [chore](test) remove some outdated datetime test case (#33476) 2024-04-12 15:09:25 +08:00
b035c7ceb4 [fix](catalog) fix resource is not reopen when rename catalog (#33432)
During the renaming of `JdbcCatalog`, I noticed that the `jdbcClient` was being closed, 
resulting in exceptions during subsequent queries. This happens because the `removeCatalog` 
method is invoked when changing the name, which in turn calls the `onClose` method of the catalog. 
Ideally, the client should not be closed when renaming the catalog. 
However, to avoid extra checks in the `removeCatalog` method, we can simply execute `onRefresh` 
in the `addCatalog` method to address this issue.
2024-04-12 15:09:25 +08:00
9b7af4c0cf [feature](schema change) unified schema change for parquet and orc reader (#32873)
Following #25138, unified schema change interface for parquet and orc reader, and can be applied to other format readers as well.
Unified schema change interface for all format readers:
- First, read the data according to the column type of the file into source column;
- Second, convert source column to the destination column with type planned by FE.
2024-04-12 15:09:25 +08:00
6f96e2b64a [fix](plsql) Fix handle select that fe can do without be (#33363)
CREATE OR REPLACE PROCEDURE procedure_test1()
BEGIN
select 1;
END; 

call procedure_test1()

fix `ERROR 2027 (HY000): Malformed packet`
2024-04-12 15:09:25 +08:00
215f402df7 [fix](nereids)when clause cannot be regarded as common sub expression (#33358)
* when clause cannot be regarded as common sub expression
2024-04-12 15:09:25 +08:00
18fb8407ae [feature](insert)use optional location and add hive regression test (#33153) 2024-04-12 10:38:54 +08:00
07f296734a [regression](insert)add hive DDL and CTAS regression case (#32924)
Issue Number: #31442

dependent on #32824

add ddl(create and drop) test
add ctas test
add complex type test
TODO:
bucketed table test
truncate test
add/drop partition test
2024-04-12 10:24:23 +08:00
716c146750 [fix](insert)fix hive external return msgs and exception and pass all columns to BE (#32824)
[fix](insert)fix hive external return msgs and exception and pass all columns to BE
2024-04-12 10:23:52 +08:00
9ada38327b [feature](txn insert) txn insert support insert into select (#31666) 2024-04-12 10:11:22 +08:00
3d66723214 [branch-2.1](auto-partition) pick auto partition and some more prs (#33523) 2024-04-11 17:12:17 +08:00
e4eb76212a [fix](Nereids): add order for constraint test (#33323) 2024-04-11 09:31:50 +08:00
Pxl
3070eda58c [Bug](load) fix stream load file on hll type mv column (#33373)
fix stream load file on hll type mv column
2024-04-11 09:31:50 +08:00
ea1e542e31 [fix](partial-update) remove unnecessary DECHEK on IndexChannel::num_rows_filtered (#33160) 2024-04-11 09:31:50 +08:00
Pxl
6462264e77 [Improvement](materialized-view) adjust priority of materialized view match rule (#33305)
adjust priority of materialized view match rule
2024-04-10 16:23:04 +08:00
4eee1a1f0d [fix](nereids) make runtime filter targets in fixed order (#33191)
* make runtime filter targets in fixed order
2024-04-10 16:22:39 +08:00