6913d68ba0
[Enhancement](merge-on-write) use delete bitmap to mark delete for rows with delete sign when sequence column doesn't exist ( #24011 )
2023-09-12 08:56:46 +08:00
6e28d878b5
[fix](hudi) compatible with hudi spark configuration and support skip merge ( #24067 )
...
Fix three bugs:
1. Hudi slice maybe has log files only, so `new Path(filePath)` will throw errors.
2. Hive column names are lowercase only, so match column names in ignore-case-mode.
3. Compatible with [Spark Datasource Configs](https://hudi.apache.org/docs/configurations/#Read-Options ), so users can add `hoodie.datasource.merge.type=skip_merge` in catalog properties to skip merge logs files.
2023-09-11 19:54:59 +08:00
115969c3fb
[opt](nereids) improve eliminate outerjoin in cascades ( #24120 )
...
* eliminate outer join cascading
2023-09-11 19:42:05 +08:00
9c441a4a16
[feature](Nereids) support create table and ctas ( #24150 )
...
Co-authored-by: sohardforaname <organic_chemistry@foxmail.com >
2023-09-11 12:37:58 +08:00
cd13f9e8c6
[BUG](view) fix can't create view with lambda function ( #23942 )
...
before the lambda function Expr not implement toSqlImpl() function.
so it's call parent function, which is not suit for lambda function.
and will be have error when create view.
2023-09-11 10:04:00 +08:00
dcde83d6e6
[Improve](regresstests)add boundary regress tests for map & array #24133
2023-09-11 08:28:11 +08:00
71db844c64
[feature](invert index) add tokenizer CharFilter preprocessing ( #24102 )
2023-09-10 23:08:28 +08:00
69f599bb53
[regression-test](fix)add test_ifnull. ( #23956 )
2023-09-10 12:11:43 +08:00
93c1151f1a
[fix](join) incorrect result of mark join ( #24112 )
2023-09-10 11:30:45 +08:00
232b58a27d
[fix](broker-load) make sequence column name case insensitive ( #24071 )
2023-09-10 10:51:07 +08:00
650af8f4df
[fix](test) fix broker load with default value test case ( #24123 )
2023-09-10 10:28:22 +08:00
f9a75b5c4f
[feature](csv_serde)1.append csv serde for serialize to csv and deserialize from csv. 2.let csvReader use csv serde not text_converter. ( #23352 )
...
1. append csv serde for serialize to csv and deserialize from csv.
2. let csvReader use csv serde not text_converter.
2023-09-10 00:16:21 +08:00
21e30d4374
[fix](planner)ctas's query part is not analyzed correctly ( #24111 )
...
* [fix](planner)ctas's query part is not analyzed correctly
2023-09-09 20:55:09 +08:00
8c2a721873
[opt](nereids)push down filter through window #23935
...
select rank() over (partition by A, B) as r, sum(x) over(A, C) as s from T;
A is a common partition key for all windowExpressions, that is A is intersection of {A,B} and {A, C}
we could push filter A=1 through this window, since A is a common Partition key:
select * from (select a, row_number() over (partition by a) from win) T where a=1;
origin plan:
----filter((T.a = 1))
----------PhysicalWindow
------------PhysicalQuickSort
--------------PhysicalProject
------------------PhysicalOlapScan[win]
transformed to
----PhysicalWindow
------PhysicalQuickSort
--------PhysicalProject
----------filter((T.a = 1))
------------PhysicalOlapScan[win]
But C=1 can not be pushed through window.
2023-09-09 20:53:31 +08:00
69868f18d6
[Bug](join) fix nested loop join some problems ( #24034 )
2023-09-08 17:40:41 +08:00
161520feb4
[feature](Nereids): enable convert CASE WHEN to IF ( #24050 )
...
enable rule to convert CASE WHEN to IF.
2023-09-08 16:58:33 +08:00
82dc970916
[feature](insert) Support group commit insert ( #22829 )
2023-09-08 15:51:03 +08:00
576855acb2
[fix](Nereids): fix regression-test ( #24065 )
2023-09-08 14:14:48 +08:00
b73f345479
[fix](intersect) fix wrong result of intersect node ( #24044 )
...
Issue Number: close #24046
2023-09-08 10:27:37 +08:00
a27349c83a
[fix](Export) Concatenation the outfile sql for Export ( #23635 )
...
In the original logic, the `Export` statement generates `Selectstmt` for execution. But there is no way to make the `SelectStmt` use the new optimizer.
Now, we change the `Export` statement to generate the `outfile SQL`, and then use the new optimizer to parse the SQL so that outfile can use the new optimizer.
2023-09-08 10:20:18 +08:00
0bdd078b41
[fix](jdbc catalog) fixed the sqlserver jdbc url parm concatenation error ( #23841 )
2023-09-08 09:58:20 +08:00
68acb8597b
[fix](nested_loop_join) null value should be output in semi-anti join ( #23971 )
...
create table t1
(k1 bigint, k2 bigint)
ENGINE=OLAP
DUPLICATE KEY(k1, k2)
COMMENT 'OLAP'
DISTRIBUTED BY HASH(k2) BUCKETS 1
PROPERTIES (
"replication_allocation" = "tag.location.default: 1",
"is_being_synced" = "false",
"storage_format" = "V2",
"light_schema_change" = "true",
"disable_auto_compaction" = "false",
"enable_single_replica_compaction" = "false"
);
create table t3
(k1 bigint, k2 bigint)
ENGINE=OLAP
DUPLICATE KEY(k1, k2)
COMMENT 'OLAP'
DISTRIBUTED BY HASH(k2) BUCKETS 1
PROPERTIES (
"replication_allocation" = "tag.location.default: 1",
"is_being_synced" = "false",
"storage_format" = "V2",
"light_schema_change" = "true",
"disable_auto_compaction" = "false",
"enable_single_replica_compaction" = "false"
);
Data:
insert into t1 values (1,null),(null,1),(1,2), (null,2),(1,3), (2,4), (2,5), (3,3), (3,4), (20,2), (22,3), (24,4),(null,null);
insert into t3 values (1,null),(null,1),(1,4), (1,2), (null,3), (2,4), (3,7), (3,9),(null,null),(5,1);
Query:
select t1.* from t1 where not exists ( select k1 from t3 where t1.k2 < t3.k2 );
Result:
Empty set
Expect result:
+------+------+
| k1 | k2 |
+------+------+
| NULL | NULL |
| 1 | NULL |
+------+------+
2023-09-08 09:28:55 +08:00
ab7c2b9d22
[Bug](type) fix wildcard char's tostring get wrong result ( #24041 )
...
fix wildcard char's tostring get wrong result
2023-09-07 20:25:38 +08:00
20b3e5eafe
[feature](Datetime) add from_microsecond / from_millisecond function ( #23902 )
2023-09-07 19:03:49 +08:00
fdb7a44f57
Revert "[Feature](partitions) Support auto partition" ( #24024 )
...
* Revert "[Feature](partitions) Support auto partition (#23236 )"
This reverts commit 6c544dd2011d731b8c9c51384c77bcf19c017981.
* Update config.h
2023-09-07 17:08:26 +08:00
9b494f4b36
[Fix](autoinc) skip to fill the auto increment column when the input column is not nullable ( #23905 )
2023-09-07 11:13:22 +08:00
45dfbeb8b3
[feature](agg) add the aggregation function 'array_agg' #23474
...
This function requires one arguments just as ARRAY_AGG(col) and col means the column whose values you want to aggregate.
This function Aggregates the values including NULL in a column into an array and returns a value of the ARRAY data type.
2023-09-07 10:21:55 +08:00
a532a08944
[Feature](Nereids)support insert overwrite for Nereids ( #23682 )
...
- support insert overwrite for Nereids
- Solving the problem of table ID changing（When no partition is specified，automatically specify all partitions for table ）
todo:
- Under the current processing logic, if the master restarts midway, it will generate dirty data (temp partition)
- for old planner ,insert overwrite table ,table id will change
2023-09-06 22:48:15 +08:00
6c544dd201
[Feature](partitions) Support auto partition ( #23236 )
...
Co-authored-by: zhangstar333 <2561612514@qq.com >
2023-09-06 16:26:45 +08:00
cb9acf4918
[fix](planner)fix 'char' function's toSql implementation is wrong ( #23860 )
2023-09-06 16:16:16 +08:00
a96adc01aa
[Chore](function) refactor of quantile_state ( #23862 )
...
refactor of quantile_state
2023-09-06 15:39:19 +08:00
7625d1514a
[pipelineX](feature) support select operator and add TPCH test cases ( #23954 )
2023-09-06 15:34:31 +08:00
dc28878f0e
[FIX](function) fix size function for array map ( #23920 )
...
Issue Number: close #xxx
now we use select size(map(1, 2)); which will make be core
and we can make size function handle array & map column both
2023-09-06 14:32:06 +08:00
2cb7536c6c
[fix](regression)fix case test_external_catalog_es ( #23908 )
2023-09-06 12:12:43 +08:00
b8c8e4ca96
[Fix](regression) fix wrong regression result ( #23944 )
2023-09-06 11:48:29 +08:00
527f87117b
[feature](nereids) support non_nullable function ( #23762 )
2023-09-06 08:52:47 +08:00
d7f1a4fb9a
[Fix](inverted index) need_read_data only supports DUP_KEYS ( #23818 )
...
need_read_data optimization does not work properly when table is UNQIUE_KEY MOR
2023-09-05 19:57:22 +08:00
d9461d77ba
[Fix](bitmap index) like predicate does not work in bitmap index ( #23819 )
2023-09-05 11:43:16 +08:00
4dac2d3b94
[Fix](Plan)StreamLoad cannot be parsed correctly when it contains complex where conditions ( #23874 )
2023-09-05 11:26:59 +08:00
e525e021ee
[Enhancement](Load) stream tvf support csv header ( #23797 )
...
Co-authored-by: yiguolei <676222867@qq.com >
2023-09-05 11:15:45 +08:00
a02ee8e1d0
[test](regression) Reduce dependency of other database in test_alter_table_column ( #22547 )
2023-09-05 10:15:01 +08:00
db306a51b3
[fix](nereids) missing return value of resetLogicalProperties() ( #23850 )
...
physicalPlan.resetLogicalProperties(); will not change the origin plan but create a new plan with no logical property. So should update the plan using resetLogicalProperties()'s return value.
2023-09-04 17:37:26 +08:00
1c73fd2687
[pipelineX](ssb) add SSB test cases ( #23844 )
2023-09-04 16:38:37 +08:00
422159bd94
[pipelineX](refactor) add repeat node in pipelineX ( #23750 )
2023-09-04 15:55:09 +08:00
bb3fadc5d3
[Bug](materialized-view) fix mv not match because cast and alias name ( #23580 )
...
fix mv not match because cast and alias name
2023-09-04 12:46:33 +08:00
a664bc5fca
[enhancement](jdbc catalog ) support postgresql partition table ( #23744 )
2023-09-03 10:52:01 +08:00
347cceb530
[Feature](inverted index) push count on index down to scan node ( #22687 )
...
Co-authored-by: airborne12 <airborne12@gmail.com >
2023-09-02 22:24:43 +08:00
228f0ac5bb
[Feature](Multi-Catalog) support query doris bitmap column in external jdbc catalog ( #23021 )
2023-09-02 12:46:33 +08:00
68aa4867b0
[fix](map_agg) lost scale information for decimal type ( #23776 )
2023-09-02 08:03:33 +08:00
657e927d50
[fix](json)Fix the bug that read json file Out of bounds access ( #23411 )
2023-09-02 01:11:37 +08:00
