Commit Graph

2600 Commits

Author SHA1 Message Date
e91d16854b [fix](function) fix date_format function execution error on fe (#31645) 2024-03-07 16:53:19 +08:00
21ce85dc14 [fix](money_format) fix money_format #31883 2024-03-07 16:53:19 +08:00
5905ffa1da [enhancement](nereids) allow reorder mark join (#30644) 2024-03-07 16:53:19 +08:00
9bf22a872a [Bug](fix) fix or and "<=>" cause coredump in query (#31884) 2024-03-07 16:53:19 +08:00
5b00f4fbeb [improvement](jdbc catalog) opt get db2 schema list & xml type mapping (#31856)
1. Trim Schema Names: Adapted the system to remove trailing spaces from DB2 schema names, ensuring compatibility without affecting query operations.
2. XML Mapping: Implemented a feature to directly map XML types to String.
2024-03-07 16:53:19 +08:00
561709451c [fix](Nereids) fix group_concat(distinct) failed (#31873) 2024-03-07 16:16:05 +08:00
1d9e9fc884 [regression test] Test the unique model by modify a value type from TINYINT to other type (#31682) 2024-03-07 16:16:05 +08:00
Pxl
3716f8a171 [Bug](partition) fix npe when prune partition with not exist partition column in mv #31860 2024-03-07 16:16:05 +08:00
Pxl
dc9de4b6b5 [Bug](load) fix wrong data in mv when routine load with function mapping (#31787) 2024-03-07 16:16:05 +08:00
4f174c4fb9 [feature](function) Support for aggregate function foreach combiner (#31526) 2024-03-06 13:08:30 +08:00
9af64d848f [fix](pipelineX) fix error distribution in DistinctStreamingAggOperatorX (#31804) 2024-03-06 13:08:30 +08:00
2e9bd268cd [improvement](jdbc catalog) support sqlserver timestamp type read (#31805) 2024-03-06 13:08:04 +08:00
aba58b0f7b [test](Nereids) add grouping sets test (#31675)
Co-authored-by: feiniaofeiafei <moailing@selectdb.com>
2024-03-06 13:08:04 +08:00
1d2d0bd411 [fix](update) Update set value should consider sequence column (#31626)
When using update command to set column value, if the column is sequence column, the column 'DORIS_SEQUENCE_COL' should also be set to the same value.
2024-03-06 13:08:04 +08:00
1434d3983b [enhancement](test) Test the unique model by modify a key type from TINYINT to other type (#31713) 2024-03-06 13:07:59 +08:00
2d6e975d5a [fix](cast) fix wrong result while cast string to float (#31781)
Issue Number: close #31518
2024-03-06 13:07:59 +08:00
97640ee0e8 [test](leading) add leading tpc-ds regression test cases (#31681)
Co-authored-by: libinfeng <libinfeng@selectdb.com>
2024-03-06 13:07:49 +08:00
7998da4691 [fix](cast) wrong result while cast const to double then to string (#31657)
Issue Number: close #31514
2024-03-06 13:06:27 +08:00
7c30cb20fd [Fix](partial update) Fix partial update load false when schema includes auto increment column (#31725)
Problem:
When partially updating columns without specifying the auto-increment column, and the imported data contains new keys, an error stating the auto-increment column could not be found occurs.

Reason:
The logic for partial column updates does not account for new keys in auto-increment columns. Since auto-increment columns can be generated by the system, it's possible to omit this column data during import. However, partial column updates treat this as a regular column, expecting it to be nullable or have a default value for automatic filling, overlooking the fact that auto-increment columns can also be auto-filled. This oversight leads to the error.

Solution:
Incorporate a check for auto-increment columns into the partial column update logic, and include the logic for generating auto-increment column values in the process of completing partial updates.
2024-03-06 13:06:27 +08:00
cca0773261 [feature](function) round function defaults to rounding normally 2024-03-06 13:06:26 +08:00
231768db0d [Performance](exec) Support runtime filter in <=> join (#31754) 2024-03-06 13:06:26 +08:00
3777ffb43f [enhancement](nereids)support null partition for list partition (#31613) 2024-03-06 13:05:22 +08:00
535e3b23ea [Fix](nereids) fix leading with left join without leftside condition (#31387)
When left join with no edge in outer side, we should add outer side to minimal require of left tables


Co-authored-by: libinfeng <libinfeng@selectdb.com>
2024-03-01 16:59:53 +08:00
07224686ef [feature](jdbc catalog) support db2 jdbc catalog (#31627) 2024-03-01 14:19:28 +08:00
2047b9416f [Fix](nereids) Only rewrite the slots that appear both in trival-agg func and grouping sets (#31600)
* [Fix](nereids) Only rewrite the slots that appear both in trival-agg func and grouping sets

* [Fix](nereids) Only rewrite the slots that appear both in trival-agg func and grouping sets

---------

Co-authored-by: feiniaofeiafei <moailing@selectdb.com>
2024-03-01 04:25:43 +08:00
1f825ee2d6 [improve](export) Support partition data consistency (#31290) 2024-03-01 04:25:43 +08:00
0aa7108ee2 [fix](merge-on-write) incorrect result caused by key range filter with pk (#31456) 2024-02-29 19:51:47 +08:00
92e3b31f50 [feature](invert index) match_phrase_edge feature added (#31142) 2024-02-29 19:51:18 +08:00
686938f5db [fix](nereids) window function with grouping sets work not well (#31475)
```sql        
select a, c, sum(sum(b)) over(partition by c order by c rows between unbounded preceding and current row) 
from test_window_table2 group by grouping sets((a),( c)) having a > 1 order by 1,2,3;
```
for this kind of case:
sum(sum(col)) over, nereids has cannot find slot problem.
the output slot of repeat and aggregate is computed wrongly.
Only collecting the trival-agg  in NormalizeRepeat can fix this problem.


Co-authored-by: feiniaofeiafei <moailing@selectdb.com>
2024-02-29 16:44:40 +08:00
9c4708ee74 [function](random_bytes)add random_bytes function (#31547)
SELECT random_bytes(10);

random_bytes(10) |
----------------------+
0x9b8ea00b7d1084bc5b26|
2024-02-29 16:44:39 +08:00
4636b6195b [Fix](JNI) fix BE core when using JNI to query the empty map type value (#31502) 2024-02-29 14:03:38 +08:00
ff3c7765d6 [Bug](Func) Fix negative number in bitmap return error result (#31563) 2024-02-29 12:38:03 +08:00
Pxl
6af6997f1d [Improvement](materialized-view) add approx_count_distinct/ndv to FunctionAlias rule (#31535)
add approx_count_distinct/ndv to FunctionAlias rule
2024-02-29 12:38:03 +08:00
3c37fb085c [refactor](jdbc catalog) split jdbc executor for different data sources (step-1) (#31406) 2024-02-29 12:38:03 +08:00
0fcdab468d [nereids] config global partition topn (#31476)
* [nereids] config global partition topn

* [nereids] config global partition topn

---------

Co-authored-by: zhongjian.xzj <zhongjian.xzj@zhongjianxzjdeMacBook-Pro.local>
2024-02-29 08:42:35 +08:00
4de25ede85 [fix](Nereids): other cond should be kept for each anti join when expanding anti join such as (#31521) 2024-02-29 08:42:35 +08:00
d88caca44a [fix](Nereids) push down topn distinct through join by mistake (#31396)
should not push down topn distinct through join when the output
columns of the corresponding child of join is more than
aggregate distinct columns.

for example for LEFT_OUTER_JOIN:

left child of join's output is: c1, c2, c3.
distinct columns is: c1, c2
topn: limit 2

if we push down topn distinct, we could get result of join like this:

```
c1    c2    c3, ...
1     2     1
1     2     2
```

and the final result we get is:

```
c1    c2
1     2
```

this is wrong, because we need 2 lines, but only return 1.
2024-02-28 17:51:32 +08:00
5824f8a4bc [Feat](nereids) support multi-leading (#30379)
support multi-leading in each query block
2024-02-28 13:07:47 +08:00
47d112e4e6 [fix](MySQL) implement SHOW CHARSET statement. (#31389) 2024-02-28 13:07:23 +08:00
481d94c3fc [feature](nereids) deal the slots that appear both in agg func and grouping sets (#31318)
this PR support slot appearing both in agg func and grouping sets.
sql like below:
select sum(a) from t group by grouping sets ((a)); 

Before this PR, Nereids throw exception like below:
col_int_undef_signed cannot both in select list and aggregate functions when using GROUPING SETS/CUBE/ROLLUP, please use union instead.

This PR removes the restriction and supports this situation.
2024-02-27 10:12:33 +08:00
1127b0065a [Improment](executor)Add scanbytes/scanrows condition (#31364)
* Add scanbytes/scanrows condition

* fix reg
2024-02-27 10:12:33 +08:00
f163d56a98 [feature](function) support sequence function(alias of array_range), enhance both to handle datetimev2 (#30823) 2024-02-27 10:12:19 +08:00
5d4a2d93a6 [feature](nereids) support join with joinRelation (#30909) 2024-02-26 19:07:11 +08:00
Pxl
3acffaa205 [Feature](agg-state) support write_column_to_pb from DataTypeFixedLengthObjectSerDe (#31171) 2024-02-26 19:07:10 +08:00
Pxl
3b7261abe4 [Mv] check delete from column exists on mv (#31321) 2024-02-26 19:07:10 +08:00
3451cd6c23 [fix](datetime) fix hour 24 on be (#31304) 2024-02-26 19:07:10 +08:00
7a9fe5d275 [enhance](mtmv)MTMV supports Hive multi-level partitioning (#31060)
Issue Number: close #xxx

For example, the hive table is partitioned by `date` and `region`, with the following 6 partitions
```
20200101
        beijing
        shanghai
20200102
        beijing
        shanghai
20200103
        beijing
        shanghai
```

If the MTMV is partitioned by `date`, then the MTMV will have three partitions: 20200101, 202000102, 20200103

If the MTMV is partitioned by `region`, then the MTMV will have two partitions: beijing, shanghai
2024-02-25 18:08:19 +08:00
93cbc96141 [regression test]Opt Test UniqueModel Schema Key Change (#31262)
* Opt Test UniqueModel Schema Key Change

* Opt Test UniqueModel Schema key Change
2024-02-24 16:26:44 +08:00
db58104bc3 [fix](inverted index) Fix inverted index for MOR unique table #31051 (#31354) 2024-02-23 23:10:36 +08:00
3576304e3e [regression test]Test UniqueModel Schema Value Change (#31260)
* Test UniqueModel Schema Value Change

* Opt Test UniqueModel Schema value Change
2024-02-23 19:03:28 +08:00