Commit Graph

5188 Commits

Author SHA1 Message Date
12ed2951c4 [fix] (inverted index) remove tmp columns in block (#39369) (#39533) 2024-08-20 20:53:23 +08:00
621d394a5e [enhance](Backup) Do connectivity check when creating repository (#38350) (#39538)
Previously when creating repository, FE would not do connectivity check.
It might result in confusing error when using backup restore.

pick #38350

Co-authored-by: AlexYue <yj976240184@gmail.com>
2024-08-19 22:16:02 +08:00
fb17f204d7 [fix](http) fix http url with incorrect character notation (#38420) (#39535)
## Proposed changes

pick from master #38420
2024-08-19 15:03:19 +08:00
273a62584c [opt](inverted index) unified optimization judgment to prevent omissions (#39473)
https://github.com/apache/doris/pull/38027
2024-08-17 16:57:19 +08:00
b0da8430bc [opt](inverted index) Optimize the usage of the multi_match function (#39472)
## Proposed changes

https://github.com/apache/doris/pull/39193

<!--Describe your changes.-->
2024-08-17 16:53:52 +08:00
20936fe054 [branch-2.1][improvement](jdbc catalog) Compatible with ojdbc6 by adding version check (#39408)
pick (#39341)

In previous versions, we used a method based on JDBC 4.2 to read data,
so it was equivalent to abandoning support for ojdbc6. However, we
recently found that a large number of users still use Oracle version
11g, which will have some unexpected compatibility issues when using
ojdbc8 to connect. Therefore, I use version verification to make it
compatible with both ojdbc6 and ojdbc8, so that good compatibility can
be obtained through ojdbc6, and better reading efficiency can be
obtained through ojdbc8.
2024-08-17 16:43:01 +08:00
79aa079cc6 [fix](array-funcs) array min/max #39307 (#39484) 2024-08-17 10:56:44 +08:00
7687f2c53a [fix](ip-funcs) fix ip inet6_aton funcs #39415 (#39513) 2024-08-17 10:56:06 +08:00
472565cd48 [enhance](mtmv) refresh hms table before run mtmv task (#38212) (#39490)
pick from master #38212​
2024-08-16 20:05:52 +08:00
ae8073f155 [opt](mtmv) partition rollup support week and quarter (#39286) (#39477)
pick from master #39286​
2024-08-16 20:01:06 +08:00
2948b5ea2b [branch-2.1][fix](jdbc scan) Remove the conjuncts.remove call in JdbcScan (#39407)
pick (#39180)

In #37565, due to the change in the calling order of finalize, the final
generated Plan will be missing the PREDICATES that have been pushed down
in Jdbc. Although this behavior is correct, before perfectly handling
the push down of various PREDICATES, we need to keep all conjuncts to
ensure that we can still filter data normally when the data returned by
Jdbc is a superset.
2024-08-16 19:01:40 +08:00
4458302a77 [Fix](Planner) fix delete from using does not attach partition information (#39020) (#39418)
cherry-pick from master #39020
Problem:
when use delete from using clause and assign partition information, it
would delete more data from other partition
Solved:
add partition information when transfer delete clause into insert into
select clause
2024-08-16 17:16:08 +08:00
e249e00586 [fix](Nereids) fix explain plan with sql cache (#39431) (#39463)
introduced by #38950, explain plan with sql cache will throw an exception
```
errCode = 2, detailMessage = Cannot invoke "org.apache.doris.nereids.trees.plans.Plan.treeString()" because "this.optimizedPlan" is null
```
2024-08-16 15:51:47 +08:00
824f035b98 [pick](Row store) fix row store with invalid json string in variant ty… (#39456)
#39394
2024-08-16 14:43:11 +08:00
6aafc5adb4 [improvement](statistics)Support drop cached stats. (#39367) (#39462)
backport: https://github.com/apache/doris/pull/39367
2024-08-16 14:12:58 +08:00
d56000e924 [opt](Nereids) polish aggregate function signature matching (#39352) (#39460)
pick from master #39352

use double to match string
- corr
- covar
- covar_samp
- stddev
- stddev_samp

use largeint to match string
- group_bit_and
- group_bit_or
- group_git_xor

use double to match decimalv3
- topn_weighted

optimize error message
- multi_distinct_sum
- multi_distinct_sum0
2024-08-16 13:57:11 +08:00
ec0e413317 [fix](Nereids) npe when delete with cte and without using (#39379) (#39441)
pick from master #39379
2024-08-16 13:56:24 +08:00
eea3676791 [fix](group commit) fix group commit insert rpc may stuck (#39391) (#39458)
pick https://github.com/apache/doris/pull/39391
2024-08-16 13:19:00 +08:00
021678c7c3 [fix](window_funnel) fix wrong result of window_funnel #38954 (#39270)
## Proposed changes

BP #38954
2024-08-16 09:59:31 +08:00
289096692f [opt](Nereids) support parse sql with semicolon at beginning (#39399) (#39442)
pick from master #39399

For statement with semicolon at beginning like: ;SELECT 1; MySQL could
not parse it, but legacy planner could
2024-08-16 09:57:44 +08:00
4e889bbc6d [fix](Nereids) support implicit cast ip types to string (#39318) (#39440)
pick from master #39318
2024-08-16 09:57:02 +08:00
4380f3cb51 [fix](variable) support all type functions (#39144) (#39438)
pick from master #39144
2024-08-16 09:51:02 +08:00
887d4fc42e [chore](test) rename conflict name under same database in test cases (#39243) (#39439)
pick from master #39243
2024-08-16 09:50:28 +08:00
46dc0d2192 [fix](variant) fix variant cast (#39426)
## Proposed changes
backport: https://github.com/apache/doris/pull/39377
Issue Number: close #xxx

<!--Describe your changes.-->
2024-08-16 09:47:44 +08:00
3aaee8f7d5 [fix](Nereids) polish function signature search algorithm (#38497) (#39436)
pick from master #38497 and #39342

use array<double> for array<string>
- array_avg
- array_cum_sum
- array_difference
- array_product

use array<bigint> for array<string>
- bitmap_from_array

use double first
- fmod
- pmod

let high order function throw friendly exception
- array_filter
- array_first
- array_last
- array_reverse_split
- array_sort_by
- array_split

let return type same as parameter's type
- array_push_back
- array_push_front
- array_with_constant
- if

let greatest / least work same as mysql's greatest
2024-08-16 08:24:25 +08:00
6257e706fa [improve](ip)update ip for bloom_filter (#39414)
## Proposed changes
backport: https://github.com/apache/doris/pull/39253
Issue Number: close #xxx

<!--Describe your changes.-->
2024-08-16 08:20:19 +08:00
fff26fe2fc [fix](group commit) fix group commit core if be inject FragmentMgr.exec_plan_fragment.failed (#39339) (#39396)
pick https://github.com/apache/doris/pull/39339
2024-08-15 17:54:11 +08:00
b3597ea898 [improvement](statistics)Drop column stats after schema change. (#39101) (#39401)
backport: https://github.com/apache/doris/pull/39101
2024-08-15 17:02:42 +08:00
aebc70d75a revert [improvement](mv) Support to use cast when create sync materialized view #38008 (#39378)
## Proposed changes

this is brought by https://github.com/apache/doris/pull/38008
if use cast(FLOOR(MINUTE(time) / 15) as decimal(9, 0)) in group by
clause when sync materialized view. if downgrade from 2.1.6 to 2.1.5 or
upgrade 2.1.6 to 3.0.0
this may cause fe can not run. So revert the function.
2024-08-15 14:16:57 +08:00
4acd69590d [Fix](function) fix wrong nullable signature of function corr (#39380)
## Proposed changes

Issue Number: close #xxx

before `corr(nullable_x, nullable_y)` will core dump. not fixed.
no need to patch in master because the refactor
https://github.com/apache/doris/pull/37330 already changed the
implementation context
2024-08-15 14:10:09 +08:00
1accde9fb3 [fix](nestedtype) support nested type for schema change reorder (#39392)
## Proposed changes
backport: https://github.com/apache/doris/pull/39210
Issue Number: close #xxx

<!--Describe your changes.-->
2024-08-15 14:03:03 +08:00
7e9aa2b9ac [feature](restore) Support clean_tables/clean_partitions properties for restore job #39028 (#39363)
cherry pick from #39028
2024-08-15 09:58:26 +08:00
c12137a8d6 [branch-2.1][fix](expr) Enhance SQL Expression Handling by Introducing printSqlInParens to CompoundPredicate (#39082)
pick #39064
2024-08-14 21:14:58 +08:00
226e01889c [fix](array_apply) pick array apply fix (#39328)
## Proposed changes
backport: https://github.com/apache/doris/pull/39105
Issue Number: close #xxx

<!--Describe your changes.-->
2024-08-14 18:52:29 +08:00
78d6e318fb [fix](ip)pick ip rowstore (#39345)
## Proposed changes
backport: https://github.com/apache/doris/pull/39258
Issue Number: close #xxx

<!--Describe your changes.-->
2024-08-14 18:51:58 +08:00
b26af32934 [fix](function) fix error return type in corr(float32,float32) (#39251) (#39350)
https://github.com/apache/doris/pull/39251
```
mysql [test11]>select corr(cast(x as float),cast(y as float)) from test_corr;
ERROR 1105 (HY000): errCode = 2, detailMessage = (127.0.0.1)[INTERNAL_ERROR]column_type not match data_types in agg node, column_type=Nullable(Float64), data_types=Nullable(Float32),column name=

```

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
2024-08-14 18:47:14 +08:00
a9692a305e [fix](function)timediff with now function causes a error signature (… (#39349)
…#39322)
https://github.com/apache/doris/pull/39322
## Proposed changes

```
mysql [(none)]>select round(timediff(now(),'2024-08-15')/60/60,2);
ERROR 1105 (HY000): errCode = 2, detailMessage = argument 1 requires datetimev2 type, however 'now()' is of datetime type
```
The reason is that the function parameter types were modified in
expectedInputTypes, which led to no match being found. The code here is
from a long time ago. Because the precision of datetimev2 could not be
deduced in the past, a separate implementation was made here. This code
can be safely deleted.


<!--Describe your changes.-->

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
2024-08-14 18:36:14 +08:00
701e23b65b [Fix](nereids) fix condition function partition prune (#39298) (#39332)
cherry-pick #39298 to branch-2.1
2024-08-14 18:32:50 +08:00
d2709f8600 [fix](test) fix test_numbers case (#39303)
bp: #38687

we use `order_qt` rather than `qt` to promise the order of results.
2024-08-13 22:26:39 +08:00
aeffb18ba6 [test](mtmv)Add the cases of upgrade and downgrade of mtmv (#39289)
pick from #38868 

## Proposed changes

[test](mtmv)Add the cases of upgrade and downgrade of mtmv
2024-08-13 17:46:26 +08:00
677435cef8 [Pick](Branch-2.1) pick json reader fix and support specify $. as column (#39271)
#39206
#38213
2024-08-13 17:44:45 +08:00
Pxl
33220109f7 [Bug](materialized-view) fix analyze where clause failed on mv (#39061) (#39209)
## Proposed changes
pick from #39061
fix analyze where clause failed on mv
do not analyze slot after replaceSlot to avoid duplicate columns in desc
2024-08-13 16:08:20 +08:00
000ea20562 [fix](inverted index)Add exception check when write bkd index (#39248) (#39277)
bp #39248
2024-08-13 15:14:16 +08:00
43cc8d648d [fix](ES Catalog)Check isArray before parse json to array (#39104) (#39273)
## Proposed changes

bp #39104
2024-08-13 15:13:40 +08:00
8f9a858452 [fix](suites) fix backup cancelled cases #39204 (#39274)
cherry pick from #39204
2024-08-13 14:08:58 +08:00
Pxl
8cc9830b26 [Bug](http-api) fix core dump on /api/reset_rpc_channel coz exec_env not initialized (#39207)
pick from #38375
2024-08-13 11:54:36 +08:00
228f78b80d [fix] (nereids) fix Match Expreesion in filter estimation (#39050) (#39215)
## Proposed changes

pick from master #39050
2024-08-13 10:57:53 +08:00
60eeec3754 [fix] (inverted index) Fix match function without inverted index (#38989) (#39220)
## Proposed changes

pick from #38989
2024-08-13 10:55:54 +08:00
a6155a517d [fix] (topn) fix uncleared block in topn_next() (#39119) (#39224)
## Proposed changes

pick from master #39119
2024-08-13 10:34:17 +08:00
ee6f9d0e90 [branch-2.1](regression) Fix case test_schema_change_unique_mow (#39173)(#39196) (#39213)
## Proposed changes

picks https://github.com/apache/doris/pull/39173 and
https://github.com/apache/doris/pull/39196
2024-08-12 19:04:49 +08:00