Commit Graph

7925 Commits

Author SHA1 Message Date
ae56739f88 [enhancement](sequence col) add session variable to skip sequence column check while INSERT INTO (#41655) (#41720)
cp #41655
2024-10-12 15:30:20 +08:00
b2bac26c17 [fix](jdbc catalog) Disable oracle scan null operator pushdown (#41563) (#41712)
Because Oracle versions below Oracle21 do not support null as an
operator, and considering that most users' Oracle versions are below
Oracle21, we disable Oracle's null operator pushdown by default.
pick (#41563)
2024-10-11 21:01:05 +08:00
379e00f421 [improve](group commit) set internal group commit timeout (#41404) (#41688)
pick https://github.com/apache/doris/pull/41404
2024-10-11 17:55:43 +08:00
18cb395496 [fix] (inverted index) fix the error result in the query when using count on index (#41375) (#41690)
## Proposed changes

pick from master  #41375

<!--Describe your changes.-->
2024-10-11 17:15:14 +08:00
4ac07fe918 [Feature](json) Support json_search function in 2.1 (#41590)
cherry-pick #40948 

Like mysql, json_search returns the path which point to a json string
witch match the pattern.
`SELECT JSON_SEARCH('["A",[{"B":"1"}],{"C":"AB"},{"D":"BC"}]', 'one',
'A_') as res;`
```
+----------+
| res      |
+----------+
| "$[2].C" |
+----------+
```

Co-authored-by: liutang123 <liulijia@gmail.com>
2024-10-11 16:33:07 +08:00
8c0f73cb90 [Enhancement](MaxCompute)Refactoring maxCompute catalog using Storage API.(#40225 , #40888 ,#41386 ) (#41610)
bp #40225 , #40888 ,#41386

## Proposed changes
Among them, #40225 is the new api of mc,
#40888 is used to fix the bug when reading null between the new and old
apis,
#41386 is used for compatibility between the new and old versions
2024-10-11 11:55:41 +08:00
01acd60e2b [Pick][Improment]Add workload group query detail (#40050) (#41594)
pick #40050
2024-10-11 09:32:33 +08:00
0fb42d3a48 [Enhancement](tvf)catalog tvf implements user permission checks and hides sensitive information (#41497) (#41604)
bp #41497 

before #21790
## Proposed changes
This PR unifies the duplicate parts of `catalog tvf` and `show
catalogs`, adds permission check when querying `catalog tvf`, and hides
sensitive information.
2024-10-10 17:55:40 +08:00
3120bfb6e3 [fix](pipelinex) fix fragment instance progress reports (part 2) (#40694) (#41641)
backport #40694
2024-10-10 17:49:41 +08:00
070bd6cf99 [fix](jdbc) fix Unknown command(27) (#41621) (#41637)
pick https://github.com/apache/doris/pull/41621
2024-10-10 17:46:45 +08:00
cd26713c26 [fix](Nereids) could not parse date/datetime with blank + zone (#41394) (#41620)
pick from master #41394

for example:
2008-08-08 20:08:08 +08:00 parse failed because the blank before +08:00
2024-10-10 14:03:41 +08:00
33fad04341 [opt](Nereids) use 1 instead narrowest column when do column pruning (#41548) (#41627)
pick from master #41548
2024-10-10 14:02:23 +08:00
eb348ec10e [improve](binlog) Filter the truncated partitions #41611 (#41630)
cherry pick from #41611
2024-10-10 11:59:27 +08:00
6a3dc98361 [fix](Nereids) merge request properties map of group should compare cost (#40819) (#41615)
pick from master #40819

when do merge, we should update target requestPropertiesMap ONLY IF the
cost of source's request property lower than target one. Otherwise, the
requestPropertiesMap will not sync with lowestCostTable. Then, we will
get wrong output property when get the final plan.
2024-10-10 11:16:00 +08:00
032c00e3c9 [branch-2.1](create table) show failed detail msg #41463 (#41544)
cherry-pick: #41463
2024-10-10 11:15:02 +08:00
a45dc8796a [fix](Nereids) simplify decimal comparison wrong when cast to smaller scale (#41151) (#41618)
pick from master #41151
2024-10-09 23:03:01 +08:00
649cefd70f [opt](Nereids) forbid distribute under project and filter (#39812) (#41622)
pick from master #39812
2024-10-09 23:02:06 +08:00
1d2e2adb74 [fix](parser) should not use selectHint in any place (#41260) (#41619)
pick from master #41260

because all comment has been redirect to channel 2, we should not use
hint in any place. selectHint only use to parse hint.
2024-10-09 23:00:04 +08:00
ade86c0600 Revert "[branch-2.1][improvement](jdbc catalog) Optimize JdbcCatalog case mapping stability" (#41588)
Reverts apache/doris#41330
2024-10-09 20:51:58 +08:00
b6c0603e0d [fix](mtmv)Fix show mtmv time should wrapped in double quotation mark… (#41501)
pick: https://github.com/apache/doris/pull/41419
2024-10-09 12:19:28 +08:00
25684f487b [2.1][improvement](jdbc catalog) Improve JdbcClientException to accommodate various identifier formats (#41530)
pick (#40931)

In some cases, JDBC returns exceptions with various identifiers that
cannot be formatted correctly, such as `%`. This PR optimizes this.
2024-10-09 10:32:41 +08:00
ca59c8537c [chore](query err) fix cutting err msg incorrectly #41456 (#41538)
cherry pick from #41456
2024-10-08 16:09:15 +08:00
3355ebe5e7 [enhancement](delete) Using insert timeout session var to control delete job timeout (#41063) (#41475)
## Proposed changes

pick: #41063
2024-10-08 12:03:30 +08:00
4f81fc474c [bugfix](paimon)Get the file format by file name (#41020) (#41487)
bp #41020
2024-09-30 15:46:13 +08:00
b5e94b65c0 [fix](mtmv) Generate mtmv cache should use ADMIN user, and rewritten plan should not check privilege (#40374) (#41450)
## Proposed changes

pr: https://github.com/apache/doris/pull/40374
commitId: f3d92e48
2024-09-30 09:59:54 +08:00
a7b88af861 [bugfix](external)Memory leak problem for external table with insert operation for 2.1 (#40440) (#41479)
bp: #40440
2024-09-29 23:09:25 +08:00
9864324286 [enhancement](err-msg) Clarify error msg for forward queries (#40802) (#41474)
## Proposed changes

pick: #40802
2024-09-29 22:23:13 +08:00
9a9226e541 [fix](block_rule) SQL block rule not working after FE restart (#41228) (#41250)
pick: https://github.com/apache/doris/pull/41228
2024-09-28 10:08:59 +08:00
514184ee12 [fix](audit_loader) fix that old external audit loader plugin not work because of incompatibility with new audit plugin (#40565) (#41400)
bp #40565

Co-authored-by: caiconghui <55968745+caiconghui@users.noreply.github.com>
Co-authored-by: caiconghui1 <caiconghui1@jd.com>
2024-09-28 08:38:14 +08:00
afd87e9aee [fix](nereids)should prune logicalSink's child node in ColumnPruning (#41289) (#41424)
## Proposed changes

pick from master https://github.com/apache/doris/pull/41289

<!--Describe your changes.-->
2024-09-28 08:35:35 +08:00
6dd94c6bda [opt](log) add warn log for saving and pushing image failure (#41216) (#41434)
bp #41216
2024-09-28 08:34:45 +08:00
1aea05b5b0 [improvement](statistics)Support partition row count return -1 when it is not fully reported. (#41348) (#41408)
backport: https://github.com/apache/doris/pull/41348
2024-09-27 22:17:16 +08:00
0b4552f74b [cherry-pick](branch-2.1) pick hive text write from master (#40537)
## Proposed changes
pick prs:
https://github.com/apache/doris/pull/38549
https://github.com/apache/doris/pull/40183
https://github.com/apache/doris/pull/40315

---------

Co-authored-by: Calvin Kirs <kirs@apache.org>
2024-09-27 20:57:07 +08:00
82228358b9 [Fix](nereids) fix create view with nullable column (#41234) (#41393)
cherry-pick from master #41234
2024-09-27 19:13:54 +08:00
0c51ee26ea [fix](function) add time type in conditional-functions (#41270) (#41379)
## Proposed changes
https://github.com/apache/doris/pull/41270

<!--Describe your changes.-->
2024-09-27 17:19:54 +08:00
1baf0dbc59 [improvement](statistics)Reduce partition column sample BE memory consumption. (#41203) (#41387)
backport: https://github.com/apache/doris/pull/41203
2024-09-27 13:46:24 +08:00
21b8887b54 [feat](profile) Print changed session var in profile #41016 (#41317)
cherry pick from #41016
2024-09-27 09:55:43 +08:00
c487fc0e5f [enhance](mtmv)Optimize the logic of mtmv lock (#41010) (#41254)
pick: https://github.com/apache/doris/pull/41010
2024-09-26 23:02:02 +08:00
eb13cd4154 [branch-2.1] Picks "[Fix](partial update) Fix __DORIS_SEQUENCE_COL__ is not set for newly inserted rows in partial update #40272" (#40964)
picks https://github.com/apache/doris/pull/40272
2024-09-26 22:54:27 +08:00
e6ce4a2c26 [fix](meta) fix Unknown column 'mva_SUM__CAST` (#41285)
## Proposed changes

cherry pick from #41283
2024-09-26 22:52:51 +08:00
3eda77b3d9 [branch-2.1][improvement](jdbc catalog) Optimize JdbcCatalog case mapping stability (#41330)
pick #40891
This PR makes the following changes to the uppercase and lowercase
mapping of JdbcCatalog
1. The identifierMapping is managed by JdbcExternalCatalog instead of
JdbcClient to better control its lifecycle
2. The identifierMapping no longer loads remoteName alone, but Catalog
controls the loading uniformly
3. The identifierMapping will be loaded when each FE performs
makeSureInitialized() to ensure that each FE has a mapping
4. The initialization of mapping will only be performed once in
makeSureInitialized(), which means that even if you use metaCache, if
your source data is updated when identifierMapping is enabled, you must
refresh the catalog to query normally.
5. The identifierMapping is only responsible for the properties of the
Catalog and is no longer affected by the fe config, simplifying the
processing logic
6. If lower_case_mete_names is false and meta_names_mapping is empty in
the catalog properties, the identifierMapping will no longer take
effect, further enhancing the stability of the default settings
7. The JdbcClient is no longer closed during onRefreshCache, reducing
the repeated creation of resources, improving reuse, and reducing the
leakage of some global shared threads
2024-09-26 22:49:55 +08:00
5e6cd8749e [branch-2.1](hudi) fix fe memory leak(#41256) (#41225)
bp #41256
2024-09-26 22:34:16 +08:00
d89e5de815 [improvement](tablet scheduler) fix higher priority tablet add failed due to pending queue full #41076 (#41268)
cherry pick from #41076
2024-09-26 22:31:20 +08:00
f422b82e24 [Fix](Nereids) fix append_trailing_char_if_absent function return null (#41157)
cherry-pick: https://github.com/apache/doris/pull/40820
example: select append_trailing_char_if_absent('it','a') would return
null in original design, it can not return null when folding constant on
fe any time
2024-09-26 16:18:12 +08:00
43c1066bc7 [fix](analysis) Fix ColumnDef to sql result #41205 (#41325)
cherry pick from #41205
2024-09-26 14:11:13 +08:00
4deda2fce7 [improvement](nereids) Simplify ScanNode projection handling by removing redundant conditions (#40801) (#41315)
pick from master #40801

This PR simplifies the handling of `ScanNode` projection logic.
Previously, the code included multiple conditional checks to determine
whether a `projectionTuple` should be generated. These conditions have
been removed, and now `projectionTuple `is always generated for
`ScanNode`, ensuring a consistent projection setup. Additionally,
redundant handling of `SlotId` and `SlotRef` has been eliminated, making
the code cleaner and easier to maintain. The behavior for `OlapScanNode`
remains unchanged.
2024-09-26 10:35:01 +08:00
1459517568 [improvement](binlog) filter dropped indexes #41246 (#41300)
cherry pick from #41246
2024-09-26 08:38:28 +08:00
63555ea970 [fix](mysql)Support COM_CHANGE_USER and other mysql command. (#40932) (#41251)
backport: https://github.com/apache/doris/pull/40932
2024-09-25 15:37:02 +08:00
ba491af9f1 [branch-2.1] do not fallbacl to origin planner for call stmt (#41263)
`CALL xxx` statement is not supported by old planner.
And if fallback, the error message will be confused.
2024-09-25 14:30:56 +08:00
5bf210ffa4 [revert](storage) storage medium of partition should not inherit from… (#41212)
… table (#41192)
pick: https://github.com/apache/doris/pull/41192
2024-09-25 14:15:24 +08:00