3d6689d32c
[fix](restore) Reset next version for restored partitions ( #38343 )
...
Cherry-pick #38321
2024-07-25 16:07:54 +08:00
32530113cf
[cherry-pick](branch-2.1) Fix some group commit forward to master problems ( #38345 )
...
## Proposed changes
<!--Describe your changes.-->
Pick #38228 #38265
2024-07-25 14:48:02 +08:00
a8139d9f39
[Improvement](profile) Provide more info for schedule time ( #38290 ) ( #38338 )
...
## Proposed changes
pick #38290
<!--Describe your changes.-->
2024-07-25 14:43:52 +08:00
e396f853a0
Pick "[enhance](Cooldown) Use config to control whether use cooldown replica for scanning first" ( #38322 )
...
## Proposed changes
<!--Describe your changes.-->
Same as master #37492
2024-07-25 12:17:38 +08:00
c23228e331
[Improvement](profile) add catalog info in profile ( #38302 )
...
bp #38283
2024-07-25 11:57:42 +08:00
bc7fc4106d
[branch-2.1](function) fix FE impl of some time functions ( #37746 ) ( #38316 )
...
pick https://github.com/apache/doris/pull/37746
before:
```sql
mysql> select date_ceil("2020-12-12 12:12:12.123", interval 2 second);
+-----------------------+
| '2020-12-12 12:12:12' |
+-----------------------+
| 2020-12-12 12:12:12 |
+-----------------------+
1 row in set (0.10 sec)
mysql> select CONVERT_TZ('9999-12-31 23:59:59.999999', 'Pacific/Galapagos', 'Pacific/Galapagos');
+------+
| NULL |
+------+
| NULL |
+------+
1 row in set (0.09 sec)
mysql [(none)]>select CONVERT_TZ('9999-12-31 23:59:59.999999', 'Pacific/Galapagos', 'Pacific/GalapaGoS');
+-----------------------------------------------------------------------------------------------------------+
| convert_tz(cast('9999-12-31 23:59:59.999999' as DATETIMEV2(6)), 'Pacific/Galapagos', 'Pacific/GalapaGoS') |
+-----------------------------------------------------------------------------------------------------------+
| 9999-12-31 23:59:59.999999 |
+-----------------------------------------------------------------------------------------------------------+
1 row in set (0.08 sec) --- gone to BE
```
after:
```sql
mysql> select date_ceil("2020-12-12 12:12:12.123", interval 2 second);
+------------------------------+
| '2020-12-12 12:12:14.000000' |
+------------------------------+
| 2020-12-12 12:12:14 |
+------------------------------+
1 row in set (0.11 sec)
mysql> select CONVERT_TZ('9999-12-31 23:59:59.999999', 'Pacific/Galapagos', 'Pacific/Galapagos');
+-----------------------------------------------------------------------------------------------------------+
| convert_tz(cast('9999-12-31 23:59:59.999999' as DATETIMEV2(6)), 'Pacific/Galapagos', 'Pacific/Galapagos') |
+-----------------------------------------------------------------------------------------------------------+
| 9999-12-31 23:59:59.999999 |
+-----------------------------------------------------------------------------------------------------------+
1 row in set (0.23 sec)
mysql> select CONVERT_TZ('9999-12-31 23:59:59.999999', 'Pacific/Galapagos', 'Pacific/GalapaGoS');
+------------------------------+
| '9999-12-31 23:59:59.999999' |
+------------------------------+
| 9999-12-31 23:59:59.999999 |
+------------------------------+
1 row in set (0.11 sec) --- finished in FE
```
2024-07-25 11:38:27 +08:00
8ad4390edb
[fix](nereids) refine row count estimation for mark join ( #38270 ) ( #38297 )
...
pick from master #38270
2024-07-25 10:19:13 +08:00
e23c1339a8
[fix](group commit) Fix prepare stmt setNull return too many filtered rows error ( #38262 ) ( #38276 )
...
Pick https://github.com/apache/doris/pull/38262
2024-07-24 19:02:59 +08:00
17c33665e5
[fix](agg_state) adjust nullable should apply on agg_state inner type too ( #37489 ) ( #38281 )
...
pick from master #37489
after adjust nullable, some children nullable has changed. so, we need
to update agg_state type inner type nullable too.
2024-07-24 13:55:09 +08:00
792bd7c74a
[cherry-pick](branch-2.1) Pick "[Enhancement](group commit)Optimize be select for group commit #35558 " ( #37830 )
...
Pick #35558
2024-07-24 09:21:07 +08:00
798b8573c9
[fix](load data) decommission replica don't load data when it misses versions #38198 ( #38256 )
...
cherry pick from #38198
2024-07-23 20:28:05 +08:00
b591837e13
[fix](insert) Pick Insert init plan error may forget to abort txn ( #34220 ) ( #38260 )
...
Pick https://github.com/apache/doris/pull/34220
2024-07-23 20:21:38 +08:00
ca7d8325af
[Fix](Nereids) TopN should forbid two-phase topN if child is DistributionSpecGather ( #36877 ) ( #38164 )
...
pick from master #36877
2024-07-23 16:27:55 +08:00
ba5d23aea8
[enhancement](log) Changed log of waiting previous transactions to info level for debug convenience ( #38133 ) ( #38230 )
...
## Proposed changes
As title.
2024-07-23 14:16:04 +08:00
33e04b12ff
[conf](parallel) Reduce parallel tasks for large cluster ( #38196 ) ( #38233 )
...
For large cluster, too many parallel tasks will cause performance issue.
So this PR limit the max parallel tasks in Doris.
pick #38196
<!--Describe your changes.-->
2024-07-23 13:42:31 +08:00
fdc8b454d2
[chore](backup) Remove delete_if_exists properties for creating repository ( #38192 )
...
delete_if_exists is a temporary solution introduced in #25847 , to avoid
concurrent testing conflicts.
Cherry-pick #38190
2024-07-23 11:09:02 +08:00
41b355721f
[fix](multicatalog) make lastdbofcatalog a session variable ( #38114 )
...
bp #37828
2024-07-23 10:34:40 +08:00
193be20c86
[feature](csv)Supports reading CSV data using LF and CRLF as line separators. ( #37687 ) ( #38099 )
...
bp #37687
2024-07-22 22:53:04 +08:00
87306858a1
[Fix](JobManager)Release the lock immediately after modifying job metadata to avoid holding the lock for an extended period.#38162 ( #38163 )
...
## Proposed changes
#38162
<!--Describe your changes.-->
2024-07-20 10:52:21 +08:00
b24774c340
[branch-2.1][fix](compatibility) Enhance SSL and Long Flag Handling in MySQL Handshake ( #38140 )
...
pick(#38086 )
Issue Number: close #38065
This PR introduces a fix to the SSL and CLIENT_LONG_FLAG handling in the
MySQL handshake process. It ensures that the flags are correctly set
according to the protocol requirements and resolves the issues related
to decimal data fetching when SSL is enabled.
2024-07-19 22:50:51 +08:00
dfc82a4011
[fix](inverted index)Support rename column with inverted index ( #38079 ) ( #38121 )
...
## Proposed changes
backport #38079
2024-07-19 17:58:49 +08:00
27d7461644
[Opt](Iceberg) Simplify the code of getting time travel snapshotId ( #34299 ) ( #38101 )
...
bp #34299
Co-authored-by: Butao Zhang <zhangbutao@cmss.chinamobile.com >
2024-07-19 09:45:56 +08:00
bb2b7774df
[feature](iceberg) iceberg write support insert overwrite and optimize hive write transaction statistics and ( #37191 ) ( #38097 )
...
bp #37191
Co-authored-by: kang <35803862+ghkang98@users.noreply.github.com >
Co-authored-by: lik40 <lik40@chinatelecom.cn >
2024-07-19 09:45:41 +08:00
301ff6af22
[Fix](nereids) add backquote and qualifier to udf, column, column alias and table alias when create view ( #37237 ) ( #37984 )
...
cherry-pick #37237 to branch-2.1
---------
Co-authored-by: feiniaofeiafei <moailing@selectdb.com >
2024-07-19 00:56:26 +08:00
5d17ff9d95
[fix](mtmv)Mtmv support force replica 2.1 ( #38090 )
...
master do not have bug, because has call
`PropertyAnalyzer.getInstance().rewriteOlapProperties`
2.1 not have this method ,should call
`PropertyAnalyzer.rewriteReplicaAllocationProperties`
2024-07-18 23:52:34 +08:00
426493a7f1
[cherrypick] Fix profile eject ( #38095 )
...
pick https://github.com/apache/doris/pull/38030
2024-07-18 23:38:56 +08:00
6733c7c226
[chore](fe) Returns dropped partitions in GetMeta request ( #38069 )
...
Cherry-pick #37196 , #37326 .
The CCR syncer needs to know the distribution of tables, partitions,
indexes, and replicas when synchronizing binlogs. If a partition is
deleted before the binlog synchronization is complete, the CCR syncer
cannot continue synchronizing. This PR will record the deleted
partitions and include them in the get meta response, allowing the CCR
syncer to filter out the binlogs that belong to these partitions.
The CCR syncer part PR is selectdb/ccr-syncer#117 .
2024-07-18 21:20:57 +08:00
4f03bde1e8
[Feat](job)scheduled job allows the time to be set in the past ( #36732 ) ( #37573 )
...
## Proposed changes
In some cases, users want to set the start time to a time in the past.
**For periodic scheduling tasks only, the start time of a one-time task
must be set to the future or present time**
**If the start time is the past time, then the next execution time
should be the start time + time interval.**
(cherry picked from commit fa6061070f6d4063b1966e99fc285e4e6b9c3750)
#36732
2024-07-18 19:15:10 +08:00
c30c1d2436
[branch-2.1] Picks "[opt](delete) Delete job should retry for failure that is not DELETE_INVALID_XXX #37834 " ( #38032 )
...
## Proposed changes
picks https://github.com/apache/doris/pull/37834 and
https://github.com/apache/doris/pull/38043
2024-07-18 14:50:30 +08:00
7dd01b1fa9
[fix](publish) Pick Catch exception in genPublishTask to make one failed txn does not block the other txns ( #37724 ) ( #38042 )
...
Pick https://github.com/apache/doris/pull/37724
2024-07-18 14:15:49 +08:00
ebc178af23
[fix](nereids)acos function should return null literal instead of NaN value ( #37996 )
...
pick from master https://github.com/apache/doris/pull/37932
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
2024-07-18 09:28:56 +08:00
dcfc72bb36
[fix](nereids) bug: after is-null stats derive, other column stats are dropped ( #37809 ) ( #38024 )
...
## Proposed changes
pick from #37809
Issue Number: close #xxx
<!--Describe your changes.-->
2024-07-18 08:11:09 +08:00
b16cd30a20
[opt](inverted index) set support_phrase default true if parser is set ( #38028 )
...
## Proposed changes
https://github.com/apache/doris/pull/37949
<!--Describe your changes.-->
2024-07-18 08:07:16 +08:00
629d7ff048
[fix](nereids) fix aggr node colocate flag local shuffle depends on ( #38016 )
...
## Proposed changes
pick from https://github.com/apache/doris/pull/37788
Co-authored-by: zhongjian.xzj <zhongjian.xzj@zhongjianxzjdeMacBook-Pro.local >
2024-07-17 21:34:30 +08:00
78d51ca320
[fix](Nereids) tablet prune wrong when decimal value scale is nagtive ( #37889 ) ( #38013 )
...
pick from master #37889
we use unscaled value of BigDecimal in tablet prune. So we need to
ensure BigDecimal's precision and scale is same with the literal
contains it.
2024-07-17 20:12:14 +08:00
7180290d68
[fix](dynamic partition) fix create dynamic partition with overlap not throw exception #37924 ( #37964 )
...
cherry-pick: #37924
2024-07-17 19:13:35 +08:00
2e6f49af15
[opt](Nereids) support no-key hint parameter ( #37720 ) ( #37988 )
...
pick from master #37720
support hint use parameter without key, like:
```sql
SELECT /*+ query_timeout(3000) */ * FROM t;
```
2024-07-17 16:59:25 +08:00
b2a4cff51b
[fix](nereids)fix nullable property of ForEachCombinator ( #37980 )
...
## Proposed changes
pick from master https://github.com/apache/doris/pull/37796
<!--Describe your changes.-->
2024-07-17 13:48:21 +08:00
f2cf8b7d6f
[Fix](InternalSchema) Compute nodes should not be used for Internal schema three replica ( #36130 ) ( #37961 )
...
bp #36130
Co-authored-by: HB <137497191@qq.com >
Co-authored-by: camby <104178625@qq.com >
2024-07-17 13:45:32 +08:00
ba66ff5768
[fix](multi-catalog)fix paimon meta properties convert ( #37249 ) ( #37958 )
...
bp #37249
Co-authored-by: slothever <18522955+wsjz@users.noreply.github.com >
2024-07-17 01:04:33 +08:00
dbd2b0abf7
[improvement](mtmv) improve mv rewrite performance by reuse the shuttled expression ( #37197 ) ( #37935 )
...
## Proposed changes
chrry-pick 2.1
pr: https://github.com/apache/doris/pull/37197
commitId: 701c7db4
2024-07-17 00:52:58 +08:00
f7068b5658
[cherry-pick](branch-2.1) Make doris read hive text table parameters and behavior consistent with hive ( #37840 )
...
## Proposed changes
pick from master https://github.com/apache/doris/pull/37638
<!--Describe your changes.-->
2024-07-16 22:24:50 +08:00
49faccbf50
[enhancement](nereids) speedup sql cache with use variable as partition predicate ( #37943 )
...
follow up #37090
support reuse sql cache when use variable as partition predicate and variable change:
```sql
set @dt='2024-07-16';
-- create cache 1
select * from tbl where dt = @dt;
set @dt='2024-07-17';
-- create cache 2, will not invalidate cache 1
select * from tbl where dt = @dt;
set @dt='2024-07-16';
-- reuse cache 1
select * from tbl where dt = @dt;
```
2024-07-16 22:11:18 +08:00
2edb9501b5
[Pick]Fix show role stmt missing grouo info ( #37920 )
...
## Proposed changes
pick #36032
2024-07-16 20:41:59 +08:00
adf11737f0
[fix](catalog)Fix internal program error causing client to get stuck … ( #37821 )
...
…(#37551 )
pick: https://github.com/apache/doris/pull/37551
2024-07-16 20:22:47 +08:00
9ff129b630
[fix](stream_load) fix stream load may failed caused by column name with keyword ( #35822 ) ( #37890 )
...
#35938 #35822
let
KW_SQL,
KW_CACHE,
KW_COLOCATE,
KW_COMPRESS_TYPE,
KW_DORIS_INTERNAL_TABLE_ID,
KW_HOTSPOT,
KW_PRIVILEGES,
KW_RECENT,
KW_STAGES,
KW_WARM,
KW_UP,
KW_CONVERT_LSC
be as non-reserved
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
---------
Co-authored-by: caiconghui1 <caiconghui1@jd.com >
2024-07-16 20:20:34 +08:00
faa425bee5
[fix](colocate) fix colocate join while multi tables ( #37729 ) ( #37859 )
...
cherry-pick #37729 to branch-2.1
2024-07-16 19:02:36 +08:00
81a7542cae
[pick]Add audit log event queue size limit ( #37914 )
...
## Proposed changes
pick #37786
2024-07-16 19:00:22 +08:00
638d4b6f27
[Bug](function) Fix function for cast string as date/datetime ( #35637 ) ( #37891 )
...
## Proposed changes
Issue Number: close #35635
cherry-pick https://github.com/apache/doris/pull/35637 from master to
branch-2.1
<!--Describe your changes.-->
Cast rules：Consistent with mysql.
String：Date
The first part is 1-digit x: 000x-month-day
The first part is 2-digit xy: 20xy-month-day / 19xy-month-day The first
part is 3-digit xyz: 20xy-0z-day / 19xy-0z-day The first part is 4-digit
xyzw: xyzw-month-day
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
2024-07-16 17:22:03 +08:00
9fd4795080
[opt](iceberg)Add a new appearance to display the pushDown count for 2.1 ( #37046 ) ( #34928 ) ( #37810 )
...
## Proposed changes
bp: #37046 #34928
2024-07-16 16:03:44 +08:00
