4e453dc1bb
Revert "[improvement](statistics)Return -1 to neredis if report olap table row count for new table is not done for all tablets. ( #40457 )" ( #40616 )
...
Reverts apache/doris#40540
2024-09-10 17:17:13 +08:00
e43e6e2bba
[improvement](statistics)Return -1 to neredis if report olap table row count for new table is not done for all tablets. ( #40457 ) ( #40540 )
...
backport: https://github.com/apache/doris/pull/40457
2024-09-10 12:55:53 +08:00
db8fb66dba
[fix](mtmv)fix nested mtmv not refresh ( #40433 ) ( #40560 )
...
pick: https://github.com/apache/doris/pull/40433
2024-09-10 11:51:41 +08:00
05f82df1f1
[fix](restore) update is_being_synced properties #40194 ( #40556 )
...
cherry pick from #40194
2024-09-10 11:15:50 +08:00
8eda15ae16
[opt](routine load) support routine load perceived schema change ( #39412 ) ( #40508 )
...
pick #39412
At present, if the table structure changes, the routine load cannot
perceive it. As a long-running load, it should be able to perceive the
changes in the table structure.
2024-09-10 11:05:58 +08:00
978fcf12b6
[Chore](dependnecies)remove javax.el and upgrade jackson to 2.16.0 ( #38843 ) ( #39389 )
...
…
## Proposed changes
javax.el is the API for Java Expression Language, which provides a
simple and flexible way for Java Web applications to access and
manipulate data. EL expressions are commonly used in JSP pages, but we
are not involved in their use here, so I removed it.
- upgrade jackson to 2.16.0
(cherry picked from commit 33a0ea5565c2e0d7e5909ca3b512fb0b539f05f4)
#38843
Issue Number: close #xxx
<!--Describe your changes.-->
2024-09-09 21:58:47 +08:00
68d1064aa9
[chore](persist) reduce truncate table operation log size #40422 ( #40555 )
...
cherry pick from #40422
2024-09-09 20:01:07 +08:00
8f37eccbf2
[Cherry-pick](branch-2.1) Pick "[Featrue](default value) Support bitmap_empty default value ( #40364 )" ( #40487 )
...
## Proposed changes
Pick #40364
<!--Describe your changes.-->
2024-09-09 16:57:38 +08:00
2235c1abd3
[Feat](OSS)Decouple COS and OBS dependencies with optional inclusion mechanism ( #39831 )
...
…
improve the flexibility of the project by decoupling direct dependencies
on the hadoop-cos and hadoop-huaweicloud libraries. These changes allow
users to control whether COS and OBS dependencies are included in the
final build, enabling a more customizable setup.
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
2024-09-09 16:50:12 +08:00
9c9827b92e
[chore](routine load) make error msg clear if routine load name illegal ( #40037 ) ( #40510 )
...
pick (#40037 )
2024-09-09 15:47:18 +08:00
5f4d1931ff
[feature](ES Catalog)Add FE open API for ES Catalog ( #40444 ) ( #40526 )
...
bp #40444
2024-09-09 14:25:43 +08:00
4b72a4c04e
[improve](routine load) delay schedule EOF tasks to avoid too many small transactions ( #39975 ) ( #40509 )
...
pick (#39975 )
We encountered a scenario where a large number of small transactions
were generated, resulting in an impact on query performance: Kafka's
data comes in batches of very small data every very short time, which
leads to tasks being frequently scheduled and ending very quickly,
resulting in a large number of small transactions.
To solve this problem, we delay the scheduling of tasks that perceive
EOF, which would not delay data consumption, for perceiving EOF
indicates that the consumption speed is greater than the production
speed.
2024-09-09 11:36:22 +08:00
01a651e573
[enhance](mtmv) During cache generation, no longer hold the write loc… ( #40489 )
...
…k for mtmv (#40402 )
pick: https://github.com/apache/doris/pull/40402
2024-09-09 11:35:22 +08:00
a67f20f073
[opt](mtmv) Support to contain select constant clause when create async materialized view ( #40244 ) ( #40435 )
...
## Proposed changes
commitId: 518a0fc0
pr: https://github.com/apache/doris/pull/40244
2024-09-09 11:23:15 +08:00
4084ff2c4a
[enhance](mtmv)Optimize the speed of obtaining the last update time o… ( #40421 )
...
…f Hive (#40169 )
pick: https://github.com/apache/doris/pull/40169
2024-09-09 11:20:40 +08:00
fc4c2aed64
[enhance](mtmv) Prohibit creating materialized views in external cata… ( #40420 )
...
…logs (#39557 )
pick: https://github.com/apache/doris/pull/39557
2024-09-09 11:16:40 +08:00
ecb75c2e7d
[fix](mtmv) Mtmv support set both immediate and starttime ( #39573 ) ( #40418 )
...
pick: https://github.com/apache/doris/pull/39573
2024-09-09 11:13:51 +08:00
a3eba2aad5
[fix](auth) ordinary users can see the processes of other users ( #39747 ) ( #40415 )
...
pick: https://github.com/apache/doris/pull/39747
2024-09-09 11:13:18 +08:00
e1e09badfa
[chore](mtmv)add plan log for UpdateMvByPartitionCommand ( #40072 ) ( #40413 )
...
pick: https://github.com/apache/doris/pull/40072
2024-09-09 11:09:58 +08:00
93a2518d12
[fix](auth)Fix some issues with incorrect permission verification (#3… ( #40410 )
...
…9726)
pick: https://github.com/apache/doris/pull/39726
2024-09-09 11:04:49 +08:00
d373ca7da1
[enhancement](schema-change) Support light schema change on hash columns and agg key columns with varchar type to change length ( #39319 ) ( #40236 )
...
## Proposed changes
1. Schema change should rebuild distribution info after modifying
columns, especially distribution columns. Or it may cause dynamic
partition failed when checking distribution columns' equality.
2. Support hash key columns to do light schema change. For unique key or
dup key columns, could not be enabled temporarily due to some historical
reasons. See #39798 .
2024-09-09 10:55:32 +08:00
a963709fed
[opt](scanner) Control the degree of parallelism of scanner when only limit involved #39927 ( #40357 )
...
cherry pick from #39927
2024-09-09 10:42:19 +08:00
962c382077
[fix](jdbc catalog) Fix type recognition error when using query tvf to query doris ( #40481 )
...
pick (#40122 )
Using string to match Doris type will not work with query tvf, so use
field matching instead
2024-09-06 19:30:32 +08:00
8104b992d1
[fix](ES Catalog)Do not extract doc_values of field with ignore_above setting ( #40314 ) ( #40464 )
...
bp #40314
2024-09-06 16:25:30 +08:00
88750e7e5d
[cherry-pick](branch-2.1) Pick "[Fix](group commit) Fix table not found fault when disable group commit ( #39731 )" ( #40323 )
...
<!--Describe your changes.-->
## Proposed changes
Pick #39731
<!--Describe your changes.-->
2024-09-05 21:39:15 +08:00
0928c9c6ed
[fix](unary function) Fix wrong result of asin, acos and sqrt when processing invalid input #40267 ( #40358 )
...
cherry pick from #40267
2024-09-05 19:51:01 +08:00
961d2c9af5
[feat](job)Internal job cancellation immediately and the strong association with the STARTS parameter ( #36805 ) ( #38110 )
...
…
## Proposed changes
For internal tasks, such as MTMV, the start time may already be set, or
the time may be adjusted immediately.
<!--Describe your changes.-->
(cherry picked from commit 904a6c0fc1a804520285533de874fe4d0ffff2c1)
## Proposed changes
Issue Number: close #36805
<!--Describe your changes.-->
2024-09-05 16:28:35 +08:00
52393f829d
[cherry-pick](branch-2.1) Fix some group commit fault ( #40319 )
...
## Proposed changes
Pick #39986 #40120
<!--Describe your changes.-->
2024-09-05 15:26:59 +08:00
40d10bdb0a
Revert "Revert "[cherry-pick](branch-2.1) Fix enable_mow_light_delete default value"" ( #40411 )
...
Reverts apache/doris#40359
2024-09-05 14:59:01 +08:00
c6771da240
[fix](nereids) fix bug for A>n, where A.max is infinity #39936 ( #40368 )
...
## Proposed changes
pick #39936
Issue Number: close #xxx
<!--Describe your changes.-->
2024-09-05 08:33:04 +08:00
08fb91f161
Revert "[cherry-pick](branch-2.1) Fix enable_mow_light_delete default value" ( #40359 )
...
Reverts apache/doris#40283
2024-09-04 17:55:25 +08:00
b4beec8ea8
[fix](OrcWriter) fix be core when upgrading BE without upgrading FE ( #40303 )
...
bp: #40282
2024-09-04 10:24:41 +08:00
db06c678a3
[Fix](planner) legacy planner repeat group by has grouping report error ( #40281 ) ( #40305 )
...
cherry-pick #40281 to branch-2.1
2024-09-04 08:50:21 +08:00
3cf5d15b49
[fix](fe) avoid setting the value of batch_size excessively high ( #35941 ) ( #40312 )
...
Issue Number: close #xxx
pick #35941
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
2024-09-03 19:49:26 +08:00
c49b345342
[cherry-pick](branch-2.1) Fix enable_mow_light_delete default value ( #40283 )
...
## Proposed changes
Pick #40104 #40197
<!--Describe your changes.-->
2024-09-03 11:19:05 +08:00
92752b90e7
[feature](metacache) add system table catalog_meta_cache_statistics #40155 ( #40210 )
...
bp #40155
2024-09-02 23:23:35 +08:00
1527099e35
[opt](Nereids) forbid one step topn with distribute as child ( #40066 ) ( #40278 )
...
pick from master #40066
2024-09-02 22:26:26 +08:00
a5efdc6085
[improve](statistics)Clean expired TableStatsMeta. ( #39779 ) ( #40253 )
...
backport: https://github.com/apache/doris/pull/39779
2024-09-02 22:13:42 +08:00
8ff068eb7c
[feature](statistics) Support get row count for JDBC external table. ( #38889 ) ( #40248 )
...
backport: https://github.com/apache/doris/pull/38889
2024-09-02 16:39:42 +08:00
98e039ae31
[pick](Variant) truncate key when column is variant type when crea… ( #40238 )
...
#39988
2024-09-02 13:54:04 +08:00
5db2c8b8bc
[branch-2.1](arrow-flight-sql) Fix return result from FE Arrow Flight server error 0.0.0.0:xxx, connection refused ( #40229 )
...
pick #40002
2024-09-02 09:45:27 +08:00
2a12251b83
[branch-2.1][Feat](Job)After a job is paused, it can be manually triggered to execute. ( #40180 )
...
…
pick (#39565 )
2024-08-31 19:24:53 +08:00
e4c48cfec5
[fix](multi-catalog)fix max compute null parts table read ( #40046 ) ( #40179 )
...
bp #40046
Co-authored-by: slothever <18522955+wsjz@users.noreply.github.com >
2024-08-30 22:56:08 +08:00
32a32808a7
[improvement](nereids) support convert_tz in partition prune ( #40047 ) ( #40127 )
...
cherry-pick #40047 to branch-2.1
2024-08-30 13:39:37 +08:00
80508fb595
[fix](restore) Reset next version for remote table when restore #40118 ( #40165 )
...
cherry pick from #40118
Co-authored-by: smallx <e9999e@163.com >
2024-08-30 13:22:23 +08:00
1288676b68
[fix](nereids)change lag/lead function signature ( #40161 )
...
## Proposed changes
pick from master https://github.com/apache/doris/pull/40060
<!--Describe your changes.-->
2024-08-30 12:05:47 +08:00
163193b1d4
[branch-2.1](function) fix random_bytes return same data for multi rows ( #39891 ) ( #40137 )
...
pick https://github.com/apache/doris/pull/39891
Issue Number: close #xxx
before:
```sql
mysql [optest]>SELECT random_bytes(10) a FROM numbers("number" = "10");
+------------------------+
| a |
+------------------------+
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
+------------------------+
```
now:
```sql
mysql [optest]>SELECT random_bytes(10) a FROM numbers("number" = "10");
+------------------------+
| a |
+------------------------+
| 0xd82cf60825b29ef2a0fd |
| 0x6f8c808415bdbaa6d257 |
| 0x7c26b5214297a151c25c |
| 0x43f02c77293063900437 |
| 0x5e5727569dec5e24f96b |
| 0x434f20bf74d7759640b7 |
| 0x087ed96b739750c733a6 |
| 0xdf05f6d7ede4972eb846 |
| 0xcefab471912264b5c54f |
| 0x1bddc019409d1926aa10 |
+------------------------+
```
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
2024-08-30 10:43:42 +08:00
e4d543f060
[branch-2.1](load) fix commit txn timeout when loading to table with many tablet ( #40031 ) ( #40139 )
...
pick https://github.com/apache/doris/pull/40031
2024-08-30 10:34:43 +08:00
70daa1f85d
[opt](inverted index) Controls whether the in_list can execute fast_execute. ( #40141 )
...
https://github.com/apache/doris/pull/40022
2024-08-30 10:32:43 +08:00
ca07a00c93
Revert "[branch-2.1](hive) support hive write text table ( #38549 ) (#4… ( #40157 )
...
…0063)"
This reverts commit c6df7c21a3c09ae1664deabacb88dfcea9d94b68.
## Proposed changes
Issue Number: close #xxx
<!--Describe your changes.-->
Co-authored-by: yiguolei <yiguolei@gmail.com >
2024-08-30 10:25:38 +08:00
