Commit Graph

20619 Commits

Author SHA1 Message Date
443e87e203 2.1.7-rc03 (#43338)
Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-11-06 15:21:23 +08:00
8fedb03fdc branch-2.1: [fix](auth)Fix after fe restarting, external permissions are lost (#43307)
Cherry-picked from #43275

Co-authored-by: zhangdong <493738387@qq.com>
2024-11-06 14:54:59 +08:00
876fae6f71 [fix](regression) test_primary_key_partial_update fail due to the U.S. change from DST to standard time (#43288) (#43300)
cherry-pick #43288
2024-11-06 14:10:29 +08:00
9d5ebd6f34 [fix](export) make sure the export task being executed after writing "create export job" edit log #43250 (#43291)
cherry-pick #43250
2024-11-06 09:47:14 +08:00
0efca04183 2.1.7-rc02 (#43285)
### What problem does this PR solve?
<!--
You need to clearly describe your PR in this part:

1. What problem was fixed (it's best to include specific error reporting
information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what
is it now, why was it modified, and what possible impacts might there
be.
3. What features were added. Why this function was added.
4. Which codes were refactored and why this part of the code was
refactored.
5. Which functions were optimized and what is the difference before and
after the optimization.

The description of the PR needs to enable reviewers to quickly and
clearly understand the logic of the code modification.
-->

<!--
If there are related issues, please fill in the issue number.
- If you want the issue to be closed after the PR is merged, please use
"close #12345". Otherwise, use "ref #12345"
-->
Issue Number: close #xxx

<!--
If this PR is followup a preivous PR, for example, fix the bug that
introduced by a related PR,
link the PR here
-->
Related PR: #xxx

Problem Summary:

### Check List (For Committer)

- Test <!-- At least one of them must be included. -->

    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No colde files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:

    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?

    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

- Release note

    <!-- bugfix, feat, behavior changed need a release note -->
    <!-- Add one line release note for this PR. -->
    None

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->

Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-11-05 20:17:25 +08:00
46afbfca01 branch-2.1: [fix](ip) fix datatype serde for ipv6 with rowstore (#43252)
Cherry-picked from #43065

Co-authored-by: amory <wangqiannan@selectdb.com>
2024-11-05 20:09:14 +08:00
71e33d9208 [feat](nereids)disable join reorder if any table row count is not available #43000 (branch-2.1) (#43247)
### What problem does this PR solve?
pick #43000 

<!--
You need to clearly describe your PR in this part:

1. What problem was fixed (it's best to include specific error reporting
information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what
is it now, why was it modified, and what possible impacts might there
be.
3. What features were added. Why this function was added.
4. Which codes were refactored and why this part of the code was
refactored.
5. Which functions were optimized and what is the difference before and
after the optimization.

The description of the PR needs to enable reviewers to quickly and
clearly understand the logic of the code modification.
-->

<!--
If there are related issues, please fill in the issue number.
- If you want the issue to be closed after the PR is merged, please use
"close #12345". Otherwise, use "ref #12345"
-->
Issue Number: close #xxx

<!--
If this PR is followup a preivous PR, for example, fix the bug that
introduced by a related PR,
link the PR here
-->
Related PR: #xxx

Problem Summary:

### Check List (For Committer)

- Test <!-- At least one of them must be included. -->

    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No colde files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:

    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?

    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

- Release note

    <!-- bugfix, feat, behavior changed need a release note -->
    <!-- Add one line release note for this PR. -->
    None

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2024-11-05 18:29:50 +08:00
2e00fa475b [fix](persist) Fix the generated dropping tmp partition sql #43079 (#43170)
cherry pick from #43079
2024-11-05 17:44:31 +08:00
4031808f00 [enhancement](rowset-meta) Remove rowset meta from olap meta directly when rowsets deleted (#41716) (#43183)
pick: #41716
2024-11-05 00:40:12 +08:00
a365965c55 [fix](regression-test) Revome "UNIQUE KEY (k1)" case for test_dynamic_partition_mod_distribution_key (#41002) (#43179)
pick: #4100
1. Remove "UNIQUE KEY (k1)" case, because for unique table hash column
must be key column, but for that historical bugs, this case will fail if
adding k2 unique key.
2. Seperate a p0 suite from docker suite because docker suite will not
be triggered in community doris p0 CI.
2024-11-05 00:39:21 +08:00
09f2516c7c [fix](test) fix some test cases #43217 (#43216)
bp #43217
2024-11-04 22:15:30 +08:00
2863bf8aaa [fix](regression) fix flaky partial update cases (#42072) (#43198)
cherry-pick #42072 and #42908
2024-11-04 21:38:05 +08:00
72bb6e79e4 [fix](index compaction)Skip writing terms with a doc frequency of 0(#43113) (#43115)
bp #43113
2024-11-04 17:49:56 +08:00
493b1d8469 [Fix](auto-pick)Allow doris-robot to trigger the workflow (#43190)
### What problem does this PR solve?
<!--
You need to clearly describe your PR in this part:

1. What problem was fixed (it's best to include specific error reporting
information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what
is it now, why was it modified, and what possible impacts might there
be.
3. What features were added. Why this function was added.
4. Which codes were refactored and why this part of the code was
refactored.
5. Which functions were optimized and what is the difference before and
after the optimization.

The description of the PR needs to enable reviewers to quickly and
clearly understand the logic of the code modification.
-->

<!--
If there are related issues, please fill in the issue number.
- If you want the issue to be closed after the PR is merged, please use
"close #12345". Otherwise, use "ref #12345"
-->
Issue Number: close #xxx

<!--
If this PR is followup a preivous PR, for example, fix the bug that
introduced by a related PR,
link the PR here
-->
Related PR: #xxx

Problem Summary:

### Check List (For Committer)

- Test <!-- At least one of them must be included. -->

    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No colde files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:

    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?

    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

- Release note

    <!-- bugfix, feat, behavior changed need a release note -->
    <!-- Add one line release note for this PR. -->
    None

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2024-11-04 12:10:24 +08:00
8eb6a2a7c1 [enhancement](Nereids) boost characterLiteralTypeCoercion (#42941) (#43092)
cherry pick from #42941
2024-11-04 11:50:44 +08:00
c0b7374020 [Fix](auto-pick)Allow doris-robot to trigger the workflow (#43166) (#43176)
### What problem does this PR solve?
 https://github.com/orgs/community/discussions/55906

Due to security restrictions, the GitHub Actions bot cannot trigger
workflows directly. Instead, we use doris-robot to initiate other CI
runs. Only automatically generated PRs will prompt doris-robot to
comment, and we verify the author and content to determine whether a
workflow should be triggered. This approach does not affect the original
CI process; it simply adds an additional trigger condition.

### Check List (For Committer)

- Test <!-- At least one of them must be included. -->

    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [x] No need to test or manual test. Explain why:
- [x] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change. - [ ] No colde files have
been changed. - [ ] Other reason <!-- Add your reason? -->

- Behavior changed:

    - [x] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?

    - [x] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

- Release note

    <!-- bugfix, feat, behavior changed need a release note -->
    <!-- Add one line release note for this PR. -->
    None

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->

(cherry picked from commit f04f0449268bd2f7251e32b867da223643bb7c4a)

### What problem does this PR solve?
<!--
You need to clearly describe your PR in this part:

1. What problem was fixed (it's best to include specific error reporting
information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what
is it now, why was it modified, and what possible impacts might there
be.
3. What features were added. Why this function was added.
4. Which codes were refactored and why this part of the code was
refactored.
5. Which functions were optimized and what is the difference before and
after the optimization.

The description of the PR needs to enable reviewers to quickly and
clearly understand the logic of the code modification.
-->

<!--
If there are related issues, please fill in the issue number.
- If you want the issue to be closed after the PR is merged, please use
"close #12345". Otherwise, use "ref #12345"
-->
Issue Number: close #xxx

<!--
If this PR is followup a preivous PR, for example, fix the bug that
introduced by a related PR,
link the PR here
-->
Related PR: #xxx

Problem Summary:

### Check List (For Committer)

- Test <!-- At least one of them must be included. -->

    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No colde files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:

    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?

    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

- Release note

    <!-- bugfix, feat, behavior changed need a release note -->
    <!-- Add one line release note for this PR. -->
    None

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
2024-11-04 11:32:50 +08:00
d1e63c5201 [improvement](external)add some improvements for external scan (#38946) (#43156)
bp #38946

Co-authored-by: wuwenchi <wuwenchihdu@hotmail.com>
2024-11-04 09:40:08 +08:00
e3170d6319 [bugfix](hive)fix after insert overwrite hive table, data error for 2.1 (#43049) (#43127)
bp: #43049
2024-11-03 23:38:47 +08:00
ac7d9f61f8 [fix](case) adjust error msg checking to real msg (#42000) (#43138) 2024-11-03 00:41:47 +08:00
9e7a1c9ae8 [fix](export) fix potential export concurrency issue #43109 (#43117)
bp #43109
2024-11-02 08:53:43 +08:00
98d3db03b1 [fix](regression-test) fix test_hive_serde_prop #42886 (#43098)
cherry pick from #42886

Co-authored-by: Socrates <suxiaogang223@icloud.com>
2024-11-01 23:11:45 +08:00
c5374be1bc [cherry-pick](branch-2.1) Pick "[Feature](schema change) Support add column bitmap with default value bitmap_empty (#42331)" (#42702)
Pick #42331
2024-11-01 21:46:38 +08:00
03a614f945 [fix](Export) fix dead lock of parallel Exporting (#43084)
### What problem does this PR solve?
bp: #43083
2024-11-01 21:27:25 +08:00
6c3a273d46 [fix](Nereids) replay create function npe since connect context not exists (#43075)
nereids will parse function statement when replay create alias function,
when do parse, need to get sql mode and other variable. because replay
thread do not have connect context, NPE be thrown when parsing
2024-11-01 21:01:24 +08:00
47fa76e0f6 [2.1][fix](jdbc catalog) Optimize JDBC Connection Closing to Ensure Proper Resource Release (#43074)
bp #43059
2024-11-01 20:34:18 +08:00
3dcaa949e9 [fix](case)Fix multiple case tables with the same name causing case failure (#42203) (#43042)
pick: from master #42203
2024-11-01 19:22:55 +08:00
cb25c40ddf [fix](export) fix concurrent modification issue with export job #43051 (#43052)
bp #43051
2024-11-01 17:19:29 +08:00
355170a921 [cherry-pick](branch2.1) impl scalar functions trim_in、ltrim_in and rtrim_in (#42641)
pick https://github.com/apache/doris/pull/41681
2024-11-01 09:55:50 +08:00
efd9866b33 [chore](profile) add task state in profile (#42895) (#43013)
## Proposed changes
pick #42895
2024-11-01 09:51:55 +08:00
e85e6c89ba [fix](auditlog) add missing audit log fields and duplicate audit log error #42262 (#43015)
bp #42262
2024-11-01 08:24:55 +08:00
df2bd6d525 [improvement](statistics)Improve statisitcs insert into audit log. (#42983) (#43032)
backport: https://github.com/apache/doris/pull/42983
2024-11-01 07:47:40 +08:00
a5efe1c66e [improve](restore) Log create replica task progress #42350 (#42967)
cherry pick from #42350
2024-10-31 14:36:34 +08:00
c169fd392f [fix](jdbc catalog) fix be crash when create jdbc catalog without driver jar #42948 (#42972)
cherry pick from #42948

Co-authored-by: zy-kkk <zhongyk10@gmail.com>
2024-10-31 14:33:40 +08:00
c573351e4e [fix](tvf) fix FE cannot start when replay alter view from tvf (#42866)
bp: #40872
2024-10-31 14:15:48 +08:00
6c3d42e09a [cherry-pick](branch-21) cherry-pick pr about (#42488) (#42099) (#42055) (#42916)
## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
2024-10-31 14:14:19 +08:00
0b640b938c [Test](bloom filter) add retry query for bloom filter test case #42937 (#42975)
cherry pick from #42937
2024-10-31 14:02:50 +08:00
cc30a7e78e [fix](test) fix some unstable external p0 test cases (#42685) (#42943)
cherry-pick #42685

Co-authored-by: daidai <2017501503@qq.com>
2024-10-31 12:36:03 +08:00
012c2d746c [fix](remote scan) Fix remote scan paralism (#42633) 2024-10-31 11:39:32 +08:00
597bcb112b [fix](Nerieds) fix FoldConstantRuleOnFE too slow when CaseWhen clause nesting multiple layers (#42897) (#42907)
cherry pick from master #42897
2024-10-31 11:15:34 +08:00
cdd32d9582 [enhance](hive) support reading hive table with OpenCSVSerde #42257 (#42940)
cherry pick from #42257

Co-authored-by: Socrates <suxiaogang223@icloud.com>
2024-10-31 11:12:07 +08:00
d67007c72d [improvement](statistics)Add log for estimating hive table row count logic. (#42921) (#42931)
backport: https://github.com/apache/doris/pull/42921
2024-10-31 10:30:41 +08:00
0f9c2c2f0a [fix](export) remove export task executor in TransientTaskExecutor (#42880)
### What problem does this PR solve?

Problem Summary:

There is a memory leak on FE side when continue running `export`
command.
The `ExportTaskExecutor` instances will be added to
`TransientTaskManager` and never be removed.

This PR mainly changes:
1. Remove unused `ExportTaskRegister` in `Env`.
2. Remove `ExportTaskExecutor` from `TransientTaskManager` once export
job is finished or cancelled.

### Release note

Fix FE memory leak by removing `ExportTaskExecutor` from
`TransientTaskManager` once export job is finished or cancelled.
2024-10-31 09:57:40 +08:00
668c65deb3 [enhancement](jni)append exception check for jni code. (#42507) (#42945)
bp #42507

Co-authored-by: daidai <2017501503@qq.com>
2024-10-31 09:56:11 +08:00
ae88d032db [chore](ddl) support force_enable_feature_binlog #41796 (#42926)
cherry pick from #41796
2024-10-31 09:53:45 +08:00
fce4695f37 [Configuration](transactional-hive) Add skip_checking_acid_version_file session var to skip checking acid version file in some hive envs. (#42111)(#42225) (#42939)
cherry-pick (#42111)(#42225)

---------

Co-authored-by: Qi Chen <kaka11.chen@gmail.com>
2024-10-31 09:52:20 +08:00
0d008b5a43 [feat](nereids)disable join reorder if column stats is invalid #41790 (branch-2.1) (#42902)
## Proposed changes
pick #41790
Issue Number: close #xxx

<!--Describe your changes.-->
2024-10-30 23:47:03 +08:00
6d23020cb8 [fix](regression) do not test "set global sessionVar" on multi-fe cluster (#42922)
## Proposed changes
fix unstable rt case: test_statistic_global_variable

Issue Number: close #xxx

<!--Describe your changes.-->
2024-10-30 21:35:32 +08:00
e43d255815 [fix](array) fix add_array_null_value for inverted index_builder (#42899)
## Proposed changes
Pick #42638 to doris branch 2.1
Issue Number: close #xxx

<!--Describe your changes.-->
2024-10-30 19:16:59 +08:00
53cccfb08c [test](move memtable) fix move memtable injection case fail (#42639) (#42891)
Fix move memtable injection case fail.

pick (#42639)
2024-10-30 16:29:40 +08:00
2a6d6c15c5 [enhance](hive)hive event code optimization #42637 (#42876)
cherry pick from #42637

Co-authored-by: zhangdong <493738387@qq.com>
2024-10-30 12:52:21 +08:00