database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
20,904 Commits 1 Branch 0 Tags
2ed306d0b160d1f6574d4e30cc6dfd12502ddc04
Commit Graph

20904 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Xinyi Zou
2ed306d0b1 [pick](branch-2.1) pick #44286 (#45055) 2024-12-06 14:33:47 +08:00
github-actions[bot]
d3c10f01e3 branch-2.1: [Chore](Job)Add scheduling logs for periodic jobs. #44843 (#45071) 
Cherry-picked from #44843

Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>
 2024-12-06 14:09:15 +08:00
github-actions[bot]
f038f28682 branch-2.1: [fix](mtmv) Fixed an error in setting variables in nested materialize views #44960 (#45038) 
Cherry-picked from #44960
 2024-12-06 14:04:13 +08:00
hui lai
682cec9b9a [test](stream load) add cases to ensure the correctness of the document (#43016) (#44385) 
pick #43016
 2024-12-06 11:04:51 +08:00
github-actions[bot]
94746b764d branch-2.1: [fix](memory) Fix UT ThreadMemTrackerMgrTest #44147 (#44936) 
Cherry-picked from #44147

Co-authored-by: Xinyi Zou <zouxinyi@selectdb.com>
 2024-12-06 10:54:12 +08:00
Mryange
21e21f5e3b [opt](exec) Use PASSTHROUGH to improve the concurrency of the ADAPTIV… (#44971) 
…E_PASSTHROUGH SINK. (#44925)
https://github.com/apache/doris/pull/44925
before
```
op -> local sink(1) -> local source (n)
```
now
```
op -> local passthrough(1) -> local passthrough(n) ->  local sink(n) -> local source (n)
```

profile
```
                  Pipeline  :  1(instance_num=3):
                      AGGREGATION_SINK_OPERATOR  (id=4  ,  nereids_id=255):
                          CROSS_JOIN_OPERATOR  (id=3  ,  nereids_id=245):
                              LOCAL_EXCHANGE_OPERATOR  (ADAPTIVE_PASSTHROUGH)  (id=-5):
                  Pipeline  :  2(instance_num=3):
                      LOCAL_EXCHANGE_SINK_OPERATOR  (ADAPTIVE_PASSTHROUGH)  (id=-5):
                          LOCAL_EXCHANGE_OPERATOR  (PASSTHROUGH)  (id=-6):
                  Pipeline  :  3(instance_num=1):
                      LOCAL_EXCHANGE_SINK_OPERATOR  (PASSTHROUGH)  (id=-6):
                          OLAP_SCAN_OPERATOR  (id=2.  nereids_id=234.  table  name  =  nums1(nums1)):
```
 2024-12-06 09:57:31 +08:00
daidai
0c3bc1bcd0 [fix](auth)fix be enable http auth, some request link never return. (#44959) (#45063) 
bp #44959

if you `enable_all_http_auth = true` in be.conf, then restart be, and
keep using `curl -u "xxxx:xxxx" http://127.0.0.1:8040/api/health` while
be is starting. You may encounter a situation where the link does not
return.
Reason:
When be is still starting, there is no information about fe master. When
you make an api request to be http port, be needs to request
authentication information from fe, which will cause it to request a
machine with empty ip and port 0. This rpc call will definitely fail
(this is not equivalent to a password error). After receiving this
failure, be does not `send_reply` to the api requester, so this api
request cannot be returned.
 2024-12-06 09:39:13 +08:00
github-actions[bot]
a67bbf3c58 branch-2.1: [Fix](job)Fix CAS competition failure leading to message publishing failure. #45018 (#45030) 
Cherry-picked from #45018

Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>
 2024-12-06 09:32:15 +08:00
yujun
041a1fe095 branch-2.1: [fix](nereids) fix compare with long min for simplify comparison rule (#44922) 2024-12-06 09:31:02 +08:00
github-actions[bot]
a98cb8c8bf branch-2.1: [fix](arrow-flight-sql) Fix query result is empty and not return query error message #45023 (#45053) 
Cherry-picked from #45023

Co-authored-by: Xinyi Zou <zouxinyi@selectdb.com>
 2024-12-06 09:27:35 +08:00
Gabriel
405b50b1b7 [Improvement](queue) Return value of concurrent queue should be proce… (#45032) 
…… (#44986)

…ssed

Push items into concurrent queue will return false due to some
unexpected error (e.g. poor memory available).
 2024-12-05 17:46:18 +08:00
MoanasDaddyXu
11c517fe1e [enhancement][docker]update routine docker file (#45048) 2024-12-05 17:27:44 +08:00
Xinyi Zou
fa633d751d [pick](branch-2.1) pick #44489 #44894 (#45024) 
### What problem does this PR solve?

pick #44489 #44894
 2024-12-05 17:04:15 +08:00
github-actions[bot]
a8f36a4813 branch-2.1: [fix](ctas) fix wrong resultType of column typeDef #43828 (#44985) 
Cherry-picked from #43828

Co-authored-by: TsukiokaKogane <cby141994@gmail.com>
 2024-12-04 22:10:11 +08:00
github-actions[bot]
c4bd0e8fa6 branch-2.1: [fix](memory) Fix compatibility with CgroupV2 #44579 (#44934) 
Cherry-picked from #44579

Co-authored-by: Xinyi Zou <zouxinyi@selectdb.com>
 2024-12-04 22:09:16 +08:00
github-actions[bot]
dcf3eb3434 branch-2.1: [fix](agg) Fixed a core dump when using the IPv6 type in array_agg. #44877 (#44918) 
Cherry-picked from #44877

Co-authored-by: Mryange <yanxuecheng@selectdb.com>
 2024-12-04 22:05:00 +08:00
github-actions[bot]
823c24d7b5 branch-2.1: [fix](mtmv) Fix filter position different but same causing rewritten by materialized view fail #44575 (#44900) 
Cherry-picked from #44575

Co-authored-by: seawinde <wusi@selectdb.com>
 2024-12-04 22:03:26 +08:00
github-actions[bot]
b94baf1f76 branch-2.1: [fix](profile) Change the check of whether the username is admin and root to check whether the user specifically corresponds to admin permissions #41714 (#44865) 
Cherry-picked from #41714

Co-authored-by: xyf <116467462+xyfsjq@users.noreply.github.com>
 2024-12-04 22:02:54 +08:00
github-actions[bot]
191c86bb8b branch-2.1: [fix](nereids) fix merge_percentile_to_array when has same agg function #44783 (#44879) 
Cherry-picked from #44783

Co-authored-by: feiniaofeiafei <moailing@selectdb.com>
 2024-12-04 22:02:16 +08:00
daidai
60dde858b9 [fix](maxcompute)add mc catalog read partition table partition prune(#44508) (#44830) 2024-12-04 18:06:31 +08:00
zhangdong
385cd1fcdb [fix](auth)Fix the need for low-level table permissions when querying… (#44842) 
… views in certain situations (#44621)

pick: https://github.com/apache/doris/pull/44621
 2024-12-04 18:01:28 +08:00
Xinyi Zou
6714936f8b [pick](branch-2.1) pick #39962 #40304 (#44931) 2024-12-04 17:56:58 +08:00
Xinyi Zou
02fdf5307c [pick](branch-2.1) pick #42059 (#44938) 2024-12-04 17:49:08 +08:00
hui lai
19b63bb437 [fix](load) fix load cast throw exception when plan (#44968) 
### What problem does this PR solve?

Use the load command with the following columns mapping:
```
curl --location-trusted -u root: \
    -H "Expect:100-continue" \
    -H "column_separator:," \
    -H "columns:cache_key,result_cnt,result,result=bitmap_from_base64(result)" \
    -T 1.txt \
    -XPUT http://10.16.10.6:48733/api/mutong/task_result_dev/_stream_load 
```

```
{
    "TxnId": 37092,
    "Label": "54b9e25b-853d-4ecc-aca9-82bd4ed91875",
    "Comment": "",
    "TwoPhaseCommit": "false",
    "Status": "Fail",
    "Message": "[ANALYSIS_ERROR]TStatus: errCode = 2, detailMessage = can not cast from origin type bitmap to target type=varchar(65533)",
    "NumberTotalRows": 0,
    "NumberLoadedRows": 0,
    "NumberFilteredRows": 0,
    "NumberUnselectedRows": 0,
    "LoadBytes": 0,
    "LoadTimeMs": 0,
    "BeginTxnTimeMs": 1,
    "StreamLoadPutTimeMs": 2,
    "ReadDataTimeMs": 0,
    "WriteDataTimeMs": 0,
    "CommitAndPublishTimeMs": 0
} 
```

The pr aims to fix this issue.

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->
 2024-12-04 17:45:05 +08:00
github-actions[bot]
5c686c32a2 branch-2.1: [fix](mtmv) Fix mv rewrite fail when base table add column #44867 (#44956) 
Cherry-picked from #44867

Co-authored-by: seawinde <wusi@selectdb.com>
 2024-12-04 17:43:49 +08:00
Siyang Tang
3820657e28 [fix](regression-test) Fix some schema change case not pick to 2.1 sufficiently (#44921) 
pick: #37390 (partly)
 2024-12-04 14:26:41 +08:00
github-actions[bot]
00c7394813 branch-2.1: [fix](scanner) Delete meaningless finish dependency in schema scanner #44915 (#44963) 
Cherry-picked from #44915

Co-authored-by: Gabriel <liwenqiang@selectdb.com>
 2024-12-04 13:16:08 +08:00
github-actions[bot]
bbd0dcb26e branch-2.1: [fix](ForkJoinPool) we should not new a thread pool every call #44891 (#44940) 
Cherry-picked from #44891
 2024-12-04 11:23:56 +08:00
github-actions[bot]
e10f1e9889 branch-2.1: [fix](unique key) getEnableUniqueKeyMergeOnWrite should check keys type #44692 (#44896) 
Cherry-picked from #44692

Co-authored-by: zhannngchen <zhangchen@selectdb.com>
 2024-12-03 20:17:19 +08:00
zhangyuan
d10cb19b07 [fix](sync mv) fix rewrite wrongly with sync mv (#44866) 
Cherry-picked from https://github.com/apache/doris/pull/39284
 2024-12-03 19:59:41 +08:00
feiniaofeiafei
3e1692298b [Feat](nereids) support pull up predicate from set operator (#39450) (#44056) 
cherry-pick #39450 to branch-2.1
 2024-12-03 16:24:52 +08:00
github-actions[bot]
d9ef316cdb branch-2.1: [fix](catalog) Fix infinity partition key toSql #44834 (#44890) 
Cherry-picked from #44834

Co-authored-by: walter <maochuan@selectdb.com>
 2024-12-03 14:12:47 +08:00
bobhan1
f612290597 branch-2.1: [Fix](TPartitionVersionInfo) Fix duplicate TPartitionVersionInfo in PublishVersionTask.partitionVersionInfos #44846 (#44885) 
pick https://github.com/apache/doris/pull/44846
 2024-12-03 10:47:49 +08:00
github-actions[bot]
4b7f5f8035 branch-2.1: [chore](binlog) Save table type in CreateTableRecord #44735 (#44836) 
Cherry-picked from #44735

Co-authored-by: walter <maochuan@selectdb.com>
 2024-12-03 10:03:07 +08:00
Pxl
0b681d3d51 [Bug](runtime-filter) fix publish not inited rf when broadcast join meet wake up by downsteam #44823 (#44859) 
pick from #44823
 2024-12-02 21:32:38 +08:00
htyoung
eee4f59f3d [fix](cache) fix same sql return wrong result when switch database with use db and enable sql cache (44782) (#44847) 
It will return wrong result when running same sql in different db with `enable_sql_cache=true`

for example, the `db1` and `db2` has the same table `tbl` but the data are not equals, 
if execute the below sql in `db1` and cache the result, then execute it in `db2`, it will return the wrong result
```sql
select count(*) from tbl
```
 2024-12-02 18:29:17 +08:00
zfr95
82afa7fbc1 [test](stats) Add upgrade and downgrade test for analyze (#42855) (#43456) 
pick from #42855
 2024-12-02 17:32:00 +08:00
Gabriel
7278a86c20 [profile](pipeline) Add key metrics for pipeline initialization (#35073) (#44738) 
pick #35073
 2024-12-02 16:02:19 +08:00
github-actions[bot]
8843b17a28 branch-2.1: [improve](routine load) ensure load balance after scaling up BE nodes #44693 (#44799) 
Cherry-picked from #44693

Co-authored-by: hui lai <laihui@selectdb.com>
 2024-12-02 15:04:11 +08:00
github-actions[bot]
54e2249a56 branch-2.1: [Fix](http)Enhanced Security Checks for Audit Log File Names #44612 (#44833) 
Cherry-picked from #44612

Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>
 2024-12-02 14:56:20 +08:00
github-actions[bot]
abf1d5f07d branch-2.1: [improvement](mysql)Support mysql COM_RESET_CONNECTION command. #44747 (#44788) 
Cherry-picked from #44747

Co-authored-by: James <lijibing@selectdb.com>
 2024-12-02 12:58:38 +08:00
924060929
a8a86f82d4 [fix](sql_cache) fix sql cache result wrong of from_unixtime(col, 'yyyy-MM-dd HH:mm:ss') (#44631) (#44810) 
fix sql cache result wrong of from_unixtime(col, 'yyyy-MM-dd HH:mm:ss') which introduced by #33262

the wrong result: is `yyyy-MM-dd HH:mm:ss`
 2024-12-02 11:46:37 +08:00
924060929
6b74db7cdc [fix](nereids) fix months_add/ months_sub/ years_add/years_sub compute wrong result because SimplifyArithmeticComparisonRule (#44725) (#44812) 
cherry pick from #44725
 2024-12-02 11:28:18 +08:00
github-actions[bot]
5f952cf6ed branch-2.1: [fix](iceberg)Bring field_id with parquet files And fix map type's key optional #44470 (#44828) 
Cherry-picked from #44470

Co-authored-by: wuwenchi <wuwenchi@selectdb.com>
 2024-12-02 10:24:07 +08:00
github-actions[bot]
a95201aa3a branch-2.1: [fix](outfile) add regression test for outfile #44734 (#44809) 
Cherry-picked from #44734

Co-authored-by: Tiewei Fang <fangtiewei@selectdb.com>
 2024-12-02 10:08:05 +08:00
Luwei
6726c9bf2f [improvement](compaction) reduce tablet skip compaction time (#44273) (#44791) 
pick master #44273

The time for tablet skip compaction is 120 seconds, which is too long.
In the scenario of high-frequency import (mow), it leads to a high
compaction score. Therefore, reducing the skip time to 10 seconds is
necessary.
 2024-12-02 10:07:17 +08:00
github-actions[bot]
e322955286 branch-2.1: [fix](jdbc catalog) set enable_jdbc_cast_predicate_push_down default true #44548 (#44760) 
Cherry-picked from #44548

Co-authored-by: zy-kkk <zhongyongkang@selectdb.com>
 2024-12-02 09:33:46 +08:00
github-actions[bot]
f9b3863848 branch-2.1: [Fix](Job)Fix some issues in the Insert job. #44543 (#44597) 
Cherry-picked from #44543

---------

Co-authored-by: Calvin Kirs <guoqiang@selectdb.com>
 2024-11-30 11:18:55 +08:00
Mingyu Chen (Rayner)
4b15b1f263 [fix](orc) check all the cases before build_search_argument (#44615) (#44801) 
cherry-pick #44615

Co-authored-by: Socrates <suyiteng@selectdb.com>
 2024-11-30 09:17:56 +08:00
yujun
4d023cf0b2 [fix](nereids) fix UnknownValue's reference in simplify range rule #44637 (#44771) 
cherry pick from #44637
 2024-11-29 20:58:40 +08:00