database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
19,601 Commits 1 Branch 0 Tags
a4e793752fccef8e49d98d6992df3ed352eb0942
Commit Graph

8056 Commits

Author SHA1 Message Date
Gabriel
3e5255a862 [pipeline](fix) Fix blocking task which is not triggered by 2nd RPC (… (#38694) 
…#38568)

Once a query is cancelled due to any reason, BE may not receive 2nd RPC
from FE. If so, we must ensure the execution dependency is ready so
tasks will not be blocked.
 2024-08-01 18:23:41 +08:00
Gabriel
82c681595e [fix](local exchange) Fix local exchange blocked by a huge data block… (#38693) 
… (#38657)

If a huge block is push into local exchanger, it will be blocked due to
concurrent problems. This PR use a unique lock to resolve it .
 2024-08-01 18:04:19 +08:00
meiyi
e8690b62ee [fix](group commit) Pick add debug log show why group commit not work; delete wal when replay success (#38611) (#38659) 
Pick https://github.com/apache/doris/pull/38611
 2024-08-01 16:59:54 +08:00
Gabriel
9d23ccf1f2 [Improvement](schema scan) Use async scanner for schema scanners (#38… (#38666) 
…403)
 2024-08-01 16:05:24 +08:00
Qi Chen
4042cdf553 [Fix](memory) Fix allocator.h compiling failed on mac. (#38646) 
Backport #38562. Fix allocator.h compiling failed on mac which
introduced by #37257.
 2024-08-01 13:56:53 +08:00
Xin Liao
63a3ff570b [Opt](load) print tablet id when memtable flush coredump #38618 (#38656) 
cherry pick from #38618
 2024-08-01 13:52:50 +08:00
HappenLee
28998300d4 [Bug](fix) fix ubsan use int32_t pointer access bool value (#38621) 
## Proposed changes

Issue Number: close #38617

<!--Describe your changes.-->
 2024-08-01 13:52:12 +08:00
amory
338fa32303 [pick](simdjson) fix simdjson with object array when jsonroot is not empty (#38633) 
## Proposed changes
backport: https://github.com/apache/doris/pull/38490
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-01 11:04:54 +08:00
wuwenchi
41fa7bc9fd [bugfix](paimon)Fixed the reading of timestamp with time zone type data for 2.1 (#37716) (#38592) 
bp: #37716
 2024-08-01 10:23:06 +08:00
amory
184b8cbbe4 [pick](json)fix jsonb deseriaze (#38630) 
## Proposed changes
backport: https://github.com/apache/doris/pull/37251
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-01 10:18:27 +08:00
airborne12
66ebf709ba [Fix](inverted index) fix fast execute for not_in expr #37745 (#38594) 
cherry pick from #37745
 2024-07-31 19:58:12 +08:00
airborne12
7730aa2170 [Fix](inverted index) fix wrong no need read data when same column in inverted index and like function #36687 (#38581) 
cherry pick from #36687
 2024-07-31 19:41:39 +08:00
airborne12
a75511ae08 [Feature](inverted index) add no need read data optimize config (#38584) 
pick from #36686
 2024-07-31 19:39:17 +08:00
airborne12
232ee74566 [Fix](inverted index) fix memory leak for index compaction (#38586) 
Pick from (#36209)
 2024-07-31 19:19:38 +08:00
airborne12
aed0cc8ba0 [Fix](inverted index) remove duplicate stats of inverted_index_query_cache_miss #36707 (#38580) 
cherry pick from #36707
 2024-07-31 19:18:58 +08:00
airborne12
7357d7bd3b [Update](inverted index) Add column name to debug point for "no need to read data" optimization #37649 (#38579) 
cherry pick from #37649
 2024-07-31 19:17:46 +08:00
HappenLee
3b234cfab6 [performance](exec) Performance problem create too many scanner task (#38460) 
## Proposed changes

cherry pick the pr: #38430

<!--Describe your changes.-->
 2024-07-31 14:34:01 +08:00
lihangyu
aa9bdd76d0 [Pick](Variant) pick some fix #38413 #38364 (#38512) 2024-07-31 11:03:31 +08:00
walter
182bf4d323 [chore](fe) Returns dropped tables in GetMeta request (#38541) 
Cherry-pick #38019
 2024-07-31 10:57:00 +08:00
Mryange
017dad8c54 [fix](type)support runtime predicate for time type (#38258) (#38465) 
## Proposed changes
https://github.com/apache/doris/pull/38258
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-31 10:27:36 +08:00
camby
715bcd13f1 [opt](mow) opt mow lookup with sequence column (#38287) (#38406) 2024-07-30 09:46:09 +08:00
airborne12
cefee4dbc0 [Pick 2.1](clucene) update clucene version (#38496) 
## Proposed changes

backport #38482
 2024-07-30 09:40:04 +08:00
hui lai
17d351af80 [fix](csv reader) fix csv parser incorrect if enclosing line_delimiter (#38347) (#38445) 
Csv reader parse data incorrect when data enclosing line_delimiter, for
example, line_delimiter is \n and enclose is ', data as follows:
```
'aaaaaaaaaaaa
bbbb'
```
it will be parsed as two columns: `'aaaaaaaaaaaa` and `bbbb',` rather
than one column
```
'aaaaaaaaaaaa
bbbb'
```

The reason why this happened is csv reader will not reset result when
not match enclose in this `output_buf_read`, causing incorrect
truncation was made.

Co-authored-by: Xin Liao <liaoxinbit@126.com>
 2024-07-29 14:55:45 +08:00
Jerry Hu
87cf2d1fb4 [fix](spill) Duplicate calls to Dependency::set_ready() in hash join(#37461) (#38399) 
## Proposed changes

pick #37461
Duplicate calling the function `Dependency::set_ready()` will cause
pipeline tasks to be scheduled incorrectly.
 2024-07-29 09:44:48 +08:00
Xin Liao
e9f12fac47 [fix](load) fix no error url for stream load #38325 (#38417) 
cherry pick from #38325
 2024-07-28 19:06:57 +08:00
Xin Liao
d8744cd3d0 [Opt](load) don't print stack when some errors occur for stream load #38332 (#38418) 
cherry pick from #38332
 2024-07-28 19:04:24 +08:00
Gabriel
c93f3bd24e [Improvement](bloom filter) Forbid small bloom filter (#38349) (#38392) 
Bloom filter has a expected filter ratio when data is enough. This PR
forbid too small bloom filter which has a big bias for filter ratio.

pick #38349
 2024-07-26 10:11:31 +08:00
Kaijie Chen
0152a4e86f [config](be) add be config migration_lock_timeout_ms (#38000) (#38337) 
backport #38000
 2024-07-25 17:36:34 +08:00
meiyi
91f0301b43 [fix](group commit) Pick some group commit pr (#38320) 
Pick https://github.com/apache/doris/pull/38292,
https://github.com/apache/doris/pull/34021,
https://github.com/apache/doris/pull/38228, some modify of
https://github.com/apache/doris/pull/37260, some modify of
https://github.com/apache/doris/pull/37595
 2024-07-25 17:32:44 +08:00
Kaijie Chen
54787f51c7 [log](load) print reason when LoadPathMgr failed to delete file (#38235) (#38340) 
backport #38235
 2024-07-25 17:28:09 +08:00
Jerry Hu
4288f09e8c [fix](debug_point) Add debug point:reached_limit_early & return_empty_block (#38328) 
## Proposed changes

pick #38127 and #37629
 2024-07-25 14:46:40 +08:00
Gabriel
a8139d9f39 [Improvement](profile) Provide more info for schedule time (#38290) (#38338) 
## Proposed changes

pick #38290

<!--Describe your changes.-->
 2024-07-25 14:43:52 +08:00
zzzxl
e2bb86e7f8 [fix](inverted index) fixed in_list condition not indexed on pipelinex (#38178) 
## Proposed changes

https://github.com/apache/doris/pull/36565
https://github.com/apache/doris/pull/37842
https://github.com/apache/doris/pull/37921
https://github.com/apache/doris/pull/37386

<!--Describe your changes.-->
 2024-07-25 14:42:34 +08:00
Qi Chen
a751372e76 [Feature](multi-catalog) Add memory tracker for orc reader/writer and arrow parquet writer。 (#37257) 
## Proposed changes

backport #37234
 2024-07-25 13:51:59 +08:00
lihangyu
73fc55b203 [Pick](Variant) fix some issue by RQG (#38336) 
#38318 
#38291
 2024-07-25 12:19:07 +08:00
zhangstar333
70cde39fe0 [cherry-pick](branch-21) fix conv function get wrong result as parse overflow (#38001) (#38309) 
## Proposed changes

cherry-pick from https://github.com/apache/doris/pull/38001

<!--Describe your changes.-->
 2024-07-25 12:06:46 +08:00
zhangstar333
e9052e2180 [cherry-pick](branch-21) fix mod function cause core dump (#37999) (#38308) 
## Proposed changes
cherry-pick from master https://github.com/apache/doris/pull/37999

<!--Describe your changes.-->

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-25 12:06:21 +08:00
zhangstar333
57864e8554 [cherry-pick](branch-21) fix collect_set function core dump without arena pool (#38234) (#38307) 
## Proposed changes

cherry-pick from master #38234

<!--Describe your changes.-->
 2024-07-25 12:05:52 +08:00
Jerry Hu
d19b3a5cb6 [fix](function) the bucket number arg of width_bucket should be a positive integer value (#37892) (#38295) 
## Proposed changes

pick #37892
 2024-07-25 11:56:47 +08:00
zclllhhjj
21b3fc3d1e [branch-2.1](function) fix coredump for MULTI_MATCH_ANY (#37959) (#38314) 
pick https://github.com/apache/doris/pull/37959

[INVALID_ARGUMENT][E33] Compile regexp expression failed. got Embedded
start anchors not supported.. some expressions may be illegal
 2024-07-25 11:34:22 +08:00
zclllhhjj
79a6496bb6 [branch-2.1](function) fix wrong result when convert_tz is out of bound (#37358) (#38313) 
## Proposed changes

pick https://github.com/apache/doris/pull/37358

before:
```sql
mysql> select CONVERT_TZ(cast('0000-01-01 00:00:00.00001'  as DATETIMEV1), cast('Asia/Shanghai' as VARCHAR(65533)), cast('America/Los_Angeles' as VARCHAR(65533)));
+---------------------------------------------------------------------------------------------------------------------------------------------------+
| convert_tz(cast('0000-01-01 00:00:00.00001' as DATETIME), cast('Asia/Shanghai' as VARCHAR(65533)), cast('America/Los_Angeles' as VARCHAR(65533))) |
+---------------------------------------------------------------------------------------------------------------------------------------------------+
| q535-12-31 08:01:19                                                                                                                               |
+---------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.12 sec)
```
now:
```sql
mysql> select CONVERT_TZ(cast('0000-01-01 00:00:00.00001'  as DATETIMEV1), cast('Asia/Shanghai' as VARCHAR(65533)), cast('America/Los_Angeles' as VARCHAR(65533)));
+---------------------------------------------------------------------------------------------------------------------------------------------------+
| convert_tz(cast('0000-01-01 00:00:00.00001' as DATETIME), cast('Asia/Shanghai' as VARCHAR(65533)), cast('America/Los_Angeles' as VARCHAR(65533))) |
+---------------------------------------------------------------------------------------------------------------------------------------------------+
| NULL                                                                                                                                              |
+---------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.09 sec)
```
 2024-07-25 11:32:44 +08:00
Mingyu Chen
3ea26a8c95 [fix](external) record not found file number (#38253) (#38285) 
bp #38253
 2024-07-25 11:03:19 +08:00
Luwei
9a40cd5f9f [fix](compaction) fix duplicate key in agg/mor table caused by ordered data compaction (#38224) (#38299) 
pick master #38224
 2024-07-24 19:03:40 +08:00
Qi Chen
ef00dad680 [Fix](multi-catalog) Fix some undefined behaviors. (#38274) 
## Proposed changes

backport #37845
 2024-07-24 16:14:34 +08:00
zzzxl
c6cd1e2e3d [opt](inverted index) opt value extraction from column to string (#38179) 
## Proposed changes

https://github.com/apache/doris/pull/37395
 2024-07-24 15:38:41 +08:00
Xinyi Zou
10c5c336d8 [branch-2.1](arrow-flight-sql) Add config arrow_flight_result_sink_buffer_size_rows (#38223) 
pick #38221
 2024-07-24 15:15:39 +08:00
airborne12
bb2bc77717 [Pick 2.1](inverted index) skip index compaction for inverted index V2 (#38278) 
Enabling index compaction in the inverted index V2 format currently
causes unexpected errors, especially in the case of tables with hybrid
indexes, such as BKD index and Fulltext index together.

backport #38209
 2024-07-24 14:38:25 +08:00
abmdocrt
792bd7c74a [cherry-pick](branch-2.1) Pick "[Enhancement](group commit)Optimize be select for group commit #35558" (#37830) 
Pick #35558
 2024-07-24 09:21:07 +08:00
Pxl
cf2120a44a [Chore](brpc) add gc for abafreelist to avoid eagain and set brpc tim… (#38177) 
pick from #37888
 2024-07-23 21:24:36 +08:00
daidai
193be20c86 [feature](csv)Supports reading CSV data using LF and CRLF as line separators. (#37687) (#38099) 
bp #37687
 2024-07-22 22:53:04 +08:00