database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
13,721 Commits 1 Branch 0 Tags
b38b8b4494de38b43c1c6c19a39de23538e12b22
Commit Graph

13721 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Pxl
fa3bb2eabe [Bug](materialized-veiw) fix error happens when parsing create materialized view stmt #21095 2023-06-22 15:58:32 +08:00
shenxingwuying
eb6202e8be [minor](fe) remove several unnecessary codes (#21046) 
1. The class 'ExternalDatabase' has implemented the 'GsonPostProcessable' interface, so
there is redundant codes in some subclass of 'ExternalDatabase'.
2. A LOG object is not used in this file.
 2023-06-22 15:29:25 +08:00
shuke
6a2226867c [fix](test) load case bug in tpch_sf10_unique_p2 #21093 2023-06-22 15:19:46 +08:00
Yu Hou
37c9a08e56 [Bug] The PID_DIR variable in the Doris stop script does not follow the conf file (#20881) 2023-06-22 10:26:26 +08:00
shuke
d5455dd04b [fix](regression-test) test case variable scope bug (#21071) 2023-06-22 09:07:44 +08:00
yuanyuan8983
bc6692a1b8 [typo](doc) add automatic service start (#20974) 
* add automatic service start

* add automatic service start

* add docs sidebars

* Update automatic-service-start.md
 2023-06-22 08:45:09 +08:00
shuke
9f0aa8a9de [fix](fuzzy)nereids and pipeline config changed by fuzzy in non-pipeline env. (#21092) 
* fix: nereids and pipeline config  changed by fuzzy in non-pipeline env.

* fix: format

* fix: format
 2023-06-22 08:36:19 +08:00
zhangstar333
a33521b2ce [enhancement](exchange) add filter for exchange node in BE (#21087) 2023-06-22 01:04:47 +08:00
Calvin Kirs
b192082b62 [Improve](load)Solve the problem of RoutineLoadTaskScheduler idling when there is no data (#20986) 
Since the polling interval is 0, the CPU will be polled all the time when there is no data

Before and after comparison test, the CPU usage time is reduced by 2000 times
 2023-06-22 00:41:45 +08:00
starocean999
fff308352f [fix](nereids)the microseconds value is wrong when create datatimev2 literal from LocalDateTime (#21089) 
* [fix](nereids)the microseconds value is wrong when create datatimev2 literal from LocalDateTime

* fix code style
 2023-06-22 00:40:53 +08:00
zclllyybb
49bbe88327 [fix](log) fix the too large warning log of BE (#21027) 2023-06-22 00:39:04 +08:00
TsukiokaKogane
3dfeee3946 [fix](typesystem) fix wrong return type argument cause type check fail (#21082) 2023-06-22 00:04:46 +08:00
Xinyi Zou
2c9bdd64fa [fix](memory) arena support memory reuse after clear() (#21033) 2023-06-21 23:27:21 +08:00
plat1ko
e060ffab96 [Fix](cooldown) Fix incorrect judgement of isDropTableOrPartition (#21084) 2023-06-21 23:00:58 +08:00
Gabriel
2ce8cfbebd [profile](sort) add some metrics in profile (#21056) 2023-06-21 22:57:46 +08:00
Xinyi Zou
661e1ae7c5 [fix](memory) no switch bthread context in UBSAN compile (#21064) 
When UBSAN is compiled, all memory will be tracked to the orphan (unknown) mem tracker, and the bthread context and mem tracker will no longer be switched.

The supplementary fixes are as follows: #20999
 2023-06-21 21:14:07 +08:00
HHoflittlefish777
b2c4e51be1 [fix](load) delete lazy open DCheck when unkown load id (#21083) 2023-06-21 20:42:31 +08:00
Chenyang Sun
18a0824eb3 [fix](compaction)Modify time series compaction policy default config (#21079) 2023-06-21 20:29:58 +08:00
starocean999
8b561cfb03 [fix](nereids)create datev2 and datetimev2 literal if enable_date_conversion is true (#21065) 2023-06-21 20:29:36 +08:00
DongLiang-0
442a734ef5 [improvement](config) update be config max_runnings_transactions_per_txn_map default value (#21060) 2023-06-21 20:29:13 +08:00
airborne12
6ac0bfeceb [Feature](inverted index) add unicode parser for inverted index (#21035) 2023-06-21 20:14:06 +08:00
zhannngchen
cc53391c9a Revert "[feature](merge-on-write) enable merge on write by default (#… (#21041) 2023-06-21 18:36:46 +08:00
HHoflittlefish777
2beed11256 [Bug](streamload) fix inconsistent load result of be and fe (#20950) 2023-06-21 18:12:51 +08:00
Xinyi Zou
84b97860a1 [fix](memory) Fix memory exceed limit and query has been canceled, Allocator will block 100ms (#20959) 2023-06-21 17:35:19 +08:00
Mryange
8bcd42d3f6 [test](regression) update some case in brown_p2 #21037 2023-06-21 16:25:07 +08:00
zhannngchen
85ce6a22c0 [enhancement](merge-on-write) some misc optimizations (#21039) 2023-06-21 16:16:06 +08:00
shuke
4766e75d4e [test](regression) add analyze after stream load (#21066) 2023-06-21 16:03:46 +08:00
DeadlineFen
4d84cd8ca1 Revert "Revert "[Test](regression) CCR syncer thrift interface regression test (#20935)" (#20990)" (#21022) 
This reverts commit 2a294801f1324a999570158eea3224239eefbb29.
 2023-06-21 15:20:21 +08:00
Yongqiang YANG
b65b821813 [enhancement](pk) add bvar stating cached io (#20977) 2023-06-21 15:02:10 +08:00
Yongqiang YANG
c5560b8f93 [fix](load) segcompaction does not signal waiters when an error hanppens (#21043) 
This leads to a deadlock.
 2023-06-21 14:56:34 +08:00
Qi Chen
bad22dd4e2 [Fix](orc-reader) Fix orc dict filter null value issue in _convert_dict_cols_to_string_cols which caused incorrect result. (#21047) 
Query results should not have empty values.
```
use regresssion.multi_catalog;
select commit_id from github_events_orc WHERE (event_type = 'CommitCommentEvent') AND commit_id != "" limit 10;
```
```
+------------------------------------------+
| commit_id                                |
+------------------------------------------+
| 685c1fd8dbbdc10c042932f9a9f88be00ff96c75 |
| 685c1fd8dbbdc10c042932f9a9f88be00ff96c75 |
| 4e3ab2ff2d2474f5d51334b9b0fdf17e9845a166 |
|                                          |
|                                          |
|                                          |
|                                          |
|                                          |
|                                          |
| 7191c20cb49da07a7fc16aa32dc0de4faff528b2 |
+------------------------------------------+
10 rows in set (0.54 sec) 
```
 2023-06-21 14:54:01 +08:00
zhannngchen
564b3533cf [enhancement](merge-on-write) update publish/streamload/compaction co… (#21040) 2023-06-21 14:49:51 +08:00
lihangyu
62fb0e642e [chore](dynamic schema) deprecated create dynamic schema table (#21058) 2023-06-21 14:44:57 +08:00
plat1ko
6f20cac1da [bugfix](cooldown) Fix potential deadlock while calling handleCooldownConf (#20975) 2023-06-21 14:44:01 +08:00
Gabriel
81abdeffbc [Improvement](pipeline) Improve shared scan performance (#20785) 2023-06-21 14:36:05 +08:00
Pxl
5f0bb49d46 [Feature](materialized-view) support create mv contain aggstate column (#20812) 
support create mv contain aggstate column
 2023-06-21 13:06:52 +08:00
lihangyu
fcd778fb4f [Fix](mysql proto) avoid send duplicated OK packet (#21032) 
1. The Mysql Go driver has a logic that terminates when it reads an EOF (end-of-file) and expects no data in the buffer. However, the front-end (FE) mistakenly returns an additional OK packet, which causes an exception to be thrown when reading the buffer.

2. Refactor some logic to support full prepared not just in where clause, like 
```
select ?, ? from tbl
```
 2023-06-21 12:00:22 +08:00
amory
18beb822a3 [FIX](array-type) fix array string output with fe const expr (#21042) 
fe foldconstRule make array() function expr with const literal , and would not pass this array literal to be . but we should make fe array string output format is same with be array string output
 2023-06-21 11:52:02 +08:00
Jerry Hu
5f760a8939 [fix](runtime_filter) remove incorrect DCHECK (#21050) 2023-06-21 11:27:53 +08:00
Ashin Gau
ef17289925 [feature](jni) add jni metrics and attach to BE profile automatically (#21004) 
Add JNI metrics, for example:
```
-  HudiJniScanner:  0ns
  -  FillBlockTime:  31.29ms
  -  GetRecordReaderTime:  1m5s
  -  JavaScanTime:  35s991ms
  -  OpenScannerTime:  1m6s
```
Add three common performance metrics for JNI scanner:
1. `OpenScannerTime`: Time to init and open JNI scanner
2. `JavaScanTime`: Time to scan data and insert into vector table in java side
3. `FillBlockTime`: Time to convert java vector table to c++ block

And support user defined metrics in java side, for example: `OpenScannerTime` is a long time for the open process, we want to determine which sub-process takes too much time, so we add `GetRecordReaderTime` in java side.
The user defined metrics in java side can be attached to BE profile automatically.
 2023-06-21 11:19:02 +08:00
Pxl
b4773e1195 [Chore](materialized-view) enable nereids planner on regression test mv_p0 (#21023) 
enable nereids planner on regression test mv_p0
 2023-06-21 10:01:27 +08:00
dujl
0cf9de8cef [fix](decimalv3) fix result error when cast a round decimalv3 to double (#20678) 2023-06-21 00:02:48 +08:00
HappenLee
ca6f51fcd5 [Performance] disable mmap alloc for doris performance (#21034) 
disable mmap alloc for some benchmark
 2023-06-20 23:27:49 +08:00
Xinyi Zou
6d579d924d [fix](profile) delete useless profile add_child #20989 2023-06-20 23:21:52 +08:00
yuanyuan8983
d7cc05502a [typo](doc) To access a Kafka cluster with PLAIN authentication enabled (#21019) 2023-06-20 23:19:59 +08:00
Xin Liao
b70a14d9c9 [fix](merge-on-write) fix that delete bitmap is not calculated correctly when has sequence column (#20955) 2023-06-20 21:36:47 +08:00
Kang
2c11ce0a02 [bugfix](topn) fix key topn merge block conflict with index predicate result columns (#20820) 2023-06-20 21:23:00 +08:00
airborne12
7a58a69aa9 [Fix](inverted index) skip index compaction when src rs did not have inverted index (#21010) 2023-06-20 21:22:25 +08:00
Xinyi Zou
ce1b39e79d [fix](profile) avoid unnecessary refresh profile of TabletsChannel 
Before, refresh the TabletsChannel profile in the LoadChannelMgr refresh memory statistics thread

This means that enable_profile=false will refresh and have performance loss in stress test
 2023-06-20 21:09:43 +08:00
Xinyi Zou
622ef63c69 [fix](memory) fix bthread_setspecific error in rpc done.run() (#20999) 2023-06-20 21:00:45 +08:00