database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
6,689 Commits 1 Branch 0 Tags
71d2d61d33fe079bbfefece3b8e94bca22bac31e
Commit Graph

6689 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
yiguolei
71d2d61d33 [chore](build release) remove doris home and user info from doris_be --version output (#13344) 
There will be personal info in doris_be --version, like this:

doris-0.0.0-trunk RELEASE (build git://hk-dev01/mnt/disk2/ygl/code/github/apache-doris/be/../@8b7d928af26318f71098f1be2ab03ed83b1955fd)
Built on Wed, 12 Oct 2022 18:36:44 CST by ygl@hk-dev01

Since we always not need this info, commit id is enough, I remove these redundant info, the new result is like this:

doris-0.0.0-trunk RELEASE (build git://hk-dev01@8b7d928)
Built on Thu, 13 Oct 2022 15:03:01 CST by hk-dev01
 2022-10-13 18:24:04 +08:00
zhannngchen
fe1524a287 [Enhancement](load) remove load mem limit (#13111) 
#12716 removed the mem limit for single load task, in this PR I propose to remove the session variable load_mem_limit, to avoid confusing.

For compatibility, load_mem_limit in thrift not removed, the value is set equal to exec_mem_limit in FE
 2022-10-13 17:19:22 +08:00
jakevin
4a6eb01ccb [refactor](Nereids): refactor UT by using Pattern and rename to remove consecutive (#13337) 
* rename

* refactor UT
 2022-10-13 16:41:51 +08:00
Gabriel
baf2689610 [Improvement](join) compute hash values by vectorized way (#13335) 2022-10-13 16:04:58 +08:00
yixiutt
87793b7c00 [bugfix](datatimev2) fix value column loss precision and scale (#13233) 
Co-authored-by: yixiutt <yixiu@selectdb.com>
 2022-10-13 15:39:53 +08:00
Zhengguo Yang
0ff04e81bc [fix](DynamicPartition) Not check max_dynamic_partition_num when disable DynamicPartition (#13267) 
Disable max_dynamic_partition_num check when disable DynamicPartition by ALTER TABLE tbl_name SET ("dynamic_partition.enable" = "false"), when max_dynamic_partition_num changed to larger and then changed to a lower value, the actual dynamic partition num may larger than max_dynamic_partition_num, and cannot disable DynamicPartition
 2022-10-13 14:37:39 +08:00
Pxl
c1ed7d4d7d [Bug](function) fix core dump on case when have 1000 condition #13315 2022-10-13 14:37:03 +08:00
Mingyu Chen
bdb8e08bd3 [fix](ci) rename the checks name for branch-1.1 (#13342) 2022-10-13 14:36:18 +08:00
starocean999
830183984a [fix](hash)update_hashes_with_value method should handle if input value is null (#13332) 
* [fix](hash)update_hashes_with_value method should handle if input value is null

* remove unnessasery xxHash64NullWithSeed
 2022-10-13 14:36:01 +08:00
jakevin
db7f955a70 [improve](Nereids): split otherJoinCondition with List. (#13216) 
* split otherJoinCondition with List.
 2022-10-13 13:49:46 +08:00
jakevin
4248c6f37c [improve](Nereids): avoid duplicated stats derive. (#13293) 2022-10-13 13:49:21 +08:00
Gabriel
3e84c04195 [Bug](predicate) fix nullptr in scan node (#13316) 2022-10-13 12:14:42 +08:00
xueweizhang
e08ba8d573 [feature](restore) Add new property 'reserve_dynamic_partition_enable' to restore statement (#12498) 
Add restore new property 'reserve_dynamic_partition_enable', which means you can
get a table with dynamic_partition_enable property which has the same value
as before the backup. before this commit, you always get a table with property
'dynamic_partition_enable=false' when restore.
 2022-10-13 11:16:15 +08:00
liujinhui
7147c77f22 [Enhancement](broker)Doris support obs broker load (#12781) 
1. Upgrade fs_broker module hadoop2.7.3->hadoop2.8.3
2. Support obs broker load

org.apache.doris.broker.hdfs.FileSystemManager add getOBSFileSystem method
 2022-10-13 09:44:13 +08:00
Jerry Hu
9b590ac4cb [improvement](olap) cache value of has_null in ColumnNullable (#13289) 2022-10-13 09:12:02 +08:00
Xinyi Zou
c494ca0ed4 [enhancement](memtracker) Print query memory usage log every second when memory_verbose_track is enabled (#13302) 2022-10-13 09:11:23 +08:00
Gabriel
d430aec3ae [Bug](bloomfilter) fix concurrency bug caused by bloom filter (#13306) 2022-10-13 09:10:02 +08:00
Zhengguo Yang
a27bcbe882 [bugfix](compatibility) Fix a compatibility problem case by emun value changed (#13305) 
Fix a compatibility problem case by enum value changed
 2022-10-13 09:09:19 +08:00
TengJianPing
f80ac34786 [regression](bloom filter) add test case for is not null (#13310) 2022-10-13 09:08:22 +08:00
Yongqiang YANG
9b0978fad9 [fix](test) make output of test stable (#13317) 2022-10-13 09:07:59 +08:00
minghong
0732ccbf0f [fix](regression) rowsDisjointWindows output is not ordered #13333 2022-10-13 09:05:07 +08:00
Pxl
a77808e103 [Enhancement](function) optimize decimal minus and plus #13320 2022-10-13 09:00:05 +08:00
luozenglin
d63a80eaba [fix](bitmap_intersect) fix bitmap_intersect result error (#13298) 2022-10-12 19:12:11 +08:00
Gabriel
dfe308f501 [Improvement](join) refine prefetch strategy (#13286) 2022-10-12 19:02:06 +08:00
slothever
4fc7a048d2 [feature-wip](parquet-reader) fix string test and support decimal64 (#13184) 
1. Refactor arguments list of parquet min max filter, pass parquet type for  min max value parsing
2. Fix the filter of string min max

Co-authored-by: jinzhe <jinzhe@selectdb.com>
 2022-10-12 16:52:28 +08:00
Ashin Gau
bb4414e303 [feature-wip](multi-catalog) optimize parquet profile & add null map timer (#13257) 
Use indentation to make `ParquetReader`'s profile more readable
Add `ParquetReader.DecodeNullMapTime` to show the time of parsing `NullMap` for `NullableColumn`

```
VFILE_SCAN_NODE  (id=0):(Active:  279.62ms,  %  non-child:  85.83%)
    -  FileReadBytes:  2.36  MB
    -  FileReadCalls:  20
    -  FileReadTime:  5.686ms
    -  MaxScannerThreadNum:  1
    -  NewlyCreateFreeBlocksNum:  125
    -  NumScanners:  1
    -  ParquetReader:  0ns
        -  ColumnReadTime:  259.946ms
        -  DecodeDictTime:  0ns
        -  DecodeHeaderTime:  437.707us
        -  DecodeLevelTime:  30.101us
        -  DecodeNullMapTime:  53.295ms
        -  DecodeValueTime:  62.607ms
        -  DecompressCount:  511
        -  DecompressTime:  1.159ms
        -  FilteredBytes:  0.00  
        -  FilteredGroups:  0
        -  FilteredRowsByGroup:  0
        -  FilteredRowsByPage:  0
        -  ParseMetaTime:  22.517ms
        -  ReadBytes:  2.36  MB
        -  ReadGroups:  20
```
 2022-10-12 16:51:06 +08:00
Tiewei Fang
b7621e1615 [feature-wip](new-scan) support csv reader (#13282) 
Issue Number: close #12574
This pr adds CsvReader which implements GenericReader interface to support read csv format file.
 2022-10-12 16:22:13 +08:00
zhannngchen
4a5095f00d [cleanup](config) remove unused config push_write_mbytes_per_sec (#13290) 2022-10-12 15:58:04 +08:00
Kang
1bd14f1d82 [feature-wip](jsonb) jsonb parse function and load (#13129) 
add function to parse json string to jsonb format and use it to support stream load.
 2022-10-12 13:56:37 +08:00
Xin Liao
239e5b9943 [enhancement](storage) set the segment cache capacity according to the open file limit of the process (#13269) 2022-10-12 12:10:58 +08:00
pengxiangyu
af7b6524f2 add hide config to hide config in webserver for safety. (#13255) 2022-10-12 10:27:09 +08:00
Liqf
917d35a184 [typo](docs)Fix a document problem #13296 2022-10-12 10:08:48 +08:00
Xinyi Zou
89b295c6cc [enhancement](memory) Print memory usage log when memory allocation fails (#13301) 2022-10-12 10:08:25 +08:00
zhangstar333
16999ef02d [Vectorized][Function] support date_trunc and countequal function (#13039) 2022-10-12 10:01:09 +08:00
Mingyu Chen
1f034a5d1f [community](asf) add branch-1.1-lts protection (#12993) 2022-10-12 09:41:52 +08:00
Liqf
022cfb6979 [typo](docs)delete duplicate document and fix some problem (#13274) 2022-10-12 09:09:05 +08:00
Pxl
5c68f69362 [improvement](config) set enable_local_exchange default value to true (#13292) 2022-10-12 09:07:24 +08:00
Xinyi Zou
df54c6b63a [enhancement](memtracker) Add independent and unique scanner mem tracker for each query (#13262) 2022-10-11 19:47:12 +08:00
Kikyou1997
3c5e7e2f24 [feature](nereids) refactor statistics framework and introduce StatsCalculatorV2 (#12987) 
* squash

change data type of metrics to double

unit test

add stats for some function

add stats for arithmeticExpr

1. set max/min of ColumnStats to double
2. add stats for binaryExpr/compoundExpr

in predicate

* Add LiteralExpr in ColumnStat just for user display only.
 2022-10-11 17:23:49 +08:00
Mingyu Chen
5af1439934 [feature](auth) support user password policy and alter user stmt (#13051) 2022-10-11 16:37:35 +08:00
赵立伟
334708dc8c [fix](memory): avoid coredump when list pointer is null (#12919) 2022-10-11 16:00:23 +08:00
zhannngchen
48b182023f [docs](broker load) add doc for property load_parallelism (#13041) 2022-10-11 15:53:25 +08:00
Yongqiang YANG
e8e171e0a3 [improvement](log) limit nums of logging disable auto compaction (#13113) 2022-10-11 15:52:56 +08:00
Gabriel
e20921fa87 [function](datev2) Complements for datev2 functions (#13276) 2022-10-11 15:52:00 +08:00
Gabriel
1724a91f53 [Bug](predicate) Cover all const predicates in scan node (#13238) 
For an vectorized expression which meets the condition vexpr->is_constant(), a const column is expected to return.
But now we still don't cover all predicates for const expression.
For example, for query SELECT col FROM tbl WHERE 'PROMOTION' LIKE 'AAA%', predicate like will return a ColumnVector which contains a single value.

This PR want to cover all const predicates in scan node whether it returns a constcolumn or not
 2022-10-11 15:49:53 +08:00
Liqf
230efa29dd [typo](docs)add orthogonal bitmap function note. #13078 2022-10-11 15:46:56 +08:00
Hong Liu
88264b3e7b [regression-test](unique) Check the table version upper 1 before select operation. #13205 
Co-authored-by: smallhibiscus <8449081280@qq.com>
 2022-10-11 15:44:17 +08:00
jiafeng.zhang
9b42f7e479 [typo](docs)Modification instructions and examples for adding schema change key columns (#13280) 2022-10-11 15:42:14 +08:00
camby
4e4f8afa28 [fix](array-type) fix get_data_at for zero element array #13225 
Co-authored-by: cambyzju <zhuxiaoli01@baidu.com>
 2022-10-11 15:41:34 +08:00
caoliang-web
eb60976c25 [typo](docs)fix error url (#13171) 
* fix error url
 2022-10-11 15:41:00 +08:00