database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
8,276 Commits 1 Branch 0 Tags
3894de49d289aa531cc6f4f7bd161cc55dc48c5b
Commit Graph

8276 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
liujinhui
7147c77f22 [Enhancement](broker)Doris support obs broker load (#12781) 
1. Upgrade fs_broker module hadoop2.7.3->hadoop2.8.3
2. Support obs broker load

org.apache.doris.broker.hdfs.FileSystemManager add getOBSFileSystem method
 2022-10-13 09:44:13 +08:00
Jerry Hu
9b590ac4cb [improvement](olap) cache value of has_null in ColumnNullable (#13289) 2022-10-13 09:12:02 +08:00
Xinyi Zou
c494ca0ed4 [enhancement](memtracker) Print query memory usage log every second when memory_verbose_track is enabled (#13302) 2022-10-13 09:11:23 +08:00
Gabriel
d430aec3ae [Bug](bloomfilter) fix concurrency bug caused by bloom filter (#13306) 2022-10-13 09:10:02 +08:00
Zhengguo Yang
a27bcbe882 [bugfix](compatibility) Fix a compatibility problem case by emun value changed (#13305) 
Fix a compatibility problem case by enum value changed
 2022-10-13 09:09:19 +08:00
TengJianPing
f80ac34786 [regression](bloom filter) add test case for is not null (#13310) 2022-10-13 09:08:22 +08:00
Yongqiang YANG
9b0978fad9 [fix](test) make output of test stable (#13317) 2022-10-13 09:07:59 +08:00
minghong
0732ccbf0f [fix](regression) rowsDisjointWindows output is not ordered #13333 2022-10-13 09:05:07 +08:00
Pxl
a77808e103 [Enhancement](function) optimize decimal minus and plus #13320 2022-10-13 09:00:05 +08:00
luozenglin
d63a80eaba [fix](bitmap_intersect) fix bitmap_intersect result error (#13298) 2022-10-12 19:12:11 +08:00
Gabriel
dfe308f501 [Improvement](join) refine prefetch strategy (#13286) 2022-10-12 19:02:06 +08:00
slothever
4fc7a048d2 [feature-wip](parquet-reader) fix string test and support decimal64 (#13184) 
1. Refactor arguments list of parquet min max filter, pass parquet type for  min max value parsing
2. Fix the filter of string min max

Co-authored-by: jinzhe <jinzhe@selectdb.com>
 2022-10-12 16:52:28 +08:00
Ashin Gau
bb4414e303 [feature-wip](multi-catalog) optimize parquet profile & add null map timer (#13257) 
Use indentation to make `ParquetReader`'s profile more readable
Add `ParquetReader.DecodeNullMapTime` to show the time of parsing `NullMap` for `NullableColumn`

```
VFILE_SCAN_NODE  (id=0):(Active:  279.62ms,  %  non-child:  85.83%)
    -  FileReadBytes:  2.36  MB
    -  FileReadCalls:  20
    -  FileReadTime:  5.686ms
    -  MaxScannerThreadNum:  1
    -  NewlyCreateFreeBlocksNum:  125
    -  NumScanners:  1
    -  ParquetReader:  0ns
        -  ColumnReadTime:  259.946ms
        -  DecodeDictTime:  0ns
        -  DecodeHeaderTime:  437.707us
        -  DecodeLevelTime:  30.101us
        -  DecodeNullMapTime:  53.295ms
        -  DecodeValueTime:  62.607ms
        -  DecompressCount:  511
        -  DecompressTime:  1.159ms
        -  FilteredBytes:  0.00  
        -  FilteredGroups:  0
        -  FilteredRowsByGroup:  0
        -  FilteredRowsByPage:  0
        -  ParseMetaTime:  22.517ms
        -  ReadBytes:  2.36  MB
        -  ReadGroups:  20
```
 2022-10-12 16:51:06 +08:00
Tiewei Fang
b7621e1615 [feature-wip](new-scan) support csv reader (#13282) 
Issue Number: close #12574
This pr adds CsvReader which implements GenericReader interface to support read csv format file.
 2022-10-12 16:22:13 +08:00
zhannngchen
4a5095f00d [cleanup](config) remove unused config push_write_mbytes_per_sec (#13290) 2022-10-12 15:58:04 +08:00
Kang
1bd14f1d82 [feature-wip](jsonb) jsonb parse function and load (#13129) 
add function to parse json string to jsonb format and use it to support stream load.
 2022-10-12 13:56:37 +08:00
Xin Liao
239e5b9943 [enhancement](storage) set the segment cache capacity according to the open file limit of the process (#13269) 2022-10-12 12:10:58 +08:00
pengxiangyu
af7b6524f2 add hide config to hide config in webserver for safety. (#13255) 2022-10-12 10:27:09 +08:00
Liqf
917d35a184 [typo](docs)Fix a document problem #13296 2022-10-12 10:08:48 +08:00
Xinyi Zou
89b295c6cc [enhancement](memory) Print memory usage log when memory allocation fails (#13301) 2022-10-12 10:08:25 +08:00
zhangstar333
16999ef02d [Vectorized][Function] support date_trunc and countequal function (#13039) 2022-10-12 10:01:09 +08:00
Mingyu Chen
1f034a5d1f [community](asf) add branch-1.1-lts protection (#12993) 2022-10-12 09:41:52 +08:00
Liqf
022cfb6979 [typo](docs)delete duplicate document and fix some problem (#13274) 2022-10-12 09:09:05 +08:00
Pxl
5c68f69362 [improvement](config) set enable_local_exchange default value to true (#13292) 2022-10-12 09:07:24 +08:00
Xinyi Zou
df54c6b63a [enhancement](memtracker) Add independent and unique scanner mem tracker for each query (#13262) 2022-10-11 19:47:12 +08:00
Kikyou1997
3c5e7e2f24 [feature](nereids) refactor statistics framework and introduce StatsCalculatorV2 (#12987) 
* squash

change data type of metrics to double

unit test

add stats for some function

add stats for arithmeticExpr

1. set max/min of ColumnStats to double
2. add stats for binaryExpr/compoundExpr

in predicate

* Add LiteralExpr in ColumnStat just for user display only.
 2022-10-11 17:23:49 +08:00
Mingyu Chen
5af1439934 [feature](auth) support user password policy and alter user stmt (#13051) 2022-10-11 16:37:35 +08:00
赵立伟
334708dc8c [fix](memory): avoid coredump when list pointer is null (#12919) 2022-10-11 16:00:23 +08:00
zhannngchen
48b182023f [docs](broker load) add doc for property load_parallelism (#13041) 2022-10-11 15:53:25 +08:00
Yongqiang YANG
e8e171e0a3 [improvement](log) limit nums of logging disable auto compaction (#13113) 2022-10-11 15:52:56 +08:00
Gabriel
e20921fa87 [function](datev2) Complements for datev2 functions (#13276) 2022-10-11 15:52:00 +08:00
Gabriel
1724a91f53 [Bug](predicate) Cover all const predicates in scan node (#13238) 
For an vectorized expression which meets the condition vexpr->is_constant(), a const column is expected to return.
But now we still don't cover all predicates for const expression.
For example, for query SELECT col FROM tbl WHERE 'PROMOTION' LIKE 'AAA%', predicate like will return a ColumnVector which contains a single value.

This PR want to cover all const predicates in scan node whether it returns a constcolumn or not
 2022-10-11 15:49:53 +08:00
Liqf
230efa29dd [typo](docs)add orthogonal bitmap function note. #13078 2022-10-11 15:46:56 +08:00
Hong Liu
88264b3e7b [regression-test](unique) Check the table version upper 1 before select operation. #13205 
Co-authored-by: smallhibiscus <8449081280@qq.com>
 2022-10-11 15:44:17 +08:00
jiafeng.zhang
9b42f7e479 [typo](docs)Modification instructions and examples for adding schema change key columns (#13280) 2022-10-11 15:42:14 +08:00
camby
4e4f8afa28 [fix](array-type) fix get_data_at for zero element array #13225 
Co-authored-by: cambyzju <zhuxiaoli01@baidu.com>
 2022-10-11 15:41:34 +08:00
caoliang-web
eb60976c25 [typo](docs)fix error url (#13171) 
* fix error url
 2022-10-11 15:41:00 +08:00
Yongqiang YANG
55fc55d5e3 [improvement](tcmalloc) increase tcmalloc upper limit to 90% (#13245) 2022-10-11 15:40:24 +08:00
FreeOnePlus
a716c74412 [typo](docs)Fix Docs Error Urls (#13176) 
* fix doc
 2022-10-11 15:40:03 +08:00
yuanyuan8983
b1cd87d635 [typo](docs)Fix FE Configuration Jump Link 404 (#13149) 
* [typo](docs)Fix FE Configuration Jump Link 404
 2022-10-11 15:39:25 +08:00
Jerry Hu
606b514329 [fix](olap) fix core dump caused by LikeColumnPredicate with nullable column (#13250) 2022-10-11 15:38:55 +08:00
Adonis Ling
9350c0be80 [Bug](aarch64) Fix the BRPC "sched_to itself" error when building BE by Clang on Linux aarch64 (#13270) 2022-10-11 15:38:28 +08:00
yuanyuan8983
6dad7ee5f5 [typo](docs) Fix jump link 404 in elastic-expansion.md (#13168) 
* [typo](docs) Fix jump link 404
 2022-10-11 15:38:17 +08:00
yuanyuan8983
9c776c1011 [typo](docs) Fix the jump link 404 in basic usage.md (#13169) 
* [typo](docs) Fix the jump link 404
 2022-10-11 15:38:00 +08:00
Mingyu Chen
c1ce48ffe4 [fix](new-scann) scanner may be marked close twice (#13263) 2022-10-11 15:37:15 +08:00
jiafeng.zhang
ed19562cb3 And ali datax unified configuration naming, modify maxBatchSize to batchSize(#13278) 
And ali datax unified configuration naming, modify maxBatchSize to batchSize
 2022-10-11 14:51:19 +08:00
caoliang-web
0b9e9ac209 metadata operation fix 404 error url (#13215) 
metadata operation fix 404 error url
 2022-10-11 14:11:11 +08:00
Kikyou1997
2b5db24d82 remove unstable q21 (#13271) 2022-10-11 12:07:14 +08:00
Kikyou1997
b5da751c2a [enhancement](Nereids) remove redundant log when fall back to legacy parser (#13243) 2022-10-11 10:53:07 +08:00
jiafeng.zhang
6ee150755a [refactor](datax)Refactoring doris writer code (#13226) 
* Refactoring doris writer code
 2022-10-11 08:47:05 +08:00