46ca012e2b
[fix](bloom-filter) Fix error when handle empty string in bloom filter ( #7448 )
2021-12-31 16:05:33 +08:00
7357089e4e
[fix] change percentile_approx return from nan to null ( #7512 )
...
Change function percentile_approx return value from nan to null (like hive.) to ensure that return value of function percentile_approxcan be parsed by JDBC successfully.
Co-authored-by: weizuo <weizuo@xiaomi.com >
2021-12-30 10:24:35 +08:00
e93360791f
Revert "[improvement](planner) make BinaryPredicate do not cast date to datetime/varchar ( #7045 )" ( #7517 )
2021-12-28 23:05:27 +08:00
9fb89004aa
[revert] part of "[improvement](planner) make BinaryPredicate do not cast date to datetime/varchar ( #7045 )" ( #7501 )
2021-12-28 15:07:10 +08:00
07e2acb2f3
[feature] Suport national secret (national commercial password) algorithm SM3/SM4 ( #7464 )
...
SM3 is password hash algorithm
SM4 is a block cipher used to replace DES / AES and other international algorithms.
2021-12-28 10:39:54 +08:00
0c154733e0
[feature](function) support bitmap_union/intersect have more columns parameters ( #7379 )
...
support multi bitmap parameter for all bitmap aggregation function
2021-12-26 11:03:20 +08:00
3ba6dcf236
[fix](function) fix round function for inaccuracy ( #7421 )
2021-12-24 21:23:11 +08:00
ff5a0e98b0
[improvement](planner) make BinaryPredicate do not cast date to datetime/varchar ( #7045 )
2021-12-24 21:22:43 +08:00
0499b2211b
[feat](lateral-view) Support execution of lateral view stmt ( #7255 )
...
1. Add table function node
2. Add 3 table functions: explode_split, explode_bitmap and explode_json_array
2021-12-16 10:46:15 +08:00
62d12067aa
[feature](udf) make orthogonal bitmap udaf as build in functions ( #7211 )
...
move orthogonal bitmap udaf as build in functions
add three buildin bitmap functions:
- orthogonal_bitmap_intersect
- orthogonal_bitmap_intersect_count
- orthogonal_bitmap_union_count
2021-12-07 09:57:26 +08:00
d3316ff567
[performance](function) Support SIMD function in some string function ( #7236 )
...
Support SIMD function in some string function：lrtim，rtrim，trim，reverse，hex
2021-12-06 10:24:26 +08:00
d8ba6e3eb6
1. Fix an error when fetch string type field may cause malform packet error. ( #7262 )
...
This is beacuse of an const MAX_PHYSICAL_PACKET_LENGTH in fe should be 2^24 -1,
but it is set as 2^24 -2 by mistake.
2. Fix bitmap_to_string may failed when the result is large than 2G
2021-12-01 10:02:34 +08:00
a1bf2878c0
[feat-opt](json-function) optimize get_json_xx function ( #7157 )
...
Avoid repeated parsing json string is the first parameter of function is constant.
2021-11-26 10:12:55 +08:00
c9e578032b
optimize bitmap function count, use roaring cardinality method, this will more fast than current version ( #7151 )
2021-11-24 14:42:48 +08:00
a74fdf184c
[refactor](be) refactor predicate function creator ( #7054 )
...
Refactor predicate function creator, make MinMaxFunction/HybridSet/BloomFilter
use a unified interface through template to get function.
2021-11-24 10:39:29 +08:00
6c6380969b
[refactor] replace boost smart ptr with stl ( #6856 )
...
1. replace all boost::shared_ptr to std::shared_ptr
2. replace all boost::scopted_ptr to std::unique_ptr
3. replace all boost::scoped_array to std::unique<T[]>
4. replace all boost:thread to std::thread
2021-11-17 10:18:35 +08:00
632f8fcc75
[libhdfs] Add errno for hdfs writer. when no dir, hdfs writer open failed, the dir need to be created. ( #7050 )
...
1. Add errno message for hdfs writer failed.
2. When call openWrite for hdfs, the dir will be created when it doesn't exist,
2021-11-11 15:21:21 +08:00
29ca77622f
[Refactor] Refactor part of RuntimeFilter's code ( #6998 )
...
#6997
2021-11-07 17:40:45 +08:00
e69249c082
sub_bitmap ( #6977 )
...
Starting from the offset position, intercept the specified limit bitmap elements and return a bitmap subset.
Types of chang
2021-11-06 13:31:03 +08:00
760fc02bfe
Added bprc stub cache check and reset api, used to test whether the bprc stub cache is available, and reset the bprc stub cache ( #6916 )
...
Added bprc stub cache check and reset api, used to test whether the bprc stub cache is available, and reset the bprc stub cache
add a config used for auto check and reset bprc stub
2021-11-05 09:45:37 +08:00
599ecb1f30
[Function] Add bitmap function bitmap_subset_limit ( #6980 )
...
Add bitmap function bitmap_subset_limit.
This function will return subset in specified index.
2021-11-04 12:14:47 +08:00
aeec9c45e6
[Function] Add bitmap-xor-count function for doris ( #6982 )
...
Add bitmap-xor-count function for doris
relate to #6875
2021-11-02 16:37:00 +08:00
1ff3d708ca
[Function] add functions of bitmap_and/or_count ( #6912 )
...
issue #6875
add bitmap_and_count/ bitmap_or_count
2021-11-01 14:00:07 +08:00
c7a3116f98
[Function] add bitmap function of bitmap_has_all ( #6918 )
...
The 'bitmap_has_all' function returns true if the first bitmap contains all the elements of the second bitmap.
2021-11-01 12:50:47 +08:00
65ded82778
[Function] add BE bitmap function bitmap_subset_in_range ( #6917 )
...
Add bitmap function bitmap_subset_in_range.
This function will return subset in specified range (not include the range_end).
2021-11-01 11:05:19 +08:00
28030294f7
[Feature] Support bitmap_and_not & bitmap_and_not_count ( #6910 )
...
Support bitmap_and_not & bitmap_and_not_count.
2021-11-01 10:11:54 +08:00
a842d41b87
[Function] add BE bitmap function bitmap_max ( #6942 )
...
Support bitmap_max.
2021-10-30 18:16:38 +08:00
77a954d02c
[Bug] Fix treat tuple_is_null_predicate is const expr cause core problem ( #6919 )
...
Fix treat tuple_is_null_predicate is const expr cause core problem
2021-10-27 09:54:25 +08:00
adb6bfdf74
[Bug] Fix bug that truncate table may change the storage medium property ( #6905 )
2021-10-25 10:07:27 +08:00
24d38614a0
[Dependency] Upgrade thirdparty libs ( #6766 )
...
Upgrade the following dependecies:
libevent -> 2.1.12
OpenSSL 1.0.2k -> 1.1.1l
thrift 0.9.3 -> 0.13.0
protobuf 3.5.1 -> 3.14.0
gflags 2.2.0 -> 2.2.2
glog 0.3.3 -> 0.4.0
googletest 1.8.0 -> 1.10.0
snappy 1.1.7 -> 1.1.8
gperftools 2.7 -> 2.9.1
lz4 1.7.5 -> 1.9.3
curl 7.54.1 -> 7.79.0
re2 2017-05-01 -> 2021-02-02
zstd 1.3.7 -> 1.5.0
brotli 1.0.7 -> 1.0.9
flatbuffers 1.10.0 -> 2.0.0
apache-arrow 0.15.1 -> 5.0.0
CRoaring 0.2.60 -> 0.3.4
orc 1.5.8 -> 1.6.6
libdivide 4.0.0 -> 5.0
brpc 0.97 -> 1.0.0-rc02
librdkafka 1.7.0 -> 1.8.0
after this pr compile doris should use build-env:1.4.0
2021-10-15 13:03:04 +08:00
adb9b0d9c6
[Bug] Return 0 when hex(0) ( #6837 )
2021-10-15 10:18:55 +08:00
58440b90f0
[Bug] Left() string function behaves not identically to the mysql implementation ( #6811 )
...
See Fix #6810
2021-10-15 10:17:21 +08:00
ad949c2f65
Optimize Hex and add related Doc ( #6697 )
...
I tested hex in a 1000w times for loop with random numbers，
old hex avg time cost is 4.92 s，optimize hex avg time cost is 0.46 s which faster nearly 10x.
2021-10-13 11:36:14 +08:00
0941322dd6
[Optimiaze] Optimize HyperLogLog ( #6625 )
...
1. Replace std::max with a ternary expression, std::max is much heavier than the ternary operator
2. Replace std::set with arrays, std::set is based on red-black trees, traversal will follow the chain domain, and cache hits are not good
3. Optimize the serialize function, improve the calculation speed of num_non_zero_registers by reducing branches, and the serialization of _registers after optimization is faster
4. The test found that the performance improvement is more obvious
2021-10-10 23:04:39 +08:00
020282e885
[Bug] Fix aes_decrypt to handle null input correctly. ( #6636 )
2021-09-14 11:19:55 +08:00
225bdb1fda
[Bug] fix replace function bug ( #6605 )
...
* fix replace function bug
* fix replace docs
2021-09-14 09:59:13 +08:00
b2f1e21a3b
[Bugs] Fix some bugs ( #6586 )
...
* fix regex lazy
* fix result file core
* fix dynamic partition replica and table name length bug
* fix replicanum 0
* fix delete bug
* renew proxy
Co-authored-by: morningman <chenmingyu@baidu.com >
2021-09-10 09:53:30 +08:00
4f744333c2
fix some core in local test: ( #6594 )
...
1. insert very large string value may coredump
2. some analitic functiuon and agg function result may be incorrect
3. string compare may be coredump when string type is too large
4. string type in delete condition can not process correctly
5. add text/blob as alias of string to compitable with mysql
6. fix string type min/max agg may process incorrectly
2021-09-10 09:52:03 +08:00
577ff01094
[Bug][Function] Fix pad function wrong result when len.val==str_char_size ( #6564 )
...
like #6563 and #6562
2021-09-07 11:55:49 +08:00
1206f9a17f
[RuntimeFilter] support TYPE_STRING in runtime filter ( #6573 )
2021-09-06 11:55:11 +08:00
7a15e583a7
[Feature]Support functions of json_array, json_object, json_quote ( #6504 )
2021-09-02 09:59:02 +08:00
bfb2252175
[RuntimeFilter] provide no simd block bloom filter implement to support arm ( #6511 )
2021-08-27 10:22:36 +08:00
acc5fd2f21
[BUG] Fix string type cast bug and runtime filter may core when not support avx2 ( #6495 )
...
* fix string type cast bug and runtime filter instructions may not support
* add arm support
2021-08-26 09:14:31 +08:00
7e30b28f3a
[Optimize] Speed up converting the data of other types to string in mysql_result_writer ( #6384 )
...
Co-authored-by: caiconghui <caiconghui@xiaomi.com >
2021-08-24 22:30:58 +08:00
66a7a4b294
[Feature] Support exact percentile aggregate function ( #6410 )
...
Support to calculate the exact percentile value array of numeric column `col` at the given percentage(s).
2021-08-18 15:56:06 +08:00
8738ce380b
Add long text type STRING, with a maximum length of 2GB. Usage is similar to varchar, and there is no guarantee for the performance of storing extremely long data ( #6391 )
2021-08-18 09:05:40 +08:00
285d44cd48
[BUG] Fix potential overflow exception when do money format for double ( #6408 )
...
* [BUG] Fix potential overflow bug when do money format for double
Co-authored-by: caiconghui <caiconghui@xiaomi.com >
2021-08-15 18:40:26 +08:00
9216735cfa
[New Featrue] Support Vectorization Execution Engine Interface For Doris ( #6329 )
...
1. FE vectorized plan code
2. Function register vec function
3. Diff function nullable type
4. New thirdparty code and new thrift struct
2021-08-11 14:54:06 +08:00
2f5b06ae70
[Bug][Optimize] Fix race condition problem and optimize do_money_format function ( #6350 )
...
* [Bug][Optimize] Fix race condition problem and optimize do_money_format function
Co-authored-by: caiconghui <caiconghui@xiaomi.com >
2021-08-06 16:29:34 +08:00
d1007afe80
Use fmt and std::from_chars to make convert integer to string and convert string to integer more efficient ( #6361 )
...
* [Optimize] optimize the speed of converting integer to string
* Use fmt and std::from_chars to make convert integer to string and convert string to integer more efficient
Co-authored-by: caiconghui <caiconghui@xiaomi.com >
2021-08-04 10:55:19 +08:00
