database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
4,827 Commits 1 Branch 0 Tags
19bc14cf8d96bd7990c0ad52d9356cf7e686a441
Commit Graph

80 Commits

Author SHA1 Message Date
yinzhijian
19bc14cf8d [feature-wip](array-type) Add array type support for vectorized parquet-orc scanner (#9856) 
Only support one level array now.
for example:
- nullable(array(nullable(tinyint))) is **support**.
- nullable(array(nullable(array(xx))) is **not support**.
 2022-06-09 12:11:47 +08:00
Pxl
f2aa5f32b8 [Feature] [Vectorized] Some pre-refactorings or interface additions for schema change (#9811) 
Some pre-refactorings or interface additions for schema change
 2022-06-07 15:04:57 +08:00
zhengshengjun
49d4798276 [fix](function) fix bug in time_round function (#9712) 2022-06-06 08:58:22 +08:00
zhangstar333
4ea5782838 [fix](Function) fix to_bitmap to return always not nullable (#9859) 2022-06-02 10:37:45 +08:00
Pxl
ac08c7ac91 [fix](vectorized) fix vcast expr input wrong row number (#9520) 2022-06-01 15:19:31 +08:00
Gabriel
632f7a3d3d [Feature] add weekday function on vectorized engine (#9901) 2022-06-01 14:47:37 +08:00
HappenLee
35f99faa0a [Bug][Vectorized] fix core dump on vcase_expr::close (#9893) 
Co-authored-by: lihaopeng <lihaopeng@baidu.com>
 2022-06-01 08:05:09 +08:00
Pxl
fa50b63cee fix core dump on vcase_expr::close (#9875) 2022-05-31 15:45:39 +08:00
Adonis Ling
f377c26bf7 [refactor][be] Optimize headers (#9708) 2022-05-30 16:12:10 +08:00
Jing Shen
7b98dd438d [feature](function) Add nvl function (#9726) 2022-05-30 09:43:00 +08:00
Pxl
f33ef32d92 [Bug] [Bitmap] change to_bitmap to always_not_nullable (#9716) 2022-05-28 17:33:55 +08:00
zhangstar333
6f61af7682 [Vectorized][java-udf] add datetime&&largeint&&decimal type to java-udf (#9440) 2022-05-20 10:26:09 +08:00
yinzhijian
bee5c2f8aa [feature-wip](parquet-vec) Support parquet scanner in vectorized engine (#9433) 2022-05-17 09:37:17 +08:00
camby
650e3a6ba0 [feature-wip](array-type) array_contains support more nested data types (#9170) 
Co-authored-by: cambyzju <zhuxiaoli01@baidu.com>
 2022-05-13 12:42:40 +08:00
Adonis Ling
718a51a388 [refactor][style] Use clang-format to sort includes (#9483) 2022-05-10 21:25:35 +08:00
camby
7e86c1beab [fix] UT MathFunctionTest.round_test fix (#9447) 
Function round support two format round(double) and round(double, int), the argument is variadic.
But FunctionBinaryArithmetic not support variadic argument now, make get_function for round(double, int) failed.

reproduce steps:
1. set enable_vectorized_engine=true;
2. try to call round(double, int);
```
> select round(10.12345,2);
ERROR 1105 (HY000): errCode = 2, detailMessage = Function round is not implemented
```
 2022-05-09 09:37:27 +08:00
Pxl
98bfeaf560 [Enhancement] [Vectorized] Refactor and optimize BinaryOperation (#9087) 2022-05-07 10:55:15 +08:00
Pxl
d2374dbd5e [fix](Lateral-View) fix outer combinator not work on non-vectorized (#9212) 2022-05-01 22:09:50 +08:00
chenlinzhong
c9961c9bb9 [style] clang-format all c++ code (#9305) 
- sh build-support/clang-format.sh  to  clang-format all c++ code
 2022-04-29 16:14:22 +08:00
Pxl
2d83167e50 [Feature] [Lateral-View] support outer combinator of table function (#9147) 2022-04-24 12:09:40 +08:00
Zhengguo Yang
ae680b4248 [UDF] support RPC udaf part 1: support create RPC udaf in fe (#8510) 2022-04-21 17:38:58 +08:00
Pxl
681f960257 [fix](storage)(vectorized) query get wrong result when read datetime type column (#8872) 2022-04-18 19:34:06 +08:00
Zhengguo Yang
290366787c [refactor] refactor code, replace some file with stl libs (#8759) 
1. replace ConditionVariables with std::condition_variable
2. repalace Mutex with std::mutex
3. repalce MonoTime with std::chrono
 2022-04-13 09:55:29 +08:00
Pxl
64cf64d1f8 remove unused code and opt int_div (#8966) 2022-04-13 09:51:01 +08:00
Zhengguo Yang
5a44eeaf62 [refactor] Unify all unit tests into one binary file (#8958) 
1. solved the previous delayed unit test file size is too large (1.7G+) and the unit test link time is too long problem problems
2. Unify all unit tests into one file to significantly reduce unit test execution time to less than 3 mins
3. temporarily disable stream_load_test.cpp, metrics_action_test.cpp, load_channel_mgr_test.cpp because it will re-implement part of the code and affect other tests
 2022-04-12 15:30:40 +08:00
Mingyu Chen
067309c466 [fix](compile) fix compilation bug (#8950) 2022-04-11 13:12:34 +08:00
Pxl
8a066e2586 [fix](vectorized) core dump on ST_AsText (#8870) 2022-04-11 09:39:32 +08:00
Gabriel
7f7172807f [feature](function)(vectorized) Support all geolocation functions on vectorized engine (#8846) 2022-04-11 09:36:53 +08:00
Gabriel
0d761f9909 [feature-wip][UDF][DIP-1] Support variable-size input and output for Java UDF (#8678) 
This feature is proposed in DSIP-1. This PR support variable-length input and output Java UDF.
 2022-04-11 09:36:16 +08:00
zbtzbtzbt
6ed59bb98b [refactor](code_style) remove useless inline #8933 
1.Member functions defined in a class are inline by default (implicitly), and do not need to be added
2.inline is a keyword used for implementation, which has no effect when placed before the function declaration
 2022-04-10 18:29:55 +08:00
camby
c5718928df [feature-wip](array-type) support explode and explode_outer table function (#8766) 
explode(ArrayColumn) desc:
> Create a row for each element in the array column. 

explode_outer(ArrayColumn) desc:
> Create a row for each element in the array column. Unlike explode, if the array is null or empty, it returns null.

Usage example:
1. create a table with array column, and insert some data;
2. open enable_lateral_view and enable_vectorized_engine;
```
set enable_lateral_view = true;
set enable_vectorized_engine=true;
```
3. use explode_outer
```
> select * from array_test;
+------+------+--------+
| k1   | k2   | k3     |
+------+------+--------+
|    3 | NULL | NULL   |
|    1 |    2 | [1, 2] |
|    2 |    3 | NULL   |
|    4 | NULL | []     |
+------+------+--------+

> select k1,explode_column from array_test LATERAL VIEW explode_outer(k3) TempExplodeView as explode_column;
+------+----------------+
| k1   | explode_column |
+------+----------------+
|    1 |              1 |
|    1 |              2 |
|    2 |           NULL |
|    4 |           NULL |
|    3 |           NULL |
+------+----------------+
```
4. explode usage example. explode return empty rows while the ARRAY is null or empty
```
> select k1,explode_column from array_test LATERAL VIEW explode(k3) TempExplodeView as explode_column;
+------+----------------+
| k1   | explode_column |
+------+----------------+
|    1 |              1 |
|    1 |              2 |
+------+----------------+
```
 2022-04-08 12:11:04 +08:00
Pxl
dbbc6549bd [feature](vectorized) support vexplode_bitmap (#8890) 2022-04-08 09:20:26 +08:00
Gabriel
3f04220d49 [typo] Fix typo in function.cpp (#8873) 2022-04-08 09:09:19 +08:00
Xinyi Zou
f90a1a1919 [fix](ut)(compile) Fix ut failure at functions_geo and compilation bug (#8843) 2022-04-05 21:30:40 +08:00
Gabriel
f3c6ddf651 [feature](function) Support geolocation functions on vectorized engine (#8790) 2022-04-03 10:50:54 +08:00
Lightman
6b0a642390 [feature][vectorized] Support explode json array func #8526 (#8539) 2022-04-03 10:06:47 +08:00
camby
4d516bece8 [feature-wip](array-type)Add element_at and subscript functions (#8597) 
Describe the overview of changes.
1. add function element_at;
2. support element_subscript([]) to get element of array, col_array[N] <==> element_at(col_array, N);
3. return error message instead of BE crash while array function execute failed;

element_at(array, index) desc:
>   Returns element of array at given **(1-based)** index. 
  If **index < 0**, accesses elements from the last to the first. 
  Returns NULL if the index exceeds the length of the array or the array is NULL.

Usage example:
1. create table with ARRAY type column and insert some data:
```
+------+------+--------+
| k1   | k2   | k3     |
+------+------+--------+
|    1 |    2 | [1, 2] |
|    2 |    3 | NULL   |
|    4 | NULL | []     |
|    3 | NULL | NULL   |
+------+------+--------+
```
2. enable vectorized:
```
set enable_vectorized_engine=true;
```
3. element_subscript([]) usage example:
```
> select k1,k3,k3[1] from array_test;
+------+--------+----------------------------+
| k1   | k3     | %element_extract%(`k3`, 1) |
+------+--------+----------------------------+
|    3 | NULL   |                       NULL |
|    1 | [1, 2] |                          1 |
|    2 | NULL   |                       NULL |
|    4 | []     |                       NULL |
+------+--------+----------------------------+
```
4. element_at function usage example:
```
> select k1,k3 from array_test where element_at(k3, -1) = 2;
+------+--------+
| k1   | k3     |
+------+--------+
|    1 | [1, 2] |
+------+--------+
```
 2022-04-02 12:03:56 +08:00
Gabriel
1ddfe20950 fix typo (#8714) 
fix typo
 2022-03-29 18:21:16 +08:00
wangbo
726eaa68ea [fix](vectorization) Vectorization decimal arithmetic inconsistent (#8626) 2022-03-28 10:12:39 +08:00
Gabriel
b89e4c7bba [feature-wip](java-udf) support java UDF with fixed-length input and output (#8516) 
This feature is propsoed in [DSIP-1](https://cwiki.apache.org/confluence/display/DORIS/DSIP-001%3A+Java+UDF). 
This PR support fixed-length input and output Java UDF. Phase I in DIP-1 is done after this PR.

To support Java UDF effeciently, I use no data copy in JNI call and all compute operations are off-heap in Java.
To achieve that, I use a UdfExecutor instead. 

For users, a UDF class must have a public evaluate method.
 2022-03-23 10:32:50 +08:00
camby
71ce3c4a6e [feature-wip](array-type) Add codes and UT for array_contains and array_position functions (#8401) (#8589) 
array_contains function Usage example:
1. create table with ARRAY column, and insert some data:
```
> select * from array_test;
+------+------+--------+
| k1   | k2   | k3     |
+------+------+--------+
|    1 |    2 | [1, 2] |
|    2 |    3 | NULL   |
|    4 | NULL | []     |
|    3 | NULL | NULL   |
+------+------+--------+
```
2. enable vectorized:
```
> set enable_vectorized_engine=true;
```
3. select with array_contains:
```
> select k1,array_contains(k3,1) from array_test;
+------+-------------------------+
| k1   | array_contains(`k3`, 1) |
+------+-------------------------+
|    3 |                    NULL |
|    1 |                       1 |
|    2 |                    NULL |
|    4 |                       0 |
+------+-------------------------+
```
4. also we can use array_contains in where condition
```
> select * from array_test where array_contains(k3,1);
+------+------+--------+
| k1   | k2   | k3     |
+------+------+--------+
|    1 |    2 | [1, 2] |
+------+------+--------+
```
5. array_position usage example
```
> select k1,k3,array_position(k3,2) from array_test;
+------+--------+-------------------------+
| k1   | k3     | array_position(`k3`, 2) |
+------+--------+-------------------------+
|    3 | NULL   |                    NULL |
|    1 | [1, 2] |                       2 |
|    2 | NULL   |                    NULL |
|    4 | []     |                       0 |
+------+--------+-------------------------+
```
 2022-03-22 15:42:40 +08:00
Pxl
be3d203289 [feature][vectorized] support table function explode_numbers() (#8509) 2022-03-22 11:38:00 +08:00
Pxl
fc3ad371c8 [fix](vec) fix regexp_replace get wrong result on clang (#8505) 2022-03-20 23:11:24 +08:00
Pxl
a8af8d2981 [fix](vectorized) fix core dump on get_json_string and add some ut (#8496) 2022-03-17 10:08:31 +08:00
Pxl
a824c3e489 [feature](vectorized) support lateral view (#8448) 2022-03-17 10:04:24 +08:00
zhangstar333
e0ef9b8f6c [refactor](vectorized) to_bitmap(-1) return NULL instead of return parse failed error_message (#8373) 2022-03-11 17:21:47 +08:00
Pxl
10c3712aa1 [fix](vectorized) fix arithmetic calculate get wrong result(#8226) 2022-03-09 13:03:57 +08:00
Pxl
cd8694e532 [feature][vectorized] support replace() (#8384) 2022-03-08 18:57:12 +08:00
zhangstar333
454b45bea3 [feature](vectorize)(function) support regexp&&sm4&&aes functions (#8307) 2022-03-08 13:14:02 +08:00
HappenLee
be4dd0c820 [fix][vectorized] Fix error cast to boolean (#8345) 2022-03-06 13:47:46 +08:00