15abafee71
[Bug](runtime filters) support late-arrival runtime filters ( #11599 )
2022-08-12 11:55:15 +08:00
f5fe622a1b
[Bug](materialized view) fix create materialized view fail
...
1. remove referenced_column(seems unused now).
2. fix mv slot ref id wrong.
3. add type check for hll_hash.
4. enable non-nullable column change to nullable column.
2022-08-12 09:49:16 +08:00
ed7f7dead9
[Refactor](push-down predicate) Derive push-down predicate from vconjuncts ( #11468 )
...
* [Refactor](push-down predicate) Derive push-down predicate from vconjuncts
2022-08-08 19:19:26 +08:00
092a394782
[improvement](agg)limit the output of agg node ( #11461 )
...
* [improvement](agg)limit the output of agg node
2022-08-05 07:53:55 +08:00
0b1d06bfd6
[Vectorized] Support order by aggregate function ( #11187 )
...
Co-authored-by: lihaopeng <lihaopeng@baidu.com >
2022-07-28 09:12:58 +08:00
72d2feae99
[feature-wip] Support all date functions for datev2/datetimev2 ( #11265 )
...
* [feature-wip] (datetimev2) support convert_tz function
* [feature-wip] Support all date functions for datev2/datetimev2
2022-07-28 08:18:59 +08:00
babab5d535
[feature-wip] support datetimev2 ( #11085 )
2022-07-23 16:07:59 +08:00
fdb4193e1b
[Vectorized][Refactor] Refactor the function of tuple_is_null, only do work in hash join node ( #11109 )
2022-07-23 11:50:07 +08:00
b7c9007776
[improvement][agg]Process aggregated results in the vectorized way ( #11084 )
2022-07-22 22:04:43 +08:00
4960043f5e
[enhancement] Refactor to improve the usability of MemTracker (step2) ( #10823 )
2022-07-21 17:11:28 +08:00
56e036e68b
[feature-wip](multi-catalog) Support runtime filter for file scan node ( #11000 )
...
* [feature-wip](multi-catalog) Support runtime filter for file scan node
Co-authored-by: morningman <morningman@apache.org >
2022-07-20 12:36:57 +08:00
6736e06679
[feature](udf) Vectorization support remote udaf #10683 ( #10685 )
2022-07-18 17:15:34 +08:00
09d19e3f0f
[feature-wip](array-type) explode support more sub types ( #10673 )
...
1. explode support more sub types;
2. explode support nullable elements;
Co-authored-by: cambyzju <zhuxiaoli01@baidu.com >
2022-07-17 18:08:30 +08:00
ad4751972c
[feature-wip] Support in predicate for datev2 type ( #10810 )
2022-07-15 14:32:40 +08:00
18348a83ad
[chore][compile] fix java udf compile error ( #10841 )
2022-07-14 20:51:44 +08:00
3b46242483
[feature-wip] Optimize Decimal type ( #10794 )
...
* [feature-wip](decimalv3) support decimalv3
* [feature-wip] Optimize Decimal type
Co-authored-by: liaoxin <liaoxinbit@126.com >
2022-07-14 10:50:50 +08:00
08384fea1c
[BUG] fix DCHECK failed for vectorized InPredicate ( #10709 )
2022-07-09 06:25:32 +08:00
f58a071605
[Bug][Function] pass intermediate argument list to be ( #10650 )
2022-07-08 20:50:05 +08:00
7bfe438884
[BUG] fix bug in literal debug_string when literal is null ( #10567 )
2022-07-04 08:57:55 +08:00
c9f86bc7e2
[refactor] Refactoring Status static methods to format message using fmt( #9533 )
2022-07-02 18:58:23 +08:00
ca94867b4e
[Feature-wip] add date v2 type ( #9916 )
2022-06-26 16:07:56 +08:00
ed1e130ef6
[BUGFIX] fix wrong children quantity in debug string ( #10348 )
2022-06-23 09:10:30 +08:00
200557052a
[BUGFIX] wrong answer with with as + two phase agg ( #10303 )
2022-06-22 14:39:39 +08:00
84f57398d9
[Improvement] set debug string for VExpressions ( #10166 )
2022-06-21 07:43:25 +08:00
588634ddf6
[feature] support runtime filter on vectorized engine ( #10103 )
2022-06-20 09:46:38 +08:00
4c24586865
[Vectorized][UDF] support java-udaf ( #9930 )
2022-06-15 10:53:44 +08:00
f2aa5f32b8
[Feature] [Vectorized] Some pre-refactorings or interface additions for schema change ( #9811 )
...
Some pre-refactorings or interface additions for schema change
2022-06-07 15:04:57 +08:00
ac08c7ac91
[fix](vectorized) fix vcast expr input wrong row number ( #9520 )
2022-06-01 15:19:31 +08:00
35f99faa0a
[Bug][Vectorized] fix core dump on vcase_expr::close ( #9893 )
...
Co-authored-by: lihaopeng <lihaopeng@baidu.com >
2022-06-01 08:05:09 +08:00
fa50b63cee
fix core dump on vcase_expr::close ( #9875 )
2022-05-31 15:45:39 +08:00
f377c26bf7
[refactor][be] Optimize headers ( #9708 )
2022-05-30 16:12:10 +08:00
63aab5ee5d
[Bugfix(Vec)] Fix some memory leak issues ( #9824 )
2022-05-29 23:04:11 +08:00
f4dd3bf013
[bugfix] fix memleak in olapscannode( #9736 )
2022-05-26 15:06:54 +08:00
718a51a388
[refactor][style] Use clang-format to sort includes ( #9483 )
2022-05-10 21:25:35 +08:00
c9961c9bb9
[style] clang-format all c++ code ( #9305 )
...
- sh build-support/clang-format.sh to clang-format all c++ code
2022-04-29 16:14:22 +08:00
d330bc3806
[Vectorized](stream-load-vec) Support stream load in vectorized engine ( #8709 ) ( #9280 )
...
Implement vectorized stream load.
Added fe configuration option `enable_vectorized_load` to enable vectorized stream load.
Co-authored-by: tengjp@outlook.com
Co-authored-by: mrhhsg@gmail.com
Co-authored-by: minghong.zhou@163.com
Co-authored-by: HappenLee <happenlee@hotmail.com >
Co-authored-by: zhoubintao <35688959+zbtzbtzbt@users.noreply.github.com >
2022-04-29 09:50:51 +08:00
2d83167e50
[Feature] [Lateral-View] support outer combinator of table function ( #9147 )
2022-04-24 12:09:40 +08:00
ae680b4248
[UDF] support RPC udaf part 1: support create RPC udaf in fe ( #8510 )
2022-04-21 17:38:58 +08:00
c5718928df
[feature-wip](array-type) support explode and explode_outer table function ( #8766 )
...
explode(ArrayColumn) desc:
> Create a row for each element in the array column.
explode_outer(ArrayColumn) desc:
> Create a row for each element in the array column. Unlike explode, if the array is null or empty, it returns null.
Usage example:
1. create a table with array column, and insert some data;
2. open enable_lateral_view and enable_vectorized_engine;
```
set enable_lateral_view = true;
set enable_vectorized_engine=true;
```
3. use explode_outer
```
> select * from array_test;
+------+------+--------+
| k1 | k2 | k3 |
+------+------+--------+
| 3 | NULL | NULL |
| 1 | 2 | [1, 2] |
| 2 | 3 | NULL |
| 4 | NULL | [] |
+------+------+--------+
> select k1,explode_column from array_test LATERAL VIEW explode_outer(k3) TempExplodeView as explode_column;
+------+----------------+
| k1 | explode_column |
+------+----------------+
| 1 | 1 |
| 1 | 2 |
| 2 | NULL |
| 4 | NULL |
| 3 | NULL |
+------+----------------+
```
4. explode usage example. explode return empty rows while the ARRAY is null or empty
```
> select k1,explode_column from array_test LATERAL VIEW explode(k3) TempExplodeView as explode_column;
+------+----------------+
| k1 | explode_column |
+------+----------------+
| 1 | 1 |
| 1 | 2 |
+------+----------------+
```
2022-04-08 12:11:04 +08:00
dbbc6549bd
[feature](vectorized) support vexplode_bitmap ( #8890 )
2022-04-08 09:20:26 +08:00
519305cb22
[feature-wip] (memory tracker) (step4) Switch TLS mem tracker to separate more detailed memory usage ( #8669 )
...
Based on #8605 , Separate out the memory usage of each operator from the Query/Load/StorageEngine mem tracker.
2022-04-08 09:02:26 +08:00
6b0a642390
[feature][vectorized] Support explode json array func #8526 ( #8539 )
2022-04-03 10:06:47 +08:00
b89e4c7bba
[feature-wip](java-udf) support java UDF with fixed-length input and output ( #8516 )
...
This feature is propsoed in [DSIP-1](https://cwiki.apache.org/confluence/display/DORIS/DSIP-001%3A+Java+UDF ).
This PR support fixed-length input and output Java UDF. Phase I in DIP-1 is done after this PR.
To support Java UDF effeciently, I use no data copy in JNI call and all compute operations are off-heap in Java.
To achieve that, I use a UdfExecutor instead.
For users, a UDF class must have a public evaluate method.
2022-03-23 10:32:50 +08:00
2580da4f72
[feature-wip](array-type) Support insertion for vectorized engine. ( #8494 ) ( #8590 )
...
Please refer to #8493
2022-03-22 15:48:13 +08:00
a498463ab5
[feature-wip](array-type)support select ARRAY data type on vectorized engine ( #8217 ) ( #8584 )
...
Usage Example:
1. create table for test;
```
`CREATE TABLE `array_test` (
`k1` tinyint(4) NOT NULL COMMENT "",
`k2` smallint(6) NULL COMMENT "",
`k3` ARRAY<int(11)> NULL COMMENT ""
) ENGINE=OLAP
DUPLICATE KEY(`k1`)
COMMENT "OLAP"
DISTRIBUTED BY HASH(`k1`) BUCKETS 5
PROPERTIES (
"replication_allocation" = "tag.location.default: 1",
"in_memory" = "false",
"storage_format" = "V2"
);`
```
2. insert some data
```
`insert into array_test values(1, 2, [1, 2]);`
`insert into array_test values(2, 3, null);`
`insert into array_test values(3, null, null);`
`insert into array_test values(4, null, []);`
```
3. open vectorized
`set enable_vectorized_engine=true;`
4. query array data
`select * from array_test;`
+------+------+--------+
| k1 | k2 | k3 |
+------+------+--------+
| 4 | NULL | [] |
| 2 | 3 | NULL |
| 1 | 2 | [1, 2] |
| 3 | NULL | NULL |
+------+------+--------+
4 rows in set (0.061 sec)
Code Changes include：
1. add column_array, data_type_array codes;
2. codes about data_type creation by Field, TabletColumn, TypeDescriptor, PColumnMeta move to DataTypeFactory;
3. support create data_type for ARRAY date type;
4. RowBlockV2::convert_to_vec_block support ARRAY date type;
5. VMysqlResultWriter::append_block support ARRAY date type;
6. vectorized::Block serialize and deserialize support ARRAY date type;
2022-03-22 15:21:44 +08:00
be3d203289
[feature][vectorized] support table function explode_numbers() ( #8509 )
2022-03-22 11:38:00 +08:00
eeae516e37
[Feature](Memory) Hook TCMalloc new/delete automatically counts to MemTracker ( #8476 )
...
Early Design Documentation: https://shimo.im/docs/DT6JXDRkdTvdyV3G
Implement a new way of memory statistics based on TCMalloc New/Delete Hook,
MemTracker and TLS, and it is expected that all memory new/delete/malloc/free
of the BE process can be counted.
2022-03-20 23:06:54 +08:00
a824c3e489
[feature](vectorized) support lateral view ( #8448 )
2022-03-17 10:04:24 +08:00
2c63fc1d6c
[improvement](vectorized) Support BetweenPredicate enable fold const expr ( #8450 )
2022-03-13 09:36:24 +08:00
68dd799796
[improvement](vectorized) Support function tuple is null ( #8442 )
2022-03-11 16:54:37 +08:00
