62e3bd338e
[refactor](BE) return error status when vslot_ref contains invalid slot_id ( #12106 )
...
In current implementation, we detect invalid slot at execute phase. At execute phase, it is hard to get useful information for further debug. This pr moves error detection ahead to prepare phase, so that we can log related tuple descriptors.
2022-08-29 12:07:08 +08:00
17b809210a
[Bug](runtime filter) fix bug for late-arrival runtime filters ( #12049 )
2022-08-26 09:13:10 +08:00
d06edd4b8b
[minor](runtime-filter) add DCHECK for runtimefilter bug ( #11996 )
...
Not a fix, just add debug info to try find root cause of #11995
2022-08-24 07:53:30 +08:00
dc8f64b3e3
[improvement](agg) Serialize the fixed-length aggregation results with corresponding columns instead of ColumnString ( #11801 )
2022-08-22 10:12:06 +08:00
64dc3b360f
[Bug](function) fix dcheck fail on close vexpr ctx ( #11908 )
2022-08-19 19:11:10 +08:00
8b10a1a3f7
[enhancement](VSlotRef) enhance column_id check in execute function during runtime ( #11862 )
...
The column id check in VSlotRef::execute function before is too strict for fuzzy test to continuously produce random query. Temporarily loosen the check logic.
Moreover, there exists some careless call to VExpr::get_const_col, it might return a nullptr but not every function call checks if it's valid. It's an underlying problem.
2022-08-18 09:12:26 +08:00
288b440b14
[improvement](vectorized) Improve count distinct performance by using fastunion ( #11516 )
...
Improve count distinct performance by using fastunion.
Testing our user real data has a 10-40% performance improvement.
2022-08-16 12:18:46 +08:00
15abafee71
[Bug](runtime filters) support late-arrival runtime filters ( #11599 )
2022-08-12 11:55:15 +08:00
f5fe622a1b
[Bug](materialized view) fix create materialized view fail
...
1. remove referenced_column(seems unused now).
2. fix mv slot ref id wrong.
3. add type check for hll_hash.
4. enable non-nullable column change to nullable column.
2022-08-12 09:49:16 +08:00
ed7f7dead9
[Refactor](push-down predicate) Derive push-down predicate from vconjuncts ( #11468 )
...
* [Refactor](push-down predicate) Derive push-down predicate from vconjuncts
2022-08-08 19:19:26 +08:00
092a394782
[improvement](agg)limit the output of agg node ( #11461 )
...
* [improvement](agg)limit the output of agg node
2022-08-05 07:53:55 +08:00
0b1d06bfd6
[Vectorized] Support order by aggregate function ( #11187 )
...
Co-authored-by: lihaopeng <lihaopeng@baidu.com >
2022-07-28 09:12:58 +08:00
72d2feae99
[feature-wip] Support all date functions for datev2/datetimev2 ( #11265 )
...
* [feature-wip] (datetimev2) support convert_tz function
* [feature-wip] Support all date functions for datev2/datetimev2
2022-07-28 08:18:59 +08:00
babab5d535
[feature-wip] support datetimev2 ( #11085 )
2022-07-23 16:07:59 +08:00
fdb4193e1b
[Vectorized][Refactor] Refactor the function of tuple_is_null, only do work in hash join node ( #11109 )
2022-07-23 11:50:07 +08:00
b7c9007776
[improvement][agg]Process aggregated results in the vectorized way ( #11084 )
2022-07-22 22:04:43 +08:00
4960043f5e
[enhancement] Refactor to improve the usability of MemTracker (step2) ( #10823 )
2022-07-21 17:11:28 +08:00
56e036e68b
[feature-wip](multi-catalog) Support runtime filter for file scan node ( #11000 )
...
* [feature-wip](multi-catalog) Support runtime filter for file scan node
Co-authored-by: morningman <morningman@apache.org >
2022-07-20 12:36:57 +08:00
6736e06679
[feature](udf) Vectorization support remote udaf #10683 ( #10685 )
2022-07-18 17:15:34 +08:00
09d19e3f0f
[feature-wip](array-type) explode support more sub types ( #10673 )
...
1. explode support more sub types;
2. explode support nullable elements;
Co-authored-by: cambyzju <zhuxiaoli01@baidu.com >
2022-07-17 18:08:30 +08:00
ad4751972c
[feature-wip] Support in predicate for datev2 type ( #10810 )
2022-07-15 14:32:40 +08:00
18348a83ad
[chore][compile] fix java udf compile error ( #10841 )
2022-07-14 20:51:44 +08:00
3b46242483
[feature-wip] Optimize Decimal type ( #10794 )
...
* [feature-wip](decimalv3) support decimalv3
* [feature-wip] Optimize Decimal type
Co-authored-by: liaoxin <liaoxinbit@126.com >
2022-07-14 10:50:50 +08:00
08384fea1c
[BUG] fix DCHECK failed for vectorized InPredicate ( #10709 )
2022-07-09 06:25:32 +08:00
f58a071605
[Bug][Function] pass intermediate argument list to be ( #10650 )
2022-07-08 20:50:05 +08:00
7bfe438884
[BUG] fix bug in literal debug_string when literal is null ( #10567 )
2022-07-04 08:57:55 +08:00
c9f86bc7e2
[refactor] Refactoring Status static methods to format message using fmt( #9533 )
2022-07-02 18:58:23 +08:00
ca94867b4e
[Feature-wip] add date v2 type ( #9916 )
2022-06-26 16:07:56 +08:00
ed1e130ef6
[BUGFIX] fix wrong children quantity in debug string ( #10348 )
2022-06-23 09:10:30 +08:00
200557052a
[BUGFIX] wrong answer with with as + two phase agg ( #10303 )
2022-06-22 14:39:39 +08:00
84f57398d9
[Improvement] set debug string for VExpressions ( #10166 )
2022-06-21 07:43:25 +08:00
588634ddf6
[feature] support runtime filter on vectorized engine ( #10103 )
2022-06-20 09:46:38 +08:00
4c24586865
[Vectorized][UDF] support java-udaf ( #9930 )
2022-06-15 10:53:44 +08:00
f2aa5f32b8
[Feature] [Vectorized] Some pre-refactorings or interface additions for schema change ( #9811 )
...
Some pre-refactorings or interface additions for schema change
2022-06-07 15:04:57 +08:00
ac08c7ac91
[fix](vectorized) fix vcast expr input wrong row number ( #9520 )
2022-06-01 15:19:31 +08:00
35f99faa0a
[Bug][Vectorized] fix core dump on vcase_expr::close ( #9893 )
...
Co-authored-by: lihaopeng <lihaopeng@baidu.com >
2022-06-01 08:05:09 +08:00
fa50b63cee
fix core dump on vcase_expr::close ( #9875 )
2022-05-31 15:45:39 +08:00
f377c26bf7
[refactor][be] Optimize headers ( #9708 )
2022-05-30 16:12:10 +08:00
63aab5ee5d
[Bugfix(Vec)] Fix some memory leak issues ( #9824 )
2022-05-29 23:04:11 +08:00
f4dd3bf013
[bugfix] fix memleak in olapscannode( #9736 )
2022-05-26 15:06:54 +08:00
718a51a388
[refactor][style] Use clang-format to sort includes ( #9483 )
2022-05-10 21:25:35 +08:00
c9961c9bb9
[style] clang-format all c++ code ( #9305 )
...
- sh build-support/clang-format.sh to clang-format all c++ code
2022-04-29 16:14:22 +08:00
d330bc3806
[Vectorized](stream-load-vec) Support stream load in vectorized engine ( #8709 ) ( #9280 )
...
Implement vectorized stream load.
Added fe configuration option `enable_vectorized_load` to enable vectorized stream load.
Co-authored-by: tengjp@outlook.com
Co-authored-by: mrhhsg@gmail.com
Co-authored-by: minghong.zhou@163.com
Co-authored-by: HappenLee <happenlee@hotmail.com >
Co-authored-by: zhoubintao <35688959+zbtzbtzbt@users.noreply.github.com >
2022-04-29 09:50:51 +08:00
2d83167e50
[Feature] [Lateral-View] support outer combinator of table function ( #9147 )
2022-04-24 12:09:40 +08:00
ae680b4248
[UDF] support RPC udaf part 1: support create RPC udaf in fe ( #8510 )
2022-04-21 17:38:58 +08:00
c5718928df
[feature-wip](array-type) support explode and explode_outer table function ( #8766 )
...
explode(ArrayColumn) desc:
> Create a row for each element in the array column.
explode_outer(ArrayColumn) desc:
> Create a row for each element in the array column. Unlike explode, if the array is null or empty, it returns null.
Usage example:
1. create a table with array column, and insert some data;
2. open enable_lateral_view and enable_vectorized_engine;
```
set enable_lateral_view = true;
set enable_vectorized_engine=true;
```
3. use explode_outer
```
> select * from array_test;
+------+------+--------+
| k1 | k2 | k3 |
+------+------+--------+
| 3 | NULL | NULL |
| 1 | 2 | [1, 2] |
| 2 | 3 | NULL |
| 4 | NULL | [] |
+------+------+--------+
> select k1,explode_column from array_test LATERAL VIEW explode_outer(k3) TempExplodeView as explode_column;
+------+----------------+
| k1 | explode_column |
+------+----------------+
| 1 | 1 |
| 1 | 2 |
| 2 | NULL |
| 4 | NULL |
| 3 | NULL |
+------+----------------+
```
4. explode usage example. explode return empty rows while the ARRAY is null or empty
```
> select k1,explode_column from array_test LATERAL VIEW explode(k3) TempExplodeView as explode_column;
+------+----------------+
| k1 | explode_column |
+------+----------------+
| 1 | 1 |
| 1 | 2 |
+------+----------------+
```
2022-04-08 12:11:04 +08:00
dbbc6549bd
[feature](vectorized) support vexplode_bitmap ( #8890 )
2022-04-08 09:20:26 +08:00
519305cb22
[feature-wip] (memory tracker) (step4) Switch TLS mem tracker to separate more detailed memory usage ( #8669 )
...
Based on #8605 , Separate out the memory usage of each operator from the Query/Load/StorageEngine mem tracker.
2022-04-08 09:02:26 +08:00
6b0a642390
[feature][vectorized] Support explode json array func #8526 ( #8539 )
2022-04-03 10:06:47 +08:00
b89e4c7bba
[feature-wip](java-udf) support java UDF with fixed-length input and output ( #8516 )
...
This feature is propsoed in [DSIP-1](https://cwiki.apache.org/confluence/display/DORIS/DSIP-001%3A+Java+UDF ).
This PR support fixed-length input and output Java UDF. Phase I in DIP-1 is done after this PR.
To support Java UDF effeciently, I use no data copy in JNI call and all compute operations are off-heap in Java.
To achieve that, I use a UdfExecutor instead.
For users, a UDF class must have a public evaluate method.
2022-03-23 10:32:50 +08:00
