bf808e9aa6
[fix](Nereids): tolerate DateLike overflow in SQL CAST/CONVERT ( #24943 )
...
- explicit type cast, we need tolerate overflow and convert it to be NULL
- implicit type cast, throw exception
2023-09-28 12:11:50 +08:00
f0fad61db4
[pipelineX](bug) Fix file scan operator ( #24989 )
2023-09-28 11:12:27 +08:00
188d9ab94e
[enhancement](statistics) collect table level loaded rows on BE to make RPC light weight ( #24609 )
2023-09-28 10:51:50 +08:00
42207df89f
[refactor](nereids)update NormalizeRepeat comments ( #24893 )
...
update NormalizeRepeat comments
2023-09-28 10:42:16 +08:00
4ff1ab7a4d
[fix](regression-test) regenerate test_http_stream_properties.out file ( #24946 )
2023-09-28 10:39:15 +08:00
21d6f41492
[fix](regresion-test) Fix the problem of occasional failure of test_outfile_exception regression-test case ( #24937 )
2023-09-28 10:05:43 +08:00
584646c054
[improvement](nereids)dphyper GraphSimplifier should consider missed edges when estimating join cost ( #21747 )
2023-09-28 09:30:57 +08:00
430634367a
[pipelineX](node)support file scan operator ( #24924 )
2023-09-27 22:10:43 +08:00
68087f6c82
[fix](json function) Fix the slow performance of get_json_path when processing JSONB ( #24631 )
...
When processing JSONB, automatically convert to jsonb_extract_string
2023-09-27 21:17:39 +08:00
732f821c15
[Fix](inverted index) make parser mode coarse grained by default ( #24949 )
2023-09-27 21:04:41 +08:00
d4e823950a
[bug](json)Fix some problems of json function on Nereids ( #24898 )
...
Fix some problems of json_length and json_contains function on Nereids
fix wrong result of json_contains function
Regression test jsonb_p0 to enable Nereids
2023-09-27 21:01:45 +08:00
391a4e29eb
[fix](schema) Table column order is changed if add a column and do truncate ( #24981 )
2023-09-27 20:59:11 +08:00
63b283a848
[fix](Nereids) init Date/DateV2Literal should check non-zero time fields ( #24971 )
2023-09-27 20:48:36 +08:00
947b116318
[pipelineX](fix) Fix BE crash due to ES scan operator ( #24983 )
2023-09-27 20:45:38 +08:00
1fb9022d07
[pipelineX](bug) Fix meta scan operator ( #24963 )
2023-09-27 20:34:47 +08:00
671b5f0a0a
[Bug](pipeline) Fix block reusing for union source operator ( #24977 )
...
[CANCELLED][INTERNAL_ERROR]Merge block not match, self:[String], input:[String, Nullable(String), Nullable(String), Nullable(String), Nullable(String), DateV2]
2023-09-27 19:41:56 +08:00
bb7f8d18a8
[fix](nereids) push down filter through partition topn ( #24944 )
...
support pushing down filter through partition topn if the filter can pass through window.
fix CreatePartitionTopNFromWindow bug which may generate two partition topn unexpectly.
case:
select * from (select c2, row_number() over (partition by c2) as rn from t1) T where rn<=1 and c2 = 1;
before this pr:
| PhysicalResultSink |
| --PhysicalDistribute |
| ----filter((rn <= 1)) |
| ------PhysicalWindow |
| --------PhysicalQuickSort |
| ----------PhysicalDistribute |
| ------------PhysicalPartitionTopN |
| --------------filter((T.c2 = 1)) |
| ----------------PhysicalPartitionTopN |
| ------------------PhysicalProject |
| --------------------PhysicalOlapScan[t1] |
+------------------------------------------+
after:
| PhysicalResultSink |
| --PhysicalDistribute |
| ----filter((rn <= 1)) |
| ------PhysicalWindow |
| --------PhysicalQuickSort |
| ----------PhysicalDistribute |
| ------------PhysicalPartitionTopN |
| --------------PhysicalProject |
| ----------------filter((T.c2 = 1)) |
| ------------------PhysicalOlapScan[t1] |
+----------------------------------------+
2023-09-27 19:38:04 +08:00
00786a3295
[fix](Nereids) could not prune datev1 partition column ( #24959 )
...
because storage engine could not process date comparison predicates.
we convert it to datetime comparison predicates.
however, partition prunner could not process cast(slot) cp literal.
so, we convert back in partition pruner to let it work well.
TODO:
move convert date to datetime in translate stage
and only convert predicates for storage engine.
2023-09-27 18:41:56 +08:00
a6bc0e7668
Revert "[chore](clang-tidy) Apply uninitialized variables check of clang-tidy ( #23497 )" ( #24976 )
2023-09-27 18:28:24 +08:00
5d138b6928
[remove](function) make execute_impl const and remove running_difference function ( #24935 )
2023-09-27 18:17:28 +08:00
100d76510c
[Fix](HttpServer) Refactor API Endpoints to Only Allow GET Requests for Enhanced Security ( #24855 )
2023-09-27 17:10:11 +08:00
00e8d1c3b4
[Fix](Planner) disable bitmap type in compare expression ( #24792 )
...
Problem:
be core because of bitmap calculation.
Reason:
when be check failed, it would core directly.
Example:
SELECT id_bitmap FROM test_bitmap WHERE id_bitmap IN (NULL) LIMIT 20;
Solved:
Forbidden this kind of expression in fe when analyze. And also forbid bitmap type comparing in other unsupported expressions.
2023-09-27 16:57:06 +08:00
0227292c85
[bug](profile) query profile api of fe cann't get result if non-root user query on the other fe #24858 ( #24914 )
...
Issue Number: #24858
If isAllNode is true, the api should only distribute the query to all fe and do not run checkAuthByUserAndQueryId.
If isAllNode is false, the api queries profile on the fe, at this time the api should run checkAuthByUserAndQueryId.
2023-09-27 16:50:41 +08:00
c04078f3b8
[improvement](compaction) output tablet_id when be core dumped. ( #24952 )
2023-09-27 16:50:18 +08:00
19cff5d167
[fix](compile) failed on arm platform, with clang compiler and pch on ( #24636 )
...
failed on arm platform, with clang compiler and pch on
2023-09-27 16:47:02 +08:00
9562e280af
[enhancement](Nereids): remove stats derivation in CostAndEnforce job ( #24945 )
...
1. remove stats derivation in CostAndEnforce job
2. enforce valid for each stats after estimating
2023-09-27 16:31:03 +08:00
5fc04b6aeb
[Improvement](hash) some refactor of process hash table probe impl ( #24461 )
...
some refactor of process hash table probe impl
2023-09-27 16:14:49 +08:00
83f5ff7b22
[typo](doc)modify error result of explode_split function. ( #24185 )
2023-09-27 03:04:18 -05:00
41dfbdac14
[typo](doc)modify the wrong parameter in the example. ( #24307 )
2023-09-27 03:04:07 -05:00
53025ce3fc
[typo](doc)modify error description of GROUP_CONCAT fiction. ( #24619 )
2023-09-27 03:03:56 -05:00
a6d1da0db9
[typo](doc)modify error link description in fe-config-template ( #24676 )
2023-09-27 02:51:47 -05:00
aa4dbbedc7
[pipelineX](bug) Fix dead lock in exchange sink operator ( #24947 )
2023-09-27 15:40:25 +08:00
87a30dc41d
[feature-wip](arrow-flight)(step3) Support authentication and user session ( #24772 )
2023-09-27 14:53:58 +08:00
26818de9c8
[feature](jni) support complex types in jni framework ( #24810 )
...
Support complex types in jni framework, and successfully run end-to-end on hudi.
### How to Use
Other scanners only need to implement three interfaces in `ColumnValue`:
```
// Get array elements and append into values
void unpackArray(List<ColumnValue> values);
// Get map key array&value array, and append into keys&values
void unpackMap(List<ColumnValue> keys, List<ColumnValue> values);
// Get the struct fields specified by `structFieldIndex`, and append into values
void unpackStruct(List<Integer> structFieldIndex, List<ColumnValue> values);
```
Developers can take `HudiColumnValue` as an example.
2023-09-27 14:47:41 +08:00
a1ab8f96a1
[fix](nereids) mark two phase partition topn global to notice be passthrough logic ( #24886 )
...
mark partition topn phase to notice be to handle passthrough logic well, this pr is fe part code.
be side logic: the the phase equals to PTopNPhase.TWO_PAHSE_GLOBAL, it should skip the bypass logic and do the second phase ptopn operation anyway.
2023-09-27 14:08:59 +08:00
1b0e3246ea
[pipelineX](fix) Fix exception reporting and Nereids plan ( #24936 )
2023-09-27 13:15:40 +08:00
c04e5bac39
[bug](pipelineX) fix java-udaf failed with open pipelineX ( #24939 )
2023-09-27 13:14:10 +08:00
452318a9fc
[Enhancement](streamload) stream tvf support user specified label ( #24219 )
...
stream tvf support user specified label
example:
curl -v --location-trusted -u root: -H "sql: insert into test.t1 WITH LABEL label1 select c1,c2 from http_stream(\"format\" = \"CSV\", \"column_separator\" = \",\")" -T example.csv http://127.0.0.1:8030/api/_http_stream
return:
{
"TxnId": 2064,
"Label": "label1",
"Comment": "",
"TwoPhaseCommit": "false",
"Status": "Success",
"Message": "OK",
"NumberTotalRows": 2,
"NumberLoadedRows": 2,
"NumberFilteredRows": 0,
"NumberUnselectedRows": 0,
"LoadBytes": 27,
"LoadTimeMs": 152,
"BeginTxnTimeMs": 0,
"StreamLoadPutTimeMs": 83,
"ReadDataTimeMs": 92,
"WriteDataTimeMs": 41,
"CommitAndPublishTimeMs": 24
}
2023-09-27 12:09:35 +08:00
18b5f70a7c
[Bug](materialized-view) enable rewrite on select materialized index with aggregate mode ( #24691 )
...
enable rewrite on select materialized index with aggregate mode
2023-09-27 11:30:36 +08:00
6b64b7fec7
[typo](doc)Add flink to read the doris table and use doris.filter.query to configure the display ( #24736 )
2023-09-27 11:22:58 +08:00
a8f312794e
[feature](nereids)support stats estimation for is-null predicate ( #24764 )
...
1. condition order: filter/hashCondition/otherCondition,
2. update regression out
3. remove tpch_sf500 shape case(covered by tpch sf1000)
4. implement is-null stats estimation
5. update ssb shape
2023-09-27 10:04:35 +08:00
b9928a017b
[fix](fe)add variant type in decimalv3's compatibilityMatrix table ( #24749 )
2023-09-27 10:03:13 +08:00
0dd57b1982
[feature](nereids)support subquey in join condition ( #24598 )
2023-09-27 10:02:56 +08:00
6d27a016b9
[Improvement](regression-test) add http_stream case ( #24930 )
2023-09-27 09:55:52 +08:00
2e8f9a6429
[improvement](streamload) QeProcessorImpl#registerQuery should be removed ( #24836 )
2023-09-27 09:48:27 +08:00
f7f359d36c
[fix](analysis)fix use regex determine whether time part exists may cause backtracking ( #24882 )
2023-09-27 09:26:42 +08:00
24ee3607e1
[Bug](pipeline) nullprt may be close the sink if init failed ( #24926 )
2023-09-27 09:11:06 +08:00
f59e69b852
[feature](autobucekt) Add support autobucket min buckets config Add support autobucket ( #24920 )
...
Signed-off-by: Jack Drogon <jack.xsuperman@gmail.com >
2023-09-27 08:13:53 +08:00
504ab2d726
[fix](es-catalog) If the returned data is incorrect, it will be directly unified as the index was not found ( #24831 )
2023-09-26 22:59:38 +08:00
841e6188bf
[fix](Nereids): fix roundFloor ( #24901 )
...
"roundFloor()" does not consider the issue of carry.
2023-09-26 22:37:42 +08:00
