c2b46e4df7
[fix](move-memtable) exclude rpc memory in flush mem-tracker ( #24722 )
2023-10-05 22:10:53 +08:00
db6c16058a
[improve](move-memtable) always share load streams ( #24763 )
2023-10-05 22:09:59 +08:00
93eedaff62
[opt](function) Use Dict to opt the function of time_round ( #25029 )
...
Before：
select hour_floor(`@timestamp`, 7) as t, count() as cnt from httplogs_date group by t order by t limit 10;
+---------------------+--------+
| t | cnt |
+---------------------+--------+
| 1998-04-30 21:00:00 | 324 |
| 1998-05-01 04:00:00 | 286156 |
| 1998-05-01 11:00:00 | 266130 |
| 1998-05-01 18:00:00 | 483765 |
| 1998-05-02 01:00:00 | 276706 |
| 1998-05-02 08:00:00 | 169945 |
| 1998-05-02 15:00:00 | 223593 |
| 1998-05-02 22:00:00 | 272616 |
| 1998-05-03 05:00:00 | 188689 |
| 1998-05-03 12:00:00 | 184405 |
+---------------------+--------+
10 rows in set (3.39 sec)
after:
select hour_floor(`@timestamp`, 7) as t, count() as cnt from httplogs_date group by t order by t limit 10;
+---------------------+--------+
| t | cnt |
+---------------------+--------+
| 1998-04-30 21:00:00 | 324 |
| 1998-05-01 04:00:00 | 286156 |
| 1998-05-01 11:00:00 | 266130 |
| 1998-05-01 18:00:00 | 483765 |
| 1998-05-02 01:00:00 | 276706 |
| 1998-05-02 08:00:00 | 169945 |
| 1998-05-02 15:00:00 | 223593 |
| 1998-05-02 22:00:00 | 272616 |
| 1998-05-03 05:00:00 | 188689 |
| 1998-05-03 12:00:00 | 184405 |
+---------------------+--------+
10 rows in set (2.19 sec)
2023-10-04 23:34:24 +08:00
4ce5213b1c
[fix](insert) Fix test_group_commit_stream_load and add more regression in test_group_commit_http_stream ( #24954 )
2023-10-03 20:56:24 +08:00
6e836fe381
[fix](jdbc catalog) fix jdbc catalog read bitmap data crash ( #25034 )
2023-10-03 20:52:47 +08:00
10f0c63896
[FIX](complex-type) fix agg table with complex type with replace state ( #24873 )
...
fix agg table with complex type with replace state
2023-10-03 16:32:58 +08:00
f8a3034dca
[Opt](performance) refactor and opt time round floor function ( #25026 )
...
refactor and opt time round floor function
2023-10-01 11:51:26 +08:00
642e5cdb69
[Fix](Status) Make Status [[nodiscard]] and handle returned Status correctly ( #23395 )
2023-09-29 22:38:52 +08:00
d23bedf170
[fix](single-replica-load) fix duplicated done run in request_slave_tablet_pull_rowset ( #25013 )
...
BE will crash because done run twice when try_offer() failed in
request_slave_tablet_pull_rowset.
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com >
2023-09-28 21:08:18 +08:00
864a0f9bcb
[opt](pipeline) Make pipeline fragment context send_report asynchronized ( #23142 )
2023-09-28 17:55:53 +08:00
2ec50dcfc7
[log](compaction) add more stats for compaction log ( #24984 )
2023-09-28 15:29:15 +08:00
b6babf3af4
[pipelineX](sink) support jdbc table sink ( #24970 )
...
* [pipelineX](sink) support jdbc table sink
2023-09-28 14:39:32 +08:00
b35171b582
[pipelineX](bug) fix distinct streaming agg ( #24995 )
2023-09-28 14:01:26 +08:00
f0fad61db4
[pipelineX](bug) Fix file scan operator ( #24989 )
2023-09-28 11:12:27 +08:00
188d9ab94e
[enhancement](statistics) collect table level loaded rows on BE to make RPC light weight ( #24609 )
2023-09-28 10:51:50 +08:00
430634367a
[pipelineX](node)support file scan operator ( #24924 )
2023-09-27 22:10:43 +08:00
68087f6c82
[fix](json function) Fix the slow performance of get_json_path when processing JSONB ( #24631 )
...
When processing JSONB, automatically convert to jsonb_extract_string
2023-09-27 21:17:39 +08:00
d4e823950a
[bug](json)Fix some problems of json function on Nereids ( #24898 )
...
Fix some problems of json_length and json_contains function on Nereids
fix wrong result of json_contains function
Regression test jsonb_p0 to enable Nereids
2023-09-27 21:01:45 +08:00
947b116318
[pipelineX](fix) Fix BE crash due to ES scan operator ( #24983 )
2023-09-27 20:45:38 +08:00
1fb9022d07
[pipelineX](bug) Fix meta scan operator ( #24963 )
2023-09-27 20:34:47 +08:00
671b5f0a0a
[Bug](pipeline) Fix block reusing for union source operator ( #24977 )
...
[CANCELLED][INTERNAL_ERROR]Merge block not match, self:[String], input:[String, Nullable(String), Nullable(String), Nullable(String), Nullable(String), DateV2]
2023-09-27 19:41:56 +08:00
5d138b6928
[remove](function) make execute_impl const and remove running_difference function ( #24935 )
2023-09-27 18:17:28 +08:00
c04078f3b8
[improvement](compaction) output tablet_id when be core dumped. ( #24952 )
2023-09-27 16:50:18 +08:00
19cff5d167
[fix](compile) failed on arm platform, with clang compiler and pch on ( #24636 )
...
failed on arm platform, with clang compiler and pch on
2023-09-27 16:47:02 +08:00
5fc04b6aeb
[Improvement](hash) some refactor of process hash table probe impl ( #24461 )
...
some refactor of process hash table probe impl
2023-09-27 16:14:49 +08:00
aa4dbbedc7
[pipelineX](bug) Fix dead lock in exchange sink operator ( #24947 )
2023-09-27 15:40:25 +08:00
87a30dc41d
[feature-wip](arrow-flight)(step3) Support authentication and user session ( #24772 )
2023-09-27 14:53:58 +08:00
26818de9c8
[feature](jni) support complex types in jni framework ( #24810 )
...
Support complex types in jni framework, and successfully run end-to-end on hudi.
### How to Use
Other scanners only need to implement three interfaces in `ColumnValue`:
```
// Get array elements and append into values
void unpackArray(List<ColumnValue> values);
// Get map key array&value array, and append into keys&values
void unpackMap(List<ColumnValue> keys, List<ColumnValue> values);
// Get the struct fields specified by `structFieldIndex`, and append into values
void unpackStruct(List<Integer> structFieldIndex, List<ColumnValue> values);
```
Developers can take `HudiColumnValue` as an example.
2023-09-27 14:47:41 +08:00
1b0e3246ea
[pipelineX](fix) Fix exception reporting and Nereids plan ( #24936 )
2023-09-27 13:15:40 +08:00
c04e5bac39
[bug](pipelineX) fix java-udaf failed with open pipelineX ( #24939 )
2023-09-27 13:14:10 +08:00
452318a9fc
[Enhancement](streamload) stream tvf support user specified label ( #24219 )
...
stream tvf support user specified label
example:
curl -v --location-trusted -u root: -H "sql: insert into test.t1 WITH LABEL label1 select c1,c2 from http_stream(\"format\" = \"CSV\", \"column_separator\" = \",\")" -T example.csv http://127.0.0.1:8030/api/_http_stream
return:
{
"TxnId": 2064,
"Label": "label1",
"Comment": "",
"TwoPhaseCommit": "false",
"Status": "Success",
"Message": "OK",
"NumberTotalRows": 2,
"NumberLoadedRows": 2,
"NumberFilteredRows": 0,
"NumberUnselectedRows": 0,
"LoadBytes": 27,
"LoadTimeMs": 152,
"BeginTxnTimeMs": 0,
"StreamLoadPutTimeMs": 83,
"ReadDataTimeMs": 92,
"WriteDataTimeMs": 41,
"CommitAndPublishTimeMs": 24
}
2023-09-27 12:09:35 +08:00
24ee3607e1
[Bug](pipeline) nullprt may be close the sink if init failed ( #24926 )
2023-09-27 09:11:06 +08:00
a689a2fbb1
[pipelineX](fix) Fix projection expression ( #24923 )
2023-09-26 21:48:28 +08:00
55d1090137
[feature](insert) Support group commit stream load ( #24304 )
2023-09-26 20:57:02 +08:00
fe2879d8fe
[fix](merge-on-write) MergeIndexDeleteBitmapCalculator stack overflow ( #24913 )
2023-09-26 20:32:23 +08:00
77e864df12
[enhancement](delete) use column id in delete push task instead of column name ( #24549 )
2023-09-26 19:54:55 +08:00
ded8ba108f
[test](be-ut) skip some be ut case ( #24917 )
...
skip be ut "VTimestampFunctionsTest.convert_tz_test"
2023-09-26 19:51:15 +08:00
28869b0f82
[fix](Outfile) Use data_type_serde to export data to orc file format ( #24812 )
2023-09-26 19:46:42 +08:00
94082ae59c
[Fix](inverted index) fix tokenize function coredump ( #24896 )
2023-09-26 17:31:10 +08:00
156b7b80a4
[pipelineX](bug) Fix scan operator if aggregation type is COUNT_ON_INDEX ( #24909 )
2023-09-26 16:42:29 +08:00
1abda1c446
[Fix](merge-on-write) Correct the alignment process when the existing rows with same key has marked delete sign ( #24877 )
2023-09-26 16:09:20 +08:00
082bcd820b
[feature](insert) Support wal for group commit insert ( #23053 )
2023-09-26 14:46:24 +08:00
422170ccb1
[Bugfix](tablet) change WriteCooldownMetaExecutors's thread pool type to prevent singleton's dtor order ( #23834 )
2023-09-26 14:29:35 +08:00
a3427cb822
[pipelineX](fix) Fix nested loop join operator ( #24885 )
2023-09-26 13:27:34 +08:00
733b71828c
[fix](pipelineX) fix do not set per_fragment_instance_idx ( #24890 )
2023-09-26 13:10:30 +08:00
513e37bdbf
[pipelineX](node)support jdbc scan operator ( #24851 )
2023-09-26 10:02:51 +08:00
8ee2e40812
[enhancement](zone_map_index) use direct_copy_may_cut instead of direct_copy( #24880 )
2023-09-25 22:32:18 +08:00
8191cd1dad
[Bug](ScanNode) Fix potential incorrect query result caused by concurrent NewOlapScanNode initialization and Compaction ( #24638 )
...
* Optimize fetch delete predicates
* Fix incorrect query result when compaction eliminate delete predicates between `NewOlapScanNode::_init_scanners` and `NewOlapScanner::init`
* Fix be ut
2023-09-25 22:24:35 +08:00
8d4fd76a16
[Feature](StreamLoad2PC) Support commit and abort streamload2PC by label ( #24613 )
2023-09-25 22:21:27 +08:00
b38b8b4494
[pipelineX](fix) Fix BE crash caused by join and constant expr ( #24862 )
2023-09-25 21:01:09 +08:00
