d809bee46e
[refactor](profilev2) add BlocksProduced RowsProduced counter #27291
2023-11-21 12:01:11 +08:00
3e8177bbbd
Revert "[improvement](routine-load) add routine load rows check ( #25818 )" ( #27304 )
...
This reverts commit 5e3277e8fbc92f08344bc86e9b2c74033a8bbc01.
2023-11-21 11:28:05 +08:00
2548e27c97
[fix](memory) Fix work load group meaningless GC #27307
2023-11-21 09:59:21 +08:00
87a7de7660
[pipelineX](bug) Fix 'No broadcast buffer left' ( #27298 )
2023-11-20 22:37:21 +08:00
c0f22e8feb
[FIX](complextype)fix struct nested complex collection type and and regresstest ( #26973 )
2023-11-20 22:29:12 +08:00
840f3b6439
[pipelineX](dependency) Wake by task by read dependency ( #27260 )
2023-11-20 22:00:59 +08:00
20d7ab061b
[PipelineX](complex type pushdown) do not push down complex type to storage layer based on PR#25007 ( #27278 )
2023-11-20 18:47:59 +08:00
273cbfc36c
[fix](inverted index) reset fs_writer to nullptr before throw exception ( #27202 )
2023-11-20 17:40:56 +08:00
6ed0be8e3c
[refactor](profilev2) unify the counter name in shuffle operator and normal operator ( #27267 )
...
using blocksproduced and rowsproduced to unify the counter name in DataStreamSender and other exec node, or exchange operator and other operators.
blocks produced and rows produced are more easy to understand.
---------
Co-authored-by: yiguolei <yiguolei@gmail.com >
2023-11-20 14:21:39 +08:00
459f75073f
[pipelineX](dependency) remove OrDependency ( #27242 )
2023-11-20 13:05:34 +08:00
58aefbc33f
[feature](clucene) coverage compilation option added ( #27162 )
2023-11-19 05:33:08 -06:00
c7cefe27d5
[pipelineX](broadcast) Add error logs ( #27243 )
2023-11-19 17:02:00 +08:00
70e070182f
[feature](executor)Make workload group property not required ( #27229 )
...
* Make workload group property not required
* remove useless UT
2023-11-19 17:01:51 +08:00
febd60c75f
[fix](join) incorrect result of left join with other conjuncts ( #27238 )
2023-11-19 15:36:39 +08:00
836cda65d8
[refactor](profilev2) split merged profile to a single runtime profile to make the logic more clear ( #27184 )
2023-11-19 13:21:50 +08:00
7b817197b2
[pipelineX](sink) simplify result sink dependency ( #27226 )
2023-11-19 12:42:10 +08:00
b42828cf69
[fix](window_function) min/max/sum/avg should be always nullable ( #27104 )
...
Co-authored-by: starocean999 <40539150+starocean999@users.noreply.github.com >
2023-11-18 18:41:42 +08:00
2f41e0c823
[FIX](complextype)fix information schema for complex type ( #27203 )
...
when we select in information schema , here do not show complex type information
2023-11-18 11:32:32 +08:00
be7273da83
[refactor](executor)Refactor workload meta update to be #26710
2023-11-18 11:19:38 +08:00
b1eef30b49
[pipelineX](dependency) Wake up task by dependencies ( #26879 )
...
---------
Co-authored-by: Mryange <2319153948@qq.com >
2023-11-18 03:20:24 +08:00
0aec436ef8
[chore](be) format reader parameter settings ( #22964 )
2023-11-18 00:11:46 +08:00
5fb27eb652
[fix](compile) fix BE compile failure on Mac ( #27206 )
2023-11-17 23:52:51 +08:00
5d548935e0
[improvement](insert) support schema change and decommission for group commit ( #26359 )
2023-11-17 21:41:38 +08:00
0a1a6cf02f
[fix](topn) add defensive code in topn opt to avoid crash due to column not in tablet schema
2023-11-17 21:14:10 +08:00
c459408580
[fix](jni) avoid BE crash and NPE when close paimon reader ( #27129 )
...
1. Do not use FATAL log when jni encounter error, to avoid crash.
2. Fix NPE when closing PaimonReader, the reader may not be assigned if PaimonReader open failed.
2023-11-17 20:01:08 +08:00
52995c528e
[fix](iceberg) iceberg use customer method to encode special characters of field name ( #27108 )
...
Fix two bugs:
1. Missing column is case sensitive, change the column name to lower case in FE for hive/iceberg/hudi
2. Iceberg use custom method to encode special characters in column name. Decode the column name to match the right column in parquet reader.
2023-11-17 18:38:55 +08:00
fdec286e82
[optimize](cooldown)Shorten the _meta_lock lock interval ( #27118 )
...
Change the two passes of _rs_version_map to one, reducing cpu overhead and shortening the lock interval of _meta_lock
Co-authored-by: xingying01@corp.netease.com <xingying01@corp.netease.com >
2023-11-17 16:59:36 +08:00
593e3662b0
[Fix](match) fix match null for no index ( #26983 )
...
This pull request addresses an issue observed with inverted index tables or tables without indices when querying null values using the MATCH function.
Previously, executing a query like `SELECT * FROM table WHERE column MATCH null;` would yield incorrect results.
The update introduces enhanced handling of nullable columns within the MATCH function, ensuring accurate query results when null values are involved.
2023-11-17 15:57:50 +08:00
4d2fb1fffb
[fix](load) add lock in active_memtable_mem_consumption ( #27101 )
2023-11-17 15:03:15 +08:00
e1b180d53d
[improve](streamload) Explicitly judge the return value of close #27134
2023-11-17 14:17:09 +08:00
a0661ed9d2
[Fix](multi-catalog) Fix complex type crash when using dict filter facility in the parquet-reader. ( #27151 )
...
- Fix complex type crash when using the dict filter facility in the parquet-reader by turning off the dict filter facility in this case.
- Add orc complex types regression test.
2023-11-17 13:43:58 +08:00
4fff9a5937
[Improvement](inverted index) delay inverted index col read to reduce IO ( #26080 ) ( #26337 )
2023-11-17 13:12:12 +08:00
91af86bc78
[fix](function) fix error when use negative number in explode_numbers #27020
2023-11-17 12:02:14 +08:00
1188d88a10
[Chore](status) catch some error status on storage ( #27132 )
...
catch some error status on storage
2023-11-17 12:00:39 +08:00
334260dff7
[feature](function) support ip function ipv4stringtonum(ordefault, ornull), inet_aton ( #25510 )
2023-11-17 10:27:07 +08:00
a4d78682ff
[Optimize](point query) clear names to reduce mem consumption and cpu cost related to block column name ( #26931 )
2023-11-17 10:18:21 +08:00
0c264c8a14
[fix](pipelineX) fix scheduling bug in union operator ( #27131 )
2023-11-17 10:02:54 +08:00
afffcfd14c
[fix](load) skip cancel already cancelled channels ( #27111 )
2023-11-16 18:38:40 +08:00
e29d8cb110
[feature](move-memtable) support pipelineX in sink v2 ( #27067 )
2023-11-16 15:00:55 +08:00
54989175fb
[case] Load json data with enable_simdjson_reader=false ( #26601 )
2023-11-16 14:40:59 +08:00
f10ab4e113
[enhancement](JNI) Provide default environment variables if it is unset ( #27037 )
2023-11-16 14:37:11 +08:00
7e82e7651a
[Improve](txn) Add some fuzzy test stub in txn ( #26712 )
2023-11-16 11:50:06 +08:00
7fbc6d26a7
[debug](log) add some log to debug issue about insert ( #27045 )
2023-11-16 11:46:47 +08:00
042f6e8458
[cleanup](move-memtable) cleanup unused fields in rowset writer v2 ( #27073 )
2023-11-16 10:13:00 +08:00
b8b86a7262
[enhance](cooldown) Reduce the locking interval for cooldown task ( #26984 )
...
Co-authored-by: xingying01 <xingying01@corp.netease.com >
2023-11-16 10:02:32 +08:00
0eabe9a651
[Test](orc-reader) Add orc submodule's unit tests. ( #26878 )
2023-11-16 09:53:42 +08:00
7ef1f7e511
[Bug](pipeline) try fix the exchange sink buffer result error ( #27052 )
2023-11-16 09:20:56 +08:00
02f3762ab3
[refactor](status) define error code and enable stacktrace in same place ( #27065 )
2023-11-16 08:41:40 +08:00
3ad865fef9
[refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits ( #26191 )
2023-11-15 21:34:49 +08:00
10ee48bb6f
[fix](publish version) publish version task no need return VERSION_NOT_EXIST #27005
...
if BE's tablet not contains a txn, publish txn on them will no error, when check version exists it will indicate the tablet as error_tablet_id in task's response, so FE can know this tablet has fail.
Also for task, it's no need to set its status as "VERSION_NOT_EXIST". Because if set it as not ok, the BE will try this task two times. Since not contains this tablet's txn, the retry is in vain.
2023-11-15 21:09:54 +08:00
