eebedbc879
[optimize](cooldown)Reduce unnecessary sort operations for vector ( #27147 )
...
Co-authored-by: xingying01 <xingying01@corp.netease.com >
2023-12-15 00:13:56 +08:00
415c6d854d
[fix](partial update) Fix some bugs about partial update ( #28358 )
2023-12-15 00:04:29 +08:00
8ca7bd8f98
[enhancement](bitmap)support bitmap type for non-key column in duplicate table ( #28392 )
2023-12-14 23:59:12 +08:00
310d1ab9a9
[feature](pipelineX)add parent-child relationship between the sink and downstream operators in profile ( #28406 )
2023-12-14 23:54:54 +08:00
4300fdce43
[feature](merge-on-write) enable merge-on-write by default again ( #28105 )
...
fix #27188 , #28096
2023-12-14 19:35:52 +08:00
a540b7eb43
[fix](Nereids) multi_distinct_sum should inherit NullableAggregateFunction ( #28349 )
2023-12-14 19:27:06 +08:00
82a91380e6
[enhancement](compaction) Add support for limiting low priority compaction scheduling ( #27648 )
2023-12-14 18:31:23 +08:00
429a3ed4df
[fix](Nereids) simplify range produce true when reference is nullable ( #28386 )
...
if reference is nullable, even if range is all, we should not return
true, but should return reference is not null. for example,
before simplify: c1 > 5 or c1 < 10
after simplify:
c1 is nullable: c1 IS NOT NULL
c1 is not nullable: TRUE
2023-12-14 18:10:17 +08:00
23941ef305
[improve](pipelineX) improve partition node dependency logical ( #28399 )
2023-12-14 18:05:39 +08:00
01c94a554d
[fix](autoinc) Fix broker load when target table has autoinc column ( #28402 )
2023-12-14 18:02:54 +08:00
c08ab9edc7
[feature](HiveCatalog) Support for getting hive meta data from relational databases under HMS ( #28188 )
2023-12-14 17:50:17 +08:00
38e79e32fa
[fix](mtmv)fix start time can not be earlier than the current time ( #28379 )
2023-12-14 17:28:04 +08:00
9fe2fce306
[minor](refactor) remove unused code ( #28383 )
2023-12-14 17:16:41 +08:00
6c9c1ef2f2
[test](regression) add flink sync db case ( #19328 )
...
add flink sync db case
Mock cdc sync data write
Contains multi-table import, add and drop columns
2023-12-14 17:06:17 +08:00
e1f7b6cff4
[typo](doc) update spark connector doc for version 1.3.0 ( #28239 )
...
Co-authored-by: gnehil <gnehil489@github>
2023-12-14 16:48:10 +08:00
634292ec1d
[Enhancement](build support) make run_clang_format compatible with python3 ( #28404 )
...
run_clang_format.py in python3 env will produce error like "ModuleNotFoundError: No module named 'distutils.util' ", try to fix this.
2023-12-14 16:02:47 +08:00
9e62e3e868
[doc](readme)Update README.md ( #28391 )
2023-12-14 14:50:43 +08:00
ee24667b9f
[fix](group commit) Fix some group commit problems ( #28319 )
2023-12-14 14:38:56 +08:00
e53cfa09da
[fix](join) incorrect result of right anti join with nullable ( #28301 )
2023-12-14 14:07:12 +08:00
bb23078d43
[feture](Nereids) support delete from without using ( #28083 )
...
support sql: DELETE FROM <table_name> WHERE <predicates>
2023-12-14 13:54:33 +08:00
e5a57f82ec
[fix](Variant Type) Fixes the desc failure ( #28343 )
...
fix the desc failure when there is no decomposition of columns in the variant column.
2023-12-14 13:20:43 +08:00
79f03d6fde
[docs](releasenote)2.0.3 release note ( #28373 )
2023-12-14 13:07:24 +08:00
c00dca70e6
[pipelineX](local shuffle) Support parallel execution despite of tablet number ( #28266 )
2023-12-14 12:53:54 +08:00
b443db6e3e
[fix](doriswriter)Fix multiple concurrent import label duplication issues #28344
2023-12-14 12:39:22 +08:00
48937fef48
[Performance](json reader) optimize filling default values ( #25542 )
...
Add a faster path for filling default values, since looking up value map is relatively slow
2023-12-14 10:20:29 +08:00
97df5981a1
[Fix](inverted index) update clucene ( #28356 )
2023-12-14 10:01:23 +08:00
1901f0f35b
[fix](function) bad performance caused by creating Status::RuntimeError ( #28362 )
2023-12-14 09:29:22 +08:00
e5be751697
[bug](exchange) Fix core dump if prepare failed ( #28363 )
2023-12-13 22:59:12 +08:00
33e24436d6
[fix](regression) mow cases should use assertEquals not assertTrue ( #28338 )
2023-12-13 22:14:32 +08:00
a271fee3c5
[test](statistics)Add external empty table test case. ( #28267 )
2023-12-13 21:48:01 +08:00
e4f7b25276
[optimize](invert index) inverted indexes optimize the file cache ( #27887 )
...
Arrange idx files in the order of segment, fnm, tii, tis to ensure that data opened during the open index phase is added to the idx file cache.
2023-12-13 20:48:53 +08:00
099b1b7106
[fix](statistics)Fix column stats trigger info bug ( #28303 )
...
Before, we didn't update the jobType info in ColStatsMeta. This will case the jobType always be the type
when it first be set. For example, if we manually analyzed table, the jobType will always be MANUAL,
even if this table is auto analyzed again later.
2023-12-13 20:31:03 +08:00
0868526f6a
[fix](sequence column) fix update fail on nereids planner ( #28031 )
...
1. if we set enable_fallback_to_original_planner to false, the UPDATE sql in regression case test_unique_table_sequence will fail due to：“Table test_uniq_sequence has sequence column, need to specify the sequence column”，The bug is introduced by [fix](sequence column) insert into should require sequence column in all scenario #27780
2. fix insert fail in transaction mode, which is introduced by [refactor](Nereids) let insert into compatible with legacy planner #27947
3. add cases with session variable enable_fallback_to_original_planner false
2023-12-13 19:41:41 +08:00
dbc2b081d7
[fix](Nereids) could not do partition prunning when predicates is NULL ( #28294 )
2023-12-13 19:36:42 +08:00
ec91dd1129
[opt](vfilescanner) interrupt running parquet/orc readers when scannode is finished ( #28223 )
...
VScanNode::get_next will check whether the ScanNode has reached limit condition, and send eos to TaskScheduler, and TaskScheduler will try to close ScanNode.
However, ScanNode must wait all running scanners finished, so even if ScanNode has reached limit condition, it can't be closed immediately.
This PR try to interrupt the running readers, and make ScanNode to end as soon as possible.
2023-12-13 19:31:08 +08:00
524bce3c07
[doc](auditlog) update description of field sql_digest ( #28340 )
2023-12-13 19:27:48 +08:00
4f5821407f
[case]Load data with load_parallelism=any > 1 and stream load with compress type ( #27306 )
2023-12-13 18:41:14 +08:00
43327383c3
[regression test](broker laod) add exception case for merge type ( #27840 )
2023-12-13 18:34:34 +08:00
9b94faf403
fix](memory_leak) fix memory leak on vrow_distribution #28336
2023-12-13 18:22:14 +08:00
ea832744df
[fix](journal) ensure txns are matched with the master before replaying ( #28192 )
2023-12-13 18:14:51 +08:00
62859f38c1
[fix](nereids)cast string to integer type use wrong datatype's valueOf method ( #28174 )
...
select cast('12.31' as tinyint);
select cast('12.31' as smallint);
select cast('12.31' as int);
should return NULL
2023-12-13 17:53:07 +08:00
ac262fa176
[fix](partial update) Fix session variable enable_unique_key_partial_update will affect the behavior of insert statement when the target table is not unique table ( #28321 )
2023-12-13 17:14:30 +08:00
e6e8632167
[improvement](merge-on-write) Optimize publish when there are missing versions ( #28012 )
...
1. Do not retry publishing on be When there are too many missing versions, just
add to async publish task.
2. To reduce memory consumption, clean up the tasks when there are too many
async publish tasks.
2023-12-13 16:59:25 +08:00
dee89d2c4a
[refactor](Nereids) let create table compatible with legacy planner ( #28078 )
2023-12-13 16:35:40 +08:00
8ac52aeda7
[fix](nereids) wrong result when simplify int compare with decimal literal ( #28195 )
...
SimplifyComparisonPredicate rule create wrong result when simplify int compare with decimal literal
2023-12-13 16:13:42 +08:00
3e08bec27f
[opt](regression test) use select count from tablet api ( #28327 )
2023-12-13 16:12:56 +08:00
2005fee430
[fix](user) Avoid throw unknown UserProperty after FE rollback version ( #28325 )
...
After using SET PROPERTY to modify the user property, if FE rolls back to a version without this property, `Unknown user property` error will be reported when replay EditLog, just ignore it.
2023-12-13 16:00:04 +08:00
5f66335e54
[feature](pipelineX) add local_shuffle in set_operation / assert_num operator ( #28293 )
2023-12-13 15:15:20 +08:00
cd6d75e518
[fix](memory) TabletSchema and Schema no longer track memory, only track columns count. ( #28149 )
...
TabletSchema and Schema no longer track memory, only track columns count. because cannot accurately track memory size.
TabletMeta MemTracker changed to track TabletSchema columns count.
Segment::_meta_mem_usage Unknown value overflow, causes the value of SegmentMeta MemTracker is similar to -2912341218700198079. So, temporarily put it in experimental type tracker.
2023-12-13 15:06:46 +08:00
613b0bd552
[fix](decimal) fix decimal overflow caused by null value ( #28260 )
2023-12-13 14:53:39 +08:00
