2018ab23f0
[chore](build) Add MVN_OPT env variable to enrich building FE with extra options ( #28375 )
...
e.g. just export or add it to custom_env.sh
```
export MVN_OPT="-o"
```
will build FE with maven option "-o" (offline), which means maven does
not need to download meta from maven repo, it is useful for saving time
if the internet is unstable or unusable.
2023-12-15 13:20:39 +08:00
e6b135c76a
[improvement](fe) Add reason log when Env is not ready ( #28286 )
2023-12-15 12:22:06 +08:00
6f3fb81965
[fix](doc) spell errors fixes multi-tenant.md ( #28436 )
2023-12-15 12:21:46 +08:00
4c51558f6b
[feature](nereids) Support basic aggregate rewrite and function rollup using materialized view ( #28269 )
...
Add aggregate materializedviewRules for query rewrite.
it support the query rewrite as following:
def mv = "select lineitem.L_LINENUMBER, orders.O_CUSTKEY, sum(O_TOTALPRICE) as sum_alias " +
"from lineitem " +
"inner join orders on lineitem.L_ORDERKEY = orders.O_ORDERKEY " +
"group by lineitem.L_LINENUMBER, orders.O_CUSTKEY "
def query = "select lineitem.L_LINENUMBER, sum(O_TOTALPRICE) as sum_alias " +
"from lineitem " +
"inner join orders on lineitem.L_ORDERKEY = orders.O_ORDERKEY " +
"group by lineitem.L_LINENUMBER"
2023-12-15 11:30:02 +08:00
c4242ab69e
[Chore](Job)Add the configuration of the maximum number of persistence tasks for the job ( #28411 )
2023-12-15 11:14:06 +08:00
8661b5ec21
[Bug](function) fix npe on select http_stream directly ( #28423 )
...
fix npe on select http_stream directly
2023-12-15 11:09:45 +08:00
ce60064573
[regression-test](variant) fix unstable query result m… ( #28390 )
...
* [regression-test](variant) fix unstable query result for unique key model
* fix p2 case
* add msg
2023-12-15 10:52:50 +08:00
1877389f12
[fix](Nereids) set card to olap table break card block rule ( #28417 )
...
we have card block rule to avoid scan too many data.
so we must set olap scan card by only scanned bucket.
2023-12-15 10:28:05 +08:00
4d9b6c272d
[Fix](vcompound pred) Corrected evaluation for compound predicates with constant columns ( #28421 )
2023-12-15 10:10:48 +08:00
eb99e4270d
[Fix](parquet_reader) Fix dict filtering doesn't work with plain dict encoding in parquet reader. ( #28290 )
2023-12-15 09:27:02 +08:00
eebedbc879
[optimize](cooldown)Reduce unnecessary sort operations for vector ( #27147 )
...
Co-authored-by: xingying01 <xingying01@corp.netease.com >
2023-12-15 00:13:56 +08:00
415c6d854d
[fix](partial update) Fix some bugs about partial update ( #28358 )
2023-12-15 00:04:29 +08:00
8ca7bd8f98
[enhancement](bitmap)support bitmap type for non-key column in duplicate table ( #28392 )
2023-12-14 23:59:12 +08:00
310d1ab9a9
[feature](pipelineX)add parent-child relationship between the sink and downstream operators in profile ( #28406 )
2023-12-14 23:54:54 +08:00
4300fdce43
[feature](merge-on-write) enable merge-on-write by default again ( #28105 )
...
fix #27188 , #28096
2023-12-14 19:35:52 +08:00
a540b7eb43
[fix](Nereids) multi_distinct_sum should inherit NullableAggregateFunction ( #28349 )
2023-12-14 19:27:06 +08:00
82a91380e6
[enhancement](compaction) Add support for limiting low priority compaction scheduling ( #27648 )
2023-12-14 18:31:23 +08:00
429a3ed4df
[fix](Nereids) simplify range produce true when reference is nullable ( #28386 )
...
if reference is nullable, even if range is all, we should not return
true, but should return reference is not null. for example,
before simplify: c1 > 5 or c1 < 10
after simplify:
c1 is nullable: c1 IS NOT NULL
c1 is not nullable: TRUE
2023-12-14 18:10:17 +08:00
23941ef305
[improve](pipelineX) improve partition node dependency logical ( #28399 )
2023-12-14 18:05:39 +08:00
01c94a554d
[fix](autoinc) Fix broker load when target table has autoinc column ( #28402 )
2023-12-14 18:02:54 +08:00
c08ab9edc7
[feature](HiveCatalog) Support for getting hive meta data from relational databases under HMS ( #28188 )
2023-12-14 17:50:17 +08:00
38e79e32fa
[fix](mtmv)fix start time can not be earlier than the current time ( #28379 )
2023-12-14 17:28:04 +08:00
9fe2fce306
[minor](refactor) remove unused code ( #28383 )
2023-12-14 17:16:41 +08:00
6c9c1ef2f2
[test](regression) add flink sync db case ( #19328 )
...
add flink sync db case
Mock cdc sync data write
Contains multi-table import, add and drop columns
2023-12-14 17:06:17 +08:00
e1f7b6cff4
[typo](doc) update spark connector doc for version 1.3.0 ( #28239 )
...
Co-authored-by: gnehil <gnehil489@github>
2023-12-14 16:48:10 +08:00
634292ec1d
[Enhancement](build support) make run_clang_format compatible with python3 ( #28404 )
...
run_clang_format.py in python3 env will produce error like "ModuleNotFoundError: No module named 'distutils.util' ", try to fix this.
2023-12-14 16:02:47 +08:00
9e62e3e868
[doc](readme)Update README.md ( #28391 )
2023-12-14 14:50:43 +08:00
ee24667b9f
[fix](group commit) Fix some group commit problems ( #28319 )
2023-12-14 14:38:56 +08:00
e53cfa09da
[fix](join) incorrect result of right anti join with nullable ( #28301 )
2023-12-14 14:07:12 +08:00
bb23078d43
[feture](Nereids) support delete from without using ( #28083 )
...
support sql: DELETE FROM <table_name> WHERE <predicates>
2023-12-14 13:54:33 +08:00
e5a57f82ec
[fix](Variant Type) Fixes the desc failure ( #28343 )
...
fix the desc failure when there is no decomposition of columns in the variant column.
2023-12-14 13:20:43 +08:00
79f03d6fde
[docs](releasenote)2.0.3 release note ( #28373 )
2023-12-14 13:07:24 +08:00
c00dca70e6
[pipelineX](local shuffle) Support parallel execution despite of tablet number ( #28266 )
2023-12-14 12:53:54 +08:00
b443db6e3e
[fix](doriswriter)Fix multiple concurrent import label duplication issues #28344
2023-12-14 12:39:22 +08:00
48937fef48
[Performance](json reader) optimize filling default values ( #25542 )
...
Add a faster path for filling default values, since looking up value map is relatively slow
2023-12-14 10:20:29 +08:00
97df5981a1
[Fix](inverted index) update clucene ( #28356 )
2023-12-14 10:01:23 +08:00
1901f0f35b
[fix](function) bad performance caused by creating Status::RuntimeError ( #28362 )
2023-12-14 09:29:22 +08:00
e5be751697
[bug](exchange) Fix core dump if prepare failed ( #28363 )
2023-12-13 22:59:12 +08:00
33e24436d6
[fix](regression) mow cases should use assertEquals not assertTrue ( #28338 )
2023-12-13 22:14:32 +08:00
a271fee3c5
[test](statistics)Add external empty table test case. ( #28267 )
2023-12-13 21:48:01 +08:00
e4f7b25276
[optimize](invert index) inverted indexes optimize the file cache ( #27887 )
...
Arrange idx files in the order of segment, fnm, tii, tis to ensure that data opened during the open index phase is added to the idx file cache.
2023-12-13 20:48:53 +08:00
099b1b7106
[fix](statistics)Fix column stats trigger info bug ( #28303 )
...
Before, we didn't update the jobType info in ColStatsMeta. This will case the jobType always be the type
when it first be set. For example, if we manually analyzed table, the jobType will always be MANUAL,
even if this table is auto analyzed again later.
2023-12-13 20:31:03 +08:00
0868526f6a
[fix](sequence column) fix update fail on nereids planner ( #28031 )
...
1. if we set enable_fallback_to_original_planner to false, the UPDATE sql in regression case test_unique_table_sequence will fail due to：“Table test_uniq_sequence has sequence column, need to specify the sequence column”，The bug is introduced by [fix](sequence column) insert into should require sequence column in all scenario #27780
2. fix insert fail in transaction mode, which is introduced by [refactor](Nereids) let insert into compatible with legacy planner #27947
3. add cases with session variable enable_fallback_to_original_planner false
2023-12-13 19:41:41 +08:00
dbc2b081d7
[fix](Nereids) could not do partition prunning when predicates is NULL ( #28294 )
2023-12-13 19:36:42 +08:00
ec91dd1129
[opt](vfilescanner) interrupt running parquet/orc readers when scannode is finished ( #28223 )
...
VScanNode::get_next will check whether the ScanNode has reached limit condition, and send eos to TaskScheduler, and TaskScheduler will try to close ScanNode.
However, ScanNode must wait all running scanners finished, so even if ScanNode has reached limit condition, it can't be closed immediately.
This PR try to interrupt the running readers, and make ScanNode to end as soon as possible.
2023-12-13 19:31:08 +08:00
524bce3c07
[doc](auditlog) update description of field sql_digest ( #28340 )
2023-12-13 19:27:48 +08:00
4f5821407f
[case]Load data with load_parallelism=any > 1 and stream load with compress type ( #27306 )
2023-12-13 18:41:14 +08:00
43327383c3
[regression test](broker laod) add exception case for merge type ( #27840 )
2023-12-13 18:34:34 +08:00
9b94faf403
fix](memory_leak) fix memory leak on vrow_distribution #28336
2023-12-13 18:22:14 +08:00
ea832744df
[fix](journal) ensure txns are matched with the master before replaying ( #28192 )
2023-12-13 18:14:51 +08:00
