bc12a05915
[fix](Nereids) explain graph insert-select NPE ( #28007 )
2023-12-07 17:25:44 +08:00
9c63dfd692
[improve](move-memtable) support variant in move-memtable ( #28084 )
...
* [improve](move-memtable) support variant in move-memtable
Pass the flush schema to the destination end of the sink.
2023-12-07 16:41:38 +08:00
1a46cf6fb5
[fix](split_by_string) Fix split by string core on column string ( #28030 )
2023-12-07 16:36:13 +08:00
495c01bdfd
[fix](session-variable) enable_unicode_name_support need forward to master ( #28112 )
...
Session variable `enable_unicode_name_support` is used to enable using unicode for table/column names.
And it should be forward to master when executing `create table` stmt on non-master FE.
Otherwise, even if we set `enable_unicode_name_support` to true on non-master FE,
we can create table with unicode name.
2023-12-07 15:43:11 +08:00
8526b9ffbe
[imporvement](table property) support for alter table property disable_auto_compaction ( #27961 )
...
in some case, some tablets may cause coredump or OOM when compaction, and it is necessary to manually close the compaction of a specific table by 'disable_auto_compaction' to make be service available
This commit allow modify disable_auto_compaction table property in schema change.
---------
Signed-off-by: nextdreamblue <zxw520blue1@163.com >
2023-12-07 15:08:39 +08:00
cb9a6f63ab
[refactor](simd_json_reader) refactor simd json parse to adapt stream parse ( #27972 )
2023-12-07 14:45:15 +08:00
3f3843fd4f
[fix](load) fix loaded rows error ( #28061 )
2023-12-07 14:43:51 +08:00
1dabd0af43
[Fix](inverted index) fix need read data optimize problem ( #28104 )
2023-12-07 14:05:29 +08:00
f2477a3fcc
[Regression](bitmap index) fix bitmap index test case ( #28109 )
2023-12-07 13:29:24 +08:00
c5f00fa746
[fix](move-memtable) temporary disable load stream profile ( #28029 )
2023-12-07 12:05:29 +08:00
8c79b86f5b
Revert "[feature](merge-on-write) enable merge-on-write by default ( #27188 )" ( #28096 )
...
This reverts commit 00c8bab84de8154052f9d323800b436cd0ad36e5.
2023-12-07 11:31:36 +08:00
3a7a8bb107
[opt](resource-tag) root and admin user can use any resource tag by default ( #28088 )
...
In #25331 , I change the behavior of user's default resource tag, that is, if a user does not set resource tag,
it can only use default resource tag.
This PR change this logic. The normal user can only use default resource tag if resource tag is not set,
but root and admin user can use any resource tag if resource tag is not set.
2023-12-07 11:22:30 +08:00
8df94f0d07
[fix](remote-scanner-pool) missing _remote_thread_pool_max_size value ( #28057 )
2023-12-07 11:18:42 +08:00
0f1da35c25
[community](slack)Update slack channel links ( #28100 )
2023-12-07 11:02:08 +08:00
6f89fae22c
[fix](profile) fix profile TotalTime is 0ns when pipeline ( #28048 )
...
* [fix](profile) fix profile TotalTime is 0ns when pipeline
Signed-off-by: nextdreamblue <zxw520blue1@163.com >
* fix
Signed-off-by: nextdreamblue <zxw520blue1@163.com >
---------
Signed-off-by: nextdreamblue <zxw520blue1@163.com >
2023-12-07 10:30:41 +08:00
00bcf0d762
[doc](paimon) add more example for paimon catalog ( #28087 )
2023-12-07 10:21:02 +08:00
b6722653cf
[test](Job)Delete the JOB show syntax (now we use TVF) and add tvf case ( #28058 )
2023-12-07 10:17:52 +08:00
630de740ea
[fix](meta) fix bug for using full name in show_full_columns stmt ( #28019 )
2023-12-07 10:17:25 +08:00
616ba2823f
[regression test](broker load) Test case missing files ( #27580 )
2023-12-07 10:17:07 +08:00
4cac07be30
[improvement](statistics)Analyze empty table. #28077
...
Analyze a table even when it's empty. The result should be like this:
mysql> show column stats nation;
+-------------+-------+------+----------+-----------+---------------+------+------+--------+--------------+---------+-------------+---------------------+
| column_name | count | ndv | num_null | data_size | avg_size_byte | min | max | method | type | trigger | query_times | updated_time |
+-------------+-------+------+----------+-----------+---------------+------+------+--------+--------------+---------+-------------+---------------------+
| n_comment | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | N/A | N/A | FULL | FUNDAMENTALS | MANUAL | 0 | 2023-12-06 19:22:09 |
| n_nationkey | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | N/A | N/A | FULL | FUNDAMENTALS | MANUAL | 0 | 2023-12-06 19:22:09 |
| n_regionkey | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | N/A | N/A | FULL | FUNDAMENTALS | MANUAL | 0 | 2023-12-06 19:22:09 |
| n_name | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | N/A | N/A | FULL | FUNDAMENTALS | MANUAL | 0 | 2023-12-06 19:22:09 |
+-------------+-------+------+----------+-----------+---------------+------+------+--------+--------------+---------+----
2023-12-07 10:16:52 +08:00
42b3dd35bb
[regression test](broker load) add case for without filepath ( #27658 )
2023-12-07 10:15:37 +08:00
e3e78a79d7
[chore](StoragePolicy) Don't print stack when invoking the storage manager #28091
2023-12-07 10:06:31 +08:00
54d062ddee
[feature](stream load) (step one)Add arrow data type for stream load ( #26709 )
...
By using the Arrow data format, we can reduce the streamload of data transferred and improve the data import performance
2023-12-06 23:29:46 +08:00
2ca66ff61c
[typo](test) Use correct sql stmt in regression test ( #27992 )
2023-12-06 23:28:29 +08:00
4a4d137402
[feature](workloadgroup) support nereids internal query and all dml query ( #28054 )
...
support nereids internal query to bind a workload group
support insert into select bind workload group
support create table as select bind workload group
change token wait timeout to be query timeout or queue timeout
query queue should not bind to pipeline engine, it could be used every where.
2023-12-06 21:07:55 +08:00
00c8bab84d
[feature](merge-on-write) enable merge-on-write by default ( #27188 )
2023-12-06 21:06:58 +08:00
d0697db7b8
[fix](doris compose) fix up command ( #27573 )
2023-12-06 21:06:10 +08:00
d4f9e12ec7
[fix](load) fix memtable mem_tracker too large ( #25205 )
2023-12-06 21:04:21 +08:00
0ff5a1cc25
[fix](doc) spell error and aligned with code ( #27609 )
2023-12-06 20:58:39 +08:00
bd808ae3b4
[cleanup](load) remove unused LoadChannel::get_tablets_channels ( #27981 )
2023-12-06 20:50:26 +08:00
0282baaf56
[typo](docker) add logs console ( #27570 )
...
doris 2.0.2+ need use start_xx.sh --console
2023-12-06 20:50:09 +08:00
3e8c75e246
[minor](orc) opt the log info in orc reader ( #27951 )
2023-12-06 20:47:36 +08:00
ec08850c08
[Config](multi-catalog) Enable query hive views as default. ( #27906 )
...
Remove EXPERIMENTAL tag for enable_query_hive_views and set enable_query_hive_views to true as default.
This feature has been used on our cluster which has more then a hundred thousands of tables for several months, i think it is fine to enable it as default.
2023-12-06 20:46:09 +08:00
3868926088
[fix](ci) pass PULL_REQUEST_NUM while trigger new build ( #28065 )
...
Co-authored-by: stephen <hello-stephen@qq.com >
2023-12-06 20:42:53 +08:00
299fcc443e
[Bug](agg-state) fix stream load failed on agg-state column ( #28050 )
2023-12-06 20:41:29 +08:00
990b8d4fa5
[minor](s3client) use LOG(INFO) to print s3 client debug log if enabled ( #28059 )
2023-12-06 20:39:12 +08:00
5168422da7
[Chore](case) remove dynamic partition on test_delete case #28075
...
remove dynamic partition on test_delete case to reduce logout
2023-12-06 16:23:12,733 WARN (DynamicPartitionScheduler|58) [DynamicPartitionScheduler.recordCreatePartitionFailedMsg():581] dynamic add partition failed: errCode = 2, detailMessage = Range [types: [DATETIMEV2]; keys: [0000-01-01 00:00:00]; ..types: [DATETIMEV2]; keys: [2021-01-01 00:00:00]; ) is intersected with range: [types: [DATETIMEV2]; keys: [2020-03-01 00:00:00]; ..types: [DATETIMEV2]; keys: [2020-04-01 00:00:00]; ), db: default_cluster:regression_test_delete_p0, table: dwd_pay
2023-12-06 20:35:08 +08:00
ff56605c2e
[pipelineX](bug) Fix hash partition shuffle ( #28071 )
2023-12-06 19:26:46 +08:00
ffd7023987
[feature](nereids) Support to get partition related table from mv and check the query operator ( #28064 )
...
Function 1:
check the select query plan is contain the stmt as following or not
SELECT
[hint_statement, ...]
[ALL | DISTINCT | DISTINCTROW | ALL EXCEPT ( col_name1 [, col_name2, col_name3, ...] )]
elect_expr [, select_expr ...]
[FROM table_references
PARTITION partition_list]
[TABLET tabletid_list]
[TABLESAMPLE sample_value [ROWS | PERCENT]
[REPEATABLE pos_seek]]
[WHERE where_condition]
[GROUP BY [GROUPING SETS | ROLLUP | CUBE] {col_name | expr | position}]
[HAVING where_condition]
[ORDER BY {col_name | expr | position}
[ASC | DESC], ...]
[LIMIT {[offset,] row_count | row_count OFFSET offset}]
[INTO OUTFILE 'file_name']
if analyzedPlan contains the stmt as following
[PARTITION partition_list]
[TABLET tabletid_list] or
[TABLESAMPLE sample_value [ROWS | PERCENT]
[REPEATABLE pos_seek]]
this method will return true.
Function 2:
Get related base table info which materialized view plan column reference,
input param plan should be rewritten plan that sub query should be eliminated
2023-12-06 19:15:21 +08:00
e695cb18b1
[improvement](regression test) add fe debug point ( #28047 )
2023-12-06 18:18:05 +08:00
17be32eefa
[fix](regression test) fix table name too long ( #28049 )
2023-12-06 18:16:17 +08:00
ddb6eb5ad7
[feature](Nereids) add command for updating mv with partitions ( #28060 )
2023-12-06 17:45:09 +08:00
1aa1b2f607
[opt](Nereids) add switch to control whether use pipeline in DML ( #28037 )
...
to turn on pipeline for DML in Nerieds, please:
set enable_nereids_dml_with_pipeline = true;
2023-12-06 17:06:11 +08:00
bb67e39558
[fix](invert index) fix reader is not close ( #28033 )
2023-12-06 17:02:26 +08:00
28817990b7
[pipelineX](improvement) enable local shuffle by default ( #28046 )
2023-12-06 16:39:48 +08:00
fa5096f510
[feature](analyze_cmd) add show-tablets-belong stmt for analyzing a batch of tablet-ids ( #27994 )
2023-12-06 15:59:00 +08:00
994c5c6f6e
[chore](log) Add log to trace query cancel #28020
2023-12-06 15:51:21 +08:00
605257ccb7
[Enhancement](group commit) Add regression case for wal limit ( #27949 )
2023-12-06 14:23:50 +08:00
e791e31b7f
[test](Nereids): add regression test eliminate/infer rules ( #27985 )
2023-12-06 14:21:06 +08:00
393c491820
[FIX](map/struct)fix map/struct literal from fe ( #28026 )
2023-12-06 13:56:56 +08:00
