Commit Graph

12453 Commits

Author SHA1 Message Date
1cd99c27e2 [CI](docs)Add deadlink check (#22736) 2023-08-09 14:58:37 +08:00
77d3d4e324 [fix](cache) add sql cache conf cache_result_max_data_size (#22645)
Only the maximum number of rows in sql cache cache_result_max_row_count is not enough. If a row of data is too large, FE may OOM.
2023-08-09 14:46:23 +08:00
1d1077c3b6 [bugfix](fd) Recycle the segment file fds directly when delete stale rowset (#22705) 2023-08-09 14:45:56 +08:00
19a2617d70 [docs](streamload) improve some formatting (#22659) 2023-08-09 14:38:22 +08:00
690a519742 [fix](Nereids) disable or expansion when pipeline engine is disable (#22719) 2023-08-09 14:33:50 +08:00
31a4d6059e [fix](case) if enable_feature_binlog=false in frontend config, no nee… (#22692)
if enable_feature_binlog=false in frontend config, no nee…
2023-08-09 13:15:50 +08:00
445bebb4cb [docs](docs)Rename Titles & URL and Add Histogram Files of Aggregate Functions (#22720) 2023-08-09 13:13:07 +08:00
512af9b2ff [docs](docs)Rename Title and URL of Struct Functions (#22712) 2023-08-09 13:12:48 +08:00
7164dd09e6 [docs](docs) Rename Title and URL of String Functions (#22711) 2023-08-09 13:12:26 +08:00
d7ea27a5c7 [docs](docs) Rename Title and URL of GIS Functions (#22704) 2023-08-09 13:12:10 +08:00
9533918d4f [fix](delete) Fix parsing error of delete where date statement (#22690) 2023-08-09 12:33:03 +08:00
a8d690272f [refactor](Nereids) let topn runtime filter as PhysicalTopN's attr (#22745)
The original implement use MutableMap on PhysicalTopN.
It is easy to lose if we rewrite the plan after this processor.
The new implement use attr to indict whether use topn runtime filter
2023-08-09 12:13:21 +08:00
7890e464ee [fix](case) 1. disable unstable case window_function 2. add sync after stream load (#22677)
* Update test_dup_tab_auto_inc_with_null.groovy,add sync after streamload

* Update test_unique_table_auto_inc.groovy, add sync after streamload* [fix](case) disable unstable case window_function
2023-08-09 11:03:31 +08:00
4608dcb2d9 [fix](agg) fix coredump caused by push down count aggregation (#22699)
fix coredump caused by push down count aggregation
2023-08-09 10:21:20 +08:00
a778027569 [typo](docs)Modified description of JSON /String size (#21694) 2023-08-09 10:00:25 +08:00
508cbe030b [Chore](binlog) Refactor TABLET_MISSING in ingest_binlog && set_tstatus (#22727)
Signed-off-by: Jack Drogon <jack.xsuperman@gmail.com>
2023-08-09 09:58:54 +08:00
d3baac2952 [improvement](resource-tag) Add Backend tag location check (#22670)
Add Backend tag location check.
Avoid user set a bad backend tag, cause create table and dynamic partitions failed.
For example, the default value for all backends tag is default, When setting the replication_allocation of a table, user use the following command: ALTER TABLE example_db.mysql_table SET ("replication_allocation" = "tag.location.tag1: 1");, it can set success, but tag1 is not exist, cause dynamic partition can't create.
2023-08-09 00:08:34 +08:00
4359089b9c [fix](delete-pred) fix special char in delete sub condition #22667
For some users, their delete condition may contain special chars like '$', which will cause failure in parsing delete condition.
2023-08-09 00:04:26 +08:00
7bfcee6e71 [improvement](variable) add annotations for variables (#22292) 2023-08-08 22:16:42 +08:00
124c1b16cf [performance](load) remove unnecessary lock in TabletsChannel::add_batch (#22703)
This lock was introduced by lazy open in #18874.
It's unnecessary and costly to hold a lock while writing data to DeltaWriter in the first place.

However, since lazy open is reverted in #21821, we can completely omit this lock.
_tablet_writers is not supposed to be changed once we've reached TabletsChannel::add_batch.
2023-08-08 22:08:21 +08:00
9581d2b4eb [refactor](load) split memtable writer out of delta writer (#21892) 2023-08-08 22:02:42 +08:00
19f8264076 Refactor be CMakeLists BUILD_INDEX_TOOL && BUILD_META_TOOL, MAKE_TEST (#22730)
option

Signed-off-by: Jack Drogon <jack.xsuperman@gmail.com>
2023-08-08 20:51:10 +08:00
30ceb7aea7 [fix](chore] need to remove reference in assert_cast (#22706) 2023-08-08 20:36:05 +08:00
897db3cca5 [Chore](inverted index) refine log in DorisCompoundDirectory::FSIndexOutput (#22716) 2023-08-08 20:32:31 +08:00
b5d7e6e7d8 [improvement](stats) Add lifecycle hooks to AnalysisTask to make codes more clear (#22658) 2023-08-08 19:06:47 +08:00
a04e30d087 [Fix](Job)Fix Job schedule calculation start time (#22707)
Since we use division calculation, when the start time is not specified,
it may have a wrong deviation from our expected time.

For example, if it is the 7th minute now, the cycle is executed every two minutes.
Then it is calculated that the first execution is 8 minutes Because 7/2=3
3+1=4
But ideally we think it should be executed at the 9th minute
2023-08-08 18:30:38 +08:00
50dd318183 [style](jdbc catalog) Tidy the jdbc catalog java file directory (#22691) 2023-08-08 18:21:21 +08:00
f2dca848db [chore](Nereids): optimize to handle enforcer in MergeGroup() (#22709) 2023-08-08 16:56:34 +08:00
edd36fe86b [Chore](tablet) Remove unused BaseTablet::is_memory (#22688)
Signed-off-by: Jack Drogon <jack.xsuperman@gmail.com>
2023-08-08 16:42:59 +08:00
f2731185c9 [fix](memory) fix cache clean thread (#22472)
fix page cache update last visit time.
fix cache clean thread
2023-08-08 15:38:29 +08:00
0f15d86c43 [fix](nereids) decimalv2 and float like type's common type should be consistant with old planner in arithmetic expr (#22654)
when both decimalv2 and float like type in the arithmetic expr, the common type is depend on roundPreciseDecimalV2Value session variable. If it's true, the common type is DecimalV2Type.SYSTEM_DEFAULT, otherwise its double type.
2023-08-08 15:22:04 +08:00
8ef38637ae [docs](docs) Rename Title and URL of Date Functions (#22686) 2023-08-08 14:44:05 +08:00
0c972288ef [docs](docs)Rename Title and URL of Array Functions for SEO (#22669) 2023-08-08 14:32:05 +08:00
1d2046de64 [docs](docs)Rename Title of zh-CN Docs (#22662) 2023-08-08 14:31:28 +08:00
66784cef71 [Enhancement](Load) Stream Load using SQL (#22509)
This PR was originally #16940 , but it has not been updated for a long time due to the original author @Cai-Yao . At present, we will merge some of the code into the master first.

thanks @Cai-Yao @yiguolei
2023-08-08 13:49:04 +08:00
c4def9db5c [feature](Nereids): add enforcers in Group (#22660) 2023-08-08 13:39:55 +08:00
1617368ee1 [fix](planner) fix bug of push constant conjuncts through set operation node (#22695)
when pushing down constant conjunct into set operation node, we should assign the conjunct to agg node if there is one. This is consistant with pushing constant conjunct into inlineview.
2023-08-08 12:25:42 +08:00
d77b77a33f [feature](Nereids) eliminate sort that is not directly below result sink (#22550)
eliminate sort that is not directly below result sink.
TODO:
handle select c1 + c2 from (select c1, c2 from t order by c1) v;
2023-08-08 11:19:10 +08:00
91b15183e7 [enhance][external]enhance and fix external cases 0807 (#22689)
enhance and fix external cases 0807
2023-08-08 10:53:08 +08:00
e578e1e6a2 [opt](Nereids) turnoff pipeline when dml temporary (#22693)
pipeline could not work well for dml
2023-08-08 10:26:40 +08:00
22cbf43b14 [Improvement](binlog) Add full/incr engine clone with binlog (#22678)
Signed-off-by: Jack Drogon <jack.xsuperman@gmail.com>
2023-08-08 10:03:11 +08:00
36cea89c22 [Fix](planner)support delete conditions contain non-key columns and add check in analyze phase for delete. (#22673) 2023-08-07 21:49:53 +08:00
c9dc715c5d [fix](broker-load) fix error when using multi data description for same table in load stmt (#22666)
For load request, there are 2 tuples on scan node, input tuple and output tuple.
The input tuple is for reading file, and it will be converted to output tuple based on user specified column mappings.

And the broker load support different column mapping in different data description to same table(or partition).
So for each scanner, the output tuples are same but the input tuple can be different.

The previous implements save the input tuple in scan node level, causing different scanner using same input tuple,
which is incorrect.
This PR remove the input tuple from scan node and save them in each scanners.
2023-08-07 20:03:03 +08:00
f074909d3c [opt](Nereids) disable strict consistency dml by default temporary (#22672)
TODO:
1. optimize exchange performance
2. let table sink do merge on one replica
2023-08-07 19:38:35 +08:00
d1a2473944 [Feature](broker)Support GCS (#20904) 2023-08-07 19:37:18 +08:00
4b20f62f79 [community](github) remove Latest-Master-Code-Check (#22668) 2023-08-07 19:25:20 +08:00
77e772e103 [enhancement](config) add some pre-process and pre-check for BE storage config attentions in docs (#22486) 2023-08-07 18:16:57 +08:00
bc697ca9d6 [fix](time) fix error in time_to_sec 2023-08-07 17:33:24 +08:00
d9c93aaa1c [fix](regression) fix failed test delete_p0 in branch-2.0 #22652 2023-08-07 16:42:19 +08:00
f036cdfde6 [feature](compaction) support delete in cumulative compaction (#19609) 2023-08-07 15:22:21 +08:00