Commit Graph

18263 Commits

Author SHA1 Message Date
f816d13c56 [feature](Nereids): eliminate groupby (#28615) 2023-12-28 14:00:41 +08:00
bc08535285 [fix](Nereids) throw readable exception when meet unsupport sup-query (#29147) 2023-12-28 13:26:09 +08:00
xy
fd90c3a6a6 [optimize](cooldown)Reduce the number of calls to the pick_cooldown_rowset (#27091)
Co-authored-by: xingying01 <xingying01@corp.netease.com>
2023-12-28 13:03:33 +08:00
a7c0dddbc9 [refactor](rename) Rename some variables in pipeline for better readability (#29140)
* rft-rename

* format
2023-12-28 12:54:47 +08:00
82a8232c8a [fix](expr) Fix BE core dump while common expr filter delete condition column (#29107)
pred column also needs to be filtered by expr, exclude delete condition column, delete condition column not need to be filtered, query engine does not need it, after _output_column_by_sel_idx, delete condition materialize column will be erase at the end of the block.
Eg:
delete from table where a = 10;
select b from table;
a column only effective in segment iterator, the block from query engine only contain the b column, so no need to filter a column by expr.
2023-12-28 11:39:54 +08:00
0562999f91 [fix](doc) spell errors fixes and align with code log for memory tracker. (#28000)
Spell corrected for LastestSuccessChannelCache and aligned that with the docs
2023-12-28 11:12:35 +08:00
3dc3e81734 [Improvement](datatype) Update Parser for IPv4/v6 data types (#29044)
Transforming from parsing std:: string to parsing char * to accelerate the parsing of ipv4/v6 data types.
2023-12-28 11:00:38 +08:00
8a169b9906 [case](regression) Test enable pipeline load (#28172)
Co-authored-by: qinhao <qinhao@newland.com.cn>
2023-12-28 10:49:19 +08:00
e5b6826de6 [fix](partial update) update error code when failed to fill the missing fields (#29103)
1. InternalError is not clear for such error, use InvalidSchema Error instead
2. avoid some useless stacktrace on InternalError when load failed
2023-12-28 10:33:03 +08:00
1284975b9b [Improve](Job) Create task adds concurrency control (#29144) 2023-12-28 10:24:39 +08:00
8b225c6c3c [pipelineX](fix) Fix core dump if cancelled (#29138) 2023-12-28 10:04:51 +08:00
1aa9ac4fe4 Prevent making snapshot on remote rowset in single replica compaction (#28716) 2023-12-27 23:43:43 +08:00
f4c5ce260b [fix](statistics)Fix rowCount==0 while analyzing bug (#28969)
Sample analyzing need to get row count by using table.getRowCount(). This method is not updated in real time, which may cause the sample task to scan whole table.
This pr is to fix this. Set the flag that indicate the analyze job is for an empty table and skip scan the table. Meanwhile, don't reset updatedRows in this case.

Set hugeTableAutoAnalyzeIntervalInMillis = 0 because all default huge table size has been set to 0.
2023-12-27 23:04:37 +08:00
0bff387577 [fix](tablet stat) fix tablet stat thread block #29151 2023-12-27 22:02:42 +08:00
0cc4ee52bf [fix](move-memtable) fix streams for node memory leak in sink v2 (#29146) 2023-12-27 21:48:32 +08:00
d96278ab21 [bug](fix) show create table show comment error (#28346) 2023-12-27 21:17:20 +08:00
224677af7c delete unused code in delta writer v2 (#29131) 2023-12-27 21:04:58 +08:00
abbd2cedff [fix](Nereids) merge limit should use bottom phase (#29142) 2023-12-27 21:04:00 +08:00
0436013baf [fix](decimal) fix cast decimal overflow and add test cases for casting decimalv2 to decimalv3 (#29165) 2023-12-27 20:58:37 +08:00
5f71691401 [fix](read) fix unexpected overflow of uninitialized column data in VStatisticsIterator::next_batch (#29141) 2023-12-27 20:58:02 +08:00
9715db61d4 [FIX](complextype)fix count func with complex type (#28873) 2023-12-27 20:38:44 +08:00
6d817bc253 [fix](topn opt) avoid using topn runtime predicate which segment does not contain such column(column unique id) when pruning segment (#29148) 2023-12-27 20:31:03 +08:00
286af3bce9 [doc](file-cache) fix wrong config (#29156)
* [doc](file-cache) fix wrong config

* 2
2023-12-27 18:51:01 +08:00
d70743a2fc [fix](doc) spell errors fixes ha-action.md (#28155) 2023-12-27 16:57:07 +08:00
c1303e9eee [fix](doc) example case fixed in best practices (#28156) 2023-12-27 16:56:44 +08:00
522e48c764 [fix](doc) spell errors fixes for be-olap-error-code docs (#27866) 2023-12-27 16:55:59 +08:00
ef8ae75e37 [fix](doc) updated node version as per given in the url. (#28427) 2023-12-27 16:55:37 +08:00
cec7eba1cc [fix](doc) spell errors fixes for table-repair and table local debug (#28153) 2023-12-27 16:51:58 +08:00
4269041563 [enhancement](Nereids): catch exception when calling getConstraintMap for external map (#29112) 2023-12-27 16:45:03 +08:00
1c0bbcf416 [fix](doc) typo fix in sql-interception docs (#28445) 2023-12-27 16:42:43 +08:00
64ca508c0b [fix](doc) typo fix in schema change docs (#28453) 2023-12-27 16:42:28 +08:00
26842049f3 [fix](doc) typo fix in fqdn page (#28509) 2023-12-27 16:42:00 +08:00
c75e63a2a5 [Improvement](scan) Use scanner to do projection of scan node (#29124) 2023-12-27 16:00:52 +08:00
cd1e109cc3 [debug string](pipeline) Add necessary debug info (#29119) 2023-12-27 15:57:22 +08:00
2d2f14bc75 [fix](paimon) use SlotDescriptor to parse the required fields (#28990)
Before this PR, Paimon has created the schema of `VectorTable` by accessing meta information. However, once the schema of `VectorTable` in java is not same as `Block` in c++, BE will crashed, and there is no good way to troubleshoot errors.
2023-12-27 15:45:53 +08:00
cfed36afbf [Fix](topn opt) prevent from merge __TEMP__ column in segment iterator (#29121) 2023-12-27 15:42:48 +08:00
0607760efa [typo](resource) fix create odbc resource error msg (#29019) 2023-12-27 15:30:52 +08:00
8035287010 [fix](doc) spell errors fixes for memory tracker docs (#27865) 2023-12-27 15:22:43 +08:00
d3e1afa2e4 [fix](doc) typo fix in metaInfoActionV2 page (#28510) 2023-12-27 15:21:20 +08:00
e0014308bd [fix](load) Recycle progress before removing bulk load job (#29066) 2023-12-27 15:17:04 +08:00
578a7dadc1 [improvement](statistics)Use key column id to query column statistic table. (#28885)
Use id as where predicate to load column statistic cache. This could improve performance, because id is the first order key in column statistics table.
2023-12-27 14:38:55 +08:00
fc6a587352 [test](Nereids) add subquery pull up rule group ut (#26725)
add subquery pull up rule group ut:
single table part
(1) subquery_basic_pullup_basic: basic part
(2) subquery_basic_pullup_uk: using unique key
(3) subquery_basic_pullup_and: with and
(4) subquery_basic_pullup_or: with or
(5) subquery_basic_pullup_and_subquery: with and subquery
(6) subquery_basic_pullup_or_subquery: with or subquery

multi table part
(1) subquery_multitable_pullup_basic: basic part
(2) subquery_multitable_pullup_and: with and
(3) subquery_multitable_pullup_or: with or
(4) subquery_multitable_pullup_and_subquery: with and subquery
(5) subquery_multitable_pullup_or_subquery: with or subquery

top op related, such as group by/having, order by, win func, select list:
(1) subquery_topop_pullup_groupby: group by
(2) subquery_topop_pullup_having: having
(3) subquery_topop_pullup_orderby: order by
(4) subquery_topop_pullup_winfunc: window function
(5) subquery_topop_pullup_selectlist: select list

misc, such as group by correlated, view, dml, etc:
(1) subquery_misc_pullup_misc: group by correlated, view, etc
(2) subquery_misc_pullup_dml: insert/update/delete
2023-12-27 14:03:53 +08:00
0ac9b3d113 [Fix](Job)cancel task is not cleared from running task (#29114) 2023-12-27 13:59:12 +08:00
05f185ff44 [fix][compile]unused variable (#28992) 2023-12-27 13:00:43 +08:00
51bd402b58 [fix](planner) Fix delete statement on MOR unique table (#28968)
add conditions for mor table removed in #26776
2023-12-27 12:59:15 +08:00
e33395a0ee [fix](paimon)Remove the static attribute of the source for paimon (#29032) 2023-12-27 12:57:39 +08:00
6f5672f318 [Refact](inverted index) refactor inverted index writer init (#29072) 2023-12-27 12:49:26 +08:00
6ed09c3f46 [fix](mtmv)add log for resolve pending task (#29078) 2023-12-27 12:36:17 +08:00
0b5fe681e4 [fix](paimon) read batch by doris' batch size (#29039) 2023-12-27 12:35:17 +08:00
3e5c8d9949 [fix](read) remove logic of estimating count of rows to read in segment iterator to avoid wrong result of unique key. (#29109) 2023-12-27 12:25:14 +08:00