database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

[opt](hive)opt select count(*) stmt push down agg on parquet in hive . (#22115)

Browse Source 
Optimization "select count(*) from table" stmtement , push down "count" type to BE.
support file type : parquet ，orc in hive .

1. 4kfiles , 60kwline num 
    before:  1 min 37.70 sec 
    after:   50.18 sec

2. 50files , 60kwline num
    before: 1.12 sec
    after: 0.82 sec
This commit is contained in:
daidai
2023-07-29 00:31:01 +08:00
committed by GitHub
parent 53d255f482
commit ae8a26335c
21 changed files with 473 additions and 65 deletions
Download Patch File Download Diff File Expand all files Collapse all files

13
be/src/vec/exec/scan/vscan_node.cpp
View File

@ -113,6 +113,19 @@ Status VScanNode::init(const TPlanNode& tnode, RuntimeState* state) {
} else {
_max_pushdown_conditions_per_column = config::max_pushdown_conditions_per_column;
}
// tnode.olap_scan_node.push_down_agg_type_opt field is deprecated
// Introduced a new field : tnode.push_down_agg_type_opt
//
// make it compatible here
if (tnode.__isset.push_down_agg_type_opt) {
_push_down_agg_type = tnode.push_down_agg_type_opt;
} else if (tnode.olap_scan_node.__isset.push_down_agg_type_opt) {
_push_down_agg_type = tnode.olap_scan_node.push_down_agg_type_opt;
} else {
_push_down_agg_type = TPushAggOp::type::NONE;
}
return Status::OK();
}