cf66280e60
In the previous implementation, when aggregating partition statistics into column statistics, the calculation of distinct values (ndv) for the entire column was performed without using sampling, resulting in reduced efficiency of the sampling process.
Before this PR analyze below table which has 1000000 lines would cost 5.75sec, after this PR, it would cost 3.39sec.
```sql
CREATE TABLE IF NOT EXISTS `duplicate_all` (
`k3` int(11) null comment "",
`k0` boolean null comment "",
`k1` tinyint(4) null comment "",
`k2` smallint(6) null comment "",
`k4` bigint(20) null comment "",
`k5` decimalv3(9, 3) null comment "",
`k6` char(36) null comment "",
`k10` date null comment "",
`k11` datetime null comment "",
`k7` varchar(64) null comment "",
`k8` double null comment "",
`k9` float null comment "",
`k12` string null comment "",
`k13` largeint(40) null comment ""
) engine=olap
DUPLICATE KEY(`k3`)
DISTRIBUTED BY HASH(`k3`) BUCKETS 5 properties("replication_num" = "3")
```
# Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information # regarding copyright ownership. The ASF licenses this file # to you under the Apache License, Version 2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, # software distributed under the License is distributed on an # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. # fe-common This module is used to store some common classes of other modules. # spark-dpp This module is Spark DPP program, used for Spark Load function. Depends: fe-common # fe-core This module is the main process module of FE. Depends: fe-common, spark-dpp