For hive version lower than 2.3.7, there is no enum ClientCapability.INSERT_ONLY_TABLES.
So if we send this enum to the server side, the server side will get a null,
and this will cause some undefined behavior, eg, failed to get tables infos from hms.
normally, mv column's data type should be same as base table. This pr plays as a fail-safe, if mv column's data type is different from base table accidentally, fall back to select base table to make the query works.
### How to reproduce
1. create a database db1 and a table tbl1;
2. insert some data and export with label L1;
3. drop the db1 and tbl1, and recreate them with same name.
4. insert some data and export with same label L1;
Expect: export success
Actual: error: Label L1 have already been used.
This PR fix it.
Before, the auto analyze job start time was the job creation time, not the start to execute time, which is inaccurate. This pr is to change the start time to the first task start to execute time.
If there exists huge datasets with many database and may tables and many columns, Auto collector might be submit too many jobs which would occupy too much of FE memory.
In this PR, limit job each round could submit up to 5
In the previous PR #27124, we used `objectMapper.readValue` for deserialization. However, this method does not handle null fields, which can lead to issues when upgrading from older versions. Specifically, if a required field is missing in the persistent data, `String realColumnNamesJson = serializeMap.get(REAL_COLUMNS);` will return null, resulting in deserialization errors and frontend startup failure. This issue is likely to occur when upgrading from an older version that uses Jdbc Catalog to a new version including PR #27124. As this represents a specific upgrade scenario involving compatibility with old version data structures, it was not covered in the regular PR test cases. Given the specificity and difficulty in replicating such a scenario, no special test cases were added for this PR.
```
java.lang.NullPointerException: null
at com.sleepycat.je.rep.util.ReplicationGroupAdmin.getMasterSocket(ReplicationGroupAdmin.java:191)
at com.sleepycat.je.rep.util.ReplicationGroupAdmin.doMessageExchange(ReplicationGroupAdmin.java:607)
at com.sleepycat.je.rep.util.ReplicationGroupAdmin.getGroup(ReplicationGroupAdmin.java:406)
at org.apache.doris.ha.BDBHA.getElectableNodes(BDBHA.java:132)
at org.apache.doris.common.proc.FrontendsProcNode.getFrontendsInfo(FrontendsProcNode.java:84)
at org.apache.doris.qe.ShowExecutor.handleShowFrontends(ShowExecutor.java:1923)
at org.apache.doris.qe.ShowExecutor.execute(ShowExecutor.java:355)
at org.apache.doris.qe.StmtExecutor.handleShow(StmtExecutor.java:2113)
...
```
1. optimize rf prune when col stats are not avaliable
2. add regression case to check plan and rf for tpcds_sf100 with stats
3. add regression case to check plan and rf for tpcds_sf100 without stats
fill miss slot in having subquery.
such as
```
select * from t group by k having max(k) in (select k from t2)
```
the max(k) should be push down aggregate
1.
```
java.lang.NullPointerException: null
at org.apache.doris.catalog.Env.getCurrentSystemInfo(Env.java:793) ~[classes/:?]
at org.apache.doris.qe.SimpleScheduler$UpdateBlacklistThread.run(SimpleScheduler.java:206) ~[classes/:?]
at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_382]
java.lang.NullPointerException
at org.apache.doris.qe.OlapQueryCacheTest.setUp(OlapQueryCacheTest.java:226)
```
2.
```
[ERROR] testSqlCacheKeyWithNestedViewForNereids Time elapsed: 1.962 s <<< FAILURE!
java.lang.AssertionError: SELECT command denied to user 'testCluster:testUser'@'192.168.1.1' for table 'internal: testCluster:testDb: appevent'
at org.apache.doris.qe.OlapQueryCacheTest.parseSqlByNereids(OlapQueryCacheTest.java:579)
at org.apache.doris.qe.OlapQueryCacheTest.testSqlCacheKeyWithNestedViewForNereids(OlapQueryCacheTest.java:1338)
```
3.
```
[ERROR] Tests run: 28, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 113.63 s <<< FAILURE! - in org.apache.doris.qe.OlapQueryCacheTest
[ERROR] testCacheModeTable Time elapsed: 1.657 s <<< ERROR!
java.lang.IllegalArgumentException: Value of type org.apache.doris.qe.QueryState incompatible with return type org.apache.doris.system.SystemInfoService of org.apache.doris.catalog.Env#getCurrentSystemInfo()
at org.apache.doris.qe.OlapQueryCacheTest.setUp(OlapQueryCacheTest.java:156)
```
This reverts commit 5b641ebd40fff71e632ee9be4ede58b744b602b9.
Currently, Deltalake Catalog is not a usable feature. We will continue to implement it in the datalake plug-in system in the future, so we will delete it from the FE code for now.
Single key column of unique/agg olap table has the same value of count and ndv, for this kind of column,
don't need to calculate ndv, simply use count as ndv.
```
2023-11-05 22:52:01,914 WARN (mysql-nio-pool-167|1655) [StmtExecutor.analyze():992] Analyze failed. stmt[222101, ec1adf0f9f7b405d-8cb71a522bc50b82]
java.lang.NullPointerException: null
at org.apache.doris.planner.OlapScanNode.computeSampleTabletIds(OlapScanNode.java:952) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.planner.OlapScanNode.init(OlapScanNode.java:548) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.planner.SingleNodePlanner.createScanNode(SingleNodePlanner.java:2064) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.planner.SingleNodePlanner.createTableRefNode(SingleNodePlanner.java:2213) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.planner.SingleNodePlanner.createSelectPlan(SingleNodePlanner.java:1244) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.planner.SingleNodePlanner.createQueryPlan(SingleNodePlanner.java:266) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.planner.SingleNodePlanner.createSingleNodePlan(SingleNodePlanner.java:189) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.planner.OriginalPlanner.createPlanFragments(OriginalPlanner.java:160) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.planner.OriginalPlanner.plan(OriginalPlanner.java:101) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.analyzeAndGenerateQueryPlan(StmtExecutor.java:1141) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.analyze(StmtExecutor.java:975) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.executeByLegacy(StmtExecutor.java:673) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:451) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:422) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:435) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.dispatch(ConnectProcessor.java:583) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:834) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_333]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_333]
at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_333]
2023-11-05 22:52:01,914 WARN (mysql-nio-pool-167|1655) [StmtExecutor.executeByLegacy():776] execute Exception. stmt[222101, ec1adf0f9f7b405d-8cb71a522bc50b82]
org.apache.doris.common.AnalysisException: errCode = 2, detailMessage = Unexpected exception: null
at org.apache.doris.qe.StmtExecutor.analyze(StmtExecutor.java:993) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.executeByLegacy(StmtExecutor.java:673) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:451) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:422) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:435) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.dispatch(ConnectProcessor.java:583) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:834) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_333]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_333]
at java.lang.Thread.run(Thread.java:750) ~[?:1.8.0_333]
```
