Fix 2 issues:
The replication num analysis will throw exception when there is no BE alive when FE restart.
org.apache.doris.common.AnalysisException: errCode = 2, detailMessage = errCode = 2, detailMessage = replication num should be less than the number of available backends. replication num is 1, available backend num is 0
at org.apache.doris.common.util.PropertyAnalyzer.analyzeReplicaAllocation(PropertyAnalyzer.java:1018) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.TableProperty.buildReplicaAllocation(TableProperty.java:483) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.TableProperty.read(TableProperty.java:529) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.OlapTable.readFields(OlapTable.java:1370) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.Table.read(Table.java:383) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.Database.readFields(Database.java:631) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.datasource.InternalCatalog.loadDb(InternalCatalog.java:3075) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.Env.loadDb(Env.java:1764) ~[doris-fe.jar:1.2-SNAPSHOT]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_341]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_341]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_341]
at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_341]
at org.apache.doris.persist.meta.MetaReader.read(MetaReader.java:116) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.Env.loadImage(Env.java:1701) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.master.Checkpoint.doCheckpoint(Checkpoint.java:130) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.master.Checkpoint.runAfterCatalogReady(Checkpoint.java:79) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.common.util.MasterDaemon.runOneCycle(MasterDaemon.java:58) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.common.util.Daemon.run(Daemon.java:116) ~[doris-fe.jar:1.2-SNAPSHOT]
Paimon File Catalog meta issue
Fix#23366
Sometimes the first run of query may be longer then former given threshold, which case test fail.
Also add a new session variable test_query_cache_hit
So that we can use it to test if cache is hit in regression test
1. Change from using string matching function to using Expr matching
2. Replace the `nvl` function with `ifnull` when pushed down to MySQL
3. Adapt ClickHouse's `from_unixtime` function to push down
4. Non-function filtering can still be pushed down when `enable_func_pushdown` is set to false
1. Fix hive partition prune bug, introduced from #23845, will fail `test_hive_default_partition` test case.
2. Fix `test_local_tvf.groovy` test case, the path of local tvf should be relative path.
3. Fix `test_external_catalog_hive` test case, the `partitions` is now reserve keywords
4. Support `local` tvf in Nereids, but fix related issue like:
```
Caused by: java.lang.NullPointerException
at org.apache.doris.nereids.stats.ExpressionEstimation.castMinMax(ExpressionEstimation.java:171) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.nereids.stats.ExpressionEstimation.visitCast(ExpressionEstimation.java:167) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.nereids.stats.ExpressionEstimation.visitCast(ExpressionEstimation.java:109) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.nereids.trees.expressions.Cast.accept(Cast.java:55) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.nereids.stats.ExpressionEstimation.visitAlias(ExpressionEstimation.java:394) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.nereids.stats.ExpressionEstimation.visitAlias(ExpressionEstimation.java:109) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.nereids.trees.expressions.Alias.accept(Alias.java:145) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.nereids.stats.ExpressionEstimation.estimate(ExpressionEstimation.java:119) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.nereids.stats.StatsCalculator.lambda$computeProject$7(StatsCalculator.java:785) ~[doris-fe.jar:1.2-SNAPSHOT]
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_341]
at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948) ~[?:1.8.0_341]
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482) ~[?:1.8.0_341]
```
1. Change the external hive docker network mode from the bridge mode to the host mode to support the external test of the multi-node doris cluster
2. Added more hive test data in various formats
3. Added a test case with hive
Add file cache regression test in tpch 1g on orc&parquet format.
tpch will run 3 times:
1. running without file cache
2. running with file cache for the first time
3. running with file cache for the second time
The file cache configuration is already added in `be/conf/be.conf` on the regression test environment, and the available capacity is 100MB. After running the tpch 1g test, the metrics introduced by https://github.com/apache/doris/pull/19177 is like:
```
doris_be_file_cache_normal_queue_curr_size{path="/mnt/datadisk1/gaoxin/file_cache"} 92808933
doris_be_file_cache_normal_queue_curr_elements{path="/mnt/datadisk1/gaoxin/file_cache"} 59
doris_be_file_cache_normal_queue_max_elements{path="/mnt/datadisk1/gaoxin/file_cache"} 102400
doris_be_file_cache_normal_queue_max_size{path="/mnt/datadisk1/gaoxin/file_cache"} 89128960
doris_be_file_cache_removed_elements{path="/mnt/datadisk1/gaoxin/file_cache"} 2132
doris_be_file_cache_segment_reader_cache_size{path="/mnt/datadisk1/gaoxin/file_cache"} 54
```
CREATE EXTERNAL TABLE `dim_server` (
`col1` varchar(50) NOT NULL,
`col2` varchar(50) NOT NULL
)
create view ads_oreo_sid_report
(
`col1` ,
`col2`
)
AS
select
tmp.col1,tmp.col2
from (
select 'abc' as col1,'def' as col2
) tmp
inner join dim_server ds on tmp.col1 = ds.col1 and tmp.col2 = ds.col2;
select * from ads_oreo_sid_report where col1='abc' and col2='def';
before this pr, col1='abc' and col2='def' can't be pushed to dim_server. now the 2 predicates can be pushed to odbc table.
1. ctas should support without distribution desc
2. ctas should support column name list
3. ctas should throw exception when excution failed
4. ctas should convert null type to tinyint
5. ctas should support type conversion
6. ctas should convert first column from string to varchar
The changes of this PR for JdbcOracleClient are as follows:
#### bug fixes:
1. Fix the problem that if there is an approximate table name for Schema synchronization with a table name with `/` characters, the synchronization Column will be confused
2. Fix the NPE problem of metadata synchronization after enabling lower_case_table_names configuration
#### improvement:
1. Modify the method of synchronizing Oracle User to Doris Database mapping, use `metadata.getSchemas` instead of `SELECT DISTINCT OWNER FROM all_tables`
2. When synchronizing metadata, change `null` at the catalog level to `conn.getcatalog`
Load data from hdfs in hive will move the source directory into table's location directory, leading the error like Can not get first file, please check uri in tvf test.
sort out the test cases of external table.
After modify, there are 2 directories:
1. `external_table_p0`: all p0 cases of external tables: hive, es, jdbc and tvf
2. `external_table_p2`: all p2 cases of external tables: hive, es, mysql, pg, iceberg and tvf
So that we can run it with one line command like:
```
sh run-regression-test.sh --run -d external_table_p0,external_table_p2
```
