This PR makes the following changes to the connection pool of JDBC Catalog
1. Set the maximum connection survival time, the default is 30 minutes
- Moreover, one-half of the maximum survival time is the recyclable time,
- One-tenth is the check interval for recycling connections
2. Keepalive only takes effect on the connection pool on BE, and will be activated based on one-fifth of the maximum survival time.
3. The maximum number of existing connections is changed from 100 to 10
4. Add the connection cache recycling thread on BE, and add a parameter to control the recycling time, the default is 28800 (8 hours)
5. Add CatalogID to the key of the connection pool cache to achieve better isolation, requires refresh catalog to take effect
6. Upgrade druid connection pool to version 1.2.20
7. Added JdbcResource's setting of default parameters when upgrading the FE version to avoid errors due to unset parameters.
If there are too many backup/restore jobs, it may cause OOM. This PR allows the user to skip all backup/restore jobs if max_backup_restore_job_num_per_db is set to 0.
* add left anti join ut
* forbidden the self join partition column get
* [Fix](nereids) Disable getting partition related table and column when self join
* fix code style
Issue Number: #30484
The objects stored in PriorityQueue must implement the Comparable interface or passed into the customized `Comparator`.
If we don't do this, run the program in the JDK17 environment will report an exception:
```java
Caused by: java.lang.AssertionError: Expect exception msg contains 'query wait timeout', but meet
'java.sql.SQLException: ClassCastException,
msg: class org.apache.doris.resource.workloadgroup.QueueToken cannot be cast to class java.lang.Comparable
(org.apache.doris.resource.workloadgroup.QueueToken is in unnamed module of loader 'app'; java.lang.Comparable is in module java.base of loader 'bootstrap')'
```
When a partition in OlapTable is removed, we should use partition id to delete the related stats record in column_statistics. Before, it was using id, which may cause delete useful stats of other partition.
1. Skip parquet file which has only 4 bytes length: PAR1
2. Refactor the schema init method of iceberg/hudi/hive table in hms catalog
1. Remove some redundant methods of `getIcebergTable`
2. Fix issue described in #23771
3. Support HoodieParquetInputFormatBase, treat it as normal hive table format
4. When listing file, skip all hidden dirs and files
1. Missing workload_group column in audit table
2. Extract the definition of internal schema's tables into a new class
3. Fix bug that audit loader has no authorization to load data to audit_table, introduced from #29790
4. Fix bug that audit_log can not be modified to 3 replica because it is partitioned table
* [feature](journal) Add a method to write a set of journals in batch (#30380)
* [feature](journal) Add log and metric to improve the observability of journal batch (#30401)
write edit log in lock scope to ensure the order of log sequence.
To avoid the sequence like:
```
add primary key pk1
add foreign key ref pk1
log foreign key
log primary key
```
LinkedList is not thread safe and there may be issues with concurrent queries and writes. Change it to ConcurrentLinkedQueue
Caused by: java.util.ConcurrentModificationException
at java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966) ~[?:1.8.0_131]
at java.util.LinkedList$ListItr.next(LinkedList.java:888) ~[?:1.8.0_131]
at org.apache.doris.tablefunction.MetadataGenerator.taskMetadataResult(MetadataGenerator.java:694) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.tablefunction.MetadataGenerator.getMetadataTable(MetadataGenerator.java:119) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.service.FrontendServiceImpl.fetchSchemaTableData(FrontendServiceImpl.java:2195) ~[doris-fe.jar:1.2-SNAPSHOT]
... 13 more
Issue Number: close#30484
problem:
gson will use Java's reflection mechanism to generate a default Adapter, but JDK17 is prohibited from visiting such an access.
solution:
gson has provided solutions since 2.9.1, which can bypass this problem: Add support for reflection access filter by Marcono1234 · Pull Request #1905 · google/gson
We need to upgrade the gson version and use this solution
