pick #40891
This PR makes the following changes to the uppercase and lowercase
mapping of JdbcCatalog
1. The identifierMapping is managed by JdbcExternalCatalog instead of
JdbcClient to better control its lifecycle
2. The identifierMapping no longer loads remoteName alone, but Catalog
controls the loading uniformly
3. The identifierMapping will be loaded when each FE performs
makeSureInitialized() to ensure that each FE has a mapping
4. The initialization of mapping will only be performed once in
makeSureInitialized(), which means that even if you use metaCache, if
your source data is updated when identifierMapping is enabled, you must
refresh the catalog to query normally.
5. The identifierMapping is only responsible for the properties of the
Catalog and is no longer affected by the fe config, simplifying the
processing logic
6. If lower_case_mete_names is false and meta_names_mapping is empty in
the catalog properties, the identifierMapping will no longer take
effect, further enhancing the stability of the default settings
7. The JdbcClient is no longer closed during onRefreshCache, reducing
the repeated creation of resources, improving reuse, and reducing the
leakage of some global shared threads
cherry-pick： https://github.com/apache/doris/pull/40820
example: select append_trailing_char_if_absent('it','a') would return
null in original design, it can not return null when folding constant on
fe any time
pick from master #40801
This PR simplifies the handling of `ScanNode` projection logic.
Previously, the code included multiple conditional checks to determine
whether a `projectionTuple` should be generated. These conditions have
been removed, and now `projectionTuple `is always generated for
`ScanNode`, ensuring a consistent projection setup. Additionally,
redundant handling of `SlotId` and `SlotRef` has been eliminated, making
the code cleaner and easier to maintain. The behavior for `OlapScanNode`
remains unchanged.
pick #40818
If IO/CPU resources are tight, routine load task is likely to timeout.
The current method is self-adaption backoff
https://github.com/apache/doris/pull/32227, but the problem is it will
do some ineffective work to match proper timeout. For one routine load
task, a better way to handle task is finishing executing instead of
retry when resources are tight. Therefore, this pr increase timeout to
make "task always finish even if it is slow when resources are tight".
Remove analyze retry logic when task failed. Because usually retry would
fail again and retry would bring a long time of sleep, which cause the
analyze job running too slow.
Master pr: https://github.com/apache/doris/pull/33703
pick (#40728)
```
2024-09-11 20:00:53,079 ERROR (replayer|105) [RoutineLoadManager.replayChangeRoutineLoadJob():836] should not happened
org.apache.doris.common.DdlException: errCode = 2, detailMessage = Could not transform PAUSED to PAUSED
at org.apache.doris.load.routineload.RoutineLoadJob.checkStateTransform(RoutineLoadJob.java:855) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.load.routineload.RoutineLoadJob.unprotectUpdateState(RoutineLoadJob.java:1407) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.load.routineload.RoutineLoadJob.updateState(RoutineLoadJob.java:1394) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.load.routineload.RoutineLoadManager.replayChangeRoutineLoadJob(RoutineLoadManager.java:834) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.persist.EditLog.loadJournal(EditLog.java:717) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.Env.replayJournal(Env.java:2913) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.catalog.Env$4.runOneCycle(Env.java:2675) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.common.util.Daemon.run(Daemon.java:116) ~[doris-fe.jar:1.2-SNAPSHOT]
```
`unprotectNeedReschedule()` will change job state to
`JobState.NEED_SCHEDULE` without `logOpRoutineLoadJob`.If job is paused
then rescheduled and paused finally, the record of two consecutive edit
logs will be 'PAUSED', the correct
replay sequence should be: `PAUSED` -> `NEED_SCHEDULE` ->` PAUSED`.
Therefore, it is need to write edit log when rescheduled job.
