Push down limit-distinct through left/right outer join or cross join.
such as select t1.c1 from t1 left join t2 on t1.c1 = t2.c1 order by t1.c1 limit 1;
it is painful to align node in `explain` and node in `explain physical plan`, since they use two different sets of node IDs.
This pr makes 'explain' command use node IDs of their correspond node in 'explain physical plan'
If user has database with same name mysql, will introduce problem when doing checkpoint.
Solution:
Add check for this situation, if duplicate, exit and print log info to prevent damage of metadata;
Add fe config field: mysqldb_replace_name to make things correct if user already has mysql db.
Related pr: #23087#22868
PR https://github.com/apache/doris/pull/24606 has updated hbase version to 2.5.5, but it conflict with hudi, causing error like:
```
org.apache.doris.common.AnalysisException: errCode = 2, detailMessage = Unexpected exception: Failed to get hudi partitions
at org.apache.doris.qe.StmtExecutor.analyze(StmtExecutor.java:1021) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.executeByLegacy(StmtExecutor.java:696) ~[doris-fe.jar:1.2-SNAPSHOT]
...
Caused by: java.lang.NullPointerException
at org.apache.hadoop.fs.FilterFileSystem.getConf(FilterFileSystem.java:524) ~[hadoop-common-3.3.6.jar:?]
at org.apache.hadoop.hbase.io.hfile.ReaderContext.<init>(ReaderContext.java:53) ~[hbase-server-2.5.5.jar:2.5.5]
at org.apache.hadoop.hbase.io.hfile.ReaderContextBuilder.build(ReaderContextBuilder.java:106) ~[hbase-server-2.5.5.jar:2.5.5]
```
1. Add support for Doris to parse ES date field without time zone info. eg: `2023-04-17T23:01:18.151`, this time will be treated as UTC time, since ES assumes that the time zone for time fields without time zones is UTC.
2. Change local time zone convertion from system local time zone to session variable time zone.
1. To ensure compatibility with the original optimizer, expose the non-lambda signature of highorder function externally.
2. fix some bugs in toSql function in the original optimizer
The session variable in export job should be copied from session variable in connection context.
Because both session variable in connection context and in export job may be modified at same time,
cause ConcurrentModificationException like:
2023-10-07 22:56:12,818 WARN (mysql-nio-pool-2|249) [ConnectProcessor.handleQueryException():396] Process one query failed because unknown reason:
java.util.ConcurrentModificationException: null
at java.util.HashMap$HashIterator.nextNode(HashMap.java:1437) ~[?:1.8.0_131]
at java.util.HashMap$KeyIterator.next(HashMap.java:1461) ~[?:1.8.0_131]
at org.apache.doris.qe.VariableMgr.revertSessionValue(VariableMgr.java:238) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:474) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:438) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:353) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.dispatch(ConnectProcessor.java:501) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:752) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_131]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_131]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_131]
This error is reported by external_table_p0/export/test_export_external_table.groovy
If a failure occurs, doris may retry. Due to ctx->is_read_schema is a global variable that has not been reset in a timely manner, which may cause exceptions.
---------
Co-authored-by: yiguolei <676222867@qq.com>
Issue Number: close#25023
The detail of this bug has been described at the above issue. We can check if current FE is a master node to avoid such problems.
1. Add a abstraction for column stats status which is required so furthur optimization and feature development
2. Enable analyze test in p0 that disabled unexpectedly before
This pull request addresses the behavior of the `lower_case_table_names` parameter for jdbc catalog's based on the configuration of the internal table's corresponding parameter.
Changes:
- For internal tables, if `lower_case_table_names` is set to 1 or 2, thejdbc catalog's parameter is forcefully set to `true`.
- For internal tables, if `lower_case_table_names` is set to 0, the jdbc catalog's parameter can be either `true` or `false` with a default value of `false`.
These adjustments ensure consistency and predictability when working with both internal and external table configurations in Doris.
Modify the BE fd number check to 60000,
because the default fd number value of some system is 65535, which is smaller than previous threshold 65536,
so reduce to 60000 to let Doris start normally in most of system.
