f2fd66ad3b
**Thanks for** pr #21855 to provide a wonderful reference. Maybe it is very difficult and **cost-expensive** to implement **a comprehensive logical plan adapter**, maybe there is just some small syntax variations between doris and some other engines (such as hive/spark), so we can just **focus on** the **difference** here. This pr mainly focus on the **syntax difference between doris and spark-sql**. For instance, do some function tranformations and override some syntax validations. - add a dialect named `spark_sql` - move method `NereidsParser#parseSQLWithDialect` to `TrinoParser` - extract some `FnCallTransformer`/`FnCallTransformers` classes, so we can reuse the logic about the function transformers - allow derived tables without alias when we set dialect to `spark_sql`(legacy and nereids parser are both supported) - add some function transformers for hive/spark built-in functions ### Test case (from our online doris cluster) - Test derived table without alias ```sql MySQL [(none)]> show variables like '%dialect%'; +---------------+-------+---------------+---------+ | Variable_name | Value | Default_Value | Changed | +---------------+-------+---------------+---------+ | sql_dialect | spark_sql | doris | 1 | +---------------+-------+---------------+---------+ 1 row in set (0.01 sec) MySQL [(none)]> select * from (select 1); +------+ | 1 | +------+ | 1 | +------+ 1 row in set (0.03 sec) MySQL [(none)]> select __auto_generated_subquery_name.a from (select 1 as a); +------+ | a | +------+ | 1 | +------+ 1 row in set (0.03 sec) MySQL [(none)]> set sql_dialect=doris; Query OK, 0 rows affected (0.02 sec) MySQL [(none)]> select * from (select 1); ERROR 1248 (42000): errCode = 2, detailMessage = Every derived table must have its own alias MySQL [(none)]> ``` - Test spark-sql/hive built-in functions ```sql MySQL [(none)]> show global functions; Empty set (0.01 sec) MySQL [(none)]> show variables like '%dialect%'; +---------------+-------+---------------+---------+ | Variable_name | Value | Default_Value | Changed | +---------------+-------+---------------+---------+ | sql_dialect | spark_sql | doris | 1 | +---------------+-------+---------------+---------+ 1 row in set (0.01 sec) MySQL [(none)]> select get_json_object('{"a":"b"}', '$.a'); +----------------------------------+ | json_extract('{"a":"b"}', '$.a') | +----------------------------------+ | "b" | +----------------------------------+ 1 row in set (0.04 sec) MySQL [(none)]> select split("a b c", " "); +-------------------------------+ | split_by_string('a b c', ' ') | +-------------------------------+ | ["a", "b", "c"] | +-------------------------------+ 1 row in set (1.17 sec) ```
# Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information # regarding copyright ownership. The ASF licenses this file # to you under the Apache License, Version 2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, # software distributed under the License is distributed on an # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. # fe-common This module is used to store some common classes of other modules. # spark-dpp This module is Spark DPP program, used for Spark Load function. Depends: fe-common # fe-core This module is the main process module of FE. Depends: fe-common, spark-dpp