7588abe76b
this PR 1. refactor physical properties, property deriver and property regular to ensure Nereids could generate plan with sufficent PhysicalDistribute. 2. refactor PhyscialPlanTranslator to ensure all ExchangeNode generated by PhysicalDistribute, except CTEConsumer. We will refactor all cte related node later. the detail changes of this PR: 1. update DistributionSpec of physical properties: - Any: random distribution, used in output and require - StorageAny: random distribution but constrained by where the data is stored, used in output - ExecutionAny: random distribution to present random shuffle, used in output - Gather: gather distribution, used in output and require - StorageGather: gather distribution but constrained by where the data is stored, used in output - Replicated: broadcast distribution - Hash: bucket distribution 2. update shuffle type of DistributionSpecHash - REQUIRE: used in require - NATURAL: distribution as storage engine hash algorithm, constrained by where the data is stored - STORAGE_BUCKETED: distribution as storage engine hash algorithm - EXECUTION_BUCKETED: distribution as execution engine hash algorithm 3. update HideOneRowRelationUnderSetOperation to MergeOneRowRelationIntoSetOperation 4. update property deriver of SetOperation to ensure suitable PhysicalDistribute be added at top and below of SetOperation 5. refactor PhysicalPlanTranslator to ensure no unplanned exchange node will be added
# Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information # regarding copyright ownership. The ASF licenses this file # to you under the Apache License, Version 2.0 (the # "License"); you may not use this file except in compliance # with the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, # software distributed under the License is distributed on an # "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. # fe-common This module is used to store some common classes of other modules. # spark-dpp This module is Spark DPP program, used for Spark Load function. Depends: fe-common # fe-core This module is the main process module of FE. Depends: fe-common, spark-dpp