database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity

[Enhancement](Nereids)refactor plan node into plan + operator (#9755)

Browse Source 
Close #9623 

Summary:
This pr refactor plan node into plan + operator.

In the previous version in nereids, a plan node consists of children and relational algebra, e.g.
```java
class LogicalJoin extends LogicalBinary {
  private Plan left, right;
}
```
This structure above is easy to understand, but it difficult to optimize `Memo.copyIn`: rule generate complete sub-plan,
and Memo must compare the complete sub-plan to distinct GroupExpression and hurt performance.

First, we need change the rule to generate partial sub-plan, and replace some children plan to a placeholder, e.g. LeafOp in Columbia optimizer. And then mark some children in sub-plan to unchanged, and bind the relate group, so don't have to compare and copy some sub-plan if relate group exists.

Second, we need separate the origin `Plan` into `Plan` and `Operator`, which Plan contains children and Operator, and Operator just denote relation relational algebra(no children/ input field). This design make operator and children not affect each other. So plan-group binder can generate placeholder plan(contains relate group) for the sub-query, don't have to generate current plan node case by case because the plan is immutable(means generate a new plan with replace children). And rule implementer can reuse the placeholder to generate partial sub-plan.

Operator and Plan have the similar inheritance structure like below. XxxPlan contains XxxOperator, e.g. LogicalBinary contains a LogicalBinaryOperator.
```
          TreeNode
             │
             │
     ┌───────┴────────┐                                                   Operator
     │                │                                                       │
     │                │                                                       │
     │                │                                                       │
     ▼                ▼                                                       ▼
Expression          Plan                                                PlanOperator
                      │                                                       │
                      │                                                       │
          ┌───────────┴─────────┐                                             │
          │                     │                                 ┌───────────┴──────────────────┐
          │                     │                                 │                              │
          │                     │                                 │                              │
          ▼                     ▼                                 ▼                              ▼
     LogicalPlan          PhysicalPlan                   LogicalPlanOperator           PhysicalPlanOperator
          │                     │                                 │                              │
          │                     │                                 │                              │
          │                     │                                 │                              │
          │                     │                                 │                              │
          │                     │                                 │                              │
          │                     │                                 │                              │
          ├───►LogicalLeaf      ├──►PhysicalLeaf                  ├──► LogicalLeafOperator       ├───►PhysicalLeafOperator
          │                     │                                 │                              │
          │                     │                                 │                              │
          │                     │                                 │                              │
          ├───►LogicalUnary     ├──►PhysicalUnary                 ├──► LogicalUnaryOperator      ├───►PhysicalUnaryOperator
          │                     │                                 │                              │
          │                     │                                 │                              │
          │                     │                                 │                              │
          └───►LogicalBinary    └──►PhysicalBinary                └──► LogicalBinaryOperator     └───►PhysicalBinaryOperator
```

The concrete operator extends the XxxNaryOperator, e.g.
```java
class LogicalJoin extends LogicalBinaryOperator;
class PhysicalProject extends PhysicalUnaryOperator;
class LogicalRelation extends LogicalLeafOperator;
```

So the first example change to this:
```java
class LogicalBinary extends AbstractLogicalPlan implements BinaryPlan {
  private Plan left, right;
  private LogicalBinaryOperator operator;
}

class LogicalJoin extends LogicalBinaryOperator {}
```

Under such changes, Rule must build the plan and operator as needed, not only the plan like before.
for example: JoinCommutative Rule
```java
public Rule<Plan> build() {
  // the plan override function can automatic build plan, according to the Operator's type,
  // so return a LogicalBinary(LogicalJoin, Plan, Plan)
  return innerLogicalJoin().then(join -> plan(
    // operator
    new LogicalJoin(join.op.getJoinType().swap(), join.op.getOnClause()),
    // children
    join.right(),
    join.left()
  )).toRule(RuleType.LOGICAL_JOIN_COMMUTATIVE);
}
```
This commit is contained in:
924060929
2022-05-24 20:53:24 +08:00
committed by GitHub
parent 90e8cda5f2
commit cc9321a09b
62 changed files with 1114 additions and 521 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
fe/fe-core/src/main/java/org/apache/doris/nereids/analyzer/UnboundRelation.java
View File

@ -19,9 +19,9 @@ package org.apache.doris.nereids.analyzer;
import org.apache.doris.nereids.analyzer.identifier.TableIdentifier;
import org.apache.doris.nereids.exceptions.UnboundException;
import org.apache.doris.nereids.trees.NodeType;
import org.apache.doris.nereids.operators.OperatorType;
import org.apache.doris.nereids.operators.plans.logical.LogicalLeafOperator;
import org.apache.doris.nereids.trees.expressions.Slot;
import org.apache.doris.nereids.trees.plans.logical.LogicalLeaf;
import org.apache.doris.nereids.util.Utils;
import com.clearspring.analytics.util.Lists;
@ -32,11 +32,11 @@ import java.util.List;
/**
* Represent a relation plan node that has not been bound.
*/
public class UnboundRelation extends LogicalLeaf<UnboundRelation> {
public class UnboundRelation extends LogicalLeafOperator<UnboundRelation> {
private final List<String> nameParts;
public UnboundRelation(List<String> nameParts) {
super(NodeType.LOGICAL_UNBOUND_RELATION);
super(OperatorType.LOGICAL_UNBOUND_RELATION);
this.nameParts = nameParts;
}
@ -46,7 +46,7 @@ public class UnboundRelation extends LogicalLeaf<UnboundRelation> {
* @param identifier relation identifier
*/
public UnboundRelation(TableIdentifier identifier) {
super(NodeType.LOGICAL_UNBOUND_RELATION);
super(OperatorType.LOGICAL_UNBOUND_RELATION);
this.nameParts = Lists.newArrayList();
if (identifier.getDatabaseName().isPresent()) {
nameParts.add(identifier.getDatabaseName().get());
@ -64,8 +64,9 @@ public class UnboundRelation extends LogicalLeaf<UnboundRelation> {
}
@Override
public List<Slot> getOutput() throws UnboundException {
throw new UnboundException("output");
public List<Slot> doComputeOutput() {
// fixme: throw unchecked exception
throw new IllegalStateException(new UnboundException("output"));
}
@Override

2
fe/fe-core/src/main/java/org/apache/doris/nereids/jobs/cascades/ApplyRuleJob.java
View File

@ -59,7 +59,7 @@ public class ApplyRuleJob extends Job {
// TODO: need to find all plan reference tree that match this pattern
PatternMatching patternMatching = new PatternMatching();
for (Plan<?> plan : patternMatching) {
for (Plan<?, ?> plan : patternMatching) {
if (!rule.check(plan, context)) {
continue;
}

8
fe/fe-core/src/main/java/org/apache/doris/nereids/memo/Group.java
View File

@ -18,7 +18,6 @@
package org.apache.doris.nereids.memo;
import org.apache.doris.common.Pair;
import org.apache.doris.nereids.exceptions.UnboundException;
import org.apache.doris.nereids.properties.LogicalProperties;
import org.apache.doris.nereids.properties.PhysicalProperties;
import org.apache.doris.nereids.trees.plans.logical.LogicalPlan;
@ -55,12 +54,7 @@ public class Group {
} else {
this.physicalPlanList.add(groupExpression);
}
logicalProperties = new LogicalProperties();
try {
logicalProperties.setOutput(groupExpression.getPlan().getOutput());
} catch (UnboundException e) {
throw new RuntimeException(e);
}
logicalProperties = groupExpression.getParent().getLogicalProperties();
groupExpression.setParent(this);
}

8
fe/fe-core/src/main/java/org/apache/doris/nereids/memo/GroupExpression.java
View File

@ -32,11 +32,11 @@ import java.util.List;
public class GroupExpression {
private Group parent;
private List<Group> children;
private final Plan<?> plan;
private final Plan<?, ?> plan;
private final BitSet ruleMasks;
private boolean statDerived;
public GroupExpression(Plan<?> plan) {
public GroupExpression(Plan<?, ?> plan) {
this(plan, Lists.newArrayList());
}
@ -46,7 +46,7 @@ public class GroupExpression {
* @param plan {@link Plan} to reference
* @param children children groups in memo
*/
public GroupExpression(Plan<?> plan, List<Group> children) {
public GroupExpression(Plan<?, ?> plan, List<Group> children) {
this.plan = plan;
this.children = children;
this.ruleMasks = new BitSet(RuleType.SENTINEL.ordinal());
@ -65,7 +65,7 @@ public class GroupExpression {
this.parent = parent;
}
public Plan<?> getPlan() {
public Plan<?, ?> getPlan() {
return plan;
}

4
fe/fe-core/src/main/java/org/apache/doris/nereids/memo/Memo.java
View File

@ -50,9 +50,9 @@ public class Memo {
* @return Reference of plan in Memo
*/
// TODO: need to merge PlanRefSet if new PlanRef is same with some one already in memo
public GroupExpression newGroupExpression(Plan<?> plan, Group target) {
public GroupExpression newGroupExpression(Plan<?, ?> plan, Group target) {
List<GroupExpression> childGroupExpr = Lists.newArrayList();
for (Plan<?> childrenPlan : plan.children()) {
for (Plan<?, ?> childrenPlan : plan.children()) {
childGroupExpr.add(newGroupExpression(childrenPlan, null));
}
GroupExpression newGroupExpression = new GroupExpression(plan);

36
fe/fe-core/src/main/java/org/apache/doris/nereids/operators/AbstractOperator.java Normal file
View File

@ -0,0 +1,36 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.operators;
import java.util.Objects;
/**
* Abstract class for all concrete operator.
*/
public abstract class AbstractOperator<TYPE extends AbstractOperator<TYPE>> implements Operator<TYPE> {
protected final OperatorType type;
public AbstractOperator(OperatorType type) {
this.type = Objects.requireNonNull(type, "type can not be null");
}
@Override
public OperatorType getType() {
return type;
}
}

25
fe/fe-core/src/main/java/org/apache/doris/nereids/operators/Operator.java Normal file
View File

@ -0,0 +1,25 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.operators;
/**
* interface for all concrete operator.
*/
public interface Operator<TYPE extends Operator<TYPE>> {
OperatorType getType();
}

40
fe/fe-core/src/main/java/org/apache/doris/nereids/operators/OperatorType.java Normal file
View File

@ -0,0 +1,40 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.operators;
/**
* Types for all Operator in Nereids, include PlanOperator and Pattern placeholder type.
*/
public enum OperatorType {
// logical plan
LOGICAL_UNBOUND_RELATION,
LOGICAL_BOUND_RELATION,
LOGICAL_PROJECT,
LOGICAL_FILTER,
LOGICAL_JOIN,
// physical plan
PHYSICAL_OLAP_SCAN,
PHYSICAL_PROJECT,
PHYSICAL_FILTER,
PHYSICAL_BROADCAST_HASH_JOIN,
// pattern
ANY,
MULTI,
}

29
fe/fe-core/src/main/java/org/apache/doris/nereids/operators/plans/BinaryPlanOperator.java Normal file
View File

@ -0,0 +1,29 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.operators.plans;
import org.apache.doris.nereids.trees.plans.Plan;
/**
* interface for all concrete binary plan operator.
*/
public interface BinaryPlanOperator<
TYPE extends BinaryPlanOperator<TYPE, LEFT_INPUT_TYPE, RIGHT_INPUT_TYPE>,
LEFT_INPUT_TYPE extends Plan,
RIGHT_INPUT_TYPE extends Plan> extends PlanOperator<TYPE> {
}

2
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/JoinType.java → fe/fe-core/src/main/java/org/apache/doris/nereids/operators/plans/JoinType.java
View File

@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.trees.plans;
package org.apache.doris.nereids.operators.plans;
import org.apache.doris.analysis.JoinOperator;
import org.apache.doris.common.AnalysisException;

24
fe/fe-core/src/main/java/org/apache/doris/nereids/operators/plans/LeafPlanOperator.java Normal file
View File

@ -0,0 +1,24 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.operators.plans;
/**
* interface for all concrete leaf plan operator.
*/
public interface LeafPlanOperator<TYPE extends LeafPlanOperator<TYPE>> extends PlanOperator<TYPE> {
}

26
fe/fe-core/src/main/java/org/apache/doris/nereids/operators/plans/PlanOperator.java Normal file
View File

@ -0,0 +1,26 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.operators.plans;
import org.apache.doris.nereids.operators.Operator;
/**
* interface for all concrete plan operator.
*/
public interface PlanOperator<TYPE extends PlanOperator<TYPE>> extends Operator<TYPE> {
}

28
fe/fe-core/src/main/java/org/apache/doris/nereids/operators/plans/UnaryPlanOperator.java Normal file
View File

@ -0,0 +1,28 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.operators.plans;
import org.apache.doris.nereids.trees.plans.Plan;
/**
* interface for all concrete unary plan operator.
*/
public interface UnaryPlanOperator<
TYPE extends UnaryPlanOperator<TYPE, INPUT_TYPE>,
INPUT_TYPE extends Plan> extends PlanOperator<TYPE> {
}

48
fe/fe-core/src/main/java/org/apache/doris/nereids/operators/plans/logical/LogicalBinaryOperator.java Normal file
View File

@ -0,0 +1,48 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.operators.plans.logical;
import org.apache.doris.nereids.operators.AbstractOperator;
import org.apache.doris.nereids.operators.OperatorType;
import org.apache.doris.nereids.operators.plans.BinaryPlanOperator;
import org.apache.doris.nereids.trees.expressions.Slot;
import org.apache.doris.nereids.trees.plans.Plan;
import java.util.List;
/**
* Abstract class for all logical binary operator that have two inputs.
*/
public abstract class LogicalBinaryOperator<
TYPE extends LogicalBinaryOperator<TYPE, LEFT_INPUT_TYPE, RIGHT_INPUT_TYPE>,
LEFT_INPUT_TYPE extends Plan,
RIGHT_INPUT_TYPE extends Plan>
extends AbstractOperator<TYPE>
implements LogicalOperator<TYPE>, BinaryPlanOperator<TYPE, LEFT_INPUT_TYPE, RIGHT_INPUT_TYPE> {
public LogicalBinaryOperator(OperatorType type) {
super(type);
}
@Override
public final List<Slot> computeOutput(Plan... inputs) {
return doComputeOutput((LEFT_INPUT_TYPE) inputs[0], (RIGHT_INPUT_TYPE) inputs[1]);
}
public abstract List<Slot> doComputeOutput(LEFT_INPUT_TYPE left, RIGHT_INPUT_TYPE right);
}

23
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalFilter.java → fe/fe-core/src/main/java/org/apache/doris/nereids/operators/plans/logical/LogicalFilter.java
View File

@ -15,36 +15,37 @@
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.trees.plans.logical;
package org.apache.doris.nereids.operators.plans.logical;
import org.apache.doris.nereids.exceptions.UnboundException;
import org.apache.doris.nereids.trees.NodeType;
import org.apache.doris.nereids.operators.OperatorType;
import org.apache.doris.nereids.trees.expressions.Expression;
import org.apache.doris.nereids.trees.expressions.Slot;
import org.apache.doris.nereids.trees.plans.Plan;
import java.util.List;
import java.util.Objects;
/**
* Logical filter plan node.
* Logical filter plan operator.
*/
public class LogicalFilter<CHILD_TYPE extends Plan>
extends LogicalUnary<LogicalFilter<CHILD_TYPE>, CHILD_TYPE> {
public class LogicalFilter<INPUT_TYPE extends Plan>
extends LogicalUnaryOperator<LogicalFilter<INPUT_TYPE>, INPUT_TYPE> {
private final Expression predicates;
public LogicalFilter(Expression predicates, CHILD_TYPE child) {
super(NodeType.LOGICAL_FILTER, child);
this.predicates = predicates;
public LogicalFilter(Expression predicates) {
super(OperatorType.LOGICAL_FILTER);
this.predicates = Objects.requireNonNull(predicates, "predicates can not be null");
}
public Expression getPredicates() {
return predicates;
}
@Override
public List<Slot> getOutput() throws UnboundException {
return output;
public List<Slot> doComputeOutput(INPUT_TYPE input) {
return input.getOutput();
}
@Override

76
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalJoin.java → fe/fe-core/src/main/java/org/apache/doris/nereids/operators/plans/logical/LogicalJoin.java
View File

@ -15,30 +15,30 @@
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.trees.plans.logical;
package org.apache.doris.nereids.operators.plans.logical;
import org.apache.doris.nereids.exceptions.UnboundException;
import org.apache.doris.nereids.trees.NodeType;
import org.apache.doris.nereids.operators.OperatorType;
import org.apache.doris.nereids.operators.plans.JoinType;
import org.apache.doris.nereids.rules.exploration.JoinReorderContext;
import org.apache.doris.nereids.trees.expressions.Expression;
import org.apache.doris.nereids.trees.expressions.Slot;
import org.apache.doris.nereids.trees.plans.JoinType;
import org.apache.doris.nereids.trees.plans.Plan;
import org.apache.commons.collections.CollectionUtils;
import org.apache.commons.lang3.StringUtils;
import com.google.common.collect.ImmutableList;
import java.util.List;
import java.util.Objects;
import java.util.Optional;
/**
* Logical join plan node.
* Logical join plan operator.
*/
public class LogicalJoin<
LEFT_CHILD_TYPE extends Plan,
RIGHT_CHILD_TYPE extends Plan>
extends LogicalBinary<LogicalJoin<LEFT_CHILD_TYPE, RIGHT_CHILD_TYPE>, LEFT_CHILD_TYPE, RIGHT_CHILD_TYPE> {
public class LogicalJoin<LEFT_INPUT_TYPE extends Plan, RIGHT_INPUT_TYPE extends Plan>
extends LogicalBinaryOperator<LogicalJoin<LEFT_INPUT_TYPE, RIGHT_INPUT_TYPE>,
LEFT_INPUT_TYPE, RIGHT_INPUT_TYPE> {
private final JoinType joinType;
private final Expression onClause;
private final Optional<Expression> onClause;
// Use for top-to-down join reorder
private final JoinReorderContext joinReorderContext = new JoinReorderContext();
@ -47,17 +47,24 @@ public class LogicalJoin<
* Constructor for LogicalJoinPlan.
*
* @param joinType logical type for join
* @param onClause on clause for join node
* @param left left child of join node
* @param right right child of join node
*/
public LogicalJoin(JoinType joinType, Expression onClause, LEFT_CHILD_TYPE left, RIGHT_CHILD_TYPE right) {
super(NodeType.LOGICAL_JOIN, left, right);
this.joinType = joinType;
this.onClause = onClause;
public LogicalJoin(JoinType joinType) {
this(joinType, Optional.empty());
}
public Expression getOnClause() {
/**
* Constructor for LogicalJoinPlan.
*
* @param joinType logical type for join
* @param onClause on clause for join node
*/
public LogicalJoin(JoinType joinType, Optional<Expression> onClause) {
super(OperatorType.LOGICAL_JOIN);
this.joinType = Objects.requireNonNull(joinType, "joinType can not be null");
this.onClause = Objects.requireNonNull(onClause, "onClause can not be null");
}
public Optional<Expression> getOnClause() {
return onClause;
}
@ -66,22 +73,18 @@ public class LogicalJoin<
}
@Override
public List<Slot> getOutput() throws UnboundException {
if (CollectionUtils.isEmpty(output)) {
switch (joinType) {
case LEFT_SEMI_JOIN:
output.addAll(left().getOutput());
break;
case RIGHT_SEMI_JOIN:
output.addAll(right().getOutput());
break;
default:
output.addAll(left().getOutput());
output.addAll(right().getOutput());
break;
}
public List<Slot> doComputeOutput(LEFT_INPUT_TYPE leftInput, RIGHT_INPUT_TYPE rightInput) {
switch (joinType) {
case LEFT_SEMI_JOIN:
return ImmutableList.copyOf(leftInput.getOutput());
case RIGHT_SEMI_JOIN:
return ImmutableList.copyOf(rightInput.getOutput());
default:
return ImmutableList.<Slot>builder()
.addAll(leftInput.getOutput())
.addAll(rightInput.getOutput())
.build();
}
return output;
}
@Override
@ -90,9 +93,6 @@ public class LogicalJoin<
if (onClause != null) {
sb.append(", ").append(onClause);
}
if (CollectionUtils.isNotEmpty(output)) {
sb.append(", output: ").append(StringUtils.join(output, ", "));
}
return sb.append(")").toString();
}

45
fe/fe-core/src/main/java/org/apache/doris/nereids/operators/plans/logical/LogicalLeafOperator.java Normal file
View File

@ -0,0 +1,45 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.operators.plans.logical;
import org.apache.doris.nereids.operators.AbstractOperator;
import org.apache.doris.nereids.operators.OperatorType;
import org.apache.doris.nereids.operators.plans.LeafPlanOperator;
import org.apache.doris.nereids.trees.expressions.Slot;
import org.apache.doris.nereids.trees.plans.Plan;
import java.util.List;
/**
* Abstract class for all logical operator that have no input.
*/
public abstract class LogicalLeafOperator<TYPE extends LogicalLeafOperator<TYPE>>
extends AbstractOperator<TYPE>
implements LogicalOperator<TYPE>, LeafPlanOperator<TYPE> {
public LogicalLeafOperator(OperatorType type) {
super(type);
}
@Override
public final List<Slot> computeOutput(Plan... inputs) {
return doComputeOutput();
}
public abstract List<Slot> doComputeOutput();
}

31
fe/fe-core/src/main/java/org/apache/doris/nereids/operators/plans/logical/LogicalOperator.java Normal file
View File

@ -0,0 +1,31 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.operators.plans.logical;
import org.apache.doris.nereids.operators.plans.PlanOperator;
import org.apache.doris.nereids.trees.expressions.Slot;
import org.apache.doris.nereids.trees.plans.Plan;
import java.util.List;
/**
* interface for all concrete logical plan operator.
*/
public interface LogicalOperator<TYPE extends LogicalOperator<TYPE>> extends PlanOperator<TYPE> {
List<Slot> computeOutput(Plan... inputs);
}

48
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalProject.java → fe/fe-core/src/main/java/org/apache/doris/nereids/operators/plans/logical/LogicalProject.java
View File

@ -15,37 +15,36 @@
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.trees.plans.logical;
package org.apache.doris.nereids.operators.plans.logical;
import org.apache.doris.nereids.exceptions.UnboundException;
import org.apache.doris.nereids.trees.NodeType;
import org.apache.doris.nereids.operators.OperatorType;
import org.apache.doris.nereids.trees.expressions.NamedExpression;
import org.apache.doris.nereids.trees.expressions.Slot;
import org.apache.doris.nereids.trees.plans.Plan;
import com.google.common.collect.Lists;
import com.google.common.collect.ImmutableList;
import org.apache.commons.lang3.StringUtils;
import java.util.List;
import java.util.Objects;
/**
* Logical project plan node.
* Logical project plan operator.
*/
public class LogicalProject<CHILD_TYPE extends Plan>
extends LogicalUnary<LogicalProject<CHILD_TYPE>, CHILD_TYPE> {
public class LogicalProject<INPUT_TYPE extends Plan>
extends LogicalUnaryOperator<LogicalProject<INPUT_TYPE>, INPUT_TYPE> {
private final List<? extends NamedExpression> projects;
/**
* Constructor for LogicalProjectPlan.
* Constructor for LogicalProject.
*
* @param projects project list
* @param child child plan node
*/
public LogicalProject(List<? extends NamedExpression> projects, CHILD_TYPE child) {
super(NodeType.LOGICAL_PROJECT, child);
this.projects = projects;
updateOutput();
public LogicalProject(List<? extends NamedExpression> projects) {
super(OperatorType.LOGICAL_PROJECT);
this.projects = Objects.requireNonNull(projects, "projects can not be null");
}
/**
@ -58,20 +57,17 @@ public class LogicalProject<CHILD_TYPE extends Plan>
}
@Override
public List<Slot> getOutput() {
return output;
}
private void updateOutput() {
output = Lists.newArrayListWithCapacity(projects.size());
for (NamedExpression projection : projects) {
try {
output.add(projection.toSlot());
} catch (UnboundException e) {
output.clear();
break;
}
}
public List<Slot> doComputeOutput(INPUT_TYPE input) {
// fixme: not throw a checked exception
return projects.stream()
.map(namedExpr -> {
try {
return namedExpr.toSlot();
} catch (UnboundException e) {
throw new IllegalStateException(e);
}
})
.collect(ImmutableList.toImmutableList());
}
@Override

34
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalRelation.java → fe/fe-core/src/main/java/org/apache/doris/nereids/operators/plans/logical/LogicalRelation.java
View File

@ -15,22 +15,23 @@
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.trees.plans.logical;
package org.apache.doris.nereids.operators.plans.logical;
import org.apache.doris.catalog.Table;
import org.apache.doris.nereids.trees.NodeType;
import org.apache.doris.nereids.operators.OperatorType;
import org.apache.doris.nereids.trees.expressions.Slot;
import org.apache.doris.nereids.trees.expressions.SlotReference;
import com.google.common.collect.ImmutableList;
import org.apache.commons.lang3.StringUtils;
import java.util.List;
import java.util.stream.Collectors;
import java.util.Objects;
/**
* Logical relation plan node.
* Logical relation plan operator.
*/
public class LogicalRelation extends LogicalLeaf<LogicalRelation> {
public class LogicalRelation extends LogicalLeafOperator<LogicalRelation> {
private final Table table;
private final List<String> qualifier;
@ -42,13 +43,9 @@ public class LogicalRelation extends LogicalLeaf<LogicalRelation> {
* @param qualifier qualified relation name
*/
public LogicalRelation(Table table, List<String> qualifier) {
super(NodeType.LOGICAL_BOUND_RELATION);
this.table = table;
this.qualifier = qualifier;
this.output = table.getBaseSchema()
.stream()
.map(col -> SlotReference.fromColumn(col, qualifier))
.collect(Collectors.toList());
super(OperatorType.LOGICAL_BOUND_RELATION);
this.table = Objects.requireNonNull(table, "table can not be null");
this.qualifier = Objects.requireNonNull(qualifier, "qualifier can not be null");
}
public Table getTable() {
@ -60,14 +57,15 @@ public class LogicalRelation extends LogicalLeaf<LogicalRelation> {
}
@Override
public List<Slot> getOutput() {
return output;
public String toString() {
return "Relation(" + StringUtils.join(qualifier, ".") + "." + table.getName() + ")";
}
@Override
public String toString() {
return "Relation(" + StringUtils.join(qualifier, ".") + "." + table.getName()
+ ", output: " + StringUtils.join(output, ", ")
+ ")";
public List<Slot> doComputeOutput() {
return table.getBaseSchema()
.stream()
.map(col -> SlotReference.fromColumn(col, qualifier))
.collect(ImmutableList.toImmutableList());
}
}

47
fe/fe-core/src/main/java/org/apache/doris/nereids/operators/plans/logical/LogicalUnaryOperator.java Normal file
View File

@ -0,0 +1,47 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.operators.plans.logical;
import org.apache.doris.nereids.operators.AbstractOperator;
import org.apache.doris.nereids.operators.OperatorType;
import org.apache.doris.nereids.operators.plans.UnaryPlanOperator;
import org.apache.doris.nereids.trees.expressions.Slot;
import org.apache.doris.nereids.trees.plans.Plan;
import java.util.List;
/**
* Abstract class for all logical operator that have one input.
*/
public abstract class LogicalUnaryOperator<
TYPE extends LogicalUnaryOperator<TYPE, INPUT_TYPE>,
INPUT_TYPE extends Plan>
extends AbstractOperator<TYPE>
implements LogicalOperator<TYPE>, UnaryPlanOperator<TYPE, INPUT_TYPE> {
public LogicalUnaryOperator(OperatorType type) {
super(type);
}
@Override
public final List<Slot> computeOutput(Plan... inputs) {
return doComputeOutput((INPUT_TYPE) inputs[0]);
}
public abstract List<Slot> doComputeOutput(INPUT_TYPE input);
}

52
fe/fe-core/src/main/java/org/apache/doris/nereids/operators/plans/physical/PhysicalBinaryOperator.java Normal file
View File

@ -0,0 +1,52 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.operators.plans.physical;
import org.apache.doris.nereids.operators.AbstractOperator;
import org.apache.doris.nereids.operators.OperatorType;
import org.apache.doris.nereids.operators.plans.BinaryPlanOperator;
import org.apache.doris.nereids.properties.LogicalProperties;
import org.apache.doris.nereids.trees.expressions.Slot;
import org.apache.doris.nereids.trees.plans.Plan;
import java.util.List;
/**
* Abstract class for all physical operator that have two inputs.
*/
public abstract class PhysicalBinaryOperator<
TYPE extends PhysicalBinaryOperator<TYPE, LEFT_INPUT_TYPE, RIGHT_INPUT_TYPE>,
LEFT_INPUT_TYPE extends Plan,
RIGHT_INPUT_TYPE extends Plan>
extends AbstractOperator<TYPE>
implements PhysicalOperator<TYPE>, BinaryPlanOperator<TYPE, LEFT_INPUT_TYPE, RIGHT_INPUT_TYPE> {
public PhysicalBinaryOperator(OperatorType type) {
super(type);
}
@Override
public final List<Slot> computeOutputs(LogicalProperties logicalProperties, Plan... inputs) {
return doComputeOutput(logicalProperties, (LEFT_INPUT_TYPE) inputs[0], (RIGHT_INPUT_TYPE) inputs[1]);
}
public List<Slot> doComputeOutput(LogicalProperties logicalProperties,
LEFT_INPUT_TYPE left, RIGHT_INPUT_TYPE right) {
return logicalProperties.getOutput();
}
}

44
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/PhysicalBroadcastHashJoin.java → fe/fe-core/src/main/java/org/apache/doris/nereids/operators/plans/physical/PhysicalBroadcastHashJoin.java
View File

@ -15,28 +15,34 @@
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.trees.plans.physical;
package org.apache.doris.nereids.operators.plans.physical;
import org.apache.doris.nereids.trees.NodeType;
import org.apache.doris.nereids.operators.OperatorType;
import org.apache.doris.nereids.operators.plans.JoinType;
import org.apache.doris.nereids.trees.expressions.Expression;
import org.apache.doris.nereids.trees.plans.JoinType;
import org.apache.doris.nereids.trees.plans.Plan;
import org.apache.commons.lang3.StringUtils;
import java.util.Objects;
import java.util.Optional;
/**
* Physical node represents broadcast hash join.
* Physical operator represents broadcast hash join.
*/
public class PhysicalBroadcastHashJoin<
LEFT_CHILD_TYPE extends Plan,
RIGHT_CHILD_TYPE extends Plan>
extends PhysicalBinary<PhysicalBroadcastHashJoin<LEFT_CHILD_TYPE, RIGHT_CHILD_TYPE>,
LEFT_CHILD_TYPE, RIGHT_CHILD_TYPE> {
public class PhysicalBroadcastHashJoin<LEFT_INPUT_TYPE extends Plan, RIGHT_INPUT_TYPE extends Plan>
extends PhysicalBinaryOperator<PhysicalBroadcastHashJoin<LEFT_INPUT_TYPE, RIGHT_INPUT_TYPE>,
LEFT_INPUT_TYPE, RIGHT_INPUT_TYPE> {
private final JoinType joinType;
private final Expression onClause;
private final Optional<Expression> onClause;
/**
* Constructor for PhysicalBroadcastHashJoin.
*
* @param joinType logical join type in Nereids
*/
public PhysicalBroadcastHashJoin(JoinType joinType) {
this(joinType, Optional.empty());
}
/**
* Constructor for PhysicalBroadcastHashJoin.
@ -44,11 +50,10 @@ public class PhysicalBroadcastHashJoin<
* @param joinType logical join type in Nereids
* @param onClause on clause expression
*/
public PhysicalBroadcastHashJoin(JoinType joinType, LEFT_CHILD_TYPE leftChild, RIGHT_CHILD_TYPE rightChild,
Expression onClause) {
super(NodeType.PHYSICAL_BROADCAST_HASH_JOIN, leftChild, rightChild);
this.joinType = joinType;
this.onClause = onClause;
public PhysicalBroadcastHashJoin(JoinType joinType, Optional<Expression> onClause) {
super(OperatorType.PHYSICAL_BROADCAST_HASH_JOIN);
this.joinType = Objects.requireNonNull(joinType, "joinType can not be null");
this.onClause = Objects.requireNonNull(onClause, "onClause can not be null");
}
public JoinType getJoinType() {
@ -56,7 +61,7 @@ public class PhysicalBroadcastHashJoin<
}
public Optional<Expression> getOnClause() {
return Optional.ofNullable(onClause);
return onClause;
}
@Override
@ -65,9 +70,6 @@ public class PhysicalBroadcastHashJoin<
if (onClause != null) {
sb.append(", ").append(onClause);
}
if (logicalProperties != null && !logicalProperties.getOutput().isEmpty()) {
sb.append(", output: ").append(StringUtils.join(logicalProperties.getOutput(), ", "));
}
return sb.append(")").toString();
}
}

18
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/PhysicalFilter.java → fe/fe-core/src/main/java/org/apache/doris/nereids/operators/plans/physical/PhysicalFilter.java
View File

@ -15,23 +15,25 @@
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.trees.plans.physical;
package org.apache.doris.nereids.operators.plans.physical;
import org.apache.doris.nereids.trees.NodeType;
import org.apache.doris.nereids.operators.OperatorType;
import org.apache.doris.nereids.trees.expressions.Expression;
import org.apache.doris.nereids.trees.plans.Plan;
import java.util.Objects;
/**
* Physical filter plan node.
* Physical filter plan operator.
*/
public class PhysicalFilter<CHILD_TYPE extends Plan>
extends PhysicalUnary<PhysicalFilter<CHILD_TYPE>, CHILD_TYPE> {
public class PhysicalFilter<INPUT_TYPE extends Plan>
extends PhysicalUnaryOperator<PhysicalFilter<INPUT_TYPE>, INPUT_TYPE> {
private final Expression predicates;
public PhysicalFilter(Expression predicates, CHILD_TYPE child) {
super(NodeType.PHYSICAL_FILTER, child);
this.predicates = predicates;
public PhysicalFilter(Expression predicates) {
super(OperatorType.PHYSICAL_FILTER);
this.predicates = Objects.requireNonNull(predicates, "predicates can not be null");
}
public Expression getPredicates() {

48
fe/fe-core/src/main/java/org/apache/doris/nereids/operators/plans/physical/PhysicalLeafOperator.java Normal file
View File

@ -0,0 +1,48 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.operators.plans.physical;
import org.apache.doris.nereids.operators.AbstractOperator;
import org.apache.doris.nereids.operators.OperatorType;
import org.apache.doris.nereids.operators.plans.LeafPlanOperator;
import org.apache.doris.nereids.properties.LogicalProperties;
import org.apache.doris.nereids.trees.expressions.Slot;
import org.apache.doris.nereids.trees.plans.Plan;
import java.util.List;
/**
* Abstract class for all physical operator that have no input.
*/
public abstract class PhysicalLeafOperator<TYPE extends PhysicalLeafOperator<TYPE>>
extends AbstractOperator<TYPE>
implements PhysicalOperator<TYPE>, LeafPlanOperator<TYPE> {
public PhysicalLeafOperator(OperatorType type) {
super(type);
}
@Override
public final List<Slot> computeOutputs(LogicalProperties logicalProperties, Plan... inputs) {
return doComputeOutput(logicalProperties);
}
public List<Slot> doComputeOutput(LogicalProperties logicalProperties) {
return logicalProperties.getOutput();
}
}

11
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/PhysicalOlapScan.java → fe/fe-core/src/main/java/org/apache/doris/nereids/operators/plans/physical/PhysicalOlapScan.java
View File

@ -15,11 +15,11 @@
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.trees.plans.physical;
package org.apache.doris.nereids.operators.plans.physical;
import org.apache.doris.catalog.OlapTable;
import org.apache.doris.catalog.Partition;
import org.apache.doris.nereids.trees.NodeType;
import org.apache.doris.nereids.operators.OperatorType;
import com.clearspring.analytics.util.Lists;
import org.apache.commons.lang3.StringUtils;
@ -27,7 +27,7 @@ import org.apache.commons.lang3.StringUtils;
import java.util.List;
/**
* Physical olap scan plan node.
* Physical olap scan plan operator.
*/
public class PhysicalOlapScan extends PhysicalScan<PhysicalOlapScan> {
private final long selectedIndexId;
@ -41,7 +41,7 @@ public class PhysicalOlapScan extends PhysicalScan<PhysicalOlapScan> {
* @param qualifier table's name
*/
public PhysicalOlapScan(OlapTable olapTable, List<String> qualifier) {
super(NodeType.PHYSICAL_OLAP_SCAN, olapTable, qualifier);
super(OperatorType.PHYSICAL_OLAP_SCAN, olapTable, qualifier);
this.selectedIndexId = olapTable.getBaseIndexId();
this.selectedTabletId = Lists.newArrayList();
this.selectedPartitionId = olapTable.getPartitionIds();
@ -65,8 +65,7 @@ public class PhysicalOlapScan extends PhysicalScan<PhysicalOlapScan> {
@Override
public String toString() {
return "Scan Olap Table " + StringUtils.join(qualifier, ".") + "." + table.getName()
+ " (output: " + logicalProperties.getOutput()
+ ", selected index id: " + selectedTabletId
+ " (selected index id: " + selectedTabletId
+ ", selected partition ids: " + selectedPartitionId
+ ", selected tablet ids: " + selectedTabletId
+ ")";

32
fe/fe-core/src/main/java/org/apache/doris/nereids/operators/plans/physical/PhysicalOperator.java Normal file
View File

@ -0,0 +1,32 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.operators.plans.physical;
import org.apache.doris.nereids.operators.plans.PlanOperator;
import org.apache.doris.nereids.properties.LogicalProperties;
import org.apache.doris.nereids.trees.expressions.Slot;
import org.apache.doris.nereids.trees.plans.Plan;
import java.util.List;
/**
* interface for all concrete physical operator.
*/
public interface PhysicalOperator<TYPE extends PhysicalOperator<TYPE>> extends PlanOperator<TYPE> {
List<Slot> computeOutputs(LogicalProperties logicalProperties, Plan... inputs);
}

17
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/PhysicalProject.java → fe/fe-core/src/main/java/org/apache/doris/nereids/operators/plans/physical/PhysicalProject.java
View File

@ -15,27 +15,28 @@
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.trees.plans.physical;
package org.apache.doris.nereids.operators.plans.physical;
import org.apache.doris.nereids.trees.NodeType;
import org.apache.doris.nereids.operators.OperatorType;
import org.apache.doris.nereids.trees.expressions.NamedExpression;
import org.apache.doris.nereids.trees.plans.Plan;
import org.apache.commons.lang3.StringUtils;
import java.util.List;
import java.util.Objects;
/**
* Physical project plan node.
* Physical project plan operator.
*/
public class PhysicalProject<CHILD_TYPE extends Plan>
extends PhysicalUnary<PhysicalProject<CHILD_TYPE>, CHILD_TYPE> {
public class PhysicalProject<INPUT_TYPE extends Plan>
extends PhysicalUnaryOperator<PhysicalProject<INPUT_TYPE>, INPUT_TYPE> {
private final List<? extends NamedExpression> projects;
public PhysicalProject(List<? extends NamedExpression> projects, CHILD_TYPE child) {
super(NodeType.PHYSICAL_PROJECT, child);
this.projects = projects;
public PhysicalProject(List<? extends NamedExpression> projects) {
super(OperatorType.PHYSICAL_PROJECT);
this.projects = Objects.requireNonNull(projects, "projects can not be null");
}
public List<? extends NamedExpression> getProjects() {

17
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/PhysicalScan.java → fe/fe-core/src/main/java/org/apache/doris/nereids/operators/plans/physical/PhysicalScan.java
View File

@ -15,18 +15,19 @@
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.trees.plans.physical;
package org.apache.doris.nereids.operators.plans.physical;
import org.apache.doris.catalog.Table;
import org.apache.doris.nereids.trees.NodeType;
import org.apache.doris.nereids.operators.OperatorType;
import java.util.List;
import java.util.Objects;
/**
* Abstract class for all physical scan node.
* Abstract class for all physical scan operator.
*/
public abstract class PhysicalScan<PLAN_TYPE extends PhysicalScan<PLAN_TYPE>>
extends PhysicalLeaf<PLAN_TYPE> {
public abstract class PhysicalScan<TYPE extends PhysicalScan<TYPE>>
extends PhysicalLeafOperator<TYPE> {
protected final Table table;
protected final List<String> qualifier;
@ -38,10 +39,10 @@ public abstract class PhysicalScan<PLAN_TYPE extends PhysicalScan<PLAN_TYPE>>
* @param table scan table
* @param qualifier table's name
*/
public PhysicalScan(NodeType type, Table table, List<String> qualifier) {
public PhysicalScan(OperatorType type, Table table, List<String> qualifier) {
super(type);
this.table = table;
this.qualifier = qualifier;
this.table = Objects.requireNonNull(table, "table can not be null");
this.qualifier = Objects.requireNonNull(qualifier, "qualifier can not be null");
}
public Table getTable() {

50
fe/fe-core/src/main/java/org/apache/doris/nereids/operators/plans/physical/PhysicalUnaryOperator.java Normal file
View File

@ -0,0 +1,50 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.operators.plans.physical;
import org.apache.doris.nereids.operators.AbstractOperator;
import org.apache.doris.nereids.operators.OperatorType;
import org.apache.doris.nereids.operators.plans.UnaryPlanOperator;
import org.apache.doris.nereids.properties.LogicalProperties;
import org.apache.doris.nereids.trees.expressions.Slot;
import org.apache.doris.nereids.trees.plans.Plan;
import java.util.List;
/**
* Abstract class for all physical operator that have one input.
*/
public abstract class PhysicalUnaryOperator<
TYPE extends PhysicalUnaryOperator<TYPE, INPUT_TYPE>,
INPUT_TYPE extends Plan>
extends AbstractOperator<TYPE>
implements PhysicalOperator<TYPE>, UnaryPlanOperator<TYPE, INPUT_TYPE> {
public PhysicalUnaryOperator(OperatorType type) {
super(type);
}
@Override
public final List<Slot> computeOutputs(LogicalProperties logicalProperties, Plan... inputs) {
return doComputeOutput(logicalProperties, (INPUT_TYPE) inputs[0]);
}
public List<Slot> doComputeOutput(LogicalProperties logicalProperties, INPUT_TYPE input) {
return logicalProperties.getOutput();
}
}

25
fe/fe-core/src/main/java/org/apache/doris/nereids/parser/LogicalPlanBuilder.java
View File

@ -49,6 +49,10 @@ import org.apache.doris.nereids.analyzer.UnboundAlias;
import org.apache.doris.nereids.analyzer.UnboundRelation;
import org.apache.doris.nereids.analyzer.UnboundSlot;
import org.apache.doris.nereids.analyzer.UnboundStar;
import org.apache.doris.nereids.operators.plans.JoinType;
import org.apache.doris.nereids.operators.plans.logical.LogicalFilter;
import org.apache.doris.nereids.operators.plans.logical.LogicalJoin;
import org.apache.doris.nereids.operators.plans.logical.LogicalProject;
import org.apache.doris.nereids.trees.expressions.Alias;
import org.apache.doris.nereids.trees.expressions.EqualTo;
import org.apache.doris.nereids.trees.expressions.Expression;
@ -60,11 +64,10 @@ import org.apache.doris.nereids.trees.expressions.Literal;
import org.apache.doris.nereids.trees.expressions.NamedExpression;
import org.apache.doris.nereids.trees.expressions.Not;
import org.apache.doris.nereids.trees.expressions.NullSafeEqual;
import org.apache.doris.nereids.trees.plans.JoinType;
import org.apache.doris.nereids.trees.plans.logical.LogicalFilter;
import org.apache.doris.nereids.trees.plans.logical.LogicalJoin;
import org.apache.doris.nereids.trees.plans.logical.LogicalBinary;
import org.apache.doris.nereids.trees.plans.logical.LogicalLeaf;
import org.apache.doris.nereids.trees.plans.logical.LogicalPlan;
import org.apache.doris.nereids.trees.plans.logical.LogicalProject;
import org.apache.doris.nereids.trees.plans.logical.LogicalUnary;
import com.google.common.collect.Lists;
import org.antlr.v4.runtime.ParserRuleContext;
@ -75,6 +78,7 @@ import org.antlr.v4.runtime.tree.TerminalNode;
import org.apache.commons.collections.CollectionUtils;
import java.util.List;
import java.util.Optional;
import java.util.function.BiFunction;
import java.util.function.Supplier;
import java.util.stream.Collectors;
@ -89,7 +93,7 @@ public class LogicalPlanBuilder extends DorisParserBaseVisitor<Object> {
*/
private final BiFunction<WhereClauseContext, LogicalPlan, LogicalPlan> withWhereClause =
(WhereClauseContext ctx, LogicalPlan plan)
-> new LogicalFilter(expression((ctx.booleanExpression())), plan);
-> new LogicalUnary(new LogicalFilter(expression((ctx.booleanExpression()))), plan);
protected <T> T typedVisit(ParseTree ctx) {
return (T) ctx.accept(this);
@ -201,7 +205,7 @@ public class LogicalPlanBuilder extends DorisParserBaseVisitor<Object> {
LogicalPlan withProject;
if (CollectionUtils.isNotEmpty(namedExpressions)) {
withProject = new LogicalProject(namedExpressions, withFilter);
withProject = new LogicalUnary(new LogicalProject(namedExpressions), withFilter);
} else {
withProject = withFilter;
}
@ -217,7 +221,7 @@ public class LogicalPlanBuilder extends DorisParserBaseVisitor<Object> {
if (left == null) {
left = right;
} else {
left = new LogicalJoin(JoinType.INNER_JOIN, null, left, right);
left = new LogicalBinary(new LogicalJoin(JoinType.INNER_JOIN, null), left, right);
}
left = withJoinRelations(left, relation);
}
@ -257,7 +261,10 @@ public class LogicalPlanBuilder extends DorisParserBaseVisitor<Object> {
condition = expression(joinCriteria.booleanExpression());
}
last = new LogicalJoin(joinType, condition, last, plan(join.relationPrimary()));
last = new LogicalBinary(
new LogicalJoin(joinType, Optional.ofNullable(condition)),
last, plan(join.relationPrimary())
);
}
return last;
}
@ -270,7 +277,7 @@ public class LogicalPlanBuilder extends DorisParserBaseVisitor<Object> {
List<String> tableId = visitMultipartIdentifier(ctx.multipartIdentifier());
UnboundRelation relation = new UnboundRelation(tableId);
// TODO: sample and time travel, alias, sub query
return relation;
return new LogicalLeaf(relation);
}
/**

33
fe/fe-core/src/main/java/org/apache/doris/nereids/pattern/Pattern.java
View File

@ -17,6 +17,7 @@
package org.apache.doris.nereids.pattern;
import org.apache.doris.nereids.operators.OperatorType;
import org.apache.doris.nereids.trees.AbstractTreeNode;
import org.apache.doris.nereids.trees.NodeType;
import org.apache.doris.nereids.trees.TreeNode;
@ -32,34 +33,34 @@ import java.util.function.Predicate;
* Pattern node used in pattern matching.
*/
public class Pattern<T extends TreeNode> extends AbstractTreeNode<Pattern<T>> {
public static final Pattern ANY = new Pattern(NodeType.ANY);
public static final Pattern MULTI = new Pattern(NodeType.MULTI);
public static final Pattern ANY = new Pattern(OperatorType.ANY);
public static final Pattern MULTI = new Pattern(OperatorType.MULTI);
public final List<Predicate<T>> predicates;
private final NodeType nodeType;
private final List<Predicate<T>> predicates;
private final OperatorType operatorType;
/**
* Constructor for Pattern.
*
* @param nodeType node type to matching
* @param operatorType operator type to matching
* @param children sub pattern
*/
public Pattern(NodeType nodeType, Pattern... children) {
public Pattern(OperatorType operatorType, Pattern... children) {
super(NodeType.PATTERN, children);
this.nodeType = nodeType;
this.operatorType = operatorType;
this.predicates = ImmutableList.of();
}
/**
* Constructor for Pattern.
*
* @param nodeType node type to matching
* @param operatorType operator type to matching
* @param predicates custom matching predicate
* @param children sub pattern
*/
public Pattern(NodeType nodeType, List<Predicate<T>> predicates, Pattern... children) {
public Pattern(OperatorType operatorType, List<Predicate<T>> predicates, Pattern... children) {
super(NodeType.PATTERN, children);
this.nodeType = nodeType;
this.operatorType = operatorType;
this.predicates = ImmutableList.copyOf(predicates);
}
@ -68,8 +69,8 @@ public class Pattern<T extends TreeNode> extends AbstractTreeNode<Pattern<T>> {
*
* @return node type in pattern
*/
public NodeType getNodeType() {
return nodeType;
public OperatorType getOperatorType() {
return operatorType;
}
/**
@ -87,11 +88,11 @@ public class Pattern<T extends TreeNode> extends AbstractTreeNode<Pattern<T>> {
return false;
}
if (nodeType == NodeType.MULTI || nodeType == NodeType.ANY) {
if (operatorType == OperatorType.MULTI || operatorType == OperatorType.ANY) {
return true;
}
return getNodeType().equals(root.getType())
return getOperatorType().equals(root.getType())
&& predicates.stream().allMatch(predicate -> predicate.test(root));
}
@ -129,12 +130,12 @@ public class Pattern<T extends TreeNode> extends AbstractTreeNode<Pattern<T>> {
return false;
}
Pattern pattern = (Pattern) o;
return nodeType == pattern.nodeType;
return operatorType == pattern.operatorType;
}
@Override
public int hashCode() {
return Objects.hash(nodeType);
return Objects.hash(operatorType);
}
@Override

2
fe/fe-core/src/main/java/org/apache/doris/nereids/pattern/PatternDescriptor.java
View File

@ -57,6 +57,6 @@ public class PatternDescriptor<INPUT_TYPE extends RULE_TYPE, RULE_TYPE extends T
public Pattern<INPUT_TYPE> patternWithPredicates() {
Pattern[] children = pattern.children().toArray(new Pattern[0]);
return new Pattern<>(pattern.getNodeType(), predicates, children);
return new Pattern<>(pattern.getOperatorType(), predicates, children);
}
}

8
fe/fe-core/src/main/java/org/apache/doris/nereids/pattern/PatternMatching.java
View File

@ -24,24 +24,24 @@ import java.util.Iterator;
/**
* Get all pattern matching subtree in query plan.
*/
public class PatternMatching implements Iterable<Plan<?>> {
public class PatternMatching implements Iterable<Plan<?, ?>> {
@Override
public Iterator<Plan<?>> iterator() {
public Iterator<Plan<?, ?>> iterator() {
return new PatternMatchingIterator();
}
/**
* Iterator to get all subtrees.
*/
public static class PatternMatchingIterator implements Iterator<Plan<?>> {
public static class PatternMatchingIterator implements Iterator<Plan<?, ?>> {
@Override
public boolean hasNext() {
return false;
}
@Override
public Plan<?> next() {
public Plan<?, ?> next() {
return null;
}
}

252
fe/fe-core/src/main/java/org/apache/doris/nereids/pattern/Patterns.java
View File

@ -17,28 +17,26 @@
package org.apache.doris.nereids.pattern;
import org.apache.doris.nereids.analyzer.UnboundAlias;
import org.apache.doris.nereids.analyzer.UnboundRelation;
import org.apache.doris.nereids.analyzer.UnboundSlot;
import org.apache.doris.nereids.analyzer.UnboundStar;
import org.apache.doris.nereids.operators.OperatorType;
import org.apache.doris.nereids.operators.plans.JoinType;
import org.apache.doris.nereids.operators.plans.logical.LogicalFilter;
import org.apache.doris.nereids.operators.plans.logical.LogicalJoin;
import org.apache.doris.nereids.operators.plans.logical.LogicalProject;
import org.apache.doris.nereids.operators.plans.logical.LogicalRelation;
import org.apache.doris.nereids.operators.plans.physical.PhysicalBroadcastHashJoin;
import org.apache.doris.nereids.operators.plans.physical.PhysicalFilter;
import org.apache.doris.nereids.operators.plans.physical.PhysicalOlapScan;
import org.apache.doris.nereids.operators.plans.physical.PhysicalProject;
import org.apache.doris.nereids.rules.RulePromise;
import org.apache.doris.nereids.trees.NodeType;
import org.apache.doris.nereids.trees.TreeNode;
import org.apache.doris.nereids.trees.expressions.Alias;
import org.apache.doris.nereids.trees.expressions.EqualTo;
import org.apache.doris.nereids.trees.expressions.Expression;
import org.apache.doris.nereids.trees.expressions.Literal;
import org.apache.doris.nereids.trees.expressions.SlotReference;
import org.apache.doris.nereids.trees.plans.JoinType;
import org.apache.doris.nereids.trees.plans.Plan;
import org.apache.doris.nereids.trees.plans.logical.LogicalFilter;
import org.apache.doris.nereids.trees.plans.logical.LogicalJoin;
import org.apache.doris.nereids.trees.plans.logical.LogicalProject;
import org.apache.doris.nereids.trees.plans.logical.LogicalRelation;
import org.apache.doris.nereids.trees.plans.physical.PhysicalBroadcastHashJoin;
import org.apache.doris.nereids.trees.plans.physical.PhysicalFilter;
import org.apache.doris.nereids.trees.plans.physical.PhysicalOlapScan;
import org.apache.doris.nereids.trees.plans.physical.PhysicalProject;
import org.apache.doris.nereids.trees.plans.logical.LogicalBinary;
import org.apache.doris.nereids.trees.plans.logical.LogicalLeaf;
import org.apache.doris.nereids.trees.plans.logical.LogicalUnary;
import org.apache.doris.nereids.trees.plans.physical.PhysicalBinary;
import org.apache.doris.nereids.trees.plans.physical.PhysicalLeaf;
import org.apache.doris.nereids.trees.plans.physical.PhysicalUnary;
/**
* An interface provided some PatternDescriptor.
@ -62,9 +60,9 @@ public interface Patterns {
/**
* create a unboundRelation pattern.
*/
default PatternDescriptor<UnboundRelation, Plan> unboundRelation() {
default PatternDescriptor<LogicalLeaf<UnboundRelation>, Plan> unboundRelation() {
return new PatternDescriptor<>(
new Pattern<>(NodeType.LOGICAL_UNBOUND_RELATION),
new Pattern<>(OperatorType.LOGICAL_UNBOUND_RELATION),
defaultPromise()
);
}
@ -72,9 +70,9 @@ public interface Patterns {
/**
* create a logicalFilter pattern.
*/
default PatternDescriptor<LogicalFilter<Plan>, Plan> logicalFilter() {
default PatternDescriptor<LogicalUnary<LogicalFilter, Plan>, Plan> logicalFilter() {
return new PatternDescriptor<>(
new Pattern<>(NodeType.LOGICAL_FILTER),
new Pattern<>(OperatorType.LOGICAL_FILTER),
defaultPromise()
);
}
@ -82,10 +80,10 @@ public interface Patterns {
/**
* create a logicalFilter pattern with child pattern.
*/
default <T extends Plan> PatternDescriptor<LogicalFilter<T>, Plan>
default <T extends Plan> PatternDescriptor<LogicalUnary<LogicalFilter, T>, Plan>
logicalFilter(PatternDescriptor<T, Plan> childPattern) {
return new PatternDescriptor<>(
new Pattern<>(NodeType.LOGICAL_FILTER, childPattern.pattern),
new Pattern<>(OperatorType.LOGICAL_FILTER, childPattern.pattern),
defaultPromise()
);
}
@ -93,9 +91,9 @@ public interface Patterns {
/**
* create a logicalProject pattern.
*/
default PatternDescriptor<LogicalProject<Plan>, Plan> logicalProject() {
default PatternDescriptor<LogicalUnary<LogicalProject, Plan>, Plan> logicalProject() {
return new PatternDescriptor<>(
new Pattern<>(NodeType.LOGICAL_PROJECT),
new Pattern<>(OperatorType.LOGICAL_PROJECT),
defaultPromise()
);
}
@ -103,10 +101,10 @@ public interface Patterns {
/**
* create a logicalProject pattern.
*/
default <T extends Plan> PatternDescriptor<LogicalProject, Plan>
default <T extends Plan> PatternDescriptor<LogicalUnary<LogicalProject, T>, Plan>
logicalProject(PatternDescriptor<T, Plan> childPattern) {
return new PatternDescriptor<>(
new Pattern<>(NodeType.LOGICAL_PROJECT, childPattern.pattern),
new Pattern<>(OperatorType.LOGICAL_PROJECT, childPattern.pattern),
defaultPromise()
);
}
@ -114,9 +112,9 @@ public interface Patterns {
/**
* create a logicalJoin pattern.
*/
default PatternDescriptor<LogicalJoin<Plan, Plan>, Plan> logicalJoin() {
default PatternDescriptor<LogicalBinary<LogicalJoin, Plan, Plan>, Plan> logicalJoin() {
return new PatternDescriptor<>(
new Pattern<>(NodeType.LOGICAL_JOIN),
new Pattern<>(OperatorType.LOGICAL_JOIN),
defaultPromise()
);
}
@ -124,32 +122,34 @@ public interface Patterns {
/**
* create a logicalJoin pattern with join type.
*/
default PatternDescriptor<LogicalJoin<Plan, Plan>, Plan> logicalJoin(JoinType joinType) {
return new PatternDescriptor<LogicalJoin<Plan, Plan>, Plan>(
new Pattern<>(NodeType.LOGICAL_JOIN),
default PatternDescriptor<LogicalBinary<LogicalJoin, Plan, Plan>, Plan> logicalJoin(JoinType joinType) {
return new PatternDescriptor<LogicalBinary<LogicalJoin, Plan, Plan>, Plan>(
new Pattern<>(OperatorType.LOGICAL_JOIN),
defaultPromise()
).when(j -> j.getJoinType() == joinType);
).when(j -> j.op.getJoinType() == joinType);
}
/**
* create a logicalJoin pattern with joinType and children patterns.
*/
default <C1 extends Plan, C2 extends Plan> PatternDescriptor<LogicalJoin<C1, C2>, Plan> logicalJoin(
JoinType joinType, PatternDescriptor<C1, Plan> leftChildPattern,
PatternDescriptor<C2, Plan> rightChildPattern) {
return new PatternDescriptor<LogicalJoin<C1, C2>, Plan>(
new Pattern<>(NodeType.LOGICAL_JOIN, leftChildPattern.pattern, rightChildPattern.pattern),
default <C1 extends Plan, C2 extends Plan>
PatternDescriptor<LogicalBinary<LogicalJoin, C1, C2>, Plan> logicalJoin(
JoinType joinType, PatternDescriptor<C1, Plan> leftChildPattern,
PatternDescriptor<C2, Plan> rightChildPattern) {
return new PatternDescriptor<LogicalBinary<LogicalJoin, C1, C2>, Plan>(
new Pattern<>(OperatorType.LOGICAL_JOIN, leftChildPattern.pattern, rightChildPattern.pattern),
defaultPromise()
).when(j -> j.getJoinType() == joinType);
).when(j -> j.op.getJoinType() == joinType);
}
/**
* create a logicalJoin pattern with children patterns.
*/
default <C1 extends Plan, C2 extends Plan> PatternDescriptor<LogicalJoin<C1, C2>, Plan> logicalJoin(
PatternDescriptor<C1, Plan> leftChildPattern, PatternDescriptor<C2, Plan> rightChildPattern) {
default <C1 extends Plan, C2 extends Plan>
PatternDescriptor<LogicalBinary<LogicalJoin, C1, C2>, Plan> logicalJoin(
PatternDescriptor<C1, Plan> leftChildPattern, PatternDescriptor<C2, Plan> rightChildPattern) {
return new PatternDescriptor<>(
new Pattern<>(NodeType.LOGICAL_JOIN, leftChildPattern.pattern, rightChildPattern.pattern),
new Pattern<>(OperatorType.LOGICAL_JOIN, leftChildPattern.pattern, rightChildPattern.pattern),
defaultPromise()
);
}
@ -157,30 +157,31 @@ public interface Patterns {
/**
* create a logicalJoin pattern with joinType is inner.
*/
default PatternDescriptor<LogicalJoin<Plan, Plan>, Plan> innerLogicalJoin() {
return new PatternDescriptor<LogicalJoin<Plan, Plan>, Plan>(
new Pattern<>(NodeType.LOGICAL_JOIN),
default PatternDescriptor<LogicalBinary<LogicalJoin, Plan, Plan>, Plan> innerLogicalJoin() {
return new PatternDescriptor<LogicalBinary<LogicalJoin, Plan, Plan>, Plan>(
new Pattern<>(OperatorType.LOGICAL_JOIN),
defaultPromise()
).when(j -> j.getJoinType() == JoinType.INNER_JOIN);
).when(j -> j.op.getJoinType() == JoinType.INNER_JOIN);
}
/**
* create a logical join pattern with join type is inner and children patterns.
*/
default <C1 extends Plan, C2 extends Plan> PatternDescriptor<LogicalJoin<C1, C2>, Plan> innerLogicalJoin(
PatternDescriptor<C1, Plan> leftChildPattern, PatternDescriptor<C2, Plan> rightChildPattern) {
return new PatternDescriptor<LogicalJoin<C1, C2>, Plan>(
new Pattern<>(NodeType.LOGICAL_JOIN, leftChildPattern.pattern, rightChildPattern.pattern),
default <C1 extends Plan, C2 extends Plan>
PatternDescriptor<LogicalBinary<LogicalJoin, C1, C2>, Plan> innerLogicalJoin(
PatternDescriptor<C1, Plan> leftChildPattern, PatternDescriptor<C2, Plan> rightChildPattern) {
return new PatternDescriptor<LogicalBinary<LogicalJoin, C1, C2>, Plan>(
new Pattern<>(OperatorType.LOGICAL_JOIN, leftChildPattern.pattern, rightChildPattern.pattern),
defaultPromise()
).when(j -> j.getJoinType() == JoinType.INNER_JOIN);
).when(j -> j.op.getJoinType() == JoinType.INNER_JOIN);
}
/**
* create a logicalRelation pattern.
*/
default PatternDescriptor<LogicalRelation, Plan> logicalRelation() {
default PatternDescriptor<LogicalLeaf<LogicalRelation>, Plan> logicalRelation() {
return new PatternDescriptor<>(
new Pattern<>(NodeType.LOGICAL_BOUND_RELATION),
new Pattern<>(OperatorType.LOGICAL_BOUND_RELATION),
defaultPromise()
);
}
@ -190,9 +191,9 @@ public interface Patterns {
/**
* create a physicalFilter pattern.
*/
default PatternDescriptor<PhysicalFilter<Plan>, Plan> physicalFilter() {
default PatternDescriptor<PhysicalUnary<PhysicalFilter, Plan>, Plan> physicalFilter() {
return new PatternDescriptor<>(
new Pattern<>(NodeType.PHYSICAL_FILTER),
new Pattern<>(OperatorType.PHYSICAL_FILTER),
defaultPromise()
);
}
@ -200,10 +201,10 @@ public interface Patterns {
/**
* create a physicalFilter pattern with child pattern.
*/
default <T extends Plan> PatternDescriptor<PhysicalFilter<T>, Plan>
default <T extends Plan> PatternDescriptor<PhysicalUnary<PhysicalFilter, T>, Plan>
physicalFilter(PatternDescriptor<T, Plan> childPattern) {
return new PatternDescriptor<>(
new Pattern<>(NodeType.PHYSICAL_FILTER, childPattern.pattern),
new Pattern<>(OperatorType.PHYSICAL_FILTER, childPattern.pattern),
defaultPromise()
);
}
@ -211,9 +212,9 @@ public interface Patterns {
/**
* create a physicalProject pattern.
*/
default PatternDescriptor<PhysicalProject<Plan>, Plan> physicalProject() {
default PatternDescriptor<PhysicalUnary<PhysicalProject, Plan>, Plan> physicalProject() {
return new PatternDescriptor<>(
new Pattern<>(NodeType.PHYSICAL_PROJECT),
new Pattern<>(OperatorType.PHYSICAL_PROJECT),
defaultPromise()
);
}
@ -221,10 +222,10 @@ public interface Patterns {
/**
* create a physicalProject pattern with child pattern.
*/
default <T extends Plan> PatternDescriptor<PhysicalProject<T>, Plan>
default <T extends Plan> PatternDescriptor<PhysicalUnary<PhysicalProject, T>, Plan>
physicalProject(PatternDescriptor<T, Plan> childPattern) {
return new PatternDescriptor<>(
new Pattern<>(NodeType.PHYSICAL_PROJECT, childPattern.pattern),
new Pattern<>(OperatorType.PHYSICAL_PROJECT, childPattern.pattern),
defaultPromise()
);
}
@ -232,9 +233,10 @@ public interface Patterns {
/**
* create a physicalBroadcastHashJoin pattern.
*/
default PatternDescriptor<PhysicalBroadcastHashJoin<Plan, Plan>, Plan> physicalBroadcastHashJoin() {
default PatternDescriptor<PhysicalBinary<PhysicalBroadcastHashJoin, Plan, Plan>, Plan>
physicalBroadcastHashJoin() {
return new PatternDescriptor<>(
new Pattern<>(NodeType.PHYSICAL_BROADCAST_HASH_JOIN),
new Pattern<>(OperatorType.PHYSICAL_BROADCAST_HASH_JOIN),
defaultPromise()
);
}
@ -242,11 +244,12 @@ public interface Patterns {
/**
* create a physicalBroadcastHashJoin pattern with children patterns.
*/
default <C1 extends Plan, C2 extends Plan> PatternDescriptor<PhysicalBroadcastHashJoin<C1, C2>, Plan>
physicalBroadcastHashJoin(PatternDescriptor<C1, Plan> leftChildPattern,
PatternDescriptor<C2, Plan> rightChildPattern) {
default <C1 extends Plan, C2 extends Plan>
PatternDescriptor<PhysicalBinary<PhysicalBroadcastHashJoin, C1, C2>, Plan>
physicalBroadcastHashJoin(PatternDescriptor<C1, Plan> leftChildPattern,
PatternDescriptor<C2, Plan> rightChildPattern) {
return new PatternDescriptor<>(
new Pattern<>(NodeType.PHYSICAL_BROADCAST_HASH_JOIN,
new Pattern<>(OperatorType.PHYSICAL_BROADCAST_HASH_JOIN,
leftChildPattern.pattern,
rightChildPattern.pattern
),
@ -257,118 +260,9 @@ public interface Patterns {
/**
* create a physicalOlapScan pattern.
*/
default PatternDescriptor<PhysicalOlapScan, Plan> physicalOlapScan() {
default PatternDescriptor<PhysicalLeaf<PhysicalOlapScan>, Plan> physicalOlapScan() {
return new PatternDescriptor<>(
new Pattern<>(NodeType.PHYSICAL_OLAP_SCAN),
defaultPromise()
);
}
// expression pattern descriptors
/**
* create a unboundAlias pattern.
*/
default PatternDescriptor<UnboundAlias<Expression>, Expression> unboundAlias() {
return new PatternDescriptor<>(
new Pattern<>(NodeType.UNBOUND_ALIAS),
defaultPromise()
);
}
/**
* create a unboundAlias pattern.
*/
default <T extends Expression> PatternDescriptor<UnboundAlias<T>, Expression>
unboundAlias(PatternDescriptor<T, Expression> childPattern) {
return new PatternDescriptor<>(
new Pattern<>(NodeType.UNBOUND_ALIAS, childPattern.pattern),
defaultPromise()
);
}
/**
* create a unboundSlot pattern.
*/
default PatternDescriptor<UnboundSlot, Expression> unboundSlot() {
return new PatternDescriptor<>(
new Pattern<>(NodeType.UNBOUND_SLOT),
defaultPromise()
);
}
/**
* create a unboundStar pattern.
*/
default PatternDescriptor<UnboundStar, Expression> unboundStar() {
return new PatternDescriptor<>(
new Pattern<>(NodeType.UNBOUND_STAR),
defaultPromise()
);
}
/**
* create a literal pattern.
*/
default PatternDescriptor<Literal, Expression> literal() {
return new PatternDescriptor<>(
new Pattern<>(NodeType.LITERAL),
defaultPromise()
);
}
/**
* create a slotReference pattern.
*/
default PatternDescriptor<SlotReference, Expression> slotReference() {
return new PatternDescriptor<>(
new Pattern<>(NodeType.SLOT_REFERENCE),
defaultPromise()
);
}
/**
* TODO create a ComparisonPredicate pattern.
*/
/**
* TODO create a ComparisonPredicate pattern with children patterns.
*/
/**
* create a equal to predicate pattern.
*/
default PatternDescriptor<EqualTo<Expression, Expression>, Expression> equalTo() {
return new PatternDescriptor<>(new Pattern<>(NodeType.EQUAL_TO), defaultPromise());
}
/**
* create a equal to predicate pattern with children patterns.
*/
default <C1 extends Expression, C2 extends Expression> PatternDescriptor<EqualTo<C1, C2>, Expression> equalTo(
PatternDescriptor<C1, Expression> leftChildPattern, PatternDescriptor<C2, Expression> rightChildPattern) {
return new PatternDescriptor<>(
new Pattern<>(NodeType.EQUAL_TO, leftChildPattern.pattern, rightChildPattern.pattern),
defaultPromise());
}
/**
* create a alias pattern.
*/
default PatternDescriptor<Alias<Expression>, Expression> alias() {
return new PatternDescriptor<>(
new Pattern<>(NodeType.ALIAS),
defaultPromise()
);
}
/**
* create a alias pattern with child pattern.
*/
default <T extends Expression> PatternDescriptor<Alias<T>, Expression>
alias(PatternDescriptor<T, Expression> childPattern) {
return new PatternDescriptor<>(
new Pattern<>(NodeType.ALIAS, childPattern.pattern),
new Pattern<>(OperatorType.PHYSICAL_OLAP_SCAN),
defaultPromise()
);
}

12
fe/fe-core/src/main/java/org/apache/doris/nereids/properties/LogicalProperties.java
View File

@ -19,7 +19,7 @@ package org.apache.doris.nereids.properties;
import org.apache.doris.nereids.trees.expressions.Slot;
import com.clearspring.analytics.util.Lists;
import com.google.common.collect.ImmutableList;
import java.util.List;
@ -27,13 +27,13 @@ import java.util.List;
* Logical properties used for analysis and optimize in Nereids.
*/
public class LogicalProperties {
protected List<Slot> output = Lists.newArrayList();
protected List<Slot> output;
public LogicalProperties(List<Slot> output) {
this.output = ImmutableList.copyOf(output);
}
public List<Slot> getOutput() {
return output;
}
public void setOutput(List<Slot> output) {
this.output = output;
}
}

3
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/PlanRuleFactory.java
View File

@ -18,9 +18,10 @@
package org.apache.doris.nereids.rules;
import org.apache.doris.nereids.trees.plans.Plan;
import org.apache.doris.nereids.trees.plans.Plans;
/**
* interface for all plan rule factories.
*/
public interface PlanRuleFactory extends RuleFactory<Plan> {
public interface PlanRuleFactory extends RuleFactory<Plan>, Plans {
}

5
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/RuleSet.java
View File

@ -22,7 +22,6 @@ import org.apache.doris.nereids.rules.exploration.join.JoinCommutative;
import org.apache.doris.nereids.rules.exploration.join.JoinLeftAssociative;
import org.apache.doris.nereids.rules.implementation.LogicalJoinToHashJoin;
import org.apache.doris.nereids.trees.TreeNode;
import org.apache.doris.nereids.trees.expressions.Expression;
import org.apache.doris.nereids.trees.plans.Plan;
import com.google.common.collect.ImmutableList;
@ -63,10 +62,6 @@ public class RuleSet {
return new RuleFactories();
}
private static RuleFactories<Expression> expressionRuleFactories() {
return new RuleFactories();
}
private static class RuleFactories<TYPE extends TreeNode> {
final Builder<Rule<TYPE>> rules = ImmutableList.builder();

10
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/analysis/AnalysisUnboundRelation.java
View File

@ -20,10 +20,10 @@ package org.apache.doris.nereids.rules.analysis;
import org.apache.doris.catalog.Catalog;
import org.apache.doris.catalog.Database;
import org.apache.doris.catalog.Table;
import org.apache.doris.nereids.operators.plans.logical.LogicalRelation;
import org.apache.doris.nereids.rules.Rule;
import org.apache.doris.nereids.rules.RuleType;
import org.apache.doris.nereids.trees.plans.Plan;
import org.apache.doris.nereids.trees.plans.logical.LogicalRelation;
import org.apache.doris.qe.ConnectContext;
import com.google.common.collect.Lists;
@ -39,19 +39,19 @@ public class AnalysisUnboundRelation extends OneAnalysisRuleFactory {
// fixme, just for example now
return unboundRelation().thenApply(ctx -> {
ConnectContext connectContext = ctx.plannerContext.getConnectContext();
List<String> nameParts = ctx.root.getNameParts();
List<String> nameParts = ctx.root.op.getNameParts();
switch (nameParts.size()) {
case 1: {
List<String> qualifier = Lists.newArrayList(connectContext.getDatabase(), nameParts.get(0));
Table table = getTable(qualifier, connectContext.getCatalog());
return new LogicalRelation(table, qualifier);
return plan(new LogicalRelation(table, qualifier));
}
case 2: {
Table table = getTable(nameParts, connectContext.getCatalog());
return new LogicalRelation(table, nameParts);
return plan(new LogicalRelation(table, nameParts));
}
default:
throw new IllegalStateException("Table name [" + ctx.root.getTableName() + "] is invalid.");
throw new IllegalStateException("Table name [" + ctx.root.op.getTableName() + "] is invalid.");
}
}).toRule(RuleType.BINDING_UNBOUND_RELATION_RULE);
}

2
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/JoinReorderContext.java → fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/JoinReorderContext.java
View File

@ -15,7 +15,7 @@
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.trees.plans.logical;
package org.apache.doris.nereids.rules.exploration;
/**

10
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/join/JoinCommutative.java
View File

@ -17,11 +17,11 @@
package org.apache.doris.nereids.rules.exploration.join;
import org.apache.doris.nereids.operators.plans.logical.LogicalJoin;
import org.apache.doris.nereids.rules.Rule;
import org.apache.doris.nereids.rules.RuleType;
import org.apache.doris.nereids.rules.exploration.OneExplorationRuleFactory;
import org.apache.doris.nereids.trees.plans.Plan;
import org.apache.doris.nereids.trees.plans.logical.LogicalJoin;
/**
* rule factory for exchange inner join's children.
@ -50,8 +50,10 @@ public class JoinCommutative extends OneExplorationRuleFactory {
@Override
public Rule<Plan> build() {
return innerLogicalJoin().then(join -> {
return new LogicalJoin(join.getJoinType().swap(), join.getOnClause(), join.right(), join.left());
}).toRule(RuleType.LOGICAL_JOIN_COMMUTATIVE);
return innerLogicalJoin().then(join -> plan(
new LogicalJoin(join.op.getJoinType().swap(), join.op.getOnClause()),
join.right(),
join.left()
)).toRule(RuleType.LOGICAL_JOIN_COMMUTATIVE);
}
}

24
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/join/JoinExchange.java
View File

@ -17,11 +17,12 @@
package org.apache.doris.nereids.rules.exploration.join;
import org.apache.doris.nereids.operators.plans.logical.LogicalJoin;
import org.apache.doris.nereids.rules.Rule;
import org.apache.doris.nereids.rules.RuleType;
import org.apache.doris.nereids.rules.exploration.OneExplorationRuleFactory;
import org.apache.doris.nereids.trees.plans.Plan;
import org.apache.doris.nereids.trees.plans.logical.LogicalJoin;
import org.apache.doris.nereids.trees.plans.logical.LogicalBinary;
/**
@ -38,19 +39,26 @@ public class JoinExchange extends OneExplorationRuleFactory {
@Override
public Rule<Plan> build() {
return innerLogicalJoin(innerLogicalJoin(), innerLogicalJoin()).then(topJoin -> {
LogicalJoin leftJoin = topJoin.left();
LogicalJoin rightJoin = topJoin.left();
LogicalBinary<LogicalJoin, Plan, Plan> leftJoin = topJoin.left();
LogicalBinary<LogicalJoin, Plan, Plan> rightJoin = topJoin.right();
Plan a = leftJoin.left();
Plan b = leftJoin.right();
Plan c = rightJoin.left();
Plan d = rightJoin.right();
LogicalJoin newLeftJoin = new LogicalJoin(leftJoin.getJoinType(), leftJoin.getOnClause(), a, c);
LogicalJoin newRightJoin = new LogicalJoin(rightJoin.getJoinType(), rightJoin.getOnClause(), b, d);
LogicalJoin newTopJoin =
new LogicalJoin(topJoin.getJoinType(), topJoin.getOnClause(), newLeftJoin, newRightJoin);
Plan newLeftJoin = plan(
new LogicalJoin(leftJoin.op.getJoinType(), leftJoin.op.getOnClause()),
a, c
);
Plan newRightJoin = plan(
new LogicalJoin(rightJoin.op.getJoinType(), rightJoin.op.getOnClause()),
b, d
);
Plan newTopJoin = plan(
new LogicalJoin(topJoin.op.getJoinType(), topJoin.op.getOnClause()),
newLeftJoin, newRightJoin
);
return newTopJoin;
}).toRule(RuleType.LOGICAL_JOIN_EXCHANGE);
}

21
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/join/JoinLAsscom.java
View File

@ -17,12 +17,12 @@
package org.apache.doris.nereids.rules.exploration.join;
import org.apache.doris.nereids.operators.plans.logical.LogicalJoin;
import org.apache.doris.nereids.rules.Rule;
import org.apache.doris.nereids.rules.RuleType;
import org.apache.doris.nereids.rules.exploration.OneExplorationRuleFactory;
import org.apache.doris.nereids.trees.plans.Plan;
import org.apache.doris.nereids.trees.plans.logical.LogicalJoin;
import org.apache.doris.nereids.trees.plans.logical.LogicalBinary;
/**
* Rule for change inner join left associative to right.
@ -38,16 +38,21 @@ public class JoinLAsscom extends OneExplorationRuleFactory {
@Override
public Rule<Plan> build() {
return innerLogicalJoin(innerLogicalJoin(), any()).then(topJoin -> {
LogicalJoin bottomJoin = topJoin.left();
LogicalBinary<LogicalJoin, Plan, Plan> bottomJoin = topJoin.left();
Plan c = topJoin.right();
Plan a = bottomJoin.left();
Plan b = bottomJoin.right();
Plan c = topJoin.right();
LogicalJoin newBottomJoin =
new LogicalJoin(bottomJoin.getJoinType(), bottomJoin.getOnClause(), a, c);
LogicalJoin newTopJoin =
new LogicalJoin(bottomJoin.getJoinType(), topJoin.getOnClause(), newBottomJoin, b);
Plan newBottomJoin = plan(
new LogicalJoin(bottomJoin.op.getJoinType(), bottomJoin.op.getOnClause()),
a, c
);
Plan newTopJoin = plan(
new LogicalJoin(bottomJoin.op.getJoinType(), topJoin.op.getOnClause()),
newBottomJoin, b
);
return newTopJoin;
}).toRule(RuleType.LOGICAL_JOIN_L_ASSCOM);
}

22
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/exploration/join/JoinLeftAssociative.java
View File

@ -17,12 +17,12 @@
package org.apache.doris.nereids.rules.exploration.join;
import org.apache.doris.nereids.operators.plans.JoinType;
import org.apache.doris.nereids.operators.plans.logical.LogicalJoin;
import org.apache.doris.nereids.rules.Rule;
import org.apache.doris.nereids.rules.RuleType;
import org.apache.doris.nereids.rules.exploration.OneExplorationRuleFactory;
import org.apache.doris.nereids.trees.plans.JoinType;
import org.apache.doris.nereids.trees.plans.Plan;
import org.apache.doris.nereids.trees.plans.logical.LogicalJoin;
/**
* Rule factory for change inner join left associative to right.
@ -39,16 +39,14 @@ public class JoinLeftAssociative extends OneExplorationRuleFactory {
public Rule<Plan> build() {
return innerLogicalJoin(innerLogicalJoin(), any()).then(root -> {
// fixme, just for example now
return new LogicalJoin(
JoinType.INNER_JOIN,
root.getOnClause(),
root.left().left(),
new LogicalJoin(
JoinType.INNER_JOIN,
root.getOnClause(),
root.left().right(),
root.right()
)
return plan(
new LogicalJoin(JoinType.INNER_JOIN, root.op.getOnClause()),
root.left().left(),
plan(
new LogicalJoin(JoinType.INNER_JOIN, root.op.getOnClause()),
root.left().right(),
root.right()
)
);
}).toRule(RuleType.LOGICAL_LEFT_JOIN_ASSOCIATIVE);
}

12
fe/fe-core/src/main/java/org/apache/doris/nereids/rules/implementation/LogicalJoinToHashJoin.java
View File

@ -17,11 +17,10 @@
package org.apache.doris.nereids.rules.implementation;
import org.apache.doris.nereids.operators.plans.physical.PhysicalBroadcastHashJoin;
import org.apache.doris.nereids.rules.Rule;
import org.apache.doris.nereids.rules.RuleType;
import org.apache.doris.nereids.trees.plans.Plan;
import org.apache.doris.nereids.trees.plans.physical.PhysicalBroadcastHashJoin;
/**
* Implementation rule that convert logical join to physical hash join.
@ -30,11 +29,10 @@ public class LogicalJoinToHashJoin extends OneImplementationRuleFactory {
@Override
public Rule<Plan> build() {
// fixme, just for example now
return logicalJoin().then(join -> new PhysicalBroadcastHashJoin(
join.getJoinType(),
join.left(),
join.right(),
join.getOnClause()
return logicalJoin().then(join -> plan(
new PhysicalBroadcastHashJoin(join.op.getJoinType(), join.op.getOnClause()),
join.getLogicalProperties(),
join.left(), join.right()
)).toRule(RuleType.LOGICAL_JOIN_TO_HASH_JOIN_RULE);
}
}

17
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/NodeType.java
View File

@ -21,20 +21,9 @@ package org.apache.doris.nereids.trees;
* Types for all TreeNode in Nereids, include Plan and Expression.
*/
public enum NodeType {
// logical plan
// plan
LOGICAL,
LOGICAL_UNBOUND_RELATION,
LOGICAL_BOUND_RELATION,
LOGICAL_PROJECT,
LOGICAL_FILTER,
LOGICAL_JOIN,
// physical plan
PHYSICAL,
PHYSICAL_OLAP_SCAN,
PHYSICAL_PROJECT,
PHYSICAL_FILTER,
PHYSICAL_BROADCAST_HASH_JOIN,
// expressions
EXPRESSION,
@ -54,7 +43,5 @@ public enum NodeType {
ALIAS,
// pattern
PATTERN,
ANY,
MULTI,
PATTERN
}

21
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/AbstractPlan.java
View File

@ -17,15 +17,15 @@
package org.apache.doris.nereids.trees.plans;
import org.apache.doris.nereids.exceptions.UnboundException;
import org.apache.doris.nereids.operators.plans.PlanOperator;
import org.apache.doris.nereids.trees.AbstractTreeNode;
import org.apache.doris.nereids.trees.NodeType;
import org.apache.doris.nereids.trees.expressions.Slot;
import org.apache.commons.lang3.StringUtils;
import java.util.ArrayList;
import java.util.List;
import java.util.Objects;
/**
* Abstract class for all concrete plan node.
@ -33,17 +33,22 @@ import java.util.List;
* @param <PLAN_TYPE> either {@link org.apache.doris.nereids.trees.plans.logical.LogicalPlan}
* or {@link org.apache.doris.nereids.trees.plans.physical.PhysicalPlan}
*/
public abstract class AbstractPlan<PLAN_TYPE extends AbstractPlan<PLAN_TYPE>>
extends AbstractTreeNode<PLAN_TYPE> implements Plan<PLAN_TYPE> {
public abstract class AbstractPlan<
PLAN_TYPE extends AbstractPlan<PLAN_TYPE, OP_TYPE>,
OP_TYPE extends PlanOperator>
extends AbstractTreeNode<PLAN_TYPE> implements Plan<PLAN_TYPE, OP_TYPE> {
protected List<Slot> output;
public final OP_TYPE op;
public AbstractPlan(NodeType type, Plan...children) {
public AbstractPlan(NodeType type, OP_TYPE operator, Plan... children) {
super(type, children);
this.op = Objects.requireNonNull(operator, "operator can not be null");
}
@Override
public abstract List<Slot> getOutput() throws UnboundException;
public OP_TYPE getOperator() {
return op;
}
@Override
public List<Plan> children() {
@ -67,7 +72,7 @@ public abstract class AbstractPlan<PLAN_TYPE extends AbstractPlan<PLAN_TYPE>>
return StringUtils.join(lines, "\n");
}
private void treeString(List<String> lines, int depth, List<Boolean> lastChildren, Plan<PLAN_TYPE> plan) {
private void treeString(List<String> lines, int depth, List<Boolean> lastChildren, Plan plan) {
StringBuilder sb = new StringBuilder();
if (depth > 0) {
if (lastChildren.size() > 1) {

6
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/BinaryPlan.java
View File

@ -17,6 +17,7 @@
package org.apache.doris.nereids.trees.plans;
import org.apache.doris.nereids.operators.plans.BinaryPlanOperator;
import org.apache.doris.nereids.trees.BinaryNode;
import java.util.List;
@ -25,10 +26,11 @@ import java.util.List;
* interface for all plan that have two children.
*/
public interface BinaryPlan<
PLAN_TYPE extends BinaryPlan<PLAN_TYPE, LEFT_CHILD_TYPE, RIGHT_CHILD_TYPE>,
PLAN_TYPE extends BinaryPlan<PLAN_TYPE, OP_TYPE, LEFT_CHILD_TYPE, RIGHT_CHILD_TYPE>,
OP_TYPE extends BinaryPlanOperator,
LEFT_CHILD_TYPE extends Plan,
RIGHT_CHILD_TYPE extends Plan>
extends Plan<PLAN_TYPE>, BinaryNode<PLAN_TYPE, LEFT_CHILD_TYPE, RIGHT_CHILD_TYPE> {
extends Plan<PLAN_TYPE, OP_TYPE>, BinaryNode<PLAN_TYPE, LEFT_CHILD_TYPE, RIGHT_CHILD_TYPE> {
@Override
List<Plan> children();

7
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/LeafPlan.java
View File

@ -17,6 +17,7 @@
package org.apache.doris.nereids.trees.plans;
import org.apache.doris.nereids.operators.plans.LeafPlanOperator;
import org.apache.doris.nereids.trees.LeafNode;
import java.util.List;
@ -24,8 +25,10 @@ import java.util.List;
/**
* Abstract class for all plan node that have no child.
*/
public interface LeafPlan<PLAN_TYPE extends LeafPlan<PLAN_TYPE>>
extends Plan<PLAN_TYPE>, LeafNode<PLAN_TYPE> {
public interface LeafPlan<
PLAN_TYPE extends LeafPlan<PLAN_TYPE, OP_TYPE>,
OP_TYPE extends LeafPlanOperator>
extends Plan<PLAN_TYPE, OP_TYPE>, LeafNode<PLAN_TYPE> {
@Override
List<Plan> children();

13
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/Plan.java
View File

@ -17,7 +17,8 @@
package org.apache.doris.nereids.trees.plans;
import org.apache.doris.nereids.exceptions.UnboundException;
import org.apache.doris.nereids.operators.plans.PlanOperator;
import org.apache.doris.nereids.properties.LogicalProperties;
import org.apache.doris.nereids.trees.TreeNode;
import org.apache.doris.nereids.trees.expressions.Slot;
@ -26,9 +27,15 @@ import java.util.List;
/**
* Abstract class for all plan node.
*/
public interface Plan<PLAN_TYPE extends Plan<PLAN_TYPE>> extends TreeNode<PLAN_TYPE> {
public interface Plan<
PLAN_TYPE extends Plan<PLAN_TYPE, OP_TYPE>,
OP_TYPE extends PlanOperator> extends TreeNode<PLAN_TYPE> {
List<Slot> getOutput() throws UnboundException;
OP_TYPE getOperator();
LogicalProperties getLogicalProperties();
List<Slot> getOutput();
String treeString();

71
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/Plans.java Normal file
View File

@ -0,0 +1,71 @@
// Licensed to the Apache Software Foundation (ASF) under one
// or more contributor license agreements. See the NOTICE file
// distributed with this work for additional information
// regarding copyright ownership. The ASF licenses this file
// to you under the Apache License, Version 2.0 (the
// "License"); you may not use this file except in compliance
// with the License. You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing,
// software distributed under the License is distributed on an
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
package org.apache.doris.nereids.trees.plans;
import org.apache.doris.nereids.operators.plans.logical.LogicalBinaryOperator;
import org.apache.doris.nereids.operators.plans.logical.LogicalLeafOperator;
import org.apache.doris.nereids.operators.plans.logical.LogicalUnaryOperator;
import org.apache.doris.nereids.operators.plans.physical.PhysicalBinaryOperator;
import org.apache.doris.nereids.operators.plans.physical.PhysicalLeafOperator;
import org.apache.doris.nereids.operators.plans.physical.PhysicalUnaryOperator;
import org.apache.doris.nereids.properties.LogicalProperties;
import org.apache.doris.nereids.trees.plans.logical.LogicalBinary;
import org.apache.doris.nereids.trees.plans.logical.LogicalLeaf;
import org.apache.doris.nereids.trees.plans.logical.LogicalUnary;
import org.apache.doris.nereids.trees.plans.physical.PhysicalBinary;
import org.apache.doris.nereids.trees.plans.physical.PhysicalLeaf;
import org.apache.doris.nereids.trees.plans.physical.PhysicalUnary;
/**
* An interface provided some builder of Plan.
* Child Interface(PlanRuleFactory) can use to build some plan for transform rule.
* You can simply use the override plan function to build a plan by the operator type.
*/
public interface Plans {
default <OP_TYPE extends LogicalLeafOperator> LogicalLeaf<OP_TYPE> plan(OP_TYPE op) {
return new LogicalLeaf(op);
}
default <OP_TYPE extends LogicalUnaryOperator, CHILD_TYPE extends Plan> LogicalUnary<OP_TYPE, CHILD_TYPE>
plan(OP_TYPE op, CHILD_TYPE child) {
return new LogicalUnary(op, child);
}
default <OP_TYPE extends LogicalBinaryOperator, LEFT_CHILD_TYPE extends Plan, RIGHT_CHILD_TYPE extends Plan>
LogicalBinary<OP_TYPE, LEFT_CHILD_TYPE, RIGHT_CHILD_TYPE>
plan(OP_TYPE op, LEFT_CHILD_TYPE leftChild, RIGHT_CHILD_TYPE rightChild) {
return new LogicalBinary(op, leftChild, rightChild);
}
default <OP_TYPE extends PhysicalLeafOperator> PhysicalLeaf<OP_TYPE>
plan(OP_TYPE op, LogicalProperties logicalProperties) {
return new PhysicalLeaf(op, logicalProperties);
}
default <OP_TYPE extends PhysicalUnaryOperator, CHILD_TYPE extends Plan> PhysicalUnary<OP_TYPE, CHILD_TYPE>
plan(OP_TYPE op, LogicalProperties logicalProperties, CHILD_TYPE child) {
return new PhysicalUnary(op, logicalProperties, child);
}
default <OP_TYPE extends PhysicalBinaryOperator, LEFT_CHILD_TYPE extends Plan, RIGHT_CHILD_TYPE extends Plan>
PhysicalBinary<OP_TYPE, LEFT_CHILD_TYPE, RIGHT_CHILD_TYPE>
plan(OP_TYPE op, LogicalProperties logicalProperties,
LEFT_CHILD_TYPE leftChild, RIGHT_CHILD_TYPE rightChild) {
return new PhysicalBinary(op, logicalProperties, leftChild, rightChild);
}
}

6
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/UnaryPlan.java
View File

@ -17,6 +17,7 @@
package org.apache.doris.nereids.trees.plans;
import org.apache.doris.nereids.operators.plans.UnaryPlanOperator;
import org.apache.doris.nereids.trees.UnaryNode;
import java.util.List;
@ -25,9 +26,10 @@ import java.util.List;
* interface for all plan node that have one child.
*/
public interface UnaryPlan<
PLAN_TYPE extends UnaryPlan<PLAN_TYPE, CHILD_TYPE>,
PLAN_TYPE extends UnaryPlan<PLAN_TYPE, OP_TYPE, CHILD_TYPE>,
OP_TYPE extends UnaryPlanOperator,
CHILD_TYPE extends Plan>
extends Plan<PLAN_TYPE>, UnaryNode<PLAN_TYPE, CHILD_TYPE> {
extends Plan<PLAN_TYPE, OP_TYPE>, UnaryNode<PLAN_TYPE, CHILD_TYPE> {
@Override
List<Plan> children();

30
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/AbstractLogicalPlan.java
View File

@ -17,18 +17,38 @@
package org.apache.doris.nereids.trees.plans.logical;
import org.apache.doris.nereids.operators.plans.logical.LogicalOperator;
import org.apache.doris.nereids.properties.LogicalProperties;
import org.apache.doris.nereids.trees.NodeType;
import org.apache.doris.nereids.trees.expressions.Slot;
import org.apache.doris.nereids.trees.plans.AbstractPlan;
import org.apache.doris.nereids.trees.plans.Plan;
import java.util.List;
/**
* Abstract class for all concrete logical plan.
*/
public abstract class AbstractLogicalPlan<PLAN_TYPE extends AbstractLogicalPlan<PLAN_TYPE>>
extends AbstractPlan<PLAN_TYPE>
implements LogicalPlan<PLAN_TYPE> {
public abstract class AbstractLogicalPlan<
PLAN_TYPE extends AbstractLogicalPlan<PLAN_TYPE, OP_TYPE>,
OP_TYPE extends LogicalOperator>
extends AbstractPlan<PLAN_TYPE, OP_TYPE>
implements LogicalPlan<PLAN_TYPE, OP_TYPE> {
public AbstractLogicalPlan(NodeType type, Plan... children) {
super(type, children);
protected final LogicalProperties logicalProperties;
public AbstractLogicalPlan(NodeType type, OP_TYPE operator, Plan... children) {
super(type, operator, children);
this.logicalProperties = new LogicalProperties(operator.computeOutput(children));
}
@Override
public LogicalProperties getLogicalProperties() {
return logicalProperties;
}
@Override
public List<Slot> getOutput() {
return logicalProperties.getOutput();
}
}

14
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalBinary.java
View File

@ -17,6 +17,7 @@
package org.apache.doris.nereids.trees.plans.logical;
import org.apache.doris.nereids.operators.plans.logical.LogicalBinaryOperator;
import org.apache.doris.nereids.trees.NodeType;
import org.apache.doris.nereids.trees.plans.BinaryPlan;
import org.apache.doris.nereids.trees.plans.Plan;
@ -24,14 +25,15 @@ import org.apache.doris.nereids.trees.plans.Plan;
/**
* Abstract class for all logical plan that have two children.
*/
public abstract class LogicalBinary<
PLAN_TYPE extends LogicalBinary<PLAN_TYPE, LEFT_CHILD_TYPE, RIGHT_CHILD_TYPE>,
public class LogicalBinary<
OP_TYPE extends LogicalBinaryOperator,
LEFT_CHILD_TYPE extends Plan,
RIGHT_CHILD_TYPE extends Plan>
extends AbstractLogicalPlan<PLAN_TYPE>
implements BinaryPlan<PLAN_TYPE, LEFT_CHILD_TYPE, RIGHT_CHILD_TYPE> {
extends AbstractLogicalPlan<LogicalBinary<OP_TYPE, LEFT_CHILD_TYPE, RIGHT_CHILD_TYPE>, OP_TYPE>
implements BinaryPlan<LogicalBinary<OP_TYPE, LEFT_CHILD_TYPE, RIGHT_CHILD_TYPE>,
OP_TYPE, LEFT_CHILD_TYPE, RIGHT_CHILD_TYPE> {
public LogicalBinary(NodeType type, LEFT_CHILD_TYPE leftChild, RIGHT_CHILD_TYPE rightChild) {
super(type, leftChild, rightChild);
public LogicalBinary(OP_TYPE operator, LEFT_CHILD_TYPE leftChild, RIGHT_CHILD_TYPE rightChild) {
super(NodeType.LOGICAL, operator, leftChild, rightChild);
}
}

11
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalLeaf.java
View File

@ -17,17 +17,18 @@
package org.apache.doris.nereids.trees.plans.logical;
import org.apache.doris.nereids.operators.plans.logical.LogicalLeafOperator;
import org.apache.doris.nereids.trees.NodeType;
import org.apache.doris.nereids.trees.plans.LeafPlan;
/**
* Abstract class for all logical plan that have no child.
*/
public abstract class LogicalLeaf<PLAN_TYPE extends LogicalLeaf<PLAN_TYPE>>
extends AbstractLogicalPlan<PLAN_TYPE>
implements LeafPlan<PLAN_TYPE> {
public class LogicalLeaf<OP_TYPE extends LogicalLeafOperator>
extends AbstractLogicalPlan<LogicalLeaf<OP_TYPE>, OP_TYPE>
implements LeafPlan<LogicalLeaf<OP_TYPE>, OP_TYPE> {
public LogicalLeaf(NodeType type) {
super(type);
public LogicalLeaf(OP_TYPE operator) {
super(NodeType.LOGICAL, operator);
}
}

6
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalPlan.java
View File

@ -17,6 +17,7 @@
package org.apache.doris.nereids.trees.plans.logical;
import org.apache.doris.nereids.operators.plans.logical.LogicalOperator;
import org.apache.doris.nereids.trees.plans.Plan;
import java.util.List;
@ -25,7 +26,10 @@ import java.util.function.BiFunction;
/**
* Abstract class for all logical plan in Nereids.
*/
public interface LogicalPlan<PLAN_TYPE extends LogicalPlan<PLAN_TYPE>> extends Plan<PLAN_TYPE> {
public interface LogicalPlan<
PLAN_TYPE extends LogicalPlan<PLAN_TYPE, OP_TYPE>,
OP_TYPE extends LogicalOperator>
extends Plan<PLAN_TYPE, OP_TYPE> {
@Override
List<Plan> children();

13
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalUnary.java
View File

@ -17,6 +17,7 @@
package org.apache.doris.nereids.trees.plans.logical;
import org.apache.doris.nereids.operators.plans.logical.LogicalUnaryOperator;
import org.apache.doris.nereids.trees.NodeType;
import org.apache.doris.nereids.trees.plans.Plan;
import org.apache.doris.nereids.trees.plans.UnaryPlan;
@ -24,13 +25,11 @@ import org.apache.doris.nereids.trees.plans.UnaryPlan;
/**
* Abstract class for all logical plan that have one child.
*/
public abstract class LogicalUnary<
PLAN_TYPE extends LogicalUnary<PLAN_TYPE, CHILD_TYPE>,
CHILD_TYPE extends Plan>
extends AbstractLogicalPlan<PLAN_TYPE>
implements UnaryPlan<PLAN_TYPE, CHILD_TYPE> {
public class LogicalUnary<OP_TYPE extends LogicalUnaryOperator, CHILD_TYPE extends Plan>
extends AbstractLogicalPlan<LogicalUnary<OP_TYPE, CHILD_TYPE>, OP_TYPE>
implements UnaryPlan<LogicalUnary<OP_TYPE, CHILD_TYPE>, OP_TYPE, CHILD_TYPE> {
public LogicalUnary(NodeType type, CHILD_TYPE child) {
super(type, child);
public LogicalUnary(OP_TYPE operator, CHILD_TYPE child) {
super(NodeType.LOGICAL, operator, child);
}
}

39
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/AbstractPhysicalPlan.java
View File

@ -17,6 +17,7 @@
package org.apache.doris.nereids.trees.plans.physical;
import org.apache.doris.nereids.operators.plans.physical.PhysicalOperator;
import org.apache.doris.nereids.properties.LogicalProperties;
import org.apache.doris.nereids.properties.PhysicalProperties;
import org.apache.doris.nereids.trees.NodeType;
@ -25,28 +26,42 @@ import org.apache.doris.nereids.trees.plans.AbstractPlan;
import org.apache.doris.nereids.trees.plans.Plan;
import java.util.List;
import java.util.Objects;
/**
* Abstract class for all concrete physical plan.
*/
public abstract class AbstractPhysicalPlan<PLAN_TYPE extends AbstractPhysicalPlan<PLAN_TYPE>>
extends AbstractPlan<PLAN_TYPE>
implements PhysicalPlan<PLAN_TYPE> {
public abstract class AbstractPhysicalPlan<
PLAN_TYPE extends AbstractPhysicalPlan<PLAN_TYPE, OP_TYPE>,
OP_TYPE extends PhysicalOperator>
extends AbstractPlan<PLAN_TYPE, OP_TYPE>
implements PhysicalPlan<PLAN_TYPE, OP_TYPE> {
protected LogicalProperties logicalProperties;
protected PhysicalProperties physicalProperties;
protected final LogicalProperties logicalProperties;
protected final PhysicalProperties physicalProperties;
public AbstractPhysicalPlan(NodeType type, Plan... children) {
super(type, children);
}
@Override
public void setLogicalProperties(LogicalProperties logicalProperties) {
this.logicalProperties = logicalProperties;
/**
* create physical plan by op, logicalProperties and children.
*/
public AbstractPhysicalPlan(NodeType type, OP_TYPE operator,
LogicalProperties logicalProperties, Plan... children) {
super(type, operator, children);
this.logicalProperties = Objects.requireNonNull(logicalProperties, "logicalProperties can not be null");
// TODO: compute physical properties
this.physicalProperties = new PhysicalProperties();
}
@Override
public List<Slot> getOutput() {
return logicalProperties.getOutput();
}
@Override
public LogicalProperties getLogicalProperties() {
return logicalProperties;
}
public PhysicalProperties getPhysicalProperties() {
return physicalProperties;
}
}

17
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/PhysicalBinary.java
View File

@ -17,6 +17,8 @@
package org.apache.doris.nereids.trees.plans.physical;
import org.apache.doris.nereids.operators.plans.physical.PhysicalBinaryOperator;
import org.apache.doris.nereids.properties.LogicalProperties;
import org.apache.doris.nereids.trees.NodeType;
import org.apache.doris.nereids.trees.plans.BinaryPlan;
import org.apache.doris.nereids.trees.plans.Plan;
@ -24,14 +26,17 @@ import org.apache.doris.nereids.trees.plans.Plan;
/**
* Abstract class for all physical plan that have two children.
*/
public abstract class PhysicalBinary<
PLAN_TYPE extends PhysicalBinary<PLAN_TYPE, LEFT_CHILD_TYPE, RIGHT_CHILD_TYPE>,
public class PhysicalBinary<
OP_TYPE extends PhysicalBinaryOperator,
LEFT_CHILD_TYPE extends Plan,
RIGHT_CHILD_TYPE extends Plan>
extends AbstractPhysicalPlan<PLAN_TYPE>
implements BinaryPlan<PLAN_TYPE, LEFT_CHILD_TYPE, RIGHT_CHILD_TYPE> {
extends AbstractPhysicalPlan<PhysicalBinary<OP_TYPE, LEFT_CHILD_TYPE, RIGHT_CHILD_TYPE>, OP_TYPE>
implements BinaryPlan<
PhysicalBinary<OP_TYPE, LEFT_CHILD_TYPE, RIGHT_CHILD_TYPE>,
OP_TYPE, LEFT_CHILD_TYPE, RIGHT_CHILD_TYPE> {
public PhysicalBinary(NodeType type, LEFT_CHILD_TYPE leftChild, RIGHT_CHILD_TYPE rightChild) {
super(type, leftChild, rightChild);
public PhysicalBinary(OP_TYPE operator, LogicalProperties logicalProperties,
LEFT_CHILD_TYPE leftChild, RIGHT_CHILD_TYPE rightChild) {
super(NodeType.PHYSICAL, operator, logicalProperties, leftChild, rightChild);
}
}

12
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/PhysicalLeaf.java
View File

@ -17,17 +17,19 @@
package org.apache.doris.nereids.trees.plans.physical;
import org.apache.doris.nereids.operators.plans.physical.PhysicalLeafOperator;
import org.apache.doris.nereids.properties.LogicalProperties;
import org.apache.doris.nereids.trees.NodeType;
import org.apache.doris.nereids.trees.plans.LeafPlan;
/**
* Abstract class for all physical plan that have no child.
*/
public abstract class PhysicalLeaf<PLAN_TYPE extends PhysicalLeaf<PLAN_TYPE>>
extends AbstractPhysicalPlan<PLAN_TYPE>
implements LeafPlan<PLAN_TYPE> {
public class PhysicalLeaf<OP_TYPE extends PhysicalLeafOperator>
extends AbstractPhysicalPlan<PhysicalLeaf<OP_TYPE>, OP_TYPE>
implements LeafPlan<PhysicalLeaf<OP_TYPE>, OP_TYPE> {
public PhysicalLeaf(NodeType type) {
super(type);
public PhysicalLeaf(OP_TYPE operator, LogicalProperties logicalProperties) {
super(NodeType.PHYSICAL, operator, logicalProperties);
}
}

10
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/PhysicalPlan.java
View File

@ -17,7 +17,7 @@
package org.apache.doris.nereids.trees.plans.physical;
import org.apache.doris.nereids.properties.LogicalProperties;
import org.apache.doris.nereids.operators.plans.physical.PhysicalOperator;
import org.apache.doris.nereids.trees.plans.Plan;
import java.util.List;
@ -25,12 +25,14 @@ import java.util.List;
/**
* interface for all physical plan.
*/
public interface PhysicalPlan<PLAN_TYPE extends PhysicalPlan<PLAN_TYPE>> extends Plan<PLAN_TYPE> {
public interface PhysicalPlan<
PLAN_TYPE extends PhysicalPlan<PLAN_TYPE, OP_TYPE>,
OP_TYPE extends PhysicalOperator>
extends Plan<PLAN_TYPE, OP_TYPE> {
@Override
List<Plan> children();
@Override
Plan child(int index);
void setLogicalProperties(LogicalProperties logicalProperties);
}

13
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/physical/PhysicalUnary.java
View File

@ -17,6 +17,8 @@
package org.apache.doris.nereids.trees.plans.physical;
import org.apache.doris.nereids.operators.plans.physical.PhysicalUnaryOperator;
import org.apache.doris.nereids.properties.LogicalProperties;
import org.apache.doris.nereids.trees.NodeType;
import org.apache.doris.nereids.trees.plans.Plan;
import org.apache.doris.nereids.trees.plans.UnaryPlan;
@ -24,13 +26,12 @@ import org.apache.doris.nereids.trees.plans.UnaryPlan;
/**
* Abstract class for all physical plan that have one child.
*/
public abstract class PhysicalUnary<
PLAN_TYPE extends PhysicalUnary<PLAN_TYPE, CHILD_TYPE>,
public class PhysicalUnary<OP_TYPE extends PhysicalUnaryOperator,
CHILD_TYPE extends Plan>
extends AbstractPhysicalPlan<PLAN_TYPE>
implements UnaryPlan<PLAN_TYPE, CHILD_TYPE> {
extends AbstractPhysicalPlan<PhysicalUnary<OP_TYPE, CHILD_TYPE>, OP_TYPE>
implements UnaryPlan<PhysicalUnary<OP_TYPE, CHILD_TYPE>, OP_TYPE, CHILD_TYPE> {
public PhysicalUnary(NodeType type, CHILD_TYPE child) {
super(type, child);
public PhysicalUnary(OP_TYPE operator, LogicalProperties logicalProperties, CHILD_TYPE child) {
super(NodeType.PHYSICAL, operator, logicalProperties, child);
}
}