Change the DatabaseIf APIs' return type to TableIf.
Use generics in DatabaseIf, to avoid changing the return type in Database.
Currently Database class use type Table, I'm try to avoid changing it to TableIf.
Because in this case, we need to change a lot of code.
When plan bucket shuffle join, we need to know left table bucket number.
Currently, we use tablet number directly based on the assumption that left table has only one partition.
But, when left table is colocated table, it could have more than one partition.
In this case, some data in right table will be dropped incorrectly and produce wrong result for query.
reproduce could follow regression test in PR.
* [fix](fe) select stmt will make BE coredump when its castExpr is like cast(int as array<>)
* fix implicit cast scalar type bug
* Revert "fix implicit cast scalar type bug"
This reverts commit 1f05b6bab72430214dca88f386b50ef9a081e60a.
* only check array cast, retrigger
Add ntile function.
For non-vectorized-engine, I just implemented like Impala, rewrite ntile to row_number and count.
But for vectorized-engine, I implemented WindowFunctionNTile.
In SetOperationNode we do passthrough, if we child output is same with itself output.
In method isChildPassthrough we only consider memory layout.
When we use vectorized engine, we need to use SlotDesc offset in TupleDesc instead of
memory layout to check whether pass-through can be performed
closed#9644
Second step of statistics derivation: implementation of nodes other than scan_node.
The statistical information derivation interface of all nodes is uniformly placed in DeriveFactory.java.
Added one-sided to verify the derivation is correct.
Statistics derivation for each node is placed in its own *StatsDerive.java
detailed design: https://docs.google.com/document/d/1u1L6XhyzKShoyYRwFQ6kE1rnvY2iFwauwg289au5Qq0/edit
Currently, only the root user has node_priv privileges.
That is, only the root user can operate the addition and deletion of nodes.
In the original design of Doris, there is an Operator role. This role can have node_priv for node operations.
This PR supports assigning node_priv to users other than root.
However, only users who have both grant_priv and node_priv can assign node_priv to other users.
This ensures that only the root user has this permission, and users who are given node_priv
cannot continue to expand this permission outward.
- set inline view's slot descriptor to nullable in register column ref
- propagate slot nullable when generate inline view's query node in SingleNodePlanner
This CL mainly changes:
1. Reducing the rpc timeout problem caused by rpc waiting for the worker thread of brpc.
1. Merge multiple fragment instances on the same BE to send requests to reduce the number of send fragment rpcs
2. If fragments size >= 3, use 2 phase RPC: one is to send all fragments, two is to start these fragments. So that there
will be at most 2 RPC for each query on one BE.
3. Set the timeout of send fragment rpc to the query timeout to ensure the consistency of users' expectation of query timeout period.
4. Do not close the connection anymore when rpc timeout occurs.
5. Change some log level from info to debug to simplify the fe.log content.
NOTICE:
1. Change the definition of execPlanFragment rpc, must first upgrade BE.
3. Remove FE config `remote_fragment_exec_timeout_ms`
1. Convert child expressions in InPredicate to column type and discard child expressions in them that cannot be converted exactly.
2. Fix the bug of ColumnRange exception caused by InPredicate child expressions type conversion.
3. Fix the problem that the tablet could not be hit due caused by InPredicate child expressions type conversion.
Issue Number: close#9627 , #9628
This PR introduce two essentials for Nereids
1. pattern match iterator used in memo
pattern match iterator is implemented by two iterators nested within each other: GroupExpressionIterator and GroupIterator.
GroupExpressionIterator use GroupIterator to get all children Plan which matching pattern and use them as children to generate pattern matched plan.
GroupIterator use GroupExpressionIterator to get all pattern matched Plan related to GroupExpressions in itself.
2. plan rewrite framework for memo
Rewrite framework is implemented by two jobs: RewriteTopDownJob and RewriteBottomUpJob
Both of them takes a group, a set of rules that need to be applied, and a context as construction parameters.
RewriteTopDownJob apply these jobs from top to down one by one.
RewriteBottomUpJob apply these jobs from bottom to up one by one.
When one rule rewrites plan tree at a plan node. This plan node will be applied all rules again until no rules can rewrite it.
This CL mainly changes:
1. Broker Load
When assigning backends, use user level resource tag to find available backends.
If user level resource tag is not set, broker load task can be assigned to any BE node,
otherwise, task can only be assigned to BE node which match the user level tags.
2. Routine Load
The current routine load job does not have user info, so it can not get user level tag when assigning tasks.
So there are 2 ways:
1. For old routine load job, use tags of replica allocation info to select BE nodes.
2. For new routine load job, the user info will be added and persisted in routine load job.
