PaloExternalSourcesService is designed for es_scan_node using tcp protocol.
But es tcp protocol need deploy a tcp jar into es code. Both es version and lucene version are upgraded,
and the tcp jar is not maintained any more.
So that I remove all the related code and thrift definitions.
Fixed#9529
When the lateral view based on a inline view which belongs to a view,
Doris could not resolve the column of lateral view in query.
When a query uses a view, it mainly refers to the string representation of the view.
That is, if the view's string representation is wrong, the view is wrong.
The string representation of the inline view lacks the handling of the lateral view.
This leads to query errors when using such views.
This PR mainly fixes the string representation of inline views.
commit #9123 introduce the bug. bitshuffle page return error when
page is full, so scalar column write cannot switch to next page, which make
ordinal index is null when flush.
All page builder should return ok when page full, and column writer procedure
shoud be append_data, check is_page_full, switch to next page
Co-authored-by: yixiutt <yixiu@selectdb.com>
Issue Number: close#9403
set below rules' severity to error and format code according check info.
a. Merge conflicts unresolved
b. Avoid using corresponding octal or Unicode escape
c. Avoid Escaped Unicode Characters
d. No Line Wrap
e. Package Name
f. Type Name
g. Annotation Location
h. Interface Type Parameter
i. CatchParameterName
j. Pattern Variable Name
k. Record Component Name
l. Record Type Parameter Name
m. Method Type Parameter Name
n. Redundant Import
o. Custom Import Order
p. Unused Imports
q. Avoid Star Import
r. tab character in file
s. Newline At End Of File
t. Trailing whitespace found
binlog load Because txn exceeds the default value, resume is a failure,
and a friendly prompt message is given to the user, instead of prompting success now,
it still fails after a while, and the user will feel inexplicable
Issue Number: close#9468
Current fe check style check all files. But some rules should be only applied on production files.
Add suppressions to suppress some rules on test files.
Describe the overview of changes.
- rename GroupExpression
- use `HashSet<GroupExpression> groupExpressions` in `memo`
- add label of `Nereids` for CI
- remove `GroupExpr` from Plan
There are many places in FE where a group of BE nodes needs to be selected according to certain requirements. for example:
1. When creating replicas for a tablet.
2. When selecting a BE to execute Insert.
3. When Stream Load forwards http requests to BE nodes.
These operations all have the same logic. So this CL mainly changes:
1. Create a new `BeSelectionPolicy` class to describe the set of conditions for selecting BE.
2. The logic of selecting BE nodes in `SystemInfoService` has been refactored, and the following two methods are used uniformly:
1. `selectBackendIdsByPolicy`: Select the required number of BE nodes according to the `BeSelectionPolicy`.
2. `selectBackendIdsForReplicaCreation`: Select the BE node for the replica creation operation.
Note that there are some changes here:
For the replica creation operation, the round-robin method was used to select BE nodes before,
but now it is changed to `random` selection for the following reasons:
1. Although the previous logic is round-robin, it is actually random.
2. The final diff of the random algorithm will not be greater than 5%, so it can be considered that the random algorithm
can distribute the data evenly.
This patch supports utf8mb4 for mysql external table.
if someone needs a mysql external table with utf8mb4 charset, but only support charset utf8 right now.
When create mysql external table, it can add an optional propertiy "charset" which can set character fom mysql connection,
default value is "utf8". You can set "utf8mb4" instead of "utf8" when you need.
1. Fix LoadTask, ChunkAllocator, TabletMeta, Brpc, the accuracy of memory track.
2. Modified some MemTracker names, deleted some unnecessary trackers, and improved readability.
3. More powerful MemTracker debugging capabilities.
4. Avoid creating TabletColumn temporary objects and improve BE startup time by 8%.
5. Fix some other details.
This pr provide a new pattern matching framework for Nereids optimizer.
The new pattern matching framework contains this concepts:
1. `Pattern`/`PatternDescriptor`: the tree node's multiple hierarchy shape, e.g. `logicalJoin(logicalJoin(), any()` pattern describe a plan that root is a `LogicalJoin` and the left child is `LogicalJoin` too.
2. `MatchedAction`: a callback function when the pattern matched, usually you can create new plan to replace the origin matched plan.
3. `MatchingContext`: the param pass through MatchedAction, contains the matched plan root and the PlannerContext.
4. `PatternMatcher`: contains PatternDescriptor and MatchedAction
5. `Rule`: a rewrite rule contains RuleType, PatternPromise, Pattern and transform function(equals to MatchedAction)
6. `RuleFactory`: the factory can help us build Rules easily. RuleFactory extends Patterns interface, and have some predefined pattern descriptors.
for example, Join commutative:
```java
public class JoinCommutative extends OneExplorationRuleFactory {
@Override
public Rule<Plan> build() {
return innerLogicalJoin().thenApply(ctx -> {
return new LogicalJoin(
JoinType.INNER_JOIN,
ctx.root.getOnClause(),
ctx.root.right(),
ctx.root.left()
);
}).toRule(RuleType.LOGICAL_JOIN_COMMUTATIVE);
}
}
```
the code above show the three step to create a Rule
1. 'innerLogicalJoin()' declare pattern is an inner logical join. 'innerLogicalJoin' is a predefined pattern.
2. invoke 'thenApply()' function to combine a MatchedAction, return a new LogicalJoin with exchange children.
3. invoke 'toRule()' function to convert to Rule
You can think the Rule contains three parts:
1. Pattern
2. transform function / MatchedAction
3. RuleType and RulePromise
So
1. `innerLogicalJoin()` create a `PatternDescriptor`, which contains a `Pattern`
2. `PatternDescriptor.then()` convert `PatternDescriptor` to `PatternMatcher,` witch contains Pattern and MatchedAction
3. `PatternMatcher.toRule()` convert `PatternMatcher` to a Rule
This three step inspired by the currying in function programing.
It should be noted, #9446 provide a generic type for TreeNode's children, so we can infer multiple hierarchy type in this pattern matching framework, so you can get the really tree node type without unsafely cast. like this:
```java
logicalJoin(logicalJoin(), any()).then(j -> {
// j can be inferred type to LogicalJoin<LogicalJoin<Plan, Plan>, Plan>
// so j.left() can be inferred type to LogicalJoin<Plan, Plan>,
// so you don't need to cast j.left() from 'Plan' to 'LogicalJoin'
var node = j.left().left();
})
```
Function round support two format round(double) and round(double, int), the argument is variadic.
But FunctionBinaryArithmetic not support variadic argument now, make get_function for round(double, int) failed.
reproduce steps:
1. set enable_vectorized_engine=true;
2. try to call round(double, int);
```
> select round(10.12345,2);
ERROR 1105 (HY000): errCode = 2, detailMessage = Function round is not implemented
```
