Commit Graph

18 Commits

Author SHA1 Message Date
9568de303a [Chore](build) update clang-format version check (#19542)
update clang-format version check
2023-05-11 19:38:58 +08:00
Pxl
9b7a419aed [Chore](build) update some doc about build enviroment (#19325)
update some doc about build enviroment
2023-05-10 16:18:44 +08:00
Pxl
ec517a53a8 [Chore](build) upgrade clang-format version to 16 && move thrift to fe-common (#19155)
upgrade clang-format version to 16
move thrift to fe-common
fix core dump on pipeline engine when operator canceled and not prepared
2023-04-28 14:14:51 +08:00
7bd6e32f85 [build-support](code-style): Add more config in IntelliJ-code-format.xml (#16420) 2023-02-06 12:06:27 +08:00
Pxl
484e9bcece [Chore](workflow) move clang-tidy to code-checks (#14758)
move clang-tidy to code-checks
2022-12-03 10:43:18 +08:00
b4101d46f0 [fix](workflow) Fix the errors when using sh to run shell scripts (#11898) 2022-08-19 21:28:52 +08:00
4fa53b4cdb [chore](workflow) Add shellcheck to check shell scripts (#11744) 2022-08-18 16:07:28 +08:00
7e3fc0d321 [enhancement](vec) Support outer join for vectorized exec engine (#11068)
Hash join node adds three new attributes.
The following will take an SQL as an example to illustrate the meaning of these three attributes

```
select t1. a from t1 left join t2 on t1. a=t2. b;
```
1. vOutputTupleDesc:Tuple2(a'')

2. vIntermediateTupleDescList: Tuple1(a', b'<nullable>)

2. vSrcToOutputSMap: <Tuple1(a'), Tuple2(a'')>

The slot in intermediatetuple corresponds to the slot in output tuple one by one through the expr calculation of the left child in vsrctooutputsmap.

This code mainly merges the contents of two PRs:
1.  [fix](vectorized) Support outer join for vectorized exec engine (https://github.com/apache/doris/pull/10323)
2. [Fix](Join) Fix the bug of outer join function under vectorization #9954

The following is the specific description of the first PR
In a vectorized scenario, the query plan will generate a new tuple for the join node.
This tuple mainly describes the output schema of the join node.
Adding this tuple mainly solves the problem that the input schema of the join node is different from the output schema.
For example:
1. The case where the null side column caused by outer join is converted to nullable.
2. The projection of the outer tuple.

The following is the specific description of the second PR
This pr mainly fixes the following problems:
1. Solve the query combined with inline view and outer join. After adding a tuple to the join operator, the position of the `tupleisnull` function is inconsistent with the row storage. Currently the vectorized `tupleisnull` will be calculated in the HashJoinNode.computeOutputTuple() function.
2. Column nullable property error problem. At present, once the outer join occurs, the column on the null-side side will be planned to be nullable in the semantic parsing stage.

For example:
```
select * from (select a as k1 from test) tmp right join b on tmp.k1=b.k1
```
At this time, the nullable property of column k1 in the `tmp` inline view should be true.

In the vectorized code, the virtual `tableRef` of tmp will be used in constructing the output tuple of HashJoinNode (specifically, the function HashJoinNode.computeOutputTuple()). So the **correctness** of the column nullable property of this tableRef is very important.
In the above case, since the tmp table needs to perform a right join with the b table, as a null-side tmp side, it is necessary to change the column attributes involved in the tmp table to nullable.

In non-vectorized code, since the virtual tableRef tmp is not used at all, it uses the `TupleIsNull` function in `outputsmp` to ensure data correctness.
That is to say, the a column of the original table test is still non-null, and it does not affect the correctness of the result.

The vectorized nullable attribute requirements are very strict.
Outer join will change the nullable attribute of the join column, thereby changing the nullable attribute of the column in the upper operator layer by layer.
Since FE has no mechanism to modify the nullable attribute in the upper operator tuple layer by layer after the analyzer.
So at present, we can only preset the attributes before the lower join as nullable in the analyzer stage in advance, so as to avoid the problem.
(At the same time, be also wrote some evasive code in order to deal with the problem of null to non-null.)

Co-authored-by: EmmyMiao87
Co-authored-by: HappenLee
Co-authored-by: morrySnow

Co-authored-by: EmmyMiao87 <522274284@qq.com>
2022-07-21 23:39:25 +08:00
4960043f5e [enhancement] Refactor to improve the usability of MemTracker (step2) (#10823) 2022-07-21 17:11:28 +08:00
aea41c2ffa support different python version in build-support scripts (#10378)
Co-authored-by: hucheng01 <hucheng01@baidu.com>
2022-07-11 23:23:45 +08:00
f164d094e8 [style](*): keep line break. (#10526) 2022-07-01 15:31:19 +08:00
46e84d39df [enhancement](style): polish the checkstyle (#9767)
* [enhancement](style): cancel auto tab for Javadoc.

* *: fix review
2022-05-30 16:10:04 +08:00
9f4291dcc9 [enhancement] (community) update intellij style file (#9783) 2022-05-29 17:34:25 +08:00
0c4b47756a [enhancement](community): enhance java style (#9693)
Enhance java style.

Now: checkstyle about code order is in this page--Class and Interface Declarations

This pr can make idea auto rearrange code
2022-05-20 15:24:30 +08:00
235d586f11 [style](fe) code correct rules and name rules (#9670)
* [style](fe) code correct rules and name rules

* revert some change according to comments
2022-05-19 16:36:03 +08:00
181aa1b4a9 [style]add IntelliJ code style xml (#9438) 2022-05-08 06:41:46 +08:00
e74bfea8e4 [chore](clang-format)(license-eye) Add Clang Format/Skywalking eyes github action (#7132)
1. The clang format action will be triggered when a PR is submitted.
2. Skywalking eyes actions will be triggered when a PR is submitted and after merging to master branch.
2021-11-24 10:41:02 +08:00
d7225d61ef [CodeFormat] Add clang-format script (#4934)
run build-support/check-format.sh to check cpp styles;
run build-support/clang-format.sh to fix cpp style issues;
2020-11-28 18:40:06 +08:00