Commit Graph

8276 Commits

Author SHA1 Message Date
570139e332 [fix][be] Delete uncivilized comments. (#10578) 2022-07-04 22:35:15 +08:00
1f1bdaa9c3 [bugfix] fix coredump of left anti join (#10591) 2022-07-04 22:29:41 +08:00
Pxl
e68ab0084b [bugfix]fix default value get wrong result because no implement read_by_rowids (#10582) 2022-07-04 19:30:49 +08:00
1cee0a7028 [feature-wip](multi-catalog) Modify the persist method about data source (#10523) 2022-07-04 18:24:14 +08:00
46bff6bba0 [fix](multi-catalog) fix the core dump on hms table (#10573)
In the funciton `TextConverter::write_vec_column`, it should execute the statement `nullable_column->get_null_map_data().push_back(0);` for every row.
Otherwise the null map will get error and cause the core dump.
2022-07-04 15:52:05 +08:00
88420deec1 [Bug][docs] Fix wrong links in README.md (#10394)
fix deadlink in README
2022-07-04 14:44:23 +08:00
e6f090e5bf [enhancement](Nereids)make nereids work (#10550)
Nereids could execute query: `select a from t;`

**enhancement**
- add a queriable interface for QueryStmt and LogicalPlanAdapter Temporarily
- refactor GroupId, GroupId extends doris.common.id now
- GroupId is generated by it's memo now, not global yet
- add varchar type
- Nereids enabled only when vectorized engine enabled

**fix**
- set output and column label to logicalPlanAdapter
- set output expression on root fragment
- set select partition and select index id to OlapScanNode
- BatchRulesJob add rule type mismatch
- add all implementation rules to rule set
- SlotReference get catalog column no longer returns null values
- bind star correctly
- implement `isNullable` in expressions

**known issue**
- could not do expression mapping(e.g. a + 1) on project node(wait intermediate tuple interface and project ability in ExecNode in be)
- aggregate do not work
- sort do not work
- filter do not work
- join do not work
2022-07-04 14:15:33 +08:00
9d4a9b95a4 [Build] fix the compile error with clang (#10570)
Co-authored-by: hucheng01 <hucheng01@baidu.com>
2022-07-04 11:13:17 +08:00
1a173a854e [fix](routine-load) Fix that routine load cannot work with old kafka version (#10554)
Co-authored-by: caiconghui1 <caiconghui1@jd.com>
2022-07-04 10:47:50 +08:00
d56d88d391 [improvement]No need to memset flags for vectorization predicates (#10564) 2022-07-04 10:23:08 +08:00
c5f85c9818 [community] modify release doc to remove incubator (#10574) 2022-07-04 10:18:23 +08:00
Pxl
0b251481d5 [Enhancement][Storage] refactor Comparison Predicates (#10380) 2022-07-04 09:22:27 +08:00
d6658f16d2 [chore][community](github)Change 'max-old-space-size' to 8192 (#10557) 2022-07-04 08:59:54 +08:00
7bfe438884 [BUG] fix bug in literal debug_string when literal is null (#10567) 2022-07-04 08:57:55 +08:00
b11e72b76b [chore] turn off java-udf by default when compiling in parallel (#10569) 2022-07-03 23:24:49 +08:00
91fca49df4 [feature-wip](unique-key-merge-on-write) add primary key index (#10529) 2022-07-03 23:20:30 +08:00
4e00584e40 [fix] fix api of table schema in http v2 (#10476)
schema in this api should be a list, just like in v1
2022-07-03 23:20:03 +08:00
8b6c46cfd1 [fix] fix create table like when having sequence column (#10464) 2022-07-03 23:19:46 +08:00
614b782d4d [feature](doris-on-es) Support es external table not assign schema (#9583) 2022-07-03 23:19:05 +08:00
aecf6e09a9 [fix] fix agg_memleak (#10571)
The previous code did not call 'destroy' to release the resource after the' create 'operation,
resulting in a memory leak. So I added Destroy
2022-07-03 20:22:26 +08:00
bfaa60b695 [fix](fe-ut) fix ut compile bug (#10562)
Introduced from #10306
2022-07-02 22:54:14 +08:00
848e0c5987 [fix](planner)infer predicate generate infered predicate using wrong information from another scope (#10519)
This PR fix a bug in predicate inference.

The original predicate inference compare two slot without SlotId. This will arise an error when a query has SetOperand and more than one SetOperand's child use same table alias. e.g.

```
select * from tb1 inner join tb2 on tb1.k1 = tb2.k1 
union
select * from tb1 inner join tb2 on tb1.k2 = tb2.k2 where tb1.k1 = 3;
```

in this case, we infer a predicate `tb2.k1 = 3` on table 'tbl2' of SetOperand's second child by mistake.
2022-07-02 22:41:04 +08:00
078cb3b4db [feature-wip](multi-catalog) end to end to support multi-catalog (#10521)
Get through the previous pull requests that support multi-catalog, and end to end to achieve multi-catalog.
2022-07-02 20:43:10 +08:00
c9f86bc7e2 [refactor] Refactoring Status static methods to format message using fmt(#9533) 2022-07-02 18:58:23 +08:00
143f519ae2 [website](doc)add package-lock.json to resolve docs build failure (#10558) 2022-07-02 17:20:11 +08:00
632ff01bbb [enhancement](Nereids) add post porcessor and error listener to prser (#10306)
add parser error listener and post processor to parser

error listener:
- throw exception when parser find unexpected syntax

post processor:
- throw exception when find error indent
- replace '``' with '`' in quoted identifier
- replace non reserved key word with normal identifier
2022-07-01 21:25:13 +08:00
3b3debf5a4 [build] Fix nested resource path error when as maven project from eclipse (#10427)
1. Fix nested resource path error when as maven project from eclipse
2. Add instructions of "Eclipse import FE as maven project" in developer guide
2022-07-01 18:03:54 +08:00
0401c04497 [chore] remove unused code for enable_lateral_view (#10438)
Co-authored-by: cambyzju <zhuxiaoli01@baidu.com>
2022-07-01 16:45:41 +08:00
5dd43fff53 [improvement] alloc aligned memory for block bloom filter to work with jemalloc (#10446) 2022-07-01 16:05:48 +08:00
558a21d7de [style](*): fix declarationOrder error by rearrange code (#10347) 2022-07-01 15:40:34 +08:00
f998c0b044 [Enhancement](Nereids) push down predicate through join (#10462)
Add filter operator to join children according to the predicate of filter and join, in order to achieving  predicate push-down

Pattern: 
```
      filter
         |
       join
      /     \
child    child
```

Transform:
```
      filter
         |
       join
      /     \
filter     filter
 |            |  
child     child
```
2022-07-01 15:39:01 +08:00
f164d094e8 [style](*): keep line break. (#10526) 2022-07-01 15:31:19 +08:00
e3d9f9430e [docs] Change the problem field in SSB test (#10540) 2022-07-01 15:30:35 +08:00
ca0906626f [BUG] fix bitmap function bug (#10502)
* fix bitmap function bug

* add regression test
2022-07-01 15:30:16 +08:00
97996c9275 [fix](Insert) fix 5 concurrent "insert...select..." OOM (#10501)
* [hotfix](dev-1.0.1) 5 concurrent insert...select... OOM

Co-authored-by: minghong <minghong.zhou@163.com>
Co-authored-by: yiguolei <yiguolei@gmail.com>
2022-07-01 15:29:26 +08:00
a44a222f76 [Feature](neireids) Add support of ProjectNode in PlanTranslator (#10499)
Since we will do the column prune with project node, so we need compact the project outputs to the PlanNode in PhysicalPlanTranslator::visitPhysicalProject

1. Add support for ProjectNode to make column prune available.
2. Add SortNode to PlanFragment when it is unpartitioned piggyback
2022-07-01 13:17:49 +08:00
d0b757c03a [bugfix](fe) fix add follower failed due to conflict socket . (#10429) 2022-07-01 11:12:36 +08:00
2708289816 [Doc]Change the download url for the binary package #10527 (#10528) 2022-07-01 10:47:14 +08:00
d52da675aa [docs](array-type)Fix keywords in array functions' help documents (#10500)
* save code

* save code
2022-07-01 10:43:04 +08:00
659e863bd7 [bugfix] fix tcmalooc hook cancel deadlock (#10514) 2022-07-01 10:41:59 +08:00
Pxl
a9d23ce337 [refactor] remove collator (#10518) 2022-07-01 10:35:32 +08:00
06e436b7cc [bugfix]dump_one_line failed to dump last column (#10522)
Co-authored-by: cambyzju <zhuxiaoli01@baidu.com>
2022-07-01 10:33:49 +08:00
6c2e76e39f [enhancement](proc) Support showing more details in show proc "/dbs" (#10471) 2022-07-01 10:27:38 +08:00
0d99b27d89 [fix] (mem tracker) Fix BE hangs at startup, stuck in tcmalloc hook call ExecEnv::GetInstance() (#10517)
1. Added flag exec_env_existed to indicate whether ExecEnv Instance is created.
2. ThreadMemTrackerMgr::add_tracker fail when USE_MEM_TRACKER=OFF, add USE_MEM_TRACKER compile option.
2022-07-01 10:20:53 +08:00
d43d3fc35f [improvement] modify comment " to ', to be compatible with mysql. (#10327) 2022-07-01 08:59:29 +08:00
Pxl
7571263ed7 [Bug][Vectorized] fix timestamp(datetime) not regist right (#10525) 2022-07-01 07:37:44 +08:00
aab7dc956f [refactor](load) Remove mini load (#10520) 2022-06-30 23:21:41 +08:00
18ad8ebfbb [improvement]Add reading by rowids to speed up lazy materialization (#10506) 2022-06-30 21:03:41 +08:00
aae619ef2c [feature-wip](nereids) Adjust plan execution flow and fix physical bugs (#10481)
Organize the plan process, improve the batch execution of rules and the way to add jobs.
Fix the problem that the condition in PhysicalHashJoin is empty.
2022-06-30 20:07:48 +08:00
77b1565b96 [feature](nereids) Support analyze for test SSB (#10415)
Follow-up #10241, this PR go through parse and analyze the SSB and add this functions:

1. support parse parenthesizedExpression
2. support analyze LogicalAggregate and LogicalSort
3. replace the functionCall to UnboundFunction and BoundFunction
4. support sum aggregate funciton
5. fix the dead loop in the ExpressionRewriter
6. refine some code
2022-06-30 16:26:39 +08:00