database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
5,948 Commits 1 Branch 0 Tags
5c3367b67daec00ce933aa5289ef3c954535f653
Commit Graph

5948 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Dayue Gao
c09858671d [improvement][performance] improve lru cache resize performance and memory usage (#9521) 2022-05-19 23:37:59 +08:00
Zhengguo Yang
939daa07f1 [fix] fix Code Quality Analysis failed (#9685) 2022-05-19 23:13:47 +08:00
huangzhaowei
0f9ef26576 [Bug] Fix timestamp_diff issue when timeunit is year and month (#9574) 2022-05-19 21:24:43 +08:00
Shuangchi He
73c4ec7167 Fix some typos in be/. (#9681) 2022-05-19 20:55:39 +08:00
Shuangchi He
87e3904cc6 Fix some typos for docs. (#9680) 2022-05-19 20:55:21 +08:00
Stalary
cbc7b167b1 [Feature] cancel load support state (#9537) 2022-05-19 16:37:56 +08:00
Adonis Ling
119ff2c02d [enhancement] Improve debugging experience. (#9677) 2022-05-19 16:36:37 +08:00
morrySnow
235d586f11 [style](fe) code correct rules and name rules (#9670) 
* [style](fe) code correct rules and name rules

* revert some change according to comments
 2022-05-19 16:36:03 +08:00
Gabriel
7c2db79b73 [BUG] fix bug for vectorized compaction and some storage vectorization bug (#9610) 2022-05-19 16:35:15 +08:00
wudi
cbf1e20fbc [doc]update streamload 2pc doc (#9651) 
Co-authored-by: wudi <>
 2022-05-19 14:30:17 +08:00
EmmyMiao87
7a9bf5b23e [FeConfig](Project) Project optimization is enabled by default (#9667) 2022-05-19 14:03:14 +08:00
Gabriel
86b2c01e85 [refactor][regressiontest] reorder license header and import statement (#9672) 2022-05-19 14:00:33 +08:00
zy-kkk
dd5e9fa9a4 [doc] Fixed a error in the Bitmap Index section of the document (#9679) 2022-05-19 13:55:52 +08:00
Zhengguo Yang
3efe97e73c [website] fix doris website with no link to the Privacy Policy. (#9665) 
All websites must link to the Privacy Policy
 2022-05-18 22:49:49 +08:00
morrySnow
a3183ec45c [fix](planner) unnecessary cast will be added on children in CaseExpr sometimes (#9600) 
unnecessary cast will be added on children in CaseExpr because use symbolized equal to compare to `Expr`'s type.
it will lead to expression compare mistake and then lead to expression substitute failed when use `ExprSubstitutionMap`
 2022-05-18 22:44:51 +08:00
Gabriel
6602adf499 [regression test] Add compaction regression test case for different data models (#9660) 2022-05-18 17:12:20 +08:00
ZenoYang
bdaf0b3fcc [fix](storage) low_cardinality_optimize core dump when is null predicate (#9586) 
Issue Number: close #9555
Make the last value of the dictionary null, when ColumnDict inserts a null value,
add the encoding corresponding to the last value of the dictionary·
 2022-05-18 14:57:13 +08:00
yixiutt
c9ab5e22fe [fixbug](vec-load) fix core of segment_writer while it is not thread-safe (#9569) 
introduce in stream-load-vec #9280, it will cause multi-thread
operate to same segment_write cause BetaRowset enable multi-thread
of memtable flush, memtable flush call rowset_writer.add_block, it
use member variable _segment_writer to write, so it will cause
multi-thread in segment write.

Co-authored-by: yixiutt <yixiu@selectdb.com>
 2022-05-18 11:29:15 +08:00
morrySnow
94c89e8a37 [improment](planner) push down predicate past two phase aggregate (#9498) 
Push down predicate past aggregate cannot push down predicate past 2 phase aggregate.

origin plan is like this:
```
second phase agg (conjuncts on olap scan node tuples)
|
first phase agg
|
olap scan node
```
should be optimized to
```
second phase agg
|
first phase agg
|
olap scan node (conjuncts on olap scan node tuples)
```
 2022-05-18 10:09:39 +08:00
Hui Tian
682cc14182 [bug] (init) Java version check fail (#9607) 2022-05-18 07:47:03 +08:00
camby
bfb1ab059d [BUG] fix information_schema.columns results not correctly on vec engine (#9612) 
* VSchemaScanNode get_next bugfix

* add regression-test case for VSchemaScanNode

Co-authored-by: cambyzju <zhuxiaoli01@baidu.com>
 2022-05-18 07:44:32 +08:00
carlvinhust2012
b6f5c89f6c [regression test] add some case for json load regression test (#9614) 
Co-authored-by: hucheng01 <hucheng01@baidu.com>
 2022-05-18 07:43:51 +08:00
Pxl
26353ba8b5 [clang build]fix clang compile error (#9615) 2022-05-18 07:42:31 +08:00
jacktengg
908f9cb7b9 [Improvement][ASAN] make BE can exit normally and ASAN memory leak checking work (#9620) 2022-05-18 07:40:57 +08:00
Gabriel
4312ef93d7 [Improvement] reduce string size in serialization (#9550) 2022-05-17 22:38:34 +08:00
Mingyu Chen
7d9c25e718 [config] Remove some old config and session variable (#9495) 
1. Remove session variable "enable_lateral_view"
2. Remove Fe config: enable_materialized_view
3. Remove Fe config: enable_create_sync_job
4. Fe config dynamic_partition_enable is only used for disable dynamic partition scheduler.
 2022-05-17 22:37:11 +08:00
Mingyu Chen
2ba81899d0 [fix] fix bug that replica can not be repaired duo to DECOMMISSION state (#9424) 
Reset state of replica which state are in DECOMMISSION after finished scheduling.
 2022-05-17 22:36:30 +08:00
pengxiangyu
4ba75d3195 [feature] Add StoragePolicyResource for Remote Storage (#9554) 
Add StoragePolicyResource for Remote Storage
 2022-05-17 20:17:33 +08:00
Stalary
d95fe08458 [feature] group_concat support distinct (#9576) 2022-05-17 19:29:47 +08:00
Adonis Ling
ec2cd0083a [code format]Upgrade clang-format in BE Code Formatter from 8 to 13 (#9602) 2022-05-17 19:28:15 +08:00
zy-kkk
7417f9dfa3 [doc]modified the spark-load doc (#9605) 2022-05-17 19:27:02 +08:00
jiafeng.zhang
0aac9489ae [doc]add largeint doc (#9609) 
add largeint doc
 2022-05-17 19:26:45 +08:00
HappenLee
536d8ca1ed [Bug][Vectorized] Fix insert bimmap column with nullable column (#9408) 
Co-authored-by: lihaopeng <lihaopeng@baidu.com>
 2022-05-17 14:42:20 +08:00
HappenLee
1cc9653bd8 [Bug][Vectorized] Fix BE crash with delete condition and enable_storage_vectorization (#9547) 
Co-authored-by: lihaopeng <lihaopeng@baidu.com>
 2022-05-17 14:01:22 +08:00
Xin Liao
7d9fa04472 [fix](storage-vectorized) fix VMergeIterator core dump (#9564) 
It could be re appeared on rowset with many segment, it means segment overlap. Maybe could not reappear it easily.
 2022-05-17 11:58:59 +08:00
dujl
72e0042efb [feature-wip](hudi) Step1: Support create hudi external table (#9559) 
support create hudi table
support show create table for hudi table

### Design
1. create hudi table without schema(recommanded)
```sql
    CREATE [EXTERNAL] TABLE table_name
    ENGINE = HUDI
    [COMMENT "comment"]
    PROPERTIES (
    "hudi.database" = "hudi_db_in_hive_metastore",
    "hudi.table" = "hudi_table_in_hive_metastore",
    "hudi.hive.metastore.uris" = "thrift://127.0.0.1:9083"
    );
```

2. create hudi table with schema
```sql
    CREATE [EXTERNAL] TABLE table_name
    [(column_definition1[, column_definition2, ...])]
    ENGINE = HUDI
    [COMMENT "comment"]
    PROPERTIES (
    "hudi.database" = "hudi_db_in_hive_metastore",
    "hudi.table" = "hudi_table_in_hive_metastore",
    "hudi.hive.metastore.uris" = "thrift://127.0.0.1:9083"
    );
```
When create hudi table with schema, the columns must exist in corresponding table in hive metastore.
 2022-05-17 11:30:23 +08:00
yinzhijian
bee5c2f8aa [feature-wip](parquet-vec) Support parquet scanner in vectorized engine (#9433) 2022-05-17 09:37:17 +08:00
zy-kkk
7e8e14b3c6 [docs]Modifide flink-doris-connector.md (#9595) 2022-05-17 09:01:14 +08:00
Adonis Ling
5660815dc6 [chore] Fix compilation errors reported by clang (#9584) 2022-05-16 22:36:16 +08:00
morrySnow
c731e84341 [fix](planner)VecNotImplException thrown when query need rewrite and some slot cannot changed to nullable (#9589) 2022-05-16 22:34:02 +08:00
EmmyMiao87
9f9b666bc1 [Feature](Nereids) Data structure of comparison predicate (#9506) 
1. The data structure of the comparison expression
2. Refactored the inheritance and implementation relationship of tree node

```
        +-- ---- ---- ---+- ---- ---- ---- ---+- ---- ----- ---- ----TreeNode-----------------+
        |                |                    |                                               |
                                                                                              |
        |                |                    |                                               |
                                                                                              v
        v                v                    v                                           Abstract Tree Node
    Leaf Node        Unary Node          Binary Node                              +--------          ---------+
        |                |                    |                                   |        (children)         |
                                                                                  |                           |
        v                v                    v                                   v                           v
Leaf Expression   Unary Expression      Binary Expression              +------Expression----+           Plan Node
        |                |                    |                        |                    |
                                                                       |                    |
        |                |                    |                        v                    v
        |                |                    +- ---- ---- -----> Comparison Predicate     Named Expr
                                                                                       +----   -------+
        |                |                                                             v              v
        |                +- -- --- --- --- --- --- --- --- --- --- --- --- --- ---> Alias Expr      Slot
                                                                                                      ^
        |                                                                                             |
        |                                                                                             |
        +---- --- ---- ------ ---- ------- ------ ------- --- ------ ------ ----- ---- ----- ----- ---+
```
 2022-05-16 15:01:13 +08:00
zhangstar333
953429e370 [fix](function) fix last_value get wrong result when have order by clause (#9247) 2022-05-15 23:56:01 +08:00
Kang
e0c790094c [enhancement][betarowset]optimize lz4 compress and decompress speed by reusing context (#9566) 2022-05-15 21:18:32 +08:00
jiafeng.zhang
cfe22f8691 [Doc]Add show tables help documentation (#9568) 2022-05-15 10:18:33 +08:00
dataalive
1075648093 [doc]fix doc typo in data-model and date data type (#9571) 2022-05-15 10:17:46 +08:00
Stalary
9151cf717a ADD: 补充idea开发文档，添加help-resource.zip的生成步骤 (#9561) 2022-05-14 19:04:11 +08:00
jacktengg
3cfa83784e [bugfix](vectorized) vectorized write: invalid memory access caused by podarray resize (#9556) 2022-05-14 19:03:51 +08:00
yiguolei
cd105bee0a [refactor](es) Clean es tcp scannode and related thrift definitions (#9553) 
PaloExternalSourcesService is designed for es_scan_node using tcp protocol.
But es tcp protocol need deploy a tcp jar into es code. Both es version and lucene version are upgraded,
and the tcp jar is not maintained any more.

So that I remove all the related code and thrift definitions.
 2022-05-14 10:03:55 +08:00
EmmyMiao87
a9653f00bb [fix](lateral-view) Error view includes lateral view (#9530) 
Fixed #9529

When the lateral view based on a inline view which belongs to a view,
Doris could not resolve the column of lateral view in query.
When a query uses a view, it mainly refers to the string representation of the view.
That is, if the view's string representation is wrong, the view is wrong.
The string representation of the inline view lacks the handling of the lateral view.
This leads to query errors when using such views.
This PR mainly fixes the string representation of inline views.
 2022-05-14 09:57:08 +08:00
zhangstar333
fa6e4db4ca [fix](Function) fix case when function return null with abs function (#9493) 2022-05-14 09:50:45 +08:00