Commit Graph

8276 Commits

Author SHA1 Message Date
90e44fdd72 [typo](docs)Documentation 1.2 Feature version annotation (#14795) 2022-12-04 19:12:22 +08:00
3a3c43fe23 [typo](docs)add Doris version information to the head of array function (#14780)
[Docs]add Doris version information to the head of array function
2022-12-04 15:18:02 +08:00
bb2d6addaf [typo](docs)Upgrade 1.2, Java UDF function depends on the installation method (#14785) 2022-12-04 08:53:42 +08:00
97dcd2b13a [feature](nereids) merge proj-proj in post process (#14730)
* merge proj-proj

* v2this pr guarantees that the physical plan does not contains consecutive physical projects.
Like rewrite rule "merge projects", it works on physical plan, not logical plan.

* move merge-proj code into Project.java
2022-12-03 23:41:02 +08:00
7bb2343505 [chore](github) Optimize the workflow FE Code Style Checker (#14751)
We can reduce the times of checking out the codebase to boost the performance of the workflow.
2022-12-03 13:26:09 +08:00
Pxl
484e9bcece [Chore](workflow) move clang-tidy to code-checks (#14758)
move clang-tidy to code-checks
2022-12-03 10:43:18 +08:00
283b23f6da [fix](planner) wrong results when select from view which has with clause (#14747) 2022-12-02 18:10:52 +08:00
12304bc0ee [Pipeline](exec) Support pipeline exec engine (#14736)
Co-authored-by: Lijia Liu <liutang123@yeah.net>
Co-authored-by: HappenLee <happenlee@hotmail.com>
Co-authored-by: Jerry Hu <mrhhsg@gmail.com>
Co-authored-by: Pxl <952130278@qq.com>
Co-authored-by: shee <13843187+qzsee@users.noreply.github.com>
Co-authored-by: Gabriel <gabrielleebuaa@gmail.com>

## Problem Summary:

### 1. Design

DSIP: https://cwiki.apache.org/confluence/display/DORIS/DSIP-027%3A+Support+Pipeline+Exec+Engine

### 2. How to use:

Set the environment variable `set enable_pipeline_engine = true; `
2022-12-02 17:11:34 +08:00
505019e1dd [minor](decimal) degrade log level for overflow decimal (#14748) 2022-12-02 15:55:26 +08:00
5c7964d396 [minor](tools) delete unused script (#14752) 2022-12-02 15:20:01 +08:00
Pxl
c804024e5d [Chore](workflow) add clang-tidy workflow (#14737)
add clang-tidy workflow
2022-12-02 14:10:29 +08:00
15bd56cd43 [Docs]merge memory-management directory into maint-monitor (#14755) 2022-12-02 14:05:48 +08:00
7627defc88 [fix](regression-test) Add test data for test_mysql_jdbc_catalog and fix mysql-5.7.yaml about UTF8 (#14749)
Fix two things:
1. Fix that the MySQL table displays the garbled code even if the UTF8 is specified for table.
2. Fix that `test_mysql_jdbc_catalog.out` lack of returned data for table `ex_tb13`.
2022-12-02 11:58:11 +08:00
07e8af7808 [regression](test) add external regression-test base on emr environment 1.0 11-29 (#14666)
* add external regression-test base on emr environment 1.0 11-29

* delete ak sk info from regression-conf.groovy
2022-12-02 11:30:07 +08:00
ae6a007c4e [test](jdbc)add new extremum case (#14692) 2022-12-02 11:28:11 +08:00
4ef6efb59b [Docs](memory) fix document directory and fix memtracker accuracy bug (#14710) 2022-12-02 11:27:18 +08:00
f9d7983671 [fix](doris-demo)fix flink-connector demo maven dependency conflict (#14714) 2022-12-02 10:18:27 +08:00
e9799fab09 [refactor](datev2) refine function expr for datev2 (#14697)
* [refactor](datev2) refine function expr for datev2

* update
2022-12-02 10:13:11 +08:00
3dde97bff1 (compaction) opt compaction task producer and quick compaction (#13495) (#14535)
1.remove quick_compaction's rowset pick policy, call cu compaction when trigger
quick compaction
2. skip tablet's compaction task when compaction score is too small

Co-authored-by: yixiutt <yixiu@selectdb.com>
2022-12-02 10:07:44 +08:00
228e9ed01c [fix](improvement)(meta) fix alter catalog properties issues and reformat code (#14745)
1. fix NPE exception #14740
2. fix issue:
mysql> alter catalog xyz set properties ('hive.metastore.uris'='thrift://172.21.0.1:7004');
ERROR 1105 (HY000): errCode = 2, detailMessage = Can't modify the type of catalog property with name: xyz
3. change behavior. The original logic is use props in set properties clause to replace all exists props, now change to only replace the listed props in set properties clause, and new props will be added. Make it behavior like alter table property stmt.
2022-12-02 09:34:13 +08:00
5b29489c7f (tcmalloc) gc does not work in somecases (#14732)
gc does not work in some cases
2022-12-02 09:18:23 +08:00
f484bc5380 [github](ci) change required checks of branch 1.1 (#14743)
* [github](branch-1.1) change required checks of branch 1.1

* modify
2022-12-02 09:11:49 +08:00
e5000c708e [feature](statistics) Support for collecting statistics on materialized view (#14676)
1. Map muiltiple tasks to one Job
2. Remove the codes for analyzing whole default db, since this feature is not available and would create too many tasks and related code is confusing
3. support analyze materialized view
4. abstract the common logic to BaseTask
2022-12-01 22:34:13 +08:00
94a6ffb906 [feature](compaction) support vertical_compaction & ordered_data_compaction (#14524) 2022-12-01 22:15:41 +08:00
2be8235d95 [feature](nereids) support timestampdiff function (#14662)
complete timeStampDiff
supported timeunit:
 - YEAR 
 - MONTH
 - WEEK
 - DAY
 - HOUR
 - MINUTE
 - SECOND
2022-12-01 22:11:55 +08:00
9a01716a57 [chore](fix)build docs fix #14727 2022-12-01 21:57:29 +08:00
14e208354d [Feature](Nereids) support nereids event for logging the cascades states and transformation. (#13659)
Add an event producer, channel, consumer system to support the feature as title and you can turn it on using set
enable_nereids_event = true;
For more information, please see fe/fe-core/src/main/java/org/apache/doris/nereids/metrics/README.md
2022-12-01 21:42:40 +08:00
5ca6596ca3 [fix](tpch-tools) disable join reorder for Q12 in TPC-H tools (#14728) 2022-12-01 20:55:17 +08:00
302da03b18 [enhancement](Nereids): Use long bitmap in DPHyp (#14725) 2022-12-01 20:47:45 +08:00
ba9a777554 [fix](function) StringRef should not be key of timezone cache (#14719) 2022-12-01 16:31:47 +08:00
9dd1d989e8 [test](decimalv3) add regression test cases for decimalv3 (#14672) 2022-12-01 15:18:40 +08:00
f496d1972a [improvement](multi-catalog) return root cause of exception (#14708) 2022-12-01 14:58:05 +08:00
176f519fa1 [enhancement](memtracker) Optimize exec node memory tracking (#14711) 2022-12-01 14:52:21 +08:00
b4d32a0c44 [fix](join) runtime filter shared from other instance wasn't be published (#14717) 2022-12-01 14:17:23 +08:00
Pxl
bba77fa9dd [Enhancement](profile) enhance column predicates display on profile (#14664) 2022-12-01 13:07:12 +08:00
3c6b96b9be [enhancement](Nereids) avoid add project that output same with child to memo (#14180) 2022-12-01 10:49:44 +08:00
7873bc95a6 [Enhancement](bitmapfilter) Support bitmap filter to apply zone_map index to filter pages (#14635) 2022-12-01 10:41:09 +08:00
ce9a160d16 [enhancement](macOS) Make CLion work out of the box (#14689)
We can't build the project after import it to CLion on macOS. Some options must be provided by default.
2022-12-01 10:40:04 +08:00
12791f1c79 [opt](docs) Add select except usage in select doc (#14696) 2022-12-01 10:06:17 +08:00
2a3a758c75 [doc](community) update release-verify doc when gpg import (#14706) 2022-12-01 10:04:58 +08:00
6c70d794f6 [fix](bitmapfilter) fix core dump caused by bitmap filter (#14702) 2022-12-01 09:56:22 +08:00
36737fe9f4 [feature](Nereids): Add cache to avoid repeatly calculation in DPhyp (#14585) 2022-11-30 21:35:45 +08:00
9bbbcf031c [enhancement](k8s) Support fqdn mode for be in k8s enviroment (#9172)
In the k8s environment, the ip of the pod can be changed, but the hostname of pod is stable. When the host machine of the pod fails, the k8s can schedule the failed pod to the new host machine for reconstruction. After that, the newly created pod's hostname remains unchanged, and the ip address has been changed. The change of the be node's ip address can be detected by FQDNManager when enable_fqdn_mode is true

Co-authored-by: caiconghui1 <caiconghui1@jd.com>
2022-11-30 20:42:15 +08:00
80baca2643 [Docs](memory) Admin-manual adds mem tracker, memory exceeds limit, OOM analysis (#14419) 2022-11-30 18:02:05 +08:00
738c36109f rename tpch dir (#14668) 2022-11-30 17:59:13 +08:00
11735043d6 [improvement](test) logging load result (#14694)
When a load failed, we have to login to doris to investigate result.
2022-11-30 16:57:35 +08:00
593a916ae6 [feature](nereids) split AggregateDisassemble into two rules (#14611)
# Proposed changes

Issue Number: close #14280

## Problem summary

The AggregateDisassemble rule is refactored and split into two rules, which are not dependent on each other.
1. AggregateDisassemble splits the agg into two phases: Local, Global.
1.1. For count function, the implementation is as follows:distinct_multi_count(update)+ distinct_multi_count(merge)

2. DistinctAggregateDisassemble splits the agg into 4 stages: Local, Global, Distinct Local, Distinct GLobal.
2.1. For count function, the implementation is as follows:distinct_multi_count(update)+ distinct_multi_count(merge)+sum(update)+ sum(merge)
2022-11-30 14:02:42 +08:00
79688a54d6 [bug](jsonb) fix be core at insert invalid json to JSONB column (#14686) 2022-11-30 14:00:50 +08:00
f3cf83a933 (fix)[test] add some logs (#14695) 2022-11-30 12:45:12 +08:00
3ca3af2234 [improvement](planner)sort show catalogs result by name (#14684)
Result of show databases, show tables, show data are all sorted by name, so make show catalogs behavior same.
2022-11-30 11:55:14 +08:00