Commit Graph

8276 Commits

Author SHA1 Message Date
d9f2da8cf0 [bugfix] temporarily disable RF code to avoid core dump caused by vexpr destruction (#10504)
Runtime filter handling in volap_scann_ode may cause double free in VExprContext, temporarily disable it to avoid it.
2022-06-30 14:54:44 +08:00
620faf4959 [feature-wip](multi-catalog) add auth&catalog check (#10480)
This pr follows up [#10435](https://github.com/apache/doris/pull/10435). [#10435](https://github.com/apache/doris/pull/10435) had supported catalog in sql syntax, but some doris statements are only valid in internal catalog. In order to remind users of the scope of catalog usage, it is necessary to throw errors to exceptions of using catalog in the analyze phase.

## How does it effect origin behavior
It is fully compatible with the previous sql statements. Meanwhile, if using the internal catalog in the statements that all the usage of the internal catalog, the syntax is still valid, but using the external catalog will directly throw errors. For example:
```
MySQL [(none)]> show data from tpch10.lineitem;
+-----------+-----------+------------+--------------+----------+
| TableName | IndexName | Size       | ReplicaCount | RowCount |
+-----------+-----------+------------+--------------+----------+
| lineitem  | lineitem  | 210.809 MB | 32           | 6001215  |
|           | Total     | 210.809 MB | 32           |          |
+-----------+-----------+------------+--------------+----------+

MySQL [(none)]> show data from internal_catalog.tpch10.lineitem;
+-----------+-----------+------------+--------------+----------+
| TableName | IndexName | Size       | ReplicaCount | RowCount |
+-----------+-----------+------------+--------------+----------+
| lineitem  | lineitem  | 210.809 MB | 32           | 6001215  |
|           | Total     | 210.809 MB | 32           |          |
+-----------+-----------+------------+--------------+----------+

MySQL [(none)]> show data from hive.tpch10.lineitem;
ERROR 1105 (HY000): errCode = 2, detailMessage = External catalog 'hive' is not allowed in 'ShowDataStmt'
```
2022-06-30 12:04:23 +08:00
468040974e [compile]Update init-env.sh (#10451) 2022-06-30 11:28:06 +08:00
d259770b86 [Fix] avoid core dump cause by malformed bitmap type data (#10458) 2022-06-30 11:27:22 +08:00
c62c2e308f [chore]replace checkstyle action with mvn checkstyle:check (#10474) 2022-06-30 11:20:50 +08:00
ec6620ae3e [feature-wip](array-type) add function arrays_overlap (#10233) 2022-06-30 08:12:29 +08:00
73999feca7 [doc] mod alter-table-replace (#10324)
Modify alter-table-replace to alter-table-replace-column, move alter-table-replace to data-definition-statements.
2022-06-30 08:11:59 +08:00
Pxl
2c35abe940 [bugfix]fix core dump on outfile with expr (#10491)
remove log
2022-06-29 20:38:49 +08:00
e42adbb959 Fix compilation error reported by clang (#10494) 2022-06-29 20:38:06 +08:00
4ec6e3ee81 [refactor] Remove debug action since it is never used. (#10484)
Co-authored-by: yiguolei <yiguolei@gmail.com>
2022-06-29 20:37:51 +08:00
9b6ed1525d [fix](nereids) extractPlan should use physical expressions but is logical expressions actually (#10483)
extractPlan should use physical expressions but is logical expressions actually
2022-06-29 19:38:10 +08:00
c695ccb827 [fix](proc) Fix show proc '/current_query_stmts' error due to wrong index for execTime (#10488)
Co-authored-by: caiconghui1 <caiconghui1@jd.com>
2022-06-29 17:41:47 +08:00
Pxl
b9223d1657 [Build] fix build fail on clang (#10445) 2022-06-29 15:08:24 +08:00
Pxl
8e713ddfcf [Feature] [Vectorized] support vectorized schema-change (#10187) 2022-06-29 15:07:21 +08:00
Pxl
6a566ccb74 [Enhancement][Vectorized] add constexpr_loop_match (#10283) 2022-06-29 14:58:50 +08:00
d1055eacb4 [feature-wip](array-type) Use uppercase to describe columns with array type (#10193)
Use uppercase to describe columns with array type.
2022-06-29 14:07:27 +08:00
5f73668626 [log] add more error info for hdfs reader writer (#10475) 2022-06-29 12:02:27 +08:00
abd10f0f3e [feature-wip](multi-catalog) Impl FileScanNode in be (#10402)
Define a new file scanner node for hms table in be.
This file scanner node is different from broker scan node as blow:
1. Broker scan node will define src slot and dest slot, there is two memory copy in it: first is from file to src slot
    and second from src to dest slot. Otherwise FileScanNode only have one stemp memory copy just from file to dest slot.
2. Broker scan node will read all the filed in the file to src slot and FileScanNode only read the need filed.
3. Broker scan node will convert type into string type for src slot and then use cast to convert to dest slot type,
    but FileScanNode will have the final type.

Now FileScanNode is a standalone code, but we will uniform the file scan and broker scan in the feature.
2022-06-29 11:04:01 +08:00
9aa800141d [fix](ut)(nereids) the check bound function lacks recursive processing (#10357) 2022-06-29 10:40:13 +08:00
8cbdbb5658 [Enhancement] a better vec version for count_zero_num (#10472) 2022-06-29 10:26:42 +08:00
deeb3028ad [Enhancement] [Memory] [Vectorized] Stress test and optimize memory allocation (#9581)
* vec stress test, Allocator introduce chunkallocator

* fix comment
2022-06-29 02:57:51 +08:00
0a36c34326 [feature](nereids) costAndEnforcerJob interim solution (#10468)
In order to complete the ssb test, temporarily increase the implementation of costAndEnforcerJob, and create an OptimizeGroupjob for all children of the group.
2022-06-28 18:45:23 +08:00
f5936aa7ce [enhancement](Nereids): add more implmentation rules. (#10335)
Add more implmentation rules.

Current some `logical` and `physical` operator is different. I change some code to make them match.

Implementation
- Sort:only heap sort
- Agg
- OlapScan
2022-06-28 17:08:33 +08:00
7898c818e9 Revert "[improvement]Do not lazily read dict encoded columns (#10420)" (#10466)
Reason:
1. Some queries performance degradation
2. Coredump bug: #10419

This reverts commit 904e7576797c796b809823647a769bc1d4569115.
2022-06-28 15:43:48 +08:00
17eb8c00d3 [feature] add table valued function framework and numbers table valued function (#10214) 2022-06-28 14:01:57 +08:00
2f30c7cf03 [doc] Update the URL download link address in Quick Start (#10457) 2022-06-28 13:53:07 +08:00
1f2bf39140 [feature-wip](multi-catalog) get catalog name from TableName (#10435) 2022-06-28 10:42:37 +08:00
498a80547c [fix](fe-ut) fix fe ut and build.sh bug (#10432) 2022-06-27 19:01:05 +08:00
904e757679 [improvement]Do not lazily read dict encoded columns (#10420) 2022-06-26 22:08:48 +08:00
ca94867b4e [Feature-wip] add date v2 type (#9916) 2022-06-26 16:07:56 +08:00
HB
f6ef1aad5c [fix](backup) fix mkdir failed (#10422) (#10423) 2022-06-26 09:55:48 +08:00
4408231765 [fix](random-distribution) Make aggregate keys table with replace type columns and unique keys table can only have hash distribution to make data computing correctly (#10414) 2022-06-26 09:52:16 +08:00
79ad05eec6 [fix](doe) fix doe on es v8 (#10391)
doris on es8 can not work, because type change. The use of type is no longer recommended in es7,
and support for type has been removed from es8.

1. /_mapping not support include_type_name
2. /_search not support use type
2022-06-26 09:51:29 +08:00
fba0051ed3 [doc] Fix Slack invitation link on README.md not working (#10406) 2022-06-26 01:23:07 +08:00
a0e330a156 [website] add website external resource (#10416) 2022-06-26 01:22:14 +08:00
Pxl
4750e94746 set default do not build benchmark-tool && and use lld/gold (#10215) 2022-06-25 22:31:11 +08:00
69134fb3f2 [docs] Add regression test for weekday function (#10253) 2022-06-25 22:29:32 +08:00
eebfbd0c91 Revert "[fix](vectorized) Support outer join for vectorized exec engine (#10323)" (#10424)
This reverts commit 2cc670dba697a330358ae7d485d856e4b457c679.
2022-06-25 22:18:08 +08:00
7fe4b20da3 [feature-wip](multi-catalog) refactor catalog interface (#10320) 2022-06-25 21:51:54 +08:00
f12b22a51e [Bug][Vectorized] Fix cord dump of BloomFilter not support DATE type (#10417) 2022-06-25 21:29:32 +08:00
4ca257a1cd [improvement] Modify the default value of doris_scan_range_max_mb (#10232)
* [improvement] Modify the default value of doris_scan_range_max_mb

* fix regression-test
2022-06-25 19:48:49 +08:00
14a9a676e7 [BUG] fix DCHECK failed (#10396) 2022-06-25 17:08:40 +08:00
eb25df5a2c [fix] (mem tracker) Fix inaccurate mem tracker leads to load OOM (#10409)
* fix load tracker

* fix comment
2022-06-25 14:13:02 +08:00
8abd00dcd5 [feature-wip](multi-catalog) Add catalog name to information schema. (#10349)
Information schema database need to show catalog name after multi-catalog is supported.
This part is step 1, add catalog name for schemata table.
2022-06-25 11:53:04 +08:00
3757bd521a [feature](nereids)Convert the expression from nereids to stale expr. (#10343)
Add ExpressionConverter.java to convert the expression from nereids to stale expression
2022-06-25 11:16:52 +08:00
7921320124 [fix]Make sure only call once set_dict_encoding_type for each ColumnReader (#10389) 2022-06-25 04:31:19 +08:00
df908873bb [improvement]Use std::iota to set values of _block_rowids in SegmentIterator::_read_columns_by_index (#10386) 2022-06-25 04:30:23 +08:00
89860fd0e3 [opt] delete the redundant parameter of _execute_non_nullable (#10173)
1. This pr is used to delete the redundant parameter of _execute_non_nullable.
2. This modification will not affect the function "element_at".
2022-06-24 19:22:50 +08:00
476be35961 [TYPO] fix typo 'destory' -> 'destroy' (#10373) 2022-06-24 19:11:28 +08:00
e82d8433be [feature](nereids) Integrate nereids into current SQL process framework (#10304)
To integrate the nereids optimizer with new SQLParser and Planner to existing SQL process framework, I abstract a interface which named "Planner" and let the Both planner from nereids and stale optimizer implement it, to disguish it
with origin Planner, I rename the Planner to OriginalPlanner.

As we don't want to impact the existing logic too much, I defined a LogicalPlanAdapter to adapt the logicalPlan that is the output of the new paser to the existing code.

Besides, as the MySQL protocol supports sending multiple statements in one packet, so I add Nereids#SparseSQL method to handle this properly.
2022-06-24 18:26:26 +08:00