database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
13,073 Commits 1 Branch 0 Tags
0318da1df3661663cd0affdf6c073e686eb99dfe
Commit Graph

13073 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
TengJianPing
d30bb8042e [fuzzy](hash join) disable fuzzy enable_hash_join_early_start_probe (#23413) 2023-08-25 10:11:20 +08:00
zclllyybb
9cacf9535a [Opt](functions) Use preloaded cache to accelerate timezone parsing (#22694) 
* opt

* bugfix

* fix ut

* fix stylecheck
 2023-08-25 10:00:48 +08:00
zclllyybb
7cfb3cc0aa [fix](functions) fix function substitute for datetimeV1/V2 (#23344) 
* fix

* function fe
 2023-08-25 09:59:38 +08:00
amory
467e48053b [FIX](regresstest) fix regress test for export with scp files (#23417) 
* fix regress test for export with scp

* fix outfile and outputfilepath
 2023-08-25 09:22:34 +08:00
Dongyang Li
bc3d397759 [fix](case) update .out file, relate to #23272 (#23455) 
Co-authored-by: stephen <hello-stephen@qq.com>
 2023-08-25 09:15:27 +08:00
Dongyang Li
e44f14507d [github](action) rm required tag for 'BE UT (Clang)' and 'BE UT (macOS)' #23422 
These two checks often cost about 2 hours, which slowed down the CI process,
After discussion， maybe we can rm the required tag first, then set compile with clang in BE UT from TeamCity.
 2023-08-25 09:12:20 +08:00
daidai
ceb931c513 [regression-test](hdfs_tvf)append regression test that hdfs_tvf read compression file (#23454) 2023-08-25 09:00:21 +08:00
Siyang Tang
441a9fff6d [fix](planner) fix now function param type error (#23446) 2023-08-25 00:12:21 +08:00
Kaijie Chen
71071ba057 [feature](move-memtable)[4/7] add stream sink file writer (#23416) 
Co-authored-by: laihui <1353307710@qq.com>
 2023-08-25 00:08:27 +08:00
Kaijie Chen
98d0a2f6c1 [feature](move-memtable)[3/7] add load stream manager and rpc service (#23415) 
Co-authored-by: zhengyu <freeman.zhang1992@gmail.com>
Co-authored-by: Yongqiang YANG <dataroaring@gmail.com>
Co-authored-by: laihui <1353307710@qq.com>
 2023-08-25 00:08:04 +08:00
Kaijie Chen
b65023e667 [fix](load) avoid using protobuf set_allocated_id in VDataStreamSender (#23435) 2023-08-25 00:07:09 +08:00
zhangdong
6a4976921d [fix](auth)Disable column auth temporarily (#23295) 
- add config `enable_col_auth` to temporarily disable column permissions(because old/new planner has bug when select from view)
- Restore the old optimizer to the previous authentication method
- Support for new optimizer authentication（Legacy issue: When querying the view, the permissions of the base table will be authenticated. The view's own permissions should be authenticated and processed after the new optimizer is improved）
- fix: show grants for non-existent users
- fix: role:`admin` can not grant/revoke to/from user
 2023-08-24 23:37:06 +08:00
Jibing-Li
966561a6ed [improvement and fix](statistics)Load the cache for external table row count while init table (#23170) 
1. Load the cache for external table row count while init table, this could avoid no row number stats for the very first time to run an sql.
2. Show cardinality for an external scan node when explain the sql.
3. fix bugs introduced by https://github.com/apache/doris/pull/22963
 2023-08-24 23:34:16 +08:00
Tiewei Fang
f6c5c8f7b5 [Fix](Nereids) fix that select...from tablets() are invalidated when there exists predicates (#23365) 
Problem: `select...from tablets()` are invalidated when there exists predicates, such as:
```sql
// The all data is:
mysql> select * from student3;
+------+------+------+
| id   | name | age  |
+------+------+------+
|    1 | ftw  |   18 |
|    3 | yy   |   19 |
|    4 | xx   |   21 |
|    2 | cyx  |   20 |
+------+------+------+

// when we specified tablet to read:
mysql> select * from student3 tablet(131131);
+------+------+------+
| id   | name | age  |
+------+------+------+
|    1 | ftw  |   18 |
|    3 | yy   |   19 |
+------+------+------+

// Howerver, when there exists predicates, the `tablet(131131)` is invalidated
mysql> select * from student3 tablet(131131) where id > 1;
+------+------+------+
| id   | name | age  |
+------+------+------+
|    4 | xx   |   21 |
|    3 | yy   |   19 |
|    2 | cyx  |   20 |
+------+------+------+
```

After the fix, we get promising data
```sql
mysql> select * from student3 tablet(131131) where id > 1;
+------+------+------+
| id   | name | age  |
+------+------+------+
|    3 | yy   |   19 |
+------+------+------+
```
 2023-08-24 23:29:59 +08:00
Qi Chen
caddcc6215 [Fix](orc-reader) Fix decimal type check for ColumnValueRange issue and use primitive_type. (#23424) 
Fix decimal type check for ColumnValueRange issue and use primitive_type in orc_reader. Because in #22842 the `CppType` of `PrimitiveTypeTraits<TYPE_DECIMALXXX> ` were changed.
 2023-08-24 23:26:41 +08:00
Gabriel
55e572df82 [pipelineX](analytic operator) Support analytic operator (#23444) 2023-08-24 23:05:29 +08:00
KassieZ
adafcc1f7a [typo](docs) Fix Sidebar Category Name of CN Version and Capitalize Files Name (#23050) 2023-08-24 22:56:23 +08:00
starocean999
320eda78e6 [fix](nereids) remove useless cast in in-predicate (#23171) 
consider sql "select * from test_simplify_in_predicate_t where a in ('1992-01-31', '1992-02-01', '1992-02-02', '1992-02-03', '1992-02-04');"
before:

```
|   0:VOlapScanNode                                                                                                                                                                                      |
|      TABLE: default_cluster:bugfix.test_simplify_in_predicate_t(test_simplify_in_predicate_t), PREAGGREGATION: OFF. Reason: No aggregate on scan.                                                      |
|      PREDICATES: CAST(a[#0] AS DATETIMEV2(0)) IN ('1992-01-31 00:00:00', '1992-02-01 00:00:00', '1992-02-02 00:00:00', '1992-02-03 00:00:00', '1992-02-04 00:00:00') AND __DORIS_DELETE_SIGN__[#1] = 0 |
|      partitions=0/1, tablets=0/0, tabletList=                                                                                                                                                          |
|      cardinality=1, avgRowSize=0.0, numNodes=1                                                                                                                                                         |
|      pushAggOp=NONE                                                                                                                                                                                    |
|      projections: a[#0]                                                                                                                                                                                |
|      project output tuple id: 1                                                                                                                                                                        |
|      tuple ids: 0  
```
after:

```
|   0:VOlapScanNode                                                                                                                                 |
|      TABLE: default_cluster:bugfix.test_simplify_in_predicate_t(test_simplify_in_predicate_t), PREAGGREGATION: OFF. Reason: No aggregate on scan. |
|      PREDICATES: a[#0] IN ('1992-01-31', '1992-02-01', '1992-02-02', '1992-02-03', '1992-02-04') AND __DORIS_DELETE_SIGN__[#1] = 0                |
|      partitions=0/1, tablets=0/0, tabletList=                                                                                                     |
|      cardinality=1, avgRowSize=0.0, numNodes=1                                                                                                    |
|      pushAggOp=NONE                                                                                                                               |
|      projections: a[#0]                                                                                                                           |
|      project output tuple id: 1                                                                                                                   |
|      tuple ids: 0  

```
 2023-08-24 18:14:43 +08:00
amory
6c5072ffc5 [FIX](array-func) fix array index func with decimal (#23399) 
fix array index func with decimal
in old analyzer when sql with array_position or array_contains with decimal , may loss precision to which will make result wrong
 2023-08-24 17:58:20 +08:00
Gabriel
96164f3bdc [pipelinex](sort) Fix expression initialization order (#23405) 2023-08-24 17:29:24 +08:00
YangYAN
e262c8b4be [chore](fe)correct some doc errors (#23398) 2023-08-24 17:05:13 +08:00
Ashin Gau
c6ac925f5a [fix](common) implement the move assignment operator for Status (#23372) 2023-08-24 14:41:19 +08:00
Dongyang Li
303d5ac3f7 Revert "[enhancement](thirdparty) upgrade thirdparty libs (#23290)" (#23420) 
This reverts commit 2185268bab2f5890a2c3c9a3c315375a971c4416.

Co-authored-by: stephen <hello-stephen@qq.com>
 2023-08-24 14:40:51 +08:00
Kaijie Chen
c775f8e7bd [feature](move-memtable)[2/7] add protos for memtable on sink node (#23348) 
Co-authored-by: zhengyu <freeman.zhang1992@gmail.com>
Co-authored-by: laihui <1353307710@qq.com>
 2023-08-24 11:11:46 +08:00
amory
687c676160 [FIX](map)fix column map for offset next_array_item_rowid order (#23250) 
* fix column map for offset next_array_item_rowid order

* add regress test
 2023-08-24 10:57:40 +08:00
morrySnow
044423f902 [fix](Nereids) use Stopwatch to do timeout checker (#23383) 
1. avoid thread leak if exception thrown in planning
2. avoid memory release delays since the timer task hold CascadesContext
   object
 2023-08-24 09:49:35 +08:00
Pxl
376d54de41 [Chore](function) add check for comparison input type #23330 2023-08-24 08:28:31 +08:00
yujun
f0bc2c2eff [fix](tablet clone) fix partition rebalance apply move exception (#23222) 2023-08-23 21:50:50 +08:00
yujun
156f7b7699 [improvement](transaction) make commit txn fail hint more understandable (#23227) 2023-08-23 21:50:24 +08:00
zy-kkk
448b7755c6 [feature](jdbc catalog) support doris jdbc catalog array type (#23056) 2023-08-23 21:17:16 +08:00
airborne12
8bdb75e752 [Enhancement](inverted index) remove time-consuming log prints in bkd reader (#23381) 2023-08-23 21:03:01 +08:00
morrySnow
daa4db097e [fix](Nereids) array_difference and array_position get wrong result (#23331) 
1. change array_difference signature to let it return same type as arg
2. do not change precision when signature not use wildcard type
 2023-08-23 20:38:09 +08:00
Xinyi Zou
2185268bab [enhancement](thirdparty) upgrade thirdparty libs (#23290) 
protobuf 3.15.0 -> 21.11
glog 0.4.0 -> 0.6.0
lz4 1.9.3 -> 1.9.4
curl 7.79.0 -> 8.2.1
brpc 1.4.0 -> 1.6.0
zstd 1.5.2 -> 1.5.5
arrow 7.0.0 -> 13.0.0
abseil 20220623.1 -> 20230125.3
orc 1.7.2 -> 1.9.0
jemalloc for arrow 5.2.1 -> 5.3.0
xsimd 7.0.0 -> 13.0.0
opentelemetry-proto 0.19.0 -> 1.0.0
opentelemetry 1.8.3 -> 1.10.0

new:
c-ares -> 1.19.1
grpc -> 1.54.3
 2023-08-23 20:25:49 +08:00
AKIRA
dad1587799 [opt](stats) don't load stats when task is killed (#23354) 
Sync cache after analyze task is killed is meaningless and make error message confused
 2023-08-23 20:24:08 +08:00
starocean999
1b7d692d72 [fix](planner & nereids) convert to double if div decimal overflow (#23272) 2023-08-23 20:10:53 +08:00
Xin Liao
bdb2f09e9f [fix](test) fix unstable pk uk case (#23390) 2023-08-23 20:09:37 +08:00
Gabriel
9d1f2cd8e0 [Improvement](pipeline) Terminate early for short-circuit join (#23378) 2023-08-23 19:40:17 +08:00
jakevin
f0c16ab20d [fix](Nereids): visitLogicalWindow don't override (#23376) 
- visitLogicalWindow don't override
- head index is wrong
 2023-08-23 18:50:15 +08:00
jakevin
711913ee07 [refactor](Nereids): remove penalty in CostModel (#23375) 2023-08-23 18:44:42 +08:00
zclllyybb
51ac92f65c Revert "[fix](function) to_bitmap parameter parsing failure returns null instead of bitmap_empty (#21236)" (#23368) 
This reverts commit 1c3cc77a54938ed948ad8186b8dea8385977d23c.
 2023-08-23 18:27:35 +08:00
Tiewei Fang
4c751202e3 [fix](Es Catalog) Pushdown predicate of Es catalog query in nereids (#23345) 2023-08-23 18:03:39 +08:00
zhangguoqiang
c25e1b7d95 [fix](tools)fix perf tools analyze db (#23370) 2023-08-23 18:01:40 +08:00
airborne12
8140fc737e [Fix](inverted index) fix bug when match condition in hash join (#23105) 
* [Fix](inverted index) fix bug when match condition in hash join
 2023-08-23 17:48:31 +08:00
Dongyang Li
6aeacf252e [fix](pipeline) if pr addded files, also need run regresion (#23332) 
Co-authored-by: stephen <hello-stephen@qq.com>
 2023-08-23 16:45:44 +08:00
Dongyang Li
4d60418c58 Update load.groovy (#23333) 2023-08-23 16:33:40 +08:00
mch_ucchi
7f1857b4e7 [fix](regression-test) fix unstable case load_p0/insert/test_insert.groovy (#23326) 2023-08-23 16:22:49 +08:00
mch_ucchi
c3327b51b9 [Fix](Nereids) add nereids load function in read fields of GlobalFunctionMgr and Database (#23248) 
add nereids load function in read fields of GlobalFunctionMgr and Database
to fix some udf is lost when restart FE and query with Nereids.
 2023-08-23 15:59:50 +08:00
camby
22e373a799 [feature](vector-search) add 4 distance functions to support vector search (#23129) 2023-08-23 15:51:15 +08:00
daidai
2dda44d7b5 [fix](csv-reader)fix bug of multi-char delimiter in csv reader 
fix bug that csv_reader parse line in order to get column.
 2023-08-23 15:19:13 +08:00
Pxl
ab7b45a9e2 [Bug](map) support replicate for column map and remove some unused code #23356 2023-08-23 15:06:04 +08:00