database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
14,011 Commits 1 Branch 0 Tags
2014e16cfb6a61391e51ebac75677ecb3e4c4f39
Commit Graph

14011 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Guangdong Liu
2014e16cfb [fix](es catalog)fix es http timeout (#25273) 2023-10-12 10:21:55 +08:00
lihangyu
c4916184f1 [regression-test](topn opt) fix case that explain with wrong result (#25316) 
`select * from tbl where k = 1 order by k limit 1` will eliminate sort in original planner
 2023-10-12 10:16:42 +08:00
Jerry Hu
7ca63665b4 [fix](agg) garbled characters in result of map_agg (#25318) 2023-10-12 10:10:55 +08:00
lihangyu
58d96ecdbf [Improve](status) avoid print too may stack log for DATA_QUALITY_ERROR code (#25292) 2023-10-12 09:58:51 +08:00
Qi Chen
46ab4346ca [Opt](parquet reader) Optimize the performance of reading decimal in parquet reader. (#25012) 
Optimize the performance of reading decimal in parquet reader.

- Static dispatch `DecimalScaleParams`.
- Optimize `memcpy`, static dispatch copy size in fixed length cases.
- Use right shift bit operator to convert decimals.
 2023-10-12 09:53:08 +08:00
Xiangyu Wang
e41b03e530 [Fix](multi-catalog) delete hdfs hedged configs at BE side. (#25094) 
Issue Number: close #25093 

We can set hdfs hedged configs when creating catalog, just like this:
```
CREATE CATALOG `test_ctl` PROPERTIES (
...
"dfs.client.hedged.read.threadpool.size" = "128",
"dfs.client.hedged.read.threshold.millis" = "500",
...
);
```
It is redundant to set these configs at BE side, and it will brings an occasional bug at #25093 .
 2023-10-11 23:25:30 +08:00
daidai
9a4baf7ccf [fix](Nereids)Fix the bug that count(*) does not push down for tables with only one column. (#25222) 
after pr #22115 .

Fixed the bug that when selecting count(*) from table, if the table has only one column, the aggregate count is not pushed down.
 2023-10-11 23:17:30 +08:00
zhangdong
d1f59a4025 [fix](catalog)fix when modifying comments in property, it will modify the comments in the catalog (#24857) 
- fix when modifying comments in property, it will modify the comments in the catalog
- add `alter catalog modify comment` to modify comment for catalog
- abstract some logic of `alter catalog` to parent class
 2023-10-11 23:16:19 +08:00
yujun
73c3e3ab55 [Feature](x-load) support config min replica num for loading data (#21118) 2023-10-11 21:07:35 +08:00
Mryange
ba87f7d3a3 [fix](pipelineX) add table sink and some fix in pipelineX (#25314) 2023-10-11 20:18:08 +08:00
zhengyu
e94fca4949 [enhancement](bvar) add metrics to monitor load throughput (#25189) 
to monitor realtime load throughput of the BE

Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>
 2023-10-11 19:56:33 +08:00
yiguolei
f960b8c989 [bugfix](stream receiver) be will core during stop because receiver is not closed (#25298) 
Co-authored-by: yiguolei <yiguolei@gmail.com>
 2023-10-11 19:49:40 +08:00
Luzhijing
9d5ff5b55d [doc](readme)Update 2.0.2 release note in README.md (#25324) 2023-10-11 19:49:00 +08:00
Pxl
b8ba83faac [Chore](check) ignore license check on .github (#25323) 
ignore license check on .github
 2023-10-11 19:44:05 +08:00
HHoflittlefish777
f680a2141d [enhancement](regression-test) add routine load json case (#25253) 2023-10-11 19:43:08 +08:00
Dongyang Li
b17bac6323 [fix](case) use the custom DB explicitly in analyze_stats.groovy (#25285) 
Co-authored-by: stephen <hello-stephen@qq.com>
use the custom DB explicitly in analyze_stats.groovy
 2023-10-11 19:28:14 +08:00
morrySnow
46be6c07e1 [opt](Nereids) expose multi distinct functions (#25309) 2023-10-11 05:42:39 -05:00
Tiewei Fang
c6b1c903e4 [fix](Regression-test) fix that the String type in a nested type should contain double quotes and add regression-test (#25115) 2023-10-11 18:30:26 +08:00
meiyi
e514d52232 [fix](point-query) Support mow table with sequence column (#25308) 2023-10-11 18:22:16 +08:00
yujun
1e300d895d [improvement](checkpoint) checkpoint thread update tablet invert index (#25098) 2023-10-11 18:18:03 +08:00
Kaijie Chen
73f632a4e3 [fix](move-memtable) handle error in LoadStreamWriter::close (#24805) 2023-10-11 16:54:42 +08:00
Gabriel
3db33207d4 [pipelineX](fix) Fix nullable types for set operator (#25294) 2023-10-11 16:50:54 +08:00
Dongyang Li
0170a63c40 [fix](workflow) checkout the pr, not the base (#25310) 
Co-authored-by: stephen <hello-stephen@qq.com>
checkout the pr, not the base
 2023-10-11 16:50:25 +08:00
starocean999
2d19f2fbfe [fix](planner)need call materializeSrcExpr for materialized slots in join node (#25204) 2023-10-11 16:34:53 +08:00
starocean999
dabeeb0338 [fix](planner)should always use plan node's getTblRefIds method to get unassigned conjuncts for this node (#25130) 2023-10-11 16:34:21 +08:00
starocean999
2221c8e2ed [fix](planner)implicit cast should use type member variable instead of targetTypeDef (#24582) 2023-10-11 16:33:48 +08:00
starocean999
e9554e36a8 [fix](nereids)disable parallel scan in some case (#25089) 2023-10-11 16:32:09 +08:00
lsy3993
df7724d6cb [typo](docs)delete wrong description of from_unixtime (#23897) 2023-10-11 03:20:13 -05:00
lsy3993
004d3264a6 [typo](docs) add 'order by' when use 'limit m,n' (#24236) 2023-10-11 03:15:33 -05:00
Dongyang Li
193ef79df2 [chore](fix) Update .asf.yaml (#25300) 
fix issue introduced by #25101

...In order to solve this, we’ve added a new pull_request_target event, which behaves in an almost identical way to the pull_request event with the same set of filters and payload.
However, instead of running against the workflow and code from the merge commit, the event runs against the workflow and code from the base of the pull request.
 2023-10-11 03:15:16 -05:00
starocean999
a9b84ae6ee [test](nereids)add more case in PushdownFilterThroughAggregationTest (#24927) 2023-10-11 16:14:36 +08:00
starocean999
6d999f5b95 [enhancement](nereids)add eliminate filter on one row relation rule (#24980) 
1.simplify PushdownFilterThroughSetOperation rule
2.add eliminate filter on one row relation rule
 2023-10-11 16:12:24 +08:00
Pxl
48b04752a5 [Chore](cases) set enable_nereids_planner on mv cases (#25277) 
set enable_nereids_planner on mv cases
 2023-10-11 15:58:06 +08:00
谢健
47578c0fc9 [fix](Nereids) fix toSql of date literal (#25243) 
toSql should return '2023-2-1 ' for DateLiteral 2023-2-1
 2023-10-11 13:04:05 +08:00
Gabriel
0d603dd4c3 [Bug](delete) Use date as common type for date comparison (#25262) 2023-10-11 11:51:43 +08:00
Gabriel
bb670118f5 [coverage](test) Delete unused function to improve test coverage (#25233) 2023-10-11 11:50:51 +08:00
Mryange
cdf5f0fe68 [fix](pipelineX) mark join column should be nullable (#25275) 2023-10-11 11:35:43 +08:00
Xiangyu Wang
1e6d34d1d0 [Enhancement](sql-cache) Add partition update time for hms table and use it at sql-cache. (#24491) 
Now FE does not record the update time of hms tbl's partitons, so the sql cache may be hit even the hive table's partitions have changed. This pr add a field to record the partition update time, and use it when enable sql-cache.
The cache will be missed if any partition has changed at hive side.

Use System.currentTimeMillis() but not the event time of hms event because we would better keep the same measurement with the schemaUpdateTime of external table. Add this value to ExternalObjectLog and let slave FEs replay it because it is better to keep the same value with all FEs, so the sql-cache can be hit by the querys through different FEs.
 2023-10-11 11:05:16 +08:00
lihangyu
2f706cc84b [compile](simdjson reader) use __AVX2__ macro to decide whether use simdjson to parse (#25165) 2023-10-11 10:50:13 +08:00
morrySnow
b91bce8a62 [feature](Nereids) add array distance functions (#25196) 
- l1_distance
- l2_distance
- cosine_distance
- inner_product
 2023-10-10 21:35:06 -05:00
zhangstar333
8e66dbc4a8 [enhancement](log) add some decheck log to debug (#25210) 2023-10-11 10:33:13 +08:00
zhangstar333
5be29f859a [enhancement](node) add filter in partition sort node in BE #25188 
add filter in partition sort node in BE
 2023-10-11 10:30:15 +08:00
amory
2ed5245014 [FIX](array_function) fix array_map function with array index function without checkout arg… #25226 2023-10-11 10:23:33 +08:00
Gabriel
7b22ae0c80 [pipelineX](feature) Support set operation operator (#25251) 
---------

Co-authored-by: zhaochangle <zhaochangle@selectdb.com>
 2023-10-11 10:22:45 +08:00
bobhan1
1fa8720164 [regression-test](merge-on-write) Fix partial update concurrency conflict case (#25212) 2023-10-11 10:17:01 +08:00
Calvin Kirs
d4673ce28a [Feature](Job)Jobs in the Finish state will be automatically deleted after three days. (#25170) 2023-10-11 10:04:19 +08:00
HHoflittlefish777
b7ac95a970 [enhancement](regression-test) open routine load regression test by default and add data check (#25122) 2023-10-11 10:03:16 +08:00
Kaijie Chen
be11b48407 [fix](load) fix MemTableWriter::active_memtable_mem_consumption (#25207) 2023-10-10 22:33:50 +08:00
zhangstar333
fb3b888ff1 [prune](partition)support prune partition when is auto partition with function call (#24747) 
now create table use auto create partition:
AUTO PARTITION BY RANGE date_trunc(event_day, 'day')
so the value of event_day will be insert into partition of date_trunc(event_day, 'day'),
eg: select * from partition_range where date_trunc(event_day,"day")= "2023-08-07 11:00:00";
we can prune some partitions by invoke function of date_trunc("2023-08-07 11:00:00","day" );
 2023-10-10 20:39:43 +08:00
lihangyu
913282b29b [refactor](column) remove get_data_type in IColumn (#25242) 2023-10-10 20:27:15 +08:00