database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
20,786 Commits 1 Branch 0 Tags
702abbff0f3e8ebc9cd99e33dcd5bf8b3f4278fa
Commit Graph

123 Commits

Author SHA1 Message Date
Uniqueyou
edd9015de3 [branch-2.1](function) fix error result in auto partition name (#41130) (#43977) 
pick https://github.com/apache/doris/pull/41130
https://github.com/apache/doris/pull/41372

---------

Co-authored-by: zhaochangle <zhaochangle@selectdb.com>
 2024-11-15 19:11:42 +08:00
TengJianPing
64d0c55e1d [fix](decimal256) support decimal256 for many functions (#42136) (#42353) 
BP #42136
 2024-10-24 10:52:28 +08:00
amory
fb12e10272 [fix](array-funcs)fix array agg func with decimal type (#40839) (#42023) 
## Proposed changes
backport: (https://github.com/apache/doris/pull/40839)
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-10-17 20:47:39 +08:00
zhiqiang
1259fe2bd5 [fix](covar) Fix covar nullable on branch-2.1 (#40841) 
covar should not be always nullable.

This fix on branch-2.1 makes covar same with master on FE.
 2024-09-20 17:35:27 +08:00
zclllhhjj
163193b1d4 [branch-2.1](function) fix random_bytes return same data for multi rows (#39891) (#40137) 
pick https://github.com/apache/doris/pull/39891

Issue Number: close #xxx

before:
```sql
mysql [optest]>SELECT random_bytes(10) a FROM numbers("number" = "10");
+------------------------+
| a                      |
+------------------------+
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
+------------------------+
```

now:
```sql
mysql [optest]>SELECT random_bytes(10) a FROM numbers("number" = "10");
+------------------------+
| a                      |
+------------------------+
| 0xd82cf60825b29ef2a0fd |
| 0x6f8c808415bdbaa6d257 |
| 0x7c26b5214297a151c25c |
| 0x43f02c77293063900437 |
| 0x5e5727569dec5e24f96b |
| 0x434f20bf74d7759640b7 |
| 0x087ed96b739750c733a6 |
| 0xdf05f6d7ede4972eb846 |
| 0xcefab471912264b5c54f |
| 0x1bddc019409d1926aa10 |
+------------------------+
```

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-30 10:43:42 +08:00
amory
4cf769b39f [Improve](table-function) explode json array with json args (#39491) 2024-08-27 14:53:17 +08:00
zclllhhjj
db0724dfe0 [Fix-2.1](function) fix function covar core for not null input (#39943) 
## Proposed changes

Issue Number: close #xxx

add testcases like:
```groovy
    qt_notnull1 "select covar_samp(non_nullable(x), non_nullable(y)) from test_covar_samp"
    qt_notnull2 "select covar_samp(x, non_nullable(y)) from test_covar_samp"
    qt_notnull3 "select covar_samp(non_nullable(x), y) from test_covar_samp"
```

before they will all coredump in 2.1
 2024-08-27 08:39:47 +08:00
zclllhhjj
460605ae3c [branch-2.1] pick some prs (#39860) 
## Proposed changes

Issue Number: close #xxx

https://github.com/apache/doris/pull/38385 optimize parsing datetime
https://github.com/apache/doris/pull/38978 make stream load failure
message more clear and disable some error's stacktrace by default
https://github.com/apache/doris/pull/39255 fix random function coredump
https://github.com/apache/doris/pull/39324 fix function corr
inconsistency with doc
https://github.com/apache/doris/pull/39449 check auto partitoin nullity
when creating partition
https://github.com/apache/doris/pull/39695 make
DynamicPartitionScheduler immediately know interval's change
https://github.com/apache/doris/pull/39754 Add some partition expr check
on creating table
 2024-08-24 17:26:42 +08:00
morrySnow
d56000e924 [opt](Nereids) polish aggregate function signature matching (#39352) (#39460) 
pick from master #39352

use double to match string
- corr
- covar
- covar_samp
- stddev
- stddev_samp

use largeint to match string
- group_bit_and
- group_bit_or
- group_git_xor

use double to match decimalv3
- topn_weighted

optimize error message
- multi_distinct_sum
- multi_distinct_sum0
 2024-08-16 13:57:11 +08:00
morrySnow
3aaee8f7d5 [fix](Nereids) polish function signature search algorithm (#38497) (#39436) 
pick from master #38497 and #39342

use array<double> for array<string>
- array_avg
- array_cum_sum
- array_difference
- array_product

use array<bigint> for array<string>
- bitmap_from_array

use double first
- fmod
- pmod

let high order function throw friendly exception
- array_filter
- array_first
- array_last
- array_reverse_split
- array_sort_by
- array_split

let return type same as parameter's type
- array_push_back
- array_push_front
- array_with_constant
- if

let greatest / least work same as mysql's greatest
 2024-08-16 08:24:25 +08:00
zclllhhjj
4acd69590d [Fix](function) fix wrong nullable signature of function corr (#39380) 
## Proposed changes

Issue Number: close #xxx

before `corr(nullable_x, nullable_y)` will core dump. not fixed.
no need to patch in master because the refactor
https://github.com/apache/doris/pull/37330 already changed the
implementation context
 2024-08-15 14:10:09 +08:00
amory
226e01889c [fix](array_apply) pick array apply fix (#39328) 
## Proposed changes
backport: https://github.com/apache/doris/pull/39105
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-14 18:52:29 +08:00
Mryange
b26af32934 [fix](function) fix error return type in corr(float32,float32) (#39251) (#39350) 
https://github.com/apache/doris/pull/39251
```
mysql [test11]>select corr(cast(x as float),cast(y as float)) from test_corr;
ERROR 1105 (HY000): errCode = 2, detailMessage = (127.0.0.1)[INTERNAL_ERROR]column_type not match data_types in agg node, column_type=Nullable(Float64), data_types=Nullable(Float32),column name=

```

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-14 18:47:14 +08:00
lihangyu
773008d6fa [Fix](Json) fix some cast issue (#38683) (#39025) 
#38683
 2024-08-07 22:05:43 +08:00
Uniqueyou
65154f8abe [branch-2.1] (doris-future) Support auto partition name function (#38853) 
cherry-pick https://github.com/apache/doris/pull/34258 to branch-2.1
 2024-08-05 16:04:24 +08:00
amory
2653087843 [pick](array-funcs)fix array with empty arg in be behavior (#38708) 
## Proposed changes
backport: https://github.com/apache/doris/pull/36845
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-05 09:08:28 +08:00
amory
556f0fc784 [pick](json-keys) support json_keys function (#38631) 
## Proposed changes
backport: https://github.com/apache/doris/pull/36411
Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-02 19:10:00 +08:00
zhangstar333
57864e8554 [cherry-pick](branch-21) fix collect_set function core dump without arena pool (#38234) (#38307) 
## Proposed changes

cherry-pick from master #38234

<!--Describe your changes.-->
 2024-07-25 12:05:52 +08:00
starocean999
ebc178af23 [fix](nereids)acos function should return null literal instead of NaN value (#37996) 
pick from master https://github.com/apache/doris/pull/37932

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-18 09:28:56 +08:00
HappenLee
90a4dd09f3 [Fix](func) CoreDump and Result Error in percentile function (#36647) 
cherry pick #36643
 2024-06-21 23:42:45 +08:00
Mingyu Chen
0cff539810 [feature](function) support new function replace_empty (#36283) (#36656) 
#36283
 2024-06-21 16:46:22 +08:00
starocean999
cbaff8a700 [fix](nereids)change the decimal's precision and scale for cast(xx as decimal) (#36540) 
pick from master #36316

expression cast( xx as decimal )'s datatype maybe decimalv3 or decimalv2
depending on enable_decimal_conversion value in fe conf file. if
enable_decimal_conversion is true, the datatype is decimalv3(9, 0), but
the datatype was decimalv3(38, 9) in 2.0 releases. So this pr change the
datatype same as 2.0 releases to keep the behavior consistent.
 2024-06-20 17:46:11 +08:00
lihangyu
e3e5f18f26 [Fix](Json type) correct cast result for json type (#34764) 2024-05-18 18:40:17 +08:00
feiniaofeiafei
fe7d2b8159 [Fix](nereids) ignore slot implements SlotNotFromChildren when check the slot from children in NormalizeAggregate (#34171) 2024-05-07 07:48:05 +08:00
Chester
22a6b1d3f5 [feature](function) support hll functions hll_from_base64, hll_to_base64 (#32089) 
Issue Number: #31320 

Support two hll functions:

- hll_from_base64
Convert a base64 string(result of function hll_to_base64) into a hll.
- hll_to_base64
Convert an input hll to a base64 string.
 2024-04-17 23:42:13 +08:00
Chester
3096150d1b [feature](agg) support aggregate function group_array_intersect (#33265) 2024-04-17 23:42:13 +08:00
Jensen
bb8bc75af4 [feature](agg) add aggregate function sum0 (#32541) 2024-04-10 11:34:29 +08:00
zy-kkk
1b3e4322e8 [improvement](serde) Handle NaN values in number for MySQL result write (#33227) 2024-04-07 23:24:23 +08:00
Mryange
cca0773261 [feature](function) round function defaults to rounding normally 2024-03-06 13:06:26 +08:00
Guangdong Liu
9c4708ee74 [function](random_bytes)add random_bytes function (#31547) 
SELECT random_bytes(10);

random_bytes(10) |
----------------------+
0x9b8ea00b7d1084bc5b26|
 2024-02-29 16:44:39 +08:00
Chester
f163d56a98 [feature](function) support sequence function(alias of array_range), enhance both to handle datetimev2 (#30823) 2024-02-27 10:12:19 +08:00
vhwzIs
d70776af55 [feature](agg-func) support covar and covar_samp function (#30983) 2024-02-18 11:50:17 +08:00
nanfeng
e68019c10a [Function](Exec) Support windows function cume_dist (#30997) 2024-02-16 10:16:40 +08:00
yangshijie
40e1326bc9 [feature](window-func) support percent_rank window function (#30926) 2024-02-16 10:12:25 +08:00
nanfeng
2bb477bae7 [feature](agg-func) support corr function #30822 2024-02-16 10:12:24 +08:00
Guangdong Liu
1ed24117ac [function](url_decode)add url_decode function (#30667) 2024-02-05 22:23:00 +08:00
Rohit Satardekar
6442663735 [Function](exec) upport atan2 math function (#30672) 
Co-authored-by: Rohit Satardekar <rohitrs1983@gmail.com>
 2024-02-04 14:28:38 +08:00
Rohit Satardekar
19f57b544e support cosh math function (#30602) 
Co-authored-by: Rohit Satardekar <rohitrs1983@gmail.com>
 2024-01-31 23:53:39 +08:00
yangshijie
8b61b7c6cd [exec](function) Add tanh func (#30555) 2024-01-31 23:53:39 +08:00
Chester
7e19224a6c [fix](function) fix ipv4 funcs get failed error, improve an ipv6 func and exception message (#30269) 2024-01-28 18:25:31 +08:00
koarz
ca5a314765 [fix](function) make STRLEFT and STRRIGHT and SUBSTR function DEPEND_ON_ARGUMENT (#28352) 
make STRLEFT and STRRIGHT function DEPEND_ON_ARGUMENT
 2024-01-25 13:23:59 +08:00
Chester
dfde10d4c8 [improvement](function) switch inet(6)_aton alias origin function (#30196) 2024-01-23 10:09:54 +08:00
Chester
ead3b4ac1d [feature](function) support ip function is_ipv4_compat, is_ipv4_mapped (#29954) 2024-01-23 10:07:51 +08:00
yangshijie
eb4c389b0b [feature](function) support ip functions isipv4string and isipv6string (#28556) 2024-01-07 13:03:11 +08:00
Chester
7402fee1fc [feature](function) support ip function ipv6_string_to_num(_or_default, _or_null), inet6_aton (#28361) 2024-01-05 19:24:45 +08:00
Chester
2e1ce758f1 [feature](function) support ip function ipv6numtostring(alias inet6_ntoa) (#27342) 2023-12-02 11:48:19 +08:00
zclllyybb
baadc14e60 [Enhancement](function) support unix_timestamp with float (#26827) 
---------

Co-authored-by: YangWithU <plzw8@outlook.com>
 2023-11-27 09:58:53 +08:00
Chester
334260dff7 [feature](function) support ip function ipv4stringtonum(ordefault, ornull), inet_aton (#25510) 2023-11-17 10:27:07 +08:00
morrySnow
2c6d2255c3 [fix](Nereids) nested type literal type coercion and insert values with map (#26669) 2023-11-14 21:13:26 -06:00
TengJianPing
9a4fd5be79 [nereids](datetime) fix wrong result type of datetime add with interval as first arg (#26957) 
Incorrect result data type cause be cordump:

drop table if exists testaaa;
create table testaaa(k1 tinyint, k2 smallint, k3 int, k4 bigint, k5 decimal(9,3), k6 char(5), k10 date, k11 datetime, k7 varchar(20), k8 double max, k9 float sum) engine=olap distributed by hash(k1) buckets 5 properties("storage_type"="column","replication_num"="1") ;

insert into testaaa values(1,1,1,1,9.3, "k6", "2023-11-14", "2023-11-14", "k7", 9.99, 9.99);


select  interval 10 year + k10 from testaaa;
The plan result type is DATE:

mysql [test]>explain verbose select   interval 10 year + k10  from testaaa;
+-------------------------------------------------------------------------------------------------------+
| Explain String(Nereids Planner)                                                                       |
+-------------------------------------------------------------------------------------------------------+
| PLAN FRAGMENT 0                                                                                       |
|   OUTPUT EXPRS:                                                                                       |
|     years_add(k10, INTERVAL 10 YEAR)[#11]                                                             |
|   PARTITION: UNPARTITIONED                                                                            |
|                                                                                                       |
|   HAS_COLO_PLAN_NODE: false                                                                           |
|                                                                                                       |
|   VRESULT SINK                                                                                        |
|      MYSQL_PROTOCAL                                                                                   |
|                                                                                                       |
|   64:VEXCHANGE                                                                                        |
|      offset: 0                                                                                        |
|      tuple ids: 1N                                                                                    |
|                                                                                                       |
| PLAN FRAGMENT 1                                                                                       |
|                                                                                                       |
|   PARTITION: HASH_PARTITIONED: k1[#0]                                                                 |
|                                                                                                       |
|   HAS_COLO_PLAN_NODE: false                                                                           |
|                                                                                                       |
|   STREAM DATA SINK                                                                                    |
|     EXCHANGE ID: 64                                                                                   |
|     UNPARTITIONED                                                                                     |
|                                                                                                       |
|   58:VOlapScanNode                                                                                    |
|      TABLE: default_cluster:test.testaaa(testaaa), PREAGGREGATION: OFF. Reason: No aggregate on scan. |
|      partitions=1/1 (testaaa), tablets=5/5, tabletList=945025,945027,945029 ...                       |
|      cardinality=1, avgRowSize=9885.0, numNodes=1                                                     |
|      pushAggOp=NONE                                                                                   |
|      projections: years_add(k10[#6], INTERVAL 10 YEAR)                                                |
|      project output tuple id: 1                                                                       |
|      tuple ids: 0                                                                                     |
|                                                                                                       |
| Tuples:                                                                                               |
| TupleDescriptor{id=0, tbl=testaaa, byteSize=8}                                                        |
|   SlotDescriptor{id=6, col=k10, colUniqueId=6, type=DATEV2, nullable=true, isAutoIncrement=false}     |
|                                                                                                       |
| TupleDescriptor{id=1, tbl=testaaa, byteSize=32}                                                       |
|   SlotDescriptor{id=11, col=null, colUniqueId=null, type=DATE, nullable=true, isAutoIncrement=false}  |
+-------------------------------------------------------------------------------------------------------+
39 rows in set (1 min 31.50 sec)
coredump stack:

F1109 20:11:37.677680 323805 assert_cast.h:61] Bad cast from type:doris::vectorized::ColumnVector to doris::vectorized::ColumnVector
*** Check failure stack trace: ***
F1109 20:11:37.680608 323800 assert_cast.h:61] Bad cast from type:doris::vectorized::ColumnVector to doris::vectorized::ColumnVector
*** Check failure stack trace: ***
F1109 20:11:37.680608 323800 assert_cast.h:61] Bad cast from type:doris::vectorized::ColumnVector to doris::vectorized::ColumnVectorF1109 20:11:37.681102 323808 assert_cast.h:61] Bad cast from type:doris::vectorized::ColumnVector to doris::vectorized::ColumnVector
*** Check failure stack trace: ***
    @     0x56489d591d3d  google::LogMessage::Fail()
    @     0x56489d591d3d  google::LogMessage::Fail()
    @     0x56489d591d3d  google::LogMessage::Fail()
    @     0x56489d594279  google::LogMessage::SendToLog()
    @     0x56489d594279  google::LogMessage::SendToLog()
    @     0x56489d594279  google::LogMessage::SendToLog()
    @     0x56489d5918a6  google::LogMessage::Flush()
    @     0x56489d5918a6  google::LogMessage::Flush()
    @     0x56489d5918a6  google::LogMessage::Flush()
    @     0x56489d5948e9  google::LogMessageFatal::~LogMessageFatal()
    @     0x56489d5948e9  google::LogMessageFatal::~LogMessageFatal()
    @     0x56489d5948e9  google::LogMessageFatal::~LogMessageFatal()
    @     0x56487a2a8a0c  assert_cast<>()
    @     0x56487a2a8a0c  assert_cast<>()
    @     0x56487a2a8a0c  assert_cast<>()
    @     0x5648893d8312  doris::vectorized::ColumnVector<>::insert_range_from()
    @     0x5648893d8312  doris::vectorized::ColumnVector<>::insert_range_from()
    @     0x5648893d8312  doris::vectorized::ColumnVector<>::insert_range_from()
    @     0x56488924a670  doris::vectorized::ColumnNullable::insert_range_from()
    @     0x56488924a670  doris::vectorized::ColumnNullable::insert_range_from()
    @     0x56488924a670  doris::vectorized::ColumnNullable::insert_range_from()
    @     0x56487a454475  doris::ExecNode::do_projections()
    @     0x56487a454475  doris::ExecNode::do_projections()
    @     0x56487a454475  doris::ExecNode::do_projections()
    @     0x56487a454b89  doris::ExecNode::get_next_after_projects()
    @     0x56487a454b89  doris::ExecNode::get_next_after_projects()
*** Query id: a467995b35334741-b625042f56495aaf ***
*** tablet id: 0 ***
*** Aborted at 1699531898 (unix time) try "date -d @1699531898" if you are using GNU date ***
*** Current BE git commitID: 0d83327a7c ***
*** SIGABRT unknown detail explain (@0x190d64) received by PID 1641828 (TID 1642168 OR 0x7f6ff96c0700) from PID 1641828; stack trace: ***
    @     0x556ca2a3ab8f  std::_Function_handler<>::_M_invoke()
    @     0x556c9f322787  std::function<>::operator()()
    @     0x556ca29da0b0  doris::Thread::supervise_thread()
    @     0x556c9f322787  std::function<>::operator()()
    @     0x7f71b9c38609  start_thread
    @     0x556ca29da0b0  doris::Thread::supervise_thread()
    @     0x7f71b9c38609  start_thread
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/common/signal_handler.h:417
 1# 0x00007F71B9E09090 in /lib/x86_64-linux-gnu/libc.so.6
 2# raise at ../sysdeps/unix/sysv/linux/raise.c:51
 3# abort at /build/glibc-SzIz7B/glibc-2.31/stdlib/abort.c:81
 4# 0x0000556CC51F3729 in /mnt/hdd01/ci/branch20-deploy/be/lib/doris_be
 5# 0x0000556CC51E8D3D in /mnt/hdd01/ci/branch20-deploy/be/lib/doris_be
 6# google::LogMessage::SendToLog() in /mnt/hdd01/ci/branch20-deploy/be/lib/doris_be
 7# google::LogMessage::Flush() in /mnt/hdd01/ci/branch20-deploy/be/lib/doris_be
 8# google::LogMessageFatal::~LogMessageFatal() in /mnt/hdd01/ci/branch20-deploy/be/lib/doris_be
 9# doris::vectorized::ColumnVector const& assert_cast const&, doris::vectorized::IColumn const&>(doris::vectorized::IColumn const&) in /mnt/hdd01/ci/branch20-deploy/be/lib/doris_be
10# doris::vectorized::ColumnVector::insert_range_from(doris::vectorized::IColumn const&, unsigned long, unsigned long) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/vec/columns/column_vector.cpp:354
11# doris::vectorized::ColumnNullable::insert_range_from(doris::vectorized::IColumn const&, unsigned long, unsigned long) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/vec/columns/column_nullable.cpp:289
12# doris::ExecNode::do_projections(doris::vectorized::Block*, doris::vectorized::Block*) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/exec/exec_node.cpp:573
13# doris::ExecNode::get_next_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*, std::function const&, bool) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/exec/exec_node.cpp:592
14# doris::pipeline::SourceOperator::get_block(doris::RuntimeState*, doris::vectorized::Block*, doris::pipeline::SourceState&) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/pipeline/exec/operator.h:413
15# doris::pipeline::PipelineTask::execute(bool*) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/pipeline/pipeline_task.cpp:259
16# doris::pipeline::TaskScheduler::_do_work(unsigned
 2023-11-14 20:28:41 +08:00