database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
19,529 Commits 1 Branch 0 Tags
54787f51c7ed454feff1663c4e4255c011749421
Commit Graph

106 Commits

Author SHA1 Message Date
zhangstar333
57864e8554 [cherry-pick](branch-21) fix collect_set function core dump without arena pool (#38234) (#38307) 
## Proposed changes

cherry-pick from master #38234

<!--Describe your changes.-->
 2024-07-25 12:05:52 +08:00
starocean999
ebc178af23 [fix](nereids)acos function should return null literal instead of NaN value (#37996) 
pick from master https://github.com/apache/doris/pull/37932

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-07-18 09:28:56 +08:00
HappenLee
90a4dd09f3 [Fix](func) CoreDump and Result Error in percentile function (#36647) 
cherry pick #36643
 2024-06-21 23:42:45 +08:00
Mingyu Chen
0cff539810 [feature](function) support new function replace_empty (#36283) (#36656) 
#36283
 2024-06-21 16:46:22 +08:00
starocean999
cbaff8a700 [fix](nereids)change the decimal's precision and scale for cast(xx as decimal) (#36540) 
pick from master #36316

expression cast( xx as decimal )'s datatype maybe decimalv3 or decimalv2
depending on enable_decimal_conversion value in fe conf file. if
enable_decimal_conversion is true, the datatype is decimalv3(9, 0), but
the datatype was decimalv3(38, 9) in 2.0 releases. So this pr change the
datatype same as 2.0 releases to keep the behavior consistent.
 2024-06-20 17:46:11 +08:00
lihangyu
e3e5f18f26 [Fix](Json type) correct cast result for json type (#34764) 2024-05-18 18:40:17 +08:00
feiniaofeiafei
fe7d2b8159 [Fix](nereids) ignore slot implements SlotNotFromChildren when check the slot from children in NormalizeAggregate (#34171) 2024-05-07 07:48:05 +08:00
Chester
22a6b1d3f5 [feature](function) support hll functions hll_from_base64, hll_to_base64 (#32089) 
Issue Number: #31320 

Support two hll functions:

- hll_from_base64
Convert a base64 string(result of function hll_to_base64) into a hll.
- hll_to_base64
Convert an input hll to a base64 string.
 2024-04-17 23:42:13 +08:00
Chester
3096150d1b [feature](agg) support aggregate function group_array_intersect (#33265) 2024-04-17 23:42:13 +08:00
Jensen
bb8bc75af4 [feature](agg) add aggregate function sum0 (#32541) 2024-04-10 11:34:29 +08:00
zy-kkk
1b3e4322e8 [improvement](serde) Handle NaN values in number for MySQL result write (#33227) 2024-04-07 23:24:23 +08:00
Mryange
cca0773261 [feature](function) round function defaults to rounding normally 2024-03-06 13:06:26 +08:00
Guangdong Liu
9c4708ee74 [function](random_bytes)add random_bytes function (#31547) 
SELECT random_bytes(10);

random_bytes(10) |
----------------------+
0x9b8ea00b7d1084bc5b26|
 2024-02-29 16:44:39 +08:00
Chester
f163d56a98 [feature](function) support sequence function(alias of array_range), enhance both to handle datetimev2 (#30823) 2024-02-27 10:12:19 +08:00
vhwzIs
d70776af55 [feature](agg-func) support covar and covar_samp function (#30983) 2024-02-18 11:50:17 +08:00
nanfeng
e68019c10a [Function](Exec) Support windows function cume_dist (#30997) 2024-02-16 10:16:40 +08:00
yangshijie
40e1326bc9 [feature](window-func) support percent_rank window function (#30926) 2024-02-16 10:12:25 +08:00
nanfeng
2bb477bae7 [feature](agg-func) support corr function #30822 2024-02-16 10:12:24 +08:00
Guangdong Liu
1ed24117ac [function](url_decode)add url_decode function (#30667) 2024-02-05 22:23:00 +08:00
Rohit Satardekar
6442663735 [Function](exec) upport atan2 math function (#30672) 
Co-authored-by: Rohit Satardekar <rohitrs1983@gmail.com>
 2024-02-04 14:28:38 +08:00
Rohit Satardekar
19f57b544e support cosh math function (#30602) 
Co-authored-by: Rohit Satardekar <rohitrs1983@gmail.com>
 2024-01-31 23:53:39 +08:00
yangshijie
8b61b7c6cd [exec](function) Add tanh func (#30555) 2024-01-31 23:53:39 +08:00
Chester
7e19224a6c [fix](function) fix ipv4 funcs get failed error, improve an ipv6 func and exception message (#30269) 2024-01-28 18:25:31 +08:00
koarz
ca5a314765 [fix](function) make STRLEFT and STRRIGHT and SUBSTR function DEPEND_ON_ARGUMENT (#28352) 
make STRLEFT and STRRIGHT function DEPEND_ON_ARGUMENT
 2024-01-25 13:23:59 +08:00
Chester
dfde10d4c8 [improvement](function) switch inet(6)_aton alias origin function (#30196) 2024-01-23 10:09:54 +08:00
Chester
ead3b4ac1d [feature](function) support ip function is_ipv4_compat, is_ipv4_mapped (#29954) 2024-01-23 10:07:51 +08:00
yangshijie
eb4c389b0b [feature](function) support ip functions isipv4string and isipv6string (#28556) 2024-01-07 13:03:11 +08:00
Chester
7402fee1fc [feature](function) support ip function ipv6_string_to_num(_or_default, _or_null), inet6_aton (#28361) 2024-01-05 19:24:45 +08:00
Chester
2e1ce758f1 [feature](function) support ip function ipv6numtostring(alias inet6_ntoa) (#27342) 2023-12-02 11:48:19 +08:00
zclllyybb
baadc14e60 [Enhancement](function) support unix_timestamp with float (#26827) 
---------

Co-authored-by: YangWithU <plzw8@outlook.com>
 2023-11-27 09:58:53 +08:00
Chester
334260dff7 [feature](function) support ip function ipv4stringtonum(ordefault, ornull), inet_aton (#25510) 2023-11-17 10:27:07 +08:00
morrySnow
2c6d2255c3 [fix](Nereids) nested type literal type coercion and insert values with map (#26669) 2023-11-14 21:13:26 -06:00
TengJianPing
9a4fd5be79 [nereids](datetime) fix wrong result type of datetime add with interval as first arg (#26957) 
Incorrect result data type cause be cordump:

drop table if exists testaaa;
create table testaaa(k1 tinyint, k2 smallint, k3 int, k4 bigint, k5 decimal(9,3), k6 char(5), k10 date, k11 datetime, k7 varchar(20), k8 double max, k9 float sum) engine=olap distributed by hash(k1) buckets 5 properties("storage_type"="column","replication_num"="1") ;

insert into testaaa values(1,1,1,1,9.3, "k6", "2023-11-14", "2023-11-14", "k7", 9.99, 9.99);


select  interval 10 year + k10 from testaaa;
The plan result type is DATE:

mysql [test]>explain verbose select   interval 10 year + k10  from testaaa;
+-------------------------------------------------------------------------------------------------------+
| Explain String(Nereids Planner)                                                                       |
+-------------------------------------------------------------------------------------------------------+
| PLAN FRAGMENT 0                                                                                       |
|   OUTPUT EXPRS:                                                                                       |
|     years_add(k10, INTERVAL 10 YEAR)[#11]                                                             |
|   PARTITION: UNPARTITIONED                                                                            |
|                                                                                                       |
|   HAS_COLO_PLAN_NODE: false                                                                           |
|                                                                                                       |
|   VRESULT SINK                                                                                        |
|      MYSQL_PROTOCAL                                                                                   |
|                                                                                                       |
|   64:VEXCHANGE                                                                                        |
|      offset: 0                                                                                        |
|      tuple ids: 1N                                                                                    |
|                                                                                                       |
| PLAN FRAGMENT 1                                                                                       |
|                                                                                                       |
|   PARTITION: HASH_PARTITIONED: k1[#0]                                                                 |
|                                                                                                       |
|   HAS_COLO_PLAN_NODE: false                                                                           |
|                                                                                                       |
|   STREAM DATA SINK                                                                                    |
|     EXCHANGE ID: 64                                                                                   |
|     UNPARTITIONED                                                                                     |
|                                                                                                       |
|   58:VOlapScanNode                                                                                    |
|      TABLE: default_cluster:test.testaaa(testaaa), PREAGGREGATION: OFF. Reason: No aggregate on scan. |
|      partitions=1/1 (testaaa), tablets=5/5, tabletList=945025,945027,945029 ...                       |
|      cardinality=1, avgRowSize=9885.0, numNodes=1                                                     |
|      pushAggOp=NONE                                                                                   |
|      projections: years_add(k10[#6], INTERVAL 10 YEAR)                                                |
|      project output tuple id: 1                                                                       |
|      tuple ids: 0                                                                                     |
|                                                                                                       |
| Tuples:                                                                                               |
| TupleDescriptor{id=0, tbl=testaaa, byteSize=8}                                                        |
|   SlotDescriptor{id=6, col=k10, colUniqueId=6, type=DATEV2, nullable=true, isAutoIncrement=false}     |
|                                                                                                       |
| TupleDescriptor{id=1, tbl=testaaa, byteSize=32}                                                       |
|   SlotDescriptor{id=11, col=null, colUniqueId=null, type=DATE, nullable=true, isAutoIncrement=false}  |
+-------------------------------------------------------------------------------------------------------+
39 rows in set (1 min 31.50 sec)
coredump stack:

F1109 20:11:37.677680 323805 assert_cast.h:61] Bad cast from type:doris::vectorized::ColumnVector to doris::vectorized::ColumnVector
*** Check failure stack trace: ***
F1109 20:11:37.680608 323800 assert_cast.h:61] Bad cast from type:doris::vectorized::ColumnVector to doris::vectorized::ColumnVector
*** Check failure stack trace: ***
F1109 20:11:37.680608 323800 assert_cast.h:61] Bad cast from type:doris::vectorized::ColumnVector to doris::vectorized::ColumnVectorF1109 20:11:37.681102 323808 assert_cast.h:61] Bad cast from type:doris::vectorized::ColumnVector to doris::vectorized::ColumnVector
*** Check failure stack trace: ***
    @     0x56489d591d3d  google::LogMessage::Fail()
    @     0x56489d591d3d  google::LogMessage::Fail()
    @     0x56489d591d3d  google::LogMessage::Fail()
    @     0x56489d594279  google::LogMessage::SendToLog()
    @     0x56489d594279  google::LogMessage::SendToLog()
    @     0x56489d594279  google::LogMessage::SendToLog()
    @     0x56489d5918a6  google::LogMessage::Flush()
    @     0x56489d5918a6  google::LogMessage::Flush()
    @     0x56489d5918a6  google::LogMessage::Flush()
    @     0x56489d5948e9  google::LogMessageFatal::~LogMessageFatal()
    @     0x56489d5948e9  google::LogMessageFatal::~LogMessageFatal()
    @     0x56489d5948e9  google::LogMessageFatal::~LogMessageFatal()
    @     0x56487a2a8a0c  assert_cast<>()
    @     0x56487a2a8a0c  assert_cast<>()
    @     0x56487a2a8a0c  assert_cast<>()
    @     0x5648893d8312  doris::vectorized::ColumnVector<>::insert_range_from()
    @     0x5648893d8312  doris::vectorized::ColumnVector<>::insert_range_from()
    @     0x5648893d8312  doris::vectorized::ColumnVector<>::insert_range_from()
    @     0x56488924a670  doris::vectorized::ColumnNullable::insert_range_from()
    @     0x56488924a670  doris::vectorized::ColumnNullable::insert_range_from()
    @     0x56488924a670  doris::vectorized::ColumnNullable::insert_range_from()
    @     0x56487a454475  doris::ExecNode::do_projections()
    @     0x56487a454475  doris::ExecNode::do_projections()
    @     0x56487a454475  doris::ExecNode::do_projections()
    @     0x56487a454b89  doris::ExecNode::get_next_after_projects()
    @     0x56487a454b89  doris::ExecNode::get_next_after_projects()
*** Query id: a467995b35334741-b625042f56495aaf ***
*** tablet id: 0 ***
*** Aborted at 1699531898 (unix time) try "date -d @1699531898" if you are using GNU date ***
*** Current BE git commitID: 0d83327a7c ***
*** SIGABRT unknown detail explain (@0x190d64) received by PID 1641828 (TID 1642168 OR 0x7f6ff96c0700) from PID 1641828; stack trace: ***
    @     0x556ca2a3ab8f  std::_Function_handler<>::_M_invoke()
    @     0x556c9f322787  std::function<>::operator()()
    @     0x556ca29da0b0  doris::Thread::supervise_thread()
    @     0x556c9f322787  std::function<>::operator()()
    @     0x7f71b9c38609  start_thread
    @     0x556ca29da0b0  doris::Thread::supervise_thread()
    @     0x7f71b9c38609  start_thread
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/common/signal_handler.h:417
 1# 0x00007F71B9E09090 in /lib/x86_64-linux-gnu/libc.so.6
 2# raise at ../sysdeps/unix/sysv/linux/raise.c:51
 3# abort at /build/glibc-SzIz7B/glibc-2.31/stdlib/abort.c:81
 4# 0x0000556CC51F3729 in /mnt/hdd01/ci/branch20-deploy/be/lib/doris_be
 5# 0x0000556CC51E8D3D in /mnt/hdd01/ci/branch20-deploy/be/lib/doris_be
 6# google::LogMessage::SendToLog() in /mnt/hdd01/ci/branch20-deploy/be/lib/doris_be
 7# google::LogMessage::Flush() in /mnt/hdd01/ci/branch20-deploy/be/lib/doris_be
 8# google::LogMessageFatal::~LogMessageFatal() in /mnt/hdd01/ci/branch20-deploy/be/lib/doris_be
 9# doris::vectorized::ColumnVector const& assert_cast const&, doris::vectorized::IColumn const&>(doris::vectorized::IColumn const&) in /mnt/hdd01/ci/branch20-deploy/be/lib/doris_be
10# doris::vectorized::ColumnVector::insert_range_from(doris::vectorized::IColumn const&, unsigned long, unsigned long) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/vec/columns/column_vector.cpp:354
11# doris::vectorized::ColumnNullable::insert_range_from(doris::vectorized::IColumn const&, unsigned long, unsigned long) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/vec/columns/column_nullable.cpp:289
12# doris::ExecNode::do_projections(doris::vectorized::Block*, doris::vectorized::Block*) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/exec/exec_node.cpp:573
13# doris::ExecNode::get_next_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*, std::function const&, bool) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/exec/exec_node.cpp:592
14# doris::pipeline::SourceOperator::get_block(doris::RuntimeState*, doris::vectorized::Block*, doris::pipeline::SourceState&) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/pipeline/exec/operator.h:413
15# doris::pipeline::PipelineTask::execute(bool*) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/pipeline/pipeline_task.cpp:259
16# doris::pipeline::TaskScheduler::_do_work(unsigned
 2023-11-14 20:28:41 +08:00
amory
fd92810df8 [FIX](regresstest)fixed out file for test-cast-map-function test (#26389) 2023-11-03 05:47:10 -05:00
amory
93a934e775 [Improve](map) support map cast with map literal and implicate nested scala cast (#26126) 2023-11-02 09:56:42 -05:00
谢健
c46fa339d0 [feature](Nereids): make count() avaliable for all data type (#26180) 
make count() avaliable for all data type
 2023-11-02 17:20:24 +08:00
Tiewei Fang
3e10e5af39 [Fix](Serde) Fix content displayed by complex types in MySQL Client (#25946) 
This pr makes three changes to the display of complex types：
1. NULL value in complex types refers to being displayed as `null`, not `NULL`
2. struct type is displayed as "column_name": column_value
3. Time types such as `datetime` and `date`, are displayed with double quotes in complex types. like
    `{1, "2023-10-26 12:12:12"}`

This pr also do a code refactor:
1. nesting_level is set to a member variable of the `DataTypeSerDe`, rather than a parameter in methods.

What's more, this pr fix a bug that fileSize is not correct, introduced by this pr: #25854
 2023-11-01 23:48:55 +08:00
Yulei-Yang
3a954cd1aa [fix](function)return NULL rather than 'null' if path not found (#25880) 
fix json_extract not return NULL but null
 2023-10-30 14:26:44 +08:00
Liqf
57340a4187 [feature](nereids) add function array_agg (#25630) 2023-10-23 02:05:54 -05:00
TengJianPing
e77b98be88 [fix](months_diff) fix wrong result of months_diff (#25577) 2023-10-19 14:29:47 +08:00
Liqf
2ddd2e5079 [feature](Nereids) add map_agg function (#25246) 2023-10-18 06:44:36 -05:00
Jerry Hu
62d06584f1 [feature](fe) add function 'BitmapAgg' in nereids (#25508) 2023-10-18 14:24:27 +08:00
morrySnow
af8832389f [feature](Nereids) add 4 array functions (#25488) 
- array_concat
- array_pushback
- array_pushfront
- array_zip
 2023-10-17 04:45:15 -05:00
morrySnow
4c57c31c5c [fix](Nereids) count should not accept complex and json type (#25354) 2023-10-15 22:08:35 -05:00
minghong
42f8b253aa [function](nereids) support array_apply/array_repeat/group_uniq_array/ipv4numtostring (#25249) 
nereids support functions: array_apply/array_repeat/group_uniq_array/ipv4numtostring
 2023-10-12 11:08:42 +08:00
morrySnow
b91bce8a62 [feature](Nereids) add array distance functions (#25196) 
- l1_distance
- l2_distance
- cosine_distance
- inner_product
 2023-10-10 21:35:06 -05:00
morrySnow
181c58c691 [fix](Nereids) count_by_enum signature is wrong (#25167) 2023-10-10 13:05:20 +08:00
morrySnow
37247ac449 [opt](Nereids) add two args signature to trim family functions (#25169) 2023-10-09 07:17:52 -05:00
morrySnow
320709b9ff [opt](Nereids) support like and regexp function (#25148) 2023-10-09 02:55:57 -05:00
谢健
3a45001447 [fix](Nerids) fix error when the view has lambda functions (#25067) 
1. To ensure compatibility with the original optimizer, expose the non-lambda signature of highorder function externally.
2. fix some bugs in toSql function in the original optimizer
 2023-10-08 15:45:24 +08:00