database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
17,549 Commits 1 Branch 0 Tags
ea2fbfaffaf9cd09c3cfa8eabbde415dfdbeeade
Commit Graph

95 Commits

Author SHA1 Message Date
Mryange
cca0773261 [feature](function) round function defaults to rounding normally 2024-03-06 13:06:26 +08:00
Guangdong Liu
9c4708ee74 [function](random_bytes)add random_bytes function (#31547) 
SELECT random_bytes(10);

random_bytes(10) |
----------------------+
0x9b8ea00b7d1084bc5b26|
 2024-02-29 16:44:39 +08:00
Chester
f163d56a98 [feature](function) support sequence function(alias of array_range), enhance both to handle datetimev2 (#30823) 2024-02-27 10:12:19 +08:00
vhwzIs
d70776af55 [feature](agg-func) support covar and covar_samp function (#30983) 2024-02-18 11:50:17 +08:00
nanfeng
e68019c10a [Function](Exec) Support windows function cume_dist (#30997) 2024-02-16 10:16:40 +08:00
yangshijie
40e1326bc9 [feature](window-func) support percent_rank window function (#30926) 2024-02-16 10:12:25 +08:00
nanfeng
2bb477bae7 [feature](agg-func) support corr function #30822 2024-02-16 10:12:24 +08:00
Guangdong Liu
1ed24117ac [function](url_decode)add url_decode function (#30667) 2024-02-05 22:23:00 +08:00
Rohit Satardekar
6442663735 [Function](exec) upport atan2 math function (#30672) 
Co-authored-by: Rohit Satardekar <rohitrs1983@gmail.com>
 2024-02-04 14:28:38 +08:00
Rohit Satardekar
19f57b544e support cosh math function (#30602) 
Co-authored-by: Rohit Satardekar <rohitrs1983@gmail.com>
 2024-01-31 23:53:39 +08:00
yangshijie
8b61b7c6cd [exec](function) Add tanh func (#30555) 2024-01-31 23:53:39 +08:00
Chester
7e19224a6c [fix](function) fix ipv4 funcs get failed error, improve an ipv6 func and exception message (#30269) 2024-01-28 18:25:31 +08:00
koarz
ca5a314765 [fix](function) make STRLEFT and STRRIGHT and SUBSTR function DEPEND_ON_ARGUMENT (#28352) 
make STRLEFT and STRRIGHT function DEPEND_ON_ARGUMENT
 2024-01-25 13:23:59 +08:00
Chester
dfde10d4c8 [improvement](function) switch inet(6)_aton alias origin function (#30196) 2024-01-23 10:09:54 +08:00
Chester
ead3b4ac1d [feature](function) support ip function is_ipv4_compat, is_ipv4_mapped (#29954) 2024-01-23 10:07:51 +08:00
yangshijie
eb4c389b0b [feature](function) support ip functions isipv4string and isipv6string (#28556) 2024-01-07 13:03:11 +08:00
Chester
7402fee1fc [feature](function) support ip function ipv6_string_to_num(_or_default, _or_null), inet6_aton (#28361) 2024-01-05 19:24:45 +08:00
Chester
2e1ce758f1 [feature](function) support ip function ipv6numtostring(alias inet6_ntoa) (#27342) 2023-12-02 11:48:19 +08:00
zclllyybb
baadc14e60 [Enhancement](function) support unix_timestamp with float (#26827) 
---------

Co-authored-by: YangWithU <plzw8@outlook.com>
 2023-11-27 09:58:53 +08:00
Chester
334260dff7 [feature](function) support ip function ipv4stringtonum(ordefault, ornull), inet_aton (#25510) 2023-11-17 10:27:07 +08:00
morrySnow
2c6d2255c3 [fix](Nereids) nested type literal type coercion and insert values with map (#26669) 2023-11-14 21:13:26 -06:00
TengJianPing
9a4fd5be79 [nereids](datetime) fix wrong result type of datetime add with interval as first arg (#26957) 
Incorrect result data type cause be cordump:

drop table if exists testaaa;
create table testaaa(k1 tinyint, k2 smallint, k3 int, k4 bigint, k5 decimal(9,3), k6 char(5), k10 date, k11 datetime, k7 varchar(20), k8 double max, k9 float sum) engine=olap distributed by hash(k1) buckets 5 properties("storage_type"="column","replication_num"="1") ;

insert into testaaa values(1,1,1,1,9.3, "k6", "2023-11-14", "2023-11-14", "k7", 9.99, 9.99);


select  interval 10 year + k10 from testaaa;
The plan result type is DATE:

mysql [test]>explain verbose select   interval 10 year + k10  from testaaa;
+-------------------------------------------------------------------------------------------------------+
| Explain String(Nereids Planner)                                                                       |
+-------------------------------------------------------------------------------------------------------+
| PLAN FRAGMENT 0                                                                                       |
|   OUTPUT EXPRS:                                                                                       |
|     years_add(k10, INTERVAL 10 YEAR)[#11]                                                             |
|   PARTITION: UNPARTITIONED                                                                            |
|                                                                                                       |
|   HAS_COLO_PLAN_NODE: false                                                                           |
|                                                                                                       |
|   VRESULT SINK                                                                                        |
|      MYSQL_PROTOCAL                                                                                   |
|                                                                                                       |
|   64:VEXCHANGE                                                                                        |
|      offset: 0                                                                                        |
|      tuple ids: 1N                                                                                    |
|                                                                                                       |
| PLAN FRAGMENT 1                                                                                       |
|                                                                                                       |
|   PARTITION: HASH_PARTITIONED: k1[#0]                                                                 |
|                                                                                                       |
|   HAS_COLO_PLAN_NODE: false                                                                           |
|                                                                                                       |
|   STREAM DATA SINK                                                                                    |
|     EXCHANGE ID: 64                                                                                   |
|     UNPARTITIONED                                                                                     |
|                                                                                                       |
|   58:VOlapScanNode                                                                                    |
|      TABLE: default_cluster:test.testaaa(testaaa), PREAGGREGATION: OFF. Reason: No aggregate on scan. |
|      partitions=1/1 (testaaa), tablets=5/5, tabletList=945025,945027,945029 ...                       |
|      cardinality=1, avgRowSize=9885.0, numNodes=1                                                     |
|      pushAggOp=NONE                                                                                   |
|      projections: years_add(k10[#6], INTERVAL 10 YEAR)                                                |
|      project output tuple id: 1                                                                       |
|      tuple ids: 0                                                                                     |
|                                                                                                       |
| Tuples:                                                                                               |
| TupleDescriptor{id=0, tbl=testaaa, byteSize=8}                                                        |
|   SlotDescriptor{id=6, col=k10, colUniqueId=6, type=DATEV2, nullable=true, isAutoIncrement=false}     |
|                                                                                                       |
| TupleDescriptor{id=1, tbl=testaaa, byteSize=32}                                                       |
|   SlotDescriptor{id=11, col=null, colUniqueId=null, type=DATE, nullable=true, isAutoIncrement=false}  |
+-------------------------------------------------------------------------------------------------------+
39 rows in set (1 min 31.50 sec)
coredump stack:

F1109 20:11:37.677680 323805 assert_cast.h:61] Bad cast from type:doris::vectorized::ColumnVector to doris::vectorized::ColumnVector
*** Check failure stack trace: ***
F1109 20:11:37.680608 323800 assert_cast.h:61] Bad cast from type:doris::vectorized::ColumnVector to doris::vectorized::ColumnVector
*** Check failure stack trace: ***
F1109 20:11:37.680608 323800 assert_cast.h:61] Bad cast from type:doris::vectorized::ColumnVector to doris::vectorized::ColumnVectorF1109 20:11:37.681102 323808 assert_cast.h:61] Bad cast from type:doris::vectorized::ColumnVector to doris::vectorized::ColumnVector
*** Check failure stack trace: ***
    @     0x56489d591d3d  google::LogMessage::Fail()
    @     0x56489d591d3d  google::LogMessage::Fail()
    @     0x56489d591d3d  google::LogMessage::Fail()
    @     0x56489d594279  google::LogMessage::SendToLog()
    @     0x56489d594279  google::LogMessage::SendToLog()
    @     0x56489d594279  google::LogMessage::SendToLog()
    @     0x56489d5918a6  google::LogMessage::Flush()
    @     0x56489d5918a6  google::LogMessage::Flush()
    @     0x56489d5918a6  google::LogMessage::Flush()
    @     0x56489d5948e9  google::LogMessageFatal::~LogMessageFatal()
    @     0x56489d5948e9  google::LogMessageFatal::~LogMessageFatal()
    @     0x56489d5948e9  google::LogMessageFatal::~LogMessageFatal()
    @     0x56487a2a8a0c  assert_cast<>()
    @     0x56487a2a8a0c  assert_cast<>()
    @     0x56487a2a8a0c  assert_cast<>()
    @     0x5648893d8312  doris::vectorized::ColumnVector<>::insert_range_from()
    @     0x5648893d8312  doris::vectorized::ColumnVector<>::insert_range_from()
    @     0x5648893d8312  doris::vectorized::ColumnVector<>::insert_range_from()
    @     0x56488924a670  doris::vectorized::ColumnNullable::insert_range_from()
    @     0x56488924a670  doris::vectorized::ColumnNullable::insert_range_from()
    @     0x56488924a670  doris::vectorized::ColumnNullable::insert_range_from()
    @     0x56487a454475  doris::ExecNode::do_projections()
    @     0x56487a454475  doris::ExecNode::do_projections()
    @     0x56487a454475  doris::ExecNode::do_projections()
    @     0x56487a454b89  doris::ExecNode::get_next_after_projects()
    @     0x56487a454b89  doris::ExecNode::get_next_after_projects()
*** Query id: a467995b35334741-b625042f56495aaf ***
*** tablet id: 0 ***
*** Aborted at 1699531898 (unix time) try "date -d @1699531898" if you are using GNU date ***
*** Current BE git commitID: 0d83327a7c ***
*** SIGABRT unknown detail explain (@0x190d64) received by PID 1641828 (TID 1642168 OR 0x7f6ff96c0700) from PID 1641828; stack trace: ***
    @     0x556ca2a3ab8f  std::_Function_handler<>::_M_invoke()
    @     0x556c9f322787  std::function<>::operator()()
    @     0x556ca29da0b0  doris::Thread::supervise_thread()
    @     0x556c9f322787  std::function<>::operator()()
    @     0x7f71b9c38609  start_thread
    @     0x556ca29da0b0  doris::Thread::supervise_thread()
    @     0x7f71b9c38609  start_thread
 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/common/signal_handler.h:417
 1# 0x00007F71B9E09090 in /lib/x86_64-linux-gnu/libc.so.6
 2# raise at ../sysdeps/unix/sysv/linux/raise.c:51
 3# abort at /build/glibc-SzIz7B/glibc-2.31/stdlib/abort.c:81
 4# 0x0000556CC51F3729 in /mnt/hdd01/ci/branch20-deploy/be/lib/doris_be
 5# 0x0000556CC51E8D3D in /mnt/hdd01/ci/branch20-deploy/be/lib/doris_be
 6# google::LogMessage::SendToLog() in /mnt/hdd01/ci/branch20-deploy/be/lib/doris_be
 7# google::LogMessage::Flush() in /mnt/hdd01/ci/branch20-deploy/be/lib/doris_be
 8# google::LogMessageFatal::~LogMessageFatal() in /mnt/hdd01/ci/branch20-deploy/be/lib/doris_be
 9# doris::vectorized::ColumnVector const& assert_cast const&, doris::vectorized::IColumn const&>(doris::vectorized::IColumn const&) in /mnt/hdd01/ci/branch20-deploy/be/lib/doris_be
10# doris::vectorized::ColumnVector::insert_range_from(doris::vectorized::IColumn const&, unsigned long, unsigned long) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/vec/columns/column_vector.cpp:354
11# doris::vectorized::ColumnNullable::insert_range_from(doris::vectorized::IColumn const&, unsigned long, unsigned long) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/vec/columns/column_nullable.cpp:289
12# doris::ExecNode::do_projections(doris::vectorized::Block*, doris::vectorized::Block*) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/exec/exec_node.cpp:573
13# doris::ExecNode::get_next_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*, std::function const&, bool) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/exec/exec_node.cpp:592
14# doris::pipeline::SourceOperator::get_block(doris::RuntimeState*, doris::vectorized::Block*, doris::pipeline::SourceState&) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/pipeline/exec/operator.h:413
15# doris::pipeline::PipelineTask::execute(bool*) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/pipeline/pipeline_task.cpp:259
16# doris::pipeline::TaskScheduler::_do_work(unsigned
 2023-11-14 20:28:41 +08:00
amory
fd92810df8 [FIX](regresstest)fixed out file for test-cast-map-function test (#26389) 2023-11-03 05:47:10 -05:00
amory
93a934e775 [Improve](map) support map cast with map literal and implicate nested scala cast (#26126) 2023-11-02 09:56:42 -05:00
谢健
c46fa339d0 [feature](Nereids): make count() avaliable for all data type (#26180) 
make count() avaliable for all data type
 2023-11-02 17:20:24 +08:00
Tiewei Fang
3e10e5af39 [Fix](Serde) Fix content displayed by complex types in MySQL Client (#25946) 
This pr makes three changes to the display of complex types：
1. NULL value in complex types refers to being displayed as `null`, not `NULL`
2. struct type is displayed as "column_name": column_value
3. Time types such as `datetime` and `date`, are displayed with double quotes in complex types. like
    `{1, "2023-10-26 12:12:12"}`

This pr also do a code refactor:
1. nesting_level is set to a member variable of the `DataTypeSerDe`, rather than a parameter in methods.

What's more, this pr fix a bug that fileSize is not correct, introduced by this pr: #25854
 2023-11-01 23:48:55 +08:00
Yulei-Yang
3a954cd1aa [fix](function)return NULL rather than 'null' if path not found (#25880) 
fix json_extract not return NULL but null
 2023-10-30 14:26:44 +08:00
Liqf
57340a4187 [feature](nereids) add function array_agg (#25630) 2023-10-23 02:05:54 -05:00
TengJianPing
e77b98be88 [fix](months_diff) fix wrong result of months_diff (#25577) 2023-10-19 14:29:47 +08:00
Liqf
2ddd2e5079 [feature](Nereids) add map_agg function (#25246) 2023-10-18 06:44:36 -05:00
Jerry Hu
62d06584f1 [feature](fe) add function 'BitmapAgg' in nereids (#25508) 2023-10-18 14:24:27 +08:00
morrySnow
af8832389f [feature](Nereids) add 4 array functions (#25488) 
- array_concat
- array_pushback
- array_pushfront
- array_zip
 2023-10-17 04:45:15 -05:00
morrySnow
4c57c31c5c [fix](Nereids) count should not accept complex and json type (#25354) 2023-10-15 22:08:35 -05:00
minghong
42f8b253aa [function](nereids) support array_apply/array_repeat/group_uniq_array/ipv4numtostring (#25249) 
nereids support functions: array_apply/array_repeat/group_uniq_array/ipv4numtostring
 2023-10-12 11:08:42 +08:00
morrySnow
b91bce8a62 [feature](Nereids) add array distance functions (#25196) 
- l1_distance
- l2_distance
- cosine_distance
- inner_product
 2023-10-10 21:35:06 -05:00
morrySnow
181c58c691 [fix](Nereids) count_by_enum signature is wrong (#25167) 2023-10-10 13:05:20 +08:00
morrySnow
37247ac449 [opt](Nereids) add two args signature to trim family functions (#25169) 2023-10-09 07:17:52 -05:00
morrySnow
320709b9ff [opt](Nereids) support like and regexp function (#25148) 2023-10-09 02:55:57 -05:00
谢健
3a45001447 [fix](Nerids) fix error when the view has lambda functions (#25067) 
1. To ensure compatibility with the original optimizer, expose the non-lambda signature of highorder function externally.
2. fix some bugs in toSql function in the original optimizer
 2023-10-08 15:45:24 +08:00
morrySnow
12bee54cf7 [fix](Nereids) function implict cast for json is not work well (#24852) 
for sql like:

SELECT JSONB_EXTRACT('{"k1":"v31","k2":300}','$.k1');

the result should be

+------------------------------------------------+
| jsonb_extract('{"k1":"v31","k2":300}', '$.k1') |
+------------------------------------------------+
| "v31"                                          |
+------------------------------------------------+

but curent result is

+------------------------------------------------+
| jsonb_extract('{"k1":"v31","k2":300}', '$.k1') |
+------------------------------------------------+
| <null>                                         |
+------------------------------------------------+
 2023-09-25 21:10:04 +08:00
TengJianPing
22616d125d [function](bitmap) add function alias bitmap_andnot and bitmap_andnot_count (#24771) 2023-09-22 12:18:31 +08:00
谢健
5b590bbfcf [feat](Nereids) add lambda func array_last and array_first (#24682) 2023-09-21 12:23:34 +08:00
LiBinfeng
a3361df7b9 [Feat](Nereids) support json and jsonb datatype (#24156) 
Feature:
support jsonb and json type in nereids

Document:
this feature supports these two datatype in nereids optimizer like original planner, the sql reference is same as before
[JSON - Apache Doris](https://doris.apache.org/zh-CN/docs/dev/sql-manual/sql-reference/Data-Types/JSON)
 2023-09-20 14:32:22 +08:00
amory
e9435c14f8 [Improve](array-func)improve array union support multi params (#24327) 2023-09-20 14:29:48 +08:00
谢健
c3bd2a22d4 [feature](Nereids) add many array functions (#24301) 
Add function array_filter, array_sortby, array_last_index, array_first_index, array_orderby, array_count
 2023-09-19 18:58:49 +08:00
daidai
4b5cea1ef8 [enhancement](fix)change ordinary type null value is \N,complex type null value is null (#24207) 2023-09-16 21:46:42 +08:00
谢健
88adab3114 [fix](Nereids): fix be core when array_map is not nullable (#24488) 
fix be core when array_map is not nullable
 2023-09-16 20:39:15 +08:00
zclllyybb
4fbb25bc55 [Enhancement](function) Support date_trunc(date) and use it in auto partition (#24341) 
Support date_trunc(date) and use it in auto partition
 2023-09-14 16:53:09 +08:00
amory
268c867679 [Improve](serde)replace function_cast from_string to serde (#24087) 
Now we can not support streamload with column which is map/array nested map/array
serde can do this now , so we can replace it
Notice. if item data in complex type data is empty we just return error, instead of makeup default value , because now we can not define right default for complex type
 2023-09-14 13:53:16 +08:00
amory
dc28878f0e [FIX](function) fix size function for array map (#23920) 
Issue Number: close #xxx
now we use select size(map(1, 2)); which will make be core
and we can make size function handle array & map column both
 2023-09-06 14:32:06 +08:00