Commit Graph

1207 Commits

Author SHA1 Message Date
7248420cfd [chore](session_variable) Add 'data_queue_max_blocks' to prevent the DataQueue from occupying too much memory. (#34017) (#34395) 2024-05-05 21:20:33 +08:00
35f8563a75 [feature](iceberg) support iceberg equality delete (#34223) (#34327)
bp #34223

Co-authored-by: Ashin Gau <AshinGau@users.noreply.github.com>
2024-04-30 11:51:29 +08:00
7cb00a8e54 [Feature](hive-writer) Implements s3 file committer. (#34307)
Backport #33937.
2024-04-29 19:56:49 +08:00
a6bf35efdf 2.1.3-rc03 2024-04-27 20:54:06 +08:00
a34ed4643a [fix](planner)date_add function should accept date type as its param (#34035) 2024-04-26 15:04:45 +08:00
a050513c91 [Fix](clean trash) Fix clean trash use agent task (#33912) (#33972)
* [Fix](clean trash) Fix clean trash use agent task (#33912)

* add .h
2024-04-22 17:14:21 +08:00
7f61626c8d [fix](arrow_flight_sql) Fix ArrowSchema column alias (#33490)
run: select TABLE_SCHEMA as a, sum(TABLE_ROWS) as b  from tables group by TABLE_SCHEMA limit 2;
old output:

          TABLE_SCHEMA                        Nullable(Int64)_1
0  regression_test_mv_p0_sum_count           9
1  regression_test_query_p0_sql_functions_string_functions       70414
now output:

          a                        b
0  regression_test_mv_p0_sum_count            9
1  regression_test_query_p0_sql_functions_string_functions       70414
2024-04-22 11:28:22 +08:00
4aee706264 2.1.3-rc02 2024-04-21 21:07:56 +08:00
c6fad11154 2.1.3-rc01 2024-04-21 17:39:05 +08:00
03c3419265 [Refactor](executor)Add workload schedule policy table (#33729) 2024-04-20 20:06:34 +08:00
315f6e44c2 [Branch-2.1](Outfile) Fixed the problem that the concurrent Outfile wrote multiple Success files (#33870)
backport: #33016
2024-04-19 12:09:53 +08:00
20b37e7a18 Add workload group id in workload policy's property (#33483) 2024-04-17 23:42:14 +08:00
22a6b1d3f5 [feature](function) support hll functions hll_from_base64, hll_to_base64 (#32089)
Issue Number: #31320 

Support two hll functions:

- hll_from_base64
Convert a base64 string(result of function hll_to_base64) into a hll.
- hll_to_base64
Convert an input hll to a base64 string.
2024-04-17 23:42:13 +08:00
e0ec2da29b [fix](routine-load) fix get kafka offset timeout may too long (#33502) 2024-04-17 23:42:12 +08:00
06a155abb0 [branch-2.1](cherry-pick) Pick some partial-update PR from master (#33639)
* [Fix](partial-update) Fix partial update fail when the datetime default value is 'current_time' (#32926)

* Problem: When importing data that includes datetime with a default value of current time for partial column updates, the import fails.
Reason: Partial column updates do not handle the logic for datetime default values.
Solution: During partial column updates, when the default value is set to current time, read the current time from the runtime state and write it into the data.

* [Enhancement](partial update)Add timezone case for partial update timestamp #33177

* [fix](partial update) Support partial update when the date default value is 'current_date'. This PR is a extension of PR #32926. (#33394)
2024-04-17 23:42:12 +08:00
face7c42fd [enhancement](plsql) Support select * from routines (#32866)
Support show of plsql procedure using select * from routines.
2024-04-17 23:42:12 +08:00
1be753ed75 [enhancement](mysql compatible) add user and procs_priv tables to mysql db in all catalogs (#33058)
Issue Number: close #xxx

This PR aims to enhance the compatibility of BI tools (such as Dbeaver, DataGrip) when using the mysql connector to connect to Doris, because some BI tools query some tables in the mysql database. In our tests, the user and procs_priv tables were mainly queried. This PR adds these two tables and adds actual data to the user table. However, please note that most of the fields in the user table are in Doris' own format rather than mysql format, so it can only ensure that the BI tool is querying No error is reported when accessing these tables, which does not guarantee that the data is completely displayed, and the tables under Doris's mysql database do not support data modification.
Thanks to @liujiwen-up for assisting in testing
2024-04-17 23:42:12 +08:00
b2b385a4ff [improve](fold) support complex type for constant folding (#32867) 2024-04-17 23:41:59 +08:00
Pxl
5f30463bb3 [Chore](descriptors) remove unused codes for descriptors (#33408)
remove unused codes for descriptors
2024-04-12 15:09:25 +08:00
26d9082b9a [Feature](function) Add function strcmp (#33272) 2024-04-12 15:09:25 +08:00
31984bb4f0 [feature](function) support quote string function #33055 2024-04-12 15:09:25 +08:00
716c146750 [fix](insert)fix hive external return msgs and exception and pass all columns to BE (#32824)
[fix](insert)fix hive external return msgs and exception and pass all columns to BE
2024-04-12 10:23:52 +08:00
3d66723214 [branch-2.1](auto-partition) pick auto partition and some more prs (#33523) 2024-04-11 17:12:17 +08:00
ef26479282 [improve](serde) support complex type in write/read pb serde (#33124)
support complex type and ip/jsonb in DataTypeSerDe::write_column_to_pb/read_column_from_pb function
2024-04-11 09:31:50 +08:00
Pxl
3081fc584d [Improvement](runtime-filter) support sync join node build side's size to init bloom runtime filter (#32180)
support sync join node build side's size to init bloom runtime filter
2024-04-11 09:31:50 +08:00
67bb519613 [Fix](nereids) forward the user define variables to master (#33013) 2024-04-10 15:26:08 +08:00
c61d6ad1e2 [Feature] support function uuid_to_int and int_to_uuid #33005 2024-04-10 14:53:56 +08:00
8e19cdd745 [featrue](expr) support common subexpression elimination be part (#32673) 2024-04-10 11:56:21 +08:00
96b995504c [enhancement](statistics) excluded delta rows num for rollup&mv tablets (#32568)
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>
Co-authored-by: tsy <tangsiyang2001@foxmail.com>
2024-04-10 11:34:28 +08:00
b130df2488 2.1.2-rc04 2024-04-09 16:28:27 +08:00
a1f80eaa7a 2.1.2-rc03 2024-04-09 12:49:05 +08:00
b5b0181a79 2.1.2-rc02 2024-04-09 12:37:31 +08:00
0e1a15960c 2.1.2-rc01 2024-04-08 23:17:15 +08:00
9f2520537f 2.1.1-rc05 2024-03-30 21:26:27 +08:00
67856d9c22 change to 2.1.1 rc04 2024-03-29 10:52:03 +08:00
3620a93da8 2.1.1-rc03 2024-03-28 00:16:50 +08:00
6ad63bb51b 2.1.1-rc02 2024-03-27 21:46:40 +08:00
62c7d0a421 [Fix](point query) add query options for short circuit queries (#32530) (#32684)
Some options like `be_exec_version` needed for functions
2024-03-22 18:03:18 +08:00
326a264fcd [Improvement](executor)Add spill property for workload group #32554 2024-03-22 16:38:19 +08:00
baf3ae1a93 [refactor](nereids)unify outputTupleDesc and projection be part (#32439) 2024-03-22 16:35:43 +08:00
0cde0cbf19 (invert index) modify of time series compaction policy 2024-03-22 08:16:30 +08:00
e892774c9a [improvement](agg) streaming agg should not take too much memory when spilling enabled (#32426) 2024-03-21 14:07:24 +08:00
85b2c42f76 [Enhancement](jdbc catalog) Add a property to test the connection when creating a Jdbc catalog (#32125) (#32531) 2024-03-21 14:05:59 +08:00
ecadb60bcd [Pick 2.1](inverted index) support inverted index format v2 (#30145) (#32418) 2024-03-19 08:11:33 +08:00
b82de68d7e [feature][insert]add hive table sink thrift (#32274) (#32360)
bp #32274
2024-03-18 10:46:17 +08:00
1645f2e0a7 [feature](insert)add hive table sink definition (#31662) (#32347)
bp #31662
Co-authored-by: slothever <18522955+wsjz@users.noreply.github.com>
2024-03-17 20:52:44 +08:00
258dcfca97 [Refactor](executor)Add information_schema.workload_groups (#32195) (#32314) 2024-03-15 20:46:54 +08:00
e3bb499cc6 [fix](function)revert function REPEAT nullable mode #32226 2024-03-15 18:06:28 +08:00
97b35d6830 [fix](nereids)AssertNumRow node's output should be nullable (#32136)
Co-authored-by: Co-Author Jerry Hu <mrhhsg@gmail.com>
2024-03-15 18:06:28 +08:00
9c1888e7ec [RuntimeFilter](exec) support min max runtime filter and do refactor (#32210) 2024-03-15 18:06:20 +08:00