Commit Graph

2010 Commits

Author SHA1 Message Date
855852d582 [enhancement](timeout) fix set timeout failure and simplify timeout logic (#17837) 2023-03-25 21:56:06 +08:00
f36465e76e [enhancement](memory) optimize jemalloc heap profile doc (#18094) 2023-03-25 13:04:45 +08:00
2408ca5da8 [Bug](DECIMALV3) Fix wrong precision for plus/minus (#18052)
Result type for DECIMAL(x, y) plus/minus DECIMAL(m, n) should be DECIMAL(max(x - y, m - n) + max(y + n) + 1, max(y + n))
2023-03-25 09:42:39 +08:00
0523860877 [Enhancement](streamload) print profile for streamload (#18015)
When both enable_profile and enable_stream_load_profile_log is true, stream load profile is printed to the log
2023-03-24 20:17:33 +08:00
039688978c [docs](doc) Add autobucket doc (#16746)
Signed-off-by: Jack Drogon <jack.xsuperman@gmail.com>
2023-03-24 19:58:58 +08:00
562f572311 [enhancement](UDF) The user defined functions support global ('show functions'/'show create') operation (#16973) (#17964)
1. add the global keyword.

SHOW [GLOBAL] [FULL] [BUILTIN] FUNCTIONS [IN|FROM db] [LIKE 'function_pattern']

SHOW CREATE GLOBAL FUNCTION function_name(arg_type [, ...]);

2. show the details of the global udf.
2023-03-24 19:07:38 +08:00
eb7b59c1c6 [docs](plugins) Fix the information in auditlog plugin documentation #18073
The information in the document is incomplete, user may be get error message like:

mysql> INSTALL PLUGIN FROM "http://127.0.0.1:8039/auditloader.zip";
ERROR 1105 (HY000): errCode = 2, detailMessage = http://127.0.0.1:8039/auditloader.zip.md5. you should set md5sum in plugin properties or provide a md5 URI to check plugin file
2023-03-24 18:16:16 +08:00
089a91ecd5 [vectorized](function) support array_exists lambda function (#17931)
Co-authored-by: zhangyu209 <zhangyu209@meituan.com>
2023-03-23 11:11:39 +08:00
d9059ef070 [Docs](multi-catalog) add yarn.resourcemanager.principal for hive catalog with kerberos enabled. (#17930)
Co-authored-by: wangxiangyu@360shuke.com <wangxiangyu@360shuke.com>
2023-03-22 23:34:33 +08:00
e2e806a5e7 [improve](clickhouse jdbc) support clickhouse array type (#17993)
In this PR, I match the array type of ClickHouse to the array type of Doris's jdbc external.
2023-03-22 19:42:32 +08:00
ebef0c038d Revert "[fix](function) fix AES/SM3/SM4 encrypt/ decrypt algorithm initialization vector bug (#17420)" (#17887)
This reverts commit 397cc011c4f1ba5a25c770258c13f1cd3f28b47d.
2023-03-22 13:28:25 +08:00
d4ca7cb57a [chore](macOS) Specify the version of LLVM for Homebrew to install it (#17945)
Clang 16 was released last week and we haven't ported the codebase to it. If Homebrew bumped the version of LLVM, our workflows would fail.
2023-03-22 11:09:28 +08:00
173d68409c [enhencement](planner) update and delete support use alias for target table (#17914) 2023-03-22 11:07:39 +08:00
213735b5fb [doc](auth)ranger doc (#17927) 2023-03-22 09:38:54 +08:00
4193884a32 [feature](array_zip) Support array_zip function (#17696) 2023-03-21 18:44:30 +08:00
6c8ed9135d [fix](truncate) fix unable to truncate table due to wrong storage medium (#17917)
When setting FE config default_storage_medium to SSD, and set all BE storage path as SSD.
And table will be stored with storage medium SSD.
But there is a FE config storage_cooldown_second and its default value is 30 days.
So after 30 days, the storage medium of table will be changed to HDD, which is unexpected.

This PR removes the storage_cooldown_second, and use a max value to set the cooldown time of SSD
storage medium when the default_storage_medium is SSD.
2023-03-21 10:04:47 +08:00
bae9d8d7f2 [Feature-Wip](MySQL LOAD)Add trim quotes property for mysql load (#17775)
Add trim quotes property for mysql load to trim double quotes in the load files.
2023-03-21 00:32:58 +08:00
dc284b62d9 [vectorized](function) support array_filter function (#17832) 2023-03-20 23:18:10 +08:00
4ef7839567 [doc](auth)auth doc version (#17943) 2023-03-20 13:24:00 +08:00
5bd00626be [doc](releasenote) 1.2.3 release note (#17926) 2023-03-19 15:01:29 +08:00
e359e412e1 [vectorized](udaf) fix java udaf meet error of std::bad_alloc (#17848)
Now if the user code of java udaf throws exception, because c++ code of agg function nobody could deal
with it, so maybe get error of std::bad_alloc
2023-03-19 11:52:15 +08:00
0de7f9787b [doc](typo) Fix a few character errors (#17911) 2023-03-19 09:09:02 +08:00
c95eb8a67f [enhancement] Function(create/drop) support the global operation (#16973) (#17608)
Support create/drop global function. 
     When you create a custom function, it can only be used within in one database. It cannot be used in other database/catalog. When there are many databases/catalog, it needs to create function one by one.

## Problem summary

Describe your changes.
1、 When a function is created or deleted, add the global keyword.

CREATE [GLOBAL] [AGGREGATE] [ALIAS] FUNCTION function_name (arg_type [, ...]) [RETURNS ret_type] [INTERMEDIATE inter_type] [WITH PARAMETER(param [,...]) AS origin_function] [PROPERTIES ("key" = "value" [, ...]) ]

DROP [GLOBAL] FUNCTION function_name (arg_type [, ...])

2、A completely global global function is set, and the global function metadata is stored in the image. The function lookup strategy is to look in the database first, and if it can't be found, it looks in the global function.
Co-authored-by: lexluo <lexluo@tencent.com>
2023-03-18 22:06:48 +08:00
49a053b3da [typo](docs) Add a hyperlink to facilitate user redirect. (#17899) 2023-03-18 21:21:36 +08:00
ee0f6120db [typo](doc)Replace invalid urls (#17886)
Co-authored-by: hechao <hechao@selectdb.com>
2023-03-18 21:20:52 +08:00
f099b7d7b5 [Doc](typo) Remove redundant symbols and Fix some misspelled words (#17609) 2023-03-17 09:01:59 +08:00
1222403233 [doc](typo)Fix docs insert into manual #17856 2023-03-17 08:57:02 +08:00
Hao
cf56a76108 [doc](typo)fix the different between cn and en document #17875 2023-03-17 08:52:51 +08:00
24dbdb6a2f Update es.md (#17816) 2023-03-16 20:14:47 +08:00
5dde910931 [feature](profile) add clean all profile sql (#17751)
Signed-off-by: nextdreamblue <zxw520blue1@163.com>
2023-03-16 16:12:21 +08:00
0086fdbbdb [enhancement](planner) support delete from using syntax (#17787)
support syntax delete using, this syntax only support UNIQUE KEY model

use the result of `t2` join `t3` to romve rows from `t1`

```sql
-- create t1, t2, t3 tables
CREATE TABLE t1
  (id INT, c1 BIGINT, c2 STRING, c3 DOUBLE, c4 DATE)
UNIQUE KEY (id)
DISTRIBUTED BY HASH (id)
PROPERTIES('replication_num'='1', "function_column.sequence_col" = "c4");

CREATE TABLE t2
  (id INT, c1 BIGINT, c2 STRING, c3 DOUBLE, c4 DATE)
DISTRIBUTED BY HASH (id)
PROPERTIES('replication_num'='1');

CREATE TABLE t3
  (id INT)
DISTRIBUTED BY HASH (id)
PROPERTIES('replication_num'='1');

-- insert data
INSERT INTO t1 VALUES
  (1, 1, '1', 1.0, '2000-01-01'),
  (2, 2, '2', 2.0, '2000-01-02'),
  (3, 3, '3', 3.0, '2000-01-03');

INSERT INTO t2 VALUES
  (1, 10, '10', 10.0, '2000-01-10'),
  (2, 20, '20', 20.0, '2000-01-20'),
  (3, 30, '30', 30.0, '2000-01-30'),
  (4, 4, '4', 4.0, '2000-01-04'),
  (5, 5, '5', 5.0, '2000-01-05');

INSERT INTO t3 VALUES
  (1),
  (4),
  (5);

-- remove rows from t1
DELETE FROM t1
  USING t2 INNER JOIN t3 ON t2.id = t3.id
  WHERE t1.id = t2.id;
```

the expect result is only remove the row where id = 1 in table t1

```
+----+----+----+--------+------------+
| id | c1 | c2 | c3     | c4         |
+----+----+----+--------+------------+
| 2  | 2  | 2  |    2.0 | 2000-01-02 |
| 3  | 3  | 3  |    3.0 | 2000-01-03 |
+----+----+----+--------+------------+
```
2023-03-16 13:12:00 +08:00
12d9d19366 [docs](Nereids) add nereids zh-CN docs (#16743) 2023-03-16 11:52:30 +08:00
e4a1e57d6f [feature](multi-catalog) support sap hana jdbc catalog and jdbc external table (#17780) 2023-03-15 20:37:36 +08:00
85080ee3c3 [vectorized](function) support array_map function (#17581) 2023-03-15 10:51:29 +08:00
ca0367d846 FIX: es doc (#17771) 2023-03-15 10:40:53 +08:00
76f486980a [docs](user)update the users number (#17749) 2023-03-14 22:42:51 +08:00
ff9e03e2bf [Feature](add bitmap udaf) add the bitmap intersection and difference set for mixed calculation of udaf (#15588)
* Add the bitmap intersection and difference set for mixed calculation of udaf

Co-authored-by: zhangbinbin05 <zhangbinbin05@baidu.com>
2023-03-14 20:40:37 +08:00
699159698e [enhancement](planner) support update from syntax (#17639)
support update from syntax

note: enable_concurrent_update is not supported now

```
UPDATE <target_table>
  SET <col_name> = <value> [ , <col_name> = <value> , ... ]
  [ FROM <additional_tables> ]
  [ WHERE <condition> ]
```

for example:
t1
```
+----+----+----+-----+------------+
| id | c1 | c2 | c3  | c4         |
+----+----+----+-----+------------+
| 3  | 3  | 3  | 3.0 | 2000-01-03 |
| 2  | 2  | 2  | 2.0 | 2000-01-02 |
| 1  | 1  | 1  | 1.0 | 2000-01-01 |
+----+----+----+-----+------------+
```

t2
```
+----+----+----+------+------------+
| id | c1 | c2 | c3   | c4         |
+----+----+----+------+------------+
| 4  | 4  | 4  |  4.0 | 2000-01-04 |
| 2  | 20 | 20 | 20.0 | 2000-01-20 |
| 5  | 5  | 5  |  5.0 | 2000-01-05 |
| 1  | 10 | 10 | 10.0 | 2000-01-10 |
| 3  | 30 | 30 | 30.0 | 2000-01-30 |
+----+----+----+------+------------+
```

t3
```
+----+
| id |
+----+
| 1  |
| 5  |
| 4  |
+----+
```

do update
```sql
 update t1 set t1.c1 = t2.c1, t1.c3 = t2.c3 * 100 from t2 inner join t3 on t2.id = t3.id where t1.id = t2.id;
```

the result
```
+----+----+----+--------+------------+
| id | c1 | c2 | c3     | c4         |
+----+----+----+--------+------------+
| 3  | 3  | 3  |    3.0 | 2000-01-03 |
| 2  | 2  | 2  |    2.0 | 2000-01-02 |
| 1  | 10 | 1  | 1000.0 | 2000-01-01 |
+----+----+----+--------+------------+
```
2023-03-14 19:26:30 +08:00
36a0d40ac3 Fix errors in the data-partition.md (#17756) 2023-03-14 10:44:57 +08:00
76458cf091 [typo](partition)Modify the list partition document #17744 2023-03-14 08:27:26 +08:00
883ae8a86d [typo](docs) Add some content for bitmap_hash.md. (#17747) 2023-03-14 08:27:07 +08:00
c302fa2564 [Feature](array-function) Support array_pushfront function (#17584) 2023-03-13 14:26:02 +08:00
47cfc81925 [fix docs] (#17634)
Co-authored-by: shenshoucheng <shenshoucheng@jd.com>
2023-03-13 08:06:33 +08:00
33059d92cc [docs](doc) fix faq docs (#17707) 2023-03-13 08:05:12 +08:00
455c800405 [feature](parquet-reader) add rle bool and delta decoder to read AWS Glue (#17112)
Support delta encoding and rle(bool) to read Glue data
add delta bit pack decoder,
add delta length byte array decoder,
add delta byte array decoder.
add rle bool decoder.

We find some data type is read with delta encoding on AWS Glue, so it should be supported.
The definition of delta encoding can refer to the delta encoding in parquet.
2023-03-12 20:09:58 +08:00
9b687026bd [Doc](TLS) add doc for TLS connection (#17683) 2023-03-12 10:01:07 +08:00
a74ef2377f typo fix in kyuubi doc (#17672) 2023-03-11 09:11:10 +08:00
48a2fe68ad [typo](docs) Fix some display errors (#17663)
* [fix](docs) fix some errors in docs
2023-03-11 09:10:48 +08:00
e1bf9411de [feature](array function) add support for array_enumerate_uniq (#17541)
add support for array_enumerate_uniq()
2023-03-10 10:20:49 +08:00
c7aa3f9717 [fix](backup) backup throw NPE when no partition in table (#17546)
If table has no partition, backup will report error:

2023-03-06 17:35:32,971 ERROR (backupHandler|24) [Daemon.run():118] daemon thread got exception. name: backupHandler
java.util.NoSuchElementException: No value present
        at java.util.Optional.get(Optional.java:135) ~[?:1.8.0_152]
        at org.apache.doris.catalog.OlapTable.selectiveCopy(OlapTable.java:1259) ~[doris-fe.jar:1.0-SNAPSHOT]
        at org.apache.doris.backup.BackupJob.prepareBackupMeta(BackupJob.java:505) ~[doris-fe.jar:1.0-SNAPSHOT]
        at org.apache.doris.backup.BackupJob.prepareAndSendSnapshotTask(BackupJob.java:398) ~[doris-fe.jar:1.0-SNAPSHOT]
        at org.apache.doris.backup.BackupJob.run(BackupJob.java:301) ~[doris-fe.jar:1.0-SNAPSHOT]
        at org.apache.doris.backup.BackupHandler.runAfterCatalogReady(BackupHandler.java:188) ~[doris-fe.jar:1.0-SNAPSHOT]
        at org.apache.doris.common.util.MasterDaemon.runOneCycle(MasterDaemon.java:58) ~[doris-fe.jar:1.0-SNAPSHOT]
        at org.apache.doris.common.util.Daemon.run(Daemon.java:116) ~[doris-fe.jar:1.0-SNAPSHOT]
2023-03-10 10:19:37 +08:00