Commit Graph

2078 Commits

Author SHA1 Message Date
281ceee3cc [feature-wip](resource-group) Support resource group tvf (#18519)
related: #18098
2023-04-13 20:11:20 +08:00
33eec9096f [Enhancement](FE) use customized grpc threadpool to get better metric for grpc from FE to BE (#13983)
Previously in Doris FE, there is no specific thread pool for grpc-client-channel,
by default the underlying netty logic would use one dynamic unbounded cache threadpool.
The workload for this grpc threadpool is unseen.
Use ThreadpoolMgr to create one customized threadpool to get Prometheus-compatible metric data.
2023-04-13 20:09:26 +08:00
2519931a04 [vectorized](function) support time_to_sec function (#18354)
support time_to_sec function
2023-04-13 19:31:12 +08:00
05badac053 [Improve](docs)new libraries check (#18634) 2023-04-13 17:57:38 +08:00
2f64a8b387 [feature](GEO)Support read/write WKB/EWKB to gis types (#18526)
Support mutual conversion from wkb and gis types.also compatible with EWKB format
https://cwiki.apache.org/confluence/display/DORIS/DSIP-033%3A+More+GEO+functions
2023-04-13 16:25:18 +08:00
c4e9808382 [feature](multi-catalog) support trino jdbc catalog and jdbc external table (#18497) 2023-04-13 16:00:09 +08:00
f9c6d7791c [typo](docs)Optimize SeaTunnel documentation (#18584) 2023-04-12 21:33:31 +08:00
43392918cd [Optimization](functions)Optimize function call for const columns. (#18310) 2023-04-12 11:11:01 +08:00
2209b714d1 [chore](orc) Update orc lib to third party lib(1.8.3) using git submodule. (#18531) 2023-04-12 10:37:50 +08:00
cbe2e138c3 [Enhancement](HttpServer) Support https in be (#17034)
* [Enhancement](HttpServer) Support https in be
2023-04-12 10:27:07 +08:00
bdd853b951 [typo](docs) enhance docs SHOW-TABLES.md (#18572) 2023-04-12 10:17:01 +08:00
80754edae3 [typo](docs) fix docs multi-catalog.md (#18573) 2023-04-12 10:13:18 +08:00
0290f0f770 [doc](https) Add https certificate docs (#18558)
* add https certificate docs

* add version

* add version

* add version
2023-04-12 09:40:51 +08:00
c45f8c4371 [typo](doc)update hight-concurrent-point-query.md (#18570) 2023-04-12 00:21:50 +08:00
ef462fc74b modify introduction to String Type (#18562) 2023-04-11 23:40:02 +08:00
1238f6de97 [bug](array) fix be core in array_with_constant/array_repeat function when the first argument is nullable (#18404)
fix be core in array_with_constant/array_repeat function when the first argument is nullable
2023-04-11 19:46:41 +08:00
a832d24206 [doc](typo) remove the LTS version explanation about 1.2.x (#18459) 2023-04-11 12:01:40 +08:00
317c9491c2 modify datev2 (#18512) 2023-04-11 08:54:49 +08:00
6413ede8c5 [docs](clion) Add Clion remote development and debugging documents. (#18490)
* [docs] Add clion remote development and debugging documents.

* Add blanks.
2023-04-10 14:01:18 +08:00
60c0bbe272 [fix](profile) fix show load query profile (#18487)
Sometimes, `show load profile` will only show part of the insert opertion's profile.
This is because we assume that for all load operation(including insert), there is only one fragment in the plan.
But actually, there will be more than 1 fragment in plan. eg:

`insert into tbl1 select * from tbl1 limit 1` will have 2 fragments.

This PR mainly changes:

1. modify the `show load profile`
   Before:  `show load profile "/queryid/taskid/instanceid";`
   After: `show load profile "/queryid/taskid/fragmentid/instanceid";`

2. Modify the display of `ReadColumns` in OlapScanNode
    Because for wide table, the line of `ReadColumns` may be too long for show in profile.
    So I wrap it and each line contains at most 10 columns names.

3. Fix tvf not working with pipeline engine, follow up #18376
2023-04-09 08:41:18 +08:00
0517616242 [vectorized](function) support array_repeat function to be compatible with hive syntax (#18028)
---------

Co-authored-by: zhangyu209 <zhangyu209@meituan.com>
2023-04-08 15:50:28 +08:00
936807b5c9 [typo](doc) Fixed typos in SHOW-PROCESSLIST.md (#18469) 2023-04-07 22:02:37 +08:00
2bb025a474 [typo](docs) fix some error in this files (#18259)
* [typo](docs) fix some error in this files

* [typo](docs) fix some error in this files

* update files
2023-04-07 18:01:07 +08:00
5d876414b5 [typo](docs) add autobucket to sidebars (#18453) 2023-04-07 14:17:36 +08:00
759f1da32e [Enhencement](Backends) add HostName filed in backends table and delete backends table in information_schema (#18156)
1.  Add `HostName` field for `show backends` statement and `backends()` tvf.
2. delete the `backends` table in `information_schema` database
2023-04-07 08:30:42 +08:00
a059973cff [typo](doc)Add description that external table are no longer maintained (#18425)
Co-authored-by: hechao <hechao@selectdb.com>
2023-04-06 21:01:26 +08:00
e848e456be [config] modify tablet_shard to 4 and add some log (#18416)
modify the default value of BE config tablet_map_shard_size to 4. To reduce lock contention.
Add log when failed writing disk test file, for debug
2023-04-06 17:18:16 +08:00
d0219180a9 [feature-wip](multi-catalog)add properties converter (#18005)
Refactor properties of each cloud , use property converter to convert properties accessing fe
metadata and be data.
user docs #18287
2023-04-06 09:55:30 +08:00
d305c459a1 [doc](datetimefunction)Supplement the description and case of days_diff (#18244) 2023-04-06 09:04:08 +08:00
d12c4c6361 Small typos in docker run commands. (#18288) 2023-04-05 22:27:53 +08:00
cbbad5d95c [typo](doc)Update SHOW-PROC.md and SHOW-CATALOGS.md (#18398) 2023-04-05 22:24:35 +08:00
668031986b Update install-faq.md (#18385) 2023-04-05 08:38:06 +08:00
4edf2acc81 [typo](doc)Fixing broken links in docker cluster docs, improving formatting. (#18290) 2023-04-05 08:36:47 +08:00
7f8d92656e [fix](streamload) fix stream load failed when enable profile (#18364)
#18015 enables stream load profile log,  however be will encounter rpc fail when loading tpch data(see #18291). This is because when `is_report_success` is true, be will reportExecStatus to fe, but fe cannot find QueryInfo in `coordinatorMap`, thus it will return error to be.
2023-04-05 01:01:46 +08:00
7c36bef6bc [Feature-Wip](MySQL Load)Show load warning for my sql load (#18224)
1. Support the show load warnings for mysql load to get the detail error message.
2. Fix fillByteBufferAsync not mark the load as finished in same data load
3. Fix drain data only in client mode.
2023-04-04 22:44:48 +08:00
d7623028e9 [doc](developer-guide) add some debug tricks to dev-guide (#18225)
add method to debug core-dump file in vscode. and some BE debug tricks.
2023-04-04 17:10:34 +08:00
50e6c4216a [vectorized](function) suppoort date_trunc function truncate week mode (#18334)
support date_trunc could truncate week eg:
select date_trunc('2023-4-3 19:28:30', 'week');
2023-04-04 10:24:26 +08:00
8b85c55117 [vectorized](function) Support array_shuffle and shuffle function. (#18116)
---------

Co-authored-by: zhangyu209 <zhangyu209@meituan.com>
2023-04-04 08:53:13 +08:00
eb6dbc03e0 [typo](docs) add regression test doc & fix api doc (#18329) 2023-04-03 17:40:41 +08:00
aff260c06f [Enhancement](HttpServer) Support https interface (#16834)
1. Organize http documents
2. Add http interface authentication for FE
3. **Support https interface for FE**
4. Provide authentication interface
5. Add http interface authentication for BE
6. Support https interface for BE
2023-04-03 14:18:17 +08:00
ecd3fd07f6 [feature](colocate) support cross database colocate join (#18152) 2023-04-03 14:03:42 +08:00
961f5d1bb7 [feature](function)Add St_Angle/St_Azimuth function (#18293)
Add St_Angle/St_azimuth function:
St_Angle:
Enter three point, which represent two intersecting lines. Returns the angle between these lines. Point 2 and point 1 represent the first line and point 2 and point 3 represent the second line. The angle between these lines is in radians, in the range [0, 2pi). The angle is measured clockwise from the first line to the second line.

`

mysql> SELECT ST_Angle(ST_Point(1, 0),ST_Point(0, 0),ST_Point(0, 1));
+----------------------------------------------------------------------+
| st_angle(st_point(1.0, 0.0), st_point(0.0, 0.0), st_point(0.0, 1.0)) |
+----------------------------------------------------------------------+
| 4.71238898038469 |
+----------------------------------------------------------------------+
1 row in set (0.04 sec)
`

St_azimuth:
Enter two point, and returns the azimuth of the line segment formed by points 1 and 2. The azimuth is the angle in radians measured between the line from point 1 facing true North to the line segment from point 1 to point 2.
`

mysql> SELECT st_azimuth(ST_Point(0, 0),ST_Point(1, 0));
+----------------------------------------------------+
| st_azimuth(st_point(0.0, 0.0), st_point(1.0, 0.0)) |
+----------------------------------------------------+
| 1.5707963267948966 |
+----------------------------------------------------+
1 row in set (0.04 sec)
2023-04-03 13:01:59 +08:00
94e3472050 [bug](function) fix count equal function return incorrect value (#18200)
fix count equal function return incorrect value
2023-04-03 11:20:36 +08:00
03fc41ea51 [doc](catalog) add faq for hive catalog (#18298) 2023-04-03 09:01:49 +08:00
04929ff6d4 [fix](doc) suggest use window function to replace running_difference (#18281) 2023-04-02 16:35:10 +08:00
20b3bdb000 [vectorized](function) support array_first_index function (#18175)
mysql> select array_first_index(x->x+1>3, [2, 3, 4]);
+-------------------------------------------------------------------+
| array_first_index(array_map([x] -> x(0) + 1 > 3, ARRAY(2, 3, 4))) |
+-------------------------------------------------------------------+
|                                                                 2 |
+-------------------------------------------------------------------+

mysql> select array_first_index(x -> x is null, [null, 1, 2]);
+----------------------------------------------------------------------+
| array_first_index(array_map([x] -> x(0) IS NULL, ARRAY(NULL, 1, 2))) |
+----------------------------------------------------------------------+
|                                                                    1 |
+----------------------------------------------------------------------+

mysql> select array_first_index(x->power(x,2)>10, [1, 2, 3, 4]);
+---------------------------------------------------------------------------------+
| array_first_index(array_map([x] -> power(x(0), 2.0) > 10.0, ARRAY(1, 2, 3, 4))) |
+---------------------------------------------------------------------------------+
|                                                                               4 |
+---------------------------------------------------------------------------------+
2023-03-31 12:51:29 +08:00
525f15dddf [vectorized](function) support array_sortby function (#18071) 2023-03-30 11:07:49 +08:00
01d012bab7 [fix](memory) Remove page cache regular clear, disabled jemalloc prof by default (#18218)
Remove page cache regular clear
Now the page cache is turned off by default. If the user manually opens the page cache, it can be considered that the user can accept the memory usage of the page cache, and then can consider adding a manual clear command to the cache.

fix memory gc cancel top memory query

jemalloc prof is not enabled by default
2023-03-30 09:39:37 +08:00
7e575d4286 [typo](docs) fix the error of markdown syntax (#18219) 2023-03-30 07:22:14 +08:00
6964d9f99c [fix](function) resubmit-fix AES/SM3/SM4 encrypt/ decrypt algorithm initialization vector bug (#17907)
* Revert "[fix](function) fix AES/SM3/SM4 encrypt/ decrypt algorithm initialization vector bug (#17420)"

This reverts commit 397cc011c4f1ba5a25c770258c13f1cd3f28b47d.

* [fix-resubmit](function) fix AES/SM3/SM4 encrypt/ decrypt algorithm initialization vector bug (#17420)

ECB algorithm, block_encryption_mode does not take effect, it only takes effect when init vector is provided.
Solved: 192/256 supports calculation without init vector

For other algorithms, an error should be reported when there is no init vector

Initialization Vector. The default value for the block_encryption_mode system variable is aes-128-ecb, or ECB mode, which does not require an initialization vector. The alternative permitted block encryption modes CBC, CFB1, CFB8, CFB128, and OFB all require an initialization vector.

Reference: https://dev.mysql.com/doc/refman/8.0/en/encryption-functions.html#function_aes-decrypt

Note: This fix does not support smooth upgrades. during upgrade process, query may report error: funciton not found
2023-03-29 21:13:01 +08:00