f86886f8f5
[Feature](function) Support array_compact function ( #14141 )
2022-11-15 14:24:37 +08:00
93e5d8e660
[Vectorized](function) support bitmap_from_array function ( #14259 )
2022-11-15 01:55:51 +08:00
7eed5a292c
[feature-wip](multi-catalog) Support hive partition cache ( #14134 )
2022-11-14 14:12:40 +08:00
23a8c7eeb6
(fix)(multi-catalog)(es) Fix error result because not used fields_context ( #14229 )
...
Fix error result because not used fields_context
2022-11-14 14:00:55 +08:00
43490a33a5
[feature-array](array-type) Add array function array_with_constant ( #14115 )
...
Return array of constants with length num.
```
mysql> select array_with_constant(4, 1223);
+------------------------------+
| array_with_constant(4, 1223) |
+------------------------------+
| [1223, 1223, 1223, 1223] |
+------------------------------+
1 row in set (0.01 sec)
```
co-authored-by @eldenmoon
2022-11-11 22:08:43 +08:00
0ba13af8ff
[feature](running_difference) support running_difference function ( #13737 )
2022-11-11 21:22:56 +08:00
a162dab40a
[feature](docs) add docs for SHOW-CATALOG-RECYCLE-BIN ( #14185 )
2022-11-11 15:54:05 +08:00
b6ba654f5b
[Feature](Sequence) Support sequence_match and sequence_count functions ( #13785 )
2022-11-11 13:38:45 +08:00
7782fb63ca
[docs](outfile) Add ORC to outfile document ( #14153 )
2022-11-11 09:42:30 +08:00
6297ef10e9
[enhancement](plugin) import audit logs for slow queries into a separate table ( #14100 )
...
* import audit logs for slow queries into a separate table
2022-11-11 09:06:01 +08:00
b62e700f4e
[fix](doc): remove incubator. ( #14159 )
2022-11-11 08:58:42 +08:00
45a3bb87c4
[docs](recover) modify recover doc ( #13904 )
2022-11-10 20:20:39 +08:00
9b5b411112
[fix](schemeChange) fe oom because replicas too many when schema change ( #12850 )
2022-11-10 16:17:25 +08:00
0e26f28bf2
[Enhancement](runtime-filter) enlarge runtime filter in predicate threshold ( #13581 )
...
enlarge runtime filter in predicate threshold
2022-11-10 15:48:46 +08:00
90bfd87660
[feature](function) add new function uuid() ( #14092 )
2022-11-10 14:55:41 +08:00
df622d8b7d
[Bug](udf) fix java-udaf process string type error and add some tests ( #14106 )
2022-11-10 09:30:57 +08:00
55cae6202f
[typo](docs)add udf doc and optimize udf regression test ( #14000 )
2022-11-10 09:24:45 +08:00
b74d0a4747
[feature](table-valued-function) Support desc from s3() and modify the syntax of tvf ( #14047 )
...
This pr does two things:
Support desc function s3()
modify the syntax of tvf
2022-11-09 14:12:43 +08:00
7362460525
[docs](array-type) update the docs to specify how to use array function when import data ( #13995 )
...
Co-authored-by: hucheng01 <hucheng01@baidu.com >
2022-11-09 12:21:26 +08:00
287c3893b9
[typo](docs)update array type doc #14057
2022-11-09 08:40:38 +08:00
a0f136a0bc
[docs](odbc) fix docs for sqlserver odbc table ( #14017 )
...
Signed-off-by: nextdreamblue <zxw520blue1@163.com >
Signed-off-by: nextdreamblue <zxw520blue1@163.com >
2022-11-09 08:39:39 +08:00
b6f91b6eff
[improvement](profile) support ordinary user to get query profile via http api ( #14016 )
2022-11-08 20:39:01 +08:00
f7ecb6d79f
[Bug](Bitmap) fix sub_bitmap calculate wrong result to return null ( #13978 )
...
fix sub_bitmap calculate wrong result to return null
2022-11-08 14:10:12 +08:00
1c07a01038
[feature](multi-catalog) Support data on s3-compatible oss and support aliyun DLF ( #13994 )
...
Support Aliyun DLF
Support data on s3-compatible object storage, such as aliyun oss.
Refactor some interface of catalog, to make it more tidy.
Fix bug that the default text format field delimiter of hive should be \x01
Add a new class PooledHiveMetaStoreClient to wrap the IMetaStoreClient.
2022-11-08 14:02:41 +08:00
241801ca17
[typo](doc) fix get_start doc ( #14001 )
2022-11-07 21:28:45 +08:00
0031304015
[typo](docs)fix config doc #14010
2022-11-07 17:00:16 +08:00
7254999f02
[typo](docs) fix docs，delete redundant words #13849
2022-11-07 13:51:10 +08:00
e8d2fb6778
[feature](function)add search functions: multi_search_all_positions & multi_match_any ( #13763 )
...
Co-authored-by: yiliang qiu <yiliang.qiu@qq.com >
2022-11-07 11:50:55 +08:00
7ffe88b579
[feature-array](array-type) Add array function array_popback ( #13641 )
...
Remove the last element from array.
```
mysql> select array_popback(['test', NULL, 'value']);
+-----------------------------------------------------+
| array_popback(ARRAY('test', NULL, 'value')) |
+-----------------------------------------------------+
| [test, NULL] |
+-----------------------------------------------------+
```
2022-11-07 10:48:16 +08:00
380395a61f
[doc](routineload)Common mistakes in adding routine load #13975
2022-11-05 19:17:33 +08:00
087488db3b
[typo](doc) fixed spelling errors ( #13974 )
2022-11-05 15:40:55 +08:00
554f566217
[enhancement](compaction) introduce segment compaction ( #12609 ) ( #12866 )
...
## Design
### Trigger
Every time when a rowset writer produces more than N (e.g. 10) segments, we trigger segment compaction. Note that only one segment compaction job for a single rowset at a time to ensure no recursing/queuing nightmare.
### Target Selection
We collect segments during every trigger. We skip big segments whose row num > M (e.g. 10000) coz we get little benefits from compacting them comparing our effort. Hence, we only pick the 'Longest Consecutive Small" segment group to do actual compaction.
### Compaction Process
A new thread pool is introduced to help do the job. We submit the above-mentioned 'Longest Consecutive Small" segment group to the pool. Then the worker thread does the followings:
- build a MergeIterator from the target segments
- create a new segment writer
- for each block readed from MergeIterator, the Writer append it
### SegID handling
SegID must remain consecutive after segment compaction.
If a rowset has small segments named seg_0, seg_1, seg_2, seg_3 and a big segment seg_4:
- we create a segment named "seg_0-3" to save compacted data for seg_0, seg_1, seg_2 and seg_3
- delete seg_0, seg_1, seg_2 and seg_3
- rename seg_0-3 to seg_0
- rename seg_4 to seg_1
It is worth noticing that we should wait inflight segment compaction tasks to finish before building rowset meta and committing this txn.
2022-11-04 14:12:51 +08:00
1b36843664
[doc](jsonb type)add documents for JSONB datatype ( #13792 )
2022-11-03 19:33:51 +08:00
6ff306b1ea
[docs](round) complement round function documentation ( #13838 )
2022-11-03 14:30:49 +08:00
5fe3342aa3
[Vectorized](function) support bitmap_to_array function ( #13926 )
2022-11-03 14:29:28 +08:00
636bdffe62
[fix](doc) fix 404 link ( #13908 )
2022-11-03 08:46:47 +08:00
b83744d2f6
[feature](function)add regexp functions: regexp_replace_one, regexp_extract_all ( #13766 )
2022-11-02 23:15:57 +08:00
374303186c
[Vectorized](function) support topn_array function ( #13869 )
2022-11-02 19:49:23 +08:00
d5becdb4a1
[fix](dynamic-partition) fix wrong check of replication num ( #13755 )
2022-11-02 12:55:33 +08:00
bd6070d9b3
[doc](spark-doris-connetor)Add spark Doris connector to support streamload documentation #13834
2022-11-02 08:43:52 +08:00
3fc1b27c40
[docs](tablet-docs) fix the tablet-repair-and-balance.md doucument. ( #13853 )
...
Co-authored-by: wangxiangyu@360shuke.com <wangxiangyu@360shuke.com >
2022-11-02 08:43:08 +08:00
8b3afd431e
[improvement](memory) simplify memory config related to tcmalloc ( #13781 )
...
There are several configs related to tcmalloc, users do know how to config them. Actually users just want two modes, performance or compact, in performance mode, users want doris run query and load quickly while in compact mode, users want doris run with less memory usage.
If we want to config tcmalloc individually, we can use env variables which are supported by tcmalloc.
2022-11-01 21:45:19 +08:00
61c817f4cc
[feature](syntax) support SELECT * EXCEPT ( #13844 )
...
* [feature](syntax) support SELECT * EXCEPT: add regression test
2022-11-01 19:41:25 +08:00
942611c185
Revert "[enhancement](compaction) opt compaction task producer and quick compaction ( #13495 )" ( #13833 )
...
This reverts commit 4f2ea0776ca3fe5315ab5ef7e00eefabfb5771a0.
2022-11-01 14:22:12 +08:00
4f2ea0776c
[enhancement](compaction) opt compaction task producer and quick compaction ( #13495 )
...
1.remove quick_compaction's rowset pick policy, call cu compaction when trigger
quick compaction
2. skip tablet's compaction task when compaction score is too small
Co-authored-by: yixiutt <yixiu@selectdb.com >
2022-10-31 12:24:05 +08:00
2fab0c45c7
[Feature](runtime-filter) add runtime filter breaking change adapt ( #13246 )
...
add runtime filter breaking change adapt
2022-10-28 10:59:28 +08:00
5805011629
[Feature](string-function) Add function mask/mask_first_n/mask_last_n ( #13694 )
...
Implementation of mask function from hive.
2022-10-28 10:43:56 +08:00
c108554f14
[function](date function) add new date function 'to_monday' #13707
2022-10-28 08:41:16 +08:00
5dd052d386
[Function](array) support array_range function ( #13547 )
...
* array_range with 3 impl
* [Function](array) support array_range function
* update
* update code
2022-10-28 08:40:24 +08:00
43c6428aea
[Function](string) support sub_replace function ( #13736 )
...
* [Function](string) support sub_replace function
* remove conf
2022-10-28 08:40:08 +08:00
