4682b4564c
[enhance](Nereids): delete output in olapscan toString() ( #17288 )
2023-03-02 10:53:24 +08:00
a5ae3072e5
[fix](planner) ignore aux expr when do push agg op ( #17239 )
2023-03-02 10:44:40 +08:00
de5112bd90
[bugfix](merger) traverse rs_meta in lock ( #17271 )
...
tablet_schema(version) will traverse rowset_meta and it should call in meta_lock.
2023-03-02 09:47:44 +08:00
b7677beab7
[enhancement](memtracker) Add special counter for memtracker and fix thread create and destroy track #17301
...
Add a special counter for memtracker, faster, but relaxed ordering and not accurate in real time
Track thread create and destroy memory, which was previously removed due to performance loss and added back
2023-03-02 08:55:00 +08:00
d7ee542dd4
[refactor](function) refine function geo #17289
...
remove unused constant args
2023-03-02 08:42:16 +08:00
201cf9c8df
Revert "[enhancement](k8s) Support fqdn mode for fe in k8s enviroment ( #16315 )" ( #17278 )
...
This reverts commit 48afd77e37d63e2989cd85ab12b39a273fcd284e.
There is meta problem
2023-03-02 00:44:54 +08:00
527eb5b059
[Enchancement](function) nullable inline refactor of min_max_by/bitmap && add register_functio… ( #17228 )
...
1. nullable inline refactor of min_max_by/bitmap/group_concat/histogram/topn
2. add register_function_both method
3. add datetimev2 type creator of min_max_by
4. remove uint16/32/64 in FOR_INTEGER_TYPES
2023-03-02 00:00:01 +08:00
bb88f2ec7d
[fix](multi-catalog) fix not find dbname from internal catalog ( #17119 )
...
Signed-off-by: nextdreamblue <zxw520blue1@163.com >
fix not find dbname from internal catalog
2023-03-01 23:59:12 +08:00
5523859fd5
[typo](readme)Fix typo in README
2023-03-01 23:43:27 +08:00
b0c5250bf9
[Enhancement](tvf) support trim_double_quotes and skip_lines for S3 and HDFS table valued function ( #17224 )
...
support trim_double_quotes and skip_lines for S3 and HDFS table valued function
2023-03-01 23:41:31 +08:00
1244eed1cd
[Opt](exec) opt the dispose nullable column logic ( #17192 )
2023-03-01 23:25:40 +08:00
633f2d52a4
[minor](log) add some logs ( #17287 )
2023-03-01 22:41:50 +08:00
543539cf18
[Feature](multi catalog)(nereids)Support ES external table for new planner. ( #17290 )
...
Support ES external table query using Nereids planner.
2023-03-01 22:32:41 +08:00
6de02f1f46
[minor](jvm) add more error logs for JNI ( #17270 )
2023-03-01 22:09:57 +08:00
722755efe9
[fix](planner) change back legacy planner type coercion ( #17070 )
...
revert legacy planner change in #16844
2023-03-01 20:55:56 +08:00
34c5e84e9f
[fix](insert) fix txn error reason clearly ( #16997 )
...
Signed-off-by: nextdreamblue <zxw520blue1@163.com >
2023-03-01 20:28:41 +08:00
f1db0d9501
[Enhencement](File Reader) delete old file_reader ( #17261 )
...
* delete old file_reader
* fix 1
2023-03-01 20:24:03 +08:00
6b70faa638
[fix](planner) should call Expr's unwrapSlotRef instead of getSrcSlotRef o prevent null pointer ( #17265 )
2023-03-01 20:07:36 +08:00
b839353c2d
[fix](inverted index) fix BE coredump because of not ignore case ensitivity for column name when create index ( #17276 )
2023-03-01 19:32:39 +08:00
3871e989ac
[fix](memory) Avoid repeating meaningless memory gc #17258
2023-03-01 19:23:33 +08:00
a1e3b908d7
[fix](memory) split mem usage thread and gc thread to different threads ( #17213 )
...
Ensure that the memory status is refreshed in time
Avoid frequent GC
2023-03-01 19:19:05 +08:00
d44c4b1300
[improvement][fix](catalog) check required properties when creating catalog and fix jdbc catalog issue ( #17209 )
...
Check required properties when creating catalog.
To avoid some strange error when missing required properties
This PR add checks for:
hms catalog: check the validation of dfs.ha properties
jdbc catalog: check jdbc_url, driver_url, driver_class is set.
Fix NPE when init MasterCatalogExecutor
The MasterCatalogExecutor may be called by FrontendServiceImpl from BE, which does not have ConnectionContext.
Add more jdbc url param to resolve Chinese issue
add useUnicode=true&characterEncoding=utf-8 by default in jdbc catalog when connecting to MySQL
Update FAQ doc of catalog
2023-03-01 17:08:36 +08:00
a5bd71c03a
(udf) create native function return error ( #17021 )
...
after 1.2.0, doris does not support native udf, return error when create native function
2023-03-01 16:56:54 +08:00
e203678e18
[fix](Nereids): fix master problem ( #17275 )
2023-03-01 16:34:23 +08:00
b8d8cf1ac9
[regression](test) script for teamcity to check if pr need run build ( #16937 )
...
* [regression](test) script for teamcity to check if pr need run build
* Update check-pr-if-need-run-build.sh
fix
* Update check-pr-if-need-run-build.sh
fix
---------
Co-authored-by: stephen <hello_stephen@@qq.com>
2023-03-01 15:59:31 +08:00
48ef61780d
[refactor](struct-type) refactor and clean unused code for struct type ( #17257 )
...
remove unused code for struct type
2023-03-01 15:49:31 +08:00
0732eb54bc
[feature](struct-type) support csv format stream load for struct type ( #17143 )
...
Refactor from_string method in data_type_struct.cpp to support csv format stream load for struct type.
2023-03-01 15:48:48 +08:00
62440f3140
[Bug](Materialized-View) forbiden mv rewrite on create view and remove duplicate method getIsM… ( #17194 )
...
1. forbiden mv rewrite on create view to avoid select fail
2. remove duplicate method getIsMaterialized
2023-03-01 13:46:56 +08:00
ff8902370c
[improvement](doc) Supplementary Bulk Deletion Notes ( #17113 )
...
* 补充批量删除注意事项
* 按照批量删除文档前文的介绍, 用户可能会开启`show_hidden_columns`的session variable来查看表是否支持批量删除.
* 后续按示例进行DELETE/MERGE的导入作业后, 如果在同一个session中执行`select count(*) from xxx`语句时, 可能会发现结果与预期不一致
* 可能无法快速联想到是因为之前开启的session variable导致被删除的语句也被查出来了.
* supplement batch deletion notes for English doc
2023-03-01 13:35:20 +08:00
b8ebcdff78
[Bug](bloomfilter) Fix wrong result using bloomfilter with date type ( #17225 )
2023-03-01 12:29:20 +08:00
979cf42d7a
[Bug](decimalv3) Use correct decimal scale for function round ( #17232 )
...
Co-authored-by: maochongxin <maochongxin@gmail.com >
2023-03-01 12:28:41 +08:00
cbdf1af2d5
[feature](Nereids): pushdown Alias through Join. ( #17150 )
2023-03-01 11:33:37 +08:00
62ec74f4e7
segcompaction featuring verticalcompaction ( #16731 )
...
This patchset applies the following changes:
using vertical compaction machanism to do segcompaction
basic (WIP) refraction to separate segcompaction logic from BetaRowsetWriter
add segcompaction specific ut and regression tests
2023-03-01 10:55:40 +08:00
48afd77e37
[enhancement](k8s) Support fqdn mode for fe in k8s enviroment ( #16315 )
2023-03-01 10:54:39 +08:00
774f66c6bc
[Enhencement](test) enhencement regression test of java udf ( #17251 )
...
when run regression test of java udf many times.
if failed in some reason, the next time will meet error like: function already exist
Issue Number: close #xxx
2023-03-01 09:34:40 +08:00
e687f3badd
Revert "[feature-wip](BE http)Support BE http service using brpc ( #16123 )" ( #17219 )
...
This reverts commit 049ecccc578802496e5421db19e21e7eb256699d.
Merge back after streamload is handled.
2023-03-01 09:18:25 +08:00
2f471de675
[fix](FileCache) load file cache before start up daemon threads ( #17199 )
...
Daemon threads in doris_main.cpp will upload tablet metrics periodically, which will use StorageEngine::instance(). However loading file cache is a process in main thread, when it takes a lot of time to load file cache, StorageEngine::instance() will be a null pointer in daemon threads.
2023-03-01 08:35:57 +08:00
e22a9ecc3b
[enhancement](execute model) using thread pool to execute report or join task instead of staring too many thread ( #17212 )
...
* [enhancement](execute model) using thread pool to execute report or join task instead of staring too many thread
Doris will start report thread and join thread during fragment execution. There are many problems if create and destroy thread very frequently. Jemalloc may not behave very well, it may crashed.
jemalloc/jemalloc#1405
It is better to using thread pool to do these tasks.
---------
Co-authored-by: yiguolei <yiguolei@gmail.com >
2023-03-01 08:35:27 +08:00
68e9a66aa0
[Enchancement](schema scanner) add SchemaScanner profile ( #17230 )
...
Add some profile information to the schema scanner to facilitate performance optimization.
Example:
SchemaScanner:
- FillBlockTime: 9s131ms
- GetDbTime: 12.816ms
- GetDescribeTime: 1s645ms
- GetTableTime: 25.433ms
2023-03-01 08:34:27 +08:00
cfc2d45795
[typo](docs) fix typo ( #17208 )
2023-03-01 07:41:21 +08:00
eeca16d7a0
[fix](doc)adjust Flink connector document structure and add SchemaChange example ( #17231 )
2023-03-01 07:40:56 +08:00
475368c62d
[typo](docs) Add some details about AES encryption. ( #17243 )
...
* [typo](docs) Add some details about AES encryption.
* Update aes.md
* Update aes.md
* Update aes.md
* Update aes.md
2023-03-01 07:40:11 +08:00
7369261f33
[typo](docs)update hight-concurrent-point-query.md ( #17248 )
...
Co-authored-by: liuxiaodong <liuxiaodong1@corp.netease.com >
2023-03-01 07:37:27 +08:00
5d096f9fcb
[community] update collaborators ( #17263 )
2023-03-01 07:30:06 +08:00
91bf497a88
[fix](Nereids): provide BUCKETED property only when child's property is enforced for agg ( #17229 )
2023-03-01 01:11:42 +08:00
cf7e97dd27
[chore](thirdparty) Fix the linkage errors for librdkafka ( #17181 )
...
Fix the linkage errors for librdkafka
2023-02-28 21:37:27 +08:00
7f6209ede4
[fix](routine load) fix be core dump while use routine load ( #17222 )
2023-02-28 21:01:38 +08:00
e3d7f7c8d8
[feature](Nereids) add test framework for cost model ( #17071 )
...
add test-frame-work for cost model according paper Testing the Accuracy of Query Optimizers
2023-02-28 20:59:07 +08:00
1b58f7f2ea
[fix](Nereids) json object and json array should always not nullable ( #17205 )
2023-02-28 20:26:21 +08:00
9bcc3ae283
[Fix](DOE)Fix be core dump when parse es epoch_millis date format ( #17100 )
2023-02-28 20:09:35 +08:00
