cd105bee0a
[refactor](es) Clean es tcp scannode and related thrift definitions ( #9553 )
...
PaloExternalSourcesService is designed for es_scan_node using tcp protocol.
But es tcp protocol need deploy a tcp jar into es code. Both es version and lucene version are upgraded,
and the tcp jar is not maintained any more.
So that I remove all the related code and thrift definitions.
2022-05-14 10:03:55 +08:00
fa6e4db4ca
[fix](Function) fix case when function return null with abs function ( #9493 )
2022-05-14 09:50:45 +08:00
b817efd652
[feature] add vectorized vjson_scanner ( #9311 )
...
This pr is used to add the vectorized vjson_scanner, which can support vectorized json import in stream load flow.
2022-05-14 09:50:05 +08:00
4ca5be94a7
[Improvement] remove unnecessary memcpy in OlapBlockDataConvertor ( #9491 )
...
* [Improvement] remove unnecessary memcpy in OlapBlockDataConvertor
2022-05-13 16:13:07 +08:00
650e3a6ba0
[feature-wip](array-type) array_contains support more nested data types ( #9170 )
...
Co-authored-by: cambyzju <zhuxiaoli01@baidu.com >
2022-05-13 12:42:40 +08:00
e0ef04a5a7
[fix][vectorized-storage] did not check column writer's write status
2022-05-13 09:57:33 +08:00
955b7a3ba2
[bugfix](load) fix coredump in ordinal index flush ( #9518 )
...
commit #9123 introduce the bug. bitshuffle page return error when
page is full, so scalar column write cannot switch to next page, which make
ordinal index is null when flush.
All page builder should return ok when page full, and column writer procedure
shoud be append_data, check is_page_full, switch to next page
Co-authored-by: yixiutt <yixiu@selectdb.com >
2022-05-12 21:10:49 +08:00
a0b95d8fcb
[fix](storage) fix core for string predicate in storage layer ( #9500 )
...
Co-authored-by: Wang Bo <wangbo36@meituan.com >
2022-05-12 15:41:39 +08:00
4cd579b155
[refactor] Check status precise_code instead of construct OLAPInternalError ( #9514 )
...
* check status precise_code instead of construct OLAPInternalError
* move is_io_error to Status
2022-05-12 15:39:29 +08:00
d26f5d22be
[refactor]Cleanup unused empty files ( #9497 )
2022-05-12 14:58:28 +08:00
289608cc20
[fixbug]fix bug for OLAP_SUCCESS with Status ( #9427 )
2022-05-11 20:04:06 +08:00
e3bac86b43
[bugfix](vtablet_sink) fix max_pending_bytes for vtablet_sink ( #9462 )
...
Co-authored-by: yixiutt <yixiu@selectdb.com >
2022-05-11 18:00:56 +08:00
375c1bf5c0
[feature](mysql-table) support utf8mb4 for mysql external table ( #9402 )
...
This patch supports utf8mb4 for mysql external table.
if someone needs a mysql external table with utf8mb4 charset, but only support charset utf8 right now.
When create mysql external table, it can add an optional propertiy "charset" which can set character fom mysql connection,
default value is "utf8". You can set "utf8mb4" instead of "utf8" when you need.
2022-05-11 09:39:23 +08:00
718a51a388
[refactor][style] Use clang-format to sort includes ( #9483 )
2022-05-10 21:25:35 +08:00
ce926a7abb
[refactor] delete OLAP_LOG_WARNING related macro definition ( #9484 )
...
Co-authored-by: BePPPower <fangtiewei@selectdb.com >
2022-05-10 20:53:45 +08:00
b34ed43ec9
[feature-wip] (memory tracker) (step6, End) Fix some details ( #9301 )
...
1. Fix LoadTask, ChunkAllocator, TabletMeta, Brpc, the accuracy of memory track.
2. Modified some MemTracker names, deleted some unnecessary trackers, and improved readability.
3. More powerful MemTracker debugging capabilities.
4. Avoid creating TabletColumn temporary objects and improve BE startup time by 8%.
5. Fix some other details.
2022-05-10 18:17:09 +08:00
e61d296486
[Refactor] Replace '#ifndef' with '#pragma once' ( #9456 )
...
* Replace '#ifndef' with '#pragma once'
2022-05-10 09:25:59 +08:00
51db78d375
[refactor] modify all OLAP_LOG_WARNING to LOG(WARNING) ( #9473 )
...
Co-authored-by: BePPPower <fangtiewei@selectdb.com >
2022-05-10 09:25:25 +08:00
eec1dfde3a
[feature] (vec) instead of converting line to src tuple for stream load in vectorized. ( #9314 )
...
Co-authored-by: xiepengcheng01 <xiepengcheng01@xafj-palo-rpm64.xafj.baidu.com >
2022-05-09 11:24:07 +08:00
ae01862ae4
[fix](ut) fix DeltaWriter::close_wait parameter mismatch in delta_writer_test ( #9457 )
2022-05-09 09:38:12 +08:00
7e86c1beab
[fix] UT MathFunctionTest.round_test fix ( #9447 )
...
Function round support two format round(double) and round(double, int), the argument is variadic.
But FunctionBinaryArithmetic not support variadic argument now, make get_function for round(double, int) failed.
reproduce steps:
1. set enable_vectorized_engine=true;
2. try to call round(double, int);
```
> select round(10.12345,2);
ERROR 1105 (HY000): errCode = 2, detailMessage = Function round is not implemented
```
2022-05-09 09:37:27 +08:00
6834fb23ca
[fix](s3) fix s3 Temp file may write failed because of has no space on disk ( #9421 )
2022-05-09 09:28:43 +08:00
580ce38a3f
[fix](schema_hash) Fix bug that introduced by removing schema_hash ( #9449 )
2022-05-08 21:03:10 +08:00
7234c964ae
[Bug] Missing error tablet list when close_wait return error ( #9418 )
2022-05-08 06:45:28 +08:00
fd11a6b493
[fix][feature](Function) fix return type && support hll_union_agg/group_concat agg to window function ( #9119 )
2022-05-07 20:44:04 +08:00
4235db8902
[refactor] some code cleanup for min/max function. ( #8874 )
2022-05-07 20:39:44 +08:00
53574ce0ea
[Bug] (fix) DeltaWriter::mem_consumption() coredump ( #9245 )
2022-05-07 19:13:08 +08:00
49890ce9aa
[BUG][Vectorized] fix replace_if_not_null in vectorized compaction ( #9376 )
2022-05-07 17:16:54 +08:00
98bfeaf560
[Enhancement] [Vectorized] Refactor and optimize BinaryOperation ( #9087 )
2022-05-07 10:55:15 +08:00
2ccaa6338c
[enhancement](load) optimize load string data and dict page write ( #9123 )
...
* [enhancement](load) optimize load string data and dict page write
2022-05-07 10:27:27 +08:00
22439cb6a6
[Improvement] [compaction]Enable vectorized compaction by default ( #9383 )
2022-05-07 08:46:35 +08:00
dce18cb325
[doc] Add window functions sql help doc ( #9393 )
2022-05-07 08:43:51 +08:00
811f019e47
[performance][query]improve the performance of DISTINCT aggregation by using flat hash set replace unordered set ( #9401 )
...
Co-authored-by: shizhiqiang03 <shizhiqiang03@meituan.com >
2022-05-07 08:43:14 +08:00
e7f12db06c
[fixbug][compaction] update OLAP_ERR_CUMULATIVE_NO_SUITABLE_VERSION ( #9410 )
2022-05-07 08:39:20 +08:00
a9831f87f2
[refactor]refactor lazy materialized ( #8834 )
...
[refactor]refactor lazy materialized (#8834 )
2022-05-06 19:16:35 +08:00
edc833ab76
[Bug][stream-vec-load] Null data load do not skip the same place data ( #9360 )
...
Co-authored-by: lihaopeng <lihaopeng@baidu.com >
2022-05-06 16:07:45 +08:00
e130d2f233
[fix][compaction] Rowset::end_version null pointer( #9379 )
2022-05-06 14:40:08 +08:00
e3b90de2d5
remove file result writer from result sink ( #9378 )
2022-05-06 02:37:20 +08:00
e5d4cf01ed
[fix](ut) fix a potential memory leak in BE ut ( #9362 )
2022-05-05 20:47:31 +08:00
a33191e222
[fix](memtracker) DCHECK failed in vetorized exec engine fold constant execute ( #9354 )
2022-05-05 09:55:38 +08:00
832338c55e
[improvement] set name for scanner threads and fix compile error in clang ( #9336 )
2022-05-05 09:53:43 +08:00
f1aa9668af
[refactor][storage format] Forbidden rowset v1 ( #9248 )
...
- Force change the existing olaptable's storage format from V1 to V2
- Forbidden to create new olap table with storage format == v1 OR do schema change that want to create new v1 format
2022-05-04 17:32:20 +08:00
eed62695e1
[fix](function) handle merge in window_funnel_init and add test ( #9338 )
2022-05-03 22:37:06 +08:00
49a0cd1925
[fix](compaction) fix bug for vectorized compaction ( #9344 )
...
1. add a BE config to switch vectorized compaction
2. Fix vectorized compaction bug that row statistic is not right.
2022-05-03 17:31:40 +08:00
b1870faddd
[Bug] [Build] fix clang build fail ( #9323 )
...
* fix clang compile fail
2022-05-02 18:04:57 +08:00
d2374dbd5e
[fix](Lateral-View) fix outer combinator not work on non-vectorized ( #9212 )
2022-05-01 22:09:50 +08:00
c9961c9bb9
[style] clang-format all c++ code ( #9305 )
...
- sh build-support/clang-format.sh to clang-format all c++ code
2022-04-29 16:14:22 +08:00
201cd207f9
[Enhancement][Vectorized] Improve hash table build efficiency ( #9250 )
...
1. MAP_POPULATE is missing for mmap in Allocator, because macro OS_LINUX is not defined in allocator.h;
2. MAP_POPULATE has no effect for mremap as for mmap, zero-fill enlarged memory range explicitly to pre-fault the pages
2022-04-29 14:26:33 +08:00
ce7905e983
[fix](vectorized) Query get wrong result when ColumnDict concurrent predicate eval ( #9270 )
2022-04-29 11:45:04 +08:00
2fa19113ab
[fix](profile) Short-circuit and del predicate filter rows are not counted on vectorized exec ( #9268 )
2022-04-29 10:45:48 +08:00
