acf07cab6f
[refactor](minor) Init counter in prepare phase ( #39287 ) ( #39385 )
...
pick #39287
2024-08-15 13:36:12 +08:00
4042cdf553
[Fix](memory) Fix allocator.h compiling failed on mac. ( #38646 )
...
Backport #38562 . Fix allocator.h compiling failed on mac which
introduced by #37257 .
2024-08-01 13:56:53 +08:00
a751372e76
[Feature](multi-catalog) Add memory tracker for orc reader/writer and arrow parquet writer。 ( #37257 )
...
## Proposed changes
backport #37234
2024-07-25 13:51:59 +08:00
b15ccdbe98
[Pick](Variant) pick some fix ( #37922 )
...
#37674
#37839
#37883
#37857
#37794
2024-07-16 21:38:47 +08:00
e5219467dd
[Bug](join) avoid overflow on bucket_size+1 ( #37807 )
...
## Proposed changes
pick from #37493
2024-07-15 18:47:36 +08:00
ef031c5fb2
[branch-2.1](memory) Fix reserve memory compatible with memory GC and logging ( #37682 )
...
pick
#36307
#36412
2024-07-12 11:43:26 +08:00
217eac790b
[pick](Variant) pick some refactor and fix #34925 #36317 #36201 #36793 ( #37526 )
2024-07-11 21:25:34 +08:00
cf2fb6945a
[branch-2.1](memory) Refactor LRU cache policy memory tracking ( #37658 )
...
pick
#36235
#35965
2024-07-11 21:04:01 +08:00
fb344b66ca
[fix](hash join) fix numeric overflow when calculating hash table bucket size #37193 ( #37213 )
...
## Proposed changes
Bp #37193
2024-07-04 11:12:52 +08:00
fbcf63e1f5
[cherry-pick] (branch-2.1)fix variant index ( #36577 )
...
pick from master #36163
2024-06-20 17:57:26 +08:00
8f5deb10be
[be](oom) add stacktrace in debugmode to find oom reason
2024-05-26 23:39:46 +08:00
65b9e5ab69
[fix](chore) fix DCHECK failure of BufferWritable if failed to alloc memory ( #35345 )
2024-05-25 17:48:04 +08:00
f0b2f5ba36
[Fix](bug) agg limit contains null values may cause error result ( #35180 )
2024-05-22 10:57:57 +08:00
2ed6a00fd1
[opt](memory) Add GlobalMemoryArbitrator and support ReserveMemory ( #34985 ) ( #35070 )
2024-05-22 09:53:45 +08:00
26d5c50339
[Bug](Variant) fix incorrect use of column index in TabletSchema ( #35019 )
2024-05-21 12:52:26 +08:00
6656508579
[fix](agg) fix DCHECK failure of agg when failed to alloc memory ( #35011 )
...
* [fix](agg) fix DCHECK failure of agg when failed to alloc memory
* add comment
2024-05-20 10:12:16 +08:00
80dd027ce2
[opt](join) For left semi/anti join without mark join conjunct and without other conjucnts, stop probing after matching one row ( #34703 )
2024-05-18 18:08:50 +08:00
691f3c5ee7
[Performance](Variant) Improve load performance for variant type ( #33890 )
...
1. remove phmap for padding rows
2. add SimpleFieldVisitorToScarlarType for short circuit type deducing
3. correct type coercion for conflict types bettween integers
4. improve nullable column performance
5. remove shared_ptr dependancy for DataType use TypeIndex instead
6. Optimization by caching the order of fields (which is almost always the same)
and a quick check to match the next expected field, instead of searching the hash table.
benchmark:
In clickbench data, load performance:
12m36.799s ->7m10.934s about 43% latency reduce
In variant_p2/performance.groovy:
3min44s20 -> 1min15s80 about 66% latency reducy
2024-05-18 17:58:33 +08:00
0b4d814598
[fix](decimal) Fix wrong result produced by decimal128 multiply ( #34825 )
...
* [fix](decimal) Fix wrong result produced by decimal128 multiply
* update
2024-05-14 23:34:11 +08:00
ca9eb56233
[Fix](functions) fix strcmp return value #34565
2024-05-12 09:49:38 +08:00
804586b342
[Improvement](sort) insert data by batch on VSortedRunMerger::get_next ( #34363 )
...
insert data by batch on VSortedRunMerger::get_next
2024-05-10 14:36:53 +08:00
cbe8e5c010
[opt](join) For a mark join without other conjuncts, stop probing after matching one row ( #34581 )
2024-05-10 13:45:34 +08:00
8fdfbcb3c4
Revert "[Opt](func) opt the percentile func performance ( #34373 ) ( #34416 )"
...
This reverts commit 509ae425e416b4779ae94eab9c2b21f9850e03c3.
2024-05-07 07:23:48 +08:00
509ae425e4
[Opt](func) opt the percentile func performance ( #34373 ) ( #34416 )
2024-05-06 20:10:35 +08:00
25358564ca
[Fix](compile) Fix gcc compile on master ( #33864 )
...
This is imported by #33511 . wrongly used
ColumnStr<T> ();
which violate C++20 standard(see https://wg21.cmeerw.net/cwg/issue2237 ) but still supported by clang up until now(see llvm/llvm-project#58112 )
2024-04-19 23:41:37 +08:00
249a9c9875
[Feature](Variant) support aggregation model for Variant type ( #33493 )
...
refactor use `insert_from` to replace `replace_column_data` for variable lengths columns
2024-04-17 23:42:00 +08:00
50b64a111d
[refactor](heap sort) Simplify sorted block view ( #33477 )
2024-04-17 23:42:00 +08:00
3d66723214
[branch-2.1](auto-partition) pick auto partition and some more prs ( #33523 )
2024-04-11 17:12:17 +08:00
8fd6d4c41b
[Chore](build) add -Wconversion and remove some unused code ( #33127 )
...
add -Wconversion and remove some unused code
2024-04-10 15:26:08 +08:00
3b42dc73af
[improvement](spill) avoid spill if memory is enough ( #33075 )
2024-04-10 14:53:27 +08:00
e4993a19e5
[Chore](column) remove ColumnVectorHelper ( #33036 )
...
remove ColumnVectorHelper
2024-04-10 11:56:41 +08:00
cf7595d423
[opt](memory) Optimize mem tracker accuracy ( #32039 ) ( #33140 )
2024-04-10 11:42:19 +08:00
005f7af21f
[bugfix](deadlock) should not use query cancelled in fragment mgr
2024-04-09 16:09:01 +08:00
66336e59e6
[fix](join) the result of left semi join with empty right side should be false, not null ( #32477 )
2024-03-22 16:35:43 +08:00
23c12fd68f
[fix](join) core caused by null-safe-equal join ( #32623 )
2024-03-22 08:53:47 +08:00
921fab2196
[fix](memory) Fix thread context not initialized in MacOS ( #32570 )
2024-03-22 08:53:47 +08:00
617cc667fe
[Fix](Variant) fix variant serialize root node ( #31769 )
2024-03-21 14:07:50 +08:00
4268634115
[fix](memory) Fix Allocator cancel pipelinex query #32048
2024-03-12 14:20:18 +08:00
3358f76a7f
[feature](spill) Implement spill to disk for hash join, aggregation and sort for pipelineX ( #31910 )
...
Co-authored-by: Jerry Hu <mrhhsg@gmail.com >
2024-03-12 14:12:09 +08:00
e8aa5ee7d5
[Improve](Variant) support bloom filter for variant subcolumns ( #31347 )
...
* [Improve](Variant) support bloom filter for variant subcolumns
* rebase
2024-03-09 19:45:03 +08:00
0da010603e
[Improve](TabletSchemaCache) reduce duplicated memory consumption for column name and column path ( #31141 )
...
Both could be reference to related field in TabletColumn.And use shared_ptr for TabletColumn in TabletSchema for later memory reuse
2024-03-09 19:44:42 +08:00
7b79b77cc9
[Optimize](Variant) make tablet schema more well-organized ( #99 ) ( #30922 )
2024-02-18 11:50:17 +08:00
0442d5dc0e
[fix](Variant Type) Add sparse columns meta to fix compaction ( #28673 )
...
Co-authored-by: eldenmoon <15605149486@163.com >
2024-02-16 10:12:23 +08:00
2344aaf337
[fix](join) JoinHashTable::pre_build_idxs should be const ( #30837 )
2024-02-05 21:59:55 +08:00
4b42156fc0
[chore](clang-tidy): add bugprone linters ( #29521 )
...
This PR introduces 4 bugprone linter rules to .clang-tidy, these linters found some bugs in #28965 . This PR also add some comments to mute false positive reports.
2024-02-05 21:58:08 +08:00
8ff8d94697
[fix](ip) change IPv6 to little-endian byte order storage (like IPv4) ( #30730 )
2024-02-05 21:56:57 +08:00
c9ab243153
[feat-wip](join) support mark join for right semi join(without mark join conjunct) ( #30767 )
2024-02-04 14:28:38 +08:00
65e277e365
[refacotr](node) refactor partition sort node to improve readability ( #30511 )
...
* [refacotr](node) refactor partition sort node to improve readability
* update
2024-02-01 19:01:08 +08:00
1aa7a914e1
fix wrong profile on distinct agg and pass reference on uint136's compare ( #30661 )
2024-02-01 19:00:50 +08:00
77b366fc4b
[fix](join) incorrect result of mark join ( #30543 )
...
incorrect result of mark join
2024-01-31 23:53:40 +08:00
