7e91e69eb9
[fix](compaction) fix single compaction ( #33907 )
...
* [fix](compaction)Fix single compaction to get all local versions #33849
add test and comment
* remove single replica compaction prepare input rowsets
reviesd
2024-04-19 23:30:25 +08:00
a4924dabb7
[enhancement](exception) enble exception logic in pipeline execute thread ( #33437 )
...
* [enhancement](exception) enble exception logic in pipeline execute thread
* f
---------
Co-authored-by: yiguolei <yiguolei@gmail.com >
2024-04-12 15:09:25 +08:00
3d66723214
[branch-2.1](auto-partition) pick auto partition and some more prs ( #33523 )
2024-04-11 17:12:17 +08:00
8fd6d4c41b
[Chore](build) add -Wconversion and remove some unused code ( #33127 )
...
add -Wconversion and remove some unused code
2024-04-10 15:26:08 +08:00
cf7595d423
[opt](memory) Optimize mem tracker accuracy ( #32039 ) ( #33140 )
2024-04-10 11:42:19 +08:00
39fba884fb
[fix](typo) typo fix for 'delete bimap' changing to 'delete bitmap' ( #32341 )
2024-04-10 11:34:30 +08:00
28e2d89ce3
[Improve](inverted_index) update clucene and improve array inverted index writer ( #32436 )
2024-04-10 11:34:29 +08:00
7b74b199a5
[fix](memory) Fix LRU cache deleter and memory tracking ( #32080 )
...
In order to add common code to the value deleter of LRU cache, let all lru cache values inherit from LRUCacheValueBase class and tracking memory in destructor.
2024-03-15 17:57:58 +08:00
0da010603e
[Improve](TabletSchemaCache) reduce duplicated memory consumption for column name and column path ( #31141 )
...
Both could be reference to related field in TabletColumn.And use shared_ptr for TabletColumn in TabletSchema for later memory reuse
2024-03-09 19:44:42 +08:00
eea9b56f69
[fix](group commit) handle group commit create plan error ( #31757 )
2024-03-06 13:07:59 +08:00
7d1db6cd1f
[refactor](exception safe) Refactor delete handler and block column predicates to make sure exception safe ( #31618 )
2024-03-01 14:21:17 +08:00
90ab5ec2d9
[fix](invert index) fix the error issue in the unit test remove_element_only_in_table ( #31238 )
2024-02-22 13:01:49 +08:00
1abe9d4384
[fix](memory) Fix LRU cache stale sweep ( #31122 )
...
Remove LRUCacheValueBase, put last_visit_time into LRUHandle, and automatically update timestamp to last_visit_time during cache insert and lookup.
Do not rely on external modification of last_visit_time, which is often forgotten.
2024-02-21 17:01:29 +08:00
7a1bd6abb0
[improvment](group_commit) Refector scan wal function ( #30939 )
...
Co-authored-by: Yongqiang YANG <98214048+dataroaring@users.noreply.github.com >
2024-02-20 09:12:38 +08:00
b5012dc55a
[Enhancement](group commit) optimize pre allocated calculation ( #30893 )
2024-02-18 11:50:17 +08:00
0f47f7f389
[Feature](runtime filter) normalize ignore runtime filter ( #30152 )
...
normalize ignore runtime filter
2024-02-03 20:24:39 +08:00
e9c112b843
[Refact](inverted index) refact inverted index cache to decouple with reader ( #30574 )
2024-02-01 19:00:50 +08:00
ccde65b942
[fix](Cooldown) Enhance calculate logic of _has_data_to_cooldown ( #30244 ) ( #30299 )
2024-01-25 13:25:34 +08:00
1a51d04cb8
[fix](move-memtable) fix schema use-after-free in delta writer v2 ( #30254 )
2024-01-24 10:00:25 +08:00
d525f576e1
[improve] Use lru cache to count the number of column in tablet schema to control memory ( #29668 )
2024-01-12 13:58:19 +08:00
81680383e6
[UT](wal) Add wal dirs info be ut ( #29759 )
2024-01-12 11:57:16 +08:00
0d16ec7345
[improvement](cooldown) do not cooldown tablet without cold data ( #29690 )
2024-01-12 11:57:16 +08:00
7c7dbf15bc
[feature](merge-cloud) Decouple Tablet/TabletManager/TxnManager from global StorageEngine instance ( #29736 )
2024-01-12 11:57:16 +08:00
b0cac0014d
[enhance](FS) Improve FS error code ( #29432 )
2024-01-06 21:17:22 +08:00
85dd606fd1
[fix](group_commit) Fix group_commit ut ( #29587 )
2024-01-06 18:11:13 +08:00
a0c3ddf902
[fix](memory) Fix LRUCacheType::NUMBER charge ( #29588 )
...
if LRUCacheType::NUMBER, charge not add handle_size, because charge at this time is no longer the memory size, but an independent weight.
2024-01-06 10:37:56 +08:00
f40cce1406
[Fix](partition) Skip rowset partition id eq 0 smaller than config wh… ( #29510 )
2024-01-05 19:39:51 +08:00
706463781c
[refactor](group commit) refactor group commit wal code ( #29375 )
2024-01-02 15:52:03 +08:00
03901b9a7a
[enhancement](group_commit): refector relay wal code ( #29183 )
2023-12-30 12:59:46 +08:00
82635d4b59
[opt](memory) All LRU Cache inherit from LRUCachePolicy ( #28940 )
...
After all LRU Cache inherits from LRUCachePolicy, this will allow prune stale entry, eviction when memory exceeds limit, and define common properties. LRUCache constructor change to private, only allow LRUCachePolicy to construct it.
Impl DummyLRUCache, when LRU Cache capacity is 0, will no longer be meaningless insert and evict.
2023-12-29 16:15:56 +08:00
a525d5c5a3
[refactor](decimal) change type name Decimal128 to Decimal128V2, Decimal128I to Decimal128V3 to avoid confusion ( #29265 )
...
change type name Decimal128 to Decimal128V2, Decimal128I to Decimal128V3 to avoid confusion
2023-12-29 10:11:44 +08:00
fd90c3a6a6
[optimize](cooldown)Reduce the number of calls to the pick_cooldown_rowset ( #27091 )
...
Co-authored-by: xingying01 <xingying01@corp.netease.com >
2023-12-28 13:03:33 +08:00
9ff8bd2e9c
[Enhancement](Wal)Support dynamic wal space limit ( #27726 )
2023-12-27 11:51:32 +08:00
0af6bd6390
[fix](group-commit) check if wal need recovery is abnormal ( #28769 )
2023-12-22 11:06:11 +08:00
aab859be56
[enhance](partition_id) check partition id before store meta ( #28055 )
2023-12-19 21:31:41 +08:00
e6e8632167
[improvement](merge-on-write) Optimize publish when there are missing versions ( #28012 )
...
1. Do not retry publishing on be When there are too many missing versions, just
add to async publish task.
2. To reduce memory consumption, clean up the tasks when there are too many
async publish tasks.
2023-12-13 16:59:25 +08:00
a719d7a222
[fix](memory) Fix LRU Cache of type NUMBER charge ( #28175 )
2023-12-13 11:15:57 +08:00
6da36e1077
[feature](merge-cloud) Refactor write path code by abstract base class ( #26537 )
...
Refactor write path code by abstract base class. Whether to use `StorageEngine` or `CloudStorageEngine` will be determined during compilation instead of runtime `config::cloud_mode` to avoid unexpected null pointer or undefined behavior issues caused by merging code.
Class that depend on `StorageEngine` but are shared by the cloud mode need to have an abstract base class. Common code should be extracted into the base class, while the code that depends on `StorageEngine` should be implemented in a `StorageEngine` mix-in class of the base class.
2023-12-08 14:50:36 +08:00
1ed99c4d8a
[Improvement](inverted index) improve inverted index bkd performance in high concurrent scenario ( #27820 )
...
Improve BKD performance by enable bkd reader cache and improvement of fast compare and visit in compressed values in BKD tree.
2023-12-05 11:39:53 +08:00
13b26ee920
[Fix](core) Fix wal space back pressure core and add regression test ( #27311 )
2023-11-27 15:10:26 +08:00
553e4a8903
[feature-wip](merge-on-write) MOW table support different primary keys and sort keys ( #24788 )
2023-11-24 16:37:30 +08:00
d767804815
[feature](merge-cloud) Decouple rowset id generator and local rowsets gc implementation ( #25921 )
2023-11-10 10:07:02 +08:00
5f62a4462d
[Enhancement](wal) Add wal space back pressure ( #26483 )
2023-11-09 12:29:05 +08:00
8bd06aff7e
[Chore](MoW) remove unused code about rowset tree ( #26282 )
2023-11-02 20:25:27 +08:00
a5ef90dacc
[enhancement](recover) support skipping missing version in select by session variable ( #25654 )
2023-11-02 20:01:51 +08:00
cedab51676
[enhancement](UT) add unit test cases about bitmap ( #25867 )
...
* [fix](bitmap) incorrect result of operator ==
* [enhancement](UT) add unit test cases about bitmap
2023-10-27 11:27:14 +08:00
9519d7ede9
[enhancement](be-ut)Add more indexed column reader be unit test ( #25652 )
...
Added more unit tests
1. key exists or does not exist in a single page
2. key exists or does not exist in multiple pages
3. key is between two pages.
2023-10-23 10:12:53 +08:00
642c149e6a
remove datetime_value and move vecdatetime_value to doris namespace ( #25695 )
...
remove datetime_value and move vecdatetime_value to doris namespace
2023-10-20 22:08:17 +08:00
9b64286f2d
[enhancement](mow-ut)Add compaction commit delete bitmap unit test ( #25569 )
2023-10-20 19:54:12 +08:00
9c9fc84f39
[feature](merge-cloud) Abstract BaseTablet for CloudTablet ( #24929 )
2023-10-18 20:29:04 +08:00
