90ab5ec2d9
[fix](invert index) fix the error issue in the unit test remove_element_only_in_table ( #31238 )
2024-02-22 13:01:49 +08:00
1abe9d4384
[fix](memory) Fix LRU cache stale sweep ( #31122 )
...
Remove LRUCacheValueBase, put last_visit_time into LRUHandle, and automatically update timestamp to last_visit_time during cache insert and lookup.
Do not rely on external modification of last_visit_time, which is often forgotten.
2024-02-21 17:01:29 +08:00
7a1bd6abb0
[improvment](group_commit) Refector scan wal function ( #30939 )
...
Co-authored-by: Yongqiang YANG <98214048+dataroaring@users.noreply.github.com >
2024-02-20 09:12:38 +08:00
b5012dc55a
[Enhancement](group commit) optimize pre allocated calculation ( #30893 )
2024-02-18 11:50:17 +08:00
0f47f7f389
[Feature](runtime filter) normalize ignore runtime filter ( #30152 )
...
normalize ignore runtime filter
2024-02-03 20:24:39 +08:00
e9c112b843
[Refact](inverted index) refact inverted index cache to decouple with reader ( #30574 )
2024-02-01 19:00:50 +08:00
ccde65b942
[fix](Cooldown) Enhance calculate logic of _has_data_to_cooldown ( #30244 ) ( #30299 )
2024-01-25 13:25:34 +08:00
1a51d04cb8
[fix](move-memtable) fix schema use-after-free in delta writer v2 ( #30254 )
2024-01-24 10:00:25 +08:00
d525f576e1
[improve] Use lru cache to count the number of column in tablet schema to control memory ( #29668 )
2024-01-12 13:58:19 +08:00
81680383e6
[UT](wal) Add wal dirs info be ut ( #29759 )
2024-01-12 11:57:16 +08:00
0d16ec7345
[improvement](cooldown) do not cooldown tablet without cold data ( #29690 )
2024-01-12 11:57:16 +08:00
7c7dbf15bc
[feature](merge-cloud) Decouple Tablet/TabletManager/TxnManager from global StorageEngine instance ( #29736 )
2024-01-12 11:57:16 +08:00
b0cac0014d
[enhance](FS) Improve FS error code ( #29432 )
2024-01-06 21:17:22 +08:00
85dd606fd1
[fix](group_commit) Fix group_commit ut ( #29587 )
2024-01-06 18:11:13 +08:00
a0c3ddf902
[fix](memory) Fix LRUCacheType::NUMBER charge ( #29588 )
...
if LRUCacheType::NUMBER, charge not add handle_size, because charge at this time is no longer the memory size, but an independent weight.
2024-01-06 10:37:56 +08:00
f40cce1406
[Fix](partition) Skip rowset partition id eq 0 smaller than config wh… ( #29510 )
2024-01-05 19:39:51 +08:00
706463781c
[refactor](group commit) refactor group commit wal code ( #29375 )
2024-01-02 15:52:03 +08:00
03901b9a7a
[enhancement](group_commit): refector relay wal code ( #29183 )
2023-12-30 12:59:46 +08:00
82635d4b59
[opt](memory) All LRU Cache inherit from LRUCachePolicy ( #28940 )
...
After all LRU Cache inherits from LRUCachePolicy, this will allow prune stale entry, eviction when memory exceeds limit, and define common properties. LRUCache constructor change to private, only allow LRUCachePolicy to construct it.
Impl DummyLRUCache, when LRU Cache capacity is 0, will no longer be meaningless insert and evict.
2023-12-29 16:15:56 +08:00
a525d5c5a3
[refactor](decimal) change type name Decimal128 to Decimal128V2, Decimal128I to Decimal128V3 to avoid confusion ( #29265 )
...
change type name Decimal128 to Decimal128V2, Decimal128I to Decimal128V3 to avoid confusion
2023-12-29 10:11:44 +08:00
fd90c3a6a6
[optimize](cooldown)Reduce the number of calls to the pick_cooldown_rowset ( #27091 )
...
Co-authored-by: xingying01 <xingying01@corp.netease.com >
2023-12-28 13:03:33 +08:00
9ff8bd2e9c
[Enhancement](Wal)Support dynamic wal space limit ( #27726 )
2023-12-27 11:51:32 +08:00
0af6bd6390
[fix](group-commit) check if wal need recovery is abnormal ( #28769 )
2023-12-22 11:06:11 +08:00
aab859be56
[enhance](partition_id) check partition id before store meta ( #28055 )
2023-12-19 21:31:41 +08:00
e6e8632167
[improvement](merge-on-write) Optimize publish when there are missing versions ( #28012 )
...
1. Do not retry publishing on be When there are too many missing versions, just
add to async publish task.
2. To reduce memory consumption, clean up the tasks when there are too many
async publish tasks.
2023-12-13 16:59:25 +08:00
a719d7a222
[fix](memory) Fix LRU Cache of type NUMBER charge ( #28175 )
2023-12-13 11:15:57 +08:00
6da36e1077
[feature](merge-cloud) Refactor write path code by abstract base class ( #26537 )
...
Refactor write path code by abstract base class. Whether to use `StorageEngine` or `CloudStorageEngine` will be determined during compilation instead of runtime `config::cloud_mode` to avoid unexpected null pointer or undefined behavior issues caused by merging code.
Class that depend on `StorageEngine` but are shared by the cloud mode need to have an abstract base class. Common code should be extracted into the base class, while the code that depends on `StorageEngine` should be implemented in a `StorageEngine` mix-in class of the base class.
2023-12-08 14:50:36 +08:00
1ed99c4d8a
[Improvement](inverted index) improve inverted index bkd performance in high concurrent scenario ( #27820 )
...
Improve BKD performance by enable bkd reader cache and improvement of fast compare and visit in compressed values in BKD tree.
2023-12-05 11:39:53 +08:00
13b26ee920
[Fix](core) Fix wal space back pressure core and add regression test ( #27311 )
2023-11-27 15:10:26 +08:00
553e4a8903
[feature-wip](merge-on-write) MOW table support different primary keys and sort keys ( #24788 )
2023-11-24 16:37:30 +08:00
d767804815
[feature](merge-cloud) Decouple rowset id generator and local rowsets gc implementation ( #25921 )
2023-11-10 10:07:02 +08:00
5f62a4462d
[Enhancement](wal) Add wal space back pressure ( #26483 )
2023-11-09 12:29:05 +08:00
8bd06aff7e
[Chore](MoW) remove unused code about rowset tree ( #26282 )
2023-11-02 20:25:27 +08:00
a5ef90dacc
[enhancement](recover) support skipping missing version in select by session variable ( #25654 )
2023-11-02 20:01:51 +08:00
cedab51676
[enhancement](UT) add unit test cases about bitmap ( #25867 )
...
* [fix](bitmap) incorrect result of operator ==
* [enhancement](UT) add unit test cases about bitmap
2023-10-27 11:27:14 +08:00
9519d7ede9
[enhancement](be-ut)Add more indexed column reader be unit test ( #25652 )
...
Added more unit tests
1. key exists or does not exist in a single page
2. key exists or does not exist in multiple pages
3. key is between two pages.
2023-10-23 10:12:53 +08:00
642c149e6a
remove datetime_value and move vecdatetime_value to doris namespace ( #25695 )
...
remove datetime_value and move vecdatetime_value to doris namespace
2023-10-20 22:08:17 +08:00
9b64286f2d
[enhancement](mow-ut)Add compaction commit delete bitmap unit test ( #25569 )
2023-10-20 19:54:12 +08:00
9c9fc84f39
[feature](merge-cloud) Abstract BaseTablet for CloudTablet ( #24929 )
2023-10-18 20:29:04 +08:00
6cb947f72b
[refactor](unused code) delete unused method from field.h ( #25554 )
...
Co-authored-by: yiguolei <yiguolei@gmail.com >
2023-10-18 13:11:14 +08:00
cda8fb6b8b
[fix](load) return Status when error in RowsetWriter::build ( #25381 )
2023-10-17 09:40:23 +08:00
7ea456ef91
[fix](insert) make group commit wal_manager exit elegantly ( #25250 )
2023-10-14 23:14:06 +08:00
cb0076e585
[fix](insert) fix group commit be ut ( #24968 )
2023-10-07 19:50:05 +08:00
642e5cdb69
[Fix](Status) Make Status [[nodiscard]] and handle returned Status correctly ( #23395 )
2023-09-29 22:38:52 +08:00
082bcd820b
[feature](insert) Support wal for group commit insert ( #23053 )
2023-09-26 14:46:24 +08:00
8191cd1dad
[Bug](ScanNode) Fix potential incorrect query result caused by concurrent NewOlapScanNode initialization and Compaction ( #24638 )
...
* Optimize fetch delete predicates
* Fix incorrect query result when compaction eliminate delete predicates between `NewOlapScanNode::_init_scanners` and `NewOlapScanner::init`
* Fix be ut
2023-09-25 22:24:35 +08:00
8eb14eec7c
[enhancement](baddisk) record bad disk in be_custom.conf to handle ( #24639 )
2023-09-21 18:31:58 +08:00
b9ddcbf729
[feature](merge-cloud) Rewrite code related to IOContext ( #24269 )
2023-09-15 19:57:58 +08:00
c7ae2a7d22
[Refactor & Bugfix](static variables) move some static vairables to exec_env ( #24029 )
2023-09-13 09:27:03 +08:00
d8ef9dda59
[feature](merge-cloud) Rewrite FS interface ( #23953 )
2023-09-12 19:20:25 +08:00
