0002500757
branch-2.1: [Fix](inverted index) fix rename column build index bug #50056 ( #50732 )
...
pick #47562 #50056 from master
---------
Co-authored-by: qiye <luen@selectdb.com >
2025-05-09 17:13:46 +08:00
08e7d920db
branch-2.1: [fix](index build) Correct inverted index behavior after dynamically adding a column #48389 ( #48546 )
...
Cherry-picked from #48389
---------
Co-authored-by: airborne12 <jiangkai@selectdb.com >
2025-03-05 09:26:54 +08:00
1aa57a3b13
branch-2.1: [fix](array index) Correct null bitmap writing for inverted index #47846 ( #48214 )
...
cherry pick from #47846 #48231
2025-02-25 20:31:18 +08:00
08cb06396b
branch-2.1: [Fix](index build) should not append index to meta while column id is -1 #46307 ( #46345 )
...
cherry pick from #46307
Co-authored-by: airborne12 <jiangkai@selectdb.com >
2025-01-03 21:59:55 +08:00
4c224bb1dd
[improvement](build index)Optimize failed task check on same tablet ( #42295 ) ( #43589 )
...
bp #42295
Co-authored-by: qidaye <luen@selectdb.com >
2024-11-12 09:56:55 +08:00
12bb38eab3
[fix](build index)Fix build index failed on renamed column ( #42882 )( #43246 )( #43336 ) ( #43266 )
...
bp #42882 #43246 #43336
We pick these PRs together because they all fix one problem and if not
do so, the regression test won't be passed.
---------
Co-authored-by: Sun Chenyang <csun5285@gmail.com >
Co-authored-by: qidaye <luen@selectdb.com >
2024-11-07 21:32:02 +08:00
e43d255815
[fix](array) fix add_array_null_value for inverted index_builder ( #42899 )
...
## Proposed changes
Pick #42638 to doris branch 2.1
Issue Number: close #xxx
<!--Describe your changes.-->
2024-10-30 19:16:59 +08:00
cc85f7b94c
[fix](build index)Remove index_meta in tablet schema when the index is dropped. ( #37646 ) ( #37897 )
2024-07-16 20:32:30 +08:00
fbcf63e1f5
[cherry-pick] (branch-2.1)fix variant index ( #36577 )
...
pick from master #36163
2024-06-20 17:57:26 +08:00
79cd726132
[Fix](inverted index) fix race condition in index build ( #35427 )
...
Fix race condition problem introduced by #35366 , which will cause heap-use-after-free
2024-05-28 13:12:03 +08:00
5ab5ec3d0d
[Fix](inverted index) fix build index wrong size for inverted index ( #35366 )
2024-05-27 15:24:17 +08:00
b4a798240a
[fix](inverted_index) donot use int32_t for index id to avoid overflow ( #35062 )
2024-05-21 12:58:38 +08:00
8f6f4cf0eb
[Pick](Variant) pick #33734 #33766 #33707 to branch-2.1 ( #33848 )
...
* [Fix](Variant Type) forbit distribution info contains variant columns (#33707 )
* [Fix](Variant) VariantRootColumnIterator::read_by_rowids with wrong null map size (#33734 )
insert_range_from should start from `size` with `count` elements for null map
* [Fix](Variant) check column index validation for extracted columns (#33766 )
2024-04-18 19:42:44 +08:00
9dfc9665fb
[Fix](inverted index) fix build index error status when batch_next ( #33532 )
2024-04-17 23:41:59 +08:00
39fba884fb
[fix](typo) typo fix for 'delete bimap' changing to 'delete bitmap' ( #32341 )
2024-04-10 11:34:30 +08:00
ea71472d64
[fix](build index) fix core when build index for a new column which without data ( #32550 ) ( #32669 )
...
Co-authored-by: Luennng <luennng@gmail.com >
Co-authored-by: Tanya-W <tanya1218w@163,com>
2024-03-22 15:05:19 +08:00
ecadb60bcd
[Pick 2.1](inverted index) support inverted index format v2 ( #30145 ) ( #32418 )
2024-03-19 08:11:33 +08:00
779ca464a5
[Fix](Status) Handle returned overall Status correctly ( #31692 )
...
Handle returned overall Status correctly
2024-03-09 19:44:39 +08:00
61bab1f1c0
[Fix](inverted index) fix data size when drop inverted index ( #30327 )
2024-01-25 13:24:52 +08:00
a314491535
[Fix](inverted index) fix array inverted index builder error ( #29869 )
2024-01-12 13:58:19 +08:00
fa4721125e
[Fix](show data) Inverted index size not added to data size after index build ( #29568 )
2024-01-05 16:21:45 +08:00
7398c3daf1
[Feature-Variant](Variant Type) support variant type query and index ( #27676 )
2023-11-29 10:37:28 +08:00
d767804815
[feature](merge-cloud) Decouple rowset id generator and local rowsets gc implementation ( #25921 )
2023-11-10 10:07:02 +08:00
9c9fc84f39
[feature](merge-cloud) Abstract BaseTablet for CloudTablet ( #24929 )
2023-10-18 20:29:04 +08:00
239df5860b
[enhancement](tablet_meta_lock) add more trace for write lock of tablet's _meta_lock ( #25095 )
2023-10-08 10:28:10 +08:00
642e5cdb69
[Fix](Status) Make Status [[nodiscard]] and handle returned Status correctly ( #23395 )
2023-09-29 22:38:52 +08:00
9c6734e68e
[bugfix](index) Fix build index limitations ( #24358 )
...
1. skip existed index on column with different id on build index
2. allow build index for CANCELED or FINISHED state
2023-09-14 17:53:22 +08:00
f7a3d2778a
[FIX](array)update array olapconvertor and support array nested other complex type ( #23489 )
...
* update array olapconvertor and support array nested other complex type
* update for inverted index
2023-08-29 16:18:11 +08:00
d4694167a8
[Enhancement](chore) Some Status relevant enhancement ( #23072 )
2023-08-21 14:14:38 +08:00
f0ad3ef244
[fix](merge-on-write) should use write lock of tablet's header lock in #23047 ( #23161 )
2023-08-18 17:50:44 +08:00
e6fe8c05d1
[fix](inverted index change) fix update delete bitmap incompletely when build inverted index on mow table ( #23047 )
2023-08-18 11:15:39 +08:00
de98324ea7
[fix](inverted index change) make mutex for ALTER_INVERTED_INDEX task and STORAGE_MEDIUM_MIGRATE task ( #22995 )
2023-08-18 08:35:30 +08:00
00c48f7d46
[opt](regression case) add more index change case ( #21734 )
2023-07-12 21:52:48 +08:00
ca71048f7f
[Chore](status) avoid empty error msg on status ( #21454 )
...
avoid empty error msg on status
2023-07-11 13:48:16 +08:00
cf1efce824
[fix](inverted index) use index id instead of column uid to determine whether a hard link is required when build index ( #21574 )
...
Fix problem:
For the same column, there are concurrent drop index request and build index request, if build index obtain lock before drop index, build a new index file, but when drop index request execute, link file not contains all index files for the column, that lead to new index file is missed.
Based on the above questions, use index id instead of column unique id to determine whether a hard link is required when do build index
2023-07-09 16:45:27 +08:00
a2e679f767
[fix](status) Return the correct error code when clucene error occured ( #21511 )
2023-07-06 09:08:11 +08:00
92882ebd91
[fix](inverted index) update output rowset index meta with input rowset when drop inverted index ( #21248 )
2023-06-27 23:54:35 +08:00
d545e00bc7
[improve](error) include detailed messages in rowset reader init error ( #21229 )
2023-06-27 20:45:14 +08:00
960e04b0ed
[fix](inverted index) fix build inverted index failed but not return immediately ( #21165 )
2023-06-26 14:05:12 +08:00
ab8125d56f
[Improve](performance) introduce SchemaCache to cache TabletSchame & Schema ( #20037 )
...
* [Improve](performance) introduce SchemaCache to cache TabletSchame & Schema
1. When the system is under high-concurrency load with wide table point queries, the frequent memory allocation and deallocation of Schema become evident system bottlenecks. Additionally, the initialization of TabletSchema and Schema also becomes a CPU hotspot.Therefore, the introduction of a SchemaCache is implemented to cache these resources for reuse.
2. Make some variables wrapped with std::unique<unique_ptr>
Performance:
| 状态 | QPS | 平均响应时间 (avg) | P99 响应时间 |
|------------------|-----|------------------|-------------|
| 开启 SchemaCache | 501 | 20ms | 34ms |
| 关闭 SchemaCache | 321 | 31ms | 61ms |
* handle schema change with schema version
* remove useless header
* rebase
2023-05-29 17:34:53 +08:00
ae352997b4
[Enhancement](alter inverted index) Improve alter inverted index performance with light weight add or drop inverted index ( #19063 )
2023-05-28 11:23:07 +08:00
