Commit Graph

13 Commits

Author SHA1 Message Date
e68834158c [fix](inverted index)Support Chinese column name with inverted index #36321 (#36374)
1. `std::string` to `std::wstring` conversion only supports ASCII
characters. For non-ASCII characters, we need to use
`StringUtil::string_to_wstring`
2. Fix index_tool check_terms_stats_v2 and add field info to print

pick from master #36321
2024-06-17 19:41:09 +08:00
4b7608c2bf [fix](inverted index)Change index_id from int32 to int64 to avoid overflow (#35206)
Co-authored-by: Luennng <luennng@gmail.com>
2024-05-23 19:12:55 +08:00
a4a191fe56 [fix](index compaction)Fix MOW index compaction core (#32121) (#32657) 2024-03-22 14:20:19 +08:00
ecadb60bcd [Pick 2.1](inverted index) support inverted index format v2 (#30145) (#32418) 2024-03-19 08:11:33 +08:00
2e94512e07 [fix](build)Fix index_tool build error (#31834)
Co-authored-by: Luennng <luennng@gmail.com>
2024-03-07 16:11:25 +08:00
d8b9909675 [Fix](Status) Handle returned Status correctly #31434 2024-03-01 04:25:43 +08:00
7e3ed8a19a [fix](index_tool) fix index_tool compile error (#30049) 2024-01-18 10:04:21 +08:00
f8aba3faae [Enhancement](index tool) refine inverted index tool code (#29717) 2024-01-12 11:46:29 +08:00
87a6def862 [Fix](index tool) fix inverted index tool memory leak (#29701) 2024-01-12 11:44:21 +08:00
a14daca7ba [feature](inverted index)write separated index files in RAM directory to reduce IO(#28810)
Normally we write the separate index files to disk before we merge the index files into an idx compound file.
In high-frequency load scenarios, disk IO can become a bottleneck. 
In order to reduce the pressure on the disk, we write the standalone index file to the RAM directory for the first time, and then write it to the disk when merging it into a composite file.

Add config `index_inverted_index_by_ram_dir_enable`, default is `false`.
2023-12-28 17:18:59 +08:00
f698bb7be2 [Feature](inverted index) index tool add match_all and match_phrase (#26896) 2023-11-13 22:53:11 +08:00
642e5cdb69 [Fix](Status) Make Status [[nodiscard]] and handle returned Status correctly (#23395) 2023-09-29 22:38:52 +08:00
8371171e44 [Feature](inverted index) add inverted index tool (#22207) 2023-07-27 21:28:34 +08:00