Currently, there are some useless includes in the codebase. We can use a tool named include-what-you-use to optimize these includes. By using a strict include-what-you-use policy, we can get lots of benefits from it.
Fixes#2892
IMPORTANT NOTICE: this CL makes incompatible changes to V2 storage format, developers need to create new tables for test.
This CL refactors the metadata and page format for segment_v2 in order to
* make it easy to extend existing page type
* make it easy to add new page type while not sacrificing code reuse
* make it possible to use SIMD to speed up page decoding
Here we summary the main code changes
* Page and index metadata is redesigned, please see `segment_v2.proto`
* The new class `PageIO` is the single place for reading and writing all pages. This removes lots of duplicated code. `PageCompressor` and `PageDecompressor` are now useless and removed.
* The type of value ordinal is changed from `rowid_t` to 64-bits `ordinal_t`, this affects ordinal index as well.
* Column's ordinal index is now implemented by IndexPage, the same with IndexedColumn.
* Zone map index is now implemented by IndexedColumn
This PR implements the build part of bitmap index support. It follows most of the design described in #1684 , but with the following differences and enhancements
1. Bitmap indexes are now written in the segment file for simplicity. Separate index file would be helpful when we're going to support `alter table add bitmap index` in the future though.
2. We switch to a generalized index page format for all data types rather than specialize for each one. Code simplicity and reusability is preferred here than optimal compression rate.
3. We introduce a new abstraction called `IndexedColumn` to unify the processing of the dictionary section and bitmap section of bitmap index. IndexedColumn is a column with an optional ordinal index and an optional value index. Ordinal index enables us to seek to a particular rowid within the column. Value index requires IndexedColumn to store ordered values and enables us to seek to a particular value. Therefore, the dictionary section can be represented by an IndexedColumn with value index and the bitmap section can be represented by an IndexedColumn with ordinal index.
Fix direct compilation failed：
fix compile thirdparty in ubuntu will install libs to lib dir instead of lib64
fix compile error in gcc5 due to the defect of c++11 (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=60970)
fix gcc version check will not work on some OS
We create a new segment format for BetaRowset. New format merge
data file and index file into one file. And we create a new format
for short key index. In origin code index is stored in format like
RowCusor which is not efficient to compare. Now we encode multiple
column into binary, and we assure that this binary is sorted same
with the key columns.
