Ashin Gau 3e186a8821 [opt](MergedIO) optimize merge small IO, prevent amplified read (#20305) ...

Optimize the strategy of merging small IO to prevent severe read amplification, and turn off merged IO when file cache enabled.
Adjustable parameters:
```
// the max amplified read ratio when merging small IO
max_amplified_read_ratio=0.8
// the min segment size
file_cache_min_file_segment_size = 1048576
```

2023-06-03 10:51:24 +08:00

..

bool_plain_decoder.cpp

[Optimize] add has_filter template param in get_next_run() to decrease _has_filter condition checking count in the loop. (#19043)

2023-04-27 21:23:36 +08:00

bool_plain_decoder.h

[Optimize] add has_filter template param in get_next_run() to decrease _has_filter condition checking count in the loop. (#19043)

2023-04-27 21:23:36 +08:00

bool_rle_decoder.cpp

[Optimize] add has_filter template param in get_next_run() to decrease _has_filter condition checking count in the loop. (#19043)

2023-04-27 21:23:36 +08:00

bool_rle_decoder.h

[Optimize] add has_filter template param in get_next_run() to decrease _has_filter condition checking count in the loop. (#19043)

2023-04-27 21:23:36 +08:00

byte_array_dict_decoder.cpp

[Chore](ub) fix some undefined behaviors (#19986)

2023-05-26 14:08:40 +08:00

byte_array_dict_decoder.h

[Optimize] add has_filter template param in get_next_run() to decrease _has_filter condition checking count in the loop. (#19043)

2023-04-27 21:23:36 +08:00

byte_array_plain_decoder.cpp

[Optimize] add has_filter template param in get_next_run() to decrease _has_filter condition checking count in the loop. (#19043)

2023-04-27 21:23:36 +08:00

byte_array_plain_decoder.h

[Optimize] add has_filter template param in get_next_run() to decrease _has_filter condition checking count in the loop. (#19043)

2023-04-27 21:23:36 +08:00

decoder.cpp

[chore](build) Use include-what-you-use to optimize includes (PART II) (#18761)

2023-04-19 23:11:48 +08:00

decoder.h

[Optimize] add has_filter template param in get_next_run() to decrease _has_filter condition checking count in the loop. (#19043)

2023-04-27 21:23:36 +08:00

delta_bit_pack_decoder.cpp

[Optimize] add has_filter template param in get_next_run() to decrease _has_filter condition checking count in the loop. (#19043)

2023-04-27 21:23:36 +08:00

delta_bit_pack_decoder.h

[Optimize] add has_filter template param in get_next_run() to decrease _has_filter condition checking count in the loop. (#19043)

2023-04-27 21:23:36 +08:00

fix_length_dict_decoder.hpp

[Optimize] add has_filter template param in get_next_run() to decrease _has_filter condition checking count in the loop. (#19043)

2023-04-27 21:23:36 +08:00

fix_length_plain_decoder.cpp

[Optimize] add has_filter template param in get_next_run() to decrease _has_filter condition checking count in the loop. (#19043)

2023-04-27 21:23:36 +08:00

fix_length_plain_decoder.h

[Optimize] add has_filter template param in get_next_run() to decrease _has_filter condition checking count in the loop. (#19043)

2023-04-27 21:23:36 +08:00

level_decoder.cpp

[chore](build) Use include-what-you-use to optimize includes (PART II) (#18761)

2023-04-19 23:11:48 +08:00

level_decoder.h

[chore](build) Use include-what-you-use to optimize includes (PART II) (#18761)

2023-04-19 23:11:48 +08:00

parquet_common.cpp

[Optimize] add has_filter template param in get_next_run() to decrease _has_filter condition checking count in the loop. (#19043)

2023-04-27 21:23:36 +08:00

parquet_common.h

[Optimize] add has_filter template param in get_next_run() to decrease _has_filter condition checking count in the loop. (#19043)

2023-04-27 21:23:36 +08:00

parquet_pred_cmp.h

[Optimize](parquet-reader) Opt by filtering null count statistics in rowgroup and page level. (#19106)

2023-04-27 21:21:30 +08:00

parquet_thrift_util.h

[opt](FileReader) merge small IO to optimize read performace (#18796)

2023-04-23 10:51:38 +08:00

schema_desc.cpp

[fix](multi-catalog) verify the precision of datetime types for each data source (#19544)

2023-05-17 20:50:15 +08:00

schema_desc.h

[Fix](mutli-catalog) Use decimal v3 type to fix decimal loss issue in multi-catalog module. (#18835)

2023-04-20 11:02:53 +08:00

vparquet_column_chunk_reader.cpp

[fix](parquet) hasn't initialize select vector when number of nested values equals zero (#18953)

2023-04-25 14:21:33 +08:00

vparquet_column_chunk_reader.h

[chore](build) Use include-what-you-use to optimize includes (PART II) (#18761)

2023-04-19 23:11:48 +08:00

vparquet_column_reader.cpp

[fix](NestedType) throw error when reading complex nested type in orc&parquet (#19489)

2023-05-11 07:51:02 +08:00

vparquet_column_reader.h

[chore](build) Use include-what-you-use to optimize includes (PART II) (#18761)

2023-04-19 23:11:48 +08:00

vparquet_file_metadata.cpp

[chore](build) Use include-what-you-use to optimize includes (PART II) (#18761)

2023-04-19 23:11:48 +08:00

vparquet_file_metadata.h

[chore](build) Use include-what-you-use to optimize includes (PART II) (#18761)

2023-04-19 23:11:48 +08:00

vparquet_group_reader.cpp

[Fix](multi-catalog) Fix parquet bugs of #19758 'replace the single pointer with an array of 'conjuncts' in ExecNode'. (#20191)

2023-05-30 09:55:12 +08:00

vparquet_group_reader.h

[refactor](exec) replace the single pointer with an array of 'conjuncts' in ExecNode (#19758)

2023-05-29 11:47:31 +08:00

vparquet_page_index.cpp

[Optimize](parquet-reader) Opt by filtering null count statistics in rowgroup and page level. (#19106)

2023-04-27 21:21:30 +08:00

vparquet_page_index.h

[chore](build) Use include-what-you-use to optimize includes (PART II) (#18761)

2023-04-19 23:11:48 +08:00

vparquet_page_reader.cpp

[chore](build) Use include-what-you-use to optimize includes (PART II) (#18761)

2023-04-19 23:11:48 +08:00

vparquet_page_reader.h

[chore](build) Use include-what-you-use to optimize includes (PART II) (#18761)

2023-04-19 23:11:48 +08:00

vparquet_reader.cpp

[opt](MergedIO) optimize merge small IO, prevent amplified read (#20305)

2023-06-03 10:51:24 +08:00

vparquet_reader.h

[refactor](exec) replace the single pointer with an array of 'conjuncts' in ExecNode (#19758)

2023-05-29 11:47:31 +08:00