[Bug] Fix the memory expand 10~1000x of compression algorithm (#5504)

Fix the memory expand 10~1000x of compression algorithm in load and compaction
This commit is contained in:
HappenLee
2021-03-12 23:04:07 +08:00
committed by GitHub
parent 543ed46bc3
commit e9a73ee278
3 changed files with 6 additions and 2 deletions

View File

@ -50,6 +50,9 @@ Status PageIO::compress_page_body(const BlockCompressionCodec* codec, double min
double space_saving = 1.0 - static_cast<double>(buf.size()) / uncompressed_size;
// return compressed body only when it saves more than min_space_saving
if (space_saving > 0 && space_saving >= min_space_saving) {
// shrink the buf to fit the len size to avoid taking
// up the memory of the size MAX_COMPRESSED_SIZE
buf.shrink_to_fit();
*compressed_body = buf.build();
return Status::OK();
}

View File

@ -1,4 +1,3 @@
```
---
{
"title": "Bucket Shuffle Join",
@ -24,7 +23,7 @@ KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
```
# Bucket Shuffle Join
Bucket Shuffle Join is a new function officially added in Doris 0.14. The purpose is to provide local optimization for some join queries to reduce the time-consuming of data transmission between nodes and speed up the query.

View File

@ -1,7 +1,9 @@
---
{
"title": "Bucket Shuffle Join",
"language": "zh-CN"
}
---
<!--
Licensed to the Apache Software Foundation (ASF) under one