Commit Graph

9 Commits

Author SHA1 Message Date
6d6c3d9703 [Enhancement] Reduce memory consumption by releasing readers earier (#5811)
We created multiple rowset readers to read data of one tablet,
after one rowset reader has reached EOF, it can be released to
reduce resource (typically memory) consumption.
As the same, we can release segment reader when it reach EOF.
2021-06-16 09:37:50 +08:00
b423274f17 [Enhance] Make MemTracker more accurate (#5515) (#5516)
* [Enhance] Make MemTracker more accurate (#5515)
 This PR main about:
 1. Improve the readability of MemTrackers' name
 2. Add the MemTracker of:
    * Load
    * Compaction
    * SchemaChange
    * StoragePageCache
    * TabletManager
 3. Change SchemaChange to a Singleon

* revise some code for Code Review

* change the name of mem_tracker

* keep reader_context have the same lifetime of rowset_reader in schema change.

* change vlog notice to log(warning) in schema change
2021-04-08 09:14:55 +08:00
6fedf5881b [CodeFormat] Clang-format cpp sources (#4965)
Clang-format all c++ source files.
2020-11-28 18:36:49 +08:00
f76dad289e Basic implementation for BetaRowsetReader (#1718) 2019-09-03 13:52:16 +08:00
1e4dd77d2a Add bitmap agg type and udaf (#1610) 2019-08-26 14:24:42 +08:00
c5edf9dae0 Unify Field and ColumnSchema in Storage (#1561)
Currently, we have Field and ColumnSchema to access column data in a
row. These two classes are mostly the same. So we should unify these to
one class. Now, Field has offset information, which is an row attribute,
so we remove offset in Field.

RowCursor now has some logic which belong to Schema, so in this patch I
add Schema attribute to RowCursor to make RowCursor simple. After this
change, only Schema will handle Field/ColumnSchema.

I extract some logic from RowCursor to be/src/olap/row.h, then we can
use same logic to handle different types of row. Each type of row has
same function that to get Cell of this row. A cell represent a column
content with a null indicator.
2019-07-30 14:01:57 +08:00
dbc912d2df Unify ColumnSchemaV2 and ColumnSchema to one (#1545)
Currently, we have two versions of ColumnSchema, in this patch, we unify
these two classes to one class.
2019-07-25 10:48:16 +08:00
0805b05d81 Remove unused FieldInfo (#1540) 2019-07-24 19:33:30 +08:00
0c8e91adf4 Add storage rowwise iterator (#1515)
Use RowwiseIterator to uniform all data fetch in storage engine.
All objects in storage engine can be read in iterator format.
For example: Segment, Rowset.

This patch implement two generic iterators: UnionRowwiseIterator,
MergeRowwiseIterator. These two class will add iterator as its inputs.

To implement iterators, we define a new class RowBlockV2, all data read
from iterator is in this format. We define a new class other than use
old version's RowBlock is because we want to keep old code work
normally.
2019-07-22 14:35:11 +08:00