After BE down, transactions related should be cleared.
The hostname is used to retrive the transactions.
But hostname is recorded as hostname + port, the port
should be removed.
Suppose that the current available byte in a disk is `_available_bytes`. After the data with size of `incoming_data_size` is written in the disk, the left space in the disk should be calculated as follow :
`int64_t left_bytes = _available_bytes - incoming_data_size;`
rather than:
`int64_t left_bytes = _disk_capacity_bytes - _available_bytes - incoming_data_size;`
If some partitions between `dynamic_partition.start` and `dynamic_partition.end` are lost
due to some unexpected circumstances when using dynamic partition.
the lost partitions between the current time and `dynamic_partition.end` will be recreated,
but the lost partitions between `dynamic_partition.start` and the current time will not be recreated.
When forwarding show tablet stmt to Master FE. If the result contains null, FE will throw exception.
Fix it by using `\N` instead of "null" in result, to make thrift rpc happy.
The tablet and disk information reporting threads need to report to the FE periodically.
At the same time these two reporting threads will also be triggered by certain events.
The modification in PR #4440 caused these two threads to be triggered only by events,
and could not report regularly.
1. Find the cache node by SQL Key, then find the corresponding partition data by Partition Key, and then decide whether to hit Cache by LastVersion and LastVersionTime
2. Refers to the classic cache algorithm LRU, which is the least recently used algorithm, using a three-layer data structure to achieve
3. The Cache elimination algorithm is implemented by ensuring the range of the partition as much as possible, to avoid the situation of partition discontinuity, which will reduce the hit rate of the Cache partition,
4. Use the two thresholds of maximum memory and elastic memory to control to avoid frequent elimination of data
* [SQL] Support where, limit, order clause in show resourcestmt.
Grammar
SHOW RESOURCES
[
WHERE
[NAME [ = "your_resource_name" | LIKE "name_matcher"]]
[RESOURCETYPE = ["SPARK"]]
]
[ORDER BY ...]
[LIMIT limit][OFFSET offset];
issue #4501
A new FE UI interface implemented with React framework.
Need to work with the new Spring framework and RESTful API in #4596.
This is a simple UI for Doris. User can develope their own UI if they want
Proposal #4308
Use spring mvc rest to replace the original netty http rest
Created a new package `org/apache/doris/httpv2`,
and the origin implementations under `org/apache/doris/http` remain unchanged.
This part of the code will not be used at present, so it will not affect existing functions.
API document can be found in #4584
Proposal #4308
When the keystype of mv and base table is difference, Doris should execute
sorting schema change instead of linked schema change.
If doesn't, the data size of mv actually is same as base table.
This will cause mv to have no pre-aggregation effect at all.
The query will not choose mv.
This commit fixed this problem. Fixed#4586
* [Bug]Fix bug that tablet info with wrong capacity may cause fe oom
* remove unused code
* fix
* remove empty line
Co-authored-by: caiconghui [蔡聪辉] <caiconghui@xiaomi.com>
After 0.12 version, doris remove the format convert functiion which can convert from hdr_ format
to tabletmeta_ format when loading metas, the commit link: 3bca253
When we update doris version and there are old format meta in storage,
BE will not read the old format tablet. It can lead to data loss.
So we add meta format detection function to prevent data loss.
When there are old format meta in olap_meta, BE can find and print log or exit.
For[Spark Load]
1 support decimal andl largeint
2 add validate logic for char/varchar/decimal
3 check data load from hive with strict mode
4 support decimal/date/datetime aggregator
When Doris create a database, the database of data quota and replica quota is initialized by configuration(Config.default_db_data_quota_bytes and Config.default_db_replica_quota_bytes).
In other words, I convert FeConstants.default_db_data_quota_bytes to Config.default_db_data_quota_bytes
Config.default_db_data_quota_bytes can be change in configuration file.
