1. Analyze what mode of cache can be used by query
2. Query cache before executing query in StmtExecutor
3. Two cache mode, sqlcache and partitioncache, are implemented
The param of rand() function should be literal, but current compiler ignore to
validate the literal param of rand function, it is validated in execution step.
This PR make it validated in compile step, and make it more earlier to find the usage error of rand() function.
Sometimes we want to detect the hotspot of a cluster, for example, hot scanned tablet, hot wrote tablet,
but we have no insight about tablets in the cluster.
This patch introduce tablet level metrics to help to achieve this object, now support 4 metrics on tablets: `query_scan_bytes `, `query_scan_rows `, `flush_bytes `, `flush_count `.
However, one BE may holds hundreds of thousands of tablets, so I add a parameter for the metrics HTTP request,
and not return tablet level metrics by default.
In DistributedPlanner, do not add the unnecessary Exchanges.
For case 1, we only need to judge that the table's distribute hash keys is a subset of the aggregate keys.
For case 2, we should judge two conditions:
- partition keys are also hash keys.
- the table's distribute hash keys is a subset of the aggregate keys.
1. When WITH_MYSQL is off, load error hub does not suport MySQL load error hub,
we should check its return value.
2. misjudge the return value of `change_row_block` in schema_change.cpp
Segment index file content is not set as 0 when it is constructed in write procedure,
so when load index from this file, and meet a null VARCHAR cell,
the null field of this cell is 0, but the length field which is not initialized maybe a large random number,
then memory copy may cause overflow.
This patch fix this bug, and also skip useless memory copy to improve a bit of performance.
Persistence stale rowsets meta. When BE reboots, stale rowsets meta
can resume and the stale version can also be readable before stale gc time.
ISSUE: #4453
1. fix write dpp result when dpp throw exception
2. boolean value：true, false(IgnoreCase), 0, 1
3. wrong dest column for source data check
4. support * in source file path
5. if job state is cancelled or finished, submitPushTasks would throw all partitions have no load data exception,
because tableToLoadPartitions was already cleaned up
#3433
In the process of historical data transformation of materialized views, it may occur that the transformation fails due to data quality.
Add an error status code ：OLAP_ERR_DATE_QUALITY_ERR to determine if a data problem is causing the failure
#3344
Send fields after first row arrived so that error packet can be send to client
when exception thrown from coord.getNext().
Golang and Python can not identify error if fields packet arrived before error packet.
This cl will use yarn command as follows to kill or get status of application running on YARN.
```
yarn --config confdir application <-kill | -status> <Application ID>
```
1. Support convert(expr, target_type) function, which is same as CastExpr
2. Support cast (expr as signed/unsigned int)
This is just for compatibility, the signed/unsigned specification is meaningless.
1. Fix core bug wild pointer in PlanFragmentExecutor, fix issue #4447
2. Fix core bug wild pointer json load, fix issue #4452
3. Change the declare order of ODBC type in thrift for compatibility
The mv column with bitmap_union function is named `mv_bitmap_union_k1` inside of Doris.
But this column name should not be shown to user in `Show Proc` stmt.
Instead, using define expr is easier to understand.
Change-Id: Id07274fef9b3a97c97f1635dd3d6cf7b09561c1e
The DECIMAL, CHAR, VARCHAR have their own scale and precision in column.
The mv column should keep those scale and precision.
Fixed#4433
Change-Id: Ie288738a4356e60d11ea472dd274e54bc7ae6990
1. The base column of bitmap_union could must be integer. The largeint is not supported too.
2. The base column of hll_union could not be decimal.
Check error msg of const expr in Union Node
If user wants to insert a negative number into bitmap mv, Doris will thrown exception 'invalid input'.
The const value in Union Node is checked in this commit.
When creating a tablet, it is necessary to select a disk from all disks that
meet the requirements on the BE node to store the tablet.
In Doris, the current disk selection strategy is to randomly select a disk
from all disks that meet the requirements for tablet creation.
After the cluster has been running for a long time, we found that the
distribution of the number of tablets on different disks in a BE node is unbalanced.
In order to solve this problem, we introduced the algorithm of "two random choices"
for disk selection when creating the tablet:
(1) Select two disks from all disks that meet the requirements on the BE node randomly；
(2) Choose the disk with a smaller number of tablet from the two disks selected in (1) for tablet creation.
1. Disable the MySQL client and LZO library by default when building the Doris.
MySQL client library is used for MySQL external table feature.
This feature will be replaced by the new ODBC external table soon.
LZO library is used to compress/decompress data of some old data format of Doris,
which is no longer used anymore.
2. Add missing license to some files.
3. For all non-Apache-License code, all are explained in NOTICE file and the corresponding license is declared.
4. Remove the js source code from webroot, it will be downloaded as thirdparty
Since the Segment V2 has been released for a long time, we should make it as default storage format for newly created table.
This CL mainly changes:
1. For all newly created tables, their default storage format is Segment V2.
2. For all already exist tablets, their storage format remain unchanged.
3. Fix bugs described in Fix#4384 and Fix#4385
