1.
Add timer to count the time the transfer thread waits for the scaner thread to return rowbatch.
2.
Add timer to count the time that the scanner thread waits for the available worker threads in the thread pool.
Co-authored-by: chenmingyu <chenmingyu@baidu.com>
The following class
org.apache.doris.clone.ColocateTableBalancer
org.apache.doris.common.proc.ProcService
org.apache.doris.rpc.BackendServiceProxy
org.apache.doris.common.util.ProfileManager
org.apache.doris.qe.HelpModule
org.apache.doris.common.publish.ClusterStatePublisher
is not safe in multiple thread environment. This PR is to implement a secure singleton mode.
The class org.apache.doris.common.publish.ClusterStatePublisher singleton method is not used.
1.
User can export query result to local disk like:
`select * from tbl into outfile ("file:///disk1/result_");`
And modify the return result to show the details of export:
```
mysql> select * from tbl1 limit 10 into outfile "file:///home/work/path/result_";
+------------+-----------+----------+--------------+
| FileNumber | TotalRows | FileSize | URL |
+------------+-----------+----------+--------------+
| 1 | 2 | 8 | 192.168.1.10 |
+------------+-----------+----------+--------------+
```
2.
Support create a mark file after export successfully finished.
Co-authored-by: chenmingyu <chenmingyu@baidu.com>
* (#5390)fix NPE when replay colocate group
* remove table id from colocate group when duplicate create table
* remove tablet id when duplicate create table,just like ddlexception
* add ut
* [Enhance] Sort directories by available space when do trash sweep
In the case when one disk is about to be full, we want to sweep trash
data on this disk as quickly as possible. The currently trash sweep
function is to remove trashed files order by path's name, however, disk
data directories may have some large different available space because
of the load balance algorithm, this patch improve it to remove files by
directories' available space.
* add log
* [Bug] Fix bug that the log message is wrong when ReportHandler can not find tablet meta from TabletInvertedIndex
Co-authored-by: caiconghui [蔡聪辉] <caiconghui@xiaomi.com>
1. Support where clause in export stmt which only export selected rows.
The syntax is following:
Export table [table name]
where [expr]
To xxx
xxxx
It will filter table rows.
Only rows that meet the where condition can be exported.
2. Support utf8 separator
3. Support export to local
The syntax is following:
Export table [table name]
To (file:///xxx/xx/xx)
If user export rows to local, the broker properties is not requried.
User only need to create a local folder to store data, and fill in the path of the folder starting with file://
Change-Id: Ib7e7ece5accb3e359a67310b0bf006d42cd3f6f5
Co-authored-by: weixiang <weixiang06@meituan.com>
When querying sql is with limit, the query statistics will be enlarged by merging the query statistics of each batch.
Fix#5340
1.Support Bucket Shuffle Join when left table is colocate table or Colocate/Bucket Bucket Shuffle Join
2.Enable Local Rumtime Filter when there is Bucket Shuffle Join and Colocate Join
3.Add Doc for Bucket Shuffle Join
* [Bug] Fix bug that the image cannot be pulled after the new fe node is added
This is because httpv2 modified the response body of the "/info" api,
causing FE to fail to obtain info from this api.
And the system did not exit correctly.
This will also cause issues in issue #5292
from_unxitime is a cpu-exhausted function.
SQL: select filed from table where from_unixtime(field) > '2021-03-02',
if there are one million rows of data. Function from_unixtime will be called one million times,
which will make query very slow.
In issue #5443, we try to rewrite from_unixtime into timestamp to reduce calling this function.
This rewriting can bring 2 times query performance improvement.
Currently, FeMetaVersion.java is in fe-common, users may forget to copy fe-common.jar when upgrading the service.
It's really dangerous because the data may be corrupted and can not be recovered.
Some invalid config value may cause BE work in an unexpected behavior,
this patch aim to support config validating when BE bootstrap and update BE's config by API
to reject invalid value.
This is a work to accomplish PR #4423
