Fix CVE-2017-5645
In Apache Log4j 2.x before 2.8.2, when using the
TCP socket server or UDP socket server to receive
serialized log events from another application, a
specially crafted binary payload can be sent that,
when deserialized, can execute arbitrary code.
https://www.cvedetails.com/cve/CVE-2017-5645/
* [Bug] Fix bug that call frontend service failed when rpc_timeout is equal with publish_timeout
*This PR is to fix bug that call frontend service failed when rpc_timeout is equal with publish_timeout and fix some small issues with log
Co-authored-by: caiconghui [蔡聪辉] <caiconghui@xiaomi.com>
* update gcc to gcc 10 and support c++17
update brpc to 0.9.7
update boost to 1.73
remove third-party boost 1.54 for mysql
* update cmake version
* ignore jdk version
* remove unused patch
* avoid use SYS_getrandom call
Add command:
1. EXPLAIN GRAPGH SELECT ...
2. SHOW QUERY PROFILE "..."
Document will be added in next PR
Change-Id: Ifd9365e10b1f9ff4fdf8ae0556343783d97545f0
1. If cumulative compaction compact only one rowset, the old rowset will not be put into `stale_rowset_meta_map`
2. Show rowset id in `/api/compaction/show`
Co-authored-by: xxiao2018 <benghua3_1@sina.com>
The following class
org.apache.doris.clone.ColocateTableBalancer
org.apache.doris.common.proc.ProcService
org.apache.doris.rpc.BackendServiceProxy
org.apache.doris.common.util.ProfileManager
org.apache.doris.qe.HelpModule
org.apache.doris.common.publish.ClusterStatePublisher
is not safe in multiple thread environment. This PR is to implement a secure singleton mode.
The class org.apache.doris.common.publish.ClusterStatePublisher singleton method is not used.
1.
User can export query result to local disk like:
`select * from tbl into outfile ("file:///disk1/result_");`
And modify the return result to show the details of export:
```
mysql> select * from tbl1 limit 10 into outfile "file:///home/work/path/result_";
+------------+-----------+----------+--------------+
| FileNumber | TotalRows | FileSize | URL |
+------------+-----------+----------+--------------+
| 1 | 2 | 8 | 192.168.1.10 |
+------------+-----------+----------+--------------+
```
2.
Support create a mark file after export successfully finished.
Co-authored-by: chenmingyu <chenmingyu@baidu.com>
* (#5390)fix NPE when replay colocate group
* remove table id from colocate group when duplicate create table
* remove tablet id when duplicate create table,just like ddlexception
* add ut
* [Bug] Fix bug that the log message is wrong when ReportHandler can not find tablet meta from TabletInvertedIndex
Co-authored-by: caiconghui [蔡聪辉] <caiconghui@xiaomi.com>
1. Support where clause in export stmt which only export selected rows.
The syntax is following:
Export table [table name]
where [expr]
To xxx
xxxx
It will filter table rows.
Only rows that meet the where condition can be exported.
2. Support utf8 separator
3. Support export to local
The syntax is following:
Export table [table name]
To (file:///xxx/xx/xx)
If user export rows to local, the broker properties is not requried.
User only need to create a local folder to store data, and fill in the path of the folder starting with file://
Change-Id: Ib7e7ece5accb3e359a67310b0bf006d42cd3f6f5
1.Support Bucket Shuffle Join when left table is colocate table or Colocate/Bucket Bucket Shuffle Join
2.Enable Local Rumtime Filter when there is Bucket Shuffle Join and Colocate Join
3.Add Doc for Bucket Shuffle Join
* [Bug] Fix bug that the image cannot be pulled after the new fe node is added
This is because httpv2 modified the response body of the "/info" api,
causing FE to fail to obtain info from this api.
And the system did not exit correctly.
This will also cause issues in issue #5292
from_unxitime is a cpu-exhausted function.
SQL: select filed from table where from_unixtime(field) > '2021-03-02',
if there are one million rows of data. Function from_unixtime will be called one million times,
which will make query very slow.
In issue #5443, we try to rewrite from_unixtime into timestamp to reduce calling this function.
This rewriting can bring 2 times query performance improvement.
Currently, FeMetaVersion.java is in fe-common, users may forget to copy fe-common.jar when upgrading the service.
It's really dangerous because the data may be corrupted and can not be recovered.
When there is count(*) function in query, we only need to scan the smallest column.
For example:
Query: select count(*) from (select k1, k2, k3 from base) tmp;
Only k1 which is the smallest column should be scanned.
The remaining columns (k2, k3) should be pruning.
This pr achieves this optimization of column pruning.
Fixed#5409
When a table has multiple partitions, each partition has it's own
version, the version doesn't represent whether it's newer or not. When a
partition has a large version, it may be considered as the largest one
currently, this will cause incorrect query result.
Suppose there are 2 partitions:
PartitionName | VisibleVersion | VisibleVersionTime
p1 | 123 | 2021-02-17 23:31:32
p2 | 23 | 2021-02-22 11:39:19
Partition p1 will be considered as the lastest partition, and there is a
cache before p2's last update time, the cache will hit and return an
error result.
* [doris-1008] support backup and restore directly to cloud storage via aws s3 protocol
* Internal][S3DirectAccess] Support backup,restore,load,export directlyconnect to s3
1. Support load and export data from/to s3 directly.
2. Add a config to auto convert broker access to s3 acces when available
Change-Id: Iac96d4b3670776708bc96a119ff491db8cb4cde7
(cherry picked from commit 2f03832ca52221cc7436069b96c45c48c4bc7201)
* [Internal][S3DirectAccess] File path glob compatible with broker
Change-Id: Ie55e07a547aa22c6fa8d432ca926216c10384e68
(cherry picked from commit d4fb25544c0dc06d23e1ada571ec3f8edd4ba56f)
* [internal] [doris-1008] fix log4j class not found
Change-Id: I468176aca0d821383c74ee658d461aba9e7d5be3
(cherry picked from commit 029adaa9d6ded8503acbd6644c1519456f3db232)
* add poms
Co-authored-by: yangzhengguo01 <yangzhengguo01@baidu.com>
If the difference between the two times exceeds Integer.MAX_VALUE,
the compare's return value will overflow and the flow exception may be triggered when sorting profile.
