Commit Graph

1233 Commits

Author SHA1 Message Date
17af23edae 【Improvement】Avoid null host when forward to master (#4844)
Co-authored-by: wangxixu <wangxixu@xiaomi.com>
2020-11-10 10:54:29 +08:00
4204a878c8 [Bug] Fix some bugs of load job scheduler (#4869)
* [Bug] Fix some bugs of load job scheduler

1. The fix load meta bug logic should be removed since 0.12.
2. The load task thread pool's waiting queue should be as long as desired pending jobs num.
3. Submit the load task outside database lock to prevent holding lock for long time.
2020-11-10 10:50:31 +08:00
04cfcf6c36 Update fe-idea-dev.md (#4852)
* Update fe-idea-dev.md

use `brew install thrift@0.9` to install thrift 0.9.3.1
`brew edit thrift090 | head` shows thrift@0.9 uses thrift 0.9.3.1

* [Refactor] Remove the unnecessary if statement

Future<?> submit(Runnable task)
Submits a Runnable task for execution and returns a Future representing that task. The Future's get method will return null upon successful completion.
2020-11-08 20:52:15 +08:00
59c7d5021d [Bug][Load] Catch retry submit exception (#4796)
When `Load Job Task Queue` is filled, continue to submit more jobs to the queue will cause 
`RejectedExecutionException`.
But `callback.onTaskFailed` function does not catch the exception, that will cause
re-submitting job failed, and status is not updated to failed.
issue: #4795
2020-11-08 20:50:50 +08:00
bde84e4ae5 [Bug][SQL] Fix bug that query failed when SQL contains Union and Colocate join (#4842)
SQL like:
`select a join b union select c join d`;

if a b is colocate join, and c d is also colocate join, the query may failed
with error like:

`failed to get tablet. tablet_id=26846, with schema_hash=398972982, reason=tablet does not exist`
2020-11-05 20:57:11 +08:00
1d89e0670a [License] Add other license declare in NOTICE (#4831) 2020-11-05 20:30:49 +08:00
c5b034acc4 [FE UI] Fix some bugs about new FE UI (#4830)
1. Add a search boxer in the left tree view of Playground.
2. Fix some visual bugs of UI.
3. Fix bugs that link failed in QueryProfile view.
4. Fix bugs that cookie is always invalid.
5. Set cookie to HTTP_ONLY to make it more safe.
2020-11-05 20:30:09 +08:00
c8df76a807 [FEATURE]Check date type to avoid scan all partitions (#4756)
`select day from test where day='2020-10-32'`
table 'test' is parititioned by day. In this case, '2020-10-32' will be taken as CastExpr not LiteralExpr,
and condition "day='2020-10-32'" will not be recognized as partitionfilter.
This case will scan all partitions. To avoid scall all partitions, it is better to filter invalid date value.

issue: #4755
2020-11-05 20:25:53 +08:00
80d5f6e3d8 [LoadBalance] make BeLoadRebalancer extends from base class Rebalancer (#4771) 2020-11-03 20:23:48 +08:00
d0b7286099 [optimize] optimize default value for thriftserver's config key "thrift_client_timeout_ms" (#4808)
* optimize  default value for thrift server's config key "thrift_client_timeout_ms"

Co-authored-by: liwei5 <liwei5@vipkid.com.cn>
2020-10-30 17:10:03 +08:00
c03808a4e4 [TabletScheduler] Fix some bug where decommission operations cannot be completed (#4804)
1.
When we decommission some BEs with SSD disks,
if there are no SSD disks on the remaining BEs, it will be impossible to select a suitable destination path.
In this case, we need to ignore the storage medium property and try to select the destination path again.
Set `isSupplement` to true will ignore the storage medium property.

2.
When the BE nodes where all replicas of a tablet are located are decommission,
and this task is a VERSION_INCOMPLETE task.
This will lead to failure to select a suitable dest replica.
At this time, we should try to convert this task to a REPLICA_MISSING task, and schedule it again.
2020-10-30 11:50:47 +08:00
a291f4d285 [SQL][Bug] Fix union bug (#4772) (#4807) 2020-10-30 11:49:43 +08:00
44498a1ae2 [Compatibility] Add table "views" in information_schema database (#4778)
To support some tools like DBeaver
2020-10-30 11:44:44 +08:00
9099191038 remove (#4818)
Co-authored-by: yangwenbo6 <yangwenbo3@jd.com>
2020-10-30 10:05:47 +08:00
6b234fb2ce Fix create rollup may duplicate hidden column (#4816) 2020-10-30 10:04:21 +08:00
fe6ac26b74 [BUG] Cast int type to date type (#4806) 2020-10-29 20:36:45 +08:00
54fa76359b [ODBC] Support ODBC external table of PostgreSQL and revise the doc. (#4798) 2020-10-29 14:31:23 +08:00
bfdb15c730 [Bug] Fix some date functions to make their result same as MySQL (#4786)
dayofweek, dayofmonth, dayofyear, weekofyear, timediff

Also fix ut compilation problem
2020-10-27 12:52:44 +08:00
c8d496830b [Compile] Remove unused import (#4794) 2020-10-26 22:54:27 +08:00
a61eea3b2e Make some debug log settings configurable and change some log level from info to debug to avoid performance bottlenecks (#4766) 2020-10-26 16:10:19 +08:00
f89b660c94 [Feature] Support Alter Table Clause For External Table (#4699)
* Support Alter Table Clause For External Table
5 alter operation be supported:

* RENAME
* ADD COLUMN
* DROP COLUMN
* MODIFY COLUMN
* REORDER COLUMN
2020-10-25 17:17:56 +08:00
2fa3ffda7b change code style (#4776) 2020-10-24 17:36:06 +08:00
0213e93ea1 [Feature][Config] Support persistence of configuration items modified at runtime (#4704)
Support persistence of configuration items modified at runtime via HTTP API.

```
FE:
GET /api/_set_config?key=value&persist=true

BE
POST /api/update_config?key=value&persist=true
```

The modified config will be saved in `fe_custom.conf` or `be_custom.conf`.
And when process starts, it will load `fe.conf/be.conf` first, then `fe_custom.conf/be_custom.conf`.
2020-10-22 21:39:02 +08:00
16f5d223e8 [Compile] Update Repository for java-cup and cup-maven-plugin (#4769) 2020-10-22 21:38:19 +08:00
fae5c624ea Fix delete_sign predicate assigned to join node (#4770) 2020-10-22 14:29:16 +08:00
0c37b42bc7 [Bug] Because of thread-safety problem , replace connectionMap with concurrentMap from hashMap. (#4774)
Thread [Connect-Scheduler-Check-Timer] fail to iterate the connectionMap , because other threads will modify the connectionMap at the same time.
Fe's connection which is timed out can't be released because of thread [Connect-Scheduler-Check-Timer] not working.
2020-10-22 11:15:21 +08:00
349cc9ef17 [Bug] Do not push down limit operation when ODBC table do not push all conjunct as filter. (#4764) 2020-10-21 10:12:12 +08:00
a95ce69c0d [Doc] Bug fix that help commend not work (#4760)
There are 2 docs with same name "bitmap", which cause error
when building help system.
2020-10-20 09:47:51 +08:00
34d5ab8e07 [BUG] Ensure that the correct lead/lag function is selected (#4732)
* fix

* fix
2020-10-18 21:38:10 +08:00
d371d363a4 Fix bug CTE statement with nested select (#4731)
Co-authored-by: lixueyan07 <lixueyan07@meituan.com>
2020-10-17 13:36:14 +08:00
fa28542da8 Fix ut (#4752)
Change-Id: I7e3eb8bea00518ad71fd0b63d6fd9c440a536993
2020-10-16 15:35:15 +08:00
21e7428999 [Feature] Support cancel load jobs in batch (#4515)
Support statement like:
`cancel load where label like 'xxx';`
2020-10-15 22:49:39 +08:00
5a6e926c7b fix join error when the table has enbale batch delete (#4734) 2020-10-14 09:32:57 +08:00
453aeb7a19 [UT] Fixes PartitionCacheTest unit test randomly failed (#4729)
Fixes PartitionCacheTest unit test randomly failed. #4728
2020-10-14 09:31:15 +08:00
bf8f68f751 [Bug] Add regular column when materialized slot is empty in tuple (#4719)
If the materialized slot is empty in inline view, the scanner will throw exception 'no materialized slot!'.
When the outer query does not need any real columns of the inner query at all, such as the outer query only contains constant calculations,
all slots in the inner query will not be materialized.
For example:
```
select c1 from (select 'xx' c1, k1 from test) t1;
```

But this does not mean that the inner query (select 'xx' c1, k1 from test) does not need to be executed.
Because the number of rows in the outer result needs to be determined by the number of rows in the inner query.
In this case, the inner query scan node needs to add a column to ensure the correct result.
At the same time it can avoid the scan node reporting errors.
2020-10-13 10:12:03 +08:00
f431d8d94c [Enhance][Log] Make RPC error log more clear (#4702)
At present, when some rpc errors occur, the client cannot obtain the error information well.

And this CL change the RPC error returned to client like this:

```
ERROR 1064 (HY000): errCode = 2, detailMessage = there is no scanNode Backend. [10002: in black list(A error 
occurred: errorCode=2001 errorMessage:Channel inactive error!)]

ERROR 1064 (HY000): failed to send brpc batch, error=The server is overcrowded, error_text=[E1011]The server is 
overcrowded @xx.xx.xx.xx:8060 [R1][E1011]The server is overcrowded @xx.xx.xx.xx:8060 [R2][E1011]The server is 
overcrowded @xx.xx.xx.xx:8060 [R3][E1011]The server is overcrowded @xx.xx.xx.xx:8060, client: yy.yy.yy.yy
```
2020-10-13 10:08:43 +08:00
c934cf93e1 [UI Part 5] Enable HTTP Server 2 by FE config (#4684)
This is the last PR of proposal #4308

1. Add a new FE config `enable_http_server_v2` to enable new HTTP Server implementation. The default value is false.
2. Add a new FE config `http_api_extra_base_path` so that we can set base path for Frontend UI.
3. Refactor the new HTTP API response body. The return http status code is always 200, and using internal code in response body to indicate the certain error.
2020-10-13 10:07:50 +08:00
c69fe23c42 [Bug] Fix hard cardinality check which makes queries fail (#4678)
fix hard `Preconditions.checkState()` by setting `cardinality` to `-1` when computation is invalid.
2020-10-13 10:07:22 +08:00
3f0cb04376 [Feature] Support REPLACE TABLE operation. (#4669)
Implement REPLACE TABLE feature.
Details can be found in document `alter-table-replace-table.md`
2020-10-13 10:06:57 +08:00
d73d205de7 [ODBC/MySQL] Support Limit Clause Push Down For ODBC Table And MySQL Table(#4706) (#4707)
1. Support limit clause push down both odbc table and mysql table.
2. Code refactor of ODBC Scan Node, change `build_connect_string` and `query_string` from BE to FE to make it easily to modify
2020-10-11 21:11:04 +08:00
04f26e4b7f [SQL] Support Bucket Shuffle Join (#4677)
Support Bucket Shuffle Join
issue:#4394
2020-10-11 15:37:32 +08:00
28f4e922a7 [CREATE TABLE]Support new syntax CREATE TABLE LIKE to clone an existe… (#4705)
Support new synatx CREATE TABLE [IF NOT EXISTS] [db_name].table_name AS [db_name2].table_name2;
to create a new table from existed table with same table schema.
ISSUE: #4355
2020-10-10 21:16:53 +08:00
94a81e52c7 Fix typo in our code (#4696)
* fix typo in our code

* fix

* fix
2020-10-10 21:16:25 +08:00
6ab858bb10 [Bug] Fix duplicate columns in case when statement (#4693)
Fix #4692
The reason of this bug is case-when statement may produce implicit CastExpr<SlotRef>
as SelectListItem in analyze step.
And these CastExpr<SlotRef> in SelectList will not be re-anlyze after rewrite step,
which will result in the incorrect number of self-incrementing SlotDescriptor ID in resultExprs .

So we need to reset the analysis state of CastExpr<SlotRef> in rewrite step.
2020-10-10 21:16:00 +08:00
ed09ea9cf7 [Bug] Fix bug that tablet report always out of date (#4695)
This is because the report version is not set correctly in the tablet report request.

Also modify the download url of libevent Fix #4689 Fix #4691
2020-10-09 15:54:31 +08:00
725ee59824 Fix mysqlslap hang under high concurrent (#4680) 2020-10-07 12:30:18 +08:00
f0bb012dd2 [Bug] Fix Windows function lag()/lead() function throw AnalysisException. (#4666)
Add lag() and lead() char and varchar check in `isCastMatchAllowed`.
2020-10-04 10:24:36 +08:00
97dd634cdc Fix hidden cloumn may disappeared (#4686)
* xx

* xx

* xx
2020-09-30 10:01:49 +08:00
0475aa9b93 [Bug]Fix delete on clause may not work in routineLoad (#4683)
fix delete on may not work in some cases, this is describe in #4682
2020-09-30 09:56:19 +08:00
9f001efc16 [Bug]Fix information_schema.columns table column_comment does not show (#4676) 2020-09-28 13:40:33 +08:00