Commit Graph

4041 Commits

Author SHA1 Message Date
f4663ad2eb [improvement](vectorized) Merge block in scanner to speed up query with conjunct (#8395) 2022-03-09 13:11:18 +08:00
51103dcf6e [typo] translate the comments of delete_handler.cpp (#8402) 2022-03-09 13:08:28 +08:00
97020bcf80 [doc] update substring.md (#8398)
correct a slip of pen.
2022-03-09 13:07:32 +08:00
25532c65d1 [doc] Update BROKER LOAD.md (#8361) 2022-03-09 13:06:24 +08:00
d880559214 [refactor] remove old schema change code on BE (#8342) 2022-03-09 13:05:44 +08:00
3d30209e9b [docs] add document conditional-functions (#8339)
Add conditional-functions.
2022-03-09 13:05:13 +08:00
0ff7de4157 [refactor] remove agent status (#8273)
There are 3 error code types in BE: OLAPStatus AgentStatus Status.
It is very confused and sometimes conflict during write code.
I will try to unify them to Status.
2022-03-09 13:04:50 +08:00
Pxl
10c3712aa1 [fix](vectorized) fix arithmetic calculate get wrong result(#8226) 2022-03-09 13:03:57 +08:00
826467e116 [fix](replica) handle replica version missing info to avoid -214 error (#8209)
In the original tablet reporting information, the version missing information is done by combining
two pieces of information as follows:

1. the maximum consecutive version number
2. the `version_miss` field

The logic of this approach is confusing and inconsistent with the logic of checking for missing versions when querying.

After the change, we directly use the version checking logic used in the query, and set `version_miss` to true
if a missing version is found

and on the FE processing side. Originally, only the **bad replica** information was syncronized among FEs,
but not the **version missing** information. As a result, the non-master FE is not aware of the missing version information.

In the new design, we deprecate the original log persistence class `BackendTabletsInfo` and use the new 
`BackendReplicasInfo` to record replica reporting information and write both **bad** and **version missing**
information to metadata so that other FEs can synchronize these information.
2022-03-09 13:03:22 +08:00
b536c1850f [fix] fix wrong examples in week.md (#8397)
fix wrong examples in week docment
2022-03-08 21:26:43 +08:00
Pxl
58e85375ca [fix](vectorized) fix float to string inaccurate (#8392) 2022-03-08 18:58:52 +08:00
40a5fac920 [fix](vectorized) Fix the datetime type read error and is_same set error in reader (#8386) 2022-03-08 18:58:02 +08:00
Pxl
cd8694e532 [feature][vectorized] support replace() (#8384) 2022-03-08 18:57:12 +08:00
Pxl
8214f32003 [fix] fix core dump on minmax_filter with decimal type (#8381) 2022-03-08 18:56:48 +08:00
9c7d51980b [improvement](regression-test) add aggregation tests from trino to doris (#8375) 2022-03-08 18:54:31 +08:00
22bafef875 [fix](broker-load) fix bug that a cancelled job's state is LOADING (#8363)
1.
Before executing LoadLoadingTask of a broker load, we should check if the job is cancelled.

2.
Add a new column `runningTransactionNum` for `show proc "/transactions"`.
So that we can view all running txns in each db in one command.
2022-03-08 18:53:45 +08:00
1e70f992e7 [improvement][fix](insert)(replay) support SHOW LAST INSERT stmt and fix json replay bug (#8355)
1. support SHOW LAST INSERT
    In the current implementation, the insert operation returns a json string to describe the result information
    of the insert. But this information is in the session track field of the mysql protocol,
    and it is difficult to obtain programmatically.

    Therefore, I provide a new syntax `show last insert` to explicitly obtain the result of the latest insert operation,
    and return a normal query result set to facilitate the user to obtain the result information of the insert.

2. the `ReturnRows` field in fe.audit.log of insert operation will be set to the loaded row num of the insert.

3.  Fix a bug described in #8354
2022-03-08 18:53:11 +08:00
b9010e34d0 [doc] Translate Chinese comment to English (#8340) 2022-03-08 18:52:48 +08:00
3eedd15f9c [optimize] optimze tablet read, avoid to create too much scanner for small tablet (#8096) 2022-03-08 13:59:45 +08:00
454b45bea3 [feature](vectorize)(function) support regexp&&sm4&&aes functions (#8307) 2022-03-08 13:14:02 +08:00
d711d64dda [fix](vectorization)Some small fix for SegmentIter Vectorization (#8267)
1. No longer using short-circuit to evaluate date type, because the cost of read date type is small,
    lazy materialization has higher costs.
2. Fix read hll/bitmap/date type error results.
2022-03-08 13:13:17 +08:00
089ef5384b [chore] fix build with parallel parameter only (#8352) 2022-03-07 23:19:19 +08:00
50a59f3f86 [license] Organize third-party dependent licenses for bianry releases (#8350) 2022-03-07 23:18:58 +08:00
ca3be24a86 [typo]fix some typo in fe_config (#8325) 2022-03-07 23:18:38 +08:00
c647def5bc [community] Modify doris connector release doc (#8275) 2022-03-07 23:15:38 +08:00
d9c2c2cac6 Revert "[refactor] remove unused new_in_predicate code (#8263)" (#8372)
This reverts commit 757e35744d4f6319e936fca84b4be13cf043a578.
2022-03-07 15:55:38 +08:00
477b87cb28 [feature](vec) Support update stmt in vec query engine (#8296) 2022-03-07 14:03:55 +08:00
3b159a9820 support doriswriter build in macos (#8330)
support doriswriter build in macos (#8330)
2022-03-07 09:53:16 +08:00
1879623521 [docs]update http port doc to be more intuitive (#8343)
*  update http port doc to be more intuitive (#8343)
2022-03-06 16:55:17 +08:00
2519f8b9a4 format fe config title , add link for tablet_rebalancer_type (#8346)
* format fe config title , add link for tablet_rebalancer_type (#8346)
2022-03-06 16:54:26 +08:00
ea4ad1a99c [doc] Add sync job fe configuration item description (#8349) 2022-03-06 13:49:59 +08:00
be4dd0c820 [fix][vectorized] Fix error cast to boolean (#8345) 2022-03-06 13:47:46 +08:00
baa3b14870 [fix] Use fmt::to_string replace memory buffer::data() (#8311) 2022-03-06 13:44:11 +08:00
910a140b93 [fix][vectorized] coredump about const compare in exchange node (#8302) 2022-03-05 15:28:23 +08:00
c18717df53 [fix](vectorized) Fix core dump of mutable block different of block (#8280) 2022-03-05 15:27:36 +08:00
22a0011403 [fix](planner) Convert format in RewriteFromUnixTimeRule (#8235)
SQL to reproduce:
```
SELECT * FROM table WHERE where FROM_UNIXTIME(d_datekey,'%Y-%m-%d %H:%i:%s') != '1970-08-20 00:11:43';

org.apache.doris.common.AnalysisException: errCode = 2, detailMessage = Unexpected exception: Illegal pattern character 'i'
        at org.apache.doris.qe.StmtExecutor.analyze(StmtExecutor.java:584) ~[palo-fe.jar:3.4.0]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:345) ~[palo-fe.jar:3.4.0]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:318) ~[palo-fe.jar:3.4.0]
        at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:221) ~[palo-fe.jar:3.4.0]
        at org.apache.doris.qe.ConnectProcessor.dispatch(ConnectProcessor.java:361) ~[palo-fe.jar:3.4.0]
        at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:562) ~[palo-fe.jar:3.4.0]
        at org.apache.doris.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:50) ~[palo-fe.jar:3.4.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
        at java.lang.Thread.run(Thread.java:835) [?:?]
```


Describe the overview of changes.
Just support:
yyyy-MM-dd HH:mm:ss
yyyy-MM-dd
yyyyMMdd
2022-03-05 15:26:37 +08:00
f57f02bbf2 [improvement] Support show tablets stmt (#7970)
change `show tablet from tbl` to `show tablets from tbl`
2022-03-05 15:25:57 +08:00
f52d479cbc [fix](ut) fix be ut fragment_mgr_test compile failed (#8344) 2022-03-05 14:43:20 +08:00
9becb5a2b7 [typo] translate the comments of schema_change.h (#8321)
translate the comments of be/src/olap/schema_change.h
2022-03-05 14:41:20 +08:00
9961b2c860 [refactor] Remove mysql-connector and replace org.json with com.googlecode.json-simple (#8319)
1. mysql-connector-java
    mysql-connector-java is under GLPv2 license, which is not compatible with APLv2, and Doris does not use it.

2. org.json
    org.json is under JSON license, which is not compatible with APLv2. I use `json-simple` to replace it.
2022-03-05 14:41:04 +08:00
9cf2798fb3 [typo] fix error for PushTask (#8316) 2022-03-05 14:40:32 +08:00
11c9c4f987 [improvement][website] The expansion of sidebar is off by default (#8314) 2022-03-05 14:40:11 +08:00
61455ec094 [docs] fix invalid links in docker-dev document (#8313) 2022-03-05 14:39:37 +08:00
777c9a4060 [typo](comment) Translate the code comments of gensrc (#8308) 2022-03-05 14:39:16 +08:00
450273cfc2 [typo]update spark build doc (#8333)
[typo]update spark build doc (#8333)
2022-03-05 12:30:56 +08:00
0383001442 [Enhancement] Support Skipping compaction lower replica where select queryable replica for better scan performance (#8146) 2022-03-05 09:51:50 +08:00
46ca23f216 [Feature] Support Changing the bucketing mode of the table from Hash Distribution to Random Distribution (#8259)
Co-authored-by: caiconghui1 <caiconghui1@jd.com>
2022-03-04 09:05:23 +08:00
Pxl
0ee53be883 [fix][improvement](runtime-filter) fix string type length limit error && add runtime filter decimal support (#8282) 2022-03-03 22:44:49 +08:00
538df28737 [improvement](routine-load) Support routine load task succeed with empty data consumed (#8256) 2022-03-03 22:35:50 +08:00
80e88159d9 [improvement](restore) allow query on part of partitions when others are in RESTORE (#8245) 2022-03-03 22:34:10 +08:00