Commit Graph

4051 Commits

Author SHA1 Message Date
e403dbc38c [feature](user-property) Support user level exec_mem_limit and load_mem_limit (#8365)
```
SET PROPERTY FOR 'jack' 'exec_mem_limit' = '2147483648';
SET PROPERTY FOR 'jack' 'load_mem_limit' = '2147483648';
```
The user level property will overwrite the value in session variables.
2022-03-11 17:20:09 +08:00
68dd799796 [improvement](vectorized) Support function tuple is null (#8442) 2022-03-11 16:54:37 +08:00
37c201adea [doc] fix metadata operation links to external docs (#8403) 2022-03-11 09:34:10 +08:00
7cfcddd8df [fix] brpc will check required field in proto and need_gen_rollup is moved will throw exception (#8420) 2022-03-11 00:28:33 +08:00
Pxl
bc923b8c63 [fix](vectorized) core dump on runtime filter insert because of block address change (#8429) 2022-03-10 19:02:19 +08:00
Pxl
d3d8301a13 [feature](function) support vectorized digital_masking (#8409) 2022-03-10 09:07:07 +08:00
548fde60e5 [doc] Fix dead link in build docker environment in developer guide (#8379) 2022-03-10 09:06:39 +08:00
8c8416026d [doc](vectorized) Add vectorized execution engine docs (#8358) 2022-03-10 09:06:08 +08:00
8eec4bf99d [feature](thread-local) Add thread local variable ThreadContext (#7234)
The thread context saves some info about a working thread.
1. thread_id:   Current thread id, Auto generated.
2. type:           The type is a enum value indicating which type of task current thread is running.
                        For example: QUERY, LOAD, COMPACTION, ...
3. task id:       A unique id to identify this task. maybe query id, load job id, etc.

Using gcc11 compiles thread_local variable on lower versions of GLIBC will report an error, see https://github.com/apache/incubator-doris/pull/7911
This is very difficult to solve, so kudu Class-scoped static thread local implementation was introduced.
Solve the above problem by Thread-scopedthread local + Class-scoped thread local.
See the comments for ThreadContextPtr for details.
2022-03-10 09:05:40 +08:00
2a433136bc [fix] fix compile error (#8410)
* [fix] fix compile error
2022-03-09 18:55:37 +08:00
f4663ad2eb [improvement](vectorized) Merge block in scanner to speed up query with conjunct (#8395) 2022-03-09 13:11:18 +08:00
51103dcf6e [typo] translate the comments of delete_handler.cpp (#8402) 2022-03-09 13:08:28 +08:00
97020bcf80 [doc] update substring.md (#8398)
correct a slip of pen.
2022-03-09 13:07:32 +08:00
25532c65d1 [doc] Update BROKER LOAD.md (#8361) 2022-03-09 13:06:24 +08:00
d880559214 [refactor] remove old schema change code on BE (#8342) 2022-03-09 13:05:44 +08:00
3d30209e9b [docs] add document conditional-functions (#8339)
Add conditional-functions.
2022-03-09 13:05:13 +08:00
0ff7de4157 [refactor] remove agent status (#8273)
There are 3 error code types in BE: OLAPStatus AgentStatus Status.
It is very confused and sometimes conflict during write code.
I will try to unify them to Status.
2022-03-09 13:04:50 +08:00
Pxl
10c3712aa1 [fix](vectorized) fix arithmetic calculate get wrong result(#8226) 2022-03-09 13:03:57 +08:00
826467e116 [fix](replica) handle replica version missing info to avoid -214 error (#8209)
In the original tablet reporting information, the version missing information is done by combining
two pieces of information as follows:

1. the maximum consecutive version number
2. the `version_miss` field

The logic of this approach is confusing and inconsistent with the logic of checking for missing versions when querying.

After the change, we directly use the version checking logic used in the query, and set `version_miss` to true
if a missing version is found

and on the FE processing side. Originally, only the **bad replica** information was syncronized among FEs,
but not the **version missing** information. As a result, the non-master FE is not aware of the missing version information.

In the new design, we deprecate the original log persistence class `BackendTabletsInfo` and use the new 
`BackendReplicasInfo` to record replica reporting information and write both **bad** and **version missing**
information to metadata so that other FEs can synchronize these information.
2022-03-09 13:03:22 +08:00
b536c1850f [fix] fix wrong examples in week.md (#8397)
fix wrong examples in week docment
2022-03-08 21:26:43 +08:00
Pxl
58e85375ca [fix](vectorized) fix float to string inaccurate (#8392) 2022-03-08 18:58:52 +08:00
40a5fac920 [fix](vectorized) Fix the datetime type read error and is_same set error in reader (#8386) 2022-03-08 18:58:02 +08:00
Pxl
cd8694e532 [feature][vectorized] support replace() (#8384) 2022-03-08 18:57:12 +08:00
Pxl
8214f32003 [fix] fix core dump on minmax_filter with decimal type (#8381) 2022-03-08 18:56:48 +08:00
9c7d51980b [improvement](regression-test) add aggregation tests from trino to doris (#8375) 2022-03-08 18:54:31 +08:00
22bafef875 [fix](broker-load) fix bug that a cancelled job's state is LOADING (#8363)
1.
Before executing LoadLoadingTask of a broker load, we should check if the job is cancelled.

2.
Add a new column `runningTransactionNum` for `show proc "/transactions"`.
So that we can view all running txns in each db in one command.
2022-03-08 18:53:45 +08:00
1e70f992e7 [improvement][fix](insert)(replay) support SHOW LAST INSERT stmt and fix json replay bug (#8355)
1. support SHOW LAST INSERT
    In the current implementation, the insert operation returns a json string to describe the result information
    of the insert. But this information is in the session track field of the mysql protocol,
    and it is difficult to obtain programmatically.

    Therefore, I provide a new syntax `show last insert` to explicitly obtain the result of the latest insert operation,
    and return a normal query result set to facilitate the user to obtain the result information of the insert.

2. the `ReturnRows` field in fe.audit.log of insert operation will be set to the loaded row num of the insert.

3.  Fix a bug described in #8354
2022-03-08 18:53:11 +08:00
b9010e34d0 [doc] Translate Chinese comment to English (#8340) 2022-03-08 18:52:48 +08:00
3eedd15f9c [optimize] optimze tablet read, avoid to create too much scanner for small tablet (#8096) 2022-03-08 13:59:45 +08:00
454b45bea3 [feature](vectorize)(function) support regexp&&sm4&&aes functions (#8307) 2022-03-08 13:14:02 +08:00
d711d64dda [fix](vectorization)Some small fix for SegmentIter Vectorization (#8267)
1. No longer using short-circuit to evaluate date type, because the cost of read date type is small,
    lazy materialization has higher costs.
2. Fix read hll/bitmap/date type error results.
2022-03-08 13:13:17 +08:00
089ef5384b [chore] fix build with parallel parameter only (#8352) 2022-03-07 23:19:19 +08:00
50a59f3f86 [license] Organize third-party dependent licenses for bianry releases (#8350) 2022-03-07 23:18:58 +08:00
ca3be24a86 [typo]fix some typo in fe_config (#8325) 2022-03-07 23:18:38 +08:00
c647def5bc [community] Modify doris connector release doc (#8275) 2022-03-07 23:15:38 +08:00
d9c2c2cac6 Revert "[refactor] remove unused new_in_predicate code (#8263)" (#8372)
This reverts commit 757e35744d4f6319e936fca84b4be13cf043a578.
2022-03-07 15:55:38 +08:00
477b87cb28 [feature](vec) Support update stmt in vec query engine (#8296) 2022-03-07 14:03:55 +08:00
3b159a9820 support doriswriter build in macos (#8330)
support doriswriter build in macos (#8330)
2022-03-07 09:53:16 +08:00
1879623521 [docs]update http port doc to be more intuitive (#8343)
*  update http port doc to be more intuitive (#8343)
2022-03-06 16:55:17 +08:00
2519f8b9a4 format fe config title , add link for tablet_rebalancer_type (#8346)
* format fe config title , add link for tablet_rebalancer_type (#8346)
2022-03-06 16:54:26 +08:00
ea4ad1a99c [doc] Add sync job fe configuration item description (#8349) 2022-03-06 13:49:59 +08:00
be4dd0c820 [fix][vectorized] Fix error cast to boolean (#8345) 2022-03-06 13:47:46 +08:00
baa3b14870 [fix] Use fmt::to_string replace memory buffer::data() (#8311) 2022-03-06 13:44:11 +08:00
910a140b93 [fix][vectorized] coredump about const compare in exchange node (#8302) 2022-03-05 15:28:23 +08:00
c18717df53 [fix](vectorized) Fix core dump of mutable block different of block (#8280) 2022-03-05 15:27:36 +08:00
22a0011403 [fix](planner) Convert format in RewriteFromUnixTimeRule (#8235)
SQL to reproduce:
```
SELECT * FROM table WHERE where FROM_UNIXTIME(d_datekey,'%Y-%m-%d %H:%i:%s') != '1970-08-20 00:11:43';

org.apache.doris.common.AnalysisException: errCode = 2, detailMessage = Unexpected exception: Illegal pattern character 'i'
        at org.apache.doris.qe.StmtExecutor.analyze(StmtExecutor.java:584) ~[palo-fe.jar:3.4.0]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:345) ~[palo-fe.jar:3.4.0]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:318) ~[palo-fe.jar:3.4.0]
        at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:221) ~[palo-fe.jar:3.4.0]
        at org.apache.doris.qe.ConnectProcessor.dispatch(ConnectProcessor.java:361) ~[palo-fe.jar:3.4.0]
        at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:562) ~[palo-fe.jar:3.4.0]
        at org.apache.doris.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:50) ~[palo-fe.jar:3.4.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
        at java.lang.Thread.run(Thread.java:835) [?:?]
```


Describe the overview of changes.
Just support:
yyyy-MM-dd HH:mm:ss
yyyy-MM-dd
yyyyMMdd
2022-03-05 15:26:37 +08:00
f57f02bbf2 [improvement] Support show tablets stmt (#7970)
change `show tablet from tbl` to `show tablets from tbl`
2022-03-05 15:25:57 +08:00
f52d479cbc [fix](ut) fix be ut fragment_mgr_test compile failed (#8344) 2022-03-05 14:43:20 +08:00
9becb5a2b7 [typo] translate the comments of schema_change.h (#8321)
translate the comments of be/src/olap/schema_change.h
2022-03-05 14:41:20 +08:00
9961b2c860 [refactor] Remove mysql-connector and replace org.json with com.googlecode.json-simple (#8319)
1. mysql-connector-java
    mysql-connector-java is under GLPv2 license, which is not compatible with APLv2, and Doris does not use it.

2. org.json
    org.json is under JSON license, which is not compatible with APLv2. I use `json-simple` to replace it.
2022-03-05 14:41:04 +08:00