Commit Graph

967 Commits

Author SHA1 Message Date
e403dbc38c [feature](user-property) Support user level exec_mem_limit and load_mem_limit (#8365)
```
SET PROPERTY FOR 'jack' 'exec_mem_limit' = '2147483648';
SET PROPERTY FOR 'jack' 'load_mem_limit' = '2147483648';
```
The user level property will overwrite the value in session variables.
2022-03-11 17:20:09 +08:00
37c201adea [doc] fix metadata operation links to external docs (#8403) 2022-03-11 09:34:10 +08:00
548fde60e5 [doc] Fix dead link in build docker environment in developer guide (#8379) 2022-03-10 09:06:39 +08:00
8c8416026d [doc](vectorized) Add vectorized execution engine docs (#8358) 2022-03-10 09:06:08 +08:00
97020bcf80 [doc] update substring.md (#8398)
correct a slip of pen.
2022-03-09 13:07:32 +08:00
25532c65d1 [doc] Update BROKER LOAD.md (#8361) 2022-03-09 13:06:24 +08:00
3d30209e9b [docs] add document conditional-functions (#8339)
Add conditional-functions.
2022-03-09 13:05:13 +08:00
b536c1850f [fix] fix wrong examples in week.md (#8397)
fix wrong examples in week docment
2022-03-08 21:26:43 +08:00
1e70f992e7 [improvement][fix](insert)(replay) support SHOW LAST INSERT stmt and fix json replay bug (#8355)
1. support SHOW LAST INSERT
    In the current implementation, the insert operation returns a json string to describe the result information
    of the insert. But this information is in the session track field of the mysql protocol,
    and it is difficult to obtain programmatically.

    Therefore, I provide a new syntax `show last insert` to explicitly obtain the result of the latest insert operation,
    and return a normal query result set to facilitate the user to obtain the result information of the insert.

2. the `ReturnRows` field in fe.audit.log of insert operation will be set to the loaded row num of the insert.

3.  Fix a bug described in #8354
2022-03-08 18:53:11 +08:00
3eedd15f9c [optimize] optimze tablet read, avoid to create too much scanner for small tablet (#8096) 2022-03-08 13:59:45 +08:00
ca3be24a86 [typo]fix some typo in fe_config (#8325) 2022-03-07 23:18:38 +08:00
c647def5bc [community] Modify doris connector release doc (#8275) 2022-03-07 23:15:38 +08:00
1879623521 [docs]update http port doc to be more intuitive (#8343)
*  update http port doc to be more intuitive (#8343)
2022-03-06 16:55:17 +08:00
2519f8b9a4 format fe config title , add link for tablet_rebalancer_type (#8346)
* format fe config title , add link for tablet_rebalancer_type (#8346)
2022-03-06 16:54:26 +08:00
ea4ad1a99c [doc] Add sync job fe configuration item description (#8349) 2022-03-06 13:49:59 +08:00
22a0011403 [fix](planner) Convert format in RewriteFromUnixTimeRule (#8235)
SQL to reproduce:
```
SELECT * FROM table WHERE where FROM_UNIXTIME(d_datekey,'%Y-%m-%d %H:%i:%s') != '1970-08-20 00:11:43';

org.apache.doris.common.AnalysisException: errCode = 2, detailMessage = Unexpected exception: Illegal pattern character 'i'
        at org.apache.doris.qe.StmtExecutor.analyze(StmtExecutor.java:584) ~[palo-fe.jar:3.4.0]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:345) ~[palo-fe.jar:3.4.0]
        at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:318) ~[palo-fe.jar:3.4.0]
        at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:221) ~[palo-fe.jar:3.4.0]
        at org.apache.doris.qe.ConnectProcessor.dispatch(ConnectProcessor.java:361) ~[palo-fe.jar:3.4.0]
        at org.apache.doris.qe.ConnectProcessor.processOnce(ConnectProcessor.java:562) ~[palo-fe.jar:3.4.0]
        at org.apache.doris.mysql.nio.ReadListener.lambda$handleEvent$0(ReadListener.java:50) ~[palo-fe.jar:3.4.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
        at java.lang.Thread.run(Thread.java:835) [?:?]
```


Describe the overview of changes.
Just support:
yyyy-MM-dd HH:mm:ss
yyyy-MM-dd
yyyyMMdd
2022-03-05 15:26:37 +08:00
f57f02bbf2 [improvement] Support show tablets stmt (#7970)
change `show tablet from tbl` to `show tablets from tbl`
2022-03-05 15:25:57 +08:00
11c9c4f987 [improvement][website] The expansion of sidebar is off by default (#8314) 2022-03-05 14:40:11 +08:00
61455ec094 [docs] fix invalid links in docker-dev document (#8313) 2022-03-05 14:39:37 +08:00
450273cfc2 [typo]update spark build doc (#8333)
[typo]update spark build doc (#8333)
2022-03-05 12:30:56 +08:00
0383001442 [Enhancement] Support Skipping compaction lower replica where select queryable replica for better scan performance (#8146) 2022-03-05 09:51:50 +08:00
46ca23f216 [Feature] Support Changing the bucketing mode of the table from Hash Distribution to Random Distribution (#8259)
Co-authored-by: caiconghui1 <caiconghui1@jd.com>
2022-03-04 09:05:23 +08:00
09bfb8b9d3 [fix] (rpc-udf) Fixed the problem that the query could not be interrupted (#8248)
if an error occurred in the rpc server during the execution of rpc-udf.
Add java,cpp,python demo of rpc-udf server
2022-03-03 09:30:03 +08:00
efe88be087 [docs] Fix a typo in downloads.md (#8274) 2022-03-02 10:18:18 +08:00
d5b6428c6d [improvement] Upgrade MySQL version to 5.7.37 to reduce unnecessary CVE issues (#8247) 2022-03-02 10:16:02 +08:00
236105daa0 [feature][show-transaction] Support view transactions info for specified status by SHOW TRANSACTION stmt (#8156)
SHOW TRANSACTION WHERE STATUS = 'prepare/precommitted/committed/visible/aborted';
2022-03-02 10:14:42 +08:00
5cc62fa166 [docs] fix document date-time-functions typo (#8053)
1. en: add convert_tz.md
2. en: fix curdate.md, the fix is to modify the title
3. en: add curtime.md
4. en: fix str_to_date.md, the fix is to modify the title
5. zh-CN: fix convert_tz.md, the fix is to modify the description
6. zh-CN: fix curdate.md, the fix is to modify the title
2022-03-02 10:07:44 +08:00
2b9b0fc1ec [Fix] Function percentile input null return null (#8238) 2022-03-01 14:42:48 +08:00
13f007bfd5 [doc] Modify document of compilation on ARM64 (#8254) 2022-03-01 11:05:18 +08:00
30fbf1da68 [typo](doc)fix some confusing doc content (#8239) 2022-02-26 17:05:08 +08:00
83521a826a [Feature](create_table) Support create table with random distribution to avoid data skew (#8041)
In some scenarios, users cannot find a suitable hash key to avoid data skew, so we need to provide an additional data distribution for olap table to avoid data skew

example:
CREATE TABLE random_table
(
siteid INT DEFAULT '10',
citycode SMALLINT,
username VARCHAR(32) DEFAULT '',
pv BIGINT SUM DEFAULT '0'
)
AGGREGATE KEY(siteid, citycode, username)
DISTRIBUTED BY random BUCKETS 10
PROPERTIES("replication_num" = "1");

Co-authored-by: caiconghui1 <caiconghui1@jd.com>
2022-02-26 10:38:55 +08:00
a630e037b9 [Enhancement](routine_load) Support show routine load statement with like predicate (#8188)
* [Enhancement](routine_load) Support show routine load with like predicate

Co-authored-by: caiconghui1 <caiconghui1@jd.com>
2022-02-26 10:35:38 +08:00
Pxl
4c5d7c27df [Bug] group_concat(value,null) not return null 2022-02-25 11:03:23 +08:00
f4c6de554b [docs] update backup and restore docs (#8161)
Co-authored-by: 段绪建 <duanxujian@jd.com>
2022-02-23 11:41:14 +08:00
c3348b8023 [docs] fix config enable_force_drop_redundant_replica name not correct in docs (#8154) 2022-02-21 09:40:21 +08:00
16bf60e0da [docs] Update data-model-rollup.md (#8126) 2022-02-21 09:36:31 +08:00
826738d97f [docs]Some doc improvements and typo fix (#8153) 2022-02-21 09:36:01 +08:00
b27122df8b [Docs] add rpc function document (#7984)
* add rpc function document
2022-02-20 23:26:52 +08:00
13d217b947 [community](doc) Refactor the release and verify doc (#8136)
Because of the latest moving of some code to a new repository, the documentation for release and verification
needs to be reorganized. There are 5 relevant documents as follows.

1. release-prepare.md
    General instructions for the release and related preparation work.
2. release-doris-core.md
    The Doris Core release process
3. release-doris-connectors
    The Doris Connectors release process
4. release-complete.md
   Release completion after release polling passed.
5. release-verify.md
    Release verification methods.
2022-02-20 16:31:26 +08:00
02531cc536 [doc]fix start/stop command (#8150) 2022-02-20 12:03:47 +08:00
694e6433cc [feature](test) Add regression test framework (#8125)
Add scalable regression testing framework(#7584)
contains
- Test framework making by groovy, and support built-in **readable DSL** named as `Action`
- Demo exists in `${DORIS_HOME}/regression-test/data/demo`
- Chinese doc exist in `${DORIS_HOME}/docs/zh-CN/developer-guide/regression-testing.md`

English document coming soon
2022-02-19 12:05:50 +08:00
Pxl
1943a5da5f [docs] fix some document error and adjust some function document location
1. move `group_concat` from string-functions to aggregate-functions.
2. add `json_array`/`json_object`/`json_quote` to sidebar file.
3. move  `json_array`/`json_object`/`json_quote`/`get_json_double`/`get_json_int`/`get_json_string` to json-functions.
4.  change `group_concat` document to uppercase
2022-02-19 11:59:55 +08:00
2b1aec871d [docs] Modify the SSB document (#8101)
Add one step of preparation before load
2022-02-18 11:14:20 +08:00
4a16b6611b Update commit-format-specification.md (#8121)
I want to add some key words for commit format
2022-02-18 11:05:09 +08:00
7471873e6f Add version upgrade instructions (#8057)
Add version upgrade instructions
2022-02-17 18:06:41 +08:00
8939b743e5 Update sql-block.md (#8106)
Documentation error
2022-02-17 18:06:03 +08:00
289aacb78c [improvement] enable check_java_version (#8034)
Enable to check the Java version when Doris starts, to prevent the user experience caused by the inconsistency 
between the compiled version and the running version.
If the Java version is compiled and the Java version is run, it will not start, and a prompt message will be given.
2022-02-17 11:16:45 +08:00
486a0586ac [community] modify the doc of verifying apache release (#8084) 2022-02-17 10:53:31 +08:00
GRX
d9535c29f6 [doc] update alter table docs (#8076) 2022-02-17 10:52:59 +08:00
26289c28b0 [fix](load)(compaction) Fix NodeChannel coredump bug and modify some compaction logic (#8072)
1. Fix the problem of BE crash caused by destruct sequence. (close #8058)
2. Add a new BE config `compaction_task_num_per_fast_disk`

    This config specify the max concurrent compaction task num on fast disk(typically .SSD).
    So that for high speed disk, we can execute more compaction task at same time,
    to compact the data as soon as possible

3. Avoid frequent selection of unqualified tablet to perform compaction.
4. Modify some log level to reduce the log size of BE.
5. Modify some clone logic to handle error correctly.
2022-02-17 10:52:08 +08:00