Commit Graph

2764 Commits

Author SHA1 Message Date
e41b03e530 [Fix](multi-catalog) delete hdfs hedged configs at BE side. (#25094)
Issue Number: close #25093 

We can set hdfs hedged configs when creating catalog, just like this:
```
CREATE CATALOG `test_ctl` PROPERTIES (
...
"dfs.client.hedged.read.threadpool.size" = "128",
"dfs.client.hedged.read.threshold.millis" = "500",
...
);
```
It is redundant to set these configs at BE side, and it will brings an occasional bug at #25093 .
2023-10-11 23:25:30 +08:00
d1f59a4025 [fix](catalog)fix when modifying comments in property, it will modify the comments in the catalog (#24857)
- fix when modifying comments in property, it will modify the comments in the catalog
- add `alter catalog modify comment` to modify comment for catalog
- abstract some logic of `alter catalog` to parent class
2023-10-11 23:16:19 +08:00
73c3e3ab55 [Feature](x-load) support config min replica num for loading data (#21118) 2023-10-11 21:07:35 +08:00
df7724d6cb [typo](docs)delete wrong description of from_unixtime (#23897) 2023-10-11 03:20:13 -05:00
004d3264a6 [typo](docs) add 'order by' when use 'limit m,n' (#24236) 2023-10-11 03:15:33 -05:00
62a6b132be [Fix](func numbers) Remove backend_nums argument of numbers function (#25200) 2023-10-10 20:25:58 +08:00
643f7cad0e [typo](docs) Delete wrong schema change memory parameters (#25234) 2023-10-10 04:49:40 -05:00
d702bc3c13 [typo](doc) hot and cold stratification increases FAQ (#24974) 2023-10-10 17:38:43 +08:00
3a29bb4bc5 [fix](doc) spelling error for colocate join #25053 (#25202)
Issue: 25053

Translation text not cleaned up
2023-10-10 10:10:55 +08:00
aa1704c50b [doc](data-model) update data-model doc (#24941) 2023-10-08 21:08:16 -05:00
f41b6a5fc3 [minor](doc) update the doc for docker env and custom_lib dir (#25088)
1. Update the doc for `apache/doris:build-env-for-2.0`
2. Update the doc for `custom_dir`
2023-10-09 09:50:31 +08:00
7af4be1ee3 [fix](mysqldb) Fix mysqldb upgrade (#25111)
If user has database with same name mysql, will introduce problem when doing checkpoint.

Solution:

Add check for this situation, if duplicate, exit and print log info to prevent damage of metadata;
Add fe config field: mysqldb_replace_name to make things correct if user already has mysql db.
Related pr: #23087 #22868
2023-10-09 09:40:56 +08:00
541f48a754 [feature](es-catalog) add include_hidden_indexin order to get the hidden index. (#24826) 2023-10-08 14:35:08 +08:00
934e9d5617 [typo](docs) Add example for create sql block rule (#24754) 2023-10-08 01:18:11 -05:00
961ca76bd3 [doc](fix)fix doc misspell (#25072) 2023-10-08 10:24:59 +08:00
f8e4cefb8c [typo](doc)Add be's enable_java_support configuration document (#25069) 2023-10-07 23:56:14 +08:00
f3e95608cb (Fix)(RoutineLoad)Query the transaction status NPE when the task has not yet started scheduling (#25074) 2023-10-07 07:26:49 -05:00
cb03703990 [fix](doc) spelling error for colocate join #25053 (#25054)
Issue: 25053

Change spell error for Colocate Join.
2023-10-07 19:51:07 +08:00
5130a6c006 [improvement](jdbc catalog)Adjustment to JDBC External Table Configuration Based on Internal Table Settings (#25059)
This pull request addresses the behavior of the `lower_case_table_names` parameter for jdbc catalog's based on the configuration of the internal table's corresponding parameter.

Changes:
- For internal tables, if `lower_case_table_names` is set to 1 or 2, thejdbc catalog's parameter is forcefully set to `true`.
- For internal tables, if `lower_case_table_names` is set to 0, the jdbc catalog's parameter can be either `true` or `false` with a default value of `false`.

These adjustments ensure consistency and predictability when working with both internal and external table configurations in Doris.
2023-10-07 06:25:52 -05:00
xy
94eec9be0f [Monitor](doc)modify incorrect name for Cumulative Compaction Score (#25082)
Co-authored-by: xingying01 <xingying01@corp.netease.com>
2023-10-07 18:53:13 +08:00
xy
20a7df6ecc [monitor][doc]modify incorrect status for doris_be_engine_requests_total (#25075)
Co-authored-by: xingying01 <xingying01@corp.netease.com>
2023-10-07 16:51:52 +08:00
5ffb01a068 [doc](doc)Update doc: delete Z-order (#24955) 2023-09-28 13:45:48 +08:00
83f5ff7b22 [typo](doc)modify error result of explode_split function. (#24185) 2023-09-27 03:04:18 -05:00
41dfbdac14 [typo](doc)modify the wrong parameter in the example. (#24307) 2023-09-27 03:04:07 -05:00
53025ce3fc [typo](doc)modify error description of GROUP_CONCAT fiction. (#24619) 2023-09-27 03:03:56 -05:00
a6d1da0db9 [typo](doc)modify error link description in fe-config-template (#24676) 2023-09-27 02:51:47 -05:00
452318a9fc [Enhancement](streamload) stream tvf support user specified label (#24219)
stream tvf support user specified label
example:

curl -v --location-trusted -u root: -H "sql: insert into test.t1 WITH LABEL label1 select c1,c2 from http_stream(\"format\" = \"CSV\", \"column_separator\" = \",\")" -T example.csv http://127.0.0.1:8030/api/_http_stream
return:

{
    "TxnId": 2064,
    "Label": "label1",
    "Comment": "",
    "TwoPhaseCommit": "false",
    "Status": "Success",
    "Message": "OK",
    "NumberTotalRows": 2,
    "NumberLoadedRows": 2,
    "NumberFilteredRows": 0,
    "NumberUnselectedRows": 0,
    "LoadBytes": 27,
    "LoadTimeMs": 152,
    "BeginTxnTimeMs": 0,
    "StreamLoadPutTimeMs": 83,
    "ReadDataTimeMs": 92,
    "WriteDataTimeMs": 41,
    "CommitAndPublishTimeMs": 24
}
2023-09-27 12:09:35 +08:00
6b64b7fec7 [typo](doc)Add flink to read the doris table and use doris.filter.query to configure the display (#24736) 2023-09-27 11:22:58 +08:00
f59e69b852 [feature](autobucekt) Add support autobucket min buckets config Add support autobucket (#24920)
Signed-off-by: Jack Drogon <jack.xsuperman@gmail.com>
2023-09-27 08:13:53 +08:00
e9eaf49db5 [feature](alter colocate group) Support alter colocate group replica allocation (#23320) 2023-09-26 19:55:25 +08:00
6cef918a71 [docs](doc)Add FAQ for "trustAnchors parameter must be non-empty" (#24848) 2023-09-26 19:23:23 +08:00
43433c3fab [improvement](processlist) Add more information for show processlist (#24000) 2023-09-26 14:25:42 +08:00
4c9ffd210d [typo][doc]modify incorrect argument type. (#24095) 2023-09-26 00:25:39 +08:00
8d4fd76a16 [Feature](StreamLoad2PC) Support commit and abort streamload2PC by label (#24613) 2023-09-25 22:21:27 +08:00
59c494eb27 [fix](Docs) Modify documents about SELECT INTO OUTFILE and EXPORT (#24641) 2023-09-25 19:51:15 +08:00
8679095e5c [feature](debug) support debug point used in debug code (#24502) 2023-09-25 17:56:12 +08:00
xfz
1b95ce1d93 [feature](json-function) add json_insert, json_replace, json_set functions (#24384)
[feature](json-function) add three json funcitons
2023-09-25 12:52:29 +08:00
2d0ac7117a [enhancement](show-frontends-disks) Add disk info corresponding to deploy dir in frontend disk info (#24726) 2023-09-25 11:14:03 +08:00
312576b93c [typo](docs) modify flink connector document error (#23967) 2023-09-25 10:11:33 +08:00
129ffb7dd7 [docs](docs)fix display error #24775 2023-09-25 08:38:51 +08:00
bacc310502 [Docs](StartUp)Add StartUp methods docs and metadata recovery docs (#24816) 2023-09-23 18:45:57 +08:00
c943a05065 [fix](stats) Fix data size calculation of auto sample (#24672)
1. Fix data size calculation of auto sample, before this pr, the data size is include all the replicas
2. Move some auto analyze related options to global session variable
3. Add some logs
2023-09-22 18:12:39 +08:00
22616d125d [function](bitmap) add function alias bitmap_andnot and bitmap_andnot_count (#24771) 2023-09-22 12:18:31 +08:00
cec3fcdf78 [typo](doc) modifying old parameters (#24565)
* modifying old parameters

* Supplementary demo
2023-09-22 09:09:06 +08:00
269b9eaf8b [typo](docs)add FAQ for broker load error: Cannot locate default realm (#24693) 2023-09-22 09:06:36 +08:00
58ab25ccaa Revert "[Feature](merge-on-write)Support ignore mode for merge-on-write unique table (#21773)" (#24731)
This reverts commit 3ee89aea35726197cb7e94bb4f2c36bc9d50da84.
2023-09-21 21:01:28 +08:00
5e82aebd25 [fix](fe-meta) fix paimon file catalog meta issue and replication num analysis issue (#24681)
Fix 2 issues:

The replication num analysis will throw exception when there is no BE alive when FE restart.
org.apache.doris.common.AnalysisException: errCode = 2, detailMessage = errCode = 2, detailMessage = replication num should be less than the number of available backends. replication num is 1, available backend num is 0
        at org.apache.doris.common.util.PropertyAnalyzer.analyzeReplicaAllocation(PropertyAnalyzer.java:1018) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.TableProperty.buildReplicaAllocation(TableProperty.java:483) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.TableProperty.read(TableProperty.java:529) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.OlapTable.readFields(OlapTable.java:1370) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Table.read(Table.java:383) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Database.readFields(Database.java:631) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.datasource.InternalCatalog.loadDb(InternalCatalog.java:3075) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env.loadDb(Env.java:1764) ~[doris-fe.jar:1.2-SNAPSHOT]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_341]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_341]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_341]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_341]
        at org.apache.doris.persist.meta.MetaReader.read(MetaReader.java:116) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env.loadImage(Env.java:1701) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.master.Checkpoint.doCheckpoint(Checkpoint.java:130) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.master.Checkpoint.runAfterCatalogReady(Checkpoint.java:79) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.common.util.MasterDaemon.runOneCycle(MasterDaemon.java:58) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.common.util.Daemon.run(Daemon.java:116) ~[doris-fe.jar:1.2-SNAPSHOT]
Paimon File Catalog meta issue
Fix #23366
2023-09-21 08:41:03 +08:00
2e85e0163d Revert "[feature](function) add json->operator convert to json_extract (#19899)" (#24679)
Revert "[feature](function) add json->operator convert to json_extract (#19899)"
because it conflict with lambda syntax
This reverts commit f54a068d82e88e8535f3ed55a4224886b752e46b.
2023-09-20 21:16:19 +08:00
e4b551e2ce [fix](Config): Remove unused config max_connection_scheduler_threads_num (#24597) 2023-09-20 18:11:56 +08:00
2a260be10c [improvement](jdbc catalog) when lower_case_table_names of jdbc catalog properties is set to true, use the real table name to query the jdbc data source (#24520) 2023-09-20 17:47:11 +08:00