Commit Graph

2739 Commits

Author SHA1 Message Date
a6d1da0db9 [typo](doc)modify error link description in fe-config-template (#24676) 2023-09-27 02:51:47 -05:00
452318a9fc [Enhancement](streamload) stream tvf support user specified label (#24219)
stream tvf support user specified label
example:

curl -v --location-trusted -u root: -H "sql: insert into test.t1 WITH LABEL label1 select c1,c2 from http_stream(\"format\" = \"CSV\", \"column_separator\" = \",\")" -T example.csv http://127.0.0.1:8030/api/_http_stream
return:

{
    "TxnId": 2064,
    "Label": "label1",
    "Comment": "",
    "TwoPhaseCommit": "false",
    "Status": "Success",
    "Message": "OK",
    "NumberTotalRows": 2,
    "NumberLoadedRows": 2,
    "NumberFilteredRows": 0,
    "NumberUnselectedRows": 0,
    "LoadBytes": 27,
    "LoadTimeMs": 152,
    "BeginTxnTimeMs": 0,
    "StreamLoadPutTimeMs": 83,
    "ReadDataTimeMs": 92,
    "WriteDataTimeMs": 41,
    "CommitAndPublishTimeMs": 24
}
2023-09-27 12:09:35 +08:00
6b64b7fec7 [typo](doc)Add flink to read the doris table and use doris.filter.query to configure the display (#24736) 2023-09-27 11:22:58 +08:00
f59e69b852 [feature](autobucekt) Add support autobucket min buckets config Add support autobucket (#24920)
Signed-off-by: Jack Drogon <jack.xsuperman@gmail.com>
2023-09-27 08:13:53 +08:00
e9eaf49db5 [feature](alter colocate group) Support alter colocate group replica allocation (#23320) 2023-09-26 19:55:25 +08:00
6cef918a71 [docs](doc)Add FAQ for "trustAnchors parameter must be non-empty" (#24848) 2023-09-26 19:23:23 +08:00
43433c3fab [improvement](processlist) Add more information for show processlist (#24000) 2023-09-26 14:25:42 +08:00
4c9ffd210d [typo][doc]modify incorrect argument type. (#24095) 2023-09-26 00:25:39 +08:00
8d4fd76a16 [Feature](StreamLoad2PC) Support commit and abort streamload2PC by label (#24613) 2023-09-25 22:21:27 +08:00
59c494eb27 [fix](Docs) Modify documents about SELECT INTO OUTFILE and EXPORT (#24641) 2023-09-25 19:51:15 +08:00
8679095e5c [feature](debug) support debug point used in debug code (#24502) 2023-09-25 17:56:12 +08:00
xfz
1b95ce1d93 [feature](json-function) add json_insert, json_replace, json_set functions (#24384)
[feature](json-function) add three json funcitons
2023-09-25 12:52:29 +08:00
2d0ac7117a [enhancement](show-frontends-disks) Add disk info corresponding to deploy dir in frontend disk info (#24726) 2023-09-25 11:14:03 +08:00
312576b93c [typo](docs) modify flink connector document error (#23967) 2023-09-25 10:11:33 +08:00
129ffb7dd7 [docs](docs)fix display error #24775 2023-09-25 08:38:51 +08:00
bacc310502 [Docs](StartUp)Add StartUp methods docs and metadata recovery docs (#24816) 2023-09-23 18:45:57 +08:00
c943a05065 [fix](stats) Fix data size calculation of auto sample (#24672)
1. Fix data size calculation of auto sample, before this pr, the data size is include all the replicas
2. Move some auto analyze related options to global session variable
3. Add some logs
2023-09-22 18:12:39 +08:00
22616d125d [function](bitmap) add function alias bitmap_andnot and bitmap_andnot_count (#24771) 2023-09-22 12:18:31 +08:00
cec3fcdf78 [typo](doc) modifying old parameters (#24565)
* modifying old parameters

* Supplementary demo
2023-09-22 09:09:06 +08:00
269b9eaf8b [typo](docs)add FAQ for broker load error: Cannot locate default realm (#24693) 2023-09-22 09:06:36 +08:00
58ab25ccaa Revert "[Feature](merge-on-write)Support ignore mode for merge-on-write unique table (#21773)" (#24731)
This reverts commit 3ee89aea35726197cb7e94bb4f2c36bc9d50da84.
2023-09-21 21:01:28 +08:00
5e82aebd25 [fix](fe-meta) fix paimon file catalog meta issue and replication num analysis issue (#24681)
Fix 2 issues:

The replication num analysis will throw exception when there is no BE alive when FE restart.
org.apache.doris.common.AnalysisException: errCode = 2, detailMessage = errCode = 2, detailMessage = replication num should be less than the number of available backends. replication num is 1, available backend num is 0
        at org.apache.doris.common.util.PropertyAnalyzer.analyzeReplicaAllocation(PropertyAnalyzer.java:1018) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.TableProperty.buildReplicaAllocation(TableProperty.java:483) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.TableProperty.read(TableProperty.java:529) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.OlapTable.readFields(OlapTable.java:1370) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Table.read(Table.java:383) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Database.readFields(Database.java:631) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.datasource.InternalCatalog.loadDb(InternalCatalog.java:3075) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env.loadDb(Env.java:1764) ~[doris-fe.jar:1.2-SNAPSHOT]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_341]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_341]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_341]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_341]
        at org.apache.doris.persist.meta.MetaReader.read(MetaReader.java:116) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env.loadImage(Env.java:1701) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.master.Checkpoint.doCheckpoint(Checkpoint.java:130) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.master.Checkpoint.runAfterCatalogReady(Checkpoint.java:79) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.common.util.MasterDaemon.runOneCycle(MasterDaemon.java:58) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.common.util.Daemon.run(Daemon.java:116) ~[doris-fe.jar:1.2-SNAPSHOT]
Paimon File Catalog meta issue
Fix #23366
2023-09-21 08:41:03 +08:00
2e85e0163d Revert "[feature](function) add json->operator convert to json_extract (#19899)" (#24679)
Revert "[feature](function) add json->operator convert to json_extract (#19899)"
because it conflict with lambda syntax
This reverts commit f54a068d82e88e8535f3ed55a4224886b752e46b.
2023-09-20 21:16:19 +08:00
e4b551e2ce [fix](Config): Remove unused config max_connection_scheduler_threads_num (#24597) 2023-09-20 18:11:56 +08:00
2a260be10c [improvement](jdbc catalog) when lower_case_table_names of jdbc catalog properties is set to true, use the real table name to query the jdbc data source (#24520) 2023-09-20 17:47:11 +08:00
81e65f4a12 [feature](function) Support SHA family functions (#24342) 2023-09-20 17:21:45 +08:00
fc12362a6d [feature-wip](arrow-flight)(step2) FE support Arrow Flight server (#24314)
This is a POC, the design documentation will be updated soon
2023-09-20 14:42:54 +08:00
ca56921481 [docs](partition) Auto partition docs (#24574) 2023-09-20 14:28:23 +08:00
075552ead4 [feature](partitions)support batch delete partition (#23986)
ALTER TABLE example_db.my_table
DROP PARTITION p1,
DROP PARTITION p2,
DROP PARTITION p3;
2023-09-20 11:45:52 +08:00
5a0ccd702c [typo](docs) fix error in routine load doc (#24623) 2023-09-20 11:13:14 +08:00
14bd290aec [feature](jsonb)support json_length and json_contains function (#24332) 2023-09-20 10:40:44 +08:00
e59aa49f28 [feature](datetime-func)support milliseconds_add/sub/diff and microseconds_diff (#24114) 2023-09-20 10:38:56 +08:00
3f0d9d182f [doc](optimizer) Rewrite docs of statistics according to the current master (#24438)
Remove verbsoe sample, stale grammar, deleted function
And add some use advices
2023-09-18 18:25:43 +08:00
b4432ce577 [Feature](statistics)Support external table analyze partition (#24154)
Enable collect partition level stats for hive external table.
2023-09-18 14:59:26 +08:00
ae0b58fcde [typo](doc)modify error word (#24456) 2023-09-18 11:27:52 +08:00
990d6c02ec [Feature](new function) Add a uuid-numeric function, returns uuid in largerint type, 20x faster than uuid (#24395) 2023-09-16 18:26:13 +08:00
0ccb032d79 [parameter](query timeout) change default query timeout to 15min (#24480)
Co-authored-by: yiguolei <yiguolei@gmail.com>
2023-09-16 18:17:58 +08:00
ed8db3727c [feature](partial update) support MOW partial update for insert statement (#21597) 2023-09-16 17:11:59 +08:00
c270a2d89f [typo] (docs)Row storage mode can only be turned on when creating a table (#24257) 2023-09-15 17:40:46 +08:00
ddc0eb3508 [doc](flink-connector) add json string write (#24422) 2023-09-15 14:08:02 +08:00
29fe87982f [improve](outfile) add file_suffix options for outfile (#24334) 2023-09-15 12:58:41 +08:00
f57c75f935 [doc](agg_state) add agg_state in data-model (#24348)
add agg_state in data-model
2023-09-15 11:45:21 +08:00
320f1e9bbf [improve](routineload) improve show routine load output (#24264) 2023-09-15 10:22:47 +08:00
f61e6483bf [enhancement](broker-load) support compress type for old broker load, and split compress type from file format (#23882) 2023-09-14 21:42:28 +08:00
0488c87a38 [typo][doc]modify broken link. (#23988) 2023-09-14 21:41:40 +08:00
07720d3ff9 [feature](replica version) Add admin set replica version statement (#23706) 2023-09-14 21:12:00 +08:00
927de33166 [config](log) disable StreamLoad log default and enable in regression pipeline (#24354)
disable StreamLoad log default and enable in regression pipeline
2023-09-14 20:47:26 +08:00
3ee89aea35 [Feature](merge-on-write)Support ignore mode for merge-on-write unique table (#21773) 2023-09-14 18:03:51 +08:00
85e5b49d4c update structured streaming doc (#24016) 2023-09-14 17:47:27 +08:00
51a5895464 [Feature](RoutineLoad) Support max filter ratio for routine load (#24035) 2023-09-14 15:30:40 +08:00