Commit Graph

2730 Commits

Author SHA1 Message Date
59c494eb27 [fix](Docs) Modify documents about SELECT INTO OUTFILE and EXPORT (#24641) 2023-09-25 19:51:15 +08:00
8679095e5c [feature](debug) support debug point used in debug code (#24502) 2023-09-25 17:56:12 +08:00
xfz
1b95ce1d93 [feature](json-function) add json_insert, json_replace, json_set functions (#24384)
[feature](json-function) add three json funcitons
2023-09-25 12:52:29 +08:00
2d0ac7117a [enhancement](show-frontends-disks) Add disk info corresponding to deploy dir in frontend disk info (#24726) 2023-09-25 11:14:03 +08:00
312576b93c [typo](docs) modify flink connector document error (#23967) 2023-09-25 10:11:33 +08:00
129ffb7dd7 [docs](docs)fix display error #24775 2023-09-25 08:38:51 +08:00
bacc310502 [Docs](StartUp)Add StartUp methods docs and metadata recovery docs (#24816) 2023-09-23 18:45:57 +08:00
c943a05065 [fix](stats) Fix data size calculation of auto sample (#24672)
1. Fix data size calculation of auto sample, before this pr, the data size is include all the replicas
2. Move some auto analyze related options to global session variable
3. Add some logs
2023-09-22 18:12:39 +08:00
22616d125d [function](bitmap) add function alias bitmap_andnot and bitmap_andnot_count (#24771) 2023-09-22 12:18:31 +08:00
cec3fcdf78 [typo](doc) modifying old parameters (#24565)
* modifying old parameters

* Supplementary demo
2023-09-22 09:09:06 +08:00
269b9eaf8b [typo](docs)add FAQ for broker load error: Cannot locate default realm (#24693) 2023-09-22 09:06:36 +08:00
58ab25ccaa Revert "[Feature](merge-on-write)Support ignore mode for merge-on-write unique table (#21773)" (#24731)
This reverts commit 3ee89aea35726197cb7e94bb4f2c36bc9d50da84.
2023-09-21 21:01:28 +08:00
5e82aebd25 [fix](fe-meta) fix paimon file catalog meta issue and replication num analysis issue (#24681)
Fix 2 issues:

The replication num analysis will throw exception when there is no BE alive when FE restart.
org.apache.doris.common.AnalysisException: errCode = 2, detailMessage = errCode = 2, detailMessage = replication num should be less than the number of available backends. replication num is 1, available backend num is 0
        at org.apache.doris.common.util.PropertyAnalyzer.analyzeReplicaAllocation(PropertyAnalyzer.java:1018) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.TableProperty.buildReplicaAllocation(TableProperty.java:483) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.TableProperty.read(TableProperty.java:529) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.OlapTable.readFields(OlapTable.java:1370) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Table.read(Table.java:383) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Database.readFields(Database.java:631) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.datasource.InternalCatalog.loadDb(InternalCatalog.java:3075) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env.loadDb(Env.java:1764) ~[doris-fe.jar:1.2-SNAPSHOT]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_341]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_341]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_341]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_341]
        at org.apache.doris.persist.meta.MetaReader.read(MetaReader.java:116) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.catalog.Env.loadImage(Env.java:1701) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.master.Checkpoint.doCheckpoint(Checkpoint.java:130) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.master.Checkpoint.runAfterCatalogReady(Checkpoint.java:79) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.common.util.MasterDaemon.runOneCycle(MasterDaemon.java:58) ~[doris-fe.jar:1.2-SNAPSHOT]
        at org.apache.doris.common.util.Daemon.run(Daemon.java:116) ~[doris-fe.jar:1.2-SNAPSHOT]
Paimon File Catalog meta issue
Fix #23366
2023-09-21 08:41:03 +08:00
2e85e0163d Revert "[feature](function) add json->operator convert to json_extract (#19899)" (#24679)
Revert "[feature](function) add json->operator convert to json_extract (#19899)"
because it conflict with lambda syntax
This reverts commit f54a068d82e88e8535f3ed55a4224886b752e46b.
2023-09-20 21:16:19 +08:00
e4b551e2ce [fix](Config): Remove unused config max_connection_scheduler_threads_num (#24597) 2023-09-20 18:11:56 +08:00
2a260be10c [improvement](jdbc catalog) when lower_case_table_names of jdbc catalog properties is set to true, use the real table name to query the jdbc data source (#24520) 2023-09-20 17:47:11 +08:00
81e65f4a12 [feature](function) Support SHA family functions (#24342) 2023-09-20 17:21:45 +08:00
fc12362a6d [feature-wip](arrow-flight)(step2) FE support Arrow Flight server (#24314)
This is a POC, the design documentation will be updated soon
2023-09-20 14:42:54 +08:00
ca56921481 [docs](partition) Auto partition docs (#24574) 2023-09-20 14:28:23 +08:00
075552ead4 [feature](partitions)support batch delete partition (#23986)
ALTER TABLE example_db.my_table
DROP PARTITION p1,
DROP PARTITION p2,
DROP PARTITION p3;
2023-09-20 11:45:52 +08:00
5a0ccd702c [typo](docs) fix error in routine load doc (#24623) 2023-09-20 11:13:14 +08:00
14bd290aec [feature](jsonb)support json_length and json_contains function (#24332) 2023-09-20 10:40:44 +08:00
e59aa49f28 [feature](datetime-func)support milliseconds_add/sub/diff and microseconds_diff (#24114) 2023-09-20 10:38:56 +08:00
3f0d9d182f [doc](optimizer) Rewrite docs of statistics according to the current master (#24438)
Remove verbsoe sample, stale grammar, deleted function
And add some use advices
2023-09-18 18:25:43 +08:00
b4432ce577 [Feature](statistics)Support external table analyze partition (#24154)
Enable collect partition level stats for hive external table.
2023-09-18 14:59:26 +08:00
ae0b58fcde [typo](doc)modify error word (#24456) 2023-09-18 11:27:52 +08:00
990d6c02ec [Feature](new function) Add a uuid-numeric function, returns uuid in largerint type, 20x faster than uuid (#24395) 2023-09-16 18:26:13 +08:00
0ccb032d79 [parameter](query timeout) change default query timeout to 15min (#24480)
Co-authored-by: yiguolei <yiguolei@gmail.com>
2023-09-16 18:17:58 +08:00
ed8db3727c [feature](partial update) support MOW partial update for insert statement (#21597) 2023-09-16 17:11:59 +08:00
c270a2d89f [typo] (docs)Row storage mode can only be turned on when creating a table (#24257) 2023-09-15 17:40:46 +08:00
ddc0eb3508 [doc](flink-connector) add json string write (#24422) 2023-09-15 14:08:02 +08:00
29fe87982f [improve](outfile) add file_suffix options for outfile (#24334) 2023-09-15 12:58:41 +08:00
f57c75f935 [doc](agg_state) add agg_state in data-model (#24348)
add agg_state in data-model
2023-09-15 11:45:21 +08:00
320f1e9bbf [improve](routineload) improve show routine load output (#24264) 2023-09-15 10:22:47 +08:00
f61e6483bf [enhancement](broker-load) support compress type for old broker load, and split compress type from file format (#23882) 2023-09-14 21:42:28 +08:00
0488c87a38 [typo][doc]modify broken link. (#23988) 2023-09-14 21:41:40 +08:00
07720d3ff9 [feature](replica version) Add admin set replica version statement (#23706) 2023-09-14 21:12:00 +08:00
927de33166 [config](log) disable StreamLoad log default and enable in regression pipeline (#24354)
disable StreamLoad log default and enable in regression pipeline
2023-09-14 20:47:26 +08:00
3ee89aea35 [Feature](merge-on-write)Support ignore mode for merge-on-write unique table (#21773) 2023-09-14 18:03:51 +08:00
85e5b49d4c update structured streaming doc (#24016) 2023-09-14 17:47:27 +08:00
51a5895464 [Feature](RoutineLoad) Support max filter ratio for routine load (#24035) 2023-09-14 15:30:40 +08:00
51811f72b3 [doc](partial update) add complete doc for partial update (#24317) 2023-09-14 14:49:24 +08:00
25763d0ed9 [typo](doc) fix error description about load_to_single_tablet (#24270)
* fix error description about load_to_single_tablet

* fix review
2023-09-14 11:48:06 +08:00
64337a8698 [Improve](metadata)Start the script to set metadata_failure_recovery (#24308) 2023-09-14 10:02:35 +08:00
86aa3802cf [log](config) set streamload record default to enable 2023-09-13 16:32:30 +08:00
a6f05e89f5 [doc](catalog) update cache refresh doc (#24183)
Some cache refresh doc is missing
2023-09-13 11:45:26 +08:00
1f769291b5 [doc](invert index) add invert index char_filter doc (#24205) 2023-09-13 10:02:45 +08:00
9df72a96f3 [Feature](multi-catalog) Support hadoop viewfs. (#24168)
### Feature

Support hadoop viewfs.

### Test

- Regression tests: 
  - hive viewfs test.
  - tvf viewfs test.

- Broker load with broker and with hdfs tests manually.
2023-09-13 00:20:12 +08:00
4bb9a12038 [function](bitmap) support bitmap_remove (#24190) 2023-09-12 14:52:04 +08:00
aa850fc9c3 [doc](hive) add faq for multi delimiter config (#24179)
when a user sets `serde` as `org.apache.hadoop.hive.contrib.serde2.MultiDelimitserDe`, we should add a configuration in hive-site.xml
2023-09-12 10:34:23 +08:00