Commit Graph

16052 Commits

Author SHA1 Message Date
2ed122b787 [improvement](task exec context) add parent class HasTaskExecutionCtx to own the task ctx (#29388)
---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-01-02 15:28:27 +08:00
4581618b09 [improvement](disk) pick disk randomly when usage is less than 0.7 (#29368) 2024-01-02 14:08:09 +08:00
eac96005b3 [Fix](Job)Concurrency may result in event loss (#29385) 2024-01-02 14:06:16 +08:00
f6443efe1a [doc](fix)fix kill doc error and add example (#29313) 2024-01-02 14:05:19 +08:00
bf6488e518 [fix](doc) show catalog example output fix (#29150)
Co-authored-by: Rohit Satardekar <rohitrs1983@gmail.com>
2024-01-02 14:05:05 +08:00
90b2ee90b2 [nereids] consider numNulls in filter estimation (#29184)
consider numNulls in filter estimation
2024-01-02 13:51:11 +08:00
b7487430da Revert "[improve](move-memtable) cancel load rapidly when stream close wait (#29322)" (#29371)
This reverts commit bbf58c5aa42d40e66bc6ccc9ed91a4fcb4bdfff7.
2024-01-02 11:32:14 +08:00
7cb3239599 [fix](sink) add writting restriction for OlapTableSinkV2Operator (#29340) 2024-01-02 11:16:06 +08:00
cec69f4cbd [Enhancement](wal) Add timout for wal memory back pressure (#29178) 2024-01-02 11:02:17 +08:00
4cbbd25d8c [fix](ctx) manager the lifecycle of connection context (#29346)
In FrontendService, we may create some connection context and set it as a thread local varaible.
These context should be removed from thread local after call.
Otherwise, it may be reused by other thread incorrectly.
2024-01-01 23:32:28 +08:00
738abac9ed [minor](context) duplicate query context in fragment ctx (#29364)
---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-01-01 22:08:23 +08:00
216b53058d [doc](mtmv)mtmv doc (#27549) 2024-01-01 17:46:21 +08:00
3d59f24403 [fix](compatibility) add some keywords (#29251)
1. Add `BINARY` keywords
    `BINARY` is used for tell MySQL to treat a String(or String column) as case sensitive.
     eg: `"abc" = "ABC"` is true, but `BINARY "abc" = "ABC"` is false.

    But in Doris, `"abc" = "ABC"` is false by default.

     I add this `BINARY` keyword just for compatibility, it will take no effect.

2. Add `PARTITIONS` and `AUTO_INCREMENT` as reserved words.

    `PARTITIONS` is the table name in `information_schema` database.
    `AUTO_INCREMENT` is a column name of a table in `information_schema`
2024-01-01 17:20:45 +08:00
01a0f0915f [fix](mtmv)fix insert overwrite will generate garbage temporary partition when restarting FE (#29075) 2024-01-01 08:12:26 +08:00
5985d216f3 [feature](mtmv)support cancel mtmv task command (#29252)
- `CANCEL MATERIALIZED VIEW TASK taskId on mvName`
- CANCEL MATERIALIZED VIEW TASK, tasks("type"="mv") and jobs("type"="mv") support check auth use priv of mv
- tasks and jobs add column mvName and mvDbName,you can use `select * from tasks("type"="mv") where MvName="xxx"` get all tasks of mv
- fix `desc mv all` error
- fix p0 The task sequence is incorrect
2023-12-31 23:10:30 +08:00
b07ce175de Revert "[fix](move-memtable) only check missing tablets when commit i… (#29352)
* Revert "[fix](move-memtable) only check missing tablets when commit info is not empty (#29326)"

This reverts commit 4634c723d7fb86f434060c0e6fa6168d92630cb9.

* Revert "[fix](move-memtable) check missing tablets before commit (#29223)"

This reverts commit fb0ed8c253f5241c23c71fb88f031604b0bf5a1f.
2023-12-31 22:25:25 +08:00
3c6c652997 [Fix](schema change) disable convert light schema change (#28205) (#29300) 2023-12-31 17:02:15 +08:00
cc40cdf2d4 [improve](load) don't limit add_segment running count in load stream (#29332)
Co-authored-by: Yongqiang YANG <98214048+dataroaring@users.noreply.github.com>
2023-12-31 16:47:46 +08:00
9a90c1cd57 [improve](move-memtable)add multi replica fault injection (#29348) 2023-12-31 16:30:27 +08:00
bbf58c5aa4 [improve](move-memtable) cancel load rapidly when stream close wait (#29322) 2023-12-31 16:26:41 +08:00
6c74313f60 [Fix](ut)(show) added UT cases for show backup stmt (#28604)
* Added UT for show backup
 * fixed some toSql() space and missing CLAUSE
 * Updated Doc with relevant syntax
2023-12-31 15:49:57 +08:00
4634c723d7 [fix](move-memtable) only check missing tablets when commit info is not empty (#29326)
Some sink may not get any success tablets, because load stream will only report them at the last CLOSE_LOAD.
For these sinks (no commit info), we should not check missing tablets since it's expected to be empty.
2023-12-31 13:50:13 +08:00
f92c0acba0 [Bug] When using the hive catalog the hive table name is uppercase, the Doris query is reported wrong #29058 (#29059) 2023-12-31 13:48:08 +08:00
74202fe7e2 [Enhancement] support user custom defined privite settings,xml for build fe (#29015)
Issue Number: close #29014
2023-12-31 13:42:56 +08:00
3661c316c9 Revert "[improvement](create tablet) backend create tablet round robin among disks (#23218)" (#29347)
This reverts commit df5b5ae0cb2f30f026ec104a64b4d9a5ce2904f3.
2023-12-31 12:51:21 +08:00
cc6013bfab [fix](partial update) Fix error message when doing strict mode partial update on a table with column that is non-nullable and has no default value #29218 2023-12-31 11:18:37 +08:00
fcc4cfb900 [Fix](Variant) add more info before crash in serialization (#29344) 2023-12-31 11:17:36 +08:00
ac636a2aba [fix](nereids) partition prunner evaluates "not x=const" on single node range #29164 2023-12-31 08:50:16 +08:00
5b503e0a7b [chore] remove and add collaborators (#29286)
Removed: nanfeng1999, gitccl
Added: gavinchou, yujun777
2023-12-30 23:41:15 +08:00
10368a71a4 [fix][security]security optimize for executable binary file doris_be access should be restricted (#29303) 2023-12-30 23:39:16 +08:00
6aaa829870 [fix](Nereids) connection_id() should return bigint type (#29287) 2023-12-30 22:25:27 +08:00
e91f6cb069 [chore](docs) straight_join is not supported (#28268) 2023-12-30 21:34:04 +08:00
aa4b3d02f1 [bug]https://github.com/apache/doris/issues/29149 (#29155)
Co-authored-by: 86157 <mapengfei@lnxjit.commapengfei@lnxjit.com>
2023-12-30 21:30:59 +08:00
9a7c15b635 [fix](log) PipelineFragmentContext has already been destoried in try_close_task before logging it #29237 2023-12-30 21:29:58 +08:00
018d8b1326 [fix](doc) spell errors fixes and align with code log for check-rpc-channel. (#28151) 2023-12-30 20:56:56 +08:00
56c3487a10 [test](ut) added UT cases for show backend stmt (#28603) 2023-12-30 20:55:28 +08:00
56648e0bd6 [Bug](security) BE SingleReplicaCompaction download_files function exists log print sensitive msg #28929 (#28930)
Co-authored-by: Rohit Satardekar <rohitrs1983@gmail.com>
2023-12-30 20:50:08 +08:00
f5e2ea1699 [fix][fe]Fixed show proc tablet inaccuracies (#29186)
Both tables simply need to change the tablet id to get the same result from the show proc statement.

before:
image
Just change the tablet id to get the result, 10139 does not belong to the tablet in the table id of 10127

later:
image
2023-12-30 20:49:01 +08:00
673a0137ce [fix] (schema change) fix missing data bug in schema change (#28938) (#29064) 2023-12-30 20:45:58 +08:00
7623b5cc31 [cleanup](move-memtable) remove namespace stream_load (#27441) 2023-12-30 20:08:23 +08:00
13f3d30652 [fix](hive-udf) Fixed the calculation result error in Hive bitmap UDF (#29073) 2023-12-30 17:25:33 +08:00
80fb496523 [improve](proc) Show journal size in bdbje journal dir (#29314) 2023-12-30 16:43:06 +08:00
e500b42142 Disable global cancel policy in regression test (#29335) 2023-12-30 14:35:17 +08:00
03901b9a7a [enhancement](group_commit): refector relay wal code (#29183) 2023-12-30 12:59:46 +08:00
30b7357604 [improve](logging) make glog FLAGS_v configurable (#29182) 2023-12-30 12:47:47 +08:00
e7d67e9411 [fix](be) resolves some unused-raii and used-after-moved issues (#29285) 2023-12-30 12:14:49 +08:00
51cb15d032 [improve](move-memtable) cancel load immediately when back pressure in delta writer v2 (#29280) 2023-12-30 10:45:06 +08:00
94cbabf675 [fix](case) update load_colddata_to_hdfs.groovy (#29325) 2023-12-30 10:30:44 +08:00
2c4e52e44e [fix](es catalog) only es_query function can push down to ES (#29320)
Issue Number: close #29318 
1. Only push down `es_query` function to ES
2. Add null check where ES query result not have `_source` or `fields` fields.
2023-12-30 09:33:26 +08:00
56058623f6 [fix](expr) Re fix BE core dump while common expr filter delete condition column (#29328)
Additional deleted filter condition will be materialized column be at the end of the block,
after _output_column_by_sel_idx will be erase, we not need to filter it,
so erase it from _columns_to_filter in the first next_batch.
Eg:
delete from table where a = 10;
select b from table;
a column only effective in segment iterator, the block from query engine only contain the b column,
so no need to filter a column by expr.
2023-12-30 08:40:03 +08:00