database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
18,263 Commits 1 Branch 0 Tags
42e91149e4df989a5cbdbcdeb72b5377e58ed653
Commit Graph

18263 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Kaijie Chen
7623b5cc31 [cleanup](move-memtable) remove namespace stream_load (#27441) 2023-12-30 20:08:23 +08:00
lihuigang
13f3d30652 [fix](hive-udf) Fixed the calculation result error in Hive bitmap UDF (#29073) 2023-12-30 17:25:33 +08:00
walter
80fb496523 [improve](proc) Show journal size in bdbje journal dir (#29314) 2023-12-30 16:43:06 +08:00
wangbo
e500b42142 Disable global cancel policy in regression test (#29335) 2023-12-30 14:35:17 +08:00
huanghaibin
03901b9a7a [enhancement](group_commit): refector relay wal code (#29183) 2023-12-30 12:59:46 +08:00
Kaijie Chen
30b7357604 [improve](logging) make glog FLAGS_v configurable (#29182) 2023-12-30 12:47:47 +08:00
py023
e7d67e9411 [fix](be) resolves some unused-raii and used-after-moved issues (#29285) 2023-12-30 12:14:49 +08:00
HHoflittlefish777
51cb15d032 [improve](move-memtable) cancel load immediately when back pressure in delta writer v2 (#29280) 2023-12-30 10:45:06 +08:00
Dongyang Li
94cbabf675 [fix](case) update load_colddata_to_hdfs.groovy (#29325) 2023-12-30 10:30:44 +08:00
qiye
2c4e52e44e [fix](es catalog) only es_query function can push down to ES (#29320) 
Issue Number: close #29318 
1. Only push down `es_query` function to ES
2. Add null check where ES query result not have `_source` or `fields` fields.
 2023-12-30 09:33:26 +08:00
Xinyi Zou
56058623f6 [fix](expr) Re fix BE core dump while common expr filter delete condition column (#29328) 
Additional deleted filter condition will be materialized column be at the end of the block,
after _output_column_by_sel_idx will be erase, we not need to filter it,
so erase it from _columns_to_filter in the first next_batch.
Eg:
delete from table where a = 10;
select b from table;
a column only effective in segment iterator, the block from query engine only contain the b column,
so no need to filter a column by expr.
 2023-12-30 08:40:03 +08:00
airborne12
5c9e0d906a [Fix](segment iterator) fix wrong segment cache remove (#29333) 2023-12-30 08:36:15 +08:00
HHoflittlefish777
5bc72bda78 [improve](move-memtable) add fault injection in writer v2 (#29177) 2023-12-30 01:32:26 +08:00
HHoflittlefish777
e62857e5a4 [improve](move-memtable) return error when back pressure cost too much time (#29327) 2023-12-30 01:31:05 +08:00
HHoflittlefish777
41d907912a [improve](move-memtable) add fault injection in load stream stub (#29105) 2023-12-30 01:29:28 +08:00
Xiangyu Wang
8407490053 [feature-wip](nereids) Support some spark-sql built-in functions when set dialect=spark_sql (#28531) 2023-12-30 00:10:35 +08:00
Nitin-Kashyap
445f72b395 [ut](stats) Added tests for HMS analysis tasks (#28583) 2023-12-30 00:09:50 +08:00
deardeng
d7001ce6e1 [fix](partition) Comment out some exceptions with partition equal to … (#28702) 
* [fix](partition) Comment out some exceptions with partition equal to 0 and return

In some scenarios, the partition id in the table meta is 0, but the rowset meta partition is not 0, (#28055) PR check partition id will caused be failed to start when upgrade

meta like this

```
{
    "table_id": 1266302,
    "partition_id": 0,
    "tablet_id": 1289222,
    "schema_hash": 816078096,
    "shard_id": 949,
    "creation_time": 1684805433,
    "cumulative_layer_point": -1,
    "tablet_state": "PB_RUNNING",
    "schema": {
        "keys_type": "DUP_KEYS",
        "column": [
	    ...
        ],
        "num_short_key_columns": 1,
        "num_rows_per_row_block": 1024,
	...
        "skip_write_index_on_load": false
    },
    "rs_metas": [
        {
            "rowset_id": 0,
            "partition_id": 1289193,
            "tablet_id": 1289222,
            "tablet_schema_hash": 816078096,
            "newest_write_timestamp": 1684805433,
	    ...
            "segments_overlap_pb": "NONOVERLAPPING"
        }
    ],
    "in_restore_mode": false,
    "tablet_uid": {
        "hi": -4301918259617109775,
        "lo": 2820124814688031668
    },
    "preferred_rowset_type": "BETA_ROWSET",
    "tablet_type": "TABLET_TYPE_DISK",
    "replica_id": 1812028,
    "enable_unique_key_merge_on_write": false,
    "binlog_config": {
        "enable": false,
        "ttl_seconds": 9223372036854775807,
        "max_bytes": 9223372036854775807,
        "max_history_nums": 9223372036854775807
    },
    "compaction_policy": "size_based",
    "time_series_compaction_goal_size_mbytes": 1024,
    "time_series_compaction_file_count_threshold": 2000,
    "time_series_compaction_time_threshold_seconds": 3600
}
```

* fix
 2023-12-30 00:07:43 +08:00
Nitin-Kashyap
0bf0f945a2 [test](ut) added UT cases for show broker stmt (#28629) 2023-12-30 00:04:01 +08:00
Petrichor
4e56adee15 [improvement](docs) add Json load case doc (#29001) 2023-12-30 00:02:43 +08:00
Luwei
b8cdbf4ffe Fix](schema change) fix metadata inconsistency bug caused by enable light schema change (#28255) (#29060) 2023-12-30 00:02:01 +08:00
Mryange
7f105facc4 [fix](pipelineX) fix unable to merge profiles in multi be (#29204) 2023-12-29 23:48:46 +08:00
Siyang Tang
2518ed64ea [enhancement](bulk-load) strict second level granularity for timeout between (0,1) (#29028) 2023-12-29 23:44:43 +08:00
morrySnow
989d20e0ac [opt](Nereids) remove decimalv2 signature from min, max, sum, nvl and case when (#29282) 2023-12-29 23:22:32 +08:00
wangbo
03ece437f0 Fix topicPublisher thread may NPE when no topic exists (#29306) 2023-12-29 23:04:20 +08:00
Sun Chenyang
59fdd5e42b [fix](compaction) time series compaction policy -> compact consecutive empty rowsets (#27299) 
Sometimes we need to merge a large number of empty versions to reduce meta-information.
 2023-12-29 22:20:41 +08:00
morrySnow
fcb90dc808 [opt](Nereids) use enable_nereids_dml to control ctas and create table (#29284) 2023-12-29 20:47:22 +08:00
Mryange
07bd65abe0 [pipelineX](local shuffle) remove unused code in local shuffle to improve performance #29292 2023-12-29 20:30:28 +08:00
HappenLee
7c5fda1169 [thirdparty](lib) Add streamvbyte thirdparty lib (#29308) 2023-12-29 19:22:07 +08:00
Gabriel
f0629da10a [pipelineX](fix) Fix illegal memory access (#29283) 2023-12-29 18:35:41 +08:00
Xiangyu Wang
4266432d11 [Fix](dialect) Fix trino dialect converter when sql does not end with delimiter. (#29106) 
Co-authored-by: wangxiangyu <wangxiangyu@360shuke.com>
 2023-12-29 18:02:57 +08:00
zhangguoqiang
7604401b06 [Enhance](regression)Do path creation ahead of time for case test_export_external_table (#28616) 
Do path creation ahead of time for case test_export_external_table
 2023-12-29 17:59:20 +08:00
yujun
7e0616ba77 [fix](regression test) fix test_publish_one_succ (#28993) 2023-12-29 17:31:55 +08:00
Dongyang Li
9c57483aa3 [ci](trigger) only master run performance check (#29290) 
Co-authored-by: stephen <hello-stephen@qq.com>
 2023-12-29 17:01:31 +08:00
Siyang Tang
ada5cf4b2e [fix](chore) fix session var memtable on sink for bulk load (#29102) 2023-12-29 17:00:02 +08:00
Vallish Pai
1a2ec2fbd9 [fix](meta)Fix error code for unknown table #27751 (#27976) 
This fix will improve the error code for unknown table
exception cases.
 2023-12-29 16:28:04 +08:00
zy-kkk
236276e9c1 [typo](docs) fix lakehouse doc 404 (#29299) 2023-12-29 16:27:48 +08:00
Xinyi Zou
82635d4b59 [opt](memory) All LRU Cache inherit from LRUCachePolicy (#28940) 
After all LRU Cache inherits from LRUCachePolicy, this will allow prune stale entry, eviction when memory exceeds limit, and define common properties. LRUCache constructor change to private, only allow LRUCachePolicy to construct it.

Impl DummyLRUCache, when LRU Cache capacity is 0, will no longer be meaningless insert and evict.
 2023-12-29 16:15:56 +08:00
starocean999
866fd5d32a [feature](fe)support last column or index definition end with comma in create table statement (#29167) 
allow the column list end with COMMA in create table statement. This is a issue rooted from history. So nereids has to keep the same behavior as old planner.
CREATE TABLE t
(
k1 int,
)
 2023-12-29 16:12:32 +08:00
starocean999
9925f7be8e [fix](nereids)FillUpMissingSlots rule didn't process standalone having clause correctly (#29143) 2023-12-29 16:12:05 +08:00
wangbo
c3c34e10bb [feature](executor) Add some check when create workload group/workload schedule policy (#29236) 2023-12-29 15:41:16 +08:00
赵硕
d6dcf962a9 [Enhancement](page cache) insert into setting to disable page cache (#28913) 2023-12-29 15:11:41 +08:00
yujun
a7c4abde9f [fix](regression test) fix test_schema_change_concurrent_with_txn (#28994) 2023-12-29 15:07:11 +08:00
yujun
24cdc00b60 [improvement](doris compose) avoid docker network conflict (#28957) 2023-12-29 15:05:28 +08:00
Calvin Kirs
660f12c717 [dependency](fe)Upgrade dependency library (#29049) 2023-12-29 14:50:46 +08:00
HappenLee
411f819306 [Opt](join) opt the join copy column from build side (#29007) 2023-12-29 14:39:33 +08:00
Jibing-Li
2308881e9f [improvement](statistics) Analyze partition columns when new partition loaded data for the first time. (#29154) 
The first time load data to a partition, we need to analyze the partition columns even when the health rate is high. Because if not, the min max value of the column may not include the new partition values, which may cause bad plan.
 2023-12-29 14:36:48 +08:00
Dongyang Li
02b27c1673 [ci](trigger) support "run performance" (#29259) 
* [ci](trigger) support "run performance"
Co-authored-by: stephen <hello-stephen@qq.com>
 2023-12-29 14:21:23 +08:00
xy
4fc5860a7c [Fix](doc)Fix word spelling errors (#29268) 
Co-authored-by: xingying01 <xingying01@corp.netease.com>
 2023-12-29 14:15:37 +08:00
fornaix
36b45b570b [fix](mysql) fix mysql channel infinite blocking (#28808) 
Call the Channels blocking method with timeout instead.

Using session variables net_write_timeout and net_read_timeout as the timeout parameter.
 2023-12-29 13:57:22 +08:00