database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
18,726 Commits 1 Branch 0 Tags
acf741fa80a3d03dc3e4bd39f3d9ed122ffdcebc
Commit Graph

18726 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
walter
acf741fa80 [feature](binlog) Support gc binlogs by history nums and size (#35250) 
* [chore](binlog) Add logs about binlog gc (#34359)

* [feature](binlog) Support gc binlogs by history nums and size (#34888)
 2024-05-23 14:39:57 +08:00
jakevin
0b440685d9 [fix](nereids): fix PlanPostProcessor use visitor (#35244) 
(cherry picked from commit 46e004a358b9e13adb492d376f77e4317e558a6a)
 2024-05-23 14:12:25 +08:00
Pxl
e962a7309b [Chore](runtime-filter) adjust some check and error msg on runtime filter (#35018) (#35251) 
adjust some check and error msg on runtime filter
 2024-05-23 11:20:02 +08:00
Mingyu Chen
adc364a6fd [feature](Paimon) support deletion vector for Paimon naive reader (#34743) (#35241) 
bp #34743
Co-authored-by: 苏小刚 <suxiaogang223@icloud.com>
 2024-05-23 00:01:30 +08:00
HHoflittlefish777
50f50cf8cc Revert "[fix][docker] fix kafka test scritps (#33417)" (#35229) 
This reverts commit c35b2becdd08ab9255b3a0c2a19d74970f621388.
 2024-05-22 20:33:14 +08:00
zgxme
bc70968019 [chore](regression) Modify character encoding to be consistent with Doris (#35228) 2024-05-22 20:04:50 +08:00
zy-kkk
3a5fb6265a [refactor](jdbc catalog) split trino jdbc executor (#34932) (#35176) 
pick #34932
 2024-05-22 19:09:57 +08:00
zy-kkk
05a390e050 [refactor](jdbc catalog) split oceanbase jdbc executor (#34869) (#35175) 
pick #34869
 2024-05-22 19:09:35 +08:00
zy-kkk
24990383ff [refactor](jdbc catalog) split clickhouse jdbc executor (#34794) (#35174) 
pick master #34794
 2024-05-22 19:09:05 +08:00
Qi Chen
291cf57c54 [Configurations](multi-catalog) Add enable_parquet_filter_by_min_max and enable_orc_filter_by_min_max Session variables. (#35012) (#35164) 
backport #35012
 2024-05-22 19:06:12 +08:00
Mingyu Chen
05cedfca4e [fix](hudi) catch exception when getting hudi partition (#35027) (#35159) 
bp #35027
 2024-05-22 18:44:19 +08:00
wuwenchi
d63c3ae2d4 [bugfix](hive)fix testcase for viewfs for 2.1 #35178 2024-05-22 18:13:09 +08:00
Xinyi Zou
72f2d0d449 [fix](memory) Allow flush memtable failed when process exceed memlimit #35150 2024-05-22 18:11:59 +08:00
morrySnow
9ed4a2023b [fix](Nereids) DatetimeV2 round floor and round ceiling is wrong (#35153) (#35155) 
pick from master #35153

1.  round floor was incorrectly implemented as round
2. round ceiling not really round because use double type when divide
 2024-05-22 16:23:20 +08:00
shuke
30a66a4f9d [regression-test](fix) fix case bug #35201 2024-05-22 15:58:37 +08:00
feiniaofeiafei
15f70c8183 [Feat](planner)create table stmt offer default distribution attribute :random distribution and auto bucket (#35189) 
Co-authored-by: feiniaofeiafei <moailing@selectdb.com>
 2024-05-22 15:18:29 +08:00
Gabriel
c23384ff07 [fix](decimal) Fix long string casting to decimalv2 (#35121) 2024-05-22 14:32:29 +08:00
Pxl
84f7bfffe2 [Bug](bitmap-filter) fix empty bitmap when rf do merge (#34182) 
fix empty bitmap when rf do merge
 2024-05-22 14:29:50 +08:00
Xinyi Zou
9d7c65b4d8 [fix](memory) Avoid frequently refresh cgroup memory info (#35083) (#35182) 
pick #35083
 2024-05-22 11:42:08 +08:00
HappenLee
f0b2f5ba36 [Fix](bug) agg limit contains null values may cause error result (#35180) 2024-05-22 10:57:57 +08:00
HHoflittlefish777
7ca7458f44 [branch-2.1](routine-load) fix routine load case fail (#35173) 
* fix routine load case error
 2024-05-22 10:38:55 +08:00
yiguolei
dbf7a76592 Revert "[Chore](rollup) check duplicate column name when create table with rollup (#34827)" 
This reverts commit 4a8df535537e8eab8fa2ad54934a185e17d4e660.
 2024-05-22 10:19:51 +08:00
Xujian Duan
af7b16f213 [optimize](desc) display the correct data type of aggStateType (#34968) 
If a table column is AGG_STATE type, we can't get the clear defined data type if we use `desc tbl` statement.

create table a_table(
    k1 int null,
    k2 agg_state<max_by(int not null,int)> generic,
    k3 agg_state<group_concat(string)> generic
)
aggregate key (k1)
distributed BY hash(k1) buckets 3
properties("replication_num" = "1");

before optimize:

mysql> desc a_table;
+-------+------------------------------------------------+------+-------+---------+---------+
| Field | Type                                           | Null | Key   | Default | Extra   |
+-------+------------------------------------------------+------+-------+---------+---------+
| k1    | INT                                            | Yes  | true  | NULL    |         |
| k2    | org.apache.doris.catalog.AggStateType@239f771c | No   | false | NULL    | GENERIC |
| k3    | org.apache.doris.catalog.AggStateType@2e535f50 | No   | false | NULL    | GENERIC |
+-------+------------------------------------------------+------+-------+---------+---------+
3 rows in set (0.00 sec)


after optimize:

mysql> desc a_table;
+-------+------------------------------------+------+-------+---------+---------+
| Field | Type                               | Null | Key   | Default | Extra   |
+-------+------------------------------------+------+-------+---------+---------+
| k1    | INT                                | Yes  | true  | NULL    |         |
| k2    | AGG_STATE<max_by(INT, INT NULL)>   | No   | false | NULL    | GENERIC |
| k3    | AGG_STATE<group_concat(TEXT NULL)> | No   | false | NULL    | GENERIC |
+-------+------------------------------------+------+-------+---------+---------+


Co-authored-by: duanxujian <duanxujian@jd.com>
 2024-05-22 10:03:31 +08:00
zclllyybb
a8c24d7698 [Fix](function) fix overflow of date_add function (#35080) 
fix overflow of date_add function
 2024-05-22 10:02:59 +08:00
Jerry Hu
ced0093d74 [fix](mem_tracker] attach mem tracker in FragmentMgr::apply_filter (#35128) 2024-05-22 10:02:46 +08:00
zhiqiang
e8fb47bec1 [fix](broker load) Make Config.enable_pipeline_load works as expected for BrokerLoad (#35105) 
* FIX LOAD PROFILE

* FIX
 2024-05-22 10:02:02 +08:00
zclllyybb
b96148c9cd [Fix](function) fix days/weeks_diff result wrong on BE #35104 
select days_diff('2024-01-01 00:00:00', '2023-12-31 23:59:59');
should be 0 but got 1 on BE.
 2024-05-22 10:00:26 +08:00
xzj7019
7ae83b60fd [opt](Nereids) opt locality under multi-replica (#34927) 
Make tablet locality fixed under multi-replica cases.
Session variable: set enable_ordered_scan_range_locations = true, default false;
3 replica tpcds 100g: 7% improvement
 2024-05-22 10:00:13 +08:00
lihangyu
c962c3a025 [regression-test](prepared statment) optimize prepared_stmt.groovy to make test stable (#35087) 2024-05-22 09:59:30 +08:00
Xinyi Zou
2ed6a00fd1 [opt](memory) Add GlobalMemoryArbitrator and support ReserveMemory (#34985) (#35070) 2024-05-22 09:53:45 +08:00
yiguolei
11971eddb4 [atomicstatus](be) add atomic status to share state between multi thread (#35002) 2024-05-22 01:11:07 +08:00
yiguolei
9bc2c88384 [enhancement](memory) add exception check in page builder to avoid be oom during flush memtable (#35138) 2024-05-22 01:07:50 +08:00
yiguolei
1e4a83e17b fix compile 2024-05-22 01:04:34 +08:00
yiguolei
f38ecd349c [enhancement](memory) return error if allocate memory failed during add rows method (#35085) 
* return error when add rows failed

* f

---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
 2024-05-22 00:53:34 +08:00
Yongqiang YANG
b11f2ad9f0 [fix](cases) fix broker_load & schema_change & table_property cases (#35061) 
1. log more info to investigate problem.
2. use def in groovy to avoid gloabl namespace
 2024-05-22 00:46:11 +08:00
HHoflittlefish777
9fc38fb011 [branch-2.1](routine-load) fix routine load case error (#35161) 
* fix routine load case error

* update
 2024-05-21 23:16:55 +08:00
Jibing-Li
37f1bf317c [fix](statistics)Disable fetch min/max column stats through HMS, because the value may inaccurate and misleading. (#35124) (#35145) 
backport #35124
 2024-05-21 22:58:12 +08:00
wuwenchi
009ab77c25 [feature](iceberg)Support write to iceberg for 2.1 (#35103) #34257 #33629 
bp: #34257 #33629
 2024-05-21 22:46:37 +08:00
Mingyu Chen
903ff32021 [opt](fe) exit FE when transfer to (non)master failed (#34809) (#35158) 
bp #34809
 2024-05-21 22:31:47 +08:00
Ashin Gau
98f8eb5c43 [opt](split) get file splits in batch mode (#34032) (#35107) 
bp  #34032
 2024-05-21 22:27:07 +08:00
wangbo
037de3dedd [Fix](executor)Fix workload policy test #35146 2024-05-21 19:04:04 +08:00
GoGoWen
0599cb2efd fix replica's remote data size set to data size (#35098) 
fix replica's remote data size set to data size
 2024-05-21 16:48:08 +08:00
lide
367603a6c9 [security] fix fastjson security issues. (#35120) 
cherry pick from #34627
Co-authored-by: derenli <derenli@tencent.com>
 2024-05-21 16:35:42 +08:00
lide
b0ecf76131 [fix][build](audit-loader) Fix a build error for AuditLoaderPlugin. (#35119) 2024-05-21 16:34:48 +08:00
yujun
428a6fd6ab fix test_decommission_with_replica_num_fail (#35123) 2024-05-21 15:49:31 +08:00
yujun
706c9c473b [fix](autobucket) calc bucket num exclude today's partition #34304 #35129 2024-05-21 15:49:16 +08:00
Xin Liao
5019aa03e9 [enhancement](be-meta) disable sync rocksdb by default for better performance (#32714) (#35122) 2024-05-21 15:30:49 +08:00
HHoflittlefish777
44bb2bb639 [opt](routine-load) do not schedule invalid task (#34918) 2024-05-21 13:02:42 +08:00
shee
fb28d0b185 [BUG] fix scan range boundary handling is incorrect (#34832) 
fix scan range boundary handling is incorrect
Co-authored-by: shizhiqiang03 <shizhiqiang03@meituan.com>
 2024-05-21 13:00:50 +08:00
Qi Chen
74d66e9650 [Fix](parquet-reader) Fix Timestamp Int96 min-max statistics is incorrect when was written by some old parquet writers by disable it. (#35041) 
Parquet INT96 timestamp values were compared incorrectly for the purposes of producing statistics
by older parquet writers, so PARQUET-1065 deprecated them. The result is that any writer that produced
stats was producing unusable incorrect values, except the special case where min == max and an incorrect
ordering would not be material to the result. PARQUET-1026 made binary stats available and valid in that special case.
 2024-05-21 13:00:22 +08:00