database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
20,050 Commits 1 Branch 0 Tags
0e9fa3dff7a7ff73597995e4ae097489fc1fefef
Commit Graph

20050 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
amory
0e9fa3dff7 [fix](decimaltype) handle exception with tablet init (#40263) 
## Proposed changes
to avoid be core like 
```
terminate called after throwing an instance of 'doris::Exception'
  what():  [E6] meet invalid precision: real_precision=28, max_decimal_precision=27, min_decimal_precision=1

        0#  doris::Exception::Exception(int, std::basic_string_view<char, std::char_traits<char> > const&) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/common/exception
.cpp:0
        1#  doris::Exception::Exception<unsigned int const&, unsigned long>(int, std::basic_string_view<char, std::char_traits<char> > const&, unsigned int const&, unsigned long&
&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/basic_string.h:187
        2#  doris::vectorized::DataTypeDecimal<doris::vectorized::Decimal<__int128> >::DataTypeDecimal(unsigned int, unsigned int, unsigned int, unsigned int) at /home/zcp/repo_c
enter/doris_branch-3.0/doris/be/src/vec/data_types/data_type_decimal.h:0
        3#  doris::vectorized::DataTypeFactory::create_data_type(doris::TypeDescriptor const&, bool) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/data_types/data_ty
pe_factory.cpp:0
        4#  doris::vectorized::DataTypeFactory::create_data_type(doris::TypeDescriptor const&, bool) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/data_types/data_ty
pe_factory.cpp:0
        5#  doris::vectorized::DataTypeFactory::create_data_type(doris::TypeDescriptor const&, bool) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/data_types/data_ty
pe_factory.cpp:0
        6#  doris::SlotDescriptor::get_empty_mutable_column() const at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/shared_ptr_base
.h:1295
        7#  doris::VOlapTablePartitionParam::VOlapTablePartitionParam(std::shared_ptr<doris::OlapTableSchemaParam>&, doris::TOlapTablePartitionParam const&) at /home/zcp/repo_cen
ter/doris_branch-3.0/doris/be/src/vec/common/cow.h:154
        8#  doris::vectorized::VTabletWriter::_init(doris::RuntimeState*, doris::RuntimeProfile*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/vec/sink/writer/vtablet_w
riter.cpp:1177
        9#  doris::vectorized::VTabletWriter::open(doris::RuntimeState*, doris::RuntimeProfile*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/common/status.h:488
        10# doris::vectorized::AsyncResultWriter::process_block(doris::RuntimeState*, doris::RuntimeProfile*) at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/common/status
.h:488
        11# std::_Function_handler<void (), doris::vectorized::AsyncResultWriter::start_writer(doris::RuntimeState*, doris::RuntimeProfile*)::$_0>::_M_invoke(std::_Any_data const
&) at /var/local/ldb-toolchain/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/ext/atomicity.h:98
        12# doris::ThreadPool::dispatch_thread() at /home/zcp/repo_center/doris_branch-3.0/doris/be/src/util/threadpool.cpp:0
        13# doris::Thread::supervise_thread(void*) at /var/local/ldb-toolchain/bin/../usr/include/pthread.h:562
        14# ?
```

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-09-03 14:38:16 +08:00
Mingyu Chen
67a2daf33c [fix](execution) fix wrong check when blocking result sink (#40289) 
Otherwise, when client stop fetch result data, the result sink will
still adding batch to the result queue,
which causing BE OOM.
 2024-09-03 14:17:42 +08:00
abmdocrt
c49b345342 [cherry-pick](branch-2.1) Fix enable_mow_light_delete default value (#40283) 
## Proposed changes

Pick #40104 #40197

<!--Describe your changes.-->
 2024-09-03 11:19:05 +08:00
qiye
41271ecba0 [fix](ES Catalog)Do not push down limit to ES when predicates can not be processed by ES. (#40111) (#40265) 
bp #40111
 2024-09-03 11:17:24 +08:00
Mingyu Chen
92752b90e7 [feature](metacache) add system table catalog_meta_cache_statistics #40155 (#40210) 
bp #40155
 2024-09-02 23:23:35 +08:00
morrySnow
1527099e35 [opt](Nereids) forbid one step topn with distribute as child (#40066) (#40278) 
pick from master #40066
 2024-09-02 22:26:26 +08:00
Jibing-Li
a5efdc6085 [improve](statistics)Clean expired TableStatsMeta. (#39779) (#40253) 
backport: https://github.com/apache/doris/pull/39779
 2024-09-02 22:13:42 +08:00
lihangyu
4a91b2d46b [pick](SchemaCache) remove redundant Schema cache (#40257) 
#40091
 2024-09-02 19:24:03 +08:00
Jibing-Li
8ff068eb7c [feature](statistics) Support get row count for JDBC external table. (#38889) (#40248) 
backport: https://github.com/apache/doris/pull/38889
 2024-09-02 16:39:42 +08:00
zhangstar333
70eeda9f50 [cherry-pick](branch-21) fix encryption_function return wrong result at const column (#40201) (#40247) 
## Proposed changes
cherry-pick from master #40201
the string object have error of order, should be src, key,iv,mode
 2024-09-02 16:09:15 +08:00
lihangyu
c8d3202595 [regression-test](cases) optimize some cases (#40240) 
#40174
 2024-09-02 14:50:48 +08:00
Gabriel
e4ee0e8379 [fix](topn) Fix wrong rows returned by TOPN sorter (#40243) 
## Proposed changes

pick #40241

<!--Describe your changes.-->
 2024-09-02 14:34:53 +08:00
lihangyu
98e039ae31 [pick](Variant) truncate key when column is variant type when crea… (#40238) 
#39988
 2024-09-02 13:54:04 +08:00
Xinyi Zou
5db2c8b8bc [branch-2.1](arrow-flight-sql) Fix return result from FE Arrow Flight server error 0.0.0.0:xxx, connection refused (#40229) 
pick #40002
 2024-09-02 09:45:27 +08:00
zhangstar333
b104e933cd [Bug](expr) execute expr should use local states instead of operators (#40189) (#40219) 
## Proposed changes

cherry-pick from master #40189

<!--Describe your changes.-->
 2024-09-01 00:41:10 +08:00
TengJianPing
d792a62409 [fix](regression) fix regression failure of window_funnel caused by duplicated table name (#40216) 
BP #40217
 2024-08-31 20:58:20 +08:00
Calvin Kirs
2a12251b83 [branch-2.1][Feat](Job)After a job is paused, it can be manually triggered to execute. (#40180) 
…

pick (#39565)
 2024-08-31 19:24:53 +08:00
yiguolei
1768169b9a Revert "[Improvement](sort) Free sort blocks if this block is exhausted (#39306)" (#40211) 
Reverts apache/doris#39956
 2024-08-31 15:58:55 +08:00
zhiqiang
44b87cb5ba [chore](script message) More convenient error msg for nofile open. #39642 (#40195) 
cherry pick from #39642
 2024-08-31 00:18:20 +08:00
zhiqiang
04fbcbd337 [chore](log) Log written rows when result sink close #39647 (#40196) 
cherry pick from #39647
 2024-08-30 23:59:39 +08:00
Dongyang Li
1ed7975e9f [fix](case) drop table if exists before create it (#40205) 
## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-30 23:22:15 +08:00
Mingyu Chen
e4c48cfec5 [fix](multi-catalog)fix max compute null parts table read (#40046) (#40179) 
bp #40046

Co-authored-by: slothever <18522955+wsjz@users.noreply.github.com>
 2024-08-30 22:56:08 +08:00
zhiqiang
d949df0c4f [chore](cancel) Make enable_pipeline_task_leakage_detect false by default #40145 (#40192) 
cherry pick from #40145
 2024-08-30 22:48:00 +08:00
feiniaofeiafei
32a32808a7 [improvement](nereids) support convert_tz in partition prune (#40047) (#40127) 
cherry-pick #40047 to branch-2.1
 2024-08-30 13:39:37 +08:00
Gabriel
3ac8347e3d [Improvement](sort) Free sort blocks if this block is exhausted (#39306) (#39956) 
## Proposed changes

pick #39306

<!--Describe your changes.-->
 2024-08-30 13:35:43 +08:00
walter
80508fb595 [fix](restore) Reset next version for remote table when restore #40118 (#40165) 
cherry pick from #40118

Co-authored-by: smallx <e9999e@163.com>
 2024-08-30 13:22:23 +08:00
Gabriel
531fe54207 [fix](join) Fix partitioned hash join strategy (#40164) 
## Proposed changes

pick #40163

<!--Describe your changes.-->
 2024-08-30 12:44:42 +08:00
starocean999
1288676b68 [fix](nereids)change lag/lead function signature (#40161) 
## Proposed changes

pick from master https://github.com/apache/doris/pull/40060

<!--Describe your changes.-->
 2024-08-30 12:05:47 +08:00
zclllhhjj
163193b1d4 [branch-2.1](function) fix random_bytes return same data for multi rows (#39891) (#40137) 
pick https://github.com/apache/doris/pull/39891

Issue Number: close #xxx

before:
```sql
mysql [optest]>SELECT random_bytes(10) a FROM numbers("number" = "10");
+------------------------+
| a                      |
+------------------------+
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
| 0x7b4e5727024bc5b59e2c |
+------------------------+
```

now:
```sql
mysql [optest]>SELECT random_bytes(10) a FROM numbers("number" = "10");
+------------------------+
| a                      |
+------------------------+
| 0xd82cf60825b29ef2a0fd |
| 0x6f8c808415bdbaa6d257 |
| 0x7c26b5214297a151c25c |
| 0x43f02c77293063900437 |
| 0x5e5727569dec5e24f96b |
| 0x434f20bf74d7759640b7 |
| 0x087ed96b739750c733a6 |
| 0xdf05f6d7ede4972eb846 |
| 0xcefab471912264b5c54f |
| 0x1bddc019409d1926aa10 |
+------------------------+
```

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->
 2024-08-30 10:43:42 +08:00
zclllhhjj
e4d543f060 [branch-2.1](load) fix commit txn timeout when loading to table with many tablet (#40031) (#40139) 
pick https://github.com/apache/doris/pull/40031
 2024-08-30 10:34:43 +08:00
zzzxl
70daa1f85d [opt](inverted index) Controls whether the in_list can execute fast_execute. (#40141) 
https://github.com/apache/doris/pull/40022
 2024-08-30 10:32:43 +08:00
yiguolei
ca07a00c93 Revert "[branch-2.1](hive) support hive write text table (#38549) (#4… (#40157) 
…0063)"

This reverts commit c6df7c21a3c09ae1664deabacb88dfcea9d94b68.

## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->

Co-authored-by: yiguolei <yiguolei@gmail.com>
 2024-08-30 10:25:38 +08:00
ChenPeng2013
519e5c88cc [test](inverted index)Add upgrade and downgrade test cases for inverted_index (#37783) 
## Proposed changes

Issue Number: close #xxx

<!--Describe your changes.-->

add case
 2024-08-30 10:19:34 +08:00
feiniaofeiafei
1ecc85b575 [Fix](ctas) forward the use_max_length_of_varchar_in_ctas (#39705) (#40138) 
cherry-pick from master #39705
 2024-08-30 10:11:29 +08:00
abmdocrt
0b9817ea4b [cherry-pick](branch-2.1) Pick "[Enhancement](txn) Block new insert into if schema change happens during transaction (#39483)" (#40115) 
## Proposed changes

Pick #39483 

<!--Describe your changes.-->
 2024-08-30 10:01:10 +08:00
seawinde
45e35e8a99 [improvement](mtmv) Optimize get available mv logic to avoid unnecessary rewrite (#39734) (#40136) 
## Proposed changes

commitId: 55ed1277
pr: https://github.com/apache/doris/pull/39734
 2024-08-30 08:02:27 +08:00
seawinde
e072ce7f68 [fix](nereids) Remove Nondeterministic to avoid ambiguity when decide a expression is nondeterministic or not (#39801) (#40130) 
## Proposed changes

commitId: 2b715a7e
pr: https://github.com/apache/doris/pull/39801
 2024-08-30 07:51:42 +08:00
TengJianPing
b86e4e8498 [fix](profile) fix possible coredump of rpc verbose profile (#40117) 
## Proposed changes

Issue Number: close #xxx

`_instance_to_rpc_stats_vec` may be updated when sorting in
`ExchangeSinkBuffer<Parent>::update_profile`, which may cause coredump.
 2024-08-29 23:54:52 +08:00
924060929
c50d2ea4e2 [fix](test) fix unstable test (#40143) 
## Proposed changes

cherry pick from #40142
 2024-08-29 23:42:24 +08:00
morrySnow
82fb747e6f [fix](Nereids) datetimev2 literal equals should compare microsecond (#40121) (#40150) 
pick from master #40121
 2024-08-29 23:41:51 +08:00
Gabriel
fe881f8676 [fix](core) Fix BUCKET_HASH_SHUFFLE local exchanger (#40149) 
## Proposed changes

pick #40148

<!--Describe your changes.-->
 2024-08-29 23:23:42 +08:00
zclllhhjj
090474a815 [branch-2.1](load) Lower lock granularity to reduce loading CPU pressure (#40135) 
pick https://github.com/apache/doris/pull/40134
 2024-08-29 23:22:58 +08:00
daidai
a7156ee775 [fix](parquet)Fix the be core issue when reading parquet unsigned types. (#39926) (#40123) 
bp #39926
 2024-08-29 21:52:52 +08:00
Gabriel
841565a40d [fix](local shuffle) Fix hash shuffle local exchanger (#40101) 
## Proposed changes

pick #40036

<!--Describe your changes.-->
 2024-08-29 16:52:28 +08:00
Socrates
c6df7c21a3 [branch-2.1](hive) support hive write text table (#38549) (#40063) 
1. Support write hive text table
2. Add SessionVariable `hive_text_compression` to write compressed hive
text table
3. Supported compression type: gzip, bzip2, snappy, lz4, zstd

pick from https://github.com/apache/doris/pull/38549
 2024-08-29 16:50:40 +08:00
zhiqiang
901344a880 [chore](log) Log written rows when result sink close #39647 (#40011) 
cherry pick from #39647
 2024-08-29 16:49:44 +08:00
xzj7019
2054cc9fc9 [fix](nereids) fix union all instance number (#40099) 
pick from https://github.com/apache/doris/pull/39999

Co-authored-by: xiongzhongjian <xiongzhongjian@selectdb.com>
 2024-08-29 16:47:55 +08:00
daidai
ac8cff34ce [fix](ut)fix be enable_http_auth ut (#40071) (#40088) 
bp #40071 

## Proposed changes
before pr #39577
 2024-08-29 16:42:53 +08:00
Luwei
51e8d9174d [fix](compaction) fix heap-use-after-free caused by compaction sample info resize (#40058) (#40077) 2024-08-29 16:42:35 +08:00
bobhan1
eb6a040a03 [branch-2.1] Picks "[Fix](partial update) Fix wrongly update autoinc column in partial update #39996" (#40073) 
## Proposed changes

pick https://github.com/apache/doris/pull/39996
 2024-08-29 16:41:51 +08:00