c51146df10
[Fix](segment) need to rebuild col_id_to_predicates when true predicates encountered ( #25685 )
2023-10-22 21:26:52 -05:00
2e2d5bcba2
[Improvements](status) catch some error status ( #25677 )
...
catch some error status
2023-10-23 10:19:08 +08:00
305c884478
[fix](rf) fix minmax runtime filter ( #25720 )
...
Template arguments for decimalv3 type is wrong when creating MinMaxNumFunc, which result in wrong decimal type like vectorized::Decimal<doris::vectorized::Decimal128I> :
class MinmaxFunctionTraits {
public:
using BasePtr = MinMaxFuncBase*;
template <PrimitiveType type, size_t N>
static BasePtr get_function() {
return new MinMaxNumFunc<std::conditional_t<
type == TYPE_DECIMAL32 || type == TYPE_DECIMAL64 || type == TYPE_DECIMAL128I,
vectorized::Decimal<typename PrimitiveTypeTraits<type>::CppType>,
typename PrimitiveTypeTraits<type>::CppType>>();
}
};
(gdb) bt
#0 doris::vectorized::Decimal<__int128>::operator<=>(doris::vectorized::Decimal<__int128> const&) const (x=..., this=<optimized out>) at /mnt/disk2/user/doris-1.1/be/src/vec/core/types.h:381
#1 doris::vectorized::Decimal<doris::vectorized::Decimal128I>::operator<=>(doris::vectorized::Decimal<doris::vectorized::Decimal128I> const&) const (x=..., this=<optimized out>)
at /mnt/disk2/user/doris-1.1/be/src/vec/core/types.h:381
#2 doris::MinMaxNumFunc<doris::vectorized::Decimal<doris::vectorized::Decimal128I> >::insert (this=0x7ef19de903c0, data=0x7c612928d860) at /mnt/disk2/user/doris-1.1/be/src/exprs/minmax_predicate.h:61
#3 0x0000558465c2844a in doris::RuntimePredicateWrapper::insert (this=0x7f47a2e3d720, value=...) at /mnt/disk2/user/doris-1.1/be/src/exprs/runtime_filter.cpp:492
#4 doris::RuntimePredicateWrapper::insert_batch (this=0x7f47a2e3d720, column=..., rows=...) at /mnt/disk2/user/doris-1.1/be/src/exprs/runtime_filter.cpp:504
#5 0x0000558465c1f6c3 in doris::IRuntimeFilter::insert_batch (this=<optimized out>, column=..., rows=...) at /mnt/disk2/user/doris-1.1/be/src/exprs/runtime_filter.cpp:1060
#6 0x000055846883dafd in doris::VRuntimeFilterSlots::insert (this=0x7d7e767f4e70, datas=...) at /mnt/disk2/user/doris-1.1/be/src/exprs/runtime_filter_slots.h:188
#7 0x0000558468840535 in doris::vectorized::ProcessRuntimeFilterBuild::operator()<doris::vectorized::MethodOneNumber<doris::vectorized::UInt128, PartitionedHashMapTable<HashMapTable<doris::vectorized::UInt128, HashMapCell<doris::vectorized::UInt128, doris::vectorized::RowRefList, HashCRC32<doris::vectorized::UInt128>, HashTableNoState>, HashCRC32<doris::vectorized::UInt128>, PartitionedHashTableGrower<8ul>, Allocator<true, true, false> > > >, doris::vectorized::HashJoinNode> (
this=this@entry=0x7e7c6a5bee40, state=0x7ef19df00400, hash_table_ctx=..., parent=0x7ef19dee3c00) at /mnt/disk2/user/doris-1.1/be/src/vec/exec/join/vhash_join_node.h:92
2023-10-23 10:16:28 +08:00
9519d7ede9
[enhancement](be-ut)Add more indexed column reader be unit test ( #25652 )
...
Added more unit tests
1. key exists or does not exist in a single page
2. key exists or does not exist in multiple pages
3. key is between two pages.
2023-10-23 10:12:53 +08:00
fbc448520a
[feature](ColdHeatSeperation) Support to upload cold data to HDFS ( #22048 )
2023-10-22 21:04:43 +08:00
642c149e6a
remove datetime_value and move vecdatetime_value to doris namespace ( #25695 )
...
remove datetime_value and move vecdatetime_value to doris namespace
2023-10-20 22:08:17 +08:00
a45c0d7b4f
[fix](log) fix the problem of parameter mismatch when printing logs ( #25649 )
2023-10-20 19:51:41 +08:00
fded8ba824
[improvement](insert) Modify insert into table_id(xxx) to doris_inter… ( #25662 )
2023-10-20 19:31:37 +08:00
a6925cc0cf
Fix exchange operator can not aware end of file ( #25562 )
2023-10-20 18:56:01 +08:00
7f1e3e48cd
[Fix](Tablet) Fix Tablet::_meta_lock shadows member inherited from BaseTablet ( #25693 )
2023-10-20 18:36:20 +08:00
2e97044706
[fix](move-memtable) fix inverted index condition ( #25684 )
2023-10-20 17:37:39 +08:00
a2ceea5951
[refactor](jni) unified jni framework for java udaf ( #25591 )
...
Follow https://github.com/apache/doris/pull/25302 , and use the unified jni framework to refactor java udaf.
This PR has removed the old interfaces to run java udf/udaf. Thanks to the ease of use of the new framework, the core code for modifying UDAF does not exceed 100 lines, and the logic is similar to that of UDF.
2023-10-20 16:13:40 +08:00
26f8c7e352
[fix](Nereids) collect_list and collect_set should always not null ( #25592 )
2023-10-20 01:54:00 -05:00
925233a271
[pipelineX](fix) Fix concurrent bug in schema scanner ( #25655 )
2023-10-20 14:29:42 +08:00
d0cd535cb9
[improvement](insert) refactor group commit stream load ( #25560 )
2023-10-20 13:27:30 +08:00
9a675fcdfc
[chore](be) Add default timezone files ( #25097 )
2023-10-20 13:12:24 +08:00
dc47087560
[fix](function) fix str_to_date default return type scale for nereids ( #24932 )
...
fix str_to_date default return type scale for nereids
2023-10-20 12:55:49 +08:00
a42536efc1
[feature](pipelineX) release dependency when task close ( #25633 )
2023-10-19 19:42:50 +08:00
8b5fd61632
[pipelineX](profile) Add pending finish dependency timer ( #25635 )
2023-10-19 19:28:45 +08:00
b964ab76b3
[refactor](shuffle) Simplify hash partitioning strategy ( #25596 )
2023-10-19 19:28:22 +08:00
fa5dd61563
[feature](pipelineX) operator id use negative numbers ( #25529 )
2023-10-19 19:27:44 +08:00
b0e70bd02b
[fix](pipelineX) fix error set delta_urls ( #25631 )
2023-10-19 19:27:31 +08:00
7385602b19
[bug](rf) fix only min/max rf return error when has remote target ( #25588 )
2023-10-19 19:26:29 +08:00
54780c62e0
[improvement](executor)Using cgroup to implement cpu hard limit ( #25489 )
...
* Using cgroup to implement cpu hard limit
* code style
2023-10-19 18:56:26 +08:00
d0d9c4267d
[fix](move-memtable) fix use-after-free in load stream stub ( #25618 )
2023-10-19 16:11:35 +08:00
159be51ea6
[bugfix](schema_change) Fix the coredump when doubly write during schema change ( #22557 )
2023-10-19 14:43:18 +08:00
e77b98be88
[fix](months_diff) fix wrong result of months_diff ( #25577 )
2023-10-19 14:29:47 +08:00
3d1206d325
[date](fix) modify push-down predicate for datev1 type ( #25571 )
...
For comparison predicate, two arguments must be cast to datetime and push down to storage if either one is date type. This PR disables predicate push-down for this case.
2023-10-19 14:18:27 +08:00
63c89df474
[enhencement](RowsetWriter) Don't delete files when beta rowset writer destructed ( #25578 )
2023-10-19 09:37:04 +08:00
dbf5787682
[fix](be) Make DorisCallOnce's function exception-safe ( #25579 )
2023-10-18 22:13:30 +08:00
11fecafb74
[fix](move-memtable) fallback if target table contains inverted index ( #25498 )
2023-10-18 22:11:59 +08:00
32fc8a1799
[chore](compaction) Do not print the stack trace when the compaction task already exists ( #25597 )
2023-10-18 21:44:17 +08:00
c21eb315b0
[feature](thrift api) support expr in MemoryScratchSink and make arrow::Schema recalculate with block info ( #24603 )
2023-10-18 07:51:56 -05:00
9c9fc84f39
[feature](merge-cloud) Abstract BaseTablet for CloudTablet ( #24929 )
2023-10-18 20:29:04 +08:00
e4a83a22d1
[opt](error msg) Make data codec error clearly when load csv data can't display ( #25540 )
...
Co-authored-by: Tanya-W <tanya1218w@163,com>
2023-10-18 16:12:22 +08:00
80e5e72202
[fix](scanner) coredump caused by 'prune_predicates_by_zone_map' ( #25555 )
2023-10-18 16:11:41 +08:00
d2400d1d7b
[feature](profile) profilev2 distinguish Sink and Operator in pipelineX ( #25491 )
...
* update
* update
2023-10-18 13:12:29 +08:00
6cb947f72b
[refactor](unused code) delete unused method from field.h ( #25554 )
...
Co-authored-by: yiguolei <yiguolei@gmail.com >
2023-10-18 13:11:14 +08:00
64aeeb971b
[Fix](partial-update) Correct the alignment process when the table has sequence column and add cases ( #25346 )
...
This PR fix the alignment process during publish phase when conflict occurs during concurrent partial updates: if we encounter a row with the same key and larger value in sequence column, it means that there exists another load which introduces a row with the same keys and larger sequence column value published successfully after the commit phase of the current load. We should act as follows:
- If the columns we update include sequence column, we should delete the current row becase the partial update on the current row has been overwritten by the previous one with larger sequence column value.
- Otherwise, we should combine the values of the missing columns in the previous row and the values of the including columns in the current row into a new row.
2023-10-18 11:32:51 +08:00
b0e0a0569a
[Fix](row store) Real default value should be used instead of default… ( #25230 )
...
Before this PR the default value is not correct, we should use default value in Frontend schema.
2023-10-18 10:13:44 +08:00
47689fd452
[refactor](jni) unified jni framework for java udf ( #25302 )
...
Use the unified jni framework to refactor java udf.
The unified jni framework takes VectorTable as the container to transform data between c++ and java, and hide the details of data format conversion.
In addition, the unified framework supports complex and nested types.
The performance of basic types remains consistent, with a 30% improvement in string types and an order of magnitude improvement in complex types.
2023-10-18 09:27:54 +08:00
18c2a13e09
[fix](multi-catalog)fix maxcompute partition filter and session creation ( #24911 )
...
add maxcompute partition support
fix maxcompute partition filter
modify maxcompute session create method
2023-10-17 22:36:10 +08:00
b74836050a
[chore](config) turnoff fuzzy for enable_simdjson_reader ( #25521 )
2023-10-17 18:42:11 +08:00
06ff59bc03
[Performance](sink) SIMD the tablet sink valied data function ( #25480 )
2023-10-17 16:21:08 +08:00
31a5e072e7
[refactor](pipelineX) Simplify set operation ( #25502 )
2023-10-17 15:11:46 +08:00
1514f78b87
[refactor](partial-update) Split partial update infos from tablet schema ( #25147 )
2023-10-17 14:21:40 +08:00
c2fe34dec7
[refine](pipelineX) refactor local state ( #25448 )
2023-10-17 11:23:29 +08:00
5f844486e3
[enhancement](invert index) read columns by index reduce seek time ( #24735 )
2023-10-17 10:34:33 +08:00
ef7d8aa99a
[fix](be)confix bug of converting outer join probe block to nullable ( #25492 )
...
_do_evaluate will add temp result column into original table block, so in order to only convert correct columns to be nullable, need call convert_block_to_null before _do_evaluate
2023-10-17 10:10:56 +08:00
cda8fb6b8b
[fix](load) return Status when error in RowsetWriter::build ( #25381 )
2023-10-17 09:40:23 +08:00
