f565f60bc3
[refactor](standard)BE:Initialize pointer variables in the class to nullptr by default ( #27587 )
2023-11-28 13:02:30 +08:00
baadc14e60
[Enhancement](function) support unix_timestamp with float ( #26827 )
...
---------
Co-authored-by: YangWithU <plzw8@outlook.com >
2023-11-27 09:58:53 +08:00
b580ee91ce
[fix](compile) fix macOS compile and format code ( #27494 )
2023-11-23 23:24:10 +08:00
2ea33518b0
[Opt](load) use batching to optimize auto partition ( #26915 )
...
use batching to optimize auto partition
2023-11-23 19:12:28 +08:00
b457856bd2
[chore](be) remove bthread scanner related codes ( #27417 )
2023-11-23 15:18:49 +08:00
604842526b
[improvement](expr) improve if expr performance ( #27296 )
2023-11-22 12:48:06 +08:00
a4d78682ff
[Optimize](point query) clear names to reduce mem consumption and cpu cost related to block column name ( #26931 )
2023-11-17 10:18:21 +08:00
3ad865fef9
[refactor](storage) Expressing the types of computation layer and storage layer in PrimitiveTypeTraits ( #26191 )
2023-11-15 21:34:49 +08:00
a5565f68b2
[Refactor](opentelemetry) Remove opentelemetry ( #26605 )
2023-11-09 18:05:34 +08:00
a3666aa87e
[feature](decimal) support decimal256 when creating table ( #26308 )
2023-11-08 15:21:01 +08:00
44b51bf0b9
[Feature](Variant) support variant load ( #26572 )
2023-11-08 00:37:57 -06:00
462db3d5be
[fix](compile) fix macos compile error ( #25944 )
2023-10-31 11:44:56 +08:00
6761dc4113
[coverage](test) improve test coverage ( #26096 )
...
improve test coverage
2023-10-30 18:01:55 +08:00
c1d64a7128
[Feature](datatype) Add IPv4/v6 data type for doris ( #24965 )
2023-10-26 17:33:28 +08:00
693982fd1a
[feature](decimal) support decimal256 ( #25386 )
2023-10-25 15:47:51 +08:00
642c149e6a
remove datetime_value and move vecdatetime_value to doris namespace ( #25695 )
...
remove datetime_value and move vecdatetime_value to doris namespace
2023-10-20 22:08:17 +08:00
d0cd535cb9
[improvement](insert) refactor group commit stream load ( #25560 )
2023-10-20 13:27:30 +08:00
be27d4d921
[fix](broker-load) fix use_count() issue when doing broker load in debug mode ( #25288 )
...
When executing broker load in ASAN mode, BE may crash with error:
```
F20231010 18:18:17.044978 185490 block.cpp:694] Check failed: d.column->use_count() == 1 (3 vs. 1)
*** Check failure stack trace: ***
@ 0x55e9d94c4e46 google::LogMessage::SendToLog()
@ 0x55e9d94c1410 google::LogMessage::Flush()
@ 0x55e9d94c5689 google::LogMessageFatal::~LogMessageFatal()
@ 0x55e9c509f80d doris::vectorized::Block::clear_column_data()
@ 0x55e9b6c170b3 doris::PlanFragmentExecutor::get_vectorized_internal()
@ 0x55e9b6c147e6 doris::PlanFragmentExecutor::open_vectorized_internal()
@ 0x55e9b6c12d9a doris::PlanFragmentExecutor::open()
@ 0x55e9b6c18426 doris::PlanFragmentExecutor::execute()
@ 0x55e9b6945cca doris::FragmentMgr::_exec_actual()
@ 0x55e9b696456c doris::FragmentMgr::exec_plan_fragment()::$_0::operator()()
```
It may happen when there is column maping like:
```
(k1,v2,v3,v4,v5,v6,v7,v8)
set (k2=v4,k3=v4,k4=v4)
```
in load stmt.
Case is covered by Baidu test cases
2023-10-12 17:04:29 +08:00
bb670118f5
[coverage](test) Delete unused function to improve test coverage ( #25233 )
2023-10-11 11:50:51 +08:00
642e5cdb69
[Fix](Status) Make Status [[nodiscard]] and handle returned Status correctly ( #23395 )
2023-09-29 22:38:52 +08:00
d31d99bf34
[pipeline](load) opt the pipeline load code ( #24708 )
...
opt the pipeline load code
2023-09-21 15:20:31 +08:00
c3b3f0f00a
[enhancement](serialize) add dcheck to ensure pb type is set ( #24645 )
...
should check the pb's type is set, or the deserialize will core.
should not return unknown type because deserialize will core.
---------
Co-authored-by: yiguolei <yiguolei@gmail.com >
2023-09-20 10:42:28 +08:00
d8ef9dda59
[feature](merge-cloud) Rewrite FS interface ( #23953 )
2023-09-12 19:20:25 +08:00
d3f1388717
[Feature](partitions) Support auto-partition ( #24153 )
...
Co-authored-by: zhangstar333 <2561612514@qq.com >
2023-09-12 15:23:15 +08:00
0143ae8266
[fix]Add logging before _builtin_unreachable() ( #24101 )
...
Co-authored-by: 宋光璠 <songguangfan@sf.com >
2023-09-09 00:30:11 +08:00
82dc970916
[feature](insert) Support group commit insert ( #22829 )
2023-09-08 15:51:03 +08:00
fdb7a44f57
Revert "[Feature](partitions) Support auto partition" ( #24024 )
...
* Revert "[Feature](partitions) Support auto partition (#23236 )"
This reverts commit 6c544dd2011d731b8c9c51384c77bcf19c017981.
* Update config.h
2023-09-07 17:08:26 +08:00
6c544dd201
[Feature](partitions) Support auto partition ( #23236 )
...
Co-authored-by: zhangstar333 <2561612514@qq.com >
2023-09-06 16:26:45 +08:00
a96adc01aa
[Chore](function) refactor of quantile_state ( #23862 )
...
refactor of quantile_state
2023-09-06 15:39:19 +08:00
09bcedb116
[feature](merge-cloud) Remove deprecated old cache ( #23881 )
...
* Remove deprecated old cache
2023-09-06 08:07:05 +08:00
251e79d9f8
[Bug](compare) fix core dump on decimal128 when be is build by debug mode ( #23840 )
...
fix core dump on decimal128 when be is build by debug mode
2023-09-04 17:40:57 +08:00
62c075bf7e
[improvement](Block) Replace Block(const PBlock&) with deserialize because it has heavy operations in ctor ( #23672 )
2023-08-31 14:44:17 +08:00
1410a15a61
[fix](compaction) print column name when checking block ColumnPtr is nullptr on get block byte ( #23338 )
2023-08-29 17:24:48 +08:00
b96b8f4370
[Bug](jdbc) support get_default on complex type ( #23325 )
...
support get_default on complex type
2023-08-25 14:08:24 +08:00
8ed4045df9
[Chore](primitive-type) remove VecPrimitiveTypeTraits ( #22842 )
2023-08-23 08:37:40 +08:00
477961dc21
[Chore](agg) refactor of hash map ( #22958 )
...
refactor of hash map
2023-08-18 17:59:30 +08:00
3f55d5d4d5
[Chore](excution) change some log fatal and dcheck to exception ( #22890 )
...
change some log fatal and dcheck to exception
2023-08-15 10:45:00 +08:00
4e880288c6
[refactor]use clear concept to replace std::enable_if_t ( #22801 )
...
---------
Signed-off-by: flynn <fenglv15@mails.ucas.ac.cn >
2023-08-12 15:10:30 +08:00
56392e21ae
[Bug](decimalv3) fix decimalv3 keyrange set wrong number #22818
2023-08-10 18:15:40 +08:00
b25d52b736
[feature](cast) remove some unused in functioncast and support some function in nereids ( #22729 )
...
1 ConvertImplGenericFromString do not need a template StringColumnType
2 remove timev1 in function cast
3 support time_to_sec , sec_to_time in nereids
2023-08-10 10:10:32 +08:00
098bab7b30
[Bug](exchange) disable implicit conversion of block to bool ( #22534 )
...
disable implicit conversion of block to bool
2023-08-03 20:37:14 +08:00
5584d7a5ba
[Improve](point query) Improve lookup connection cache from DoubleBuffer to LRU cache for better item pruning ( #22041 )
2023-07-27 22:22:50 +08:00
8973610543
[feature](datetime) "timediff" supports calculating microseconds ( #21371 )
2023-07-10 19:21:32 +08:00
36524f2b72
[improvement](functions) avoid copying of block in create_block_with_nested_columns ( #21526 )
...
avoid copying of block in create_block_with_nested_columns
2023-07-10 17:21:23 +08:00
2678afd2db
[fix][improvement](fs) add HdfsIO profile and modification time ( #21638 )
...
Refactor the interface of create_file_reader
the file_size and mtime are merged into FileDescription, not in FileReaderOptions anymore.
Now the file handle cache can get correct file's modification time from FileDescription.
Add HdfsIO for hdfs file reader
pick from [Enhancement](multi-catalog) Add hdfs read statistics profile. #21442
2023-07-08 14:49:44 +08:00
242a35fa80
[fix](s3) fix s3 fs benchmark tool ( #21401 )
...
1. fix concurrency bug of s3 fs benchmark tool, to avoid crash on multi thread.
2. Add `prefetch_read` operation to test prefetch reader.
3. add `AWS_EC2_METADATA_DISABLED` env in `start_be.sh` to avoid call ec2 metadata when creating s3 client.
4. add `AWS_MAX_ATTEMPTS` env in `start_be.sh` to avoid warning log of s3 sdk.
2023-07-05 16:20:58 +08:00
f7c724f8a3
[Bug](excution) avoid core dump on filter_block_internal and add debug information ( #21433 )
...
avoid core dump on filter_block_internal and add debug information
2023-07-03 18:10:30 +08:00
2e6d91aa99
[chore](block) temporarily disable DCHECK for column name equality in MutableBlock ( #21116 )
...
* tempororyly disable DCHECK for column name equality in MutableBlock::add_rows
* num columns EQ to LE
2023-06-26 10:49:27 +08:00
2c11ce0a02
[bugfix](topn) fix key topn merge block conflict with index predicate result columns ( #20820 )
2023-06-20 21:23:00 +08:00
08fff8923f
[improvement](serde) Optimizing the performance of mysql result writter ( #20928 )
...
When converting query results into MySQL format, it involves transforming columnar data storage into row-based storage. This process raises the question of choosing between sequential reading and sequential writing. In reality, sequential writing is the better choice for performance optimization.
Test with 9M rows contains more than 20 columns, this patch can reduce the conversion time from 20s to 11s.
2023-06-19 12:29:01 +08:00
