74e452f19c
[bug](bitmap) fix bitmap value copy operator not call reset ( #26451 )
...
when a empty bitmap assign to other bitmap
the other bitmap should reset self firstly, and then set empty type.
2023-11-09 10:05:09 +08:00
58bf79f79e
[fix](move-memtable) pass load stream num to backends ( #26198 )
2023-11-08 16:16:33 +08:00
519b48648e
[fix](move-memtable) handle status when possible ( #26526 )
2023-11-08 10:09:06 +08:00
1e2a614a46
[fix](workflow) Fix failure test cases in BE UT (macOS) ( #26425 )
...
1. Fix memory issues in LoadStreamMgrTest.
2. Skip S3FileWriterTest by default because it depends on the environment in teamcity.
3. Fix VTimestampFunctionsTest.convert_tz_test.
2023-11-06 10:44:44 +08:00
a4e415ab09
[feature](hive)Support hive tables after alter type. ( #25138 )
...
1.Reconstruct the logic of decode to read parquet. The parquet reader first reads the data according to the parquet physical type, and then performs a type conversion.
2.Support hive alter table.
2023-11-02 00:24:21 +08:00
3e10e5af39
[Fix](Serde) Fix content displayed by complex types in MySQL Client ( #25946 )
...
This pr makes three changes to the display of complex types：
1. NULL value in complex types refers to being displayed as `null`, not `NULL`
2. struct type is displayed as "column_name": column_value
3. Time types such as `datetime` and `date`, are displayed with double quotes in complex types. like
`{1, "2023-10-26 12:12:12"}`
This pr also do a code refactor:
1. nesting_level is set to a member variable of the `DataTypeSerDe`, rather than a parameter in methods.
What's more, this pr fix a bug that fileSize is not correct, introduced by this pr: #25854
2023-11-01 23:48:55 +08:00
0449a240f4
[Fix](from_unixtime) Keep consistent with MySQL & bug fix ( #25966 )
...
Bug fix: implicit convert from int32 -> int64 makes negative time stamp valid, so change signature to int64
Consistent: keep consistent with mysql.
2023-10-31 14:31:24 +08:00
8f320944a8
[fix](move-memtable) fix DeltaWriterV2 profile use-after-free ( #26110 )
...
The sink who creates the delta writer may be closed while other sinks still using this delta writer.
The parent profile is deconstructed and when the last sink trying to update the profile, it will meet use-after-free.
To address this issue, we record the profile number in delta writer,
and the last sink who close the delta writer will create and update the profile.
2023-10-31 13:52:18 +08:00
da4de17d5c
[improvement](function) improve date_trunc function performance when timeunit is const ( #25824 )
...
this PR #22602 have check function.
only support date_trunc(column, const), so the second must be const literal
and no need to check time unit every row.
2023-10-26 09:51:21 +08:00
693982fd1a
[feature](decimal) support decimal256 ( #25386 )
2023-10-25 15:47:51 +08:00
9160834606
[FIX](resize) fix array and map offsets resize with default value ( #25669 )
2023-10-24 02:50:01 -05:00
08832d9f3a
[Fix](exec) Fix date dict dead loop. ( #25570 )
2023-10-24 02:51:43 +08:00
cbc5c91aec
[fix](datetime) fix unstable str_to_date function result ( #25707 )
...
fix unstable str_to_date function result
2023-10-23 11:52:08 +08:00
642c149e6a
remove datetime_value and move vecdatetime_value to doris namespace ( #25695 )
...
remove datetime_value and move vecdatetime_value to doris namespace
2023-10-20 22:08:17 +08:00
9a675fcdfc
[chore](be) Add default timezone files ( #25097 )
2023-10-20 13:12:24 +08:00
b964ab76b3
[refactor](shuffle) Simplify hash partitioning strategy ( #25596 )
2023-10-19 19:28:22 +08:00
9c9fc84f39
[feature](merge-cloud) Abstract BaseTablet for CloudTablet ( #24929 )
2023-10-18 20:29:04 +08:00
b0e0a0569a
[Fix](row store) Real default value should be used instead of default… ( #25230 )
...
Before this PR the default value is not correct, we should use default value in Frontend schema.
2023-10-18 10:13:44 +08:00
cda8fb6b8b
[fix](load) return Status when error in RowsetWriter::build ( #25381 )
2023-10-17 09:40:23 +08:00
7ea456ef91
[fix](insert) make group commit wal_manager exit elegantly ( #25250 )
2023-10-14 23:14:06 +08:00
6f9a084d99
[Fix](Outfile) Use data_type_serde to export data to parquet file format ( #24998 )
2023-10-13 13:58:34 +08:00
c6b1c903e4
[fix](Regression-test) fix that the String type in a nested type should contain double quotes and add regression-test ( #25115 )
2023-10-11 18:30:26 +08:00
9e31cb26bb
[fix](parse_url) fix parse_url is not working in some case to extract the HOST ( #25040 )
...
Issue Number: close #24452
2023-10-09 00:14:58 +08:00
b91335dbb8
[refactor](columndecimal) is_decimal_v2 member is useless because column decimal could detect by itself ( #25110 )
...
Co-authored-by: yiguolei <yiguolei@gmail.com >
2023-10-08 18:09:19 +08:00
0df32c8e3e
[Fix](Outfile) Use data_type_serde to export data to csv file format ( #24721 )
...
Modify the outfile logic, use the data type serde framework.
2023-10-07 22:50:44 +08:00
642e5cdb69
[Fix](Status) Make Status [[nodiscard]] and handle returned Status correctly ( #23395 )
2023-09-29 22:38:52 +08:00
68087f6c82
[fix](json function) Fix the slow performance of get_json_path when processing JSONB ( #24631 )
...
When processing JSONB, automatically convert to jsonb_extract_string
2023-09-27 21:17:39 +08:00
5d138b6928
[remove](function) make execute_impl const and remove running_difference function ( #24935 )
2023-09-27 18:17:28 +08:00
ded8ba108f
[test](be-ut) skip some be ut case ( #24917 )
...
skip be ut "VTimestampFunctionsTest.convert_tz_test"
2023-09-26 19:51:15 +08:00
082bcd820b
[feature](insert) Support wal for group commit insert ( #23053 )
2023-09-26 14:46:24 +08:00
8191cd1dad
[Bug](ScanNode) Fix potential incorrect query result caused by concurrent NewOlapScanNode initialization and Compaction ( #24638 )
...
* Optimize fetch delete predicates
* Fix incorrect query result when compaction eliminate delete predicates between `NewOlapScanNode::_init_scanners` and `NewOlapScanner::init`
* Fix be ut
2023-09-25 22:24:35 +08:00
22616d125d
[function](bitmap) add function alias bitmap_andnot and bitmap_andnot_count ( #24771 )
2023-09-22 12:18:31 +08:00
85fb46bb71
[refactor](cache) Refactor preloaded timezone global cache ( #24694 )
...
Refactor preloaded timezone global cache
2023-09-21 17:26:41 +08:00
dc9fa1a4f1
[Refactor](Sink) convert to tablet sink to tablet writer ( #24474 )
2023-09-20 14:47:18 +08:00
8aea31e383
[fix](timezone) fix timezone parse when there is no tzfile ( #24578 )
2023-09-20 14:28:12 +08:00
4b5cea1ef8
[enhancement](fix)change ordinary type null value is \N,complex type null value is null ( #24207 )
2023-09-16 21:46:42 +08:00
268c867679
[Improve](serde)replace function_cast from_string to serde ( #24087 )
...
Now we can not support streamload with column which is map/array nested map/array
serde can do this now , so we can replace it
Notice. if item data in complex type data is empty we just return error, instead of makeup default value , because now we can not define right default for complex type
2023-09-14 13:53:16 +08:00
563c3f75ff
[feature](move-memtable) share delta writer v2 among sinks ( #24066 )
2023-09-13 14:39:29 +08:00
c7ae2a7d22
[Refactor & Bugfix](static variables) move some static vairables to exec_env ( #24029 )
2023-09-13 09:27:03 +08:00
4bb9a12038
[function](bitmap) support bitmap_remove ( #24190 )
2023-09-12 14:52:04 +08:00
f9a75b5c4f
[feature](csv_serde)1.append csv serde for serialize to csv and deserialize from csv. 2.let csvReader use csv serde not text_converter. ( #23352 )
...
1. append csv serde for serialize to csv and deserialize from csv.
2. let csvReader use csv serde not text_converter.
2023-09-10 00:16:21 +08:00
2f8b075b71
[improvement](bitmap) support version for ser/deser of bitmap ( #23959 )
2023-09-07 09:55:29 +08:00
a96adc01aa
[Chore](function) refactor of quantile_state ( #23862 )
...
refactor of quantile_state
2023-09-06 15:39:19 +08:00
75e2bc8a25
[function](bitmap) support bitmap_to_base64 and bitmap_from_base64 ( #23759 )
2023-09-02 00:58:48 +08:00
eaf2a6a80e
[fix](date) return right date value even if out of the range of date dictionary( #23664 )
...
PR(https://github.com/apache/doris/pull/22360 ) and PR(https://github.com/apache/doris/pull/22384 ) optimized the performance of date type. However hive supports date out of 1970~2038, leading wrong date value in tpcds benchmark.
How to fix:
1. Increase dictionary range: 1900 ~ 2038
2. The date out of 1900 ~ 2038 is regenerated.
2023-09-01 14:40:20 +08:00
62c075bf7e
[improvement](Block) Replace Block(const PBlock&) with deserialize because it has heavy operations in ctor ( #23672 )
2023-08-31 14:44:17 +08:00
94a8fa6bc9
[bug](function) fix explode_number function return wrong rows ( #23603 )
...
before the explode_number function result is random with const value.
because the _cur_size is reset, so it's can't insert values to column.
2023-08-29 19:02:49 +08:00
5be8d57f52
[fix](be-ut) fix ColumnFixedLenghtObjectTest on 32 bits system ( #23519 )
2023-08-28 14:02:05 +08:00
f80b067990
[fix](column) add unimplemented function of ColumnFixedLengthObject ( #23468 )
2023-08-25 17:38:01 +08:00
9cacf9535a
[Opt](functions) Use preloaded cache to accelerate timezone parsing ( #22694 )
...
* opt
* bugfix
* fix ut
* fix stylecheck
2023-08-25 10:00:48 +08:00
