9b7af4c0cf
[feature](schema change) unified schema change for parquet and orc reader ( #32873 )
...
Following #25138 , unified schema change interface for parquet and orc reader, and can be applied to other format readers as well.
Unified schema change interface for all format readers:
- First, read the data according to the column type of the file into source column;
- Second, convert source column to the destination column with type planned by FE.
2024-04-12 15:09:25 +08:00
5f30463bb3
[Chore](descriptors) remove unused codes for descriptors ( #33408 )
...
remove unused codes for descriptors
2024-04-12 15:09:25 +08:00
26d9082b9a
[Feature](function) Add function strcmp ( #33272 )
2024-04-12 15:09:25 +08:00
31984bb4f0
[feature](function) support quote string function #33055
2024-04-12 15:09:25 +08:00
5688c28364
[Bug](runtime-filter) try to fix heap use after free on runtime filter send filter size ( #33465 ) ( #33522 )
2024-04-11 13:10:24 +08:00
ef26479282
[improve](serde) support complex type in write/read pb serde ( #33124 )
...
support complex type and ip/jsonb in DataTypeSerDe::write_column_to_pb/read_column_from_pb function
2024-04-11 09:31:50 +08:00
8fd6d4c41b
[Chore](build) add -Wconversion and remove some unused code ( #33127 )
...
add -Wconversion and remove some unused code
2024-04-10 15:26:08 +08:00
c61d6ad1e2
[Feature] support function uuid_to_int and int_to_uuid #33005
2024-04-10 14:53:56 +08:00
bf022f9d8d
[enhancement](function truncate) truncate can use column as scale argument ( #32746 )
...
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2024-04-10 14:53:56 +08:00
cf7595d423
[opt](memory) Optimize mem tracker accuracy ( #32039 ) ( #33140 )
2024-04-10 11:42:19 +08:00
950ca68fac
[fix](move-memtable) fix timeout to get tablet schema ( #33256 ) ( #33260 )
2024-04-04 21:45:55 +08:00
df197c6a14
[fix](move-memtable) fix initial use count of streams for auto partition ( #33165 ) ( #33236 )
...
Co-authored-by: Kaijie Chen <ckj@apache.org >
2024-04-03 20:31:29 +08:00
ff0da8108b
[fix](RF) fix 'Invalid value' error of RF of decimal type ( #32749 )
2024-03-25 22:34:19 +08:00
d7a3ff1ddf
[Fix](Outfile) Fix the column type mapping in the orc/parquet file format ( #32281 )
...
| Doris Type | Orc Type | Parquet Type |
|---------------------|--------------------|------------------------|
| Date | Long (logical: DATE) | int32 (Logical: Date) |
| DateTime | TIMESTAMP (logical: TIMESTAMP) | int96 |
2024-03-22 08:52:16 +08:00
7486e96b12
[improve](function) add error msg if exceeded maximum default value in repeat function ( #32219 )
...
add some error msg from repeat function, so the user could know the count is greater than default value.
2024-03-21 14:07:49 +08:00
2196c534e8
[fix](group commit) Fix compatibility issues on serializing and deserializing wal file ( #32299 )
2024-03-21 14:07:24 +08:00
0990014e94
[fix](datetime) fix datetime rounding on BE ( #32075 )
2024-03-21 14:07:19 +08:00
ef2151ae66
[Feature-WIP](multi-catalog) Add Hive sink on BE side. ( #32306 ) ( #32364 )
...
bp #32306
Co-authored-by: Qi Chen <kaka11.chen@gmail.com >
2024-03-18 11:23:01 +08:00
0da010603e
[Improve](TabletSchemaCache) reduce duplicated memory consumption for column name and column path ( #31141 )
...
Both could be reference to related field in TabletColumn.And use shared_ptr for TabletColumn in TabletSchema for later memory reuse
2024-03-09 19:44:42 +08:00
8f77e6363a
[Feature](function) Support xxhash function like murmur hash function ( #31193 )
2024-02-23 19:03:28 +08:00
ad07dec0ed
[Improve](InPredict) enhance in predict with struct type ( #30840 )
2024-02-22 13:01:49 +08:00
b66583551c
[fix](group_commit)Fix bound checking problem when reading wal block ( #31112 )
2024-02-22 13:01:48 +08:00
f2a38e6345
[chore](columns) remove update_hashes_with_value for SipHash ( #31224 )
2024-02-22 13:01:48 +08:00
7a1bd6abb0
[improvment](group_commit) Refector scan wal function ( #30939 )
...
Co-authored-by: Yongqiang YANG <98214048+dataroaring@users.noreply.github.com >
2024-02-20 09:12:38 +08:00
bb4575a392
[Improvement](join) optimization for build_side_output_column ( #30826 )
...
optimization for build_side_output_column
2024-02-19 17:22:03 +08:00
6cf7468073
[enhancement](function) change some function nullable mode ( #30991 )
...
change some function nullable mode
2024-02-18 14:45:25 +08:00
68102fd531
[Fix](auto-partition) fix a concurrent bug of extremely long values ( #31005 )
2024-02-18 14:45:25 +08:00
45b4189bb6
[Refactor](opt) Opt rf and remove unless code ( #30900 )
...
Opt rf and remove unless code
2024-02-18 11:50:16 +08:00
8ff8d94697
[fix](ip) change IPv6 to little-endian byte order storage (like IPv4) ( #30730 )
2024-02-05 21:56:57 +08:00
3315c16383
[enhance](function) refactor from_format_str and support more format ( #30452 )
2024-02-01 19:08:37 +08:00
02a27a587a
remove some unused member function of IFunctionBase ( #30260 )
2024-01-24 09:59:45 +08:00
f7a340a2df
[improve](move-memtable) add cancel method to load stream stub ( #29994 )
2024-01-16 20:23:09 +08:00
d494674ff4
[opt](parquet-reader) Opt parquet decimal type reading. ( #29825 )
2024-01-12 13:58:19 +08:00
0d691c638b
[Feature](profile)Support report runtime workload statistics #29591
2024-01-12 11:59:27 +08:00
7c7dbf15bc
[feature](merge-cloud) Decouple Tablet/TabletManager/TxnManager from global StorageEngine instance ( #29736 )
2024-01-12 11:57:16 +08:00
fc4ca712ed
[bugfix](core) using weak ptr in data stream receiver to avoid runtime state is deconstructed ( #29410 )
2024-01-12 11:48:39 +08:00
7287c0ca15
[Opt](exec)(multi-catalog) Opt date type reading. ( #29571 )
2024-01-12 11:48:39 +08:00
48f58510a8
[refactor](tabletwriter) make tablet writer's rpc callback safe, could exit any time ( #29684 )
...
---------
Co-authored-by: yiguolei <yiguolei@gmail.com >
2024-01-12 11:46:29 +08:00
c58d18147b
[refactor](resultwriter) rename append_block to write method is more meaningful ( #29635 )
...
---------
Co-authored-by: yiguolei <yiguolei@gmail.com >
2024-01-08 00:00:33 +08:00
0b731800a0
[enhancement](group_commit) refector wal manager code ( #29560 )
2024-01-07 18:54:41 +08:00
b0cac0014d
[enhance](FS) Improve FS error code ( #29432 )
2024-01-06 21:17:22 +08:00
e3c9f535dc
[refactor](wal) refactor some wal code ( #29434 )
2024-01-03 14:45:57 +08:00
69a01e0cf5
[improve](move-memtable) skip load stream stub close wait when cancel ( #29427 )
2024-01-02 23:35:50 +08:00
706463781c
[refactor](group commit) refactor group commit wal code ( #29375 )
2024-01-02 15:52:03 +08:00
b7487430da
Revert "[improve](move-memtable) cancel load rapidly when stream close wait ( #29322 )" ( #29371 )
...
This reverts commit bbf58c5aa42d40e66bc6ccc9ed91a4fcb4bdfff7.
2024-01-02 11:32:14 +08:00
bbf58c5aa4
[improve](move-memtable) cancel load rapidly when stream close wait ( #29322 )
2023-12-31 16:26:41 +08:00
7623b5cc31
[cleanup](move-memtable) remove namespace stream_load ( #27441 )
2023-12-30 20:08:23 +08:00
51cb15d032
[improve](move-memtable) cancel load immediately when back pressure in delta writer v2 ( #29280 )
2023-12-30 10:45:06 +08:00
a525d5c5a3
[refactor](decimal) change type name Decimal128 to Decimal128V2, Decimal128I to Decimal128V3 to avoid confusion ( #29265 )
...
change type name Decimal128 to Decimal128V2, Decimal128I to Decimal128V3 to avoid confusion
2023-12-29 10:11:44 +08:00
9ff8bd2e9c
[Enhancement](Wal)Support dynamic wal space limit ( #27726 )
2023-12-27 11:51:32 +08:00
