517e7a0567
[Fix](load) fix load channel leak when load exception occurs ( #30915 )
2024-02-16 10:12:23 +08:00
c4a8d5497d
[fix](gc) process exception while iteratoring directory ( #30850 )
2024-02-16 10:12:23 +08:00
b23a785775
[Fix](Variant) support materialize view for variant and accessing variant subcolumns ( #30603 )
...
* [Fix](Variant) support materialize view for variant and accessing variant subcolumns
1. fix schema change with path lost and lead to invalid data read
2. support element_at function in BE side and use simdjson to parse data
3. fix multi slot expression
2024-02-16 10:12:23 +08:00
5b343911e8
[log](gc) add log for unused rowsets gc ( #30854 )
2024-02-16 10:12:23 +08:00
2667e10ba2
[improve](group-commit) Modify some log ( #30842 )
2024-02-16 10:12:23 +08:00
89695f43d4
[refactor](queryctx) move tg related code to task group ( #30829 )
...
init query ctx memtracker in queryctx constructor
set all task group related property during set taskgroup
2024-02-16 10:12:23 +08:00
2cb46eed94
[Feature](auto-inc) Add start value for auto increment column ( #30512 )
2024-02-16 10:12:23 +08:00
847db2c015
[Enhancement](group commit) Add retry message for group commit load while schema changing ( #30391 )
2024-02-16 10:12:23 +08:00
73940f96d3
[opt](string_to_unsigned_int) performance opt ( #30825 )
2024-02-05 22:23:16 +08:00
1ed24117ac
[function](url_decode)add url_decode function ( #30667 )
2024-02-05 22:23:00 +08:00
09ef78402e
fix build error
2024-02-05 22:15:17 +08:00
d123abc903
disable check segment when build rowset meta by default ( #30857 )
2024-02-05 22:00:36 +08:00
2c99c53812
[refactor](taskqueue) remove old task scheduler based wg ( #30832 )
...
---------
Co-authored-by: yiguolei <yiguolei@gmail.com >
2024-02-05 22:00:27 +08:00
0d32aeeaf6
[improvement](load) Enable lzo & Remove dependency on Markus F.X.J. Oberhumer's lzo library ( #30573 )
...
Issue Number: close #29406
1. increase lzop version to 0x1040,
I set to 0x1040 only for decompressing lzo files compressed by higher version of lzop,
no change of decompressing logic,
actully, 0x1040 should have "F_H_FILTER" feature,
but it mainly for audio and image data, so we do not support it.
2. use orc::lzoDecompress() instead of lzo1x_decompress_safe() to decompress lzo data
3. use crc32c::Extend() instead of lzo_crc32()
4. use olap_adler32() instead of lzo_adler32()
5. thus, remove dependency of Markus F.X.J. Oberhumer's lzo library
6. remove DORIS_WITH_LZO, so lzo file are supported by stream and broker load by default
7. add some regression test
2024-02-05 22:00:24 +08:00
499fd27ed0
[config](move-memtable) set StreamWait timeout default to 10min ( #30831 )
2024-02-05 21:59:55 +08:00
2344aaf337
[fix](join) JoinHashTable::pre_build_idxs should be const ( #30837 )
2024-02-05 21:59:55 +08:00
be31b8dc61
[Refactor](exchange) remove unless code in exchange and opt some code ( #30813 )
2024-02-05 21:59:52 +08:00
4b42156fc0
[chore](clang-tidy): add bugprone linters ( #29521 )
...
This PR introduces 4 bugprone linter rules to .clang-tidy, these linters found some bugs in #28965 . This PR also add some comments to mute false positive reports.
2024-02-05 21:58:08 +08:00
255ca143f8
[fix](chinese) fix the issue where the be crashes due to the missing chinese dict ( #30712 )
2024-02-05 21:57:29 +08:00
8ff8d94697
[fix](ip) change IPv6 to little-endian byte order storage (like IPv4) ( #30730 )
2024-02-05 21:56:57 +08:00
cd939fcca2
[Enhancement](group commit) Optimize group commit block sink wal disk space log #30811
2024-02-05 21:56:57 +08:00
48aaaa8005
[Enhancement](fuction) change function REPEAT nullable mode ( #30743 )
2024-02-04 22:21:36 +08:00
aed858a442
[improve](log) print query_id when fold constant on BE ( #30802 )
2024-02-04 22:21:36 +08:00
e891a095e7
check segment num when build rowset meta ( #30803 )
2024-02-04 18:15:12 +08:00
91a669f5fd
[chore](mac compile) remove using regex to avoid mac compile failed frequently #30783
2024-02-04 14:28:38 +08:00
1d39e16eda
[Bug](compaction) pass arena to function->add_batch_range ( #30709 )
2024-02-04 14:28:38 +08:00
6442663735
[Function](exec) upport atan2 math function ( #30672 )
...
Co-authored-by: Rohit Satardekar <rohitrs1983@gmail.com >
2024-02-04 14:28:38 +08:00
c9ab243153
[feat-wip](join) support mark join for right semi join(without mark join conjunct) ( #30767 )
2024-02-04 14:28:38 +08:00
6289f7e605
[Fix](multi-catalog) Fix truncate_char_or_varchar_column crash. ( #30731 )
2024-02-03 20:26:04 +08:00
0c7c8240ee
[Fix](ser) serialize bitmap/hll some not fixed length object maybe over INT_MAX ( #30773 )
2024-02-03 20:26:04 +08:00
4f8730d092
[improvement](jdbc catalog) Optimize connection pool parameter settings ( #30588 )
...
This PR makes the following changes to the connection pool of JDBC Catalog
1. Set the maximum connection survival time, the default is 30 minutes
- Moreover, one-half of the maximum survival time is the recyclable time,
- One-tenth is the check interval for recycling connections
2. Keepalive only takes effect on the connection pool on BE, and will be activated based on one-fifth of the maximum survival time.
3. The maximum number of existing connections is changed from 100 to 10
4. Add the connection cache recycling thread on BE, and add a parameter to control the recycling time, the default is 28800 (8 hours)
5. Add CatalogID to the key of the connection pool cache to achieve better isolation, requires refresh catalog to take effect
6. Upgrade druid connection pool to version 1.2.20
7. Added JdbcResource's setting of default parameters when upgrading the FE version to avoid errors due to unset parameters.
2024-02-03 20:26:03 +08:00
5687ca977d
[Bug](java-udf) fix core dump when javaudf input 0 row block ( #30720 )
...
fix core dump when javaudf input 0 row block
2024-02-03 20:25:25 +08:00
bc8c1875ac
[enhencement](config) change default write buffer size ( #30749 )
2024-02-03 20:25:25 +08:00
79a83e9e1a
[fix](move-memtable) use signed integer when calculating remain ms ( #30750 )
2024-02-03 20:25:25 +08:00
0e2d2f6169
[Bug](pipeline) fix pipeline task execute without wait second start rpc ( #30659 )
2024-02-03 20:24:58 +08:00
0f47f7f389
[Feature](runtime filter) normalize ignore runtime filter ( #30152 )
...
normalize ignore runtime filter
2024-02-03 20:24:39 +08:00
2ca911fb5d
[revert](move-memtable) Revert enable brpc debug log in regression pipelines ( #30389 ) ( #30611 )
...
This reverts commit 4bf47e229f930714572d8f91d6f9e94b4608bd20.
2024-02-02 13:31:47 +08:00
94eedd8ea4
[Enhancement](function)make SUBSTRING_INDEX function DEPEND_ON_ARGUMENT ( #30392 )
2024-02-02 13:31:47 +08:00
82bb3ed50f
[Fix](group commit) Fix pre allocated err handling for group commit async load and add regression test #30718
2024-02-02 13:31:47 +08:00
cd65a8c9a7
Remove useless statistics report path ( #30687 )
2024-02-01 23:14:14 +08:00
c1c633dc1f
[opt](exec) change string ser der way to improve performance ( #30693 )
2024-02-01 23:14:14 +08:00
82aa304706
[Opt](exec) opt the repeat node code ( #30683 )
2024-02-01 23:14:14 +08:00
3315c16383
[enhance](function) refactor from_format_str and support more format ( #30452 )
2024-02-01 19:08:37 +08:00
65e277e365
[refacotr](node) refactor partition sort node to improve readability ( #30511 )
...
* [refacotr](node) refactor partition sort node to improve readability
* update
2024-02-01 19:01:08 +08:00
203daba19d
[fix](outfile) fix outfile csv did not write json column with string ( #29067 )
2024-02-01 19:01:08 +08:00
822f2b1255
[improve](stream-load) add observability on receiving HTTP request #30432
2024-02-01 19:01:08 +08:00
5a0764b288
[log](rpc) print log when offer_failed in internal service ( #30454 )
2024-02-01 19:01:08 +08:00
1ac5b45180
[fix](invert index) fixed the issue of insufficient index idx generation during partial column updates. ( #30678 )
2024-02-01 19:01:08 +08:00
1d5471f805
[chore](debug) check the const exprs count in union node ( #30679 )
2024-02-01 19:00:51 +08:00
2a81ab70e4
[Enhancement](group commit) Optimize WAL insufficient space error messages ( #30614 )
2024-02-01 19:00:50 +08:00
