Commit Graph

6972 Commits

Author SHA1 Message Date
e891a095e7 check segment num when build rowset meta (#30803) 2024-02-04 18:15:12 +08:00
91a669f5fd [chore](mac compile) remove using regex to avoid mac compile failed frequently #30783 2024-02-04 14:28:38 +08:00
Pxl
1d39e16eda [Bug](compaction) pass arena to function->add_batch_range (#30709) 2024-02-04 14:28:38 +08:00
6442663735 [Function](exec) upport atan2 math function (#30672)
Co-authored-by: Rohit Satardekar <rohitrs1983@gmail.com>
2024-02-04 14:28:38 +08:00
c9ab243153 [feat-wip](join) support mark join for right semi join(without mark join conjunct) (#30767) 2024-02-04 14:28:38 +08:00
6289f7e605 [Fix](multi-catalog) Fix truncate_char_or_varchar_column crash. (#30731) 2024-02-03 20:26:04 +08:00
0c7c8240ee [Fix](ser) serialize bitmap/hll some not fixed length object maybe over INT_MAX (#30773) 2024-02-03 20:26:04 +08:00
4f8730d092 [improvement](jdbc catalog) Optimize connection pool parameter settings (#30588)
This PR makes the following changes to the connection pool of JDBC Catalog
1. Set the maximum connection survival time, the default is 30 minutes

-   Moreover, one-half of the maximum survival time is the recyclable time,
-   One-tenth is the check interval for recycling connections

2. Keepalive only takes effect on the connection pool on BE, and will be activated based on one-fifth of the maximum survival time.
3. The maximum number of existing connections is changed from 100 to 10
4. Add the connection cache recycling thread on BE, and add a parameter to control the recycling time, the default is 28800 (8 hours)
5. Add CatalogID to the key of the connection pool cache to achieve better isolation, requires refresh catalog to take effect
6. Upgrade druid connection pool to version 1.2.20
7. Added JdbcResource's setting of default parameters when upgrading the FE version to avoid errors due to unset parameters.
2024-02-03 20:26:03 +08:00
Pxl
5687ca977d [Bug](java-udf) fix core dump when javaudf input 0 row block (#30720)
fix core dump when javaudf input 0 row block
2024-02-03 20:25:25 +08:00
bc8c1875ac [enhencement](config) change default write buffer size (#30749) 2024-02-03 20:25:25 +08:00
79a83e9e1a [fix](move-memtable) use signed integer when calculating remain ms (#30750) 2024-02-03 20:25:25 +08:00
0e2d2f6169 [Bug](pipeline) fix pipeline task execute without wait second start rpc (#30659) 2024-02-03 20:24:58 +08:00
Pxl
0f47f7f389 [Feature](runtime filter) normalize ignore runtime filter (#30152)
normalize ignore runtime filter
2024-02-03 20:24:39 +08:00
2ca911fb5d [revert](move-memtable) Revert enable brpc debug log in regression pipelines (#30389) (#30611)
This reverts commit 4bf47e229f930714572d8f91d6f9e94b4608bd20.
2024-02-02 13:31:47 +08:00
94eedd8ea4 [Enhancement](function)make SUBSTRING_INDEX function DEPEND_ON_ARGUMENT (#30392) 2024-02-02 13:31:47 +08:00
82bb3ed50f [Fix](group commit) Fix pre allocated err handling for group commit async load and add regression test #30718 2024-02-02 13:31:47 +08:00
cd65a8c9a7 Remove useless statistics report path (#30687) 2024-02-01 23:14:14 +08:00
c1c633dc1f [opt](exec) change string ser der way to improve performance (#30693) 2024-02-01 23:14:14 +08:00
82aa304706 [Opt](exec) opt the repeat node code (#30683) 2024-02-01 23:14:14 +08:00
3315c16383 [enhance](function) refactor from_format_str and support more format (#30452) 2024-02-01 19:08:37 +08:00
65e277e365 [refacotr](node) refactor partition sort node to improve readability (#30511)
* [refacotr](node) refactor partition sort node to improve readability

* update
2024-02-01 19:01:08 +08:00
203daba19d [fix](outfile) fix outfile csv did not write json column with string (#29067) 2024-02-01 19:01:08 +08:00
822f2b1255 [improve](stream-load) add observability on receiving HTTP request #30432 2024-02-01 19:01:08 +08:00
5a0764b288 [log](rpc) print log when offer_failed in internal service (#30454) 2024-02-01 19:01:08 +08:00
1ac5b45180 [fix](invert index) fixed the issue of insufficient index idx generation during partial column updates. (#30678) 2024-02-01 19:01:08 +08:00
1d5471f805 [chore](debug) check the const exprs count in union node (#30679) 2024-02-01 19:00:51 +08:00
2a81ab70e4 [Enhancement](group commit) Optimize WAL insufficient space error messages (#30614) 2024-02-01 19:00:50 +08:00
90d3f0d805 [Fix](json) avoid print warn log when parse failed (#30656) 2024-02-01 19:00:50 +08:00
e9c112b843 [Refact](inverted index) refact inverted index cache to decouple with reader (#30574) 2024-02-01 19:00:50 +08:00
Pxl
1aa7a914e1 fix wrong profile on distinct agg and pass reference on uint136's compare (#30661) 2024-02-01 19:00:50 +08:00
4f1d76d646 handle create rowset error to avoid null pointer exception (#30670) 2024-02-01 11:51:51 +08:00
92cad69fc4 [Fix](parquet-reader) Fix reading fixed length byte array decimal in parquet reader. (#30535) 2024-01-31 23:53:40 +08:00
65076949ef [fix](compile)Fix Ambiguous regex Namespace Issue on MacOS Compilation (#30652) 2024-01-31 23:53:40 +08:00
73371d44f8 [fix][refactor] refactor schema init of externa table and some parquet issue (#30325)
1. Skip parquet file which has only 4 bytes length: PAR1
2. Refactor the schema init method of iceberg/hudi/hive table in hms catalog
    1. Remove some redundant methods of `getIcebergTable`
    2. Fix issue described in #23771
3. Support HoodieParquetInputFormatBase, treat it as normal hive table format
4. When listing file, skip all hidden dirs and files
2024-01-31 23:53:40 +08:00
77b366fc4b [fix](join) incorrect result of mark join (#30543)
incorrect result of mark join
2024-01-31 23:53:40 +08:00
711b156a78 [Refactor][Rf] remove unless code in RF (#30597) 2024-01-31 23:53:40 +08:00
c28ced1ebb [Feature](executor)Insert select limited by WorkloadGroup #30610 2024-01-31 23:53:40 +08:00
045225a096 [pipelineX](profile) Fix Tablet counter on pipelineX engine (#30613) 2024-01-31 23:53:39 +08:00
ef8d9ad9a4 [pipelinex](profile) improve memory counter of pipelineX (#30538) 2024-01-31 23:53:39 +08:00
9eeb7dc9e4 Refactor MemTableFlushExecutor::create_flush_token to improve (#30554)
readability && Add default type guard

Signed-off-by: Jack Drogon <jack.xsuperman@gmail.com>
2024-01-31 23:53:39 +08:00
19f57b544e support cosh math function (#30602)
Co-authored-by: Rohit Satardekar <rohitrs1983@gmail.com>
2024-01-31 23:53:39 +08:00
f35803b7a0 [feature](pipeline-load) enable pipeline load by default (#30581) 2024-01-31 23:53:39 +08:00
e6fbccd3ed [Feature](Variant) support row store for variant type (#30052) 2024-01-31 23:53:39 +08:00
8b61b7c6cd [exec](function) Add tanh func (#30555) 2024-01-31 23:53:39 +08:00
a1ccf34ecc [fix](group commit) Fix replay wal core because undefined TLoadSourceType (#30571) 2024-01-31 23:53:39 +08:00
1d632f1af4 [improvement](move-memtable) enable stream write to socket in background bthread (#30586) 2024-01-31 23:53:39 +08:00
7d037c12bf [bugfix](paimon)fix paimon testcases (#30514)
1. set default timezone
2. not supported `char` type to pushdown
2024-01-31 23:53:39 +08:00
378d9e7336 [Colo][Scan] delete the colo scan code (#30584) 2024-01-31 23:53:39 +08:00
221308f78a [fix](datatype) fix bugs for IPv4/v6 datatype and add some basic regression test cases (#30261) 2024-01-31 23:53:39 +08:00
1a8e281255 [fix](cluster by) Fix cluster_by used-after-moved in compaction (#29273) 2024-01-31 23:53:39 +08:00