Commit Graph

5709 Commits

Author SHA1 Message Date
b38b8b4494 [pipelineX](fix) Fix BE crash caused by join and constant expr (#24862) 2023-09-25 21:01:09 +08:00
6502da8917 [bugfix](restore) add partition id into convert_rowset_ids() (#24834) 2023-09-25 20:07:24 +08:00
8679095e5c [feature](debug) support debug point used in debug code (#24502) 2023-09-25 17:56:12 +08:00
3b4d8b4ac8 [pipelineX](feature) Support schema scan operator (#24850) 2023-09-25 14:42:25 +08:00
9412775686 remove useless variable in scanctx (#24849)
remove useless variable in scanctx
2023-09-25 14:36:18 +08:00
xfz
1b95ce1d93 [feature](json-function) add json_insert, json_replace, json_set functions (#24384)
[feature](json-function) add three json funcitons
2023-09-25 12:52:29 +08:00
c9ef5ef2b1 [refactor](profile) refactor join node profile when build side shared hash table (#24785)
refactor join node profile when build side shared hash table
2023-09-25 10:28:16 +08:00
39e6512a21 [bug](scanner) Fix memory out of bound in scanner scheduler (#24840) 2023-09-25 09:58:26 +08:00
9579634eac [Debug](pipeline) add log of pipeline scan bug (#24804) 2023-09-25 08:38:31 +08:00
aea0279f25 [chore](build) Fix issues with GLIBC dependency (#24837)
Linking with libabsl_random_internal_seed_material.a instead of glibc to fix this issue #24838 .
2023-09-25 08:21:19 +08:00
900b06d14e [improvement](disk balancer) reduce disk balance sensitivity (#24637) 2023-09-24 22:02:53 +08:00
fa0e0a606b [Fix](autoinc) Ensure that _fetch_autoinc_id_executor is destructed after the destructions of AutoIncIDBuffers (#24832) 2023-09-24 21:44:02 +08:00
27eed937b3 [pipelineX](es scan) Support ES scan operator (#24824)
Support ES scan operator
2023-09-24 00:32:38 +08:00
5a4d51716c [fix](pipelineX) fix null ptr when unionoperator only have constexpr #24822 2023-09-23 20:28:38 +08:00
11b6fb9d10 [PipelineX](feature) Support data gen operator (#24823) 2023-09-23 19:28:47 +08:00
a0de08255d [pipelineX](profile) Add necessary metrics (#24820) 2023-09-23 19:27:45 +08:00
329c22da00 [pipelineX](feature) Support table function operator (#24818) 2023-09-23 19:26:53 +08:00
ce79711b0d [FIX](serde) fix map/array deserialize string with quote pair (#24808) 2023-09-23 01:12:20 +08:00
3d786c58de [pipelineX](sink) Support async writer (#24793) 2023-09-22 23:04:23 +08:00
3bf95b2cfe [fix](merge-on-write) fix duplicate key in schema change (#24782) 2023-09-22 19:50:31 +08:00
f4b1e2b343 [Fix](core) Fix segment cache core when output rowset is nullptr (#24778) 2023-09-22 19:48:42 +08:00
ac55d45f79 [Fix](topn opt) fix heap use after free when shrink in fetch phase (#24774) 2023-09-22 19:48:05 +08:00
4007622ad9 [Improve](inverted index) improve match performance without index (#24751) 2023-09-22 18:45:11 +08:00
c346f4d638 [Improvement](Storage) Lazy init mini_download dir (#24649) 2023-09-22 16:23:26 +08:00
263506f8ab [refactor](pipelineX) add MultiCast operator (#24656) 2023-09-22 15:41:14 +08:00
3c99743bf2 [enhancement](csv_reader)Optimize the reading efficiency of nullable (string) columns. (#24698)
Optimize the performance of stream load tsv by reducing virtual function calls .
(Optimize read performance of nullable (string) columns by reducing virtual function calls.)
before : 600+ s
after : 560+ s
2023-09-22 13:44:37 +08:00
034582bb64 [pipelineX](fix) Fix broadcast dependency hanging (#24740) 2023-09-22 12:24:32 +08:00
d82b603b53 [bug](jsonb) fix non simd int128 overflow #24750 2023-09-22 12:23:43 +08:00
22616d125d [function](bitmap) add function alias bitmap_andnot and bitmap_andnot_count (#24771) 2023-09-22 12:18:31 +08:00
c8655d1dae [enhancement](compaction) print correct error messsage & avoid unnecessary stacktrace (#24758) 2023-09-22 11:58:11 +08:00
8a85a75b8b [chore](scanner) check columns' nullable with schema (#24724)
Add a validation to prevent potential schema inconsistency issues.
2023-09-22 11:34:53 +08:00
37a7532a49 [fix](pipelineX) get sink local state in distinctstreamingagg (#24752) 2023-09-22 11:16:46 +08:00
e9ef6c7da7 [chore](workflow) Fix security issues in Code Checks (#24761)
The workflow `Code Checks` needs write permissions granted by the event `pull_request_target` to comment on pull requests. However, if the workflow ran users' code, the malicious code would do some dangerous actions on our repository.

The following changes are made in this PR:
1. Instead of applying patches, we use `sed` to modify the `entrypoint.sh` in action-sh-checker explicitly in the workflow.
2. Revoke the write permissions when generating `compile_commands.json` which is produced by executing the build script `build.sh`.
2023-09-22 10:39:39 +08:00
a9576df74a [typo](comments) Change the Chinese comments to English #24659 2023-09-22 08:56:56 +08:00
09e03247ec [chore](readability) Better readability of ExecNode.cpp #24733 2023-09-22 08:54:57 +08:00
665e878811 [fix](Compile) fix BE complie error #24756 2023-09-22 08:53:30 +08:00
c9b2f4cb92 [workload](pipeline) Add cgroup cpu controller (#24052) 2023-09-21 21:49:33 +08:00
58ab25ccaa Revert "[Feature](merge-on-write)Support ignore mode for merge-on-write unique table (#21773)" (#24731)
This reverts commit 3ee89aea35726197cb7e94bb4f2c36bc9d50da84.
2023-09-21 21:01:28 +08:00
ce14183c3a [fix](compile) fix GHA mac ut compile failure (#24738) 2023-09-21 21:00:23 +08:00
a48b19ceb6 [feature](Outfile) select into outfile supports to export struct/map/array type data to orc file format (#24350)
We do not support nested complex type in this pr.
2023-09-21 20:15:18 +08:00
7630fe7b7b [bug](node)fix dense_rank function in partition sort node return wrong rows (#24727) 2023-09-21 19:13:30 +08:00
8eb14eec7c [enhancement](baddisk) record bad disk in be_custom.conf to handle (#24639) 2023-09-21 18:31:58 +08:00
ec987b1b7d [fix](index compaction)ignore doc which dose not exist in destination segment (#24729) 2023-09-21 18:27:08 +08:00
85fb46bb71 [refactor](cache) Refactor preloaded timezone global cache (#24694)
Refactor preloaded timezone global cache
2023-09-21 17:26:41 +08:00
b9997d69fa [refactor](pipelineX) return error when local_state can not find id (#24360) 2023-09-21 16:04:08 +08:00
36c9366a8b [improve](tablet schema) add config to modify tablet schema recycle interval (#24602) 2023-09-21 15:57:11 +08:00
2098670001 [Fix](merge-on-write) Skip to check delete bitmap correctness in commit phase if the current tablet is converting (#24675) 2023-09-21 15:48:02 +08:00
5a463fedb8 [optimize](match) optimize unnecessary conversions (#24389) 2023-09-21 15:29:10 +08:00
4eb09ce1b2 [enhancement](config) do not abort when a disk is broken (#24692) 2023-09-21 15:21:42 +08:00
d31d99bf34 [pipeline](load) opt the pipeline load code (#24708)
opt the pipeline load code
2023-09-21 15:20:31 +08:00