32853a529c
[Bug](cte) fix multi cast data stream source not open expr ( #23740 )
...
fix multi cast data stream source not open expr
2023-09-01 14:57:12 +08:00
65f41f71c1
[pipelineX](refactor) refine codes ( #23726 )
2023-09-01 07:57:35 +08:00
d22290e548
[pipelineX](join) support hash join ( #23689 )
2023-08-31 13:01:26 +08:00
f7caae08d5
[fix](union) should open/alloc_resource in sink operator instead of source ( #23637 )
2023-08-30 18:58:59 +08:00
29b94c4ed7
[pipeline](refactor) refine pipeline fragment context ( #23478 )
2023-08-28 15:55:02 +08:00
28a2e71084
[pipelineX](refactor) refine codes ( #23521 )
...
* [pipelineX](refactor) refine codes
* update
* update
2023-08-28 14:38:07 +08:00
ba351af452
[enhancement](thirdparty) upgrade thirdparty libs - again ( #23414 )
...
submit again #23290 (not upgrade brpc, because bthread local has error)
protobuf 3.15.0 -> 21.11
glog 0.4.0 -> 0.6.0
lz4 1.9.3 -> 1.9.4
curl 7.79.0 -> 8.2.1
zstd 1.5.2 -> 1.5.5
arrow 7.0.0 -> 13.0.0
abseil 20220623.1 -> 20230125.3
orc 1.7.2 -> 1.9.0
jemalloc for arrow 5.2.1 -> 5.3.0
xsimd 7.0.0 -> 13.0.0
opentelemetry-proto 0.19.0 -> 1.0.0
opentelemetry 1.8.3 -> 1.10.0
new:
c-ares -> 1.19.1
grpc -> 1.54.3
2023-08-26 22:59:10 +08:00
2b6d876280
[feature](move-memtable)[6/7] add options to enable memtable on sink node ( #23470 )
...
Co-authored-by: Siyang Tang <82279870+TangSiyang2001@users.noreply.github.com >
2023-08-25 22:32:22 +08:00
49a32c2ee0
[pipelineX](fix) fix two phase execution and add test cases ( #23353 )
2023-08-25 17:57:35 +08:00
5c37be16fe
[pipelineX](correctness) Fix close problem for local state ( #23479 )
2023-08-25 14:19:27 +08:00
d9db3f5431
[Improvement](scan) Remove redundant predicates on scan node ( #23374 )
...
* Remove redundant predicates on scan node
* update
* fix
2023-08-25 10:41:37 +08:00
55e572df82
[pipelineX](analytic operator) Support analytic operator ( #23444 )
2023-08-24 23:05:29 +08:00
96164f3bdc
[pipelinex](sort) Fix expression initialization order ( #23405 )
2023-08-24 17:29:24 +08:00
9d1f2cd8e0
[Improvement](pipeline) Terminate early for short-circuit join ( #23378 )
2023-08-23 19:40:17 +08:00
ba882dea21
[pipelineX](dependency) Build DAG between pipelines ( #23355 )
2023-08-23 13:21:32 +08:00
450b90c082
[Bug](compile) fix compiling error due to merge conflicts ( #23351 )
...
fix compiling error due to merge conflicts
2023-08-23 11:57:57 +08:00
391355eeae
[pipelineX](agg) Support streaming agg ( #23341 )
2023-08-23 08:38:12 +08:00
5c2fae7ce5
[pipeline](exec) Refactor the table sink code in remove unless code ( #23223 )
...
Refactor the table sink code in remove unless code
2023-08-22 20:42:14 +08:00
1609b6cbf2
[pipelineX](sort) Support sort operator ( #23322 )
2023-08-22 19:36:50 +08:00
12075f9853
[pipelineX](projection) Support projection and blocking agg ( #23256 )
2023-08-21 22:23:02 +08:00
dcd6c3c022
[pipelineX](refactor) propose a new pipeline execution model ( #22562 )
2023-08-21 15:38:45 +08:00
9b2323b7fd
[Pipeline](exec) support async writer in pipelien query engine ( #22901 )
2023-08-15 17:32:53 +08:00
34399e2965
[Bug](exchange) init _instance_to_rpc_ctx on register_sink ( #22976 )
...
init _instance_to_rpc_ctx on register_sink
2023-08-15 13:02:28 +08:00
abc9de07b3
[Bug](pipeline) make sure sink is not blocked before try close ( #22765 )
...
make sure sink is not blocked before try close
2023-08-13 13:20:48 +08:00
9f0a9e6fd6
[bug](distinct-agg) fix limit value not effective in some case ( #22517 )
...
fix limit value not effective in some case
2023-08-03 10:35:36 +08:00
5f25b924b3
[opt](conf) Modify brpc eovercrowded conf ( #22407 )
...
brpc ignore eovercrowded of data stream sender and exchange sink buffer
Modify the default value of brpc_socket_max_unwritten_bytes
2023-08-01 08:47:55 +08:00
bc88d34b16
[bug](distinct-agg) fix distinct-agg outblock columns size not equal key size ( #22357 )
...
* [imporve](flex) support scientific notation(aEb) parser
* update
* [bug](distinct-agg) fix distinct-agg outblock columns size not equal key size
2023-07-29 12:44:44 +08:00
1c6246f7ee
[improve](agg) support distinct agg node ( #22169 )
...
select c_name from customer union select c_name from customer
this sql used agg node to get distinct row of c_name,
so it's no need to wait for inserted all data to hash map,
could output the data which it's inserted into hash map successed.
2023-07-28 13:54:10 +08:00
aa75f79fad
[fix](executor)cancel exchange buffer rpc when query is cancelled ( #22226 )
...
when brpc client make a request to a server, if the server doesn't response and may not response forever(such as BE restart), the query can be cancelled at once, but the ExchangeSinkBuffer can not be cancelled until rpc timeout.
So we hope when the query is cancelled, the ExchangeSinkBuffer can be closed at once.
2023-07-27 14:38:25 +08:00
23e7423748
[pipeline](refactor) refactor pipeline task schedule logics ( #22028 )
2023-07-25 17:18:26 +08:00
19ba6bec38
[Improvement](pipeline) support send eos on local exchange and remove some unused code ( #22086 )
...
support send eos on local exchange and remove some unused code
2023-07-24 09:25:32 +08:00
ca71048f7f
[Chore](status) avoid empty error msg on status ( #21454 )
...
avoid empty error msg on status
2023-07-11 13:48:16 +08:00
f87a3ccba2
[fix](runtime_filter) runtime_profile was not initialized in multi_cast_data_stream_source ( #21690 )
2023-07-11 00:16:29 +08:00
ee9822fa7e
[Fix](pipeline) fix ExchangeSinkBuffer request id memory alloc problem ( #21647 )
...
Co-authored-by: airborne12 <airborne12@gmail.com >
fix ExchangeSinkBuffer request id memory alloc problem
2023-07-09 23:45:28 +08:00
90dd8716ed
[refactor](multicast) change the way multicast do filter, project and shuffle ( #21412 )
...
Co-authored-by: Jerry Hu <mrhhsg@gmail.com >
1. Filtering is done at the sending end rather than the receiving end
2. Projection is done at the sending end rather than the receiving end
3. Each sender can use different shuffle policies to send data
2023-07-04 16:51:07 +08:00
b86dd11a7d
[fix](pipeline) refactor olap table sink close ( #20771 )
...
For pipeline, olap table sink close is divided into three stages, try_close() --> pending_finish() --> close()
only after all node channels are done or canceled, pending_finish() will return false, close() will start.
this will avoid block pipeline on close().
In close, check the index channel intolerable failure status after each node channel failure,
if intolerable failure is true, the close will be terminated in advance, and all node channels will be canceled to avoid meaningless blocking.
2023-07-04 11:27:51 +08:00
f8cfe5e579
[Bug](pipeline) add DCHECK for _instance_to_sending_by_pipeline = false on _send_rpc ( #21169 )
...
add DCHECK for _instance_to_sending_by_pipeline = false on _send_rpc
2023-06-29 10:03:57 +08:00
601120db04
[Bug](pipeline) access map may cause coredump in sink buffer ( #21108 )
2023-06-24 23:03:59 +08:00
661e1ae7c5
[fix](memory) no switch bthread context in UBSAN compile ( #21064 )
...
When UBSAN is compiled, all memory will be tracked to the orphan (unknown) mem tracker, and the bthread context and mem tracker will no longer be switched.
The supplementary fixes are as follows: #20999
2023-06-21 21:14:07 +08:00
622ef63c69
[fix](memory) fix bthread_setspecific error in rpc done.run() ( #20999 )
2023-06-20 21:00:45 +08:00
99810f1ea5
[Bug](pipeline) fix hang on union_source_operator when child sink_operator all finished ( #20938 )
2023-06-19 09:46:38 +08:00
db1da6b787
[Chore](pipeline) add some profile log when pipeline canceled ( #20825 )
...
add some profile log when pipeline canceled
2023-06-16 10:54:54 +08:00
e010fa8d4f
[Chore](runtime filter) remove runtime filter ready_for_publish/publish_finally ( #20593 )
2023-06-13 11:20:49 +08:00
5fd9f58bd3
[Chore](pipeline-engine) adjus queryt canceled log on pipeline engine ( #20702 )
...
adjus queryt canceled log on pipeline engine
2023-06-12 18:23:19 +08:00
ea264ce9de
[Opt](join) short circuit probe for join node ( #20585 )
...
Support the _short_circuit_for_probe for join node
2023-06-12 16:01:09 +08:00
14f59bef1d
[improvement](profile)add sum/avg rpc time ( #20511 )
2023-06-12 11:34:49 +08:00
05438eab0d
remove DCHECK for rpc time ( #20621 )
2023-06-09 13:38:12 +08:00
22985af4d7
[Bug](pipeline) set SourceState to MORE_DATA when UnionSourceOperator have const_expr/data_queue->remaining_has_data ( #20557 )
...
set SourceState to MORE_DATA when UnionSourceOperator have const_expr/data_queue->remaining_has_data
2023-06-08 14:47:35 +08:00
3aa0c2bdbf
[Bug](memleak) Fix emptyoperator may cause node not close ( #20525 )
2023-06-07 01:27:13 +08:00
65100d8083
[improvement](profile)add max/min rpc time ( #20339 )
2023-06-06 12:03:01 +08:00
