Commit Graph

10 Commits

Author SHA1 Message Date
41271ecba0 [fix](ES Catalog)Do not push down limit to ES when predicates can not be processed by ES. (#40111) (#40265)
bp #40111
2024-09-03 11:17:24 +08:00
695d58f354 [cherry-pick](scan)scanner could eos early when reached limit (#36535) (#36736)
## Proposed changes
cherry-pick from master #36535
2024-06-25 17:22:43 +08:00
2b7fa9d6bb [pipelineX](refactor) Rebuild relationship between dep and operator (#31487) 2024-02-29 12:38:03 +08:00
52b9af06fb [pipelineX](refactor) Delete subclasses inherited from Dependency (#31216) 2024-02-22 13:01:48 +08:00
c00dca70e6 [pipelineX](local shuffle) Support parallel execution despite of tablet number (#28266) 2023-12-14 12:53:54 +08:00
1b3512d942 [pipelineX](bug) Fix cancel timeout (#27396) 2023-11-22 22:31:34 +08:00
459f75073f [pipelineX](dependency) remove OrDependency (#27242) 2023-11-20 13:05:34 +08:00
e20cab64f4 [improvement](scan) avoid too many scanners for file scan node (#25727)
In previous, when using file scan node(eq, querying hive table), the max number of scanner for each scan node
will be the `doris_scanner_thread_pool_thread_num`(default is 48).
And if the query parallelism is N, the total number of scanner would be 48 * N, which is too many.

In this PR, I change the logic, the max number of scanner for each scan node
will be the `doris_scanner_thread_pool_thread_num / query parallelism`. So that the total number of scanners
will be up to `doris_scanner_thread_pool_thread_num`.

Reduce the number of scanner can significantly reduce the memory usage of query.
2023-10-29 17:41:31 +08:00
552091f21f [performance](pipelineX) optimize pipelineX (#25713) 2023-10-25 10:13:17 +08:00
27eed937b3 [pipelineX](es scan) Support ES scan operator (#24824)
Support ES scan operator
2023-09-24 00:32:38 +08:00