database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
Files
936da4f10a6c8c5965a91ee814583311b7741e99
doris/be/src/runtime
History
yinzhijian 936da4f10a [feature](thread-pool) Support thread pool per disk for scanners (#7994) 
Support thread pool per disk for scanners to prevent pool performance from some high ioutil disks happening

key point:
1. each disk has a thread pool for scanners
2. whenever a thread pool of one disk runs out of local work, tasks can be retrieved from other threads(disks). This is done round-robin.

performance testing: 
vec version: 25% faster than single thread pool in a high io util disk test case
normal version: 8% faster than single thread pool in a high io util disk test case
2022-02-18 09:40:58 +08:00
..
bufferpool
dd a method to get doris current memory usage (#6979)
2021-11-24 10:07:54 +08:00
cache
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
memory
[fix](memory) fix bug that ~BitShufflePageDecoder destroys uninitialized chunk (#7172)
2021-11-23 15:24:25 +08:00
routine_load
[fix][refactor] fix bugs and refactor some code by lint (#7871)
2022-02-01 14:31:14 +08:00
stream_load
[Feature](Transaction) Support two phase commit (2PC) for stream load (#7473)
2022-02-16 11:55:04 +08:00
broker_mgr.cpp
[config] use thrift_rpc_timeout_ms config replace hard code value (#6637)
2021-09-16 10:22:57 +08:00
broker_mgr.h
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00
buffer_control_block.cpp
1. Fix an error when fetch string type field may cause malform packet error. (#7262)
2021-12-01 10:02:34 +08:00
buffer_control_block.h
1. Fix an error when fetch string type field may cause malform packet error. (#7262)
2021-12-01 10:02:34 +08:00
buffered_block_mgr2.cc
dd a method to get doris current memory usage (#6979)
2021-11-24 10:07:54 +08:00
buffered_block_mgr2.h
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
buffered_tuple_stream2.cc
[refactor] remove some unused code (#8022)
2022-02-12 15:17:28 +08:00
buffered_tuple_stream2.h
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
buffered_tuple_stream2.inline.h
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
buffered_tuple_stream3.cc
[refactor] remove some unused code (#8022)
2022-02-12 15:17:28 +08:00
buffered_tuple_stream3.h
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
buffered_tuple_stream3.inline.h
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00
client_cache.cpp
[improvement](executor) Optimize lock of client cache (#7543)
2022-01-11 15:05:24 +08:00
client_cache.h
[improvement](executor) Optimize lock of client cache (#7543)
2022-01-11 15:05:24 +08:00
CMakeLists.txt
[refactor] remove some unused code (#8022)
2022-02-12 15:17:28 +08:00
collection_value.cpp
[ARRAY] Support array type load and select not include access by index (#5980)
2021-07-13 14:02:39 +08:00
collection_value.h
[ARRAY] Support array type load and select not include access by index (#5980)
2021-07-13 14:02:39 +08:00
data_stream_mgr.cpp
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
data_stream_mgr.h
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
data_stream_recvr.cc
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
data_stream_recvr.h
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
data_stream_sender.cpp
[feature](rpc) (experimental)Support implement UDF through GRPC protocol. (#7519)
2022-02-08 09:25:09 +08:00
data_stream_sender.h
[fix](brpc-attachment) Fix bug that may cause BE crash when enable transfer_data_by_brpc_attachment (#7921)
2022-02-01 08:51:16 +08:00
datetime_value.cpp
[improvement](rewrite) Make RewriteDateLiteralRule to be compatible with mysql (#7876)
2022-01-27 10:32:18 +08:00
datetime_value.h
[feature](vectorization) Support Vectorized Exec Engine In Doris (#7785)
2022-01-18 10:07:15 +08:00
decimalv2_value.cpp
[Optimize] Speed up converting the data of other types to string in mysql_result_writer (#6384)
2021-08-24 22:30:58 +08:00
decimalv2_value.h
[Optimize] Speed up converting the data of other types to string in mysql_result_writer (#6384)
2021-08-24 22:30:58 +08:00
descriptor_helper.h
[refactor] Remove decimal v1 related code from code base (#6079)
2021-07-07 10:26:32 +08:00
descriptors.cpp
(fix)[planner] Fix the right tuple ids in empty set node (#7931)
2022-01-29 09:46:05 +08:00
descriptors.h
[fix](brpc-attachment) Fix bug that may cause BE crash when enable transfer_data_by_brpc_attachment (#7921)
2022-02-01 08:51:16 +08:00
disk_io_mgr_internal.h
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
disk_io_mgr_reader_context.cc
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
disk_io_mgr_scan_range.cc
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
disk_io_mgr.cc
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
disk_io_mgr.h
[refactor] remove some unused code (#8022)
2022-02-12 15:17:28 +08:00
dpp_sink_internal.cpp
[chore] Prolog of hermetic build with GCC 11 and Clang 13. (#7712)
2022-01-21 12:12:04 +08:00
dpp_sink_internal.h
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00
dpp_sink.cpp
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
dpp_sink.h
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00
dpp_writer.cpp
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
dpp_writer.h
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00
etl_job_mgr.cpp
[improvement](Load) Cancel the load job ASAP when encounter unqualified data (#6319)
2022-01-18 13:13:55 +08:00
etl_job_mgr.h
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00
exec_env_init.cpp
[feature](thread-pool) Support thread pool per disk for scanners (#7994)
2022-02-18 09:40:58 +08:00
exec_env.cpp
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00
exec_env.h
[feature](thread-pool) Support thread pool per disk for scanners (#7994)
2022-02-18 09:40:58 +08:00
export_sink.cpp
[docs] Add user manual for hdfs load and transaction. (#7497)
2021-12-30 10:22:48 +08:00
export_sink.h
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00
export_task_mgr.cpp
[LOG] Standardize the use of VLOG in code (#5264)
2021-01-21 12:09:09 +08:00
export_task_mgr.h
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00
external_scan_context_mgr.cpp
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
external_scan_context_mgr.h
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00
file_result_writer.cpp
[feature] Support return bitmp/hll data in select statement (#7276)
2021-12-15 09:48:27 +08:00
file_result_writer.h
[feature] Support return bitmp/hll data in select statement (#7276)
2021-12-15 09:48:27 +08:00
fold_constant_executor.cpp
[feature](vectorization) Support Vectorized Exec Engine In Doris (#7785)
2022-01-18 10:07:15 +08:00
fold_constant_executor.h
[feature](vectorization) Support Vectorized Exec Engine In Doris (#7785)
2022-01-18 10:07:15 +08:00
fragment_mgr.cpp
[refactor] Remove version hash from BE and related test in BE (#8027)
2022-02-14 09:29:27 +08:00
fragment_mgr.h
[Bug] fix Runtime filter can't find fragment-id when apply_filter called early (#6923)
2021-10-27 09:54:52 +08:00
free_list.hpp
Change variable and namespace name in BE (#268)
2018-11-02 10:22:32 +08:00
free_pool.hpp
[BUG] fix bugs with string type (#6538)
2021-09-01 15:59:55 +08:00
heartbeat_flags.h
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00
initial_reservations.cc
[refactor] Remove boost mutex, use std::mutex instead (#5684)
2021-04-22 11:29:36 +08:00
initial_reservations.h
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00
large_int_value.cpp
Use fmt and std::from_chars to make convert integer to string and convert string to integer more efficient (#6361)
2021-08-04 10:55:19 +08:00
large_int_value.h
[chore] Prolog of hermetic build with GCC 11 and Clang 13. (#7712)
2022-01-21 12:12:04 +08:00
load_channel_mgr.cpp
[fix](load)(compaction) Fix NodeChannel coredump bug and modify some compaction logic (#8072)
2022-02-17 10:52:08 +08:00
load_channel_mgr.h
[improvement](load) Improve load fault tolerance (#7674)
2022-01-20 09:23:21 +08:00
load_channel.cpp
[fix](load)(compaction) Fix NodeChannel coredump bug and modify some compaction logic (#8072)
2022-02-17 10:52:08 +08:00
load_channel.h
[improvement](load) Improve load fault tolerance (#7674)
2022-01-20 09:23:21 +08:00
load_path_mgr.cpp
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00
load_path_mgr.h
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00
mem_pool.cpp
dd a method to get doris current memory usage (#6979)
2021-11-24 10:07:54 +08:00
mem_pool.h
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
mem_tracker.cpp
[Optimize] Optimize mem_tracker (#6988)
2021-11-12 10:51:45 +08:00
mem_tracker.h
dd a method to get doris current memory usage (#6979)
2021-11-24 10:07:54 +08:00
memory_scratch_sink.cpp
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
memory_scratch_sink.h
[fix](brpc-attachment) Fix bug that may cause BE crash when enable transfer_data_by_brpc_attachment (#7921)
2022-02-01 08:51:16 +08:00
message_body_sink.cpp
[Bug] Fix some memory bugs (#6699)
2021-09-22 12:30:14 +08:00
message_body_sink.h
[Bug] Fix some memory bugs (#6699)
2021-09-22 12:30:14 +08:00
minidump.cpp
[feat](minidump) Add minidump support (#7124)
2021-11-20 21:41:26 +08:00
minidump.h
[feat](minidump) Add minidump support (#7124)
2021-11-20 21:41:26 +08:00
multi_precision.h
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00
mysql_result_writer.cpp
[feature](vectorization) Support Vectorized Exec Engine In Doris (#7785)
2022-01-18 10:07:15 +08:00
mysql_result_writer.h
[feature](vectorization) Support Vectorized Exec Engine In Doris (#7785)
2022-01-18 10:07:15 +08:00
mysql_table_sink.cpp
[Enhance] Improve the readability of memtrackers' name (#5455)
2021-03-11 22:33:31 +08:00
mysql_table_sink.h
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00
mysql_table_writer.cpp
[refactor](mysql_table_writer)split into two parts of vectorized and row mode (#8081)
2022-02-17 11:29:25 +08:00
mysql_table_writer.h
[refactor](mysql_table_writer)split into two parts of vectorized and row mode (#8081)
2022-02-17 11:29:25 +08:00
odbc_table_sink.cpp
[Optimize] Speed up the bulk data load to ODBC table. (#5765)
2021-05-12 10:58:52 +08:00
odbc_table_sink.h
[ODBC] Support ODBC Sink for insert into data to ODBC external table (#5033)
2020-12-13 21:53:27 +08:00
plan_fragment_executor.cpp
[fix](load)(compaction) Fix NodeChannel coredump bug and modify some compaction logic (#8072)
2022-02-17 10:52:08 +08:00
plan_fragment_executor.h
[feature](vectorization) Support Vectorized Exec Engine In Doris (#7785)
2022-01-18 10:07:15 +08:00
primitive_type.cpp
Add long text type STRING, with a maximum length of 2GB. Usage is similar to varchar, and there is no guarantee for the performance of storing extremely long data (#6391)
2021-08-18 09:05:40 +08:00
primitive_type.h
[feature](vectorization) Support Vectorized Exec Engine In Doris (#7785)
2022-01-18 10:07:15 +08:00
qsorter.cpp
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
qsorter.h
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
query_fragments_ctx.h
Add scan thread token (#6443)
2021-08-27 10:56:17 +08:00
query_statistics.cpp
[feat](memory-track) Print peak memory use of all backend after query in audit log (#7030)
2021-11-22 14:46:08 +08:00
query_statistics.h
[feat](memory-track) Print peak memory use of all backend after query in audit log (#7030)
2021-11-22 14:46:08 +08:00
raw_value.cpp
[refactor] remove some unused code (#8022)
2022-02-12 15:17:28 +08:00
raw_value.h
[feature](vectorization) Support Vectorized Exec Engine In Doris (#7785)
2022-01-18 10:07:15 +08:00
record_batch_queue.cpp
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00
record_batch_queue.h
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00
result_buffer_mgr.cpp
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
result_buffer_mgr.h
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
result_file_sink.cpp
[fix][refactor] fix bugs and refactor some code by lint (#7871)
2022-02-01 14:31:14 +08:00
result_file_sink.h
[fix](brpc-attachment) Fix bug that may cause BE crash when enable transfer_data_by_brpc_attachment (#7921)
2022-02-01 08:51:16 +08:00
result_queue_mgr.cpp
[fix][refactor] fix bugs and refactor some code by lint (#7871)
2022-02-01 14:31:14 +08:00
result_queue_mgr.h
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00
result_sink.cpp
[fix][refactor] fix bugs and refactor some code by lint (#7871)
2022-02-01 14:31:14 +08:00
result_sink.h
[fix](brpc-attachment) Fix bug that may cause BE crash when enable transfer_data_by_brpc_attachment (#7921)
2022-02-01 08:51:16 +08:00
result_writer.cpp
[OUTFILE] Support INTO OUTFILE to export query result (#3584)
2020-05-25 21:24:56 +08:00
result_writer.h
[feature] Support return bitmp/hll data in select statement (#7276)
2021-12-15 09:48:27 +08:00
row_batch_interface.hpp
Change variable and namespace name in BE (#268)
2018-11-02 10:22:32 +08:00
row_batch.cpp
[fix](compatibility) Fix compatibility issue of PRowBatch and some tablet sink bugs (#8000)
2022-02-15 11:23:19 +08:00
row_batch.h
[refactor] remove some unused code (#8022)
2022-02-12 15:17:28 +08:00
runtime_filter_mgr.cpp
[feature](rpc) (experimental)Support implement UDF through GRPC protocol. (#7519)
2022-02-08 09:25:09 +08:00
runtime_filter_mgr.h
[Optimize](Runtime Filter) Support merge in runtime filter(#7546) (#7547)
2022-01-06 19:08:35 +08:00
runtime_state.cpp
[improvement](Load) Cancel the load job ASAP when encounter unqualified data (#6319)
2022-01-18 13:13:55 +08:00
runtime_state.h
[improvement](load) Improve load fault tolerance (#7674)
2022-01-20 09:23:21 +08:00
small_file_mgr.cpp
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00
small_file_mgr.h
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00
snapshot_loader.cpp
[refactor] Remove version hash from BE and related test in BE (#8027)
2022-02-14 09:29:27 +08:00
snapshot_loader.h
[refactor] Remove version hash from BE and related test in BE (#8027)
2022-02-14 09:29:27 +08:00
sorted_run_merger.cc
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
sorted_run_merger.h
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
sorter.h
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00
spill_sorter.cc
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
spill_sorter.h
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
string_buffer.hpp
Change variable and namespace name in BE (#268)
2018-11-02 10:22:32 +08:00
string_search.hpp
[Enhancement] use std::search to replace custom search (#7999)
2022-02-11 10:47:58 +08:00
string_value.cpp
[Bug] Fix the bug of where condition a in ('A', 'B', 'V') and a in ('A') return error result (#5072)
2020-12-15 09:29:10 +08:00
string_value.h
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
string_value.hpp
[feature](vectorization) Support Vectorized Exec Engine In Doris (#7785)
2022-01-18 10:07:15 +08:00
tablets_channel.cpp
[improvement](load) Improve load fault tolerance (#7674)
2022-01-20 09:23:21 +08:00
tablets_channel.h
[improvement](load) Improve load fault tolerance (#7674)
2022-01-20 09:23:21 +08:00
thread_resource_mgr.cpp
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
thread_resource_mgr.h
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
tmp_file_mgr.cc
[refactor] remove some unused code (#8022)
2022-02-12 15:17:28 +08:00
tmp_file_mgr.h
[refactor] remove some unused code (#8022)
2022-02-12 15:17:28 +08:00
tuple_row.cpp
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00
tuple_row.h
[refactor] replace boost smart ptr with stl (#6856)
2021-11-17 10:18:35 +08:00
tuple.cpp
[fix](brpc-attachment) Fix bug that may cause BE crash when enable transfer_data_by_brpc_attachment (#7921)
2022-02-01 08:51:16 +08:00
tuple.h
[fix](brpc-attachment) Fix bug that may cause BE crash when enable transfer_data_by_brpc_attachment (#7921)
2022-02-01 08:51:16 +08:00
type_limit.h
[New Featrue] Support Vectorization Execution Engine Interface For Doris (#6329)
2021-08-11 14:54:06 +08:00
types.cpp
[refactor] remove some unused code (#8022)
2022-02-12 15:17:28 +08:00
types.h
[feature](vec) add ColumnHLL to support hll type (#7828)
2022-02-17 10:44:42 +08:00
user_function_cache.cpp
fix BE coredump in UserFunctionCache (#6331)
2021-07-30 09:24:30 +08:00
user_function_cache.h
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00
vectorized_row_batch.cpp
Add long text type STRING, with a maximum length of 2GB. Usage is similar to varchar, and there is no guarantee for the performance of storing extremely long data (#6391)
2021-08-18 09:05:40 +08:00
vectorized_row_batch.h
[CodeFormat] Clang-format cpp sources (#4965)
2020-11-28 18:36:49 +08:00