Currently, there are some useless includes in the codebase. We can use a tool named include-what-you-use to optimize these includes. By using a strict include-what-you-use policy, we can get lots of benefits from it.
Currently, there are some useless includes in the codebase. We can use a tool named include-what-you-use to optimize these includes. By using a strict include-what-you-use policy, we can get lots of benefits from it.
Currently, BE will print fail to get master client from cache. host=xxxxx, port=9228, code=THRIFT_RPC_ERROR but we did not know which step generate this error. So that I refactor error status in be and add error stack for RPC_ERROR.
W1122 10:19:21.130796 30405 utils.cpp:89] fail to get master client from cache. host=xxxx, port=9228, code=RPC error(error -1): Couldn't open transport for xxxx:9228 (open() timed out)/n @ 0x559af8f774ea doris::Status::ConstructErrorStatus()
@ 0x559af9aacbee _ZN5doris16ThriftClientImpl4openEv.cold
@ 0x559af97f563a doris::ClientCacheHelper::_create_client()
@ 0x559af97f78cd doris::ClientCacheHelper::get_client()
@ 0x559af934f38b doris::MasterServerClient::report()
@ 0x559af932e7a7 doris::TaskWorkerPool::_handle_report()
@ 0x559af932f07c doris::TaskWorkerPool::_report_task_worker_thread_callback()
@ 0x559af9b223c5 doris::ThreadPool::dispatch_thread()
@ 0x559af9b187af doris::Thread::supervise_thread()
@ 0x7f661bd8bea5 start_thread
@ 0x7f661c09eb0d __clone
Co-authored-by: yiguolei <yiguolei@gmail.com>
This PR supports rowset level data upload on the BE side, so that there can be both cold data and hot data in a tablet,
and there is no necessary to prohibit loading new data to cooled tablets.
Each rowset is bound to a `FileSystem`, so that the storage layer can read and write rowsets without
perceiving the underlying filesystem.
The abstracted `RemoteFileSystem` can try local caching strategies with different granularity,
instead of caching segment files as before.
To avoid conflicts with the code in be/src/io, we temporarily put the file system related code in the be/src/io/fs directory.
In the future, `FileReader`s and `FileWriter`s should be unified.
There are 3 error code types in BE: OLAPStatus AgentStatus Status.
It is very confused and sometimes conflict during write code.
I will try to unify them to Status.
1. replace all boost::shared_ptr to std::shared_ptr
2. replace all boost::scopted_ptr to std::unique_ptr
3. replace all boost::scoped_array to std::unique<T[]>
4. replace all boost:thread to std::thread
In the previous implementation, clone task will continue download files
even if some error happened. This may cause unexpected problem. This
Change List refactor it to that when error happends, clone task will
fail total and try to clone from another remote source.
Besides above change, I call FileUtils::remove_all and create_dir
instead of boost one, which may cause exception. What's more
AgentMasterClient is replaced with ThriftRpcHelper, by this change
conncection can be reused.
1. Unify the thrift rpc timeout from BE to FE.
Add a BE config 'thrift_rpc_timeout_ms', default is 5000
2. Add hostname in "show proc '/frontends';" stmt result.
3. Fix a lock order bug in Load.java
