1. The clang format action will be triggered when a PR is submitted.
2. Skywalking eyes actions will be triggered when a PR is submitted and after merging to master branch.
We found that many commit messages submitted at present have ambiguous information.
Clear commit messages can help developers submit pull requests more readable,
committers merge easily and Release Manager easy to release.
Therefore, we have sorted out a version of the commit format specification.
We hope that subsequent contributors can sort out the commit messages according to
the specification when submitting Pull Request.
1. setting _report_thread_active to false is not necessary protected by _report_thread_lock, because
_report_thread_active's type is bool, writing data is multi-threadly safety if size <= marchine word length
2. report_profile thread terminates early is possiable, in the function report_profile(), while (_report_thread_active) may
break if _report_thread_active is false, the thread of calling open() may be scheduled out between
_report_thread_started_cv.wait(l) and _report_thread_active = true, we should not assume that how long time elapsed
between a thread be scheduled twice
Now minidump file will be created when BE crashes.
And user can manually trigger a minidump by sending SIGUSR1 to BE process.
More details can be found in minidump.md documents
Increase compatibility with mysql
1. Added two system tables files and partitions
2. Improved the return logic of mysql error code to make the error code more compatible with mysql
3. Added lock/unlock tables statement and show columns statement for compatibility with mysql dump
4. Compatible with mysqldump tool, now you can use mysql dump to dump data and table structure from doris
now use mysqldump may print error message like
```
$ mysqldump -h127.0.0.1 -P9130 -uroot test_query_qa > a
mysqldump: Error: 'errCode = 2, detailMessage = select list expression not produced by aggregation output (missing from GROUP BY clause?): `EXTRA`' when trying to dump tablespaces
```
This error message not effect the export file, you can add `--no-tablespaces` to avoid this error
Add a new field `Lag` in result of `show routine load` stmt.
`Lag: {"0":10, "1":0}` means kafka partition 0 has 10 msg behind and partition 1 is update-to-date.
Introduce by pr #4359
VariableMgr.fillValue() method should not call in ExpressionFunctions.eval(),
because in method analyzeImpl() of SysVariableDesc, it has been already called once.
If VariableMgr.fillValue() was called twice, the type of SysVariableDesc will become BigInt,
which is incorrect.
Fix profile not working in sql_cache enabled. It will thrown NullPointerException.
The reason is that the Coordinator in init profile is null when cache is enable.
Therefore, we should perform different profile processing in the case of cache hits and misses, so as to avoid the situation of null pointers.
Fixed#7104
BitShufflePageDecoder reuses the memory for storing decoder results, allocate memory directly from the
`ChunkAllocator`, the performance is improved to a certain extent.
In the case of #6285, the total time consumption is reduced by 13.5%, and the time consumption ratio of `~Reader()`
has also been reduced from 17.65% to 1.53%, and the memory allocation is unified to `ChunkAllocator` for centralized
management , Which is conducive to subsequent memory optimization.
which can avoid the memory waste caused by `Mempool`, because the chunk can be free at any time, but the
performance is lower than the allocation from `Mempool`. The guess is that there is no `Mempool` after secondary
allocation of large chunks , Will directly apply for a large number of small chunks from `ChunkAllocator`, and it takes
longer to lock in `pop_free_chunk` and `push_free_chunk` (but this is not proven from the flame graphs of BE's cpu and
contention).
1. Forbidden non-string column as params of explode_view.
The first param of explode_view must be string column(VARCHAR/CHAR/STRING)
2. N-1 n lateral views map one TableFunctionNode
The TableFunctionNode include all of fnExprs which belongs to one table.
For example:
select pageid,mycol1, mycol2 from pageAds
lateral view explode_string(col1) myTable1 as mycol1
lateral view explode_string(col2) myTable2 as mycol2;
TableFunctionNode
|----
|- fnExprList: explode_string(col1), explode_string(col2)
1. replace all boost::shared_ptr to std::shared_ptr
2. replace all boost::scopted_ptr to std::unique_ptr
3. replace all boost::scoped_array to std::unique<T[]>
4. replace all boost:thread to std::thread
Users can directly query the data in the hive table in Doris, and can use join to perform complex queries without laboriously importing data from hive.
Main changes list below:
FE:
Extend HiveScanNode from BrokerScanNode
HiveMetaStoreClientHelper communicate with HIVE and HDFS.
BE:
Treate HiveScanNode as BrokerScanNode, treate HiveTable as BrokerTable.
broker_scanner.cpp: suppot read column from HDFS path.
orc_scanner.cpp: support read hdfs file.
POM:
Add hive.version=2.3.7, hive-metastore and hive-exec
Add hadoop.version=2.8.0, hadoop-hdfs
Upgrade commons-lang to fix incompatiblity of Java 9 and later.
Thrift:
Add THiveTable
Add read_by_column_def in TBrokerRangeDesc
