[feature](jni) add jni metrics and attach to BE profile automatically (#21004)

Add JNI metrics, for example:
```
-  HudiJniScanner:  0ns
  -  FillBlockTime:  31.29ms
  -  GetRecordReaderTime:  1m5s
  -  JavaScanTime:  35s991ms
  -  OpenScannerTime:  1m6s
```
Add three common performance metrics for JNI scanner:
1. `OpenScannerTime`: Time to init and open JNI scanner
2. `JavaScanTime`: Time to scan data and insert into vector table in java side
3. `FillBlockTime`: Time to convert java vector table to c++ block

And support user defined metrics in java side, for example: `OpenScannerTime` is a long time for the open process, we want to determine which sub-process takes too much time, so we add `GetRecordReaderTime` in java side.
The user defined metrics in java side can be attached to BE profile automatically.
This commit is contained in:
Ashin Gau
2023-06-21 11:19:02 +08:00
committed by GitHub
parent b4773e1195
commit ef17289925
6 changed files with 174 additions and 23 deletions

View File

@ -24,6 +24,8 @@ import org.apache.doris.common.jni.vec.ScanPredicate;
import org.apache.doris.common.jni.vec.VectorTable;
import java.io.IOException;
import java.util.Collections;
import java.util.Map;
public abstract class JniScanner {
protected VectorTable vectorTable;
@ -79,6 +81,15 @@ public abstract class JniScanner {
return getMetaAddress(numRows);
}
/**
* Get performance metrics. The key should be pattern like "metricType:metricName".
* Support three metric types: timer, counter and bytes.
* The c++ side will attach metricName into profile automatically.
*/
public Map<String, String> getStatistics() {
return Collections.emptyMap();
}
private long getMetaAddress(int numRows) {
vectorTable.setNumRows(numRows);
return vectorTable.getMetaAddress();