pick (#42491)
For a routine load job, it will be divided into many tasks, each of
which is a transaction. Currently, the default time
consumed(max_batch_interval) is 10 seconds. The benefits of increasing
this value are:
1. Larger batch consumption can lead to better performance.
2. Reducing the number of transactions can alleviate the pressure of
compaction and the conflicts of concurrent transaction submissions.
related doc: https://github.com/apache/doris-website/pull/1236/files
stack info :
```
2024-11-01 14:04:24,570 INFO (replayer|118) [EditLog.loadJournal():249] Begin to unprotect add partition. db = 50402 table = 3355198 partitionName = p2024110115
2024-11-01 14:04:24,572 INFO (replayer|118) [Env.replayJournal():2795] replayed journal id is 91381595, replay to journal id is 91381596
2024-11-01 14:04:24,572 INFO (replayer|118) [EditLog.loadJournal():259] Begin to unprotect drop partition. db = 50402 table = 23712088 partitionName = p2024110112
2024-11-01 14:04:24,573 INFO (replayer|118) [CatalogRecycleBin.recyclePartition():197] recycle partition[30713663-p2024110112] of table [23712088-real_result]
2024-11-01 14:04:24,573 INFO (replayer|118) [OlapTable.updateVisibleVersionAndTime():2740] updateVisibleVersionAndTime, tableName: real_result, visibleVersion, 1653, visibleVersionTime: 1730441064570
2024-11-01 14:04:24,573 WARN (mysql-nio-pool-109891|1159519) [StmtExecutor.executeByLegacy():985] execute Exception. stmt[76210227, 9822ba51b0b14b33-b8c313b570b26832]
java.lang.NullPointerException: null
at org.apache.doris.qe.cache.CacheAnalyzer.buildCacheTableForOlapScanNode(CacheAnalyzer.java:705) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.cache.CacheAnalyzer.buildCacheTableList(CacheAnalyzer.java:516) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.cache.CacheAnalyzer.innerCheckCacheMode(CacheAnalyzer.java:259) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.cache.CacheAnalyzer.getCacheData(CacheAnalyzer.java:528) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.handleCacheStmt(StmtExecutor.java:1615) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.handleQueryStmt(StmtExecutor.java:1721) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.handleQueryWithRetry(StmtExecutor.java:810) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.executeByLegacy(StmtExecutor.java:903) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:597) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:523) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:328) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:206) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:260) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:288) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:341) ~[doris-fe.jar:1.2-SNAPSHOT]
at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) ~[doris-fe.jar:1.2-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_272]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_272]
at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_272]
2024-11-01 14:04:24,578 INFO (replayer|118) [Env.replayJournal():2795] replayed journal id is 91381596, replay to journal id is 91381597
2024-11-01 14:04:24,578 INFO (replayer|118) [EditLog.loadJournal():249] Begin to unprotect add partition. db = 50402 table = 23712088 partitionName = p2024110115
2024-11-01 14:04:24,580 INFO (replayer|118) [Env.replayJournal():2795] replayed journal id is 91381597, replay to journal id is 91381598
2024-11-01 14:04:24,580 INFO (replayer|118) [EditLog.loadJournal():259] Begin to unprotect drop partition. db = 50402 table = 67267 partitionName = p2024110111
```
```
# partition get null here
Partition partition = olapTable.getPartition(partitionId);
# Unable to obtain the specified partition in idToPartition and tempPartitions through partitionId
public Partition getPartition(long partitionId) {
Partition partition = idToPartition.get(partitionId);
if (partition == null) {
partition = tempPartitions.getPartition(partitionId);
}
return partition;
}
```
Because there is reading and writing, the partition should have already
been deleted when reading again
