when olap table is dynamic partition enable, if drop and recover olap table, the table should be added to DynamicPartitionScheduler again
---------
Co-authored-by: caiconghui1 <caiconghui1@jd.com>
* [Fix](inverted index) fix memeory leak when inverted index writer do not finish correctly
* [Update](inverted index) use smart pointer to avoid memeory leak
* [Chore](format) code format
---------
Co-authored-by: airborne12 <airborne12@gmail.com>
before
mysql [(none)]>select cast("10:10:10" as time);
+-------------------------------+
| CAST('10:10:10' AS TIMEV2(0)) |
+-------------------------------+
| 00:00:00 |
+-------------------------------+
after
mysql [(none)]>select cast("10:10:10" as time);
+-------------------------------+
| CAST('10:10:10' AS TIMEV2(0)) |
+-------------------------------+
| 10:10:10 |
+-------------------------------+
In the past, we supported this syntax.
mysql [(none)]>select cast("2023:05:01 13:14:15" as time);
+------------------------------------------+
| CAST('2023:05:01 13:14:15' AS TIMEV2(0)) |
+------------------------------------------+
| 13:14:15 |
+------------------------------------------+
However, "10:10:10" is also a valid datetime.
mysql [(none)]>select cast("10:10:10" as datetime);
+-----------------------------------+
| CAST('10:10:10' AS DATETIMEV2(0)) |
+-----------------------------------+
| 2010-10-10 00:00:00 |
+-----------------------------------+
So here, the order of parsing has been adjusted.
Refactoring the filtering conditions in the current ExecNode from an expression tree to an array can simplify the process of adding runtime filters. It eliminates the need for complex merge operations and removes the requirement for the frontend to combine expressions into a single entity.
By representing the filtering conditions as an array, each condition can be treated individually, making it easier to add runtime filters without the need for complex merging logic. The array can store the individual conditions, and the runtime filter logic can iterate through the array to apply the filters as needed.
This refactoring simplifies the codebase, improves readability, and reduces the complexity associated with handling filtering conditions and adding runtime filters. It separates the conditions into discrete entities, enabling more straightforward manipulation and management within the execution node.
Increase the functionality of advanced materialized view
This feature already supported by legacy planner with PR #19650
This PR implement it in Nereids. This PR implement the features as below:
1. Support multiple columns in aggregate function. eg: select sum(c1 + c2) from t1;
2. Supports complex expressions. eg: select abs(c1), sum(abc(c1+1) + 1) from t1;
TODO:
1. Support adding where in materialized view
We encountered one confusing situation where buffered reader were trapped in one endless loop when calling readat. Then we found out that it was all due to the return data size is less than requested.
As the following picture shows, the actual data size is about 2M, and when we called readat it only retrieved about 1MB.
before: the node will wait to retrieve all data from child, then send data to parent.
now: for data from child that does not require sorting, it can be sent to parent immediately.
* [Bug](point query) checkAndSetPointQuery before checkEnableTwoPhaseRead
1. checkEnableTwoPhaseRead rely on thr short circuit flag
2. add more metric to display lookup profile
* fix rebase
#18976 introduced merge small IO facility to optimize performance, and used by parquet reader.
This PR support this facility in orc reader. Current ORC reader implementation need to reposition parent present stream when reading lazy columns in lazy materialization facility. So let it works by removing `DCHECK_GE(offset, cached_data.end_offset)`.
Fix errors when inserting string/date/datetime values into SQLServer:
ERROR 1105 (HY000): errCode = 2, detailMessage = (172.21.0.101)[INTERNAL_ERROR]UdfRuntimeException: JDBC executor sql has error:
CAUSED BY: SQLServerException: Invalid column name '2021-10-30'.
When using double quotes enclose string values, it will be parsed as column name, so we should enclose string values with single quotes.
We can not use the string as the variable key to use in the hint.
Before this PR
mysql> SET enable_nereids_planner=true;
Query OK, 0 rows affected (0.01 sec)
mysql> set enable_fallback_to_original_planner=false;
Query OK, 0 rows affected (0.10 sec)
mysql> explain select /*+ SET_var("enable_nereids_planner" = "false") */ 1;
ERROR 1105 (HY000): Exception, msg: Nereids cannot parse the SQL, and fallback disabled. caused by:
no viable alternative at input 'select /*+ SET_var("enable_nereids_planner"'(line 1, pos 27)
After this PR
mysql> SET enable_nereids_planner=true;
Query OK, 0 rows affected (0.01 sec)
mysql> set enable_fallback_to_original_planner=false;
Query OK, 0 rows affected (0.10 sec)
mysql> select /*+ SET_var("enable_nereids_planner" = "false") */ 1;
+------+
| 1 |
+------+
| 1 |
+------+
1 row in set (0.00 sec)
Describe your changes.
Support the string for the hint key in the Parser.
Before change, when doing optimize use Nereids planner, input will serialize to memory first. And when bug happen, it would be dump to minidump file when catching the exception.
We found that serialization process will cause the performance when statistic message too large or when optimization time be small enough.
So the user minidump using should change to ONLY YOU OPEN MINIDUMP SWITCH(set enable_minidump=true;) can you use it.
