Increase the functionality of advanced materialized view
This feature already supported by legacy planner with PR #19650
This PR implement it in Nereids. This PR implement the features as below:
1. Support multiple columns in aggregate function. eg: select sum(c1 + c2) from t1;
2. Supports complex expressions. eg: select abs(c1), sum(abc(c1+1) + 1) from t1;
TODO:
1. Support adding where in materialized view
We encountered one confusing situation where buffered reader were trapped in one endless loop when calling readat. Then we found out that it was all due to the return data size is less than requested.
As the following picture shows, the actual data size is about 2M, and when we called readat it only retrieved about 1MB.
before: the node will wait to retrieve all data from child, then send data to parent.
now: for data from child that does not require sorting, it can be sent to parent immediately.
* [Bug](point query) checkAndSetPointQuery before checkEnableTwoPhaseRead
1. checkEnableTwoPhaseRead rely on thr short circuit flag
2. add more metric to display lookup profile
* fix rebase
#18976 introduced merge small IO facility to optimize performance, and used by parquet reader.
This PR support this facility in orc reader. Current ORC reader implementation need to reposition parent present stream when reading lazy columns in lazy materialization facility. So let it works by removing `DCHECK_GE(offset, cached_data.end_offset)`.
Fix errors when inserting string/date/datetime values into SQLServer:
ERROR 1105 (HY000): errCode = 2, detailMessage = (172.21.0.101)[INTERNAL_ERROR]UdfRuntimeException: JDBC executor sql has error:
CAUSED BY: SQLServerException: Invalid column name '2021-10-30'.
When using double quotes enclose string values, it will be parsed as column name, so we should enclose string values with single quotes.
We can not use the string as the variable key to use in the hint.
Before this PR
mysql> SET enable_nereids_planner=true;
Query OK, 0 rows affected (0.01 sec)
mysql> set enable_fallback_to_original_planner=false;
Query OK, 0 rows affected (0.10 sec)
mysql> explain select /*+ SET_var("enable_nereids_planner" = "false") */ 1;
ERROR 1105 (HY000): Exception, msg: Nereids cannot parse the SQL, and fallback disabled. caused by:
no viable alternative at input 'select /*+ SET_var("enable_nereids_planner"'(line 1, pos 27)
After this PR
mysql> SET enable_nereids_planner=true;
Query OK, 0 rows affected (0.01 sec)
mysql> set enable_fallback_to_original_planner=false;
Query OK, 0 rows affected (0.10 sec)
mysql> select /*+ SET_var("enable_nereids_planner" = "false") */ 1;
+------+
| 1 |
+------+
| 1 |
+------+
1 row in set (0.00 sec)
Describe your changes.
Support the string for the hint key in the Parser.
Before change, when doing optimize use Nereids planner, input will serialize to memory first. And when bug happen, it would be dump to minidump file when catching the exception.
We found that serialization process will cause the performance when statistic message too large or when optimization time be small enough.
So the user minidump using should change to ONLY YOU OPEN MINIDUMP SWITCH(set enable_minidump=true;) can you use it.
1. Before this PR if rowset does not contain column which should be read for related SlotDescriptor will call `insert_default` to column, but it's not this real defautl value.Real default value relevant information should be provided by the frontend side.
2. Support fetch when light schema change is not enabled, but disable for AGG or UNIQUE MOR model
When the postgresql bit type size is 1, it reads as a java.lang.boolean via jdbc, and if we match against string,
it will display true or false. But the normal display should be a number,
so when I detect that the size of bit is 1, I will match it with boolean
When FE is deployed on a virtual machine and CN is deployed on k8s, FE needs to use a proxy IP to communicate with
CN nodes, and BE cannot resolve the proxy IP from the local network card.
We change the previous verification rules and obey the IP assigned by the master
