this pr is a quick solution, but not complete.
runtime filter on NestLoopJoin suffers this bug even without PROJECT_OTHER_JOIN_CONDITION rule.
for example, when enable Min_Max Runtime filter, the target Expression is n_regionkey, but it should be "n_regionkey - 28"
explain
select n_nationkey, nrkey
from (select n_regionkey -28 nrkey, n_nationkey from nation) T
join region on nrkey > r_regionkey;
we will refactor RuntimeFilterGenerator to completely solve this issue in following pr.
we do derive stats job eager to avoid un derive stats due to merge group and optimize group
consider:
we have two groups burned by order: G1 and G2
then we have job by order derive G2, optimize group expression in G2,
derive G1, optimize group expression in G1
if G1 merged into G2, then we maybe generated job optimize group G2 before derive G1
in this case, we will do get stats from G1's child before derive G1's child stats
then we will meet NPE in CostModel.
1. we should check output set, since we will remove top project and
result output size will diff with its child output size if there are
dup slot in result list
2. should not return null, instead we should return rewritten plan
itself, because we will use return result in many place and do not
check null at all
* Support statistics
* - Fix Failed task not showing up in the task list
- Task metadata add jobName
- Fix Finished job clear time error
- Job metadata add successCount, failedCount, totalTaskCount
* add test
pick from master #33979
commit id 65fb7d43b7e838c48502d4e8a69e2541dc73aa88
This PR:
1. add a new Command type: UnsupportedCommand to handle the statement only support parse but could not execute.
2. support syntax about set and use
3. add keyword VAULT to follow legacy planner
TODO
1. support all statment syntax in Nereids
In MySQL, it's common to use a simplified syntax like `SELECT constant FROM dual`
which is equivalent to just `SELECT constant`.
This syntax is often used by BI tools when utilizing MySQL connectors to verify connection validity.
To enhance compatibility and ensure seamless integration with such tools,
we have now implemented this feature in Doris.
### Key Changes:
- Doris now interprets `SELECT constant FROM dual` as `SELECT constant`, aligning with MySQL's behavior.
- This update ensures that BI tools can use standard MySQL connectors without modifications or errors when connecting to Doris.
In SQL syntax, `col != ''` equals `col.length() > 0`.
It means that this column must exist in ES doc fields and its content is not empty.
In this PR, we make a special translation for this binary predicate to keep the behavior of both consistent.
---------
Co-authored-by: Luennng <luennng@gmail.com>
Record increase refersh version more accurately.
The refreshVersion in the memo will increase when mv rewrite successfully.
In query rewrite, if refresh version is different from the current struct info map in group,
will refresh the group struct info or not.
Expand bucket shuffle downgrade condition, which originally requiring a single partition after pruning, basic table and bucket number < para number. Currently, we expect this option can be used for disabling bucket shuffle more efficiently, without above restrictions.
Co-authored-by: zhongjian.xzj <zhongjian.xzj@zhongjianxzjdeMacBook-Pro.local>
