mv in select materialized_view should disable show table,
because Nereids planner can output the string such as
slot#[0] in toSql() of SlotRef. Note this is only a
temporary solution, will use an expression translator later
select rank() over (partition by A, B) as r, sum(x) over(A, C) as s from T;
A is a common partition key for all windowExpressions, that is A is intersection of {A,B} and {A, C}
we could push filter A=1 through this window, since A is a common Partition key:
select * from (select a, row_number() over (partition by a) from win) T where a=1;
origin plan:
----filter((T.a = 1))
----------PhysicalWindow
------------PhysicalQuickSort
--------------PhysicalProject
------------------PhysicalOlapScan[win]
transformed to
----PhysicalWindow
------PhysicalQuickSort
--------PhysicalProject
----------filter((T.a = 1))
------------PhysicalOlapScan[win]
But C=1 can not be pushed through window.
Issue Number: close #xxx
ShowTableStmtTest.testNoDb and DropDbStmtTest.testNoPriv are unstable cases，error msg is:
java.lang.Exception: Unexpected exception, expected<org.apache.doris.common.AnalysisException> but was<mockit.internal.expectations.invocation.MissingInvocation>
we can not know what is missing ,and this issue cannot be reproduced locally,so add some log
When be load reblancer choose candidate tablets, it will try moving tablets from high load backends to low backend backends. If the higher HIGH BE has no available slot num, it should try next HIGH BE.
Sometimes FE replica's version is unreliable. FE's replica may bigger than BE's real version. Need check if BE missing version (last failed version > 0).
There are 1 security vulnerabilities found in gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b
CVE-2022-28948
What did I do？
Upgrade gopkg.in/yaml.v3 from v3.0.0-20210107192922-496545a6307b to 3.0.0 for vulnerability fix
What did you expect to happen？
Ideally, no insecure libs should be used.
How can we automate the detection of these types of issues?
By using the GitHub Actions configurations provided by murphysec, we can conduct automatic code security checks in our CI pipeline.
The specification of the pull request
PR Specification from OSCS
When executing admin clean trash, if the backend daemon clean thread is cleaning trash, then SQL command will return immediately. But for the backend daemon thread, it doesn't clean all the trashes, it clean only the expired trashes.
Also if there's lots of trashes, the daemon clean thread will busy handling trashes for a long time.
void throw analysis ex for unsupported type to make ext table goes nereids.
this will improve the nereids' availability for external table if unsupported type is in the basic table schema but not referenced in the real sql.
tested in external table env.
Consider the following case:
select pu.pk_ct_pu as id
from fms_rd_nc65_zb.NC65P.CT_PU pu
left join fms_rd_nc65_zb.NC65P.PUB_WF_INSTANCE pwi
on pu.pk_ct_pu = pwi.billid
and pu.vtrantypecode=pwi.billtype
left join fms_rd_nc65_zb.NC65P.SM_USER su
on pu.creator = su.cuserid
where pu.pk_ct_pu='1001A110000000K8XPVN';
PUB_WF_INSTANCE table has a BLOB type column and currently it will throw analysis exception and fallback to old optimizer, although this column is not referenced in the real sql. The old optimizer doesn't have the outer join -> inner join rule and the "pu.pk_ct_pu='1001A110000000K8XPVN'; " is not pushed down and the performance will drop down. After the pr, we add the unsupported type instead of throw exception directly, it will decide the unused case and can continue goes nereids and use all advanced optimization for the sql.
1. The catalog is connected to an old version of Doris, and an error is reported when using the datetime field type on the Doris surface .
2. error message [fe]:
Caused by: java.lang.NumberFormatException: For input string: "DATETIM"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) ~[?:1.8.0_291]
at java.lang.Integer.parseInt(Integer.java:580) ~[?:1.8.0_291]
at java.lang.Integer.parseInt(Integer.java:615) ~[?:1.8.0_291]
at org.apache.doris.datasource.jdbc.client.JdbcMySQLClient.dorisTypeToDoris(JdbcMySQLClient.java:401) ~[doris-fe.jar:1.2-
SNAPSHOT]
In the original logic, the `Export` statement generates `Selectstmt` for execution. But there is no way to make the `SelectStmt` use the new optimizer.
Now, we change the `Export` statement to generate the `outfile SQL`, and then use the new optimizer to parse the SQL so that outfile can use the new optimizer.
Before, Statistics Cleaner only handles olap db and table.
External db and tables would be removed without verification. So that external stats could stored no more than 2 days, which is the interval of Stats cleaner thread.
This pr is to add verification for external db and tables.
