Commit Graph

17518 Commits

Author SHA1 Message Date
fd1e0e933e [opt](Nereids) outer join with is null stats estimation enhancement (#31875) 2024-03-15 17:58:01 +08:00
94a75c27e7 [feature](pipelineX) support paritition tablet sink shuffle (#31689) 2024-03-15 17:58:01 +08:00
8a8a06d8b6 [feat](Nereids) update struct info map when there is new expr (#32119) 2024-03-15 17:58:01 +08:00
7b74b199a5 [fix](memory) Fix LRU cache deleter and memory tracking (#32080)
In order to add common code to the value deleter of LRU cache, let all lru cache values inherit from LRUCacheValueBase class and tracking memory in destructor.
2024-03-15 17:57:58 +08:00
df5ec16d7c [Refactor](exectuor)Add schema type table active_queries (#32057)
* Add schema type table active_queries
2024-03-15 17:57:28 +08:00
58675c271b [opt](statistics) Add wait row count reported logic for sample analyze. (#32030)
If getRowCount returns 0, and table visible version is 2, the table is probably not empty, so we wait a moment for the row count to be reported.
2024-03-15 17:54:43 +08:00
c20567d088 change to 2.1.1-rc01 2024-03-15 12:59:42 +08:00
6f0aa5fb84 [bug](s3) Fix object data is overwritten by empty object (#32258) 2024-03-15 10:18:45 +08:00
5f75b36ad3 [regression](framework) add config caseNamePrefix (#32266) 2024-03-14 21:48:28 +08:00
bbdce3eb5e [fix](jdbc catalog) fix jdbc-connector coredump as get env return nullptr (#32217) 2024-03-14 16:05:08 +08:00
20d6698c27 [bugfix](arm compile) could not compile on arm because -Werror=maybe-uninitialized 2024-03-14 12:11:25 +08:00
847ec368be [Fix](smooth-upgrade) Fix incompatibility when upgrade from 2.0 to 2.1 (#32220) 2024-03-14 11:23:05 +08:00
d0e296e88d [docs](docs) Fix invalid link and typo of v2.1 (#32216) 2024-03-14 11:16:05 +08:00
31827e88e3 [fix](merge-cloud) Fix ali open sdk gcc compile issue (#30308) 2024-03-14 10:59:25 +08:00
b031c95324 [Opt](exec) use libbase64 to replace base64 code in doris (#32078)
* [Opt](exec) use libbase64 to replace base64 code in doris
2024-03-14 09:20:50 +08:00
8ae493e54f [thirdparty](lib) Add Patch for base64 lib (#32059) 2024-03-14 09:20:45 +08:00
f2404ff553 [Opt](exec) remove the unless mem alloc in base64 (#32019) 2024-03-14 09:19:40 +08:00
f9ee3ff359 [thirdparty](lib) Add base64 thirdparty lib (#31850) 2024-03-14 09:18:52 +08:00
Pxl
6b08a4ec93 [Bug](top-n) do not get runtime predicate when predicate not initialized #32209 2024-03-14 09:12:09 +08:00
Pxl
2f4401189a [Bug](top-n) do not update topn filter when sort node and scan node are not in the… (#32159) 2024-03-13 16:21:36 +08:00
5539f85fec [fix](ui) fix database cannot be choosed bug (#32091) 2024-03-13 16:21:13 +08:00
6d2924668e [fix](audit-loader) fix invalid token check logic (#32095)
The check of the token should be forwarded to Master FE.
I add a new RPC method `checkToken()` in Frontend for this logic.
Otherwise, after enable the audit loader, the log from non-master FE can not be loaded to audit table
with `Invalid token` error.
2024-03-12 22:52:11 +08:00
Pxl
fa111c371b [Chore](top-n) check runtime predicate inited when scan operator open (#32140)
check runtime predicate inited when scan operator open
2024-03-12 22:52:10 +08:00
5da7cd0fba [bugfix](becore) has to use value to capture lambda value to avoid core during callback (#32132)
Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-03-12 22:51:44 +08:00
84af8e0a53 [enhance](mtmv)mtmv support hive default partition (#32051) 2024-03-12 22:51:11 +08:00
6acdd9cd48 The issue introduced by the recently added auto-increment column works fine on a single node but may result in discontinuous auto-increment IDs when running on a cluster. This PR has been modified to check for the uniqueness of the auto-increment column values instead of checking for equality to a fixed value. (#32115) 2024-03-12 21:51:36 +08:00
0159a75ced [bugfix](becore) be will core when stop because the map is modified during iterator (#32105)
Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-03-12 18:50:26 +08:00
473bd3ee64 [fix](function) incorrect result of eq_for_null (#32103) 2024-03-12 18:50:26 +08:00
4956d5de83 [fix](planner) remove input slot for aggregate slot which is not materialized (#32092)
introduced by #26886

run this sql:
SELECT
        caseId
    FROM
        (
            SELECT
                caseId,
                count(judgementDateId)
            FROM
                (
                    SELECT
                        abs(caseId) AS caseId,
                        id as judgementDateId
                    FROM
                        dr_user_test_t2
                ) AGG_RESULT
            GROUP BY
                caseId
        ) TOTAL
        order by 1;


will get:

ERROR 1105 (HY000): errCode = 2, detailMessage = (172.17.0.1)[INTERNAL_ERROR]couldn't resolve slot descriptor 1, desc: tuples:
Tuple(id=5 slots=[Slot(id=10 type=DOUBLE col=-1, colname=, nullable=1), Slot(id=11 type=VARCHAR col=-1, colname=id, nullable=1)] has_varlen_slots=1)
Tuple(id=4 slots=[Slot(id=8 type=DOUBLE col=-1, colname=, nullable=1)] has_varlen_slots=0)
Tuple(id=2 slots=[Slot(id=4 type=DOUBLE col=-1, colname=caseId, nullable=1)] has_varlen_slots=0)
Tuple(id=0 slots=[Slot(id=0 type=VARCHAR col=-1, colname=caseId, nu
2024-03-12 18:50:26 +08:00
781a45d93c [Fix](nereids) fix date function rewrite (#32060) 2024-03-12 18:50:26 +08:00
6610307eb0 [opt](routine-load) end Kafka consume when meets partition EOF #32046 2024-03-12 18:50:06 +08:00
2da57526a3 [feat](Nereids): use table map to construct struct info (#32058) 2024-03-12 18:50:06 +08:00
45824d959c [release](build script) fix build release bug when build multiple version on same machine #32104 2024-03-12 18:50:06 +08:00
cf04c9c300 [enhancement](Nereids) refine and speedup analyzer (#31792) (#32111)
## Proposed changes
1. check data type whether can applied should not throw exception when real data type is subclass of signature data type
2. merge `SlotBinder` and `FunctionBinder` to `ExpressionAnalyzer` to skip rewrite the whole expression tree multiple times.
3. `ExpressionAnalyzer.buildCustomSlotBinderAnalyzer()` provide more refined code to bind slot by different parts and different priority
4. the origin slot binder has O(n^2) complexity, this pr use `Scope.nameToSlot` to support O(n) bind
5. modify some `Collection.stream()` to `ImmutableXxx.builder()` to remove some method call which are difficult to inline by jvm in the hot path, e.g. `Expression.<init>` and `AbstractTreeNode.<init>`
6. modify some `ImmutableXxx.copyOf(xxx)` to `Utils.fastToImmutableList(xxx)` to skip addition copy of the array
7. set init size to `Immutable.builder()` to skip some useless resize
8. lazy compute and cache some heavy operations, like `Scope.nameToSlot` and `CaseWhen.computeDataTypesForCoercion()`

(cherry picked from commit 83c2f5a95827136aac4f0a78c5e841e9a099858c)
2024-03-12 17:09:38 +08:00
5f125bbaaa [improvement](binlog)Support inverted index in CCR (#31743) (#32101) 2024-03-12 15:34:08 +08:00
61928f7df5 [pipelineX](scanner) Use the actual instances num when ignore data distribution (#32081) 2024-03-12 14:20:39 +08:00
ffa904c487 [enhance](Cooldown) Skip cooldown if the tablet is dropped (#32079) 2024-03-12 14:20:39 +08:00
2ffcfb4bc5 [fix](doc) fix wrong download link (#32082) 2024-03-12 14:20:39 +08:00
ae206596c9 [DOC](Variant) add bloom filter description and correct some benchmark results (#31998) 2024-03-12 14:20:39 +08:00
65835cdd2b [doc](sql-dialect) update sql convertor tool to 1.0.1 (#32077) 2024-03-12 14:20:39 +08:00
da60a111d0 [refactor](nereids) rename PlanNode.distributeExprLists to childrenDistributeExprLists #32069 2024-03-12 14:20:39 +08:00
194f3432ab [Improvement](executor)Routine load support workload group #31671 2024-03-12 14:20:18 +08:00
b41b17ad0a [fix](spill) fix storage engine type cast error (#32071) 2024-03-12 14:20:18 +08:00
a937c54b2b [fix](docker)Fix the registration logic of Doris On Docker (#32026)
* Refactor upgrade documentation

* fix docker be entry point shell

---------

Co-authored-by: Yijia Su <suyijia@selectdb.com>
2024-03-12 14:20:18 +08:00
2ee12b8678 [improvement](docker)Optimize Dockerfile content for FE and BE (#32027)
* Refactor upgrade documentation

* Optimize Dockerfile content for FE and BE.

* Optimize Dockerfile content for FE and BE.

---------

Co-authored-by: Yijia Su <suyijia@selectdb.com>
2024-03-12 14:20:18 +08:00
aea9ddc3cb [Fix](Outfiel) fix be core when the open method of vfile_result_writer failed #32042 2024-03-12 14:20:18 +08:00
dc7d80860f [fix](case) fix export data consistency table key type (#32045) 2024-03-12 14:20:18 +08:00
926908ece2 [fix](hive) fix spelling mistakes for "separatorChar" #32061 2024-03-12 14:20:18 +08:00
4268634115 [fix](memory) Fix Allocator cancel pipelinex query #32048 2024-03-12 14:20:18 +08:00
ccd21a6ea4 [Improve](InPredict) enhance in predict with array type (#31828) 2024-03-12 14:19:14 +08:00