database/doris
2
0
Fork 0
Code Issues Pull Requests Actions 3 Packages Projects Releases Wiki Activity
18,429 Commits 1 Branch 0 Tags
3cb0deae9ceef92c71c35444c495bc5b94de6c7c
Commit Graph

18429 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
lihangyu
554f13ca12 [Fix](Variant) fix inverted index lost due to wrong column unique id (#31992) 2024-03-15 18:01:22 +08:00
lihangyu
1db57c0667 [Optimization][Scanner] Skip _init_variant_columns when there are no variant columns, and ensure inherit_tablet_index is called only once (#32174) 2024-03-15 18:01:19 +08:00
xy
e6f9bd4023 [Opt](exec) Reduce unnecessary find function calls (#32165) 2024-03-15 18:00:57 +08:00
Xin Liao
3f4ae002a8 [fix](merge-cloud) fix no cluster for common user (#32097) 2024-03-15 18:00:42 +08:00
seawinde
9ffe8842a2 [improvement](mtmv) Remove useless code and add some doc (#32047) 
1. Use visitor to remove the result sink in mv plan
2. Remove useless code
3. Add more doc
 2024-03-15 18:00:42 +08:00
jakevin
628e1d5e41 [fix](Nereids): slot set in condition can be empty (#32169) 2024-03-15 18:00:42 +08:00
Xin Liao
dc687dc4cc [fix](merge-clod) fix file not found when load for mow table (#32144) 2024-03-15 18:00:24 +08:00
924060929
2f89ec961f [enhancement](Nereids) Optimize expression (#32067) 
1. optimize expression comparison by
      a) flatter method call stack
      b) short circuit: if some simple field not equals, then return, and not compare to the big field, like Alias.name
    2. lazy compute Alias.name which is computed by toSql. Now Alias.toSlot() will not generate long name immediately
    3. cache Expresssion.inputSlots, it can save time when invoke this method multiple times
    4. always compute Expression.unbound, it can avoid traverse the big expression tree

this pr can save about 200ms when submit some long sqls
 2024-03-15 18:00:02 +08:00
starocean999
3c4234111b [fix](nereids)EliminateSemiJoin should consider empty table (#32107) 
* [fix](nereids)EliminateSemiJoin should consider empty table

* update out file
 2024-03-15 17:59:31 +08:00
谢健
c8c6e86386 [fix](Nereids): ignore project and distribute in test_cte_filter_pushdown and push_down_expression_in_hash_join (#32083) 2024-03-15 17:59:30 +08:00
feiniaofeiafei
99830b77a8 [feat](nereids) add merge aggregate rule (#31811) 2024-03-15 17:58:01 +08:00
xzj7019
fd1e0e933e [opt](Nereids) outer join with is null stats estimation enhancement (#31875) 2024-03-15 17:58:01 +08:00
zhangstar333
94a75c27e7 [feature](pipelineX) support paritition tablet sink shuffle (#31689) 2024-03-15 17:58:01 +08:00
谢健
8a8a06d8b6 [feat](Nereids) update struct info map when there is new expr (#32119) 2024-03-15 17:58:01 +08:00
Xinyi Zou
7b74b199a5 [fix](memory) Fix LRU cache deleter and memory tracking (#32080) 
In order to add common code to the value deleter of LRU cache, let all lru cache values inherit from LRUCacheValueBase class and tracking memory in destructor.
 2024-03-15 17:57:58 +08:00
wangbo
df5ec16d7c [Refactor](exectuor)Add schema type table active_queries (#32057) 
* Add schema type table active_queries
 2024-03-15 17:57:28 +08:00
Jibing-Li
58675c271b [opt](statistics) Add wait row count reported logic for sample analyze. (#32030) 
If getRowCount returns 0, and table visible version is 2, the table is probably not empty, so we wait a moment for the row count to be reported.
 2024-03-15 17:54:43 +08:00
yiguolei
c20567d088 change to 2.1.1-rc01 2024-03-15 12:59:42 +08:00
plat1ko
6f0aa5fb84 [bug](s3) Fix object data is overwritten by empty object (#32258) 2024-03-15 10:18:45 +08:00
Dongyang Li
5f75b36ad3 [regression](framework) add config caseNamePrefix (#32266) 2024-03-14 21:48:28 +08:00
zy-kkk
bbdce3eb5e [fix](jdbc catalog) fix jdbc-connector coredump as get env return nullptr (#32217) 2024-03-14 16:05:08 +08:00
yiguolei
20d6698c27 [bugfix](arm compile) could not compile on arm because -Werror=maybe-uninitialized 2024-03-14 12:11:25 +08:00
zclllyybb
847ec368be [Fix](smooth-upgrade) Fix incompatibility when upgrade from 2.0 to 2.1 (#32220) 2024-03-14 11:23:05 +08:00
KassieZ
d0e296e88d [docs](docs) Fix invalid link and typo of v2.1 (#32216) 2024-03-14 11:16:05 +08:00
walter
31827e88e3 [fix](merge-cloud) Fix ali open sdk gcc compile issue (#30308) 2024-03-14 10:59:25 +08:00
HappenLee
b031c95324 [Opt](exec) use libbase64 to replace base64 code in doris (#32078) 
* [Opt](exec) use libbase64 to replace base64 code in doris
 2024-03-14 09:20:50 +08:00
HappenLee
8ae493e54f [thirdparty](lib) Add Patch for base64 lib (#32059) 2024-03-14 09:20:45 +08:00
HappenLee
f2404ff553 [Opt](exec) remove the unless mem alloc in base64 (#32019) 2024-03-14 09:19:40 +08:00
HappenLee
f9ee3ff359 [thirdparty](lib) Add base64 thirdparty lib (#31850) 2024-03-14 09:18:52 +08:00
Pxl
6b08a4ec93 [Bug](top-n) do not get runtime predicate when predicate not initialized #32209 2024-03-14 09:12:09 +08:00
Pxl
2f4401189a [Bug](top-n) do not update topn filter when sort node and scan node are not in the… (#32159) 2024-03-13 16:21:36 +08:00
Jeffrey
5539f85fec [fix](ui) fix database cannot be choosed bug (#32091) 2024-03-13 16:21:13 +08:00
Mingyu Chen
6d2924668e [fix](audit-loader) fix invalid token check logic (#32095) 
The check of the token should be forwarded to Master FE.
I add a new RPC method `checkToken()` in Frontend for this logic.
Otherwise, after enable the audit loader, the log from non-master FE can not be loaded to audit table
with `Invalid token` error.
 2024-03-12 22:52:11 +08:00
Pxl
fa111c371b [Chore](top-n) check runtime predicate inited when scan operator open (#32140) 
check runtime predicate inited when scan operator open
 2024-03-12 22:52:10 +08:00
yiguolei
5da7cd0fba [bugfix](becore) has to use value to capture lambda value to avoid core during callback (#32132) 
Co-authored-by: yiguolei <yiguolei@gmail.com>
 2024-03-12 22:51:44 +08:00
zhangdong
84af8e0a53 [enhance](mtmv)mtmv support hive default partition (#32051) 2024-03-12 22:51:11 +08:00
abmdocrt
6acdd9cd48 The issue introduced by the recently added auto-increment column works fine on a single node but may result in discontinuous auto-increment IDs when running on a cluster. This PR has been modified to check for the uniqueness of the auto-increment column values instead of checking for equality to a fixed value. (#32115) 2024-03-12 21:51:36 +08:00
yiguolei
0159a75ced [bugfix](becore) be will core when stop because the map is modified during iterator (#32105) 
Co-authored-by: yiguolei <yiguolei@gmail.com>
 2024-03-12 18:50:26 +08:00
Jerry Hu
473bd3ee64 [fix](function) incorrect result of eq_for_null (#32103) 2024-03-12 18:50:26 +08:00
xueweizhang
4956d5de83 [fix](planner) remove input slot for aggregate slot which is not materialized (#32092) 
introduced by #26886

run this sql:
SELECT
        caseId
    FROM
        (
            SELECT
                caseId,
                count(judgementDateId)
            FROM
                (
                    SELECT
                        abs(caseId) AS caseId,
                        id as judgementDateId
                    FROM
                        dr_user_test_t2
                ) AGG_RESULT
            GROUP BY
                caseId
        ) TOTAL
        order by 1;


will get:

ERROR 1105 (HY000): errCode = 2, detailMessage = (172.17.0.1)[INTERNAL_ERROR]couldn't resolve slot descriptor 1, desc: tuples:
Tuple(id=5 slots=[Slot(id=10 type=DOUBLE col=-1, colname=, nullable=1), Slot(id=11 type=VARCHAR col=-1, colname=id, nullable=1)] has_varlen_slots=1)
Tuple(id=4 slots=[Slot(id=8 type=DOUBLE col=-1, colname=, nullable=1)] has_varlen_slots=0)
Tuple(id=2 slots=[Slot(id=4 type=DOUBLE col=-1, colname=caseId, nullable=1)] has_varlen_slots=0)
Tuple(id=0 slots=[Slot(id=0 type=VARCHAR col=-1, colname=caseId, nu
 2024-03-12 18:50:26 +08:00
feiniaofeiafei
781a45d93c [Fix](nereids) fix date function rewrite (#32060) 2024-03-12 18:50:26 +08:00
HHoflittlefish777
6610307eb0 [opt](routine-load) end Kafka consume when meets partition EOF #32046 2024-03-12 18:50:06 +08:00
谢健
2da57526a3 [feat](Nereids): use table map to construct struct info (#32058) 2024-03-12 18:50:06 +08:00
shuke
45824d959c [release](build script) fix build release bug when build multiple version on same machine #32104 2024-03-12 18:50:06 +08:00
924060929
cf04c9c300 [enhancement](Nereids) refine and speedup analyzer (#31792) (#32111) 
## Proposed changes
1. check data type whether can applied should not throw exception when real data type is subclass of signature data type
2. merge `SlotBinder` and `FunctionBinder` to `ExpressionAnalyzer` to skip rewrite the whole expression tree multiple times.
3. `ExpressionAnalyzer.buildCustomSlotBinderAnalyzer()` provide more refined code to bind slot by different parts and different priority
4. the origin slot binder has O(n^2) complexity, this pr use `Scope.nameToSlot` to support O(n) bind
5. modify some `Collection.stream()` to `ImmutableXxx.builder()` to remove some method call which are difficult to inline by jvm in the hot path, e.g. `Expression.<init>` and `AbstractTreeNode.<init>`
6. modify some `ImmutableXxx.copyOf(xxx)` to `Utils.fastToImmutableList(xxx)` to skip addition copy of the array
7. set init size to `Immutable.builder()` to skip some useless resize
8. lazy compute and cache some heavy operations, like `Scope.nameToSlot` and `CaseWhen.computeDataTypesForCoercion()`

(cherry picked from commit 83c2f5a95827136aac4f0a78c5e841e9a099858c)
 2024-03-12 17:09:38 +08:00
qiye
5f125bbaaa [improvement](binlog)Support inverted index in CCR (#31743) (#32101) 2024-03-12 15:34:08 +08:00
Gabriel
61928f7df5 [pipelineX](scanner) Use the actual instances num when ignore data distribution (#32081) 2024-03-12 14:20:39 +08:00
AlexYue
ffa904c487 [enhance](Cooldown) Skip cooldown if the tablet is dropped (#32079) 2024-03-12 14:20:39 +08:00
Mingyu Chen
2ffcfb4bc5 [fix](doc) fix wrong download link (#32082) 2024-03-12 14:20:39 +08:00
lihangyu
ae206596c9 [DOC](Variant) add bloom filter description and correct some benchmark results (#31998) 2024-03-12 14:20:39 +08:00