Commit Graph

16456 Commits

Author SHA1 Message Date
e693249f5b [fix](Nereids) remove useless substr of string like type cast (#29997) 2024-01-16 20:23:09 +08:00
f7a340a2df [improve](move-memtable) add cancel method to load stream stub (#29994) 2024-01-16 20:23:09 +08:00
d101234be1 [fix](test) fix disableDebugPointForAllBEs do not execute (#30023) 2024-01-16 20:23:09 +08:00
a0355a6580 [opt](scanner) optimize the number of threads of scanners, follow up #28640 (#30030)
follow up #28640, doris_scanner_thread_pool_thread_num may be to large in machines with 128 cores.
2024-01-16 20:23:09 +08:00
4e41e1d797 [feat](Nereids) persist constraint in table (#29767) 2024-01-16 18:49:29 +08:00
2916745cf2 [Chore](arm) Support auto-SIMD on arm and refactor compilation docs (#29956) 2024-01-16 18:49:04 +08:00
168afdb965 [fix](disk balance) Change disk rebalance unpick time to configurable (#28949) 2024-01-16 18:49:04 +08:00
74991c4af2 [bugfix](paimon)support native and jni to read paimon for minio/cos #29933 2024-01-16 18:49:01 +08:00
4bf4239d7a [feature](Nereids): optimize logical group expression in dphyp (#30000) 2024-01-16 18:48:20 +08:00
d11e797d4c [fix](decimal) fix wrong decimal overflow caused by uninitialized nested column of null value (#29960) 2024-01-16 18:48:20 +08:00
6ce9c67383 [fix](Nereids) struct type coercion call wrong api (#30018) 2024-01-16 18:48:20 +08:00
d3bf23d70d [chore](removelogs) remove debug query timeout logs (#30006)
---------

Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-01-16 18:48:18 +08:00
55a0f90003 [fix](jobs)fix create task int job (#30011) 2024-01-16 18:46:19 +08:00
75cafa8672 [enhancement](jdbc catalog) Enhance function pushdown of Jdbc Oracle Catalog (#29972) 2024-01-16 18:46:19 +08:00
f53d2c28cb [improvement](catalog) fix jdbc mysql catalog to_date fun pushdown (#29900) 2024-01-16 18:46:19 +08:00
08db92a49a [fix](regression) spare .testfile to make disk checker happy when injecting fault (#30012) 2024-01-16 18:46:19 +08:00
3c502f6444 [fix](case) add sync after streamload (#30009)
Co-authored-by: stephen <hello-stephen@qq.com>
2024-01-16 18:46:19 +08:00
22978726e3 [opt](nereids) if column stats are unknown, 10-20 table-join optimization use cascading instead of dphyp (#29902)
* if column stats are unknown, do not use dphyp
tpcds query64 is optimized in case of no stats
sf500, query64 improved from 15sec to 7sec on hdfs, and from 4sec to 3.85sec on olaptable
2024-01-16 18:46:19 +08:00
e1bcdc35fd (fix)[group-commit] Fix some group commit case (#30008) 2024-01-16 18:46:19 +08:00
4d6de1f181 [opt](invert index) optimize the code structure of inverted index queries (#29771) 2024-01-16 18:46:19 +08:00
07de535c4c [fix](Nereids) should not fold constant when do ordinal group by (#29976) 2024-01-16 18:46:19 +08:00
2297a5b6fc [doc](statistics)Update docs for statistics (#29926)
Update docs for statistics
2024-01-16 18:46:19 +08:00
8240c4e225 [Enhance](tools) change SSB perf tests to standardize behavior (#30002)
change SSB perf tests to standardize behavior
2024-01-16 18:46:19 +08:00
ce2e84e9a6 [fix](nereids)need validate auto partition columns in DDL (#29985) 2024-01-16 18:46:19 +08:00
779ed25972 [fix](scan) crashing caused by unlocked reading of tablet (#30005) 2024-01-16 18:46:19 +08:00
f7e12605a2 [fix](Nereids): fix NPE InferPredicates (#29978)
PredicatePropagation shouldn't add null into List.
2024-01-16 18:46:19 +08:00
fd5d986239 [fix](regression-test) fix err log limit test global impact for setting param (#29993) 2024-01-16 18:44:52 +08:00
2bf8c51baa [test](regression) Add debug level log of editlog for p0 p1 (#29992) 2024-01-16 18:42:09 +08:00
ece5f8e86c [pipelineX](fix) Fix input data distribution for distinct streaming agg (#29980) 2024-01-16 18:42:09 +08:00
c7d3b833c5 [fix](ci) Update .asf.yaml (#30004) 2024-01-16 18:42:09 +08:00
0afb4561cd [chore](ci) replace clickbench pipeline to performance pipeline (#29929) 2024-01-16 18:42:09 +08:00
66513d57f9 [feature](function) support ip function named ipv6_cidr_to_range(addr, cidr) (#29812) 2024-01-16 18:42:09 +08:00
43597afe2c [bugfix](core) writer status is read and write concurrently and will core by use after free (#29982)
Co-authored-by: yiguolei <yiguolei@gmail.com>
2024-01-16 18:42:09 +08:00
0ea5f0f5de [fix](Nereids): fix offset in PlanTranslator (#29789)
Current BE operator don't support `offset`, we need add offset into `ExchangeNode`
2024-01-16 18:41:21 +08:00
8433169e5e [test](regression-test) fix case bug, add 'order by' to make it stable (#29981) 2024-01-16 18:41:21 +08:00
d5dcdf3e07 [Improve](array) support array_enumerate_uniq and array_suffle for nereids (#29936) 2024-01-16 18:40:32 +08:00
c9cf9ab841 [pipelineX](improvement) Improve data distribution for streaming agg (#29969) 2024-01-16 18:40:32 +08:00
94bfa6e46d [fix](Nereids): fix wrong api calling (#29977) 2024-01-16 18:40:32 +08:00
b99b9efda8 [fix](Nereids) top-n with top project should hit top-n opt (#29971)
in PR #29312, we limit top-n opt only enable in simplest cases.
in this PR, we let go of some restrictions.
2024-01-16 18:40:32 +08:00
9e30a67a2a [Improve](topn opt) avoid crash when rpc returned row contains duplicated row entry (#29872)
1. Add more info to trace potential bug and avoid crash
2. use correct permutation size to do `column->permute`
2024-01-16 18:40:31 +08:00
e09118eb9c extract agg in struct info node (#29853) 2024-01-16 18:39:37 +08:00
ffc6f58e85 [pipelineX](fix) Fix incorrect partition number (#29963) 2024-01-16 18:39:37 +08:00
9282ff8e56 [asf] remove some collaborators (#29967)
There should be at most 10 collaborators
2024-01-16 18:39:06 +08:00
be893d792c [fix](jni) fix jni_reader function name get_nex_block to get_next_block (#29943) 2024-01-16 18:39:00 +08:00
f6dc6ea13b [improvement](catalog) Escape characters for columns in recovery predicate pushdown in SQL (#29854)
In the previous logic, when we restored the Column in the predicate pushdown based on the logical syntax tree for JdbcScanNode, in order to avoid query errors caused by keywords such as `key`, we added escape characters for it, but before we only Binary predicates are processed, which is imperfect. We should add escape characters to all columns that appear in the predicate to avoid errors with keywords or illegal characters.
2024-01-16 18:39:00 +08:00
05a65b9f81 [improve](join) remove join probe dependency of wait rf publish finish #29792 2024-01-16 18:39:00 +08:00
e35b26f4fc [feature](auditlog)Add runtime cpu time/peak memory metric (#29925) 2024-01-16 18:39:00 +08:00
97955da749 [enhancement](fe-memory) support label num threshold to reduce fe memory consumption (#22889)
Co-authored-by: Yongqiang YANG <98214048+dataroaring@users.noreply.github.com>
2024-01-16 18:38:54 +08:00
b7b8e59392 [opt](scanner) use buffered queue to avoid acquiring locks frequently (#29938) 2024-01-16 18:37:44 +08:00
3f20e28f3d [doc](jvm) Correct the jvm parameters of be (#29958) 2024-01-16 18:37:44 +08:00