Commit Graph

10605 Commits

Author SHA1 Message Date
a0c3ebeeab [Chore](build) fix typo in build.sh (#19846) 2023-05-19 11:49:20 +08:00
609b20bd02 [Feature](planner) use partial update in update from & delete from (#19262) 2023-05-19 09:46:29 +08:00
cbe12cfb69 [chore](build) Support specifying output path #19669
now we could

./build.sh --be --fe --output PATH_TO_BINARY_YOU_LIKE
did not modify the default value
2023-05-19 09:22:36 +08:00
84bad03ccb [feature](nereids) set proper min/max value for column stats when minExpr/maxExpr is not avialable #19673 2023-05-19 09:02:40 +08:00
0dd361dbf7 [fix](tracing) fix the issue that a trace may track multiple queries (#19804) 2023-05-19 08:58:53 +08:00
3e0b661267 [fix](test) Comment unstable stats test #19729 2023-05-19 08:55:28 +08:00
6f6d744a2a [fix](nereids) avoid 0 row count in stats derive #19640
row count of join estimation is at least 1 to make less error propagation.
2023-05-19 08:54:24 +08:00
14620a6766 [minor](log) add details for unqueryable replicas (#19792)
Add a new FE config: show_details_for_unaccessible_tablet.
Default is false, when set to true, if a query is unable to select a healthy replica,
the detailed information of all the replicas of the tablet including the specific reason why they are unqueryable,
will be printed out.
2023-05-19 08:53:57 +08:00
dc8a992bba [improve](nereids) check be status when column stats is unknown #19742
when forbid_unknown_col_stats is open and some column stats is unknown,
we will check the be status by StatisticsUtil.statsTblAvailable(), and report error according to be status.
2023-05-19 08:53:34 +08:00
1e8eb1c756 [fix](profile) Fix pipeline load channel profile #19828 2023-05-19 08:51:02 +08:00
3d795de2d5 [chore](build) avoid generating generated code every time (#19813)
When calling generated-source.sh in build.sh, not to remove the gensrc/build dir.
2023-05-19 08:47:36 +08:00
3e010bbee7 [improvement](profile) add profile counter 'BytesSent' for VDataBufferSender (#19826) 2023-05-19 08:46:50 +08:00
b3ce4593b1 [deps](libhdfs) update hadoop libhdfs to 3.3.4.1 for doris (#19832) 2023-05-19 08:44:32 +08:00
1d01136b1b [Fix](parquet-reader) Fix partition field conjuncts not work. (#19837)
Fix partition field conjuncts not work.
Add predicate_partition_columns in _slot_id_to_filter_conjuncts(single slot conjuncts) to _filter_conjuncts, others should had been added from not_single_slot_filter_conjuncts.
2023-05-19 08:44:02 +08:00
f788acaa6e [fix](regression-test) fix insert overwrite case same db name issue (#19839) 2023-05-19 08:43:46 +08:00
66f53a723e [typo](docs) Add a previously missed configuration item and other note (#19814)
fix enable_query_memory_overcommit config doc
2023-05-19 08:38:10 +08:00
f32deb18e9 [Update](build) change clucene from thirdparty to git module (#19352) 2023-05-19 08:25:51 +08:00
3d6a13605d [improvement](stacktrace) do not captute stack trace for txn error codes (#19817) 2023-05-18 23:58:56 +08:00
adc5522c9b [bug](MTMV) Fix the wrong interpretation for NEVER REFRESH (#19800) 2023-05-18 23:56:56 +08:00
dfc4432e83 [improvement](jdbc catalog) Add adaptation to Oracle special character / table names (#19809) 2023-05-18 22:58:33 +08:00
f2b2a568de [fix](jdbc catalog)fixed oceanbase catalog row limit bug (#19796) 2023-05-18 22:05:51 +08:00
40ab4ce305 fix select resource groups bug (#19808) 2023-05-18 21:54:31 +08:00
481e9aebdb [Refactor](spark load) remove parquet scanner (#19251) 2023-05-18 19:19:13 +08:00
ef0657c072 [Bug](pipeline) RegressionTest failed release resouce cause DCHECK failed (#19783)
RegressionTest failed release resouce cause DCHECK failed
2023-05-18 18:57:25 +08:00
f68d3a660e [improvement](opentelemetry) upgrade opentelemetry jar to v1.26.0 and opentelemetry-cpp to v1.8.3 (#19733)
why upgrade? anything wrong?

Try to fix the problem about opentelemetry::v1::ext::http::client::curl::HttpOperation::Send(), I have updated the pr info.
2023-05-18 18:46:20 +08:00
ed85a10a70 [Fix](multi-catalog) Fix sync hms event failed. (#19555)
A similar situation with #19344 , because sometimes hms meta info is newer than hms events, if we try to invoke org.apache.doris.datasource.hive.PooledHiveMetaStoreClient#getTable and this table is not exists, some error will throws and this event can not be handled.
2023-05-18 18:34:18 +08:00
e242d7dfcc [refactor-WIP](TaskWorkerPool) add DropTableTaskPool for DROP_TABLE task (#19793) 2023-05-18 18:25:13 +08:00
e67872d391 [fix](Nereids) fallback not work when cannot parse after forward (#19790) 2023-05-18 18:24:59 +08:00
07bbf741fb [enhence](memory) gc inverted index cache when there is not enough memory (#19622)
Support to gc inverted index cache when there is not enough memory.
previous problem: The inverted index cache (InvertedIndexSearcherCache and InvertedIndexQueryCache) may use 20% memory which can't be released.
2023-05-18 16:41:51 +08:00
fd4fa5c64e [Optimize](row store) optimize serialization and deserialization (#19691)
1. Get DataTypeSerde in advance to avoid get temporary DataTypeSerde iterate each column
2. Iterate the original row once is enoungh for deserializing by introducing a map for record the index of each column's unique id
2023-05-18 16:22:38 +08:00
294599ee45 [feature](jsonb) rename JSONB type name and function name to JSON (#19774)
To be more compatible with MySQL, rename JSONB type name and function name to JSON.

The old JSONB type name and jsonb_xx function can still be used for backward compatibility.

There is a function jsonb_extract remained since json_extract is used by json string function and more work need to change it. It will be changed further.
2023-05-18 16:16:52 +08:00
Pxl
a2c9ed7be8 [Chore](build) fix some undefined behavior about incomplete type vector #19753 2023-05-18 15:13:45 +08:00
068a32bc49 [Improvement](memory) faststring use Allocator #19762
After the outer catch exception, faststring resize reserve build may throw a memory alloc failure exception from the Allocator.

Currently page body compress will catch memory alloc failure exception
2023-05-18 15:00:49 +08:00
7c8b7878cd [fix](memory) Print all query/load memory before memory GC when memory_debug=true (#19720) 2023-05-18 14:55:47 +08:00
160d2be0d8 [minimal](Nereids) add more comments for the rewriter (#19788)
Only add some comments to the rewriter. Because it is fewer comments before and it's hard to understand for the newbie.
2023-05-18 14:47:25 +08:00
303bee6fa3 [Fix](single replica load) add inverted index copy for single replica load (#19663)
* [Fix](single replica load) add inverted index copy for single replica load
2023-05-18 14:13:41 +08:00
851886cc18 [minor](datev2) remove datev2 because datev2 is used by default (#19777) 2023-05-18 13:36:11 +08:00
e45bc160c9 [fix](mtmv) fix bug that should not write edit log when replaying alter mv (#19781) 2023-05-18 13:34:05 +08:00
943e5fb7e5 [improvement](MOW) use seperated cache for mow pk cache (#19686)
In mow, primary key cache have a big impact on load performance, so we add a new cache type to seperate
it from page cache to make it more flexible in some cases
2023-05-18 13:27:09 +08:00
50370dead9 [fix](load) fix unified load converted failed when forwarding to master (#19779) 2023-05-18 12:28:32 +08:00
f43e8cc98f [regressiontest](unionall) Regression_test_similar_query_boolean (#19553)
* regression_test_similar_query

* add the ORDER BY

* update ORDER BY to comfirm correctness

---------

Co-authored-by: ZI-MA <chime316@qq.com>
2023-05-18 12:21:32 +08:00
9a99d6cd7e [improvement-#19739]statement error in data-model (#19740)
Co-authored-by: fuchanghai <‘2875334588@qq.com’>
2023-05-18 12:07:01 +08:00
62458ed0f4 [enhancement](compaction) not core when init failed (#19754) 2023-05-18 12:06:22 +08:00
6a5b590873 [refactor-WIP](TaskWorkerPool) add CreateTableTaskPool class for CREATE_TABLE task (#19734) 2023-05-18 11:43:09 +08:00
18c1081659 [fix](nereids) fix some nereids bugs (#19711)
1. add json_unquote and json_extract functions
2. remove mv releated code in visitPhysicalOlapScan
3. forbid bitmap and hll type for topn node's sort exprs
4. HashDistributionInfo of olap scan node should use the slots from output not the full schema
5. SelectMaterializedIndexWithoutAggregate should use the filter node's output together with the predicate to get the correct mv
6. forbid SimplifyArithmeticRule for decimal type
7. make DecimalLiteral's type and value consistent with each other if the value is decimalv2
8. json_array need support empty argument
2023-05-18 11:33:56 +08:00
f412aec187 [improvement](load)disable shrink memory by default (#19714)
disable shrink memory by default, it becomes very slow when importing large amounts of data
you can turn on If you think it's necessary
2023-05-18 11:25:39 +08:00
a3f06e5fbd [git](Nereids): ignore apache parquet. (#19765) 2023-05-18 10:54:46 +08:00
fe42e52851 [pipeline](CTE) Support multi stream data sink in pipeline (#19519) 2023-05-18 10:34:37 +08:00
88ca4f3e6b [feature](like) make like regexp used as a sql function (#19755) 2023-05-18 10:03:12 +08:00
5707ecd0e5 [typo](docs) modify the dynamic table document (#19750) 2023-05-18 09:24:17 +08:00