Commit Graph

10591 Commits

Author SHA1 Message Date
f788acaa6e [fix](regression-test) fix insert overwrite case same db name issue (#19839) 2023-05-19 08:43:46 +08:00
66f53a723e [typo](docs) Add a previously missed configuration item and other note (#19814)
fix enable_query_memory_overcommit config doc
2023-05-19 08:38:10 +08:00
f32deb18e9 [Update](build) change clucene from thirdparty to git module (#19352) 2023-05-19 08:25:51 +08:00
3d6a13605d [improvement](stacktrace) do not captute stack trace for txn error codes (#19817) 2023-05-18 23:58:56 +08:00
adc5522c9b [bug](MTMV) Fix the wrong interpretation for NEVER REFRESH (#19800) 2023-05-18 23:56:56 +08:00
dfc4432e83 [improvement](jdbc catalog) Add adaptation to Oracle special character / table names (#19809) 2023-05-18 22:58:33 +08:00
f2b2a568de [fix](jdbc catalog)fixed oceanbase catalog row limit bug (#19796) 2023-05-18 22:05:51 +08:00
40ab4ce305 fix select resource groups bug (#19808) 2023-05-18 21:54:31 +08:00
481e9aebdb [Refactor](spark load) remove parquet scanner (#19251) 2023-05-18 19:19:13 +08:00
ef0657c072 [Bug](pipeline) RegressionTest failed release resouce cause DCHECK failed (#19783)
RegressionTest failed release resouce cause DCHECK failed
2023-05-18 18:57:25 +08:00
f68d3a660e [improvement](opentelemetry) upgrade opentelemetry jar to v1.26.0 and opentelemetry-cpp to v1.8.3 (#19733)
why upgrade? anything wrong?

Try to fix the problem about opentelemetry::v1::ext::http::client::curl::HttpOperation::Send(), I have updated the pr info.
2023-05-18 18:46:20 +08:00
ed85a10a70 [Fix](multi-catalog) Fix sync hms event failed. (#19555)
A similar situation with #19344 , because sometimes hms meta info is newer than hms events, if we try to invoke org.apache.doris.datasource.hive.PooledHiveMetaStoreClient#getTable and this table is not exists, some error will throws and this event can not be handled.
2023-05-18 18:34:18 +08:00
e242d7dfcc [refactor-WIP](TaskWorkerPool) add DropTableTaskPool for DROP_TABLE task (#19793) 2023-05-18 18:25:13 +08:00
e67872d391 [fix](Nereids) fallback not work when cannot parse after forward (#19790) 2023-05-18 18:24:59 +08:00
07bbf741fb [enhence](memory) gc inverted index cache when there is not enough memory (#19622)
Support to gc inverted index cache when there is not enough memory.
previous problem: The inverted index cache (InvertedIndexSearcherCache and InvertedIndexQueryCache) may use 20% memory which can't be released.
2023-05-18 16:41:51 +08:00
fd4fa5c64e [Optimize](row store) optimize serialization and deserialization (#19691)
1. Get DataTypeSerde in advance to avoid get temporary DataTypeSerde iterate each column
2. Iterate the original row once is enoungh for deserializing by introducing a map for record the index of each column's unique id
2023-05-18 16:22:38 +08:00
294599ee45 [feature](jsonb) rename JSONB type name and function name to JSON (#19774)
To be more compatible with MySQL, rename JSONB type name and function name to JSON.

The old JSONB type name and jsonb_xx function can still be used for backward compatibility.

There is a function jsonb_extract remained since json_extract is used by json string function and more work need to change it. It will be changed further.
2023-05-18 16:16:52 +08:00
Pxl
a2c9ed7be8 [Chore](build) fix some undefined behavior about incomplete type vector #19753 2023-05-18 15:13:45 +08:00
068a32bc49 [Improvement](memory) faststring use Allocator #19762
After the outer catch exception, faststring resize reserve build may throw a memory alloc failure exception from the Allocator.

Currently page body compress will catch memory alloc failure exception
2023-05-18 15:00:49 +08:00
7c8b7878cd [fix](memory) Print all query/load memory before memory GC when memory_debug=true (#19720) 2023-05-18 14:55:47 +08:00
160d2be0d8 [minimal](Nereids) add more comments for the rewriter (#19788)
Only add some comments to the rewriter. Because it is fewer comments before and it's hard to understand for the newbie.
2023-05-18 14:47:25 +08:00
303bee6fa3 [Fix](single replica load) add inverted index copy for single replica load (#19663)
* [Fix](single replica load) add inverted index copy for single replica load
2023-05-18 14:13:41 +08:00
851886cc18 [minor](datev2) remove datev2 because datev2 is used by default (#19777) 2023-05-18 13:36:11 +08:00
e45bc160c9 [fix](mtmv) fix bug that should not write edit log when replaying alter mv (#19781) 2023-05-18 13:34:05 +08:00
943e5fb7e5 [improvement](MOW) use seperated cache for mow pk cache (#19686)
In mow, primary key cache have a big impact on load performance, so we add a new cache type to seperate
it from page cache to make it more flexible in some cases
2023-05-18 13:27:09 +08:00
50370dead9 [fix](load) fix unified load converted failed when forwarding to master (#19779) 2023-05-18 12:28:32 +08:00
f43e8cc98f [regressiontest](unionall) Regression_test_similar_query_boolean (#19553)
* regression_test_similar_query

* add the ORDER BY

* update ORDER BY to comfirm correctness

---------

Co-authored-by: ZI-MA <chime316@qq.com>
2023-05-18 12:21:32 +08:00
9a99d6cd7e [improvement-#19739]statement error in data-model (#19740)
Co-authored-by: fuchanghai <‘2875334588@qq.com’>
2023-05-18 12:07:01 +08:00
62458ed0f4 [enhancement](compaction) not core when init failed (#19754) 2023-05-18 12:06:22 +08:00
6a5b590873 [refactor-WIP](TaskWorkerPool) add CreateTableTaskPool class for CREATE_TABLE task (#19734) 2023-05-18 11:43:09 +08:00
18c1081659 [fix](nereids) fix some nereids bugs (#19711)
1. add json_unquote and json_extract functions
2. remove mv releated code in visitPhysicalOlapScan
3. forbid bitmap and hll type for topn node's sort exprs
4. HashDistributionInfo of olap scan node should use the slots from output not the full schema
5. SelectMaterializedIndexWithoutAggregate should use the filter node's output together with the predicate to get the correct mv
6. forbid SimplifyArithmeticRule for decimal type
7. make DecimalLiteral's type and value consistent with each other if the value is decimalv2
8. json_array need support empty argument
2023-05-18 11:33:56 +08:00
f412aec187 [improvement](load)disable shrink memory by default (#19714)
disable shrink memory by default, it becomes very slow when importing large amounts of data
you can turn on If you think it's necessary
2023-05-18 11:25:39 +08:00
a3f06e5fbd [git](Nereids): ignore apache parquet. (#19765) 2023-05-18 10:54:46 +08:00
fe42e52851 [pipeline](CTE) Support multi stream data sink in pipeline (#19519) 2023-05-18 10:34:37 +08:00
88ca4f3e6b [feature](like) make like regexp used as a sql function (#19755) 2023-05-18 10:03:12 +08:00
5707ecd0e5 [typo](docs) modify the dynamic table document (#19750) 2023-05-18 09:24:17 +08:00
51a12f4f11 [Doc](map-type) add map sql doc (#19612) 2023-05-18 09:09:41 +08:00
7f9cdc41f5 [typo](docs)add new describe for grant (#19656) 2023-05-18 09:00:39 +08:00
d5d47703fe [fix](memory) remove auto option in memory config and optimize memtracker logs #19706
fix mem_limit default value
memory_gc_sleep_time_s to memory_gc_sleep_time_ms
LoadChannelMgr::_handle_mem_exceed_limit process_mem_limit to process soft mem limit
fix query mem tracker print
2023-05-18 08:54:03 +08:00
cfab124ddd [Chore](inverted index) change Status::EndOfFile to just logging info, remove useless print (#19721) 2023-05-18 08:44:18 +08:00
6a6be52bc9 [enhancement](merge-on-write) Avoiding unnecessary primary key index traversal (#19746) 2023-05-18 08:41:49 +08:00
f04f181249 [Bug](pipeline) RegressionTest failed release resouce cause DCHECK failed #19772 2023-05-18 08:41:32 +08:00
fcd85cfc14 [typo](doc)fix typo in BROKER-LOAD.md #19737 2023-05-18 08:38:12 +08:00
ba1bccc518 [typo](docs)Fix typo in monitor-alert (#19743) 2023-05-18 08:37:55 +08:00
b32ee4facc [typo](doc)fix audit plugin and spark load kerberos for yarn #19749 2023-05-18 08:37:41 +08:00
40909f49cb [typo](docs) spark load example revise and some other revise (#19756)
Co-authored-by: zhuwei <zhuwei8421@gmail.com>
2023-05-18 08:37:23 +08:00
098dac20c2 [log](Nereids): add more debug info when check logicalproperties. (#19763) 2023-05-18 08:37:10 +08:00
35ad081784 [typo](doc)Example of adding stream load import to current time (#19760) 2023-05-18 08:36:39 +08:00
5fa956b0d6 [Bug](pipeline) RegressionTest failed release resouce cause DCHECK failed #19773 2023-05-18 08:35:57 +08:00
4566281cc3 [fix](sink) disable lazy-open partition by default (#19769)
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com>
2023-05-18 07:28:04 +08:00