Commit Graph

8276 Commits

Author SHA1 Message Date
16866e3c55 [doc] Add compression properties to the create table document (#10829)
Co-authored-by: smallhibiscus <844981280>
2022-07-18 15:25:53 +08:00
108e6207b1 [doc] fix sequence_column_manual.md (#10907)
Co-authored-by: yixiutt <yixiu@selectdb.com>
2022-07-18 15:24:15 +08:00
cc7c31b080 [Bug](be) fix be coredump when receive singal(#10903). (#10953) 2022-07-18 15:23:51 +08:00
dc01ea7ad9 [fix](nereids) Fix the substring compilation error caused by merge (#10965)
Compilation error after merging due to Literal refactoring.
Compilation failure:
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/Substring.java:[40,38] org.apache.doris.nereids.trees.expressions.Literal is abstract; cannot be instantiated
2022-07-18 15:20:25 +08:00
238395e282 [Bug] fix decimal arithmetic calculations (#10963) 2022-07-18 14:35:07 +08:00
8c544b6e13 fix show storage policy null pointer and redundant log (#10906)
* fix show storage policy null pointer and redundant log
2022-07-18 14:08:54 +08:00
77ef19dbcd [BugFix](Array)Fix using Array aggregate function caused be coredump (#10649) 2022-07-18 13:47:17 +08:00
0b177669d9 [feature](nereids) support substring (#10847)
support substring, for example:
select substr(a, 2), substring(b ,3 ,4) from test1;
2022-07-18 12:38:56 +08:00
bf95440c13 [Refactor](nereids)Refactor Literal to an inheritance hierarchy (#10771)
Use inheritance hierarchy instead of combination to make the framework more clear
2022-07-18 12:01:30 +08:00
d0dc93654e [doc] update proxy_connect_timeout and proxy_timeout from 30s to 300s (#10753) 2022-07-18 10:55:41 +08:00
2d5aca18fb [feature-wip](array) add the array_sort function (#10598)
Co-authored-by: hucheng01 <hucheng01@baidu.com>
2022-07-18 10:52:42 +08:00
e3c19ded44 [enhancement](thirdparty) Support building thirdparty on macOS (#10677) 2022-07-18 10:50:30 +08:00
5c88a74792 [Enhancement] generate runtime filter only for tuples with conjunct (#8745)
Remove useless runtime filter in some primary-foreign key join scenario in TPCH case.
2022-07-18 09:37:45 +08:00
2b6cdcf599 [improvement] add an option to let regression stop when a failure happen (#10939)
For community pipeline, it is a waste of resource to run tests with errors.
2022-07-18 08:53:17 +08:00
ec5996f1f8 [improvement]do not acquire mutex in metric hook (#10941) 2022-07-18 08:52:24 +08:00
2e94674cb5 [fix](alter) fix bug that fe crash because npe on rollupBatchTask (#10943)
Co-authored-by: caiconghui1 <caiconghui1@jd.com>
2022-07-18 08:47:25 +08:00
523d395527 [refactor] Remove alpha rowset meta (#10933)
* remove alpha_rowset_meta
* remove alpha rowset related codes in compaction
* remove alpha rowset related codes in RowsetMeta
* fix be ut because some ut use alpha rowsetmeta
2022-07-18 08:45:46 +08:00
50cb26d599 revert sc case (#10945)
schema change cases leads to failure of creating partition.
2022-07-17 21:45:23 +08:00
6b1408ce41 [fix](planner) fix create view when using union (#10849) 2022-07-17 20:54:40 +08:00
c3e1b73d15 revert cast_to_string (#10940)
It leads schema change p0 tests to failure.
2022-07-17 18:34:39 +08:00
09d19e3f0f [feature-wip](array-type) explode support more sub types (#10673)
1. explode support more sub types;
2. explode support nullable elements;

Co-authored-by: cambyzju <zhuxiaoli01@baidu.com>
2022-07-17 18:08:30 +08:00
5793cb11d0 [feature-wip] (array-type) function concat_ws support array (#10749)
Issue #10052
function concat_ws support array
2022-07-17 17:50:39 +08:00
3bc6655069 [refactor] remove BlockManager (#10913)
* remove BlockManager
* remove deprecated field in tablet meta
2022-07-17 14:10:06 +08:00
464830437a [doc] add cloudcanal docs (#10813) 2022-07-17 11:50:40 +08:00
eec142ae90 [Enhancement] Use shared file reader when read a segment (#10896)
* readers under a segment use a shared FileReader

* no need to cache fd in LocalFileReader
2022-07-17 07:54:58 +08:00
c45a98d4c0 [Bug] Fix invalid nullmap (#10925) 2022-07-17 07:53:11 +08:00
02460ab4a9 change date to int, good for performance (#10889) 2022-07-16 22:35:43 +08:00
0381cdc989 [Bug] fix core for min/max runtime filter (#10899) 2022-07-16 22:22:51 +08:00
f0babfdcf8 print image's version when it is higher than FEConstants.meta_version (#10908) 2022-07-16 19:26:47 +08:00
f78db1d773 release memory allocated in agg function in vec stream load (#10739)
release memory allocated in agg function in vec stream load

When a load is cancelled, memory allocated by agg functions should
be freeed.
2022-07-16 15:32:53 +08:00
75ca21dafa [Bug] handle null map right in vectorized load (#10883) 2022-07-16 14:18:38 +08:00
00c9455f16 [fix](array-type) fix arrow column to doris array column (#10855)
* support merge array column, while convert from arrow column to doris array column

* fix typo

Co-authored-by: cambyzju <zhuxiaoli01@baidu.com>
2022-07-16 11:49:42 +08:00
b5b6e1e12c [regression] add ssb flat sf 1 test (#10894) 2022-07-16 11:40:45 +08:00
2d5ea56a06 [regression] rm unwanted sleep (#10893) 2022-07-16 06:00:42 +08:00
6a7487a302 [improvement]Remove the page button on the System page (#10900) 2022-07-16 06:00:08 +08:00
ba1c527a23 [improvement](arrow) Avoid parse timezone for each datetime value (#10869)
* [improvement](arrow) Avoid parse timezone for each datetime value

Convert arrow batch to doris block is too slow when there are datetime values.
Because we call `TimezoneUtils::find_cctz_time_zone` for each values.

After modify, the tpch-100 q1 with external table cost from 40s -> 9s

Co-authored-by: morningman <morningman@apache.org>
2022-07-15 21:19:36 +08:00
7be2ef79ed array column support read by rowids (#10886)
Co-authored-by: cambyzju <zhuxiaoli01@baidu.com>
2022-07-15 21:19:02 +08:00
b1711e94b7 (unique-key-merge-on-write) Add tablet lookup interface, DSIP-18[3/4] (#10820)
Add lookup_row_key interface for tablet and segment.
2022-07-15 20:49:28 +08:00
06ecf8bdc5 [Bug](vec compaction) fix compaction core with sequence column (#10845)
Block reader ignores sequence column but rowset writer should write this column, will core in set_source_column row_num DCHECK. 

Sequence column works across rowsets, so compaction can not discard it and should keeps it altime. 

Co-authored-by: yixiutt <yixiu@selectdb.com>
2022-07-15 20:39:50 +08:00
23dab4cbf9 [enhancement] (SSB & TPC-H) fix test scripts, cover having PASSWORD situation (#10885) 2022-07-15 20:07:47 +08:00
c8e13caf63 [docs] Fixed description about GROUPING_ID in GROUPING SETS DESIGN. (#10805) 2022-07-15 19:59:28 +08:00
6751e5b23c [fix](alter)(tablet-scheduler) fix unexpected exception with compaction_too_slow message when add rollup for olap table (#10827) 2022-07-15 19:59:00 +08:00
dc6fbcce14 [feature-wip] (datev2) modify datev2 format in memory (#10873)
* [feature-wip] (datev2) modify datev2 format in memory

* update
2022-07-15 19:57:38 +08:00
401203da6a [feature](code-data) move cold data to object storage without losing any feature(FE) (#10693)
Co-authored-by:platonekosama@gmail.com
2022-07-15 18:00:48 +08:00
cc84cfcc0e [feature-wip](array-type) add function array_remove (#10385)
Description:

array_remove function remove all elements in array which is equal to the target.
2022-07-15 17:57:49 +08:00
1275aa905a fix sc regression test (#10879) 2022-07-15 17:53:11 +08:00
97861f517a Revert "[chore][nereids] Bump the version of antlr4 to 4.10.1 (#10780)" (#10876)
This reverts commit b4927a8f151c60357387302723fa808e523d17e3.
2022-07-15 17:05:08 +08:00
ecdcc883bc [docs]Replenish-streamload-document (#10844) 2022-07-15 15:56:54 +08:00
dd70feb35f [docs]update flink-connector-doc (#10875)
Co-authored-by: wudi <>
2022-07-15 15:53:15 +08:00
ad4751972c [feature-wip] Support in predicate for datev2 type (#10810) 2022-07-15 14:32:40 +08:00