Commit Graph

5288 Commits

Author SHA1 Message Date
06ecf8bdc5 [Bug](vec compaction) fix compaction core with sequence column (#10845)
Block reader ignores sequence column but rowset writer should write this column, will core in set_source_column row_num DCHECK. 

Sequence column works across rowsets, so compaction can not discard it and should keeps it altime. 

Co-authored-by: yixiutt <yixiu@selectdb.com>
2022-07-15 20:39:50 +08:00
23dab4cbf9 [enhancement] (SSB & TPC-H) fix test scripts, cover having PASSWORD situation (#10885) 2022-07-15 20:07:47 +08:00
c8e13caf63 [docs] Fixed description about GROUPING_ID in GROUPING SETS DESIGN. (#10805) 2022-07-15 19:59:28 +08:00
6751e5b23c [fix](alter)(tablet-scheduler) fix unexpected exception with compaction_too_slow message when add rollup for olap table (#10827) 2022-07-15 19:59:00 +08:00
dc6fbcce14 [feature-wip] (datev2) modify datev2 format in memory (#10873)
* [feature-wip] (datev2) modify datev2 format in memory

* update
2022-07-15 19:57:38 +08:00
401203da6a [feature](code-data) move cold data to object storage without losing any feature(FE) (#10693)
Co-authored-by:platonekosama@gmail.com
2022-07-15 18:00:48 +08:00
cc84cfcc0e [feature-wip](array-type) add function array_remove (#10385)
Description:

array_remove function remove all elements in array which is equal to the target.
2022-07-15 17:57:49 +08:00
1275aa905a fix sc regression test (#10879) 2022-07-15 17:53:11 +08:00
97861f517a Revert "[chore][nereids] Bump the version of antlr4 to 4.10.1 (#10780)" (#10876)
This reverts commit b4927a8f151c60357387302723fa808e523d17e3.
2022-07-15 17:05:08 +08:00
ecdcc883bc [docs]Replenish-streamload-document (#10844) 2022-07-15 15:56:54 +08:00
dd70feb35f [docs]update flink-connector-doc (#10875)
Co-authored-by: wudi <>
2022-07-15 15:53:15 +08:00
ad4751972c [feature-wip] Support in predicate for datev2 type (#10810) 2022-07-15 14:32:40 +08:00
c4426aa7e2 [doc] Improve the alter table part of the documentation (#10872) 2022-07-15 14:20:46 +08:00
cc0f334cb7 [Bug] (UT) fix parsing microseconds ut (#10842) 2022-07-15 14:15:08 +08:00
82251a6bab [refactor] some refactor of delete predicates (#10816) 2022-07-15 14:13:34 +08:00
dafd41824b [fix](unique-key-merge-on-write) Fix missing delete bitmap introduced by DSIP-018 (#10548) (#10863)
Missing delete bitmap for copy constructor of `TabletMeta(TabletMeta&)`.
2022-07-15 11:58:49 +08:00
33e9d5b2da [enhance](test): remove some System.out.println in UT. (#10859) 2022-07-15 11:16:24 +08:00
39325bd666 [Bug](tablet) Fix bug that segment files are removed as trash but tablet meta is normal (#10721) 2022-07-15 11:07:16 +08:00
b4927a8f15 [chore][nereids] Bump the version of antlr4 to 4.10.1 (#10780) 2022-07-15 10:43:05 +08:00
18348a83ad [chore][compile] fix java udf compile error (#10841) 2022-07-14 20:51:44 +08:00
505758c76b [BUG] (decimalv3) fix FE UTs (#10834) 2022-07-14 19:24:50 +08:00
364c8733fa fix light schema change coredump (#10828) 2022-07-14 15:43:15 +08:00
41f71f3ade [regression] add ssb sf1 test (#10831)
Co-authored-by: stephen <hello-stephen@qq.com>
2022-07-14 15:03:40 +08:00
ca5dbb1bcc Fix olap scan node normalize_in_and_eq_predicate infinite loop bug. (#10817) 2022-07-14 14:54:57 +08:00
799300c475 [website]1.1 release formulation (#10838) 2022-07-14 14:43:33 +08:00
190daee9f3 [doc] Fixed doc typo for materialized views 2022-07-14 14:16:45 +08:00
d245ab76cc [improvement]Use uint32 instead of size_t to reduce agg key's length (#10832) 2022-07-14 14:11:55 +08:00
575a1cb173 [Release] add download links for v1.1 (#10836) 2022-07-14 14:01:52 +08:00
13e9cb146f [feature-wip](unique-key-merge-on-write) Add option to enable unique-key-merge-on-write, DSIP-018[5/1] (#10814)
* Add option in FE

* add opt in be

* some fix

* update

* fix code style

* fix typo

* fix typo

* update

* code format
2022-07-14 12:10:58 +08:00
d1573e1a4a [improvement]Use phmap for aggregation with serialized key (#10821) 2022-07-14 11:26:09 +08:00
e361eb385e [vectorized][udf] improvement java-udaf with group by clause (#10296)
save for file about udaf
add bool _destory_deserialize
update some code according reviewer
change destroy all data at once
2022-07-14 11:23:42 +08:00
3d52bff8d1 [improvement]output query_id when be core dumped. (#10822) 2022-07-14 10:55:28 +08:00
3b46242483 [feature-wip] Optimize Decimal type (#10794)
* [feature-wip](decimalv3) support decimalv3

* [feature-wip] Optimize Decimal type

Co-authored-by: liaoxin <liaoxinbit@126.com>
2022-07-14 10:50:50 +08:00
bb0d023abd [docs] Change the incubator fields before and after Doris' Graduation (#10482)
Change the incubator fields before and after Doris' Graduation
2022-07-14 10:48:20 +08:00
077ec4b114 [bug](multi-catalog) empty hadoop configuration when reading iceberg table (#10793) 2022-07-14 10:18:59 +08:00
4f4ce4674a [fix](array) regression test fix for array (#10815) 2022-07-13 21:21:34 +08:00
e78cca1009 (Refactor)[Nereids] Combine operator and plan (#10786)
in #9755, we split plan into plan & operator, but in subsequent development, we found the rule became complex and counter intuition: 
1. we must create an operator instance, then wrap a plan by the operator type.
2. relational algebra(operator) not contains children 

e.g.
```java
logicalProject().then(project -> {
    List<NamedExpression> boundSlots =
        bind(project.operator.getProjects(), project.children(), project);
    LogicalProject op = new LogicalProject(flatBoundStar(boundSlots));
    // wrap a plan
    return new LogicalUnaryPlan(op, project.child());
})
```

after combine operator and plan, the code become to:
```java
logicalProject().then(project -> {
    List<NamedExpression> boundSlots =
        bind(project.getProjects(), project.children(), project);
    return new LogicalProject(flatBoundStar(boundSlots), project.child());
})
```

Originally, we thought it would be convenient for `Memo.copyIn()` after split plan & operator, because Memo don't known how to re-new the plan(assembling child plan in the children groups) by the plan type. So plan must provide the `withChildren()` abstract method to assembling children. The less plan type, the lower code cost we have(logical/physical with leaf/unary/binary plan, about 6 plans, no concrete plan e.g. LogicalAggregatePlan). 

But the convenient make negative effect that difficult to understand, and people must known the concept then can develop some new rules, and rule become ugly. So we combine the plan & operator, make the rule as simple as possible, the negative effect is we must overwrite some withXxx for all concrete plan, e.g. LogicalAggregate, PhysicalHashJoin.
2022-07-13 19:05:15 +08:00
56b55563c6 [feature-wip](unique-key-merge-on-write) add bloom filter index for primary key, DSIP-018[1.2] (#10706) 2022-07-13 18:58:45 +08:00
ad7702f06e add flink local debug log dep doc (#10806)
add flink local debug log dep doc
2022-07-13 17:43:42 +08:00
def59a686e [improvement]output fetal log to stderr (#10789) 2022-07-13 16:34:37 +08:00
bd982ac815 [Bug] Fix array functions arguments mismatch (#10549)
Currently, we convert array<Int> to array<BigInt>

For example, the input array_sum([1, 2, 3]) can match function array_sum(Array<Int>) as well as array_sum(Array<BigInt>).

But when a function has more than one argument, the function may be match incorrectly.

For example, the input array_contains([1, 2, 3], 2147483648) will match the function array_contains(Array<BigInt>, BigInt), but the correct match should be array_contains(Array<Int>, Int)

The correct match should be:
array_contains([1, 2, 3], 1) match array_contains(Array<Int>, Int)
array_contains([1, 2, 3], 2147483648) match array_contains(Array<Int>, Int)
array_contains([2147483648, 2147483649, 2147483650], 2147483648) match array_contains(Array<BigInt>, BigInt)

now is:
array_contains([1, 2, 3], 1) match array_contains(Array<Int>, Int)
array_contains([1, 2, 3], 2147483648) match array_contains(Array<BigInt>, BigInt)
array_contains([2147483648, 2147483649, 2147483650], 2147483648) match array_contains(Array<BigInt>, BigInt)

And this will cause some trouble.

Assume that there are two functions being defined:
Int array_functions(Array<Int>, Int)
BigInt array_functions(Array<BigInt>, BigInt)

And array_functions([1,2,3], 2147483648) will match BigInt array_functions(Array<BigInt>, BigInt), but the result type should not be BigInt, but should be Int.
2022-07-13 14:54:49 +08:00
d03b5c29a8 [bugfix] fix bug of ComparisonPredicate for nullable column (#10798) 2022-07-13 12:26:05 +08:00
4719d4705f [regression] update test framework and fix cases (#10686)
and regression test exclude suite test_create_table_with_bloom_filter temporarily.

Co-authored-by: stephen <hello-stephen@qq.com>
2022-07-13 10:16:16 +08:00
7906866826 Fix show table status docs. (#10782)
Co-authored-by: smallhibiscus <844981280>
2022-07-13 08:41:49 +08:00
f9f711cd16 FIX: fix datetimev2 decimal error. (#10736) 2022-07-13 08:32:26 +08:00
a9a08d3d0b [doc]Add common errors to broker load import documentation (#10773)
* Add common errors to broker load import documentation

* Add common errors to broker load import documentation
2022-07-13 08:31:17 +08:00
Pxl
4190f7354c [Bug][Memtable] fix core dump on int128 because not aligned by 16 byte (#10775)
* fix core dump on int128 because not aligned by 16 byte

* update
2022-07-13 08:30:58 +08:00
Pxl
d6210edcda [bugfix]set IsNullPredicate to ALWAYS_NOT_NULLABLE (#10785) 2022-07-13 08:28:00 +08:00
d278f400d4 [enhancement](show data skew) Support show avg_row_count for data skew of one table (#10790) 2022-07-13 08:27:20 +08:00
6063c0c9c8 [enhancement](singal) output git commit id when the program coredump (#10788)
* [enhancement](singal) output git commit id when the program coredump

* modift output info
2022-07-13 08:24:58 +08:00