00896d8954
[fix](agg) fix coredump of multi distinct of decimal128I ( #27014 )
...
* [fix](agg) fix coredump of multi distinct of decimal128
* fix
2023-11-15 17:37:20 +08:00
2f529c1c7b
[feature](Nereids): remove True in Join condition ( #26951 )
...
Remove `True` in Join Condition like `SELECT * FROM t1 JOIN t2 on True`;
2023-11-15 15:58:33 +08:00
5dbc3cbba4
[test](information_schema)append information_schema external_table_p0 case. ( #26846 )
2023-11-15 14:30:16 +08:00
2c6d2255c3
[fix](Nereids) nested type literal type coercion and insert values with map ( #26669 )
2023-11-14 21:13:26 -06:00
df867a1531
[fix](catalog) Fix ClickHouse DataTime64 precision parsing ( #26977 )
2023-11-15 10:23:21 +08:00
cdc5626930
[regression](partial update) Add row store cases for all existing partial update cases ( #26924 )
2023-11-15 08:20:55 +08:00
add160b768
[improvement](regression-test) add more group commit regression-test ( #26952 )
2023-11-15 00:01:13 +08:00
88d909b4dd
[test](regression) Add more alter stmt regression case ( #26988 )
2023-11-14 23:58:15 +08:00
f1169d3c58
[regression-test](TRIM_DOUBLE_QUOTES) add case for TRIM_DOUBLE_QUOTES ( #26998 )
2023-11-14 23:52:40 +08:00
9a4fd5be79
[nereids](datetime) fix wrong result type of datetime add with interval as first arg ( #26957 )
...
Incorrect result data type cause be cordump:
drop table if exists testaaa;
create table testaaa(k1 tinyint, k2 smallint, k3 int, k4 bigint, k5 decimal(9,3), k6 char(5), k10 date, k11 datetime, k7 varchar(20), k8 double max, k9 float sum) engine=olap distributed by hash(k1) buckets 5 properties("storage_type"="column","replication_num"="1") ;
insert into testaaa values(1,1,1,1,9.3, "k6", "2023-11-14", "2023-11-14", "k7", 9.99, 9.99);
select interval 10 year + k10 from testaaa;
The plan result type is DATE:
mysql [test]>explain verbose select interval 10 year + k10 from testaaa;
+-------------------------------------------------------------------------------------------------------+
| Explain String(Nereids Planner) |
+-------------------------------------------------------------------------------------------------------+
| PLAN FRAGMENT 0 |
| OUTPUT EXPRS: |
| years_add(k10, INTERVAL 10 YEAR)[#11 ] |
| PARTITION: UNPARTITIONED |
| |
| HAS_COLO_PLAN_NODE: false |
| |
| VRESULT SINK |
| MYSQL_PROTOCAL |
| |
| 64:VEXCHANGE |
| offset: 0 |
| tuple ids: 1N |
| |
| PLAN FRAGMENT 1 |
| |
| PARTITION: HASH_PARTITIONED: k1[#0 ] |
| |
| HAS_COLO_PLAN_NODE: false |
| |
| STREAM DATA SINK |
| EXCHANGE ID: 64 |
| UNPARTITIONED |
| |
| 58:VOlapScanNode |
| TABLE: default_cluster:test.testaaa(testaaa), PREAGGREGATION: OFF. Reason: No aggregate on scan. |
| partitions=1/1 (testaaa), tablets=5/5, tabletList=945025,945027,945029 ... |
| cardinality=1, avgRowSize=9885.0, numNodes=1 |
| pushAggOp=NONE |
| projections: years_add(k10[#6 ], INTERVAL 10 YEAR) |
| project output tuple id: 1 |
| tuple ids: 0 |
| |
| Tuples: |
| TupleDescriptor{id=0, tbl=testaaa, byteSize=8} |
| SlotDescriptor{id=6, col=k10, colUniqueId=6, type=DATEV2, nullable=true, isAutoIncrement=false} |
| |
| TupleDescriptor{id=1, tbl=testaaa, byteSize=32} |
| SlotDescriptor{id=11, col=null, colUniqueId=null, type=DATE, nullable=true, isAutoIncrement=false} |
+-------------------------------------------------------------------------------------------------------+
39 rows in set (1 min 31.50 sec)
coredump stack:
F1109 20:11:37.677680 323805 assert_cast.h:61] Bad cast from type:doris::vectorized::ColumnVector to doris::vectorized::ColumnVector
*** Check failure stack trace: ***
F1109 20:11:37.680608 323800 assert_cast.h:61] Bad cast from type:doris::vectorized::ColumnVector to doris::vectorized::ColumnVector
*** Check failure stack trace: ***
F1109 20:11:37.680608 323800 assert_cast.h:61] Bad cast from type:doris::vectorized::ColumnVector to doris::vectorized::ColumnVectorF1109 20:11:37.681102 323808 assert_cast.h:61] Bad cast from type:doris::vectorized::ColumnVector to doris::vectorized::ColumnVector
*** Check failure stack trace: ***
@ 0x56489d591d3d google::LogMessage::Fail()
@ 0x56489d591d3d google::LogMessage::Fail()
@ 0x56489d591d3d google::LogMessage::Fail()
@ 0x56489d594279 google::LogMessage::SendToLog()
@ 0x56489d594279 google::LogMessage::SendToLog()
@ 0x56489d594279 google::LogMessage::SendToLog()
@ 0x56489d5918a6 google::LogMessage::Flush()
@ 0x56489d5918a6 google::LogMessage::Flush()
@ 0x56489d5918a6 google::LogMessage::Flush()
@ 0x56489d5948e9 google::LogMessageFatal::~LogMessageFatal()
@ 0x56489d5948e9 google::LogMessageFatal::~LogMessageFatal()
@ 0x56489d5948e9 google::LogMessageFatal::~LogMessageFatal()
@ 0x56487a2a8a0c assert_cast<>()
@ 0x56487a2a8a0c assert_cast<>()
@ 0x56487a2a8a0c assert_cast<>()
@ 0x5648893d8312 doris::vectorized::ColumnVector<>::insert_range_from()
@ 0x5648893d8312 doris::vectorized::ColumnVector<>::insert_range_from()
@ 0x5648893d8312 doris::vectorized::ColumnVector<>::insert_range_from()
@ 0x56488924a670 doris::vectorized::ColumnNullable::insert_range_from()
@ 0x56488924a670 doris::vectorized::ColumnNullable::insert_range_from()
@ 0x56488924a670 doris::vectorized::ColumnNullable::insert_range_from()
@ 0x56487a454475 doris::ExecNode::do_projections()
@ 0x56487a454475 doris::ExecNode::do_projections()
@ 0x56487a454475 doris::ExecNode::do_projections()
@ 0x56487a454b89 doris::ExecNode::get_next_after_projects()
@ 0x56487a454b89 doris::ExecNode::get_next_after_projects()
*** Query id: a467995b35334741-b625042f56495aaf ***
*** tablet id: 0 ***
*** Aborted at 1699531898 (unix time) try "date -d @1699531898" if you are using GNU date ***
*** Current BE git commitID: 0d83327a7c ***
*** SIGABRT unknown detail explain (@0x190d64) received by PID 1641828 (TID 1642168 OR 0x7f6ff96c0700) from PID 1641828; stack trace: ***
@ 0x556ca2a3ab8f std::_Function_handler<>::_M_invoke()
@ 0x556c9f322787 std::function<>::operator()()
@ 0x556ca29da0b0 doris::Thread::supervise_thread()
@ 0x556c9f322787 std::function<>::operator()()
@ 0x7f71b9c38609 start_thread
@ 0x556ca29da0b0 doris::Thread::supervise_thread()
@ 0x7f71b9c38609 start_thread
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/common/signal_handler.h:417
1# 0x00007F71B9E09090 in /lib/x86_64-linux-gnu/libc.so.6
2# raise at ../sysdeps/unix/sysv/linux/raise.c:51
3# abort at /build/glibc-SzIz7B/glibc-2.31/stdlib/abort.c:81
4# 0x0000556CC51F3729 in /mnt/hdd01/ci/branch20-deploy/be/lib/doris_be
5# 0x0000556CC51E8D3D in /mnt/hdd01/ci/branch20-deploy/be/lib/doris_be
6# google::LogMessage::SendToLog() in /mnt/hdd01/ci/branch20-deploy/be/lib/doris_be
7# google::LogMessage::Flush() in /mnt/hdd01/ci/branch20-deploy/be/lib/doris_be
8# google::LogMessageFatal::~LogMessageFatal() in /mnt/hdd01/ci/branch20-deploy/be/lib/doris_be
9# doris::vectorized::ColumnVector const& assert_cast const&, doris::vectorized::IColumn const&>(doris::vectorized::IColumn const&) in /mnt/hdd01/ci/branch20-deploy/be/lib/doris_be
10# doris::vectorized::ColumnVector::insert_range_from(doris::vectorized::IColumn const&, unsigned long, unsigned long) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/vec/columns/column_vector.cpp:354
11# doris::vectorized::ColumnNullable::insert_range_from(doris::vectorized::IColumn const&, unsigned long, unsigned long) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/vec/columns/column_nullable.cpp:289
12# doris::ExecNode::do_projections(doris::vectorized::Block*, doris::vectorized::Block*) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/exec/exec_node.cpp:573
13# doris::ExecNode::get_next_after_projects(doris::RuntimeState*, doris::vectorized::Block*, bool*, std::function const&, bool) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/exec/exec_node.cpp:592
14# doris::pipeline::SourceOperator::get_block(doris::RuntimeState*, doris::vectorized::Block*, doris::pipeline::SourceState&) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/pipeline/exec/operator.h:413
15# doris::pipeline::PipelineTask::execute(bool*) at /home/zcp/repo_center/doris_branch-2.0/doris/be/src/pipeline/pipeline_task.cpp:259
16# doris::pipeline::TaskScheduler::_do_work(unsigned
2023-11-14 20:28:41 +08:00
3585c7e216
[test](parquet)append parquet reader byte_array_decimal and rle_bool case ( #26751 )
2023-11-14 15:05:10 +08:00
a16517a061
[test](tvf) append tvf read hive_text file regression case. ( #26790 )
2023-11-14 15:03:19 +08:00
23e2bded1a
[fix](Nereids) column pruning under union broken unexpectedly ( #26884 )
...
introduced by PR #24060
2023-11-14 00:39:32 -06:00
f1d90ffc4e
[regression](nereids) add test case for partition prune ( #26849 )
...
* list selected partition name in explain
* add prune partition test case （multi-range key）
2023-11-14 11:51:32 +08:00
0a9d71ebd2
[Fix](Planner) fix varchar does not show real length ( #25171 )
...
Problem:
when we create table with datatype varchar(), we regard it to be max length by default. But when we desc, it does not show
real length but show varchar()
Reason:
when we upgrade version from 2.0.1 to 2.0.2, we support new feature of creating varchar(), and it shows the same way with
ddl schema. So user would confuse of the length of varchar
Solved:
change the showing of varchar() to varchar(65533), which in compatible with hive
2023-11-14 10:49:21 +08:00
cd7ad99de0
[improvement](regression-test) add chunked transfer json test ( #26902 )
2023-11-14 10:31:30 +08:00
fef627c0ba
[Fix](Txn) Fix transaction write to sequence column error ( #26748 )
2023-11-14 10:30:10 +08:00
ec40603b93
[fix](parquet) compressed_page_size has the same meaning in page v1 and v2 ( #26783 )
...
1. Parquet with page v2 is parsed error when using other codec except snappy. Because `compressed_page_size` has the same meaning in page v1 and v2, it always contains the bytes of definition level, repetition level and compressed data.
2. Add regression test for `fix_length_byte_array` stored decimal type, and dictionary encoded date/datetime type.
2023-11-14 08:30:42 +08:00
2853efd4ee
[fix](partial update) Fix NPE when the query statement of an update statement is a point query in OriginPlanner ( #26881 )
...
close #26882
We should not use the singleNodePlan to generate the rootPlanFragment if the query is inside a insert statement or distributedPlanner will be null.
introduced in #15491
2023-11-13 22:08:06 +08:00
4ecaa921f9
[regression-test](num_as_string) test num_as_string ( #26842 )
2023-11-13 21:48:13 +08:00
ebc15fc6cc
[fix](transaction) Fix concurrent schema change and txn cause dead lock ( #26428 )
...
Concurrent schema change and txn may cause dead lock. An example:
Txn T commit but not publish;
Run schema change or rollup on T's related partition, add alter replica R;
sc/rollup add a sched txn watermark M;
Restart fe;
After fe restart, T's loadedTblIndexes will clear because it's not save to disk;
T will publish version to all tablet, including sc/rollup's new alter replica R;
Since R not contains txn data, so the T will fail. It will then always waitting for R's data;
sc/rollup wait for txn before M to finish, only after that it will let R copy history data;
Since T's not finished, so sc/rollup will always wait, so R will nerver copy history data;
Txn T and sc/rollup will wait each other forever, cause dead lock;
Fix: because sc/rollup will ensure double write after the sched watermark M, so for finish transaction, when checking a alter replica:
if txn id is bigger than M, check it just like a normal replica;
otherwise skip check this replica, the BE will modify history data later.
2023-11-13 21:39:28 +08:00
2f32a721ee
[refactor](jni) unified jni framework for jdbc catalog ( #26317 )
...
This commit overhauls the JDBC connector logic within our project, transitioning from the previous mechanism of fetching data through JNI calls for individual ResultSet items to a more efficient and unified approach using the VectorTable data structure.
2023-11-13 14:28:15 +08:00
7e62c3c2de
[fix](Nereids) store user variable in connect context ( #26655 )
...
1.user variable should be case insensitive
2.user variable should be cleared after the connection reset
2023-11-13 12:25:08 +08:00
fa3c7d98c8
[fix](map) the implementation of ColumnMap::replicate was incorrect" ( #26647 )
2023-11-13 12:17:14 +08:00
17b1108635
[fix](nereids)support uncorrelated subquery in join condition ( #26672 )
...
sql select * from t1 a join t1 b on b.id in (select 1) and a.id = b.id; will report an error.
This pr support uncorrelated subquery in join condition to fix it
2023-11-13 11:49:11 +08:00
7e36ab838f
[regression](partial update) Add cases when the deleted rows have non nullable columns without default value ( #26776 )
2023-11-13 10:36:59 +08:00
7332b1b371
[fix](decimal) fix undefined behaviour of divide by zero when cast string to decimal ( #26822 )
...
* [fix](decimal) fix undefined behaviour of divide by zero when cast string to decimal
* fix format
2023-11-13 10:09:06 +08:00
d34dc1c133
[enhancement](regression test) stream load support direct load to be ( #26829 )
2023-11-13 10:07:10 +08:00
183c74f6ae
[decimal](test case) porting postgres regression test cases ( #26836 )
2023-11-13 10:06:43 +08:00
ce64f0c917
[enhancement](Nereids): add phase in shape string ( #26682 )
2023-11-10 14:56:28 +08:00
5c3fed216d
[fix](transaction) Fix publish txn wait too long when not meet quorum ( #26659 )
2023-11-10 14:55:26 +08:00
cdba4936b4
[feature](nereids) Support group commit insert ( #26075 )
2023-11-10 14:20:14 +08:00
7878c08e15
[Revert](merge-on-write) Don't use delete bitmap to mark delete for rows with delete sign when sequence column doesn't exist ( #26721 )
2023-11-10 13:55:40 +08:00
c02c00974f
[Bug](agg-state) fix file load insert wrong data to agg_state ( #26581 )
...
fix file load insert wrong data to agg_state
2023-11-10 11:03:13 +08:00
7ce746654a
[test](jdbc) add doris and sqlserver jdbc catalog test case ( #26656 )
2023-11-10 10:32:09 +08:00
49cffd0bc9
[fix](JdbcCatalog) fix that the predicate column name does not have back quote when querying the JDBC appearance ( #26479 )
2023-11-10 09:54:39 +08:00
328e5b120b
[fix](group commit) Group commit support sequence column for unique table ( #26652 )
2023-11-09 22:43:37 +08:00
02fe71dcd6
[test](serialisation) Serialise some cases and enable str_to_date tests #26651
...
1 enable the cases about str_to_date, which have been muted because some parallel config influence.
2 serialise some cases which called admin set config
2023-11-09 21:53:32 +08:00
8434389358
[fix](jdbc) fix clickhouse catalog arr nullable and add case ( #26639 )
2023-11-09 19:32:05 +08:00
fac1b2b192
[fix](planner)cast floating point type to bigint for bit functions ( #26598 )
2023-11-09 18:06:06 +08:00
eca747413d
[Fix](partial update) Fix core when doing partial update on tables with row column after schema change ( #26632 )
2023-11-09 18:00:05 +08:00
baae7bf339
[fix](information_schema)fix bug that metadata_name_ids error tableid and append information_schema case. ( #26238 )
...
fix bug that #24059 .
Added some information_schema scanner tests.
files
schema_privileges
table_privileges
partitions
rowsets
statistics
table_constraints
Based on infodb_support_ext_catalog=false, it currently includes tests for all tables under the information_schema database.
2023-11-09 14:07:12 +08:00
22bf2889e5
[feature](tvf)(jni-avro)jni-avro scanner add complex data types ( #26236 )
...
Support avro's enum, record, union data types
2023-11-09 13:58:49 +08:00
0929ad568f
[fix](regression) move fault-injection data to the right place ( #26618 )
...
Signed-off-by: freemandealer <freeman.zhang1992@gmail.com >
2023-11-09 12:22:13 +08:00
57ed781bb6
[fix](regression-test) Add tvf regression tests ( #26455 )
2023-11-09 12:09:32 +08:00
d1438a8563
[Fix](orc-reader) Fix orc complex types when late materialization was turned on by disabling late materialization in this case. ( #26548 )
...
Fix orc complex types when late materialization was turned on in orc reader by disabling late materialization in this case.
2023-11-09 12:05:43 +08:00
f6b7046a6e
[fix](regression-test) add tests for jdbc catalog ( #26608 )
2023-11-09 11:59:35 +08:00
55b2988bfd
[Opt](date_add/sub) Throw exception when result of date_add/sub out of range ( #26475 )
2023-11-09 08:46:51 +08:00
5bcf6bfd46
[fix](jdbc catalog) fix mysql zero date ( #26569 )
2023-11-08 21:41:56 +08:00
06343e6d68
[opt](nereids)replace scan by empty relation when all partitions are pruned ( #26514 )
...
* replace scan by empty relation when all partitions are pruned
2023-11-08 20:54:35 +08:00
