Commit Graph

50 Commits

Author SHA1 Message Date
ae56739f88 [enhancement](sequence col) add session variable to skip sequence column check while INSERT INTO (#41655) (#41720)
cp #41655
2024-10-12 15:30:20 +08:00
3c535e80dd [fix](compatibility) type toSql should return lowercase string (#38012) (#38517)
pick from master #38012

revert #25951
2024-08-09 11:35:42 +08:00
2d5eb55310 [cherry-pick](branch-2.1) Pick "[Enhancement](partial update) Add some partial update case (#36385)" (#37382)
#36385
2024-07-09 10:52:04 +08:00
c66975b244 [branch-2.1] Pick "[Fix](regression) Fix p0 case test_unique_table_auto_inc_concurrent #37048" (#37050)
## Proposed changes

pick https://github.com/apache/doris/pull/37048
2024-07-02 11:15:50 +08:00
4210a6a8d6 [branch-2.1] PIck "[Fix](autoinc) Hanlde the processing of auto_increment column on exchange node rather than on TabletWriter when using TABLET_SINK_SHUFFLE_PARTITIONED #36836" (#37029)
## Proposed changes

pick https://github.com/apache/doris/pull/36836
2024-07-01 09:56:30 +08:00
12dddfc26c [branch-2.1] Pick "[Fix](autoinc) try fix concurrent load problem with auto inc column #36421" (#37027)
## Proposed changes

pick https://github.com/apache/doris/pull/36421
2024-06-30 13:10:03 +08:00
42425808a1 [Cherry-Pick](branch-2.1) Pick "Fix multiple replica partial update auto inc data inconsistency problem #34788" (#35056)
* [Fix](auto inc) Fix multiple replica partial update auto inc data inconsistency problem (#34788)

* **Problem:** For tables with auto-increment columns, updating partial columns can cause data inconsistency among replicas.

**Cause:** Previously, the implementation for updating partial columns in tables with auto-increment columns was done independently on each BE (Backend), leading to potential inconsistencies in the auto-increment column values generated by each BE.

**Solution:** Before distributing blocks, determine if the update involves partial columns of a table with an auto-increment column. If so, add the auto-increment column to the last column of the block. After distributing to each BE, each BE will check if the data key for the partial column update exists. If it exists, the previous auto-increment column value is used; if not, the auto-increment column value from the last column of the block is used. This ensures that the auto-increment column values are consistent across different BEs.

* 2

* [Fix](regression-test) Fix auto inc partial update unstable regression test (#34940)
2024-05-20 15:43:46 +08:00
4079a7b6ab [fix](txn insert) Fix txn insert into values for sequence column or column name is keyword (#33336) 2024-04-10 16:21:31 +08:00
0537952573 [Fix](auto-inc) Fix auto inc unique table case (#32393) 2024-03-22 08:52:16 +08:00
6acdd9cd48 The issue introduced by the recently added auto-increment column works fine on a single node but may result in discontinuous auto-increment IDs when running on a cluster. This PR has been modified to check for the uniqueness of the auto-increment column values instead of checking for equality to a fixed value. (#32115) 2024-03-12 21:51:36 +08:00
1d2d0bd411 [fix](update) Update set value should consider sequence column (#31626)
When using update command to set column value, if the column is sequence column, the column 'DORIS_SEQUENCE_COL' should also be set to the same value.
2024-03-06 13:08:04 +08:00
7c30cb20fd [Fix](partial update) Fix partial update load false when schema includes auto increment column (#31725)
Problem:
When partially updating columns without specifying the auto-increment column, and the imported data contains new keys, an error stating the auto-increment column could not be found occurs.

Reason:
The logic for partial column updates does not account for new keys in auto-increment columns. Since auto-increment columns can be generated by the system, it's possible to omit this column data during import. However, partial column updates treat this as a regular column, expecting it to be nullable or have a default value for automatic filling, overlooking the fact that auto-increment columns can also be auto-filled. This oversight leads to the error.

Solution:
Incorporate a check for auto-increment columns into the partial column update logic, and include the logic for generating auto-increment column values in the process of completing partial updates.
2024-03-06 13:06:27 +08:00
2cb46eed94 [Feature](auto-inc) Add start value for auto increment column (#30512) 2024-02-16 10:12:23 +08:00
6503aaf7db [feature](planner) allow HLL and QUANTILE_STATE types on duplicate and unique table (#28546) 2023-12-19 09:54:24 +08:00
088bb80a9c [fix](test) fix case of test_unique_table_new_sequence (#28442)
pr #28105 and #28031 merging conflict caused this case to fail.
2023-12-15 15:10:39 +08:00
8ca7bd8f98 [enhancement](bitmap)support bitmap type for non-key column in duplicate table (#28392) 2023-12-14 23:59:12 +08:00
4300fdce43 [feature](merge-on-write) enable merge-on-write by default again (#28105)
fix #27188, #28096
2023-12-14 19:35:52 +08:00
0868526f6a [fix](sequence column) fix update fail on nereids planner (#28031)
1. if we set enable_fallback_to_original_planner to false, the UPDATE sql in regression case test_unique_table_sequence will fail due to:“Table test_uniq_sequence has sequence column, need to specify the sequence column”,The bug is introduced by [fix](sequence column) insert into should require sequence column in all scenario #27780
2. fix insert fail in transaction mode, which is introduced by [refactor](Nereids) let insert into compatible with legacy planner #27947
3. add cases with session variable enable_fallback_to_original_planner false
2023-12-13 19:41:41 +08:00
2b2c2dd772 [fix](sequence column) insert into should require sequence column in all scenario (#27780) 2023-11-30 23:27:58 +08:00
fef627c0ba [Fix](Txn) Fix transaction write to sequence column error (#26748) 2023-11-14 10:30:10 +08:00
9b494f4b36 [Fix](autoinc) skip to fill the auto increment column when the input column is not nullable (#23905) 2023-09-07 11:13:22 +08:00
c7b9eb5f9c [enhancement](bitmap)support bitmap type for non-key column in unique table (#23228) 2023-08-23 14:21:22 +08:00
0aa00026bb [fix](autoinc) ignore column property isAutoInc() for create table as select ... statement(#22827) 2023-08-10 23:25:54 +08:00
2b4bfe5be7 [fix](autoinc) fix _fill_auto_inc_cols when the input column is ColumnConst (#22175) 2023-07-25 14:41:36 +08:00
74313c7d54 [feature-wip](autoinc)(step-3) add auto increment support for unique table (#22036) 2023-07-21 13:24:41 +08:00
367ad9164a [feature-wip](auto-inc)(step-2) support auto-increment column for duplicate table (#19917) 2023-07-20 18:03:39 +08:00
0469c02202 [Test](regression) Temporarily disable quickTest for SHOW CREATE TABLE to adapt to enable_feature_binlog=true (#21247) 2023-07-05 10:12:02 +08:00
5506faa7b4 [datetimev2](minor) Add scale parameter for datetimev2 (#21176) 2023-06-27 19:55:35 +08:00
cc53391c9a Revert "[feature](merge-on-write) enable merge on write by default (#… (#21041) 2023-06-21 18:36:46 +08:00
4d84cd8ca1 Revert "Revert "[Test](regression) CCR syncer thrift interface regression test (#20935)" (#20990)" (#21022)
This reverts commit 2a294801f1324a999570158eea3224239eefbb29.
2023-06-21 15:20:21 +08:00
2a294801f1 Revert "[Test](regression) CCR syncer thrift interface regression test (#20935)" (#20990)
This reverts commit dd482b74c849b022862e7cfb1f1d0b933a84e3d2.
2023-06-19 21:38:03 +08:00
dd482b74c8 [Test](regression) CCR syncer thrift interface regression test (#20935) 2023-06-18 00:13:09 +08:00
187bf14d81 [feature-wip](auto-inc)(step-1) add syntax support for duplicate table (#20284)
Co-authored-by: yifeng <cnissnzg@126.com>
2023-06-07 22:01:28 +08:00
05d497d21e [fix](sequence) value predicates shouldn't be push down when has sequence column (#20408)
* (fix)[sequence] value predicates shouldn't be push down when has sequence column

* add case
2023-06-05 19:18:34 +08:00
ecdc5124be [feature-wip](duplicate-no-keys) schame change support for duplicate no keys (#19326) 2023-06-02 09:22:41 +08:00
5167dc1251 [feature](merge-on-write) enable merge on write by default (#19017) 2023-05-11 11:10:48 +08:00
8a4a92f658 [Enchancement](compatible) show dateV2/datetimeV2 to date/datetime (#18358)
show dateV2/datetimeV2 to date/datetime

modify show create table
modify desc table
use desc table all to get real type from column ColumnType
2023-04-09 10:34:14 +08:00
849b5b7b8f [fix](sequence) fix that the result is wrong when load multiple duplicate keys (#17575) 2023-03-09 20:59:23 +08:00
edead494cb [Enhancement](storage) add a new hidden column __DORIS_VERSION_COL__ for unique key table (#16509) 2023-02-23 15:47:17 +08:00
6a5277b391 [fix](sequence-column) MergeIterator does not use the correct seq column for comparison (#16494) 2023-02-10 17:51:15 +08:00
69f34cd1c3 [fix](load) sequence column do not compare correctly in memtable (#16211) 2023-02-02 11:00:23 +08:00
eef1f432dd [Bug](datetimev2/decimalv3) Fix wrong predicate infer rule (#15574) 2023-01-04 10:03:43 +08:00
5ec4e5586f [refactor]remove seek block in segmentIterator (#15413)
* remove seek block

* add reg test

Co-authored-by: Wang Bo <wangbo36@meituan.com>
2022-12-30 14:14:16 +08:00
fb140d0180 [Enhancement](sequence-column) optimize the use of sequence column (#13872)
When you create the Uniq table, you can specify the mapping of sequence column to other columns.
You no longer need to specify mapping column when importing.
2022-11-17 22:39:09 +08:00
37e4a1769d [fix](sequence) fix that update table core dump with sequence column (#13847)
* [fix](sequence) fix that update table core dump with sequence column

* update
2022-11-03 09:02:21 +08:00
6b6d548df9 [enhancement](test) add more p0 cases (#12285) 2022-09-29 10:45:17 +08:00
df9dcba6db [regression-case](improve) improve regression test case (#12979) 2022-09-27 08:53:53 +08:00
8364165e30 [regression_test](testcase) add regression test case from session variable skip_storage_engine_merge, skip_delete_predicate and show_hidden_columns (#12617)
also add this function to new olap scan node.
2022-09-16 10:33:12 +08:00
1f9eec5462 [Regression](datev2) Add test cases for datev2/datetimev2 (#11831) 2022-08-19 10:57:55 +08:00
ee4d9d4347 [improvement](test) group some cases and group a case to p0 if it is not grouped (#11548) 2022-08-06 15:12:08 +08:00