Commit Graph

88 Commits

Author SHA1 Message Date
eb35c773b5 ddl: avoid commit conflicts when updating/delete from mysql.tidb_ddl_reorg. (#38738)
* Added test case

* ddl fix #38669.

The issue was that mysql.tidb_ddl_reorg table was updated by an
inner transaction after the outer transaction started,
which then made a commit conflict in the outer transaction,
when it deleted the same row.

* Fixed typo in comment

* Added test case for #24427

* Disabled tests for CI testing

* Revert "Disabled tests for CI testing"

This reverts commit 17c28f30ba8802c578fed5653107aa6995e17607.

* Revert "Revert "Disabled tests for CI testing""

This reverts commit 65c84d94f7ab4440c739703312f5329619c6ccdc.

* removed test skips

* Clean up the tidb_ddl_reorg entry after DDL is completed

* Use a cleanup job afterwards instead.

* Fixed test

* Moved cleanup before asyncNotify

* More detailed test failure log

* Refined test error message

* Injecting timoeut to get stack traces from CI

* Updated Debug Dump on timeout

* Delete mulitple entries in tidb_ddl_reorg if needed

* Linting

* Linting

* Added CI debug logs

* Linting + CI debugs

* fixed CI debug

* Try to cleanup also if job.State == synced

* check for non-error of runErr instead of error...

* Use a new session, instead of reusing worker.sess

* Also handle case when job == nil

* Removed CI debug logs

* Misssed change session from w.sess to newly created sess

* Improved TestConcurrentDDLSwitch and added CI debug logs

* Always cleaning up all orphan mysql.tidb_ddl_reorg entries

* linting

* Also cleanup if job is nil

* Updated TestModifyColumnReorgInfo + CI debug logs

* more CI debug

* refactored the cleanupDDLReorgHandle code

* Added missing cleanup in handleDDLJobQueue

* Removed debug panic

* Code cleanup

* Test updates

* Debug cleanup

* Cleaned up test after removal of old non-concurrent DDL code merge

* Linting

* always wrap changes to tidb_ddl_reorg in an own transaction

+ fixed some typos

* Minimum fix

* Always update reorg meta, not only on error

* Issue is here :)

* Fixed newReorgHandler

* Wrapped more tidb_ddl_reorg changes into separate transactions

* linting

* Removed updateDDLReorgStartHandle

* cleanups

* Made runInTxn a method on *session, instead of normal function

* Update test

* Final touches

* Removed duplicate test

* CleanupDDLReorgHandles should only be called from HandleJobDone.

* Variable rename

* Renamed 'delete' variabel name

* Updated test

* small revert

* Removed timeout debugging code

* Simplified the cleanup to only start a new txn and not a new session

* Reverted the change of GetDDLInfo

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
2023-01-11 21:18:33 +08:00
45d71af649 ddl: support displaying sub-job reorg type in admin show ddl (#40387)
close pingcap/tidb#40386
2023-01-10 12:50:22 +08:00
b477b1c946 *: Supports create/alter/drop resource group (#40188)
ref pingcap/tidb#38825
2023-01-09 13:44:22 +08:00
cb5affbc3c ddl, parser: Implement the write-reorg state split task related functions, and the related interfaces of backfill worker (#39982)
close pingcap/tidb#37123
2023-01-04 15:20:19 +08:00
6dff69f680 ddl: Support flashback cluster with ddl history (#40209)
ref pingcap/tidb#40026
2022-12-29 21:02:19 +08:00
ad0c202088 *: fix issue of multi-schema change with foreign key (#40042)
close pingcap/tidb#40037
2022-12-21 19:58:55 +08:00
9488ca9db6 *: Add backfill job related tables and operations (#39616)
close pingcap/tidb#37122
2022-12-12 14:34:51 +08:00
7f632be1d5 ddl: add ttl related jobs / execution (#39298)
close pingcap/tidb#39269, close pingcap/tidb#39270, close pingcap/tidb#39271
2022-11-28 10:42:00 +08:00
11ab7b4953 parser: add ActionReorganizePartition (#39183)
ref pingcap/tidb#38535
2022-11-16 18:39:55 +08:00
863d35290c ddl: support add foreign key constraint with auto-create index (#38763)
close pingcap/tidb#38762
2022-11-01 20:27:59 +08:00
6d6e9c4b10 *: show backfill type in admin show ddl jobs (#38733)
ref pingcap/tidb#35983
2022-10-29 00:53:58 +08:00
4bdd20b540 *: support 2-phases flashback cluster (#38164)
ref pingcap/tidb#37961, close pingcap/tidb#38554
2022-10-26 17:01:58 +08:00
300d9e7c63 ddl: Support flashback schema (#37438)
close pingcap/tidb#20463
2022-10-19 15:03:54 +08:00
8f18fcece0 *: delete mInFlashbackCluster related codes (#38241)
ref pingcap/tidb#37197
2022-09-30 12:15:45 +08:00
40e67edcca *: redirect the DML writes to a temp index during creating index (#37709)
ref pingcap/tidb#35983
2022-09-15 11:22:59 +08:00
df844e814b ddl, parser: fix resume pd schedule and cancel for flashback cluster (#37601)
ref pingcap/tidb#37580, close pingcap/tidb#37584
2022-09-05 20:30:55 +08:00
a04000cd89 *: merge feature/flashback-cluster to master (#37529)
ref pingcap/tidb#37197
2022-09-01 10:58:24 +08:00
ccd4f347a2 *: Enable tiflash_fastscan variable and remove fast mode alter statment (#37027)
ref pingcap/tidb#35850
2022-08-29 11:28:22 +08:00
be102a3d55 *: fix timestamp conversion (#36705)
close pingcap/tidb#36707
2022-07-29 19:05:12 +08:00
893a114abc parser: fix DDL job.Clone() (#36523)
close pingcap/tidb#36522
2022-07-25 18:31:10 +08:00
c087f6dc2a table partition: update job dependence for exchange partition with table (#35751)
close pingcap/tidb#35842
2022-07-20 12:41:08 +08:00
8b30e52bcf ddl: only use one schema version for the non-revertible step (#36262)
ref pingcap/tidb#14766
2022-07-18 21:49:07 +08:00
911e7cc360 *: enable revive (#35970)
ref pingcap/tidb#35345
2022-07-15 16:03:06 +08:00
2193bfe3b7 *: remove system var tidb_enable_change_multi_schema (#36167)
ref pingcap/tidb#14766
2022-07-14 10:37:05 +08:00
2f934d67a2 *: support show ddl jobs for sub-jobs (#36168)
ref pingcap/tidb#14766
2022-07-13 20:57:05 +08:00
6e22e47f06 parser,ddl: support 'set tiflash mode' ddl and support check tiflash mode in tiflash_replica table (#35851)
ref pingcap/tidb#35850
2022-07-07 15:15:03 +08:00
d5898d03a9 ddl: remove onDropIndexes (#35960)
ref pingcap/tidb#14766
2022-07-06 12:17:02 +08:00
d1c8af6284 ddl: make ToProxyJob return value to avoid heap allocation (#35882)
ref pingcap/tidb#14766
2022-07-01 16:06:39 +08:00
bc00ddb3ca ddl: remove onDropColumns and onAddColumns (#35862)
ref pingcap/tidb#14766
2022-06-30 22:58:39 +08:00
c6103886b1 *: enable all gosimples (#35762)
ref pingcap/tidb#35345
2022-06-29 11:54:39 +08:00
2c4d1dfd36 ddl: implement the core for multi-schema change (#35429)
ref pingcap/tidb#14766
2022-06-24 19:34:38 +08:00
b3475f2570 ddl: add front-end checks for multi-schema change (#35358)
ref pingcap/tidb#14766
2022-06-15 11:04:33 +08:00
fb67ee4412 ddl: add data structure SubJob for multi-schema change (#35231)
ref pingcap/tidb#14766
2022-06-10 12:36:30 +08:00
93859ddff0 ddl: correct the job.SchemaState in DROP cases (#34235)
ref pingcap/tidb#32939
2022-04-28 12:14:51 +08:00
ffa4ba91e3 parser: turn some functions into member functions of model.Job (#34175)
ref pingcap/tidb#31716
2022-04-24 14:42:49 +08:00
67b073425f ddl: fix inappropriate schema state output by admin show ddl jobs (#33850)
ref pingcap/tidb#23494, close pingcap/tidb#24420
2022-04-11 15:26:34 +08:00
0d7602eb65 *: fix http api document and a comment (#33834)
close pingcap/tidb#33833
2022-04-11 13:50:34 +08:00
be76f885e6 ddl, parser: fix concurrent rename table (#33354)
close pingcap/tidb#32610
2022-04-07 17:48:33 +08:00
1624123b84 *: fix time zone problems encountered when changing the other type columns to timestamp type columns (#31843)
close pingcap/tidb#29585
2022-02-21 23:11:42 +08:00
c8bef987ae *: support variable tidb_last_ddl_info (#32414)
close pingcap/tidb#32402
2022-02-17 16:43:40 +08:00
e50e03c92c parser, ddl: update the DDL job's Location structure (#31199)
close pingcap/tidb#31212
2022-01-03 14:53:51 +08:00
xhe
4f30a14256 ddl: support batch create table (#28763) 2021-12-31 17:13:50 +08:00
6e6db1fd35 binlog: allow multiple ddl targets (#30904) 2021-12-24 12:55:47 +08:00
93e2c12346 *: Rename some names of placement ddl operation (#30622) 2021-12-11 00:41:57 +08:00
28446605c2 expression: fix tidb can't alter table from other-type with null value to timestamp with NOT NULL attribute (#29664) 2021-12-03 15:57:54 +08:00
a90c98816e ddl: fix none info of alter table placement (#29929) 2021-11-24 16:53:53 +08:00
a237e41d9f ddl : support alter table xx nocache operations switch a table into uncacheable (#29621) 2021-11-11 17:19:06 +08:00
49d995d577 *: Remove ALTER TABLE ... ALTER PARTITION ... (#29556) 2021-11-08 18:05:04 +08:00
a3f3924388 parser: support analyze/stats options when create/alter table (#29056) 2021-10-27 14:54:49 +08:00
1777bd0e86 ddl : support alter table xx cache operations (#29022) 2021-10-25 22:08:48 +08:00