ab513bf0ca
ddl: support concurrent ddl ( #32169 )
...
* 1. init ddl tables
create tidb_ddl_job, tidb_ddl_reorg, tidb_ddl_history tables with raw meta write, these 3 tables is use to replace the ddl job queue and reorg and history hash table
* 2. setup concurrent ddl env and add ddl worker pool
adds the ddl worker pool definition, the ddl job manager will find a job and ship it to a worker in the worker pool.
Also, this commit provides a sessionctx wrapper, only use in ddl relate. it just wraps begin, commit and execute
* 3. add ddl manager to handle ddl job
* 4. reorg handler for concurrent ddl
just implements the partner of the reorg information.
* 5. manage ddl jobs for concurrent ddl
add the partner of add job, delete job and many others related to history job
because many of the functions need a session now, we just change the caller
* 6. add metrics for concurrent ddl
add metrics
* 7. support multiple tables
* 8. fix test
* 9. migrate ddl between table and queue
support switch between the old and new ddl framework, migrate the existing ddl job between queue and table
* 10. check tikv version and set reorg worker count according cpu count
* *: add featuretag on tests
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com >
* use a determined table id for 3 tables
* remove ctx value
* add GetSchemaVersionWithNonEmptyDiff function
* address tangenta and zimulala comment
* use only one etcd path
* make ActionRenameTable support multi-schema
* reset sql digest to make top sql work correct
* add comment
* fix test
* remove 0 for schema version lock
Co-authored-by: xiongjiwei <xiongjiwei1996@outlook.com >
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io >
Co-authored-by: wjHuang <huangwenjun1997@gmail.com >
2022-07-20 21:20:07 +08:00
f84554bd07
expression: fix binary str numeric result type ( #36258 )
...
close pingcap/tidb#27698
2022-07-19 16:43:07 +08:00
4cade24ed6
*: support the binary plan through EXPLAIN format='binary' and tidb_decode_binary_plan ( #36221 )
...
ref pingcap/tidb#35889
2022-07-18 16:27:07 +08:00
7953d60c1b
*: switch to flat plan in stmt summary, slow log, etc ( #36069 )
...
ref pingcap/tidb#35888
2022-07-14 17:37:06 +08:00
2193bfe3b7
*: remove system var tidb_enable_change_multi_schema ( #36167 )
...
ref pingcap/tidb#14766
2022-07-14 10:37:05 +08:00
0f7faa67d8
ddl: refine GetDDLInfo function ( #35969 )
2022-07-06 13:19:02 +08:00
637d00c05a
ddl: refine history job code ( #35967 )
2022-07-06 11:43:02 +08:00
157c178f27
*: seperate TxnScope and ReplicaReadScope ( #35877 )
...
close pingcap/tidb#31772
2022-07-05 11:33:02 +08:00
65e0b8e541
client: KV requests with source label ( #34833 )
...
close pingcap/tidb#33963
2022-07-01 19:46:39 +08:00
5b305c34f8
txn: manage the initialization of RCCheckTS by transaction context provider ( #35554 )
...
close pingcap/tidb#35497
2022-06-29 16:14:39 +08:00
4fc9551dd1
sessionctx, types, executor: support encoding and decoding user-defined variables ( #35343 )
...
close pingcap/tidb#35288
2022-06-15 21:02:34 +08:00
5d4745e9d4
parser, executor: support show/set session_states ( #35263 )
...
close pingcap/tidb#35259
2022-06-10 15:24:31 +08:00
b0245cda58
*: clean up coprocessor streaming code ( #35277 )
...
close pingcap/tidb#35276
2022-06-10 10:50:31 +08:00
933d5523cb
parser: fix the Incompatible parser behavior for HAVING clause #34642 ( #34751 )
...
close pingcap/tidb#34642
2022-05-30 22:52:26 +08:00
1fac141f35
*: close the resource on error to aovid resource leak ( #34837 )
...
close pingcap/tidb#34666 , ref pingcap/tidb#34849
2022-05-30 18:00:27 +08:00
dbce66a055
test: fix integration-ddl-test ( #35014 )
...
close pingcap/tidb#35020
2022-05-30 16:00:27 +08:00
7f023bd61b
ddl: wrap some functions for ddl job ( #34920 )
...
close pingcap/tidb#34919
2022-05-29 15:38:26 +08:00
a9614850f9
*: move config file option oom-action to sysvar ( #34644 )
...
ref pingcap/tidb#33769
2022-05-19 05:16:38 +08:00
bbd7541322
*: refine 'tidb_snapshot' behavior ( #34573 )
...
close pingcap/tidb#34529
2022-05-11 20:54:34 +08:00
73f3435346
ddl: move some admin function to ddl ( #34527 )
...
close pingcap/tidb#34526
2022-05-10 18:14:34 +08:00
2258bc54c4
txn: Add EnterNewTxn and OnStmtStart methods to TxnManager ( #34263 )
...
close pingcap/tidb#34390
2022-05-07 21:02:32 +08:00
1bf64c60f4
*: Refactor FieldType structure ( #33569 )
...
close pingcap/tidb#32116
2022-04-26 19:28:51 +08:00
fb342ff861
topsql: fix issue of topsql failed catch the running SQL when topsql is enabled in execution ( #33861 )
...
close pingcap/tidb#33859
2022-04-26 13:16:51 +08:00
d23a81488a
*: Move stale read context to TxnManager ( #33812 )
...
close pingcap/tidb#31954 , close pingcap/tidb#33014 , close pingcap/tidb#33832
2022-04-25 16:52:51 +08:00
02de667042
testkit: add MockSessionManager ( #34124 )
...
ref pingcap/tidb#31716
2022-04-21 12:14:04 +08:00
58e2a552d3
executor: migrate test-infra to testify for executor_test.go ( #34062 )
...
close pingcap/tidb#28577 , close pingcap/tidb#33425 , close pingcap/tidb#33432
2022-04-18 14:46:03 +08:00
1fcc407b3b
executor: migrate test-infra to testify for executor_test.go testSuiteP2 ( #34049 )
...
close pingcap/tidb#33427
2022-04-18 13:10:02 +08:00
2b00d3e6e0
executor: migrate test-infra to testify for testSuite2 ( #34046 )
...
close pingcap/tidb#33431
2022-04-18 11:50:02 +08:00
345b1a886f
executor: migrate test-infra to testify for part of executor_test.go ( #34040 )
...
ref pingcap/tidb#28577
2022-04-17 18:08:36 +08:00
b5de819d00
util: fix memory.reArrangeFallback cpu usage ( #30414 )
...
close pingcap/tidb#30353
2022-04-15 12:48:35 +08:00
4dbed02d74
executor: fix unstable test of TestTimestampDefaultValueTimeZone ( #33926 )
...
close pingcap/tidb#33923
2022-04-13 16:36:36 +08:00
192482da3a
executor: migrate test-infra to testify for testSerialSuite ( #33911 )
...
close pingcap/tidb#33441
2022-04-13 15:14:36 +08:00
9445f318dc
executor: migrate test-infra to testify for testSerialSuite1 ( #33874 )
...
close pingcap/tidb#33438
2022-04-12 14:40:35 +08:00
eaf52d6628
executor: migrate test-infra to testify for testSuite1 ( #33776 )
...
Signed-off-by: tison <wander4096@gmail.com >
Co-authored-by: Weizhen Wang <wangweizhen@pingcap.com >
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io >
2022-04-07 16:53:33 +08:00
a2b2dc0745
executor: migrate test-infra to testify for part of testSuite ( #33741 )
...
ref pingcap/tidb#33425
2022-04-07 01:22:31 +08:00
1c658e5eb6
executor: migrate test-infra to testify for executor_test.testSuite(6|8) ( #33713 )
...
close pingcap/tidb#33433 , close pingcap/tidb#33434
2022-04-06 11:34:31 +08:00
27cf5bc384
executor: migrate test-infra to testify for part of executor_test ( #33709 )
...
close pingcap/tidb#28577
2022-04-06 09:48:31 +08:00
8dc1f6b092
executor: migrate test-infra to testify for executor_test.testSerialSuite2 ( #33706 )
...
close pingcap/tidb#33429
2022-04-04 23:52:31 +08:00
843b0f43db
executor: migrate test-infra to testify for executor_test.go testSuiteWithData ( #33402 )
...
ref pingcap/tidb#28577
2022-03-25 14:06:33 +08:00
30bf4117c9
executor: migrate test-infra to testify for executor_test.go testSuiteP1 ( #33394 )
...
ref pingcap/tidb#28577
2022-03-24 19:32:33 +08:00
d282940b42
*: remove syncLog ( #33306 )
...
close pingcap/tidb#33305
2022-03-23 13:12:32 +08:00
f9beb51a64
expr: fix nil panic for In function with bit column ( #33204 )
...
close pingcap/tidb#33070
2022-03-21 12:18:32 +08:00
2dd0074e4e
executor: fix wrong result of delete multiple tables using left join ( #33055 )
...
close pingcap/tidb#31321
2022-03-16 12:05:52 +08:00
ffaeb3e80d
ddl: improve test to avoid cycle import ( #32058 )
...
ref pingcap/tidb#31716
2022-03-14 12:11:51 +08:00
9a4ca3ca69
*: Remove deprecated streaming ( #32765 )
...
ref pingcap/tidb#32679
2022-03-11 03:15:50 +08:00
f60017af36
expression: validate args to check whether it is out of range ( #32898 )
...
close pingcap/tidb#32871
2022-03-10 23:43:51 +08:00
52fa48ce2b
sessionctx,executor: Revert "sessionctx,executor: disallowed set null to sysvar" ( #32988 )
...
close pingcap/tidb#32987
2022-03-10 23:13:51 +08:00
77730b6d9e
sessionctx,executor: disallowed set null to sysvar ( #32879 )
...
close pingcap/tidb#32850
2022-03-10 01:13:49 +08:00
a1d8f2f3ac
*: remove ddl dependency ( #32910 )
...
ref pingcap/tidb#31716
2022-03-09 08:15:48 +08:00
223f5f8acd
executor: migrate test-infra to testify for part3 of executor_test ( #32884 )
...
ref pingcap/tidb#28577
2022-03-08 01:41:48 +08:00
