Commit Graph

109 Commits

Author SHA1 Message Date
1de2bf6e54 Backup: add checkpoint (#38648)
close pingcap/tidb#38647
2022-11-30 12:52:00 +08:00
d61b8c423f log-backup: add flag for restore point (#39431)
close pingcap/tidb#39379
2022-11-29 11:41:59 +08:00
84703efd01 br: modify collate.newCollationEnabled according to the config of the cluster (#39173)
close pingcap/tidb#39150
2022-11-22 19:19:58 +08:00
2103466f98 br: PiTR restore batch apply kv-files (#38772)
close pingcap/tidb#38211
2022-11-22 16:43:58 +08:00
74aa891cdd br: replace GetTS with GetTSWithRetry (#38948)
ref pingcap/tidb#36910
2022-11-22 14:47:58 +08:00
6477e7521f br: remove error when create the backup client to make caller easier (#39303)
close pingcap/tidb#39302
2022-11-22 12:27:57 +08:00
96708adc93 log-backup: truncate metadata streamingly (#38870)
close pingcap/tidb#38869, ref pingcap/tidb#38939, ref pingcap/tidb#38940
2022-11-08 16:47:50 +08:00
bd081f1214 br: ebs volume snapshot backup and restore with flashback solution (#38700)
close pingcap/tidb#36215
2022-11-03 16:20:00 +08:00
b9029f19b8 log-backup: using streaming to load metadata for preventing OOM (#38386)
close pingcap/tidb#38404
2022-10-28 13:45:58 +08:00
18698d1558 extension: add more informations to extension context (#38693)
ref pingcap/tidb#38493
2022-10-27 20:15:57 +08:00
c4316fbc74 log-backup: support restart task after stop a log-backup task (#38381)
close pingcap/tidb#38382
2022-10-27 13:19:57 +08:00
ac0d36b2c9 log-backup: set checkpoint-ts - 1 as safe-point-ts (#38249)
close pingcap/tidb#38250
2022-10-26 14:19:57 +08:00
9d56ee97b1 log-backup: add progress bar for truncate (#36454)
close pingcap/tidb#36914
2022-10-26 12:29:56 +08:00
400f585578 br/restore: try keep table ID when possible (#38033)
close pingcap/tidb#38438
2022-10-14 12:21:52 +08:00
805ca3e9c1 *: close resultsets (#38146) 2022-09-26 11:25:43 +08:00
4e1860b97c log-backup: Refactor daemon (#36763)
close pingcap/tidb#36762
2022-09-23 17:25:42 +08:00
6a0638b0c1 br: fix restore full fail when restore cluster <= 6.2 using br 6.3 (#38082)
close pingcap/tidb#38055
2022-09-22 19:29:03 +08:00
b84ea2326a br: modify the column of Restore Statement (#36281)
close pingcap/tidb#36142, close pingcap/tidb#37029
2022-09-22 15:27:03 +08:00
85b8104e21 ddl, br: improve compatibility for PiTR and ingest ddl jobs (#38029)
ref pingcap/tidb#38045
2022-09-22 11:15:02 +08:00
0e3e44001b ebs-br: change command line flags of restore (#38027)
close pingcap/tidb#38011
2022-09-21 23:21:03 +08:00
181fb8e905 br: merge feature branch of ebs-based-br (#37297)
ref pingcap/tidb#35306
2022-09-18 20:37:00 +08:00
6e6b553e15 log-backup: compression type for log backup file (#37778)
ref pingcap/tidb#36924
2022-09-17 00:58:59 +08:00
fae88ae561 log-backup: refuse PiTR if the full-snapshot and stream-log do not come from the same cluster (#37546)
close pingcap/tidb#29501
2022-09-15 13:38:59 +08:00
a25917f156 cloud: support backup to s3 when object lock enabled. (#37392)
close tikv/tikv#13442
2022-09-15 12:14:59 +08:00
08b1faf2ad log-backup: restore the merge small backup files. (#36925)
close pingcap/tidb#36924
2022-09-08 12:20:57 +08:00
29591587c7 linter: fix unconvert output (#37663) 2022-09-07 18:06:57 +08:00
2858bc1571 *: refine mock session manager (#37400) 2022-08-26 15:06:21 +08:00
f16f52ca7c br: restore tiflash replica count after PiTR (#37181)
close pingcap/tidb#37208
2022-08-25 17:58:21 +08:00
14f9b043cb br: add an example in timezone-check error message (#36895)
ref pingcap/tidb#36717
2022-08-18 14:42:52 +08:00
d6d0ce7b10 *: gofmt code for adapting go 1.19 (#36832)
ref pingcap/tidb#36879
2022-08-11 12:50:49 +08:00
5b4a82dbf4 *: upgrade bazel timeout (#36776) 2022-08-02 11:10:05 +08:00
158ba1a1d9 *: enable part revive for all code (#36703) 2022-08-01 14:02:06 +08:00
1fe666add6 *: upgrade staticcheck (#36745) 2022-08-01 11:58:05 +08:00
cb211e0edf br: add flag to control whether restore priv table on default (#36652)
close pingcap/tidb#36651
2022-07-30 16:40:03 +08:00
90e9b63a29 log-backup: format the date displayed in --help (#36713)
close pingcap/tidb#36717
2022-07-29 20:33:13 +08:00
3df01375c0 log-backup: filter out unnecessary metadata files (#36656)
close pingcap/tidb#36648
2022-07-29 14:59:13 +08:00
7a47538f4e br: enhance the output infomation (#36676)
ref pingcap/tidb#29501
2022-07-29 13:25:11 +08:00
56dadec862 ddl: Preparation for add index acceleration (#36085)
ref pingcap/tidb#35983
2022-07-28 16:43:11 +08:00
9b7060393e log-backup: fixed bugs about log truncate left some files and log truncate update log-min-date (#36616)
close pingcap/tidb#36376, ref pingcap/tidb#36533, ref pingcap/tidb#36535
2022-07-28 13:13:10 +08:00
6ae88c4309 br: use one shot session to close domain ASAP (#36558)
* br: use one shot session to close domain ASAP



Co-authored-by: Ian <ArGregoryIan@gmail.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
2022-07-26 20:15:53 +08:00
bf743a6ca0 *: enable flaky test for all test (#36385)
close pingcap/tidb#36404
2022-07-22 15:35:09 +08:00
6c704f756f log-backup: remove the timezone from log-date (#36369)
close pingcap/tidb#36370
2022-07-21 20:01:09 +08:00
656b5a5e80 br: fix broken integration test (#36411) 2022-07-21 16:49:09 +08:00
ab513bf0ca ddl: support concurrent ddl (#32169)
* 1. init ddl tables

create tidb_ddl_job, tidb_ddl_reorg, tidb_ddl_history tables with raw meta write, these 3 tables is use to replace the ddl job queue and reorg and history hash table

* 2. setup concurrent ddl env and add ddl worker pool

adds the ddl worker pool definition, the ddl job manager will find a job and ship it to a worker in the worker pool.
Also, this commit provides a sessionctx wrapper, only use in ddl relate. it just wraps begin, commit and execute

* 3. add ddl manager to handle ddl job

* 4. reorg handler for concurrent ddl

just implements the partner of the reorg information.

* 5. manage ddl jobs for concurrent ddl

add the partner of add job, delete job and many others related to history job
because many of the functions need a session now, we just change the caller

* 6. add metrics for concurrent ddl

add metrics

* 7. support multiple tables

* 8. fix test

* 9. migrate ddl between table and queue

support switch between the old and new ddl framework, migrate the existing ddl job between queue and table

* 10. check tikv version and set reorg worker count according cpu count

* *: add featuretag on tests

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

* use a determined table id for 3 tables

* remove ctx value

* add GetSchemaVersionWithNonEmptyDiff function

* address tangenta and zimulala comment

* use only one etcd path

* make ActionRenameTable support multi-schema

* reset sql digest to make top sql work correct

* add comment

* fix test

* remove 0 for schema version lock

Co-authored-by: xiongjiwei <xiongjiwei1996@outlook.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
Co-authored-by: wjHuang <huangwenjun1997@gmail.com>
2022-07-20 21:20:07 +08:00
63f5d286ff log-backup: add version check for pitr (#36268)
close pingcap/tidb#36267
2022-07-18 23:27:07 +08:00
98595124b2 log-backup: added more metrics and hint; fixed a bug may cause inf loop (#36228)
ref pingcap/tidb#29501
2022-07-18 22:17:07 +08:00
48830de368 *: update client-go to improve performance for sortRegion (#36275) 2022-07-18 17:49:08 +08:00
0d6741039a log-backup: calculate global checkpoint-ts from storage (#36255)
close pingcap/tidb#36246
2022-07-18 14:09:07 +08:00
1883a4444d br: restore privilege tables on full restore by default (#35397)
close pingcap/tidb#35395
2022-07-18 13:11:06 +08:00
f47978c3c2 log-backup: add the switch for log backup (#36115)
ref pingcap/tidb#29501
2022-07-15 14:31:06 +08:00