Commit Graph

56 Commits

Author SHA1 Message Date
7c88876a7f br: add log backup/restore encryption support (#55757)
close pingcap/tidb#55834
2024-09-26 02:31:09 +00:00
835ea0451a backup: implement store based backup to solve the long tail issue (#53081)
ref pingcap/tidb#52534
2024-05-16 11:13:13 +00:00
81da4f8811 br: send backup request in batch (#52535)
ref pingcap/tidb#52534
2024-05-07 07:31:07 +00:00
07057bdb1f backup: add table-concurrency to control backup tableInfo stats and checksum. (#48570)
close pingcap/tidb#48571
2023-11-16 15:29:17 +00:00
db2e239496 types: remove param explicitTz from types.ParseTime (#48574)
close pingcap/tidb#48573
2023-11-15 08:08:17 +00:00
632cd843b0 executor/brie: use the default value from flags (#48025)
close pingcap/tidb#48000
2023-11-08 13:12:42 +00:00
249a5fe541 br: public the parameter --ignore-stats (#48009)
close pingcap/tidb#48008
2023-11-07 14:45:41 +00:00
6d64b7e89a types, stmtctx, *: use ZeroDate, ZeroInDate and InvalidDate related flags to replace variable in statement context (#47794)
close pingcap/tidb#47507, close pingcap/tidb#47508
2023-10-27 10:44:05 +00:00
895b4e0f12 *: replace mathutil.Max/Min with built-in max/min (#47700)
ref pingcap/tidb#45933
2023-10-17 07:22:59 +00:00
318e82bbb7 *: move most of the packages to 'pkg' (#47123)
ref pingcap/tidb#41241
2023-10-13 14:36:26 +00:00
8d934ea949 stmtctx: move time zone info in stmtctx.StatementContext into TypeCtx (#47592)
close pingcap/tidb#47590
2023-10-13 10:30:47 +00:00
3925f9af7a br: public the parameter --use-backupmeta-v2 and use it in default (#47114)
close pingcap/tidb#47111
2023-10-11 03:22:56 +00:00
257278d064 br: fix unstable integration tests (#47156) 2023-09-27 08:33:17 +00:00
5f41a0f460 br: update new_collation related logs (#46172)
close pingcap/tidb#46136
2023-08-17 10:02:08 +00:00
0f41cfe2a8 br: set resource group context for br kv requests (#45806)
ref pingcap/tidb#44517
2023-08-10 11:16:57 +00:00
dc5748c07c br: allow to change some configurations for checkpoint for backup (#44141)
close pingcap/tidb#44130
2023-05-24 09:21:39 +00:00
a085eff3da br: a new checkpoint file format with smaller space used (#43812)
close pingcap/tidb#43811
2023-05-18 11:21:36 +00:00
0d587a5f7b br/backup_ebs: added flag operator-paused-gc-and-scheduler. (#43687)
ref pingcap/tidb#43559
2023-05-16 03:25:17 +00:00
b51758efdb br: enhancement for checkpoint (#43384)
close pingcap/tidb#43270
2023-05-06 10:44:56 +00:00
6488c35a25 sql: implement SHOW BR JOBS <n:int> and CANCEL BR JOB <n:int> (#43074)
close pingcap/tidb#43072
2023-05-05 10:45:57 +00:00
35e5b45212 br: support backup txn kv data (#40829)
* support txn backup and restore
2023-04-19 15:11:04 +08:00
9cf0ed8e8f br: pipeline backup schemas (#43003)
close pingcap/tidb#43002
2023-04-17 22:51:18 +08:00
f7bc7b9c83 br: turn off the checkpoint mode when use noop storage (#41836)
close pingcap/tidb#41835
2023-03-15 18:28:53 +08:00
7b6e692eba br: display hidden flag of concurrency (#30085) 2023-02-27 19:07:08 +08:00
9cb4c488fc br: Support backup replica read (#40899)
ref pingcap/tidb#40898
2023-02-09 18:06:08 +08:00
8398f0fe09 *: fix a timezone data race which may cause wrong row data (#41146)
close pingcap/tidb#40710
2023-02-08 12:21:58 +08:00
e8c32ae8fb br: adaption for keyspace feature (#40532)
ref pingcap/tidb#40531
2023-02-02 12:21:57 +08:00
89de573660 BR: add lock with ttl for checkpoint backup (#40563)
close pingcap/tidb#40561
2023-01-18 14:35:51 +08:00
bf2cc45206 br: add errch buf for checkpoint (#40166)
close pingcap/tidb#40165
2023-01-04 19:52:20 +08:00
8b76580bf5 backup: config adjust for checkpoint (#39634)
close pingcap/tidb#39632
2022-12-06 17:32:03 +08:00
1de2bf6e54 Backup: add checkpoint (#38648)
close pingcap/tidb#38647
2022-11-30 12:52:00 +08:00
84703efd01 br: modify collate.newCollationEnabled according to the config of the cluster (#39173)
close pingcap/tidb#39150
2022-11-22 19:19:58 +08:00
6477e7521f br: remove error when create the backup client to make caller easier (#39303)
close pingcap/tidb#39302
2022-11-22 12:27:57 +08:00
c4316fbc74 log-backup: support restart task after stop a log-backup task (#38381)
close pingcap/tidb#38382
2022-10-27 13:19:57 +08:00
0e3e44001b ebs-br: change command line flags of restore (#38027)
close pingcap/tidb#38011
2022-09-21 23:21:03 +08:00
181fb8e905 br: merge feature branch of ebs-based-br (#37297)
ref pingcap/tidb#35306
2022-09-18 20:37:00 +08:00
a25917f156 cloud: support backup to s3 when object lock enabled. (#37392)
close tikv/tikv#13442
2022-09-15 12:14:59 +08:00
14f9b043cb br: add an example in timezone-check error message (#36895)
ref pingcap/tidb#36717
2022-08-18 14:42:52 +08:00
6ae88c4309 br: use one shot session to close domain ASAP (#36558)
* br: use one shot session to close domain ASAP



Co-authored-by: Ian <ArGregoryIan@gmail.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
2022-07-26 20:15:53 +08:00
656b5a5e80 br: fix broken integration test (#36411) 2022-07-21 16:49:09 +08:00
ab513bf0ca ddl: support concurrent ddl (#32169)
* 1. init ddl tables

create tidb_ddl_job, tidb_ddl_reorg, tidb_ddl_history tables with raw meta write, these 3 tables is use to replace the ddl job queue and reorg and history hash table

* 2. setup concurrent ddl env and add ddl worker pool

adds the ddl worker pool definition, the ddl job manager will find a job and ship it to a worker in the worker pool.
Also, this commit provides a sessionctx wrapper, only use in ddl relate. it just wraps begin, commit and execute

* 3. add ddl manager to handle ddl job

* 4. reorg handler for concurrent ddl

just implements the partner of the reorg information.

* 5. manage ddl jobs for concurrent ddl

add the partner of add job, delete job and many others related to history job
because many of the functions need a session now, we just change the caller

* 6. add metrics for concurrent ddl

add metrics

* 7. support multiple tables

* 8. fix test

* 9. migrate ddl between table and queue

support switch between the old and new ddl framework, migrate the existing ddl job between queue and table

* 10. check tikv version and set reorg worker count according cpu count

* *: add featuretag on tests

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

* use a determined table id for 3 tables

* remove ctx value

* add GetSchemaVersionWithNonEmptyDiff function

* address tangenta and zimulala comment

* use only one etcd path

* make ActionRenameTable support multi-schema

* reset sql digest to make top sql work correct

* add comment

* fix test

* remove 0 for schema version lock

Co-authored-by: xiongjiwei <xiongjiwei1996@outlook.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
Co-authored-by: wjHuang <huangwenjun1997@gmail.com>
2022-07-20 21:20:07 +08:00
63f5d286ff log-backup: add version check for pitr (#36268)
close pingcap/tidb#36267
2022-07-18 23:27:07 +08:00
5eac3b6999 log-backup: check timezone when using datatime format ts string (#35811)
ref pingcap/tidb#31975
2022-06-29 17:20:40 +08:00
c6fe032e99 br: support PiTR feature (#34409)
ref pingcap/tidb#29501
2022-05-12 17:08:35 +08:00
f88dab92f9 util: use generics to implement max/min (#34197)
close pingcap/tidb#34247
2022-05-06 10:08:57 +08:00
2578630108 br: enhance the error infomation (#33882)
close pingcap/tidb#33598
2022-04-21 21:54:04 +08:00
8f8dfafc10 br: Fix new collaction enable check (#33500)
* add glue to get global variables
2022-03-28 21:16:53 +08:00
dcafe8ec71 br: download sst again with no-crypt parameters when met decryption error for downloadSST (#32847)
close pingcap/tidb#32423
2022-03-17 14:00:30 +08:00
e0db77ecd1 br: implement backup & restore policy info (#33007)
close pingcap/tidb#32290
2022-03-15 17:39:53 +08:00
4b48e55ae9 lightning: Add source dir existence check for s3 (#30674) 2021-12-14 20:04:35 +08:00