Commit Graph

66 Commits

Author SHA1 Message Date
0bcb859280 br: fix checkpoint panic when BR executes gain (#40706)
close pingcap/tidb#40704
2023-01-29 16:41:54 +08:00
76153ab9ff lint: update config of deprecated linter (#40763)
ref pingcap/tidb#40786
2023-01-28 11:57:53 +08:00
89de573660 BR: add lock with ttl for checkpoint backup (#40563)
close pingcap/tidb#40561
2023-01-18 14:35:51 +08:00
1de2bf6e54 Backup: add checkpoint (#38648)
close pingcap/tidb#38647
2022-11-30 12:52:00 +08:00
f9a6e47192 planner: support push part of order property down to the partition table (#36108)
ref pingcap/tidb#26166
2022-11-29 13:51:59 +08:00
6477e7521f br: remove error when create the backup client to make caller easier (#39303)
close pingcap/tidb#39302
2022-11-22 12:27:57 +08:00
f9c7bbccec *: authentication using tidb_auth_token plugin (#38773)
close pingcap/tidb#38504
2022-11-04 02:54:00 +08:00
636a363f7b server: support dumpPartitionStats param for statsHandler (#38190)
ref pingcap/tidb#37977
2022-10-28 12:37:58 +08:00
c4316fbc74 log-backup: support restart task after stop a log-backup task (#38381)
close pingcap/tidb#38382
2022-10-27 13:19:57 +08:00
1eeedacc60 *: remove goleak's ignoring opencensus top function (#37908) 2022-09-19 12:29:00 +08:00
181fb8e905 br: merge feature branch of ebs-based-br (#37297)
ref pingcap/tidb#35306
2022-09-18 20:37:00 +08:00
7ca368650c br: add a test for cached table (#31191)
close pingcap/tidb#31190
2022-08-12 18:08:51 +08:00
5b4a82dbf4 *: upgrade bazel timeout (#36776) 2022-08-02 11:10:05 +08:00
xhe
9a16a8dd19 *: replace defer clean with t.Cleanup (#36722)
close pingcap/tidb#36721
2022-08-01 17:34:05 +08:00
158ba1a1d9 *: enable part revive for all code (#36703) 2022-08-01 14:02:06 +08:00
1fe666add6 *: upgrade staticcheck (#36745) 2022-08-01 11:58:05 +08:00
be102a3d55 *: fix timestamp conversion (#36705)
close pingcap/tidb#36707
2022-07-29 19:05:12 +08:00
56dadec862 ddl: Preparation for add index acceleration (#36085)
ref pingcap/tidb#35983
2022-07-28 16:43:11 +08:00
624171abcc *: decouple planner/core from ddl package (#36608) 2022-07-27 21:15:12 +08:00
f22098cc5d br: don't encode key of raw kv. (#36491)
ref pingcap/tidb#36490
2022-07-27 19:05:11 +08:00
6ae88c4309 br: use one shot session to close domain ASAP (#36558)
* br: use one shot session to close domain ASAP



Co-authored-by: Ian <ArGregoryIan@gmail.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
2022-07-26 20:15:53 +08:00
bf743a6ca0 *: enable flaky test for all test (#36385)
close pingcap/tidb#36404
2022-07-22 15:35:09 +08:00
ab513bf0ca ddl: support concurrent ddl (#32169)
* 1. init ddl tables

create tidb_ddl_job, tidb_ddl_reorg, tidb_ddl_history tables with raw meta write, these 3 tables is use to replace the ddl job queue and reorg and history hash table

* 2. setup concurrent ddl env and add ddl worker pool

adds the ddl worker pool definition, the ddl job manager will find a job and ship it to a worker in the worker pool.
Also, this commit provides a sessionctx wrapper, only use in ddl relate. it just wraps begin, commit and execute

* 3. add ddl manager to handle ddl job

* 4. reorg handler for concurrent ddl

just implements the partner of the reorg information.

* 5. manage ddl jobs for concurrent ddl

add the partner of add job, delete job and many others related to history job
because many of the functions need a session now, we just change the caller

* 6. add metrics for concurrent ddl

add metrics

* 7. support multiple tables

* 8. fix test

* 9. migrate ddl between table and queue

support switch between the old and new ddl framework, migrate the existing ddl job between queue and table

* 10. check tikv version and set reorg worker count according cpu count

* *: add featuretag on tests

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

* use a determined table id for 3 tables

* remove ctx value

* add GetSchemaVersionWithNonEmptyDiff function

* address tangenta and zimulala comment

* use only one etcd path

* make ActionRenameTable support multi-schema

* reset sql digest to make top sql work correct

* add comment

* fix test

* remove 0 for schema version lock

Co-authored-by: xiongjiwei <xiongjiwei1996@outlook.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
Co-authored-by: wjHuang <huangwenjun1997@gmail.com>
2022-07-20 21:20:07 +08:00
eb3de65c6e Br: refactor log messages (#35760)
close pingcap/tidb#35928
2022-07-08 15:17:03 +08:00
7d37ac1dde br: get bucket region from s3 to enable users not to input s3 region param (#34419)
close pingcap/tidb#34275
2022-07-07 16:11:03 +08:00
3c83cd3106 BR: fix backup ranges total time summary log (#35552)
close pingcap/tidb#35553
2022-06-23 23:12:37 +08:00
2b4a4c6d7a *: support bazel (#33691)
close pingcap/tidb#34083
2022-06-07 15:22:30 +08:00
26df62fef2 test: improve some tests (#35092)
ref pingcap/tidb#32031
2022-06-01 18:50:28 +08:00
7f023bd61b ddl: wrap some functions for ddl job (#34920)
close pingcap/tidb#34919
2022-05-29 15:38:26 +08:00
0f85650aa4 br: add error into retry list when backup (#34930)
close pingcap/tidb#34865
2022-05-26 16:54:47 +08:00
810f8ae12e *: remove pingcap/check dependency (#34734)
close pingcap/tidb#28328
2022-05-17 21:38:37 +08:00
c6fe032e99 br: support PiTR feature (#34409)
ref pingcap/tidb#29501
2022-05-12 17:08:35 +08:00
9339955f0d backup: backup empty databases (#34385)
close pingcap/tidb#33866
2022-05-07 18:30:32 +08:00
167d4ebc61 backup: retry on internal error and make more errors can be retried (#34352)
close pingcap/tidb#34350
2022-05-05 15:20:55 +08:00
6517ee3fb6 *: migrate tidb-tools (#33296)
close pingcap/tidb#32999
2022-03-25 19:08:35 +08:00
6b2e37f18c gomod: update dependency for golang1.18 (#33053)
ref pingcap/tidb#32874
2022-03-17 15:10:31 +08:00
dcafe8ec71 br: download sst again with no-crypt parameters when met decryption error for downloadSST (#32847)
close pingcap/tidb#32423
2022-03-17 14:00:30 +08:00
33e76327ab br: Consider partition table's placement for br (#33139)
ref pingcap/tidb#32290
2022-03-16 20:05:52 +08:00
e0db77ecd1 br: implement backup & restore policy info (#33007)
close pingcap/tidb#32290
2022-03-15 17:39:53 +08:00
4e69c0705e br: Fix backup rawkv failure (#32612)
close pingcap/tidb#32607
2022-03-03 15:05:47 +08:00
8948c1c92b br: give error message to user clearly when fail to backup without local storage directory/file (#30431)
close pingcap/tidb#30419
2022-03-01 20:27:45 +08:00
5988d0b2f4 *: update client-go and etcd version (#32457)
close pingcap/tidb#32456
2022-02-22 11:11:43 +08:00
7c56706622 *: support predeclared golangci-lint linter (#32066)
close pingcap/tidb#30733
2022-02-14 10:35:38 +08:00
74261e1432 backup, restore: Write correct api-version in rawkv backup/restore (#31122)
close pingcap/tidb#31121
2022-02-08 23:07:36 +08:00
5bb4be2257 br/pkg/backup: fix goroutine leak in test (#31962)
close pingcap/tidb#31961
2022-01-26 22:58:45 +08:00
bb42313355 br: refactor sendBackup (#31264)
ref pingcap/tidb#31374
2022-01-25 18:22:11 +08:00
701f126b27 br/pkg/backup: migrate test-infra to testify (#31288)
close pingcap/tidb#28164
2022-01-23 13:19:46 +08:00
50520e3985 lint: fix lint errors in pkg br. (#31041)
close pingcap/tidb#31358
2022-01-06 10:54:36 +08:00
e12342b494 br: error log optimization (#29640)
close pingcap/tidb#27015
2021-12-21 18:09:46 +08:00
4b48e55ae9 lightning: Add source dir existence check for s3 (#30674) 2021-12-14 20:04:35 +08:00