Commit Graph

597 Commits

Author SHA1 Message Date
17d70c1b7e lightning/parser: support STARTING BY (#40821)
ref pingcap/tidb#40499
2023-01-30 11:23:54 +08:00
e11cbec12b lightning: support data files with bom header (#40813)
close pingcap/tidb#40744
2023-01-30 01:13:54 +08:00
5048568825 lightning: support timeout for tikv/tidb retry (#40805)
ref pingcap/tidb#33714
2023-01-29 19:11:54 +08:00
0bcb859280 br: fix checkpoint panic when BR executes gain (#40706)
close pingcap/tidb#40704
2023-01-29 16:41:54 +08:00
8e963505c1 bazel: upgrade rules_go to prepare for Go 1.20 (#40776)
close pingcap/tidb#40798
2023-01-29 15:35:54 +08:00
06e2b29551 lightning: check hasDupe and tableID when resolve duplicate rows (#40696)
close pingcap/tidb#40657
2023-01-28 14:17:53 +08:00
76153ab9ff lint: update config of deprecated linter (#40763)
ref pingcap/tidb#40786
2023-01-28 11:57:53 +08:00
6ebbe64b5b br: split regions before pitr restore (#39941)
close pingcap/tidb#37207
2023-01-22 11:09:50 +08:00
bbd1995ea3 ddl, lightning: support detecting duplicate keys in different batch (#40701)
close pingcap/tidb#40698
2023-01-19 14:29:49 +08:00
9b64d81bcf pitr: move step iterate backup files into worker pool (#40577)
close pingcap/tidb#40576
2023-01-19 12:01:49 +08:00
dc30a5b7b3 *: LOAD DATA support load one file from s3 and other OSS (#40489)
ref pingcap/tidb#40499
2023-01-18 15:27:51 +08:00
89de573660 BR: add lock with ttl for checkpoint backup (#40563)
close pingcap/tidb#40561
2023-01-18 14:35:51 +08:00
1083f79f51 handle: use separete ctx for initStats (#40604)
close pingcap/tidb#40548
2023-01-16 18:23:48 +08:00
73f2360527 log-backup: remove k-v record in etcd when stop log-backup task (#40431)
close pingcap/tidb#40403
2023-01-14 18:03:47 +08:00
aef752a407 lightning: continue this region round and retry on next round when TiKV is busy (#40278)
* finish

Signed-off-by: lance6716 <lance6716@gmail.com>

* fix some comments

Signed-off-by: lance6716 <lance6716@gmail.com>

* fix CI

Signed-off-by: lance6716 <lance6716@gmail.com>

* address comment

Signed-off-by: lance6716 <lance6716@gmail.com>

* fix errdoc

Signed-off-by: lance6716 <lance6716@gmail.com>

* revert a change

Signed-off-by: lance6716 <lance6716@gmail.com>

* add lightning debug log

Signed-off-by: lance6716 <lance6716@gmail.com>

* fix bug

Signed-off-by: lance6716 <lance6716@gmail.com>

* address comment

Signed-off-by: lance6716 <lance6716@gmail.com>

Signed-off-by: lance6716 <lance6716@gmail.com>
Co-authored-by: Weizhen Wang <wangweizhen@pingcap.com>
2023-01-12 15:24:33 +08:00
fb29c807ce lightning: fix risk of OOM (#40443)
* fix: check buf size

* fix: terminate too long buf

* add ut

* fix dev

* local var

* fix ut

* fix lint

* fix lint

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
2023-01-12 13:48:33 +08:00
eef8438e14 lightning: not set pos to end if fail to ReadUntil (#40232)
* fix: not set pos to end

* add it

* fix: return to prevPos

* fix: return err

* fix

* fix

* fix it

* fix

* fix: not exit when err

Co-authored-by: Zak Zhao <57036248+joccau@users.noreply.github.com>
2023-01-12 12:20:33 +08:00
f88714d1af br: need init MDL variable when GetDomain (#40512)
Signed-off-by: joccau <zak.zhao@pingcap.com>

Signed-off-by: joccau <zak.zhao@pingcap.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
2023-01-11 21:00:33 +08:00
e3f8d9805e lightning: fix codecPDClient (#40496)
Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>

Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
Co-authored-by: Weizhen Wang <wangweizhen@pingcap.com>
2023-01-11 17:36:33 +08:00
be31b6cba7 br: reset cloud_admin and root after ebs restoration (#39986)
close pingcap/tidb#40418
2023-01-10 12:26:22 +08:00
199bb382a1 br: support reset_tiflash after ebs restoration (#40124)
close pingcap/tidb#40208
2023-01-09 16:54:23 +08:00
b477b1c946 *: Supports create/alter/drop resource group (#40188)
ref pingcap/tidb#38825
2023-01-09 13:44:22 +08:00
6477d2b907 log-backup: should set default value gc.ratio-threshold after pitr finished (#40141)
close pingcap/tidb#40185
2023-01-06 15:04:20 +08:00
bf2cc45206 br: add errch buf for checkpoint (#40166)
close pingcap/tidb#40165
2023-01-04 19:52:20 +08:00
00604eb19d br: fix bootstrap domain (#40307)
close pingcap/tidb#40281
2023-01-04 19:36:20 +08:00
47080d98d1 lightning: check peers write stall when switch-mode is disabled (#40228)
close pingcap/tidb#40163
2022-12-30 19:58:17 +08:00
f7de8beede *: remove the support of the old ddl framework (#39684)
ref pingcap/tidb#39824
2022-12-30 18:30:18 +08:00
1b8a2338ab restore: set config value to default when failed to get config from tikv (#36051)
close pingcap/tidb#36053
2022-12-30 15:08:17 +08:00
cf34941401 ttl: make the TTL compatible with dumpling, lightning and BR (#40216)
close pingcap/tidb#40215
2022-12-29 16:30:21 +08:00
79ab2b92a0 restore: Fix prealloc (#40176)
close pingcap/tidb#40177
2022-12-26 22:38:15 +08:00
388364d5f5 br: recursive split batch create table when restore failure on batch create table (#39829)
close pingcap/tidb#39029
2022-12-26 11:58:15 +08:00
1ace97069e lightning: disable foreign key checks (#40032)
close pingcap/tidb#40027
2022-12-26 10:58:15 +08:00
ae58fa1c8b expression: introduce cast(... as ... array) in expression index (#39992)
close pingcap/tidb#40045
2022-12-20 12:56:54 +08:00
0c1808245b *: support sample for compressed files for adjustment (#39680)
ref pingcap/tidb#38514
2022-12-19 15:42:54 +08:00
d2393e3e4d fix: ebs volume restore failure since incorrect unfinished volumes (#39975)
close pingcap/tidb#39974
2022-12-16 21:22:53 +08:00
e5c0b6b101 br: fix err overwrite (#39935)
close pingcap/tidb#39934
2022-12-14 18:50:52 +08:00
dd42b72242 br: make more sane variables of ticking timeout (#39887)
close pingcap/tidb#39620
2022-12-14 16:00:51 +08:00
90a739821c ddl: close lightning writers after the import is complete (#39879)
close pingcap/tidb#39800
2022-12-13 20:10:51 +08:00
Bin
33e5baade9 fix: update ioutil.readall (#39850) 2022-12-13 10:24:51 +08:00
f18be893f3 br: add retry for azblob read file (#39650)
close pingcap/tidb#39649
2022-12-09 17:07:42 +08:00
92836501a7 lightning: update to new TLS API of SQL driver (#39623) 2022-12-09 16:20:04 +08:00
0e1eef3b54 log-backup: PITR do not support batch if skip requirement check (#39770)
close pingcap/tidb#39722
2022-12-09 10:18:04 +08:00
4b98439f33 lightning: record network usage (#39673)
ref pingcap/tidb#39674
2022-12-08 18:24:05 +08:00
c21dc26351 log-backup: set gc disable when restore log (#39729)
close pingcap/tidb#39602
2022-12-08 16:02:24 +08:00
98d84abee8 br/streamhelper: fix subscribe error (#39689)
close pingcap/tidb#39688
2022-12-07 22:46:04 +08:00
edabc32d92 br: fix integration test cases with --with-sys-table (#39655)
close pingcap/tidb#39692
2022-12-07 18:48:05 +08:00
d7d059cea1 br/streamhelper: added timeout for ticking (#39625)
close pingcap/tidb#39620
2022-12-07 14:40:04 +08:00
032e6fd3f3 br: fix getting region failure with other s3 compatible storage (#39653)
close pingcap/tidb#39648
2022-12-07 11:18:07 +08:00
8b76580bf5 backup: config adjust for checkpoint (#39634)
close pingcap/tidb#39632
2022-12-06 17:32:03 +08:00
35819ee43f import: check table empty skips using index (#39604)
close pingcap/tidb#39477
2022-12-05 21:40:03 +08:00