Commit Graph

162 Commits

Author SHA1 Message Date
ee7f4fc3a5 lightning: enable pre-deduplication and rename the recording table (#45122)
ref pingcap/tidb#41629
2023-07-05 07:38:14 +00:00
fdf8c2cd0d chore: add test lightning_drop_other_tables_halfway to script (#45162) 2023-07-05 05:53:14 +00:00
60784a30d0 lightning: duplicate resolution support to use DELETE to cleanup clustered table (#44799)
ref pingcap/tidb#44723
2023-07-03 09:32:13 +00:00
301357e0af ci(br): support run group br&lightning tests (#44957)
close pingcap/tidb#45006
2023-07-03 06:59:20 +00:00
9eed26b133 lightning: duplicate entry should be "conflict error" in tidb backend (#44581)
close pingcap/tidb#44608
2023-07-03 06:59:13 +00:00
2f1c891d07 lightning: refine fetching table structure error handling (#44801)
close pingcap/tidb#44614
2023-06-29 13:57:49 +00:00
0f20315681 br: pipeline wait tiflash synced (#43726)
close pingcap/tidb#43828
2023-06-27 03:08:34 +00:00
89bf743227 importinto/lightning: do remote checksum via sql (#44803)
close pingcap/tidb#41941, ref pingcap/tidb#44711
2023-06-21 13:41:42 +00:00
1bfdae8e63 lightning: disable pre-deduplication test after we disable the code part (#44812)
ref pingcap/tidb#41629
2023-06-21 04:01:42 +00:00
33236ef1e4 lightning: fix uuid panic when expression index enabled (#44516)
close pingcap/tidb#44497
2023-06-09 07:04:47 +00:00
6bab55cfeb lightning: add error message for pre-deduplication (#44317)
ref pingcap/tidb#41629
2023-06-07 11:09:41 +00:00
bc45bdf3c0 lightning: deprecate send-kv-pairs (#44463)
ref pingcap/tidb#43853
2023-06-07 05:19:41 +00:00
dd54ff29ba lightning: support latin1 source file encoding (#44435)
close pingcap/tidb#44434
2023-06-06 10:13:49 +00:00
cd46add6b3 lightning: implement preprocess duplicate detection (#42647)
ref pingcap/tidb#41629
2023-05-29 04:12:40 +00:00
ca629447df lightning: add send-kv-size to avoid oom when each kv is large on default config (#43870)
close pingcap/tidb#43853
2023-05-18 09:49:36 +00:00
d9243db241 lightning: refactor switch mode for reuse (#43909)
ref pingcap/tidb#42930
2023-05-18 04:03:35 +00:00
853d2b6b75 lightning: fix grep check in integration test (#43821)
ref pingcap/tidb#43820
2023-05-15 10:33:50 +00:00
6043234d98 lightning: add option to control pause scheduler scope (#43762)
close pingcap/tidb#43657
2023-05-15 06:19:17 +00:00
fd438de6a8 br: Fix RangeTree.GetIncompleteRange when the rangeTree is empty. (#37086)
ref pingcap/tidb#37085
2023-05-08 10:22:58 +00:00
61e92b1d2d fix(dos2unix): use unix eol(LF) rather than dos(CRLF) (#43603)
close pingcap/tidb#43602
2023-05-08 07:26:57 +00:00
6e32106b93 br/lighting: add multi-valued index test (#41977)
close pingcap/tidb#42037
2023-05-06 10:06:56 +00:00
0bfe3e72d3 Lightning: adjust precheck and refine error message (#42258)
close pingcap/tidb#42835, ref pingcap/tidb#42836
2023-04-19 01:07:19 +08:00
ab086ae3cf lightning: refactor checksum to reuse (#43082)
ref pingcap/tidb#42930
2023-04-18 21:05:19 +08:00
6122ca307e linter: support lll to avoid too long line (#42718)
close pingcap/tidb#42570
2023-04-12 23:23:01 +08:00
03d19d2a2a lightning: add test about multi-rocksdb (#42243)
ref pingcap/tidb#42359
2023-03-21 15:44:41 +08:00
07178874b6 lightning: use sql to add secondary indexes (#42136)
close pingcap/tidb#42132
2023-03-17 18:34:53 +08:00
af60ff8dae br: filter resource group in br full restore (#42267)
ref pingcap/tidb#42266
2023-03-16 15:22:47 +08:00
7b5c65f108 br: add <Code>RequestTimeout</Code> to list of retryable errors (#41759)
close pingcap/tidb#41756
2023-03-15 14:42:39 +08:00
309f3aaf3d ddl: use length to determine if the slice is empty (#41862)
close pingcap/tidb#41559
2023-03-09 17:53:13 +08:00
567152e8f4 ttl, lightning: fix panic when importing TTL table data using lightning (#42014)
close pingcap/tidb#42012
2023-03-08 19:37:21 +08:00
9958a93ae8 lightning: rename and move some struct (#41946)
ref pingcap/tidb#40499
2023-03-06 17:49:33 +08:00
efe553385b log-backup: fix the issue that gc-worker owner cannot check log-backup task existed. (#41824)
close pingcap/tidb#41806
2023-03-02 12:23:09 +08:00
6837bd588d lightning: support auto_random column in composite primary key (#41463)
close pingcap/tidb#41454
2023-02-16 13:40:01 +08:00
532689cb00 lightning: fix panic when user cancel (#41236)
close pingcap/tidb#41235
2023-02-10 11:13:59 +08:00
7f88e73af6 br: skip template system table __TiDB_BR_Temporary_mysql when backup (#41000)
close pingcap/tidb#40797
2023-02-09 23:53:59 +08:00
b359ee77e7 lightning: support compression when sending kv pairs to tikv (#41164)
close pingcap/tidb#41163
2023-02-09 18:06:23 +08:00
9cb4c488fc br: Support backup replica read (#40899)
ref pingcap/tidb#40898
2023-02-09 18:06:08 +08:00
7255868dae lightning: do resolve conflict job when other lightning has local dupes (#41157)
close pingcap/tidb#40923
2023-02-09 13:24:00 +08:00
d16f4c0ed0 pitr: prevent from restore point to cluster running log backup (#40871)
ref pingcap/tidb#40797
2023-02-09 11:38:00 +08:00
37fce43abd lightning: introduce param to skip CSV header parsing (#41128)
close pingcap/tidb#40839
2023-02-09 00:03:58 +08:00
4ccf60343e br: fix br ci (#41149) 2023-02-08 10:09:58 +08:00
72ee34510f lightning: fix panic when nextKey twice (#40959)
close pingcap/tidb#40934
2023-02-02 22:47:56 +08:00
0d47a5e85c lightning: enable setting conflict max-error (#40874)
ref pingcap/tidb#40743
2023-01-31 13:45:55 +08:00
e11cbec12b lightning: support data files with bom header (#40813)
close pingcap/tidb#40744
2023-01-30 01:13:54 +08:00
06e2b29551 lightning: check hasDupe and tableID when resolve duplicate rows (#40696)
close pingcap/tidb#40657
2023-01-28 14:17:53 +08:00
eef8438e14 lightning: not set pos to end if fail to ReadUntil (#40232)
* fix: not set pos to end

* add it

* fix: return to prevPos

* fix: return err

* fix

* fix

* fix it

* fix

* fix: not exit when err

Co-authored-by: Zak Zhao <57036248+joccau@users.noreply.github.com>
2023-01-12 12:20:33 +08:00
875c002ad5 lightning: add foreign key test (#40103)
close pingcap/tidb#40226
2022-12-29 21:34:19 +08:00
cf34941401 ttl: make the TTL compatible with dumpling, lightning and BR (#40216)
close pingcap/tidb#40215
2022-12-29 16:30:21 +08:00
1ace97069e lightning: disable foreign key checks (#40032)
close pingcap/tidb#40027
2022-12-26 10:58:15 +08:00
4b98439f33 lightning: record network usage (#39673)
ref pingcap/tidb#39674
2022-12-08 18:24:05 +08:00