Commit Graph

76 Commits

Author SHA1 Message Date
7746b6b70a test: fix incorrect regexp pattern during migrating test (#30080) 2021-12-02 05:51:53 +08:00
1b6ffe95c2 lightning: avoid retry write rows and record duplicate rows in tidb backend (#30179) 2021-11-29 20:03:52 +08:00
df113a1242 unistore: get/batchGet/scan support read-through-lock (#29898) 2021-11-29 14:37:51 +08:00
f45a8f0511 br : migrate test-infra to testify for lightning/backend/importer (#30073) 2021-11-24 14:49:50 +08:00
c68791566d lightning: let ignore columns be compatible with tidb backend (#27850) 2021-11-23 19:11:50 +08:00
f7ff198c84 br : migrate test-infra to testify for lightning/backend/tidb/tidb_test.go (#30042) 2021-11-23 18:11:50 +08:00
e482122d4e br : migrate test-infra to testify for lightning/backend/backend_test (#30009) 2021-11-22 23:11:50 +08:00
b93ce18982 lightning: fix log doesn't output to stdout when passing --log-file="-" (#29896) 2021-11-19 11:19:48 +08:00
ab70e7a3d9 lightning: add unit test for lightning check multi ingest (#28959) 2021-11-19 10:55:47 +08:00
4a7aad98a1 lightning: add a precheck for csv header (#27819) 2021-11-18 13:53:48 +08:00
3a2c8c43b8 lightning: split engine logic from local.go (#29869) 2021-11-18 11:01:48 +08:00
c6f4cb5cb3 br: fix DATA RACE in the TestRunServer (#29836) 2021-11-16 17:33:08 +08:00
155c93eece lightning: fix the bug where type error row data was always empty (#29537) 2021-11-15 18:39:06 +08:00
5992cdab6d lightning: fix init error-manager (#29768) 2021-11-15 18:27:07 +08:00
4324e4e95e lightning: avoid init error manager if not needed (#29748) 2021-11-14 14:15:05 +08:00
31a4f6abbb lightning: make charset setting case-insensitive (#29713)
* make charset case-insensitive

* fmt code

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
2021-11-12 13:55:04 +08:00
b266d7d574 util: add WaitGroupWrapper (#29614) 2021-11-12 02:27:05 +08:00
6512fe5a49 lightning: fix checksum mismatch when parallel import data (#29695) 2021-11-11 19:15:05 +08:00
96a9ce1b14 lightning: do not shadow checksum error (#29626) 2021-11-10 11:07:05 +08:00
ffd6ee6583 pkg/lightning: reduce cpu&memory usage when duplicate-resolution is enabled (#29555)
* bug fix

* set default resolution algorithm to 'none'

* remove switchToNormalMode

* Update br/pkg/lightning/restore/table_restore.go

Co-authored-by: glorv <glorvs@163.com>

Co-authored-by: glorv <glorvs@163.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
2021-11-09 13:58:03 +08:00
3537d7e5af lightning,dumpling: make tidb version check more compitible (#29500) 2021-11-07 16:55:04 +08:00
b74c13a898 *: run go fmt in go1.17 (#28784) 2021-11-05 12:33:03 +08:00
8368eeed24 *: remove useless parameter (#29392) 2021-11-04 12:39:02 +08:00
dc2d5a4f45 lightning: fix duplicate check panic (#29338) 2021-11-02 22:48:51 +08:00
55a38a1c76 *: tiny clean up the RecordSet NewChunk() method (#29232) 2021-11-02 20:02:51 +08:00
c9a516981f pkg/lightning: refine config for duplicate-resolution and pause GC when duplicate-resolution is enabled (#29249) 2021-11-02 11:56:52 +08:00
bdc9987370 lightning: fix in-correctly check for session variable (#29291) 2021-11-01 19:36:52 +08:00
6a62f9d18c br/lightning: fix test lightning_duplicate_detection (#29225) 2021-10-29 13:04:49 +08:00
e60e170ee1 br/lightning: fix the integration test cases (#29062) 2021-10-27 19:02:49 +08:00
8f192f0e54 br: Set check-static enable for br (#27642) 2021-10-27 18:22:49 +08:00
5b5020d73d lightning: migrate test-infra to tesify for lightning_test.go (#29077) 2021-10-26 22:58:48 +08:00
2dc09885bb lightning: support duplicate resolution by deleting all offending entries (#28464) 2021-10-25 15:26:48 +08:00
efa94595c0 *: enable golangci-lint unconvert (#28929) 2021-10-23 21:28:47 +08:00
eca2dbb681 lightning: let session vars in config also impact kv encoder (#28877) 2021-10-21 13:32:45 +08:00
412dd4f763 parser: refactor Parse() interface to make it extensible (#28975) 2021-10-21 10:56:45 +08:00
853a2447f0 restore: add rowserrcheck to golangci-lint.yml and fix lint warning (#28966) 2021-10-21 03:46:46 +08:00
f739a53da3 br: optimizing retry-able function (#28434) 2021-10-14 10:11:27 +08:00
505299fe9c lightning: move RemoveCheckpoint and CleanupEngine from ctl/main.go (#28769) 2021-10-13 14:33:27 +08:00
xhe
94e30df8e2 parser: fix compiling, add unit tests (#28700) 2021-10-11 23:15:25 +08:00
7e184f1d01 lightning: support dm set session variables and remove checkpoint. (#28471) 2021-10-11 15:13:25 +08:00
dbfc28239b lightning_web: update npm (#28702) 2021-10-11 14:25:25 +08:00
267a3b8162 executor: add auto id allocator execution runtime stats (#28013) 2021-10-08 17:35:24 +08:00
5f797c3f24 lightning: support keep checkpoint data after success without changing schema name (#28472) 2021-09-30 19:08:47 +08:00
b2dd60d3c4 br: make lightning tidb backend compatible with mysql (#27988) 2021-09-29 15:28:47 +08:00
51ce0ddd16 lightning: fix the bug that the GBK-encoded separator could not be parsed normally (#28293) 2021-09-28 11:26:49 +08:00
1cd932c310 lightning: implement max-error reporting for type conversion and unique key conflict errors (#27181) 2021-09-27 10:56:47 +08:00
4ba43a0d4c br: Fix typo in br (#28312) 2021-09-25 17:06:46 +08:00
a088e908ff lightning: fix tidb-backend estimate data source (#28290) 2021-09-23 18:52:46 +08:00
76368e9e4a br: refactoring code - remove unnecessary variables (#28158) 2021-09-23 16:26:46 +08:00
1e8b9337e3 lightning: do not output pre-check info when disable check-requirements (#27934) 2021-09-22 19:28:45 +08:00