Commit Graph

39 Commits

Author SHA1 Message Date
58505468e2 disttask: fix flaky multi task test (#49040)
close pingcap/tidb#49039
2023-11-30 08:53:57 +00:00
ef02d72873 disttask: subtasks rebalance during task execution (#48306)
close pingcap/tidb#47298, ref pingcap/tidb#48795
2023-11-30 04:26:48 +00:00
a1bb7aa682 disttask: test scheduler with real store and refine tests (#49000)
ref pingcap/tidb#48795
2023-11-29 14:16:17 +00:00
d4b41beec9 build/linter/util: exclude files according to the key of only_files (#48999)
close pingcap/tidb#48998
2023-11-29 11:17:47 +00:00
86df166bd3 importinto: make cancel wait task done and some fixes (#48928)
close pingcap/tidb#48736
2023-11-29 09:59:48 +00:00
845b20cbc9 disttask: mock dispatcher ext in UT (#48968)
ref pingcap/tidb#48795
2023-11-29 05:20:49 +00:00
3fe327aaff importinto: add more ut (#48948)
ref pingcap/tidb#43820
2023-11-29 03:34:18 +00:00
01d441ef8a disttask: fix keep reporting error when cancel pending task (#48906)
close pingcap/tidb#48890
2023-11-27 06:16:15 +00:00
8eb191303a *: fix grpc client leak bug for AUTO_ID_CACHE=1 tables (#48870)
close pingcap/tidb#48869
2023-11-24 11:25:14 +00:00
9cf638fb08 global sort: merge to one file (#48142)
ref pingcap/tidb#45719
2023-11-23 08:46:12 +00:00
36cb29fd19 disttask: optimize metrics (#48663)
close pingcap/tidb#48592
2023-11-17 17:42:18 +00:00
844ba426b9 disttask: fix subtask finished immediately and mark success when encountering network partition (#48660)
ref pingcap/tidb#46258, close pingcap/tidb#48649
2023-11-17 13:00:48 +00:00
311d8a2957 disttask: add awserr. RequestCanceled as graceful shutdown err (#48639)
close pingcap/tidb#48635
2023-11-16 10:11:17 +00:00
5a458cf136 disttask: fix cancel from grpc mark subtasks as failed (#48339)
close pingcap/tidb#48324
2023-11-08 08:50:20 +00:00
d4618d4a5f disttask: fix wrong plan when filterByRole (#48366)
close pingcap/tidb#48368
2023-11-08 06:23:13 +00:00
8071fc76d9 merge-sort: add context to errgroup, alloc mem by concurrency and refine logs (#48341)
close pingcap/tidb#48367
2023-11-08 03:54:42 +00:00
4e49859bf7 disttask: correct the usage of context (#48343)
close pingcap/tidb#48303
2023-11-07 11:53:11 +00:00
4979e9985c *: share etcd client from domain for autoid allocator (#46647)
close pingcap/tidb#46324
2023-11-06 05:54:40 +00:00
e392766f8e importinto: calculate subtask count for global sort (#48169)
ref pingcap/tidb#47572
2023-11-03 02:59:17 +00:00
028d7b8c7b external: merge overlapping files concurrently (#48073)
close pingcap/tidb#48072
2023-11-02 04:24:37 +00:00
99a4f35146 importinto: optimize write ingest step and fix oom (#48047)
ref pingcap/tidb#47572, close pingcap/tidb#47643
2023-10-31 11:48:38 +00:00
2b18f649e2 disttask/ddl: refine manager error handling (#48095)
ref pingcap/tidb#46258, close pingcap/tidb#48064
2023-10-31 09:27:38 +00:00
fab7825845 *: remove pkg/errors (#48081) 2023-10-31 03:15:37 +00:00
a2edc738de *: move sqlescaping from pkg/util/sqlexec to pkg/util/sqlescape (#48061)
close pingcap/tidb#48055
2023-10-31 01:45:36 +00:00
ca977c5111 disttask: add more retryable error (#48033)
ref pingcap/tidb#46258, close pingcap/tidb#48034
2023-10-27 09:44:04 +00:00
d1fc638f5b *: record end key rather than max key (#47648)
close pingcap/tidb#47535
2023-10-27 08:20:05 +00:00
9224f62ef8 disttask: fix removing meta when met network partition for so long then recover from it (#48005)
close pingcap/tidb#47954
2023-10-27 03:11:05 +00:00
a7366f936d importinto: fix data race in test (#47965)
close pingcap/tidb#47776
2023-10-26 02:23:03 +00:00
18f4a3c119 pkg/disttask: add OWNERS file (#47861)
close pingcap/tidb#47845
2023-10-25 04:57:33 +00:00
caaa35a05e disttask/ddl: refine add index dispatcher for error handling (#47886)
ref pingcap/tidb#46258, close pingcap/tidb#47932
2023-10-23 14:36:33 +00:00
05b50a1ae1 external: pre-alloc mem for writer, use offset to avoid golang slice (#47793)
ref pingcap/tidb#47572
2023-10-23 08:15:32 +00:00
19784796fe importinto: add more logs to external backend, disable parallel sort (#47680)
ref pingcap/tidb#47572
2023-10-17 06:40:58 +00:00
ab4c20ba16 importinto: revert "use one writer for each kv group for all concurrent encoder" (#47666)
ref pingcap/tidb#46704
2023-10-16 14:34:58 +00:00
1cb25c74ed disttask: add sleep when subtask == nil (#47668)
ref pingcap/tidb#46258
2023-10-16 10:54:35 +00:00
46e53cdd31 disttask: cleanup framework meta when nodes shutdown (#47536)
ref pingcap/tidb#46258
2023-10-16 10:16:58 +00:00
eafb78a9c6 disttask: use merge sort meta (#47599)
ref pingcap/tidb#45719
2023-10-16 07:46:59 +00:00
b3dcf596b2 disttask: define TaskType/TaskState/Step instead of string/int64 (#47611)
ref pingcap/tidb#46258
2023-10-16 06:12:27 +00:00
9504d83392 disttask: more log (#47575)
ref pingcap/tidb#46258
2023-10-14 03:55:56 +00:00
318e82bbb7 *: move most of the packages to 'pkg' (#47123)
ref pingcap/tidb#41241
2023-10-13 14:36:26 +00:00