Commit Graph

62 Commits

Author SHA1 Message Date
36a2af4b10 dxf: start scheduler without reserve resource in nextgen (#63205)
ref pingcap/tidb#61702
2025-08-26 12:22:10 +00:00
1fa4788853 dxf: prepare work for schedule status API in nextgen (#63140)
ref pingcap/tidb#61702
2025-08-22 17:21:38 +00:00
aa875ea472 dxf: calculate max_node_count and concurrency for next-gen automatically (#62995)
ref pingcap/tidb#61702
2025-08-22 14:53:38 +00:00
658fa1a43e crossks: add real-tikv tests for info schema sync (#63008)
ref pingcap/tidb#61702
2025-08-19 17:29:50 +00:00
0a704a4bb7 *: collect subtask summary for IMPORT INTO tasks (#61600)
ref pingcap/tidb#61088
2025-07-04 15:12:30 +00:00
8e6d77ae27 dxf: add keyspace name to global task table (#61755)
ref pingcap/tidb#61702
2025-06-18 16:41:39 +00:00
254849d23a ddl, disttask: extend SubtaskSummary and add Collector interface (#61425)
ref pingcap/tidb#61088
2025-06-09 08:12:37 +00:00
dc876e5c51 disttask: gopls modernize (#60795)
close pingcap/tidb#60794
2025-04-28 10:06:26 +00:00
246a241621 dxf: add manual recovery mode (#59641)
ref pingcap/tidb#59640
2025-02-24 09:42:51 +00:00
9099090462 dxf: add tidb_max_dist_task_nodes to specify max node count (#58937)
close pingcap/tidb#58944
2025-02-18 05:12:25 +00:00
baf16005e0 ddl: part 1 of add-index integrating dxf dynamic modify param (#59046)
ref pingcap/tidb#57229, ref pingcap/tidb#57497
2025-02-17 05:19:39 +00:00
27f8ff3840 dxf: use global entry_size_limit when executing SQLs (#59516)
close pingcap/tidb#59506
2025-02-13 11:22:23 +00:00
fbdaef5c0b dxf: optimize querying eligible tasks in task executor manager (#59345)
close pingcap/tidb#59344
2025-02-10 07:33:52 +00:00
cf4d252af8 *: split definition out of variable pkg and refactor some reorg methods (#59146)
ref pingcap/tidb#57229, ref pingcap/tidb#57497
2025-01-23 09:24:44 +00:00
71e70af1b1 dxf: merge loops in Run and RunStep of task executor (#57944)
ref pingcap/tidb#57497
2024-12-09 07:32:51 +00:00
9ff83eb378 dxf: systable and scheduler change for modify task at runtime (#57557)
ref pingcap/tidb#57497
2024-11-25 07:29:05 +00:00
e4e707d4a7 dxf: refactor failpoints in scheduler (#57566)
ref pingcap/tidb#57497
2024-11-21 03:46:45 +00:00
04c66ee950 ddl: decouple job scheduler from 'ddl' and make it run/exit as owner changes (#53548)
ref pingcap/tidb#53246
2024-05-30 06:37:21 +00:00
d49a6541db test: switch some failpoints to InjectCall (#53588)
ref pingcap/tidb#49008
2024-05-28 04:50:49 +00:00
d626b6adf0 disttask: fix scope can't set different value (#52627)
close pingcap/tidb#52441
2024-04-26 07:19:42 +00:00
5954be7da2 *: let our unit test cover the infoschema v2 code (#52156)
ref pingcap/tidb#50959
2024-04-12 06:58:53 +00:00
40456f36fe *: make package sqlexec not depend on sessionctx anymore (#52031)
close pingcap/tidb#52029
2024-03-25 11:59:17 +00:00
8b02143e2b disttask: skip scheduler take slots for some states (#51022)
ref pingcap/tidb#49008
2024-03-11 07:29:08 +00:00
23190ff540 disttask: enlarge max-concurrent-task and optimize scheduler/executor loop (#51241)
ref pingcap/tidb#49008
2024-02-27 03:32:01 +00:00
a84d57c7da disttask: extract task/subtask base (#51231)
ref pingcap/tidb#49008
2024-02-23 03:55:57 +00:00
e34702f10e disttask: unify task order naming to 'rank' & refactor failpoint (#51136)
ref pingcap/tidb#49008
2024-02-21 03:37:57 +00:00
40f0d1693b disttask: alloc or occupy resource when there are subtasks to run & fix scheduler block task (#50650)
ref pingcap/tidb#49008
2024-01-24 09:16:58 +00:00
d1e6f1c291 disttask: remove revert subtask (#50550)
close pingcap/tidb#50549
2024-01-22 10:25:49 +00:00
595fa7affc disttask: refine taskTable and remove useless code (#50461)
ref pingcap/tidb#48795
2024-01-17 04:06:15 +00:00
caa5539686 disttask: fix executor err handling missing part (#50429)
close pingcap/tidb#50417
2024-01-16 09:55:59 +00:00
720983a20c disttask: merge transfer task/subtask (#50311)
ref pingcap/tidb#48795, close pingcap/tidb#50310
2024-01-15 10:28:46 +00:00
17d65b9317 disttask: change extra running subtasks to pending (#50266)
ref pingcap/tidb#49008
2024-01-12 03:48:25 +00:00
986b00849c disttask: storage.util move to testutil (#50252)
ref pingcap/tidb#48795
2024-01-11 05:28:55 +00:00
e15c517afa import/index: adjust concurrency using cpu-count of managed node depends on use case (#50091)
ref pingcap/tidb#49008
2024-01-10 08:31:54 +00:00
56c619f1f5 disttask: remove useless task state (#50071)
ref pingcap/tidb#48795
2024-01-09 10:49:53 +00:00
386bb43fea disttask: consider node load when schedule/balance subtasks (#49758)
ref pingcap/tidb#49008
2024-01-09 09:06:24 +00:00
f3d4ad5d89 disttask: fix can't reset service_scope (#50108)
close pingcap/tidb#50104
2024-01-08 11:42:30 +00:00
17d24f4cbf disttask: add more tests for task_state (#50090)
ref pingcap/tidb#48795
2024-01-05 05:47:24 +00:00
354c186066 disttask: record endtime for task and subtask (#49872)
close pingcap/tidb#49617
2024-01-04 03:03:33 +00:00
a28d46bf14 test: fix flaky case (#50008)
close pingcap/tidb#49984
2024-01-03 07:03:39 +00:00
237b2c7d50 disttask: fix panic in task executor/scheduler (#49877)
close pingcap/tidb#49869
2024-01-02 12:18:32 +00:00
99f0349bfb disttask: fix failed step is taken as success (#49971)
close pingcap/tidb#49950
2024-01-02 10:57:32 +00:00
3d939d4b6f disttask: init capacity and check concurrency using cpu count of managed node (#49875)
ref pingcap/tidb#49008
2024-01-02 05:36:31 +00:00
8eed0437b3 disttask: split task and subtask state (#49896)
ref pingcap/tidb#48795
2023-12-29 10:04:58 +00:00
84d9bc3b6a importinto: move post-process code from disttask pkg to importer pkg (#49884)
ref pingcap/tidb#49883
2023-12-29 07:27:35 +00:00
fbe232e3f8 disttask: check affacted row before start subtask (#49810)
ref pingcap/tidb#49008
2023-12-29 05:01:58 +00:00
3e9bd47dfd disttask: maintain managed nodes separately (#49623)
ref pingcap/tidb#49008
2023-12-22 08:51:24 +00:00
b2d97523fa test: fix make ut for package disttask/framework (#49480)
close pingcap/tidb#49463
2023-12-15 02:52:49 +00:00
5519033a6c disttask: insert subtasks in batch (#49282)
close pingcap/tidb#49281
2023-12-14 03:15:21 +00:00
3379450cec addindex/disttask: adjust add index task concurrency & add check when submit task (#49403)
ref pingcap/tidb#49008
2023-12-13 08:12:27 +00:00