Commit Graph

204 Commits

Author SHA1 Message Date
b08a92082f disttask: fix task metrics for distributed execute framework panel (#59940)
ref pingcap/tidb#47017
2025-03-25 17:31:28 +00:00
d51e00e5bb globalsort: reduce number of SST ingested into TiKV (#59870) (#60045)
ref pingcap/tidb#51734, close pingcap/tidb#59962
2025-03-24 12:22:43 +00:00
80d6b5683c infosync: refactor server config into dynamic and static sections (#58473)
close pingcap/tidb#58405
2025-03-18 02:36:52 +00:00
6953d3656c disttask: fix potential block on DXF scheduler (#59945)
close pingcap/tidb#59944
2025-03-14 05:41:34 +00:00
246a241621 dxf: add manual recovery mode (#59641)
ref pingcap/tidb#59640
2025-02-24 09:42:51 +00:00
e2d5d1c017 import-into: support running with 1 thread when using global sort (#58572)
close pingcap/tidb#58680
2025-02-24 06:30:23 +00:00
811be5aaa5 disttask: add local disk space to node resource managed by DXF (#59607)
ref pingcap/tidb#49008
2025-02-20 08:42:09 +00:00
b45a9e840c sysvar: add validation check for tidb_max_dist_task_nodes (#59645)
ref pingcap/tidb#58944
2025-02-20 07:05:27 +00:00
54384b7cbb ddl: modify param at runtime for add-index on DXF local sort (#59343)
ref pingcap/tidb#57229, ref pingcap/tidb#57497
2025-02-19 13:43:09 +00:00
9099090462 dxf: add tidb_max_dist_task_nodes to specify max node count (#58937)
close pingcap/tidb#58944
2025-02-18 05:12:25 +00:00
baf16005e0 ddl: part 1 of add-index integrating dxf dynamic modify param (#59046)
ref pingcap/tidb#57229, ref pingcap/tidb#57497
2025-02-17 05:19:39 +00:00
0c61a62c3d dxf: add integration test for modify param during subtask running (#59472)
ref pingcap/tidb#57497
2025-02-14 10:06:40 +00:00
fc0adfcf28 lightning/global-sort: move membuf pkg from br to lightning (#59550)
ref pingcap/tidb#57275
2025-02-14 09:19:19 +00:00
435a716670 import: fix oom on global sort when the target table has large number of indices (#59518)
close pingcap/tidb#59508
2025-02-14 06:13:19 +00:00
27f8ff3840 dxf: use global entry_size_limit when executing SQLs (#59516)
close pingcap/tidb#59506
2025-02-13 11:22:23 +00:00
fbdaef5c0b dxf: optimize querying eligible tasks in task executor manager (#59345)
close pingcap/tidb#59344
2025-02-10 07:33:52 +00:00
cf4d252af8 *: split definition out of variable pkg and refactor some reorg methods (#59146)
ref pingcap/tidb#57229, ref pingcap/tidb#57497
2025-01-23 09:24:44 +00:00
fcdbbd49bb DXF: check the cause of subtaskCtx to decide the subtask state (#58893)
close pingcap/tidb#58827
2025-01-22 16:20:26 +00:00
b9b9787f17 dxf: support param modify for active running subtask (#58935)
ref pingcap/tidb#57497
2025-01-22 04:02:05 +00:00
62f9a39346 dxf: support modify app param on scheduler part (#58990)
ref pingcap/tidb#57497
2025-01-20 09:25:14 +00:00
6cceaba4b1 dxf: add comment to explain current schedule strategy (#59033)
ref pingcap/tidb#57497
2025-01-20 06:56:23 +00:00
f7c94c80c5 ddl/dxf: some rename and merge explicit impl of scheduler.Extension with scheduler (#59013)
ref pingcap/tidb#57497
2025-01-17 09:55:29 +00:00
efb579450a dxf: add an example application (#58938)
ref pingcap/tidb#57497
2025-01-16 11:01:01 +00:00
87867ebf5f ddl/dxf: refine internal docs and some log refactor (#58881)
ref pingcap/tidb#57497
2025-01-14 06:14:08 +00:00
b2f2faaa95 parser: move 'model' to 'ast' pkg (#58704)
ref pingcap/tidb#54436
2025-01-07 10:20:04 +00:00
c199ddfcdf disttask: cancel subtask context if scheduled away (#58615)
close pingcap/tidb#58450
2025-01-07 04:12:37 +00:00
bf939fa79a *: don't handle live updates of column size (#58596)
close pingcap/tidb#58595
2025-01-06 19:42:49 +00:00
293077b761 dxf: support modify concurrency/task-specific-param for later subtask at task executor part (#58144)
ref pingcap/tidb#57497
2024-12-27 13:05:21 +00:00
14a469ab9d *: upgrade the PD client to adopt the latest refactor (#58440)
ref pingcap/tidb#58439
2024-12-23 10:36:12 +00:00
588b273f67 global sort: add boundaries to split keys when generating plan (#58323)
close pingcap/tidb#58267
2024-12-17 15:04:30 +00:00
e62963d2c2 logutil/dxf: add a error verbose logger, and use it in DXF (#58185)
ref pingcap/tidb#57497
2024-12-12 11:41:06 +00:00
3adc71cc71 dxf: merge OnFinished into RunSubtask (#58098)
ref pingcap/tidb#57497
2024-12-10 09:26:21 +00:00
71e70af1b1 dxf: merge loops in Run and RunStep of task executor (#57944)
ref pingcap/tidb#57497
2024-12-09 07:32:51 +00:00
3002c1c1c7 dxf: refactor error handling in task executor (#57837)
ref pingcap/tidb#57497
2024-12-03 11:09:38 +00:00
e631ba8a9b dxf: handle modifying task concurrency in scheduler (#57673)
ref pingcap/tidb#57497
2024-12-02 14:05:17 +00:00
2950725ec1 dxf: fix flaky scope test (#57796)
ref pingcap/tidb#57497
2024-11-28 12:17:39 +00:00
d02d48acd9 dxf: refactor failpoints on task executor (#57680)
ref pingcap/tidb#57497
2024-11-26 03:49:00 +00:00
9ff83eb378 dxf: systable and scheduler change for modify task at runtime (#57557)
ref pingcap/tidb#57497
2024-11-25 07:29:05 +00:00
2f8d1f6f68 ddl: dynamically adjusting the concurrency and batch size of reorganization job (#57468)
ref pingcap/tidb#57229
2024-11-21 12:35:02 +00:00
e4e707d4a7 dxf: refactor failpoints in scheduler (#57566)
ref pingcap/tidb#57497
2024-11-21 03:46:45 +00:00
31c068c719 dxf: reduce indent of scheduleTask and some log refactor (#57501)
ref pingcap/tidb#57497
2024-11-20 08:44:02 +00:00
ecca340037 lightning: fix id too large after parallel import (#57398)
close pingcap/tidb#56814
2024-11-19 11:03:32 +00:00
a4faee2ee5 store: make storage type a enum (#57276)
ref pingcap/tidb#57275
2024-11-12 04:44:58 +00:00
56e7093a9d ddl: use a global owner manager instance for DDL, to avoid owner change (#57179)
ref pingcap/tidb#57185
2024-11-11 05:12:48 +00:00
559f218988 dxf: fix the task num count incorrectly (#57127)
close pingcap/tidb#57172
2024-11-07 13:12:21 +00:00
7e491c3357 dxf: get task from both mysql.tidb_global_task and mysql.tidb_global_task_history (#57063)
close pingcap/tidb#56942
2024-11-05 05:26:36 +00:00
a00ba68b29 disttask: make task executor onError print error's stack. (#56618)
close pingcap/tidb#56014
2024-10-22 06:21:12 +00:00
f0b3bf0332 *: replace reflect.TypeOf((*T)(nil)).Elem() with reflect.TypeFor[T]() (#56598) 2024-10-12 04:21:17 +00:00
c43d0589be *: replace exportloopref with copyloopvar (#56503)
close pingcap/tidb#56509
2024-10-09 12:12:43 +00:00
3578e29613 lightning: job range keys and region split keys are 2 concepts (part 2) (#55759)
close pingcap/tidb#55374
2024-09-05 01:31:35 +00:00