Commit Graph

217 Commits

Author SHA1 Message Date
c71ebaf73e global-sort: accelerate adding unique index using global sort (#60848)
ref pingcap/tidb#60621
2025-05-12 09:25:58 +00:00
4ad4493246 ddl/dxf: modify param at runtime for add-index with global sort (part 1) (#60677)
ref pingcap/tidb#57229, ref pingcap/tidb#57497
2025-04-29 05:22:19 +00:00
dc876e5c51 disttask: gopls modernize (#60795)
close pingcap/tidb#60794
2025-04-28 10:06:26 +00:00
0200e5d634 localbackend: merge feature branch next-gen (#60879)
ref pingcap/tidb#60418
2025-04-28 05:18:58 +00:00
5faa7496ef ddl: fix OOM issue when adding many indexes (#60676)
ref pingcap/tidb#59508
2025-04-25 08:25:35 +00:00
531eb80580 *: add a RetryableErrorCount metric to trace error during DDL (#60485)
ref pingcap/tidb#60484
2025-04-22 14:23:02 +00:00
08d20b8ee2 IMPORTINTO: close store inside LoadDataController (#60633)
close pingcap/tidb#60352
2025-04-18 11:18:52 +00:00
813b32f4df import: add a hidden option to enable manual recover for debug purpose (#59674)
ref pingcap/tidb#59640
2025-04-08 08:46:48 +00:00
1cd39430e1 *: inject random error for DXF (part2) (#60354)
ref pingcap/tidb#60277
2025-04-08 06:18:54 +00:00
f6f5703e43 importinto: fix error not returned when writing external meta (#60350)
close pingcap/tidb#60083
2025-04-01 09:04:15 +00:00
6e5a6d2a93 importinto: check retryable schedule error (#60320)
ref pingcap/tidb#59754
2025-03-28 11:50:24 +00:00
e2d1d89c9e *: inject random error for DXF (part1) (#60278)
ref pingcap/tidb#60277
2025-03-27 12:38:13 +00:00
ed8b869a60 globalsort: write sorted kv meta to external storage (#59966)
close pingcap/tidb#60083
2025-03-26 09:39:54 +00:00
b08a92082f disttask: fix task metrics for distributed execute framework panel (#59940)
ref pingcap/tidb#47017
2025-03-25 17:31:28 +00:00
d51e00e5bb globalsort: reduce number of SST ingested into TiKV (#59870) (#60045)
ref pingcap/tidb#51734, close pingcap/tidb#59962
2025-03-24 12:22:43 +00:00
80d6b5683c infosync: refactor server config into dynamic and static sections (#58473)
close pingcap/tidb#58405
2025-03-18 02:36:52 +00:00
6953d3656c disttask: fix potential block on DXF scheduler (#59945)
close pingcap/tidb#59944
2025-03-14 05:41:34 +00:00
246a241621 dxf: add manual recovery mode (#59641)
ref pingcap/tidb#59640
2025-02-24 09:42:51 +00:00
e2d5d1c017 import-into: support running with 1 thread when using global sort (#58572)
close pingcap/tidb#58680
2025-02-24 06:30:23 +00:00
811be5aaa5 disttask: add local disk space to node resource managed by DXF (#59607)
ref pingcap/tidb#49008
2025-02-20 08:42:09 +00:00
b45a9e840c sysvar: add validation check for tidb_max_dist_task_nodes (#59645)
ref pingcap/tidb#58944
2025-02-20 07:05:27 +00:00
54384b7cbb ddl: modify param at runtime for add-index on DXF local sort (#59343)
ref pingcap/tidb#57229, ref pingcap/tidb#57497
2025-02-19 13:43:09 +00:00
9099090462 dxf: add tidb_max_dist_task_nodes to specify max node count (#58937)
close pingcap/tidb#58944
2025-02-18 05:12:25 +00:00
baf16005e0 ddl: part 1 of add-index integrating dxf dynamic modify param (#59046)
ref pingcap/tidb#57229, ref pingcap/tidb#57497
2025-02-17 05:19:39 +00:00
0c61a62c3d dxf: add integration test for modify param during subtask running (#59472)
ref pingcap/tidb#57497
2025-02-14 10:06:40 +00:00
fc0adfcf28 lightning/global-sort: move membuf pkg from br to lightning (#59550)
ref pingcap/tidb#57275
2025-02-14 09:19:19 +00:00
435a716670 import: fix oom on global sort when the target table has large number of indices (#59518)
close pingcap/tidb#59508
2025-02-14 06:13:19 +00:00
27f8ff3840 dxf: use global entry_size_limit when executing SQLs (#59516)
close pingcap/tidb#59506
2025-02-13 11:22:23 +00:00
fbdaef5c0b dxf: optimize querying eligible tasks in task executor manager (#59345)
close pingcap/tidb#59344
2025-02-10 07:33:52 +00:00
cf4d252af8 *: split definition out of variable pkg and refactor some reorg methods (#59146)
ref pingcap/tidb#57229, ref pingcap/tidb#57497
2025-01-23 09:24:44 +00:00
fcdbbd49bb DXF: check the cause of subtaskCtx to decide the subtask state (#58893)
close pingcap/tidb#58827
2025-01-22 16:20:26 +00:00
b9b9787f17 dxf: support param modify for active running subtask (#58935)
ref pingcap/tidb#57497
2025-01-22 04:02:05 +00:00
62f9a39346 dxf: support modify app param on scheduler part (#58990)
ref pingcap/tidb#57497
2025-01-20 09:25:14 +00:00
6cceaba4b1 dxf: add comment to explain current schedule strategy (#59033)
ref pingcap/tidb#57497
2025-01-20 06:56:23 +00:00
f7c94c80c5 ddl/dxf: some rename and merge explicit impl of scheduler.Extension with scheduler (#59013)
ref pingcap/tidb#57497
2025-01-17 09:55:29 +00:00
efb579450a dxf: add an example application (#58938)
ref pingcap/tidb#57497
2025-01-16 11:01:01 +00:00
87867ebf5f ddl/dxf: refine internal docs and some log refactor (#58881)
ref pingcap/tidb#57497
2025-01-14 06:14:08 +00:00
b2f2faaa95 parser: move 'model' to 'ast' pkg (#58704)
ref pingcap/tidb#54436
2025-01-07 10:20:04 +00:00
c199ddfcdf disttask: cancel subtask context if scheduled away (#58615)
close pingcap/tidb#58450
2025-01-07 04:12:37 +00:00
bf939fa79a *: don't handle live updates of column size (#58596)
close pingcap/tidb#58595
2025-01-06 19:42:49 +00:00
293077b761 dxf: support modify concurrency/task-specific-param for later subtask at task executor part (#58144)
ref pingcap/tidb#57497
2024-12-27 13:05:21 +00:00
14a469ab9d *: upgrade the PD client to adopt the latest refactor (#58440)
ref pingcap/tidb#58439
2024-12-23 10:36:12 +00:00
588b273f67 global sort: add boundaries to split keys when generating plan (#58323)
close pingcap/tidb#58267
2024-12-17 15:04:30 +00:00
e62963d2c2 logutil/dxf: add a error verbose logger, and use it in DXF (#58185)
ref pingcap/tidb#57497
2024-12-12 11:41:06 +00:00
3adc71cc71 dxf: merge OnFinished into RunSubtask (#58098)
ref pingcap/tidb#57497
2024-12-10 09:26:21 +00:00
71e70af1b1 dxf: merge loops in Run and RunStep of task executor (#57944)
ref pingcap/tidb#57497
2024-12-09 07:32:51 +00:00
3002c1c1c7 dxf: refactor error handling in task executor (#57837)
ref pingcap/tidb#57497
2024-12-03 11:09:38 +00:00
e631ba8a9b dxf: handle modifying task concurrency in scheduler (#57673)
ref pingcap/tidb#57497
2024-12-02 14:05:17 +00:00
2950725ec1 dxf: fix flaky scope test (#57796)
ref pingcap/tidb#57497
2024-11-28 12:17:39 +00:00
d02d48acd9 dxf: refactor failpoints on task executor (#57680)
ref pingcap/tidb#57497
2024-11-26 03:49:00 +00:00