Commit Graph

205 Commits

Author SHA1 Message Date
b6e32dd757 ddl: run merge-temp-index step on DXF (#62971)
close pingcap/tidb#63296
2025-10-03 10:15:04 +00:00
b3011b689d disttask: support metering on next-gen (#63397)
ref pingcap/tidb#61702
2025-09-19 02:27:35 +00:00
276ed0cf95 dxf: add api to manually tune the calculation of resource related params (#63517)
ref pingcap/tidb#61702
2025-09-18 15:09:53 +00:00
eb25197d68 disttask: collect metering data to subtask's real time summary (#63431)
ref pingcap/tidb#61702
2025-09-15 12:47:09 +00:00
941ba546bd importinto: avoid cal resource params twice and round those params (#63452)
ref pingcap/tidb#61702
2025-09-11 08:19:15 +00:00
37f8a14657 importinto: fix zero update time during validation phase and refine logs (#63425)
ref pingcap/tidb#61702
2025-09-09 12:07:54 +00:00
9825dfebc9 importinto: calculate distsql scan concurency in nextgen (#63341)
ref pingcap/tidb#61702
2025-09-03 11:52:06 +00:00
36a2af4b10 dxf: start scheduler without reserve resource in nextgen (#63205)
ref pingcap/tidb#61702
2025-08-26 12:22:10 +00:00
97c5139bf9 dxf: add schedule status api for nextgen (#63159)
ref pingcap/tidb#61702
2025-08-25 10:45:07 +00:00
1fa4788853 dxf: prepare work for schedule status API in nextgen (#63140)
ref pingcap/tidb#61702
2025-08-22 17:21:38 +00:00
aa875ea472 dxf: calculate max_node_count and concurrency for next-gen automatically (#62995)
ref pingcap/tidb#61702
2025-08-22 14:53:38 +00:00
658fa1a43e crossks: add real-tikv tests for info schema sync (#63008)
ref pingcap/tidb#61702
2025-08-19 17:29:50 +00:00
36dc5f0871 dxf: support modify max_node_count param (#62999)
ref pingcap/tidb#61702
2025-08-19 04:21:02 +00:00
d82d2baa0f bazel: update settings and close some race on tests (#63009) 2025-08-15 09:07:31 +00:00
9cd2b03833 dxf/crossks: check by inner fields not global var and make crossks real tikvtest work (#62918)
ref pingcap/tidb#61702
2025-08-14 15:00:03 +00:00
9b12530970 crossks: join online schema change server group and sync info schema (#62860)
ref pingcap/tidb#61702
2025-08-13 04:51:06 +00:00
2a0741939c disttask: refine the speed calculation of import jobs (#62638)
ref pingcap/tidb#61088
2025-08-09 01:15:27 +00:00
eb12403896 *: replace make then copy to slices.Clone (#62544)
close pingcap/tidb#62543
2025-07-31 12:27:55 +00:00
1ea1db40cd infosync: move server info sync part of structs out to a separate pkg (#62609)
ref pingcap/tidb#61702
2025-07-25 07:46:59 +00:00
5ae6ad63c2 session: rename and unify location of interfaces, and some refactor (#62498)
ref pingcap/tidb#61702
2025-07-22 07:43:38 +00:00
8060477c03 importinto: reduce def engine size to 100G and always use max node count to adjust subtask count (#62527)
ref pingcap/tidb#61702
2025-07-21 07:41:31 +00:00
0fc2d8bfd8 tests: add create index realtikv test for next_gen (#62326)
ref pingcap/tidb#61702
2025-07-18 07:16:24 +00:00
4048d4098b dxf: fix flaky test TestResourceControl/scale-in (#62489)
close pingcap/tidb#61354
2025-07-18 05:49:48 +00:00
3606de5c43 backend: add log fields for local and external pkg (#62476)
ref pingcap/tidb#61702
2025-07-18 05:49:42 +00:00
5ec4f36c85 dxf: use correct store in add-index, checksum by copr in import-into, and enable DXF service (#62224)
ref pingcap/tidb#61702
2025-07-09 08:07:27 +00:00
0a704a4bb7 *: collect subtask summary for IMPORT INTO tasks (#61600)
ref pingcap/tidb#61088
2025-07-04 15:12:30 +00:00
c56481e1d1 ddl: run add-index subtasks at system keyspace tidb (#62045)
ref pingcap/tidb#61702
2025-07-02 10:38:25 +00:00
6e13f827fc realtikvtest: make import-into/add-index tests works on nextgen (#62112)
ref pingcap/tidb#61702
2025-07-01 15:58:33 +00:00
ae70341a13 disttask, util: remove useless code (#62056) 2025-06-30 07:05:18 +00:00
704373b1e4 importinto: use correct session to submit/update/query task in DXF service (#62023)
ref pingcap/tidb#61702
2025-06-28 06:07:21 +00:00
8e6d77ae27 dxf: add keyspace name to global task table (#61755)
ref pingcap/tidb#61702
2025-06-18 16:41:39 +00:00
292c2c5caf global sort: use cluster id in file path (#61502)
close pingcap/tidb#61537
2025-06-10 06:34:40 +00:00
254849d23a ddl, disttask: extend SubtaskSummary and add Collector interface (#61425)
ref pingcap/tidb#61088
2025-06-09 08:12:37 +00:00
0f5aa223ec *: rewrite InTest to EnableInternalCheck (#60167)
ref pingcap/tidb#60176
2025-06-05 07:56:08 +00:00
c1a4550fd4 dxf: make target scope fixed to dxf_service on nextgen (#61427)
ref pingcap/tidb#60418
2025-06-04 06:53:02 +00:00
2036543b29 disttask: set moderate timeout for pkg/disttask/framework/storage (#61463)
close pingcap/tidb#61461
2025-06-03 09:26:38 +00:00
fe769f5154 dxf: include context info in the stored error (#61289)
close pingcap/tidb#60433
2025-05-23 13:28:13 +00:00
c71ebaf73e global-sort: accelerate adding unique index using global sort (#60848)
ref pingcap/tidb#60621
2025-05-12 09:25:58 +00:00
dc876e5c51 disttask: gopls modernize (#60795)
close pingcap/tidb#60794
2025-04-28 10:06:26 +00:00
531eb80580 *: add a RetryableErrorCount metric to trace error during DDL (#60485)
ref pingcap/tidb#60484
2025-04-22 14:23:02 +00:00
813b32f4df import: add a hidden option to enable manual recover for debug purpose (#59674)
ref pingcap/tidb#59640
2025-04-08 08:46:48 +00:00
1cd39430e1 *: inject random error for DXF (part2) (#60354)
ref pingcap/tidb#60277
2025-04-08 06:18:54 +00:00
e2d1d89c9e *: inject random error for DXF (part1) (#60278)
ref pingcap/tidb#60277
2025-03-27 12:38:13 +00:00
ed8b869a60 globalsort: write sorted kv meta to external storage (#59966)
close pingcap/tidb#60083
2025-03-26 09:39:54 +00:00
b08a92082f disttask: fix task metrics for distributed execute framework panel (#59940)
ref pingcap/tidb#47017
2025-03-25 17:31:28 +00:00
d51e00e5bb globalsort: reduce number of SST ingested into TiKV (#59870) (#60045)
ref pingcap/tidb#51734, close pingcap/tidb#59962
2025-03-24 12:22:43 +00:00
6953d3656c disttask: fix potential block on DXF scheduler (#59945)
close pingcap/tidb#59944
2025-03-14 05:41:34 +00:00
246a241621 dxf: add manual recovery mode (#59641)
ref pingcap/tidb#59640
2025-02-24 09:42:51 +00:00
811be5aaa5 disttask: add local disk space to node resource managed by DXF (#59607)
ref pingcap/tidb#49008
2025-02-20 08:42:09 +00:00
b45a9e840c sysvar: add validation check for tidb_max_dist_task_nodes (#59645)
ref pingcap/tidb#58944
2025-02-20 07:05:27 +00:00