Commit Graph

78 Commits

Author SHA1 Message Date
f2534055f2 ddl: send result chunks to writers after scan task complete (#62014)
close pingcap/tidb#62008
2025-07-18 12:01:49 +00:00
3606de5c43 backend: add log fields for local and external pkg (#62476)
ref pingcap/tidb#61702
2025-07-18 05:49:42 +00:00
5ec4f36c85 dxf: use correct store in add-index, checksum by copr in import-into, and enable DXF service (#62224)
ref pingcap/tidb#61702
2025-07-09 08:07:27 +00:00
86ed1cab62 backend/local: add rate limiter for split region and ingest data (#61555)
close pingcap/tidb#61553
2025-07-05 09:30:09 +00:00
0a704a4bb7 *: collect subtask summary for IMPORT INTO tasks (#61600)
ref pingcap/tidb#61088
2025-07-04 15:12:30 +00:00
6106300325 localbackend: release the service safe point on finish checksum (#62143)
ref pingcap/tidb#61702, close pingcap/tidb#62142
2025-07-02 13:45:42 +00:00
02757471f1 statistics, lightning: remove useless code (#62004) 2025-06-26 03:07:36 +00:00
cc0925eeaf Lightning: Attempt to return writeTooSlow when we experience write timeout. (#61346)
ref pingcap/tidb#61326
2025-06-06 07:09:45 +00:00
d216b7be1a ingestor: fix reused httpclient schema and move ingest api err to ingestcli pkg (#61492)
ref pingcap/tidb#60418
2025-06-05 13:10:44 +00:00
029c0a90d0 *: inject random error for global sort & injest (#60582)
close pingcap/tidb#60277
2025-06-04 11:29:23 +00:00
d0369ffcec ingestor: reuse http client and close idle connections on localbackend close for nextgen (#61473)
ref pingcap/tidb#60418
2025-06-04 05:12:05 +00:00
3ab8c7fe85 importinto: forbid set external-id explicitly and set it to keyspace name when sem enabled on nextgen (#61315)
ref pingcap/tidb#60418
2025-05-26 08:14:19 +00:00
e9853fe210 br: enable follower handler again (#60301)
close pingcap/tidb#60085
2025-05-22 10:42:31 +00:00
ced609bfc7 ingestor: support TLS on nextgen (#61212)
ref pingcap/tidb#60418
2025-05-21 12:02:16 +00:00
cb040775d7 global-sort: unify error message for adding unique index using global sort (#61094)
ref pingcap/tidb#60621, close pingcap/tidb#61084
2025-05-16 05:44:40 +00:00
c71ebaf73e global-sort: accelerate adding unique index using global sort (#60848)
ref pingcap/tidb#60621
2025-05-12 09:25:58 +00:00
c0176459da globalsort: support onDup param for engine (#60950)
ref pingcap/tidb#60621
2025-05-08 05:14:16 +00:00
1a1897ec9c lightning: gopls modernize (#60768)
close pingcap/tidb#60767
2025-05-08 03:57:43 +00:00
e774090927 localbackend: handle ingest pberror in next gen (#60895)
ref pingcap/tidb#60418
2025-04-30 03:34:20 +00:00
0200e5d634 localbackend: merge feature branch next-gen (#60879)
ref pingcap/tidb#60418
2025-04-28 05:18:58 +00:00
531eb80580 *: add a RetryableErrorCount metric to trace error during DDL (#60485)
ref pingcap/tidb#60484
2025-04-22 14:23:02 +00:00
fc7b44a20d localbackend: fix retry on ingest err 'disk full' (#60680)
close pingcap/tidb#60678
2025-04-21 10:30:19 +00:00
3a47685a38 globalsort: add definition for conflict handling and refactor (#60631)
ref pingcap/tidb#60621
2025-04-18 12:50:14 +00:00
61c09e601e build(deps): bump github.com/golangci/golangci-lint from 1.63.4 to 1.64.7 (#60029) 2025-04-03 07:44:54 +00:00
f3d0e43040 *: upgrade to the latest client-go (#59757)
ref pingcap/tidb#58780
2025-03-27 07:32:36 +00:00
d51e00e5bb globalsort: reduce number of SST ingested into TiKV (#59870) (#60045)
ref pingcap/tidb#51734, close pingcap/tidb#59962
2025-03-24 12:22:43 +00:00
514204ea91 Lightning: Make sure we are using default block size of 16KB if user does not specify one. (#60097)
close pingcap/tidb#59947
2025-03-20 05:57:42 +00:00
21f9bb7b7a ddl/ingest: add test for ADD INDEX with global sort recovery from retryable error (#59987)
ref pingcap/tidb#57670
2025-03-11 09:31:39 +00:00
fc0adfcf28 lightning/global-sort: move membuf pkg from br to lightning (#59550)
ref pingcap/tidb#57275
2025-02-14 09:19:19 +00:00
c7ce7902a8 ddl: reuse backend for DXF subtasks of same step (#59165)
ref pingcap/tidb#57229, ref pingcap/tidb#57497
2025-01-24 09:17:12 +00:00
cf4d252af8 *: split definition out of variable pkg and refactor some reorg methods (#59146)
ref pingcap/tidb#57229, ref pingcap/tidb#57497
2025-01-23 09:24:44 +00:00
db7843a5c4 ddl/ingest: remove backend context manager and refactor code (#57770)
ref pingcap/tidb#57670
2025-01-17 08:59:10 +00:00
b2f2faaa95 parser: move 'model' to 'ast' pkg (#58704)
ref pingcap/tidb#54436
2025-01-07 10:20:04 +00:00
88a2247d9f dupdetect: gRPC cancel should trigger retry (#58542)
close pingcap/tidb#58523
2025-01-06 11:13:19 +00:00
65e514e5e7 br/lightning: support pd_enable_follower_handle_region (#58653)
close pingcap/tidb#58651
2025-01-03 11:26:39 +00:00
42d4fae449 lightning: adjust TS assertion range between TiDB and PD (#58594)
close pingcap/tidb#58449
2024-12-30 12:26:30 +00:00
14a469ab9d *: upgrade the PD client to adopt the latest refactor (#58440)
ref pingcap/tidb#58439
2024-12-23 10:36:12 +00:00
588b273f67 global sort: add boundaries to split keys when generating plan (#58323)
close pingcap/tidb#58267
2024-12-17 15:04:30 +00:00
098213a180 lightning, ddl: set TS to engineMeta after ResetEngineSkipAllocTS (#57998)
close pingcap/tidb#57980
2024-12-05 15:00:05 +00:00
d3936a2f9f lightning: set the 17th bit of the txn_source to indicate the write source is lightning physical mode import (#57706)
close pingcap/tidb#57714
2024-11-29 09:46:17 +00:00
ec8b81b98e ddl: dynamically adjusting the max write speed of reorganization job (#57611)
ref pingcap/tidb#57526
2024-11-25 14:13:35 +00:00
ecca340037 lightning: fix id too large after parallel import (#57398)
close pingcap/tidb#56814
2024-11-19 11:03:32 +00:00
ef6662ed35 Restore: refine split code for both compacted sst files and logs (#56917)
ref pingcap/tidb#56522
2024-11-07 08:42:21 +00:00
b522e98080 lightning: unlimited grpc MaxCallRecvMsgSize for TikvImporter (#56771)
close pingcap/tidb#56114
2024-11-06 16:13:15 +00:00
0e8c8dadce lightning: test a simple case that golang outputs same SST (#57061)
ref pingcap/tidb#57105
2024-11-06 10:09:03 +00:00
01e2ff078e lightning: optimize a query to I_S.columns (#57117)
close pingcap/tidb#57115
2024-11-06 03:31:53 +00:00
e020f6a921 lightning: delete job.injected length check in TestDoImport (#56596)
close pingcap/tidb#56413
2024-10-12 06:38:23 +00:00
cc44527ce1 lightning: skip non-unique index when detect duplicate KV (#56541)
close pingcap/tidb#53669
2024-10-10 10:36:33 +00:00
c43d0589be *: replace exportloopref with copyloopvar (#56503)
close pingcap/tidb#56509
2024-10-09 12:12:43 +00:00
bad2ecd6b0 ddl: refine some context usage (#56243)
ref pingcap/tidb#54436, ref pingcap/tidb#56017
2024-09-27 12:03:24 +00:00