Commit Graph

85 Commits

Author SHA1 Message Date
ec8b81b98e ddl: dynamically adjusting the max write speed of reorganization job (#57611)
ref pingcap/tidb#57526
2024-11-25 14:13:35 +00:00
eb871f862e lightning: add PK to internal tables (#57480)
close pingcap/tidb#57479
2024-11-22 02:29:50 +00:00
039b38670c lightning: remove some NOT NULL constraints on lightning table (#57577)
close pingcap/tidb#57578
2024-11-21 10:45:17 +00:00
c091dba897 gomod: remove my patch dependency (#57571) 2024-11-21 07:19:14 +00:00
91beef4bb1 *: disable insert null to not-null column for single-row insertion in non-strict mode (#55477)
close pingcap/tidb#55457, close pingcap/tidb#56381
2024-11-19 12:46:11 +00:00
ecca340037 lightning: fix id too large after parallel import (#57398)
close pingcap/tidb#56814
2024-11-19 11:03:32 +00:00
0e3efb6ea8 Lightning: Skip apply ioworkers when seek(0, io.SeekCurrent) (#57454)
close pingcap/tidb#57413
2024-11-19 08:15:20 +00:00
06ac7b17ef lightning: log error cause when SampleFileCompressRatio fail (#57455) 2024-11-18 13:30:29 +00:00
zzm
0a9a231c5e lightning: sample once parquet file (#56205)
close pingcap/tidb#56104
2024-11-18 03:24:59 +00:00
91436e3ddb local-sst-writer: test only logical content and test many keys (#57203)
ref pingcap/tidb#57105
2024-11-11 04:23:27 +00:00
ef6662ed35 Restore: refine split code for both compacted sst files and logs (#56917)
ref pingcap/tidb#56522
2024-11-07 08:42:21 +00:00
b522e98080 lightning: unlimited grpc MaxCallRecvMsgSize for TikvImporter (#56771)
close pingcap/tidb#56114
2024-11-06 16:13:15 +00:00
0e8c8dadce lightning: test a simple case that golang outputs same SST (#57061)
ref pingcap/tidb#57105
2024-11-06 10:09:03 +00:00
01e2ff078e lightning: optimize a query to I_S.columns (#57117)
close pingcap/tidb#57115
2024-11-06 03:31:53 +00:00
ea55064414 lightning/importinto: set correct step to create single point allocator (#56602)
close pingcap/tidb#56476
2024-10-12 11:52:54 +00:00
f1cd69819f lightning: allow use azure as data source (#56105)
ref pingcap/tidb#56606
2024-10-12 09:34:34 +00:00
e020f6a921 lightning: delete job.injected length check in TestDoImport (#56596)
close pingcap/tidb#56413
2024-10-12 06:38:23 +00:00
cc44527ce1 lightning: skip non-unique index when detect duplicate KV (#56541)
close pingcap/tidb#53669
2024-10-10 10:36:33 +00:00
c43d0589be *: replace exportloopref with copyloopvar (#56503)
close pingcap/tidb#56509
2024-10-09 12:12:43 +00:00
8c4bab1963 *: upgrade go1.23.2 (#51126)
close pingcap/tidb#50765
2024-10-09 05:15:46 +00:00
b427e33eb9 meta: separate reader and mutator (#56376)
ref pingcap/tidb#54436
2024-09-27 15:34:11 +00:00
bad2ecd6b0 ddl: refine some context usage (#56243)
ref pingcap/tidb#54436, ref pingcap/tidb#56017
2024-09-27 12:03:24 +00:00
448d56910c lightning: fix forget to set lastRetryableErr when ingest RPC fail (#56345)
close pingcap/tidb#55808
2024-09-26 12:13:28 +00:00
918b58d466 lightning: buffer all region jobs and consider store balance (#56112)
close pingcap/tidb#54886, close pingcap/tidb#56113
2024-09-25 12:29:21 +00:00
2eb4dc8e03 lightning: support prepared statement and client stmt cache in logical import mode (#55482)
close pingcap/tidb#54850
2024-09-21 11:47:12 +00:00
6d3439a850 lightning: remove legacy code of session (#56174)
ref pingcap/tidb#53388
2024-09-20 11:13:14 +00:00
67bee7c513 * : Remove binlog related codes (#55955)
close pingcap/tidb#55949
2024-09-20 04:11:57 +00:00
68fdf55d94 *: rename some packages of context (#56108)
close pingcap/tidb#56107
2024-09-19 07:11:03 +00:00
12e5b31731 lighting: replace lightning inner contexts with manged ones (#56038)
ref pingcap/tidb#53388
2024-09-17 07:20:03 +00:00
b3095dc162 Lightning: add retry if transaction failed while fetching task metas (#55670) 2024-09-13 02:02:57 +00:00
76098f5e16 importinto: check column assignment expressions in import into ... (#55968)
close pingcap/tidb#55497
2024-09-10 17:40:45 +00:00
c403cd555d ddl/ingest: set minCommitTS when detect remote duplicate keys (#55588)
close pingcap/tidb#55587
2024-09-05 15:28:18 +00:00
3578e29613 lightning: job range keys and region split keys are 2 concepts (part 2) (#55759)
close pingcap/tidb#55374
2024-09-05 01:31:35 +00:00
3176b9c263 *: move XXInfo/Job, etc, out from model pkg in parser to meta (#55754)
ref pingcap/tidb#54436
2024-09-04 04:49:45 +00:00
dd64d25266 lightining,expression: support user variable for BuildSimpleExpr and remove PlanContext dependency in lightning context (#55617)
ref pingcap/tidb#53388
2024-09-02 04:55:18 +00:00
c65692b9b2 external: range job keys and region split keys are 2 concepts (part 1) (#55659)
ref pingcap/tidb#55374
2024-08-30 10:50:48 +00:00
1f095a3f47 lighting: expose limited fields for kv.Session (#55517)
ref pingcap/tidb#53388
2024-08-20 14:34:42 +00:00
2621a4cff6 ddl: directly use BackendConfig rather than use lightning config (#55433)
ref pingcap/tidb#54436
2024-08-20 06:49:42 +00:00
6499137a0c table: remove method Txn() in MutateContext (#55400)
close pingcap/tidb#55378
2024-08-14 11:18:07 +00:00
a9a5b97e35 lightning: optimize checkpoint size (#55230)
close pingcap/tidb#55054
2024-08-08 08:59:10 +00:00
1985662294 table: add new option DupKeyCheckMode for table mutations (#55194)
ref pingcap/tidb#54397
2024-08-06 07:18:40 +00:00
da7ed5cdc9 ddl: remove unused code related to prev fast create impl (#55116)
ref pingcap/tidb#54436
2024-08-01 08:13:21 +00:00
26378cbe48 lightning: make table in BaseKVEncoder private and provide some wrapped methods (#54673)
ref pingcap/tidb#54397
2024-07-18 03:40:31 +00:00
f158c65446 table: provide some binlog related methods for binlog in MutateContext (#54433)
ref pingcap/tidb#54392, ref pingcap/tidb#54397
2024-07-10 05:43:33 +00:00
53dcc79a6d lightning: improve log (#54483)
close pingcap/tidb#46687, close pingcap/tidb#54478
2024-07-10 03:16:33 +00:00
eec1d4fbdf lightning: reduce the "unable to get keyspace name" log level to DEBUG (#54243)
close pingcap/tidb#54232
2024-07-04 04:38:58 +00:00
1480bb127a lighting: lightning configuration tidb.tls = false should not affect cluster certificate (PD's connection) (#54211)
close pingcap/tidb#54172
2024-07-02 08:31:27 +00:00
bcd797e9e6 tables: optimize memory allocation in tables' hot path (#53832)
close pingcap/tidb#53837
2024-06-22 06:32:49 +00:00
a8de5891fe lightning: improve TotalMemoryConsume for write step (#53856)
ref pingcap/tidb#53858
2024-06-19 01:55:17 +00:00
329a9800c4 ddl: fast-reorg set memory limit for local writer based on current usage (#53873)
ref pingcap/tidb#53789
2024-06-13 12:09:27 +00:00