Commit Graph

72 Commits

Author SHA1 Message Date
01e2ff078e lightning: optimize a query to I_S.columns (#57117)
close pingcap/tidb#57115
2024-11-06 03:31:53 +00:00
ea55064414 lightning/importinto: set correct step to create single point allocator (#56602)
close pingcap/tidb#56476
2024-10-12 11:52:54 +00:00
f1cd69819f lightning: allow use azure as data source (#56105)
ref pingcap/tidb#56606
2024-10-12 09:34:34 +00:00
e020f6a921 lightning: delete job.injected length check in TestDoImport (#56596)
close pingcap/tidb#56413
2024-10-12 06:38:23 +00:00
cc44527ce1 lightning: skip non-unique index when detect duplicate KV (#56541)
close pingcap/tidb#53669
2024-10-10 10:36:33 +00:00
c43d0589be *: replace exportloopref with copyloopvar (#56503)
close pingcap/tidb#56509
2024-10-09 12:12:43 +00:00
8c4bab1963 *: upgrade go1.23.2 (#51126)
close pingcap/tidb#50765
2024-10-09 05:15:46 +00:00
b427e33eb9 meta: separate reader and mutator (#56376)
ref pingcap/tidb#54436
2024-09-27 15:34:11 +00:00
bad2ecd6b0 ddl: refine some context usage (#56243)
ref pingcap/tidb#54436, ref pingcap/tidb#56017
2024-09-27 12:03:24 +00:00
448d56910c lightning: fix forget to set lastRetryableErr when ingest RPC fail (#56345)
close pingcap/tidb#55808
2024-09-26 12:13:28 +00:00
918b58d466 lightning: buffer all region jobs and consider store balance (#56112)
close pingcap/tidb#54886, close pingcap/tidb#56113
2024-09-25 12:29:21 +00:00
2eb4dc8e03 lightning: support prepared statement and client stmt cache in logical import mode (#55482)
close pingcap/tidb#54850
2024-09-21 11:47:12 +00:00
6d3439a850 lightning: remove legacy code of session (#56174)
ref pingcap/tidb#53388
2024-09-20 11:13:14 +00:00
67bee7c513 * : Remove binlog related codes (#55955)
close pingcap/tidb#55949
2024-09-20 04:11:57 +00:00
68fdf55d94 *: rename some packages of context (#56108)
close pingcap/tidb#56107
2024-09-19 07:11:03 +00:00
12e5b31731 lighting: replace lightning inner contexts with manged ones (#56038)
ref pingcap/tidb#53388
2024-09-17 07:20:03 +00:00
b3095dc162 Lightning: add retry if transaction failed while fetching task metas (#55670) 2024-09-13 02:02:57 +00:00
76098f5e16 importinto: check column assignment expressions in import into ... (#55968)
close pingcap/tidb#55497
2024-09-10 17:40:45 +00:00
c403cd555d ddl/ingest: set minCommitTS when detect remote duplicate keys (#55588)
close pingcap/tidb#55587
2024-09-05 15:28:18 +00:00
3578e29613 lightning: job range keys and region split keys are 2 concepts (part 2) (#55759)
close pingcap/tidb#55374
2024-09-05 01:31:35 +00:00
3176b9c263 *: move XXInfo/Job, etc, out from model pkg in parser to meta (#55754)
ref pingcap/tidb#54436
2024-09-04 04:49:45 +00:00
dd64d25266 lightining,expression: support user variable for BuildSimpleExpr and remove PlanContext dependency in lightning context (#55617)
ref pingcap/tidb#53388
2024-09-02 04:55:18 +00:00
c65692b9b2 external: range job keys and region split keys are 2 concepts (part 1) (#55659)
ref pingcap/tidb#55374
2024-08-30 10:50:48 +00:00
1f095a3f47 lighting: expose limited fields for kv.Session (#55517)
ref pingcap/tidb#53388
2024-08-20 14:34:42 +00:00
2621a4cff6 ddl: directly use BackendConfig rather than use lightning config (#55433)
ref pingcap/tidb#54436
2024-08-20 06:49:42 +00:00
6499137a0c table: remove method Txn() in MutateContext (#55400)
close pingcap/tidb#55378
2024-08-14 11:18:07 +00:00
a9a5b97e35 lightning: optimize checkpoint size (#55230)
close pingcap/tidb#55054
2024-08-08 08:59:10 +00:00
1985662294 table: add new option DupKeyCheckMode for table mutations (#55194)
ref pingcap/tidb#54397
2024-08-06 07:18:40 +00:00
da7ed5cdc9 ddl: remove unused code related to prev fast create impl (#55116)
ref pingcap/tidb#54436
2024-08-01 08:13:21 +00:00
26378cbe48 lightning: make table in BaseKVEncoder private and provide some wrapped methods (#54673)
ref pingcap/tidb#54397
2024-07-18 03:40:31 +00:00
f158c65446 table: provide some binlog related methods for binlog in MutateContext (#54433)
ref pingcap/tidb#54392, ref pingcap/tidb#54397
2024-07-10 05:43:33 +00:00
53dcc79a6d lightning: improve log (#54483)
close pingcap/tidb#46687, close pingcap/tidb#54478
2024-07-10 03:16:33 +00:00
eec1d4fbdf lightning: reduce the "unable to get keyspace name" log level to DEBUG (#54243)
close pingcap/tidb#54232
2024-07-04 04:38:58 +00:00
1480bb127a lighting: lightning configuration tidb.tls = false should not affect cluster certificate (PD's connection) (#54211)
close pingcap/tidb#54172
2024-07-02 08:31:27 +00:00
bcd797e9e6 tables: optimize memory allocation in tables' hot path (#53832)
close pingcap/tidb#53837
2024-06-22 06:32:49 +00:00
a8de5891fe lightning: improve TotalMemoryConsume for write step (#53856)
ref pingcap/tidb#53858
2024-06-19 01:55:17 +00:00
329a9800c4 ddl: fast-reorg set memory limit for local writer based on current usage (#53873)
ref pingcap/tidb#53789
2024-06-13 12:09:27 +00:00
5c0d73719b build(deps): bump github.com/golangci/golangci-lint from 1.58.1 to 1.59.0 (#53869) 2024-06-08 12:28:28 +00:00
98a0a755fb ddl: unify merging unique and non-unique index for multi-schema change (#53632)
close pingcap/tidb#53677
2024-06-05 09:07:26 +00:00
827720395d go.mod: update the clients to adopt the latest fix/optimization (#53788)
ref pingcap/tidb#35319
2024-06-04 07:20:57 +00:00
c6e6c1545b lightning: turn off zstd decoding concurrency (#53589)
ref pingcap/tidb#53587
2024-06-03 09:38:54 +00:00
29bf0083a6 localbackend: fix resource leak when err on new local backend (#53664)
close pingcap/tidb#53659
2024-05-30 04:14:51 +00:00
afd8de1f62 lightning: release the requirement for NewTargetInfoGetterImpl (#53651) 2024-05-30 02:59:51 +00:00
9164182d0b config: must set line terminator when use strict-format (#53444)
close pingcap/tidb#37338
2024-05-27 08:41:50 +00:00
zzm
22dc064694 lightning: update parquet-go to optimize the memory usage when importing parquet files. (#53474)
close pingcap/tidb#53466
2024-05-23 05:25:17 +00:00
67531d0d92 lightning: provide view for users when using conflict detection (#53289)
close pingcap/tidb#52393
2024-05-21 09:43:17 +00:00
1c4a9c6434 br: separate the restore client (#53197)
ref pingcap/tidb#52877
2024-05-17 05:07:13 +00:00
dd177a808a lightning: update util.go (#53204) 2024-05-13 00:58:10 +00:00
98312088b8 lightning/config: align the behaviour of tidb.tls to doc (#53140)
close pingcap/tidb#53001
2024-05-10 14:31:09 +00:00
f46db203f9 *: caller can use a fixed TS for fast reorg (#52993)
close pingcap/tidb#46986, ref pingcap/tidb#49233
2024-05-10 13:31:09 +00:00