Commit Graph

1108 Commits

Author SHA1 Message Date
fc2e10b257 global sort: refine and add tests for readAllData (#49389)
ref pingcap/tidb#48952
2023-12-13 08:42:26 +00:00
ced8af24c4 br: use atomic.Pointer instead of atomic.Value (#49359)
ref pingcap/tidb#44736
2023-12-13 08:12:20 +00:00
93225684c1 br/storage: add scope to support service_account type of credential file & remove default GCS httpclient (#49363)
ref pingcap/tidb#49340
2023-12-12 17:16:48 +00:00
5bd02673af lightning: fix parquet import always fail (#49391)
close pingcap/tidb#49368
2023-12-12 14:03:26 +00:00
1b199c2a6e global sort: bench merge step (#49376)
ref pingcap/tidb#48952
2023-12-12 14:03:19 +00:00
b478056bbf lightning, membuf: limit the maximum memory size allocated from pool (#49210)
ref pingcap/tidb#48420
2023-12-12 04:33:17 +00:00
8c8333a9ac br/storage: disable keep alive for GCS on default to speed up request (#49341)
close pingcap/tidb#49340
2023-12-12 02:54:18 +00:00
7aba5245f1 lightning: add function getChunkCompressedSizeForParquet for solving parquet oom issue (#49021)
ref pingcap/tidb#49234
2023-12-11 07:22:48 +00:00
9e34417f97 test: ignore some expected goroutine leak (#49320)
close pingcap/tidb#43430
2023-12-11 06:43:50 +00:00
bcbccafcea *: replace exp/maps with std/maps (#49315)
ref pingcap/tidb#45933
2023-12-11 02:19:47 +00:00
e62699cb26 br: remove duplicate bind info after snapshot restore (#48946)
close pingcap/tidb#46527
2023-12-10 08:48:16 +00:00
6c30c6e6aa br: refactor error handle mechanism to tolerant unexpect kv errors. (#48646)
close pingcap/tidb#47656
2023-12-08 05:20:52 +00:00
be62f754fb ddl: wrap the sessionctx to public delete range logic to BR (#48050)
close pingcap/tidb#47738
2023-12-07 06:25:21 +00:00
94930215de gloabl sort: revert "writer upload data in background" (#49118)
ref pingcap/tidb#49117
2023-12-04 12:19:22 +00:00
7353fbeea8 infosync: fully integrated PD HTTP client into InfoSyncer (#49006)
ref pingcap/tidb#35319
2023-12-01 11:23:49 +00:00
f39e9bf1ee global sort: fix a bug and prepare large scale data to s3 (#49024)
ref pingcap/tidb#48952
2023-11-30 07:56:49 +00:00
720ae4b114 gloabl sort: add more test cases, and writer upload data in background (#48873)
ref pingcap/tidb#48779
2023-11-29 12:37:18 +00:00
d4b41beec9 build/linter/util: exclude files according to the key of only_files (#48999)
close pingcap/tidb#48998
2023-11-29 11:17:47 +00:00
31eb1eca4c br: split br and lightning tests in pipelines (#48921)
close pingcap/tidb#48955
2023-11-29 09:29:49 +00:00
4309c0f334 external: LoadIngestData download all needed data and sort all (#48871)
ref pingcap/tidb#48779
2023-11-28 10:11:56 +00:00
c741119684 lightning: still update stats_meta if analyze is skipped (#48915)
close pingcap/tidb#48562
2023-11-28 06:16:16 +00:00
56106454d3 BR: Support encryption for restored ebs volumes (#48900)
close pingcap/tidb#48899
2023-11-27 10:35:52 +00:00
c82e19c0c2 infosync: integrate PD HTTP client into the placement manager (#48858)
ref pingcap/tidb#35319
2023-11-27 09:44:51 +00:00
d3f399f25d *: fix data race of Column.GeneratedExpr (#48888)
close pingcap/tidb#44919, close pingcap/tidb#48191
2023-11-27 08:19:15 +00:00
2e277dc8f0 build(deps): bump github.com/prometheus/prometheus from 0.0.0-20190525122359-d20e84d0fb64 to 0.48.0 (#48866) 2023-11-27 03:11:44 +00:00
12634458cb *: optimize the layout of global sort files and other (#48275)
ref pingcap/tidb#48779
2023-11-24 11:56:43 +00:00
8eb191303a *: fix grpc client leak bug for AUTO_ID_CACHE=1 tables (#48870)
close pingcap/tidb#48869
2023-11-24 11:25:14 +00:00
c771e8b2f9 br/storage: enable async prefetch data (#48587)
close pingcap/tidb#48781
2023-11-24 04:44:42 +00:00
9cf638fb08 global sort: merge to one file (#48142)
ref pingcap/tidb#45719
2023-11-23 08:46:12 +00:00
67b70e5207 br: record size in backupmeta (#48772)
close pingcap/tidb#48745
2023-11-23 06:11:41 +00:00
b8515ae354 session: move session interface into new package (#48792)
ref pingcap/tidb#44940
2023-11-23 05:28:11 +00:00
eff04125e7 br, util/ranger: replace EvalWithInnerCtx with Eval for packages br and util/ranger (#48785)
close pingcap/tidb#48783
2023-11-23 03:40:11 +00:00
d2cfbdef67 ddl,lightning: fix ingest data unexpectedly using zero commit ts (#48797)
close pingcap/tidb#48804
2023-11-22 12:56:41 +00:00
5bc2136a60 membuf: add SliceLocation to reduce GC pressure (#48710)
ref pingcap/tidb#48779
2023-11-22 09:41:58 +00:00
c4677188c4 lightning: add complex unit tests for lightning post-import conflict detection "replace" mode (#48514)
ref pingcap/tidb#45774
2023-11-22 08:23:41 +00:00
9062834986 expression: add ctx to expression.Eval (#48609)
close pingcap/tidb#48608
2023-11-22 05:29:40 +00:00
711e95ff46 ebs br: make sure fsr credit is full filled (#48627)
close pingcap/tidb#48629
2023-11-21 08:59:40 +00:00
006763789b lightning: avoid copying bytes if the bytes are already non-shared (#48676)
close pingcap/tidb#48677
2023-11-21 07:10:17 +00:00
3d176fcc2c util/codec,util/errctx,*: add error context, and use it to refactor util/codec (#48613)
close pingcap/tidb#48611
2023-11-20 13:53:40 +00:00
49d1b66787 br: fix incorrect uri for the file storage (#48453)
close pingcap/tidb#48452
2023-11-20 07:50:41 +00:00
7db44c0175 br: make br support FIPS (#48421)
close pingcap/tidb#48419
2023-11-20 06:10:10 +00:00
95759f44bd *: further adaptation to PD HTTP client (#48606)
ref pingcap/tidb#35319
2023-11-20 04:20:42 +00:00
4f2a69eca7 lightning: release memory pool when external engine close (#48537)
close pingcap/tidb#48538
2023-11-17 08:53:18 +00:00
557599fb1f lightning: fix typo because lightning supports sql/parquet data files but csv. (#48655)
close pingcap/tidb#48654
2023-11-17 06:51:46 +00:00
657f0d94a9 pkg: stop using math/rand.Seed (#48653) 2023-11-17 06:00:53 +00:00
07057bdb1f backup: add table-concurrency to control backup tableInfo stats and checksum. (#48570)
close pingcap/tidb#48571
2023-11-16 15:29:17 +00:00
311d8a2957 disttask: add awserr. RequestCanceled as graceful shutdown err (#48639)
close pingcap/tidb#48635
2023-11-16 10:11:17 +00:00
db2e239496 types: remove param explicitTz from types.ParseTime (#48574)
close pingcap/tidb#48573
2023-11-15 08:08:17 +00:00
0f35b6f813 *: integrate PD HTTP client to the store helper (#48276)
ref pingcap/tidb#35319
2023-11-15 04:47:16 +00:00
032c8e599c external/test: compare the performance of merge iter (#48560) 2023-11-14 03:12:44 +00:00