17d70c1b7e
lightning/parser: support STARTING BY ( #40821 )
...
ref pingcap/tidb#40499
2023-01-30 11:23:54 +08:00
e11cbec12b
lightning: support data files with bom header ( #40813 )
...
close pingcap/tidb#40744
2023-01-30 01:13:54 +08:00
5048568825
lightning: support timeout for tikv/tidb retry ( #40805 )
...
ref pingcap/tidb#33714
2023-01-29 19:11:54 +08:00
8e963505c1
bazel: upgrade rules_go to prepare for Go 1.20 ( #40776 )
...
close pingcap/tidb#40798
2023-01-29 15:35:54 +08:00
06e2b29551
lightning: check hasDupe and tableID when resolve duplicate rows ( #40696 )
...
close pingcap/tidb#40657
2023-01-28 14:17:53 +08:00
76153ab9ff
lint: update config of deprecated linter ( #40763 )
...
ref pingcap/tidb#40786
2023-01-28 11:57:53 +08:00
bbd1995ea3
ddl, lightning: support detecting duplicate keys in different batch ( #40701 )
...
close pingcap/tidb#40698
2023-01-19 14:29:49 +08:00
aef752a407
lightning: continue this region round and retry on next round when TiKV is busy ( #40278 )
...
* finish
Signed-off-by: lance6716 <lance6716@gmail.com >
* fix some comments
Signed-off-by: lance6716 <lance6716@gmail.com >
* fix CI
Signed-off-by: lance6716 <lance6716@gmail.com >
* address comment
Signed-off-by: lance6716 <lance6716@gmail.com >
* fix errdoc
Signed-off-by: lance6716 <lance6716@gmail.com >
* revert a change
Signed-off-by: lance6716 <lance6716@gmail.com >
* add lightning debug log
Signed-off-by: lance6716 <lance6716@gmail.com >
* fix bug
Signed-off-by: lance6716 <lance6716@gmail.com >
* address comment
Signed-off-by: lance6716 <lance6716@gmail.com >
Signed-off-by: lance6716 <lance6716@gmail.com >
Co-authored-by: Weizhen Wang <wangweizhen@pingcap.com >
2023-01-12 15:24:33 +08:00
fb29c807ce
lightning: fix risk of OOM ( #40443 )
...
* fix: check buf size
* fix: terminate too long buf
* add ut
* fix dev
* local var
* fix ut
* fix lint
* fix lint
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io >
2023-01-12 13:48:33 +08:00
eef8438e14
lightning: not set pos to end if fail to ReadUntil ( #40232 )
...
* fix: not set pos to end
* add it
* fix: return to prevPos
* fix: return err
* fix
* fix
* fix it
* fix
* fix: not exit when err
Co-authored-by: Zak Zhao <57036248+joccau@users.noreply.github.com >
2023-01-12 12:20:33 +08:00
e3f8d9805e
lightning: fix codecPDClient ( #40496 )
...
Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com >
Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com >
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io >
Co-authored-by: Weizhen Wang <wangweizhen@pingcap.com >
2023-01-11 17:36:33 +08:00
47080d98d1
lightning: check peers write stall when switch-mode is disabled ( #40228 )
...
close pingcap/tidb#40163
2022-12-30 19:58:17 +08:00
cf34941401
ttl: make the TTL compatible with dumpling, lightning and BR ( #40216 )
...
close pingcap/tidb#40215
2022-12-29 16:30:21 +08:00
1ace97069e
lightning: disable foreign key checks ( #40032 )
...
close pingcap/tidb#40027
2022-12-26 10:58:15 +08:00
ae58fa1c8b
expression: introduce cast(... as ... array) in expression index ( #39992 )
...
close pingcap/tidb#40045
2022-12-20 12:56:54 +08:00
0c1808245b
*: support sample for compressed files for adjustment ( #39680 )
...
ref pingcap/tidb#38514
2022-12-19 15:42:54 +08:00
90a739821c
ddl: close lightning writers after the import is complete ( #39879 )
...
close pingcap/tidb#39800
2022-12-13 20:10:51 +08:00
92836501a7
lightning: update to new TLS API of SQL driver ( #39623 )
2022-12-09 16:20:04 +08:00
4b98439f33
lightning: record network usage ( #39673 )
...
ref pingcap/tidb#39674
2022-12-08 18:24:05 +08:00
35819ee43f
import: check table empty skips using index ( #39604 )
...
close pingcap/tidb#39477
2022-12-05 21:40:03 +08:00
eed9ceb8e2
import: fix dup-detect failure on index data ( #39571 )
...
close pingcap/tidb#39476
2022-12-02 11:38:00 +08:00
10b3bc7a11
lightning: refine progress for compress files import ( #39219 )
...
ref pingcap/tidb#38514
2022-12-02 10:34:00 +08:00
78011c5286
lightning: add WithDupIndicator to expose status to caller ( #39461 )
2022-12-01 15:20:01 +08:00
328aef88b0
lightning: add precheck about downstream CDC/PiTR ( #39338 )
...
close pingcap/tidb#39346
2022-11-30 13:17:59 +08:00
19dae1fa73
*: separate auto_increment ID allocator from _tidb_rowid allocator when AUTO_ID_CACHE=1 ( #39041 )
...
close pingcap/tidb#982
2022-11-29 20:20:00 +08:00
f9a6e47192
planner: support push part of order property down to the partition table ( #36108 )
...
ref pingcap/tidb#26166
2022-11-29 13:51:59 +08:00
f1215a5925
bazel: update shard_count ( #39435 )
2022-11-29 10:30:01 +08:00
5b9d96b948
lightning: support compress for lightning, add compress unit and integration tests ( #39153 )
...
ref pingcap/tidb#38514
2022-11-28 18:58:01 +08:00
5c9570d51e
import: fix memory leak ( #39332 )
...
close pingcap/tidb#39331
2022-11-28 11:34:00 +08:00
89522b1708
*: fix deprecated ( #39401 )
2022-11-28 09:46:00 +08:00
05d616c2d3
lightning: recover status from checkpoint after index engine closed ( #39365 )
2022-11-25 16:39:58 +08:00
f50113f6d8
lightning: finer granularity for tidb backend finished bytes ( #39318 )
2022-11-23 15:59:58 +08:00
777cb5fbc6
lightning: expose metrics when as a library ( #39218 )
...
ref pingcap/tidb#38455
2022-11-18 10:43:55 +08:00
481f5abad9
lightning: support checkpoint read for compress files ( #38946 )
...
ref pingcap/tidb#38514
2022-11-11 16:03:52 +08:00
a20b70ffa9
lightning: fix cluster index ( #39010 )
...
close pingcap/tidb#39009
2022-11-10 17:19:52 +08:00
f9c7bbccec
*: authentication using tidb_auth_token plugin ( #38773 )
...
close pingcap/tidb#38504
2022-11-04 02:54:00 +08:00
bc0b4199d8
lightning: support custom file listing policy ( #38599 )
...
close pingcap/tidb#38598
2022-10-30 17:11:58 +08:00
0f6baa3f9b
lightning: support extracting source/schema/table name to specific column ( #37790 )
...
close pingcap/tidb#37797
2022-10-26 17:47:57 +08:00
51e559c52d
bazel: update config and nogo ( #38549 )
2022-10-19 17:03:54 +08:00
ff5fd4e8c6
lightning: make default file router support compression ( #38515 )
...
ref pingcap/tidb#38514
2022-10-19 10:59:54 +08:00
d0376379d6
*: don't use DSN to avoid some security problems ( #38342 )
2022-10-19 10:27:52 +08:00
68305e9004
lightning: specify collation when parquet value to string datum ( #38391 )
...
close pingcap/tidb#38351
2022-10-17 09:23:54 +08:00
00791e7968
restore: ignore error when downstream db/table is created ( #38327 )
...
ref pingcap/tidb#37984
2022-10-12 19:09:51 +08:00
4adfaf1ec0
*: enable race for some part ( #38288 )
2022-10-08 12:03:49 +08:00
5c4b328b02
variables: enable variable hook to access storage ( #38227 )
...
close pingcap/tidb#38225
2022-10-01 02:35:45 +08:00
cab909e6b3
lightning: clone default filter for new configs ( #38157 )
...
close pingcap/tidb#38151
2022-09-27 10:47:44 +08:00
7cbbaac920
Lightning: change severity of clusterResourceCheckItem,emptyRegionCheckItem from Critical to Warn ( #37657 )
...
close pingcap/tidb#37654
2022-09-21 21:17:03 +08:00
e2abe3df41
ddl: support create index with ingest method ( #37847 )
...
ref pingcap/tidb#35983
2022-09-16 17:18:59 +08:00
1f611c109e
lightning: add option to return partial mdl on error ( #37863 )
...
close pingcap/tidb#37861
2022-09-16 16:25:00 +08:00
57febedb35
*: enable rowserrcheck for nogo ( #37761 )
2022-09-13 14:28:58 +08:00
