Commit Graph

2244 Commits

Author SHA1 Message Date
a68ac9e73d exectuor: reduce listInDisk.GetChunk memory usage(constructing of chunk in rows) (#36738)
close pingcap/tidb#35636
2022-08-02 14:46:05 +08:00
xhe
9a16a8dd19 *: replace defer clean with t.Cleanup (#36722)
close pingcap/tidb#36721
2022-08-01 17:34:05 +08:00
fc217d432c config, sysvar: add config instance.enable_ddl and sysvar tidb_enable_ddl (#35425)
ref pingcap/tidb#34960
2022-08-01 15:02:06 +08:00
158ba1a1d9 *: enable part revive for all code (#36703) 2022-08-01 14:02:06 +08:00
0aca188e7e util/mock: set sessionvars EnableChunkRPC to true for mock.Context (#36726)
close pingcap/tidb#36725
2022-07-30 14:16:03 +08:00
8e77ce264c *: introduce @@tidb_max_paging_size system variable for tuning (#36599)
close pingcap/tidb#36328
2022-07-29 15:29:12 +08:00
e3ceffa580 ddl: invisibility for plannner (#36634) 2022-07-28 13:45:10 +08:00
212d62ee41 statistics: support tracking histogram (#36170)
ref pingcap/tidb#34052
2022-07-28 11:11:11 +08:00
678bbeefd0 admin: rename the same name field in the log (#36626) 2022-07-27 21:39:11 +08:00
b70207d849 planner/core, util/plancodec: fix wrongly encoded plan which might cause panic (#36611)
close pingcap/tidb#36609
2022-07-27 19:35:11 +08:00
f22098cc5d br: don't encode key of raw kv. (#36491)
ref pingcap/tidb#36490
2022-07-27 19:05:11 +08:00
9ee028002e dbutil: fix wrong syntax for SHOW GRANTS USING (#36565)
close pingcap/tidb#36564
2022-07-26 21:57:11 +08:00
b0c6c5afa7 *: cleanup default sysvar value usage (#36487)
close pingcap/tidb#36485
2022-07-26 19:51:10 +08:00
bf743a6ca0 *: enable flaky test for all test (#36385)
close pingcap/tidb#36404
2022-07-22 15:35:09 +08:00
0b1ad27b1a server: skip check tiflash version (#36451)
close pingcap/tidb#36449
2022-07-22 13:01:09 +08:00
ab513bf0ca ddl: support concurrent ddl (#32169)
* 1. init ddl tables

create tidb_ddl_job, tidb_ddl_reorg, tidb_ddl_history tables with raw meta write, these 3 tables is use to replace the ddl job queue and reorg and history hash table

* 2. setup concurrent ddl env and add ddl worker pool

adds the ddl worker pool definition, the ddl job manager will find a job and ship it to a worker in the worker pool.
Also, this commit provides a sessionctx wrapper, only use in ddl relate. it just wraps begin, commit and execute

* 3. add ddl manager to handle ddl job

* 4. reorg handler for concurrent ddl

just implements the partner of the reorg information.

* 5. manage ddl jobs for concurrent ddl

add the partner of add job, delete job and many others related to history job
because many of the functions need a session now, we just change the caller

* 6. add metrics for concurrent ddl

add metrics

* 7. support multiple tables

* 8. fix test

* 9. migrate ddl between table and queue

support switch between the old and new ddl framework, migrate the existing ddl job between queue and table

* 10. check tikv version and set reorg worker count according cpu count

* *: add featuretag on tests

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

* use a determined table id for 3 tables

* remove ctx value

* add GetSchemaVersionWithNonEmptyDiff function

* address tangenta and zimulala comment

* use only one etcd path

* make ActionRenameTable support multi-schema

* reset sql digest to make top sql work correct

* add comment

* fix test

* remove 0 for schema version lock

Co-authored-by: xiongjiwei <xiongjiwei1996@outlook.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
Co-authored-by: wjHuang <huangwenjun1997@gmail.com>
2022-07-20 21:20:07 +08:00
fd6f5dd798 *: GC-aware memory track for analyze (#35248)
close pingcap/tidb#34996
2022-07-20 20:17:09 +08:00
a52656fde4 *: add binary plan in stmt summary and slow log/query (#35698)
close pingcap/tidb#35889
2022-07-20 18:21:09 +08:00
d00b984f4c executor, statistics: analyze use MaxUint64 ts to read data (#35232)
close pingcap/tidb#35233
2022-07-20 16:09:08 +08:00
0cecfc85af util/paging: choose min paging size default value as 128, and max value as 8192 (#36331)
close pingcap/tidb#36328
2022-07-19 21:57:08 +08:00
d48f40a09c util, session: enable paging for more ut (#36259)
ref pingcap/tidb#35633
2022-07-19 17:33:08 +08:00
8af8341cca planner: Table scan in TiFlash not keep order (#36232)
close pingcap/tidb#36231
2022-07-19 00:51:07 +08:00
8b30e52bcf ddl: only use one schema version for the non-revertible step (#36262)
ref pingcap/tidb#14766
2022-07-18 21:49:07 +08:00
4cade24ed6 *: support the binary plan through EXPLAIN format='binary' and tidb_decode_binary_plan (#36221)
ref pingcap/tidb#35889
2022-07-18 16:27:07 +08:00
df31ac65a4 execdetails: output write details in RuntimeStats (#35814)
ref pingcap/tidb#34487
2022-07-18 15:01:07 +08:00
f675cf619b ddl: ban set tiflash mode under unsupported case (#36211)
ref pingcap/tidb#35850
2022-07-18 09:53:06 +08:00
911e7cc360 *: enable revive (#35970)
ref pingcap/tidb#35345
2022-07-15 16:03:06 +08:00
ce4edc9df4 planner: fix optimizer trace may miss operators in some cases (#35058)
close pingcap/tidb#31811
2022-07-15 13:59:05 +08:00
7c72a53319 util: use pdqsort to replace sortedMapSlice (#36067)
close pingcap/tidb#36063
2022-07-15 12:13:05 +08:00
fa00fc5cd5 *: set fine grain timeout and update bazel config (#36220) 2022-07-15 11:11:05 +08:00
6da1cbec77 chunk: refactor test to fix unstable test (#36217) 2022-07-14 19:27:06 +08:00
7953d60c1b *: switch to flat plan in stmt summary, slow log, etc (#36069)
ref pingcap/tidb#35888
2022-07-14 17:37:06 +08:00
bbc2e655d1 log: add get-latest-ts duration of async commit and 1pc into slowlog (#36180)
ref pingcap/tidb#34106
2022-07-14 14:03:05 +08:00
d93bc7a4b0 table-filter: optimize table pattern message and unit tests (#36160)
close pingcap/tidb#36163
2022-07-13 20:31:05 +08:00
bdc6397023 executor: support tidb memory debug mode (#35322)
ref pingcap/tidb#33877
2022-07-13 12:33:06 +08:00
7458c3ec2e sessionctx: support signing and checking tokens (#36030)
close pingcap/tidb#36029
2022-07-12 20:33:05 +08:00
d10d254572 util: let TypeFloat should be decoded as Float32 in Chunk (#35978)
close pingcap/tidb#35910
2022-07-11 18:57:05 +08:00
1a9a7a614c statistics,planner: let index support sync stats (#35879)
ref pingcap/tidb#35764
2022-07-11 17:23:05 +08:00
756a817014 *: fix a data race on TestConnExecutionTimeout (#35923)
close pingcap/tidb#35922
2022-07-11 13:35:05 +08:00
d941cb9dea distsql, execdetails: add information into runtime stats (#35993)
ref pingcap/tidb#35889
2022-07-08 15:59:05 +08:00
f3eb0f5c26 txn: remove NewTxn and NewStaleTxnWithStartTS in session (#35885)
close pingcap/tidb#35884
2022-07-07 19:27:02 +08:00
ec9f201294 topsql: compress plan instead of dropping it (#35973)
ref pingcap/tidb#35964
2022-07-06 16:03:02 +08:00
55aea2787d *: use pdqsort to improve sort performance (#34404)
close pingcap/tidb#34250
2022-07-06 13:41:02 +08:00
114c922bc8 *: enable gosec (#35873)
ref pingcap/tidb#35345
2022-07-05 19:39:01 +08:00
d617ba2a82 plancodec: fix a miss case in conversion (#35957)
close pingcap/tidb#35956
2022-07-05 17:43:01 +08:00
a4fd5cd4d9 util: mathutil type parameters name use uppercase (#35896)
close pingcap/tidb#35804
2022-07-05 11:53:01 +08:00
e50b9430fc *: enable gci (#35823)
ref pingcap/tidb#35345
2022-07-04 14:53:02 +08:00
e00ad2b3ef *: enable errcheck for nogo (#35704)
ref pingcap/tidb#35345
2022-07-02 22:08:39 +08:00
65e0b8e541 client: KV requests with source label (#34833)
close pingcap/tidb#33963
2022-07-01 19:46:39 +08:00
16e5815f18 txn: txnManager providers GetReadSnapshot and GetForUpdateSnapshot. (#35788)
close pingcap/tidb#35810
2022-07-01 18:12:40 +08:00