Commit Graph

510 Commits

Author SHA1 Message Date
b0c6c5afa7 *: cleanup default sysvar value usage (#36487)
close pingcap/tidb#36485
2022-07-26 19:51:10 +08:00
f0717dfe07 statistics, sessionctx: introduce topn assisted cardinality estimation for string matching functions (#36210)
close pingcap/tidb#36209
2022-07-20 21:55:09 +08:00
ab513bf0ca ddl: support concurrent ddl (#32169)
* 1. init ddl tables

create tidb_ddl_job, tidb_ddl_reorg, tidb_ddl_history tables with raw meta write, these 3 tables is use to replace the ddl job queue and reorg and history hash table

* 2. setup concurrent ddl env and add ddl worker pool

adds the ddl worker pool definition, the ddl job manager will find a job and ship it to a worker in the worker pool.
Also, this commit provides a sessionctx wrapper, only use in ddl relate. it just wraps begin, commit and execute

* 3. add ddl manager to handle ddl job

* 4. reorg handler for concurrent ddl

just implements the partner of the reorg information.

* 5. manage ddl jobs for concurrent ddl

add the partner of add job, delete job and many others related to history job
because many of the functions need a session now, we just change the caller

* 6. add metrics for concurrent ddl

add metrics

* 7. support multiple tables

* 8. fix test

* 9. migrate ddl between table and queue

support switch between the old and new ddl framework, migrate the existing ddl job between queue and table

* 10. check tikv version and set reorg worker count according cpu count

* *: add featuretag on tests

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

* use a determined table id for 3 tables

* remove ctx value

* add GetSchemaVersionWithNonEmptyDiff function

* address tangenta and zimulala comment

* use only one etcd path

* make ActionRenameTable support multi-schema

* reset sql digest to make top sql work correct

* add comment

* fix test

* remove 0 for schema version lock

Co-authored-by: xiongjiwei <xiongjiwei1996@outlook.com>
Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
Co-authored-by: wjHuang <huangwenjun1997@gmail.com>
2022-07-20 21:20:07 +08:00
fbbae462f1 planner: expose two variables related to the cost model (#36356)
ref pingcap/tidb#35240
2022-07-20 20:45:09 +08:00
fd6f5dd798 *: GC-aware memory track for analyze (#35248)
close pingcap/tidb#34996
2022-07-20 20:17:09 +08:00
a52656fde4 *: add binary plan in stmt summary and slow log/query (#35698)
close pingcap/tidb#35889
2022-07-20 18:21:09 +08:00
d00b984f4c executor, statistics: analyze use MaxUint64 ts to read data (#35232)
close pingcap/tidb#35233
2022-07-20 16:09:08 +08:00
afd71bdefd executor: support dispatch cop request to closest replica adaptively (#35927)
ref pingcap/tidb#35926
2022-07-19 17:59:07 +08:00
67be460efe planner: enable the new cost interface by default (#36241)
ref pingcap/tidb#35240
2022-07-18 21:21:08 +08:00
c28bac3d72 planner: support rewrite for skew group distinct aggregate to avoid skew execution (#36181)
close pingcap/tidb#36169
2022-07-18 15:29:07 +08:00
9a2ed52119 server, sessionctx: support token-based authentication (#36152)
close pingcap/tidb#35913
2022-07-14 18:29:06 +08:00
2193bfe3b7 *: remove system var tidb_enable_change_multi_schema (#36167)
ref pingcap/tidb#14766
2022-07-14 10:37:05 +08:00
0b427e1fd6 *: add tidb_min_paging_size system variable (#36107)
close pingcap/tidb#36106
2022-07-13 22:27:05 +08:00
bdc6397023 executor: support tidb memory debug mode (#35322)
ref pingcap/tidb#33877
2022-07-13 12:33:06 +08:00
dced89a231 planner/core, planner, sessionctx/variable: remove more skipInit (#35992)
ref pingcap/tidb#35051
2022-07-13 11:27:05 +08:00
2eb101abcf sessionctx/variable: remove skipInit from non-session scoped variables (#35801)
ref pingcap/tidb#35051
2022-07-07 00:01:02 +08:00
a5e973ca5a sessionctx/variable: convert TiDBLogFileMaxDays and TiDBConfig to ScopeInstance (#35733)
close pingcap/tidb#35190
2022-07-06 17:11:02 +08:00
cdde0398fe sessionctx,kv,planner: add system variable for fine_grained_shuffle (#35256)
close pingcap/tidb#35342
2022-07-06 10:27:02 +08:00
e2b55640c1 config, sysvar: map max-server-connections to max_connections (#35453)
ref pingcap/tidb#34960
2022-06-23 15:12:37 +08:00
193f04db70 session,sessionctx: enable tidb_enable_paging by default (#35275)
close pingcap/tidb#35273
2022-06-23 08:32:37 +08:00
e0527ba27c sessionctx: support encoding and decoding session variables (#35531)
close pingcap/tidb#35414
2022-06-21 19:20:36 +08:00
f89466daa2 sessionctx: set skipInit false for TiDBOptProjectionPushDown and TiDBOptAggPushDown (#35491)
close pingcap/tidb#35083
2022-06-21 15:38:37 +08:00
5c9ad77ae1 *: add support for disabling noop variables (#35496)
close pingcap/tidb#8895
2022-06-21 14:10:36 +08:00
428
79d9d6fa75 executor, sessionctx/variable: convert some system variables to different builtin types (#35143)
close pingcap/tidb#35048
2022-06-17 02:34:34 +08:00
081bd10d54 system_variables: remove the limitation for set tidb_gc_life_time (#35411)
close pingcap/tidb#35392
2022-06-16 18:18:35 +08:00
017ac90e67 planner: introduce some new variables for the cost model version2 (#35241)
ref pingcap/tidb#35240
2022-06-09 17:52:30 +08:00
4d8cf5e1e2 metrics: add a config to unregister the unused metrics (#34990)
close pingcap/tidb#34989
2022-06-09 15:34:31 +08:00
c8fcad4d82 planner: refactor to use the standard function GetXXFactor() to get cost model factors (#35226)
ref pingcap/tidb#33945
2022-06-08 16:10:30 +08:00
7ffab33a47 variables: add concurrent DDL switch (#35028)
ref pingcap/tidb#32031
2022-06-06 14:56:29 +08:00
233a7335fd sessionctx/variable, ddl: fix sysvar API usage (#35052)
ref pingcap/tidb#35051
2022-06-06 10:34:29 +08:00
e2fd394fbf table partition: add more warnings to inform the user that setting TiDB partition prune mode to dynamic at session level (#35046)
close pingcap/tidb#35042
2022-05-31 18:04:27 +08:00
dfb22c064d config, executor, sessionctx/variable: Remove old config items (#34927)
close pingcap/tidb#34867, close pingcap/tidb#34879, close pingcap/tidb#34945
2022-05-29 05:22:48 +08:00
43f83c0799 *: kill auto analyze when it runs too long (#34950)
close pingcap/tidb#34952
2022-05-27 13:36:47 +08:00
dae0a4104a sessionctx: enable list partition by default (#34954)
close pingcap/tidb#34953
2022-05-27 12:22:47 +08:00
6a0239362d planner: use variable to control outer join reorder for fallback (#34825)
ref pingcap/tidb#19940
2022-05-19 23:14:39 +08:00
0cbccb74f3 *: move config file options prepared-plan-cache.* to sysvars (#34790)
close pingcap/tidb#30168, ref pingcap/tidb#33769
2022-05-19 16:18:39 +08:00
2f4b11dc26 planner: disable the new cost implementation by default (#34810) 2022-05-19 16:01:16 +08:00
2cf1354f72 *: global stats consistency for dynamic prune (#34695)
close pingcap/tidb#34730
2022-05-19 11:34:38 +08:00
a9614850f9 *: move config file option oom-action to sysvar (#34644)
ref pingcap/tidb#33769
2022-05-19 05:16:38 +08:00
eae2ae84f4 *: move config file option tidb_enable_auto_analyze to sysvar (#34643)
ref pingcap/tidb#33769
2022-05-19 02:34:38 +08:00
c154d4f864 variable: fix 34773 race error (#34775)
close pingcap/tidb#34773
2022-05-18 15:44:38 +08:00
95e359ff4c variable: revise TiDBStatsCacheMemQuota settting code (#34762)
ref pingcap/tidb#34052
2022-05-18 13:26:38 +08:00
d2ada35afa *: move config file option require_secure_transport to sysvar (#34261)
ref pingcap/tidb#33769
2022-05-17 22:02:37 +08:00
09b155f7a3 distsql, sessionctx/variable: add max_threads mysql variable for TiFlash (#34299)
close pingcap/tidb#34298
2022-05-17 19:00:38 +08:00
683ba092b5 planner: enable the new cost implementation by default (#34567)
close pingcap/tidb#33945
2022-05-17 00:02:38 +08:00
0017963ddd *: move config file option committer-concurrency to sysvar (#34368)
ref pingcap/tidb#33769
2022-05-14 04:36:37 +08:00
bbfbe137eb *: move config file option enable-batch-dml to sysvar (#33803)
ref pingcap/tidb#33769
2022-05-13 22:12:37 +08:00
98c31070d9 sysvar, sysvar_test: fix tidb_mem_quota_query min/max value (#34554)
fixes pingcap/tidb#34553
2022-05-12 00:08:35 +08:00
53f228af45 sysvari: enable statsCache lru by config (#34278)
* add statsquota variable

Signed-off-by: yisaer <disxiaofei@163.com>

* add test

Signed-off-by: yisaer <disxiaofei@163.com>

* address the comment

Signed-off-by: yisaer <disxiaofei@163.com>

* address the comment

Signed-off-by: yisaer <disxiaofei@163.com>

* address the comment

Signed-off-by: yisaer <disxiaofei@163.com>

* fix conflict

Signed-off-by: yisaer <disxiaofei@163.com>

* address the comment

Signed-off-by: yisaer <disxiaofei@163.com>

address the comment

Signed-off-by: yisaer <disxiaofei@163.com>

fix duplicated plan

Signed-off-by: yisaer <disxiaofei@163.com>

fix duplicated plan

Signed-off-by: yisaer <disxiaofei@163.com>

fix duplicated plan

Signed-off-by: yisaer <disxiaofei@163.com>

* fix duplicated plan

Signed-off-by: yisaer <disxiaofei@163.com>

* fix duplicated plan

Signed-off-by: yisaer <disxiaofei@163.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
2022-05-10 23:49:54 +08:00
33236ba8d7 *: track analyze memory usage separately with quota (#33054)
ref pingcap/tidb#29938, close pingcap/tidb#33065
2022-05-10 20:32:35 +08:00