Commit Graph

155 Commits

Author SHA1 Message Date
3cb091ba18 statistics: change the upper bound of the out-of-range estimation to modify count | tidb-test=pr/2012 (#39011)
close pingcap/tidb#39008
2023-01-10 15:26:23 +08:00
6af4e4b160 statistics: fix merge global stats panic if the partition hasn't been analyzed before (#39123)
close pingcap/tidb#39121
2022-11-15 18:27:55 +08:00
22b85b9b3e statistics, planner: make the content in HistColl.ColID2IdxIDs stable (#38458)
close pingcap/tidb#38457
2022-10-18 17:57:55 +08:00
7d0318cbd3 planner, statistics: refine stats loaded status and when to use pseudo stats (#37444)
close pingcap/tidb#37485
2022-09-01 20:56:24 +08:00
6e582640d7 *: upgrade glangci-linter (#37413) 2022-09-01 12:14:24 +08:00
d435267600 statistics: track used stats status during query (#37101)
ref pingcap/tidb#36481
2022-08-25 12:14:21 +08:00
96165aae7e statistics: stablize (*Table).GetColRowCount() (#37081)
close pingcap/tidb#37080
2022-08-13 00:58:48 +08:00
c63a8d83a6 statistics: support extend show histograms (#36509)
close pingcap/tidb#36481
2022-08-10 11:58:48 +08:00
212d62ee41 statistics: support tracking histogram (#36170)
ref pingcap/tidb#34052
2022-07-28 11:11:11 +08:00
34628f0f1a *: Spelling, change ture to true (#36520)
closes pingcap/tidb#36519
2022-07-27 14:35:11 +08:00
f0717dfe07 statistics, sessionctx: introduce topn assisted cardinality estimation for string matching functions (#36210)
close pingcap/tidb#36209
2022-07-20 21:55:09 +08:00
da1b82aae7 statistics: support tracking topn in statsCache (#35510)
ref pingcap/tidb#34052
2022-07-13 11:57:06 +08:00
55aea2787d *: use pdqsort to improve sort performance (#34404)
close pingcap/tidb#34250
2022-07-06 13:41:02 +08:00
aec4349cac statistics: let index support LoadNeededHistograms (#35775)
ref pingcap/tidb#35764
2022-07-04 14:03:00 +08:00
96526518b4 statistics: support tracking column cmsketch (#35135)
ref pingcap/tidb#34052
2022-06-17 17:20:35 +08:00
b24ef9a9a7 *: fix panic when mergeGlobalStats for newly-added index or column (#35066)
close pingcap/tidb#35056
2022-05-31 16:10:27 +08:00
e0c0c737a2 *: add metrics for stats healthy (#34636)
close pingcap/tidb#34691
2022-05-16 21:18:38 +08:00
0f0a2ca774 statistics: add lru cache implement for statsCacheInner (#34323)
ref pingcap/tidb#34052
2022-05-06 16:22:56 +08:00
f88dab92f9 util: use generics to implement max/min (#34197)
close pingcap/tidb#34247
2022-05-06 10:08:57 +08:00
1bf64c60f4 *: Refactor FieldType structure (#33569)
close pingcap/tidb#32116
2022-04-26 19:28:51 +08:00
ebdc78463a statistics: introduce cache interface for statsCache (#34061)
* fix

Signed-off-by: yisaer <disxiaofei@163.com>

* fix

Signed-off-by: yisaer <disxiaofei@163.com>

* fix

Signed-off-by: yisaer <disxiaofei@163.com>

* fix test

Signed-off-by: yisaer <disxiaofei@163.com>

* fix test

Signed-off-by: yisaer <disxiaofei@163.com>

* fix test

Signed-off-by: yisaer <disxiaofei@163.com>

* support forward dumpfile request

Signed-off-by: yisaer <disxiaofei@163.com>

add log

Signed-off-by: yisaer <disxiaofei@163.com>

add log

Signed-off-by: yisaer <disxiaofei@163.com>

add log

Signed-off-by: yisaer <disxiaofei@163.com>

add log

Signed-off-by: yisaer <disxiaofei@163.com>

fix get topo

Signed-off-by: yisaer <disxiaofei@163.com>

revise log

Signed-off-by: yisaer <disxiaofei@163.com>

fix lint

Signed-off-by: yisaer <disxiaofei@163.com>

fix lint

Signed-off-by: yisaer <disxiaofei@163.com>

fix lint

Signed-off-by: yisaer <disxiaofei@163.com>

revise handle error

Signed-off-by: yisaer <disxiaofei@163.com>

revise handle error

Signed-off-by: yisaer <disxiaofei@163.com>

* fix lint

Signed-off-by: yisaer <disxiaofei@163.com>

* fix lint

Signed-off-by: yisaer <disxiaofei@163.com>

* fix lint

Signed-off-by: yisaer <disxiaofei@163.com>

* remove limit

Signed-off-by: yisaer <disxiaofei@163.com>

* introduce interface

Signed-off-by: yisaer <disxiaofei@163.com>

* introduce tableMemUsage

Signed-off-by: yisaer <disxiaofei@163.com>

* add comments

Signed-off-by: yisaer <disxiaofei@163.com>

* add comments

Signed-off-by: yisaer <disxiaofei@163.com>

* address the comment

Signed-off-by: yisaer <disxiaofei@163.com>

* address the comment

Signed-off-by: yisaer <disxiaofei@163.com>

* address the comment

Signed-off-by: yisaer <disxiaofei@163.com>

Co-authored-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
2022-04-20 15:06:59 +08:00
c671ebc88b statistics: fix wrong point range in crossValidationSelectivity (#33357)
ref pingcap/tidb#28030
2022-03-28 10:52:27 +08:00
daf525a17e *: replace compareDatum by compare, range part (#30710) 2021-12-16 19:48:36 +08:00
2f42f7c0f6 planner: unify the argument of stats functions to use SessionCtx instead of StatementContext (#30668) 2021-12-14 19:36:35 +08:00
aa002f28a2 *: show cmd to check if all needed histograms are loaded (#29672) 2021-12-10 13:45:57 +08:00
f82ca6a8eb statistics, util/ranger: add cardinality estimation trace for GetRowCountBy... (#30321) 2021-12-07 19:45:56 +08:00
6ed0134902 statistics: pseudo table should not contain the hidden columns (#29584) 2021-11-09 16:27:05 +08:00
efa94595c0 *: enable golangci-lint unconvert (#28929) 2021-10-23 21:28:47 +08:00
xhe
94e30df8e2 parser: fix compiling, add unit tests (#28700) 2021-10-11 23:15:25 +08:00
7755d25aba *: Integrate SkyWalking-eyes to check license headers in CI and fix licenses headers (#27198) 2021-08-16 18:52:00 +08:00
cb65b7a2f7 statistics: improve out-of-range estimation strategy (#26502) 2021-08-02 17:43:08 +08:00
4eeff54d85 statistics: fix the fomula for checking outdated stats (#26728) 2021-07-29 22:34:17 +08:00
50ae2d8266 statistics: trigger auto-analyze based on histogram row count (#24382) 2021-07-28 18:02:17 +08:00
32cf14bd7d statistics: relax the check of the OutOfRange (#24958) 2021-06-02 22:16:25 +08:00
734084fa2b statistics: skip reading mysql.stats_histograms if cached stats is up-to-date (#24175) 2021-04-22 17:37:54 +08:00
bd12ea96cc statistics: use index fm-sketches instead of bucket NDV to calculate global NDV for indexes (#23962) 2021-04-15 14:59:55 +08:00
550ca8eaeb statistics: enables global-level stats to be generated in fast analyze mode (#22931) 2021-03-01 16:40:50 +08:00
270d497848 statistics: introduce new estimation logic when index histogram fails to estimate (#22682)
* statistics: introduce new estimation logic when index histogram fail to estimate

* address comments

* address comments

Co-authored-by: Ti Chi Robot <71242396+ti-chi-bot@users.noreply.github.com>
2021-02-24 16:30:03 +08:00
f3fe4895cb statistics: support to store FMSketch and add FMSketch to column stats (#22841)
* add FMSketch to mysql.stats_histograms

* statistics: support store FMSketch and add FMSketch to histogram

* FIX UT

* Create Stats_FM_Sketch Table

* fix UT

* add some tests

* FIX ut

* move fmSketch from histogram to column.stats

* remove unused code and comments

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* fix ut

* finish the FMSketch.MemoryUsage function

* add some comments for FMSketch.MemoryUsage function

* fix typo

Co-authored-by: ti-srebot <66930949+ti-srebot@users.noreply.github.com>
Co-authored-by: Ti Chi Robot <71242396+ti-chi-bot@users.noreply.github.com>
2021-02-23 10:39:09 +08:00
2d717c12b3 statistics: merge the partition-level stats to global-level stats (#22667) 2021-02-05 11:42:41 +08:00
343c412531 planner, executor, statistics: add tests for version 2 and fix bugs (#22461)
Co-authored-by: ti-srebot <66930949+ti-srebot@users.noreply.github.com>
2021-02-04 15:52:30 +08:00
f22ec62dd0 statistics: fix panic occurs when stats cache inconsistency (#22465) 2021-01-27 16:41:07 +08:00
8d2ab4ef7a statistics: fix inaccurate row count estimation caused by cross validation using invalid column stats (#22479) 2021-01-22 17:42:37 +08:00
3dd842f50a statistics: add bucket ndv for index histogram (#20580)
Co-authored-by: Yuanjia Zhang <zhangyuanjia@pingcap.com>
Co-authored-by: ti-srebot <66930949+ti-srebot@users.noreply.github.com>
2021-01-13 18:24:21 +08:00
76054200b6 statistics: redesign the schema for mysql.stats_extended (#22033)
Co-authored-by: ti-srebot <66930949+ti-srebot@users.noreply.github.com>
2021-01-04 17:57:24 +08:00
092a01a0a4 statistics, executor: refactor statistics on columns (#21817) 2020-12-24 13:56:30 +08:00
5acff8de7a *: Revert "#19008" and "#18788" (#21599) 2020-12-14 15:06:35 +08:00
b4eee287af *: extract topn out of histogram correctly (#21007)
Co-authored-by: Yuanjia Zhang <zhangyuanjia@pingcap.com>
2020-12-10 17:05:39 +08:00
152b60ebe4 session: increase cm_sketch column max length to 6291456(6MB) (#20152) 2020-11-02 15:21:25 +08:00
c2e3b0a6ad statistics: decouple TopN from CMSKetch (#20572) 2020-10-28 15:16:42 +08:00