tidb/roadmap.md

TiDB Roadmap

This roadmap brings you what's coming in the 1-year future, so you can see the new features or improvements in advance, follow the progress, learn about the key milestones on the way, and give feedback as the development work goes on.

In the course of development, this roadmap is subject to change based on user needs and feedback. If you have a feature request or want to prioritize a feature, please file an issue on GitHub.

TiDB kernel

Scenario	Feature	Description	Estimated Time of Delivery
Support JSON	Support JSON function.	In business scenarios that require flexible schema definitions (such as SaaS, Web3, and gaming), the application can use JSON to store information for ODS, transaction indicators, commodities, game characters, and props.	September 2022
	Support expression indexes. Support generated columns.	Provide query acceleration for specific field indexes in JSON scenarios.	October 2022
Flashback	Support cluster-level flashback.	In game rollback scenarios, the flashback can be used to achieve a fast rollback of the current cluster. This solves the common problems in the gaming industry such as version errors and bugs.	November 2022
TiFlash result write-back (supports INSERT INTO SELECT)	Easily write the analysis results in TiFlash back to TiDB. Provide complete ACID transactions, more convenient and reliable than general ETL solutions. Set a hard limit on the threshold of intermediate result size, and report an error if the threshold is exceeded. Support fully distributed transactions, and remove or relax the limit on the intermediate result size.	These features combined enable a way to materialize intermediate results. The analysis results can be easily reused, which reduces unnecessary ad-hoc queries, improves the performance of BI and other applications (by pulling results directly) and reduces system load (by avoiding duplicated computation), thereby improving the overall data pipeline efficiency and reducing costs. It will make TiFlash an online service.	November 2022
Time to live (TTL)	Support automatically deleting expired table data based on custom rules.	This feature enables automatic data cleanup in limited data archiving scenarios.	December 2022
Multi-value Index	Support array index.	Array is one of the commonly used data types in JSON scenarios. For inclusive queries in arrays, multi-value indexes can efficiently improve the query speed.	December 2022
TiFlash kernel optimization	FastScan provides weak consistency but faster table scan capability. Further optimize the join order, shuffle, and exchange algorithms to improve computing efficiency and boost performance for complex queries. Add a fine-grained data sharding mechanism to optimize the COUNT(DISTINCT) function and high cardinality aggregation.	Improve the basic computing capability of TiFlash, and optimize the performance and reliability of the underlying algorithms of the columnar storage and MPP engine.	December 2022
TiDB proxy	Implement automatic load balancing so that upgrading a cluster or modifying configurations does not affect the application. After scaling out or scaling in the cluster, the application can automatically rebalance the connection without reconnecting.	In scenarios such as upgrades and configuration changes, TiDB proxy is more business-friendly.	December 2022
PB-level scalability	Support huge region size	Scenarios with fast business growth and a large amount of data	January 2023
DDL parallel framework	Implement a single DDL parallel execution framework for DDL acceleration	Accelerate DDL in scenarios such as creating indexes for large tables, modifying a column's default value, and modifying a column type.	January 2023
Non-prepared Plan Cache	Support plan cache for general SQL statements in a session to save cache resources, improve the hit rate of general execution plans, and improve SQL performance.	Non-prepared plan cache. Improve real-time and throughputs of OLTP in general scenarios, save PoC time, and increase PoC win rate.	January 2023
SQL blocklist	Support a rule-based SQL blocklist mechanism.	In multi-service aggregation scenarios, provide SQL management and control capabilities, and improve cluster stability by prohibiting high-resource-consuming SQL statements.	February 2023
Resource management	Provide a basic resource management and control framework to effectively control the resource squeeze of background tasks on front-end tasks (user operations), and improve cluster stability.	Refine resource management in the multi-service aggregation scenario.	March 2023
Prepared Plan Cache	Support in-session subquery, expression index, and prepared plan cache for Partition.	Expand the usage scenarios of plan cache.	April 2023
PB-level scalability	Support dynamic region size adjustment (heterogeneous).	For scenarios with fast business growth and a large amount of data.	June 2023
Instance plan cache	Support cross-session plan cache, save cache resources, improve the hit rate of general execution plans, and improve SQL performance.	In general scenarios, reuse execution plans to improve memory utilization and to achieve higher throughputs.	July 2023

Diagnosis and maintenance

Scenario	Feature	Description	Estimated Time of Delivery
SQL tuning for HTAP workloads	Provide SQL execution information from the perspective of applications. Provide suggestions on optimizing SQL for TiFlash and TiKV in HTAP workloads.	Provide a dashboard that displays a SQL execution overview from the perspective of applications in HTAP workloads. For one or several HTAP scenarios, provide suggestions on SQL optimization.	October 2022

Data backup and migration

Scenario	Feature	Description	Estimated Time of Delivery
Backup and restore	AWS EBS snapshot-based backup	BR provides an MVP version of AWS EBS snapshot-based backup and restore.	September 2022
	Backup and restore based on AWS EBS or GCP persistent disk snapshots	BR provides a complete version of EBS snapshot-based backup and restore.	December 2022
Point-in-time recovery (PITR)	Table-level and database-level PITR	BR supports table-level or database-level PITR.	November 2022
Data replication to downstream systems via TiCDC	Reduce TiCDC replication latency in planned offline scenarios.	When TiKV, TiDB, PD, or TiCDC nodes are offline in a planned maintenance window, the replication latency of TiCDC can be reduced to less than 10 seconds.	October 2022
	Support replicating data to object storage such as S3.	TiCDC supports replicating data changes to common object storage services.	November 2022
Data migration	TiDB Lightning supports table-level and partition-level incremental data import.	TiDB Lightning provides comprehensive table-level and partition-level data import capabilities.	December 2022

Security

Scenario	Feature	Description	Estimated Time of Delivery
ShangMi (SM) algorithms	Encryption-at-rest (TiKV and TiFlash) supports the SM4 algorithm.	Supports encrypting data stored in TiKV and TiFlash based on the SM4 algorithm.	September 2022
	TiDB authentication supports the SM3 algorithm.	Provide a user authentication plugin based on the SM3 algorithm, which encrypts the password using the SM3 algorithm.	September 2022
Log redaction	Data redaction in execution plans in TiDB Dashboard Data redaction in TiDB-related logs	Redact sensitive information in execution plans and various logs to enhance the security of user data.	October 2022
Password complexity check	A strong password is required.	To improve security, empty passwords and weak passwords are not allowed. The required password length is not less than 8. The password must contain an uppercase letter, a lowercase letter, a number, and a character.	November 2022
Password expiration	TiDB provides password expiration management and requires users to change passwords regularly.	Reduce the security risk of password cracking or leakage caused by using the same password for a long time.	December 2022
Password policy management	TiDB provides a password reuse mechanism and brute-force cracking prevention capabilities.	TiDB supports password policy management to protect password security.	March 2023
Column-level access control	TiDB supports column-level privilege management.	TiDB already supports cluster-level, database-level, and table-level privilege management. On top of that, TiDB will support column-level privilege management to meet the principle of least privilege and provide fine-grained data access control.	June 2023
Audit logging capability refactor	Support configurable audit log policies, configurable audit filters (filter by objects, users, and operation types), and visual access to audit logs.	Improve the completeness and usability of the audit log feature.	July 2023