TiDB 4.0.2 Release Notes

Release date: July 1, 2020

TiDB version: 4.0.2

Compatibility Changes

• TiDB

  • Remove sensitive information in the slow query log and the statement summary table #18130
  • Forbid negative value in the sequence cache #18103
  • Remove tombstone TiKV and TiFlash stores from the CLUSTER_INFO table #17953
  • Change the diagnostic rule from current-load to node-check #17660

• PD

  • Persist store-limit and remove store-balance-rate #2557

New Change

• By default, TiDB and TiDB Dashboard share usage details with PingCAP to help understand how to improve the product #18180. For details about what is shared and how to disable the sharing, see Telemetry.

New Features

• TiDB

  • Support the MEMORY_QUOTA() hint in INSERT statements #18101
  • Support authentication based on the SAN field of TLS certificate #17698
  • Support collation for the REGEXP() function #17581
  • Support the sql_select_limit session and global variable #17604
  • Support splitting the Region for the newly added partition by default #17665
  • Support pushing the IF()/BITXOR()/BITNEG()/JSON_LENGTH() functions to the TiFlash Coprocessor #17651 #17592
  • Support a new aggregate function APPROX_COUNT_DISTINCT() to calculate the approximate result of COUNT(DISTINCT) #18120
  • Support collation in TiFlash and pushing collation-related functions to TiFlash #17705
  • Add the STATUS_ADDRESS column in the INFORMATION_SCHEMA.INSPECTION_RESULT table to indicate the status address of servers #17695
  • Add the SOURCE column in the MYSQL.BIND_INFO table to indicate the how the bindings are created #17587
  • Add the PLAN_IN_CACHE and PLAN_CACHE_HITS columns in the PERFORMANCE_SCHEMA.EVENTS_STATEMENTS_SUMMARY_BY_DIGEST table to indicate the plan cache usage of SQL statements #17493
  • Add the enable-collect-execution-info configuration item and the tidb_enable_collect_execution_info session variable to control whether to collect execution information of each operator and record the information in the slow query log #18073 #18072
  • Add the tidb_slow_log_masking global variable to control whether to desensitize the queries in slow query log #17694
  • Add a diagnostic rule in the INFORMATION_SCHEMA.INSPECTION_RESULT table for the storage.block-cache.capacity TiKV configuration item #17671
  • Add the BACKUP and RESTORE SQL statements to back up and restore data #15274

• TiKV

  • Support the encryption-meta command in TiKV Control #8103
  • Add a perf context metric for RocksDB::WriteImpl #7991

• PD

  • Support the operator to fail immediately when trying to remove a leader peer #2551
  • Set a suitable default store limit for TiFlash stores #2559

• TiFlash

  • Support new aggregation function APPROX_COUNT_DISTINCT in Coprocessor
  • Enable the rough set filter feature by default
  • Enable TiFlash to run on the ARM architecture
  • Support pushing down the JSON_LENGTH function in Coprocessor

• Tools

  • TiCDC

    • Support migrating sub-tasks to new captures #665
    • Add a cli command to delete the TiCDC GC TTL #652
    • Support canal protocol in MQ sink #649

Improvements

• TiDB

  • Reduce the query latency caused by the Golang memory allocation when CM-Sketch consumes too much memory #17545
  • Reduce the QPS recovery duration of a cluster when a TiKV server is in the failure recovery process #17681
  • Support pushing aggregate functions to TiKV/TiFlash Coprocessor on partition tables #17655
  • Improve the accuracy of row count estimation for index equal conditions #17611

• TiKV

  • Improve the PD client panic log #8093
  • Add back the process_cpu_seconds_total and process_start_time_seconds monitoring metrics #8029

• TiFlash

  • Improve backward compatibility when upgrading from an older version #786
  • Reduce memory consumption of delta index #787
  • Use the more efficient update algorithm for delta index #794

• Tools

  • Backup & Restore (BR)

    • Improve the performance by pipelining the restore process #266

Bug Fixes

• TiDB

  • Fix the issue of incorrect execution plan obtained from the plan cache after tidb_isolation_read_engines is changed #17570
  • Fix the occasional runtime error that occurs when executing the EXPLAIN FOR CONNECTION statement #18124
  • Fix the incorrect result of the last_plan_from_cache session variable in some cases #18111
  • Fix the runtime error that occurs when executing the UNIX_TIMESTAMP() function from the plan cache #18002 #17673
  • Fix the runtime error when the child of HashJoin executor returns the NULL column #17937
  • Fix the runtime error caused by concurrently executing the DROP DATABASE statement and other DDL statements in the same database #17659
  • Fix the incorrect result of the COERCIBILITY() function on user variables #17890
  • Fix the issue that the IndexMergeJoin executor occasionally gets stuck #18091
  • Fix the hang issue of the IndexMergeJoin executor when out of memory quota and query cancelling is triggered #17654
  • Fix the excessive counting memory usage of the Insert and Replace executors #18062
  • Fix the issue that the data replication to TiFlash storage is stopped when DROP DATABASE and DROP TABLE are executed concurrently in the same database #17901
  • Fix the BACKUP/RESTORE failure between TiDB and the object storage service #17844
  • Fix the incorrect error message of privilege check failure when access is denied #17724
  • Discard the query feedbacks generated from the DELETE/UPDATE statement #17843
  • Forbid altering AUTO_RANDOM_BASE for a table without AUTO_RANDOM property #17828
  • Fix the issue that the AUTO_RANDOM column is allocated wrong results when the table is moved between databases by ALTER TABLE ... RENAME #18243
  • Fix the issue that some system tables cannot be accessed when setting the value of tidb_isolation_read_engines without tidb #17719
  • Fix the inaccurate result of JSON comparison on large integers and float values #17717
  • Fix the incorrect decimal property for the result of the COUNT() function #17704
  • Fix the incorrect result of the HEX() function when the type of input is the binary string #17620
  • Fix the issue that an empty result is returned when querying the INFORMATION_SCHEMA.INSPECTION_SUMMARY table without filter condition #17697
  • Fix the issue that the hashed password used by the ALTER USER statement to update user information is unexpected #17646
  • Support collation for ENUM and SET values #17701
  • Fix the issue that the timeout mechanism for pre-splitting Regions does not work when creating a table #17619
  • Fix the issue that the schema is unexpectedly updated when a DDL job is retried, which might break the atomicity of DDL jobs #17608
  • Fix the incorrect result of the FIELD() function when the argument contains the column #17562
  • Fix the issue that the max_execution_time hint does not work occasionally #17536
  • Fix the issue that the concurrency information is redundantly printed in the result of EXPLAIN ANALYZE #17350
  • Fix the incompatible behavior of %h on the STR_TO_DATE function #17498
  • Fix the issue that the follower/learner keeps retrying when tidb_replica_read is set to follower and there is a network partition between the leader and the follower/learner #17443
  • Fix the issue that TiDB sends too many pings to PD follower in some cases #17947
  • Fix the issue that the range partition table of older versions cannot be loaded in TiDB v4.0 #17983
  • Fix the SQL statement timeout issue when multiple Region requests fail at the same time by assigning different Backoffer for each Region #17585
  • Fix the MySQL incompatible behavior when parsing DateTime delimiters #17501
  • Fix the issue that TiKV requests are occasionally sent to the TiFlash server #18105
  • Fix the data inconsistency issue occurred because the lock of a written and deleted primary key in one transaction is resolved by another transaction #18250

• TiKV

  • Fix a memory safety issue for the status server #8101
  • Fix the issue of lost precision in JSON numeric comparison #8087
  • Fix the wrong query slow log #8050
  • Fix the issue that a peer cannot be removed when its store is isolated during multiple merge processes #8048
  • Fix the issue that tikv-ctl recover-mvcc does not remove invalid pessimistic locks #8047
  • Fix the issue that some Titan histogram metrics are missing #7997
  • Fix the issue that TiKV returns duplicated error to TiCDC #7887

• PD

  • Check the correctness of the pd-server.dashboard-address configuration item #2517
  • Fix the panic issue of PD when setting store-limit-mode to auto #2544
  • Fix the issue that hotspots cannot be identified in some cases #2463
  • Fix the issue that placement rules prevent the store from changing to tombstone in some cases #2546
  • Fix the panic issue of PD when upgrading from earlier versions in some cases #2564

• TiFlash

  • Fix the issue that the proxy might panic when the region not found error occurs
  • Fix the issue that the I/O exception thrown in drop table might lead to synchronization failure of TiFlash schema