TiDB 5.1.4 Release Notes

Release Date: February 22, 2022

TiDB version: 5.1.4

Compatibility changes

• TiDB

  • Change the default value of the system variable tidb_analyze_version from 2 to 1 #31748
  • Since v5.1.4, if TiKV is configured with storage.enable-ttl = true, the requests from TiDB are rejected, because the TTL feature of TiKV only supports the RawKV mode #27303

• Tools

  • TiCDC

    • Set the default value of max-message-bytes to 10M #4041

Improvements

• TiDB

  • Support partition pruning for the built-in IN expression in Range partition tables #26739
  • Improve the accuracy of tracking memory usage when IndexJoin is executed #28650

• TiKV

  • Update the proc filesystem (procfs) to v0.12.0 #11702
  • Improve the error log report in the Raft client #11959
  • Increase the speed of inserting SST files by moving the verification process to the Import thread pool from the Apply thread pool #11239

• PD

  • Speed up the exit process of schedulers #4146

• TiFlash

  • Support pushing down ADDDATE() and DATE_ADD() to TiFlash
  • Support pushing down INET6_ATON() and INET6_NTOA() to TiFlash
  • Support pushing down INET_ATON() and INET_NTOA() to TiFlash
  • Increase the max supported depth of an expression or a plan tree in a DAG request from 100 to 200

• Tools

  • TiCDC

    • Add the exponential backoff mechanism for restarting a changefeed. #3329
    • Reduce the replication latency when replicating many tables #3900
    • Add metrics for observing the remaining time of incremental scan #2985
    • Reduce the count of "EventFeed retry rate limited" logs #4006
    • Add more Prometheus and Grafana monitoring metrics and alerts, including no owner alert, mounter row, table sink total row, and buffer sink total row #4054 #1606
    • Optimize rate limiting control on TiKV reloads to reduce gPRC congestion during changefeed initialization #3110
    • Reduce the time for the KV client to recover when a TiKV store is down #3191

Bug fixes

• TiDB

  • Fix a memory leak bug that occurs when the system variable tidb_analyze_version is set to 2 #32499
  • Fix the issue that the MaxDays and MaxBackups configurations do not take effect for the slow log #25716
  • Fix the issue that executing the INSERT ... SELECT ... ON DUPLICATE KEY UPDATE statement gets panic #28078
  • Fix the wrong result that might occur when performing JOIN on ENUM type columns #27831
  • Fix the issue that INDEX HASH JOIN returns the send on closed channel error #31129
  • Fix the issue that using the BatchCommands API might block sending TiDB requests to TiKV in some rare cases #32500
  • Fix the issue of potential data index inconsistency in optimistic transaction mode #30410
  • Fix the issue that window functions might return different results when using a transaction or not #29947
  • Fix the issue that the length information is wrong when casting Decimal to String #29417
  • Fix the issue that the GREATEST function returns incorrect result that occurs when setting the tidb_enable_vectorized_expression vectorized expression to off #29434
  • Fix the issue that the optimizer might cache invalid plans for join in some cases #28087
  • Fix wrong results of the microsecond and hour functions in vectorized expressions #29244 #28643
  • Fix the TiDB panic when executing the ALTER TABLE.. ADD INDEX statement in some cases #27687
  • Fix a bug that the availability detection of MPP node does not work in some corner cases #3118
  • Fix the DATA RACE issue when assigning MPP task ID #27952
  • Fix the INDEX OUT OF RANGE error for a MPP query after deleting an empty dual table #28250
  • Fix the issue of false positive error log invalid cop task execution summaries length for MPP queries #1791
  • Fix the issue that SET GLOBAL tidb_skip_isolation_level_check=1 doesn't affect new session settings #27897

• TiKV

  • Fix a bug that TiKV cannot delete a range of data (unsafe_destroy_range cannot be executed) when the GC worker is busy #11903
  • Fix the issue that destroying a peer might cause high latency #10210
  • Fix a bug that the any_value function returns a wrong result when regions are empty #11735
  • Fix the issue that deleting an uninitialized replica might cause an old replica to be recreated #10533
  • Fix the metadata corruption issue when Prepare Merge is triggered after a new election is finished but the isolated peer is not informed #11526
  • Fix the deadlock issue that happens occasionally when coroutines run too fast #11549
  • Fix the potential deadlock and memory leak issues when profiling flame graphs #11108
  • Fix the rare data inconsistency issue when retrying a prewrite request in pessimistic transactions #11187
  • Fix a bug that the configuration resource-metering.enabled does not work #11235
  • Fix the issue that some coroutines leak in resolved_ts #10965
  • Fix the issue of reporting false "GC can not work" alert under low write flow #9910
  • Fix a bug that tikv-ctl cannot return the correct Region-related information #11393
  • Fix the issue that a down TiKV node causes the resolved timestamp to lag #11351
  • Fix a panic issue that occurs when Region merge, ConfChange, and Snapshot happen at the same time in extreme conditions #11475
  • Fix the issue that TiKV cannot detect the memory lock when TiKV performs a reverse table scan #11440
  • Fix the issue of negative sign when the decimal divide result is zero #29586
  • Fix a memory leak caused by the monitoring data of statistics threads #11195
  • Fix the issue of TiCDC panic that occurs when the downstream database is missing #11123
  • Fix the issue that TiCDC adds scan retries frequently due to the Congest error #11082
  • Fix the issue that batch messages are too large in Raft client implementation #9714
  • Collapse some uncommon storage-related metrics in Grafana dashboard #11681

• PD

  • Fix a bug that the schedule generated by the region scatterer might decrease the number of peers #4565
  • Fix the issue that Region statistics are not affected by flow-round-by-digit #4295
  • Fix slow leader election caused by stucked region syncer #3936
  • Support that the evict leader scheduler can schedule regions with unhealthy peers #4093
  • Fix the issue that the cold hotspot data cannot be deleted from the hotspot statistics #4390
  • Fix a panic issue that occurs after the TiKV node is removed #4344
  • Fix the issue that the scheduling operator cannot fail fast because the target store is down #3353

• TiFlash

  • Fix the issue that the str_to_date() function incorrectly handles leading zeros when parsing microseconds
  • Fix the TiFlash crash problem when the memory limit is enabled
  • Fix the issue that when an input time is earlier than 1970-01-01 00:00:01 UTC, the behavior of unix_timestamp is inconsistent with that of TiDB or MySQL
  • Fix the potential data inconsistency caused by widening the primary key column when the primary key is handle
  • Fix the overflow bug and the issue of reporting Can't compare error when comparing data in the DECIMAL data type
  • Fix the unexpected error of 3rd arguments of function substringUTF8 must be constants.
  • Fix the issue that TiFlash fails to start on platforms without the nsl library
  • Fix the overflow bug when casting data to the DECIMAL data type
  • Fix the issue that the castStringAsReal behavior is inconsistent in TiFlash and in TiDB/TiKV
  • Fix the issue that TiFlash might return the EstablishMPPConnection error after it is restarted
  • Fix the issue that obsolete data cannot be reclaimed after setting the number of TiFlash replicas to 0
  • Fix the issue that the CastStringAsDecimal behavior is inconsistent in TiFlash and in TiDB/TiKV
  • Fix the issue that queries with the where <string> clause return wrong results
  • Fix the issue that TiFlash might panic when an MPP query is stopped
  • Fix the unexpected error of Unexpected type of column: Nullable(Nothing)

• Tools

  • TiCDC

    • Fix a bug that MySQL sink generates duplicated replace SQL statements if batch-replace-enable is disabled #4501
    • Fix the issue that the cached region monitoring metric is negative #4300
    • Fix the issue that replication cannot be performed when min.insync.replicas is smaller than replication-factor #3994
    • Fix the potential panic issue that occurs when a replication task is removed #3128
    • Fix the issue of potential data loss caused by inaccurate checkpoint #3545
    • Fix the potential issue that the deadlock causes a replication task to get stuck #4055
    • Fix the issue that special comments in DDL statements cause the replication task to stop #3755
    • Fix a bug that EtcdWorker might hang the owner and processor #3750
    • Fix the issue that stopped changefeeds resume automatically after a cluster upgrade #3473
    • Fix the issue that default values cannot be replicated #3793
    • Fix data inconsistency caused by TiCDC default value padding exceptions #3918 #3929
    • Fix a bug that an owner gets stuck when a PD leader shuts down and transfers to a new node #3615
    • Fix the TiCDC panic issue that occurs when manually cleaning the task status in etcd #2980
    • Fix the issue that the service cannot be started because of a timezone issue in the RHEL release #3584
    • Fix the issue of overly frequent warnings caused by MySQL sink deadlock #2706
    • Fix the bug that the enable-old-value configuration item is not automatically set to true on Canal and Maxwell protocols #3676
    • Fix the issue that Avro sink does not support parsing JSON type columns #3624
    • Fix the negative value error in the changefeed checkpoint lag #3010
    • Fix the OOM issue in the container environment #1798
    • Fix the TiCDC replication interruption issue when multiple TiKVs crash or during a forced restart #3288
    • Fix the memory leak issue after processing DDLs #3174
    • Fix the issue that changefeed does not fail fast enough when the ErrGCTTLExceeded error occurs #3111
    • Fix the issue that TiCDC replication task might terminate when the upstream TiDB instance unexpectedly exits #3061
    • Fix the issue that TiCDC process might panic when TiKV sends duplicate requests to the same Region #2386
    • Fix the issue that Kafka may send excessively large messages by setting the default value of max-message-bytes to 10M #3081
    • Fix the issue that TiCDC sync task might pause when an error occurs during writing a Kafka message #2978

  • Backup & Restore (BR)

    • Fix the potential issue that Regions might be unevenly distributed after a restore operation is finished #30425 #31034

  • TiDB Binlog

    • Fix the issue that DBaaS importing CSV fails with InvalidRange if CSV file size is about 256MB and strict-format is true #27763

  • TiDB Lightning

    • Fix the issue that TiDB Lightning does not report errors when the S3 storage path does not exist #28031 #30709