TiDB使用br工具进行表恢复，报错[BR:KV:ErrKVDownloadFailed]download sst failed;

[TiDBer_ndcICetK]

【 TiDB 使用环境】5节点的测试环境

【 TiDB 版本】7.1.9

【复现路径】
对一个5节点的集群做全量备份，备份后进行表恢复，恢复时目标表已经创建，但是报错[BR:KV:ErrKVDownloadFailed]download sst failed;

【遇到的问题：问题现象及影响】
现象是恢复表报错， 错误日志：error="Cannot read http://xx.xx.125.130:29848/obdata/backup/8/146547_4817_138e50d3f74adf320817cdd190ccaf06b33c56299a4917b29f691358f9106a0d_1786344454987_write.sst into /tidb-data/tidb-data/tikv-20160/import/.temp/dc9e1afb-751c-41bd-a093-acc9da8e9289_226364_779_4765_write_2.sst: failed to get object dispatch failure: [BR:KV:ErrKVDownloadFailed]download sst failed;

【资源配置】进入到 TiDB Dashboard -集群信息 (Cluster Info) -主机(Hosts) 截图此页面

已经排查过的问题：

1. 服务端 minio文件存在，并且权限正常。
2. br的客户端网络telnet能访问到ip和端口
3. 使用 rclone 在服务端和客户端都能正常下载到报错的文件

手动恢复在下载sst过程卡主：

报错日志：
[2026/08/13 16:00:17.689 +08:00] [WARN] [import.go:443] [“download file failed, retry later”] [sn=0] [region#=0] [total#=1] [region=“{ID=226689,startKey=7480000000000032FF2400000000000000F8,endKey=7480000000000032FF245F72FFFFFFFFFFFFFFFFFF0000000000FB,epoch="conf_ver:779 version:4786 ",peers="id:226690 store_id:8 ,id:226691 store_id:1 ,id:226692 store_id:9 "}”] [startKey=7480000000000032FF245F698000000000FF0000010000000000FB] [endKey=7480000000000032FF245F72FFFFFFFFFFFFFFFFFF0000000000FB] [error=“Cannot read http://10.11.125.130:29859/obdata/backup/8/146547_4852_c77b4ea2e33fcca2d68252ea99cff19402d9ab0e817a8ac94d430d04270d38c7_1786604407933_write.sst into /tidb-data/tidb-data/tikv-20160/import/.temp/40cc98a5-f556-41ce-b336-79536632956a_226689_779_4786_write_2.sst: failed to get object dispatch failure: [BR:KV:ErrKVDownloadFailed]download sst failed; Cannot read http://10.11.125.130:29859/obdata/backup/8/146547_4852_c77b4ea2e33fcca2d68252ea99cff19402d9ab0e817a8ac94d430d04270d38c7_1786604407933_write.sst into /tidb-data/tidb-data/tikv-20160/import/.temp/de96facb-9480-4c2a-a909-a621c7e59c85_226689_779_4786_write_2.sst: failed to get object dispatch failure: [BR:KV:ErrKVDownloadFailed]download sst failed; Cannot read http://10.11.125.130:29859/obdata/backup/8/146547_4852_c77b4ea2e33fcca2d68252ea99cff19402d9ab0e817a8ac94d430d04270d38c7_1786604407933_write.sst into /tidb-data/tidb-data/tikv-20160/import/.temp/35896f91-71dc-4c43-870e-eccf97eabe9b_226689_779_4786_write_2.sst: failed to get object dispatch failure: [BR:KV:ErrKVDownloadFailed]download sst failed; Cannot read http://10.11.125.130:29859/obdata/backup/8/146547_4852_e4df8b9c0b572f396899bb89f1c33b22fa3d45df980ee9e3a3f0cb1d95c3795a_1786604407933_write.sst into /tidb-data/tidb-data/tikv-20160/import/.temp/04c07cd1-7755-4d67-8118-1f2c5ae6a629_226689_779_4786_write_2.sst: failed to get object dispatch failure: [BR:KV:ErrKVDownloadFailed]download sst failed; Cannot read http://10.11.125.130:29859/obdata/backup/8/146547_4852_e4df8b9c0b572f396899bb89f1c33b22fa3d45df980ee9e3a3f0cb1d95c3795a_1786604407933_write.sst into /tidb-data/tidb-data/tikv-20160/import/.temp/b781ef1a-b730-4a41-93cd-b1c5429b78a0_226689_779_4786_write_2.sst: failed to get object dispatch failure: [BR:KV:ErrKVDownloadFailed]download sst failed; Cannot read http://10.11.125.130:29859/obdata/backup/8/146547_4852_c77b4ea2e33fcca2d68252ea99cff19402d9ab0e817a8ac94d430d04270d38c7_1786604407933_write.sst into /tidb-data/tidb-data/tikv-20160/import/.temp/eb2e984f-0621-4e70-8b94-dfa968d75761_226689_779_4786_write_2.sst: failed to get object dispatch failure: [BR:KV:ErrKVDownloadFailed]download sst failed; Cannot read http://10.11.125.130:29859/obdata/backup/8/146547_4852_c77b4ea2e33fcca2d68252ea99cff19402d9ab0e817a8ac94d430d04270d38c7_1786604407933_write.sst into /tidb-data/tidb-data/tikv-20160/import/.temp/67c068a6-343d-452a-a215-d30ae3e6a85a_226689_779_4786_write_2.sst: failed to get object dispatch failure: [BR:KV:ErrKVDownloadFailed]download sst failed; Cannot read http://10.11.125.130:29859/obdata/backup/8/146547_4852_c77b4ea2e33fcca2d68252ea99cff19402d9ab0e817a8ac94d430d04270d38c7_1786604407933_write.sst into /tidb-data/tidb-data/tikv-20160/import/.temp/0b8a82fe-12c4-49fb-918f-e602f7e4f19e_226689_779_4786_write_2.sst: failed to get object dispatch failure: [BR:KV:ErrKVDownloadFailed]download sst failed”]

[TiDBer_ndcICetK]

抓包的信息显示minio正常收到获取文件的请求，也正常返回200 OK，但是没有传输数据

[Raymond]

每台tikv 到这个网络节点都是通顺的嘛，你可以看下

[一只大芒果 (Ti D Ber L B8j9nt J)]

是把备份文件放在了kv对应的主机上吗？

[TiDBer_ndcICetK]

放在其他服务器的，通过S3协议获取文件。

[kang]

从报错看，BR在下载SST时网络中断或超时了。虽然你telnet通了，但大文件下载时连接可能不稳定。建议按顺序排查：

1. 检查TiKV日志，看是否有io timeout或connection reset，重点看tikv-20160节点
2. 调大BR下载并发和超时时间，在恢复命令加参数：

br restore table --concurrency 4 --ratelimit 128 --checksum=false --send-credentials-to-tikv=true

3. 检查TiKV的import目录磁盘空间，/tidb-data/tidb-data/tikv-20160/import/.temp满了也会报这个错
4. 如果MinIO是单机部署，确认带宽和连接数限制，试试把region下载并发调低

另外确认下目标表结构是否和备份时一致，不一致也会导致下载后apply失败。建议先用br restore table --dry-run验证下。

[TiDBer_bawpguhQ]

BR 下载 SST 失败先核 TiKV 到对象存储连通性、证书/鉴权与路径；对照 rclone 成功路径查 br 是否走了不同 endpoint。看 TiKV import 临时目录磁盘与权限，必要时降并发重试并抓完整 BR/TiKV 日志。

[独善其身 (Ti D Ber Bi Rqfz5 K)]

常见阻断原因：

1. 机器防火墙 firewalld/iptables，只放通了 TiKV 固定端口 20160，没有放通高位临时端口范围 29000‑32767；
2. 安全组 / 云主机网络 ACL 拦截高位端口；
3. 主机 bind 地址：TiKV 临时 http server 监听 IP 绑定问题，只监听 127.0.0.1，没有监听节点内网 IP；
4. SELinux 限制网络访问。

[wbslxw (Ti D Ber Cl S0j Eng)]

楼上说的有道理，测试下防火墙，看是不是没有放通高位端口