【 TiDB 使用环境】生产环境
【 TiDB 版本】 v5.4.0
【复现路径】
【遇到的问题:问题现象及影响】
业务突然报障,说tidb集群不可用,上去tiup display 发现有一个kv节点disconnect ,然后重启这个节点也还是disconnt,10分后恢复。
kv节点的日志信息:
[2023/12/20 12:48:21.858 +08:00] [WARN] [gc_worker.rs:606] [“GcKeys fail”] [err=“Error(Other("[src/server/gc_worker/gc_worker.rs:341]: [components/raftstore/src/coprocessor/region_info_accessor.rs:622]: failed to send request to region collector: channel has been closed"))”]
[2023/12/20 12:48:21.858 +08:00] [WARN] [gc_worker.rs:606] [“GcKeys fail”] [err=“Error(Other("[src/server/gc_worker/gc_worker.rs:341]: [components/raftstore/src/coprocessor/region_info_accessor.rs:622]: failed to send request to region collector: channel has been closed"))”]
[2023/12/20 12:48:21.858 +08:00] [WARN] [gc_worker.rs:606] [“GcKeys fail”] [err=“Error(Other("[src/server/gc_worker/gc_worker.rs:341]: [components/raftstore/src/coprocessor/region_info_accessor.rs:622]: failed to send request to region collector: channel has been closed"))”]
[2023/12/20 12:48:21.858 +08:00] [WARN] [gc_worker.rs:606] [“GcKeys fail”] [err=“Error(Other("[src/server/gc_worker/gc_worker.rs:341]: [components/raftstore/src/coprocessor/region_info_accessor.rs:622]: failed to send request to region collector: channel has been closed"))”]
[2023/12/20 12:48:21.858 +08:00] [WARN] [gc_worker.rs:606] [“GcKeys fail”] [err=“Error(Other("[src/server/gc_worker/gc_worker.rs:341]: [components/raftstore/src/coprocessor/region_info_accessor.rs:622]: failed to send request to region collector: channel has been closed"))”]
