【 TiDB 使用环境】Linux：3.10.0-1160.el7.x86_64 安装单实例
【概述】安装时无报错，执行tiup cluster start tidb-test时启动失败
【背景】尝试过在线安装及离线安装报错一致
【现象】执行tiup cluster display tidb-test状态均为up状态，Grafana和prometheus均无报错
【TiDB 版本】TIDB版本为V5.1.0
测试信息：
增删改查均无问题，但是无法导入导出，无法备份。
执行报错信息：
[tidb@mgtk8suat01v ~]$ tiup cluster start tidb-test
Starting component cluster: /home/tidb/.tiup/components/cluster/v1.5.2/tiup-cluster start tidb-test
Starting cluster tidb-test…
-
[ Serial ] - SSHKeySet: privateKey=/home/tidb/.tiup/storage/cluster/clusters/tidb-test/ssh/id_rsa, publicKey=/home/tidb/.tiup/storage/cluster/clusters/tidb-test/ssh/id_rsa.pub
-
[ Serial ] - StartCluster
Starting component pd
Starting instance 10.20.0.1:2379
Start instance 10.20.0.1:2379 success
Starting component tikv
Starting instance 10.20.0.1:20160
Start instance 10.20.0.1:20160 success
Starting component tidb
Starting instance 10.20.0.1:4000
Start instance 10.20.0.1:4000 success
Starting component prometheus
Starting instance 10.20.0.1:9090
Start instance 10.20.0.1:9090 success
Starting component grafana
Starting instance 10.20.0.1:3000
Start instance 10.20.0.1:3000 success
Starting component alertmanager
Starting instance 10.20.0.1:9093
Start instance 10.20.0.1:9093 success
Starting component node_exporter
Starting instance 10.20.0.1
Start 10.20.0.1 success
Starting component blackbox_exporter
Starting instance 10.20.0.1
Start 10.20.0.1 success -
[ Serial ] - UpdateTopology: cluster=tidb-test
{“level”:“warn”,“ts”:“2021-08-26T13:37:14.462+0800”,“logger”:“etcd-client”,“caller”:“v3@v3.5.0/retry_interceptor.go:62”,“msg”:“retrying of unary invoker failed”,“target”:“etcd-endpoints://0xc000174380/#initially=[10.20.0.1:2379]”,“attempt”:0,“error”:"rpc error: code = DeadlineExceeded desc = latest balancer error: last connection error: connection error: desc = "transport: Error while dialing failed to do connect handshake, response: \"HTTP/1.1 403 Forbidden\\r\\
Content-Length: 3496\\r\\
Connection: keep-alive\\r\\
Content-Language: en\\r\\
Content-Type: text/html;charset=utf-8\\r\\
Date: Thu, 26 Aug 2021 05:37:12 GMT\\r\\
Mime-Version: 1.0\\r\\
Server: squid/3.5.20\\r\\
Vary: Accept-Language\\r\\
Via: 1.1 mgtasblab01v.geniusafc.com (squid/3.5.20)\\r\\
X-Cache: MISS from mgtasblab01v.geniusafc.com\\r\\
X-Cache-Lookup: NONE from mgtasblab01v.geniusafc.com:3128\\r\\
X-Squid-Error: ERR_ACCESS_DENIED 0\\r\\
\\r\\
ERROR
\\\\The requested URL could not be retrieved
\\\\\\\\ \\\\
The following error was encountered while trying to retrieve the URL: 10.20.0.1:2379
\\\\ \\\\\\\\\\\\ \\\\Access Denied.
\\\\
Access control configuration prevents your request from being allowed at this time. Please contact your service provider if you feel this is incorrect.
\\\\ \\\\Your cache xxxxxxxxistrator is root.
\\\\\\\\
\\\\
Generated Thu, 26 Aug 2021 05:37:12 GMT by mgtasblab01v.geniusafc.com (squid/3.5.20)
\\\\ \\\\edit-config信息：
[tidb@mgtk8suat01v ~]$ tiup cluster edit-config tidb-test
Starting component cluster: /home/tidb/.tiup/components/cluster/v1.5.2/tiup-cluster edit-config tidb-test
global:
user: tidb
ssh_port: 22
ssh_type: builtin
deploy_dir: /u01/tidb/tidb-deploy
data_dir: /u01/tidb/tidb-data
os: linux
arch: amd64
monitored:
node_exporter_port: 9100
blackbox_exporter_port: 9115
deploy_dir: /u01/tidb/tidb-deploy/monitor-9100
data_dir: /u01/tidb/tidb-data/monitor-9100
log_dir: /u01/tidb/tidb-deploy/monitor-9100/log
tidb_servers:
- host: 10.20.0.1
ssh_port: 22
port: 4000
status_port: 10080
deploy_dir: /u01/tidb/tidb-deploy/tidb-4000
log_dir: /u01/tidb/tidb-deploy/tidb-4000/log
arch: amd64
os: linux
tikv_servers: - host: 10.20.0.1
ssh_port: 22
port: 20160
status_port: 20180
deploy_dir: /u01/tidb/tidb-deploy/tikv-20160
data_dir: /u01/tidb/tidb-data/tikv-20160
log_dir: /u01/tidb/tidb-deploy/tikv-20160/log
arch: amd64
os: linux
tiflash_servers:
pd_servers: - host: 10.20.0.1
ssh_port: 22
name: pd-10.20.0.1-2379
client_port: 2379
peer_port: 2380
deploy_dir: /u01/tidb/tidb-deploy/pd-2379
data_dir: /u01/tidb/tidb-data/pd-2379
log_dir: /u01/tidb/tidb-deploy/pd-2379/log
arch: amd64
os: linux
monitoring_servers: - host: 10.20.0.1
ssh_port: 22
port: 9090
deploy_dir: /u01/tidb/tidb-deploy/prometheus-9090
data_dir: /u01/tidb/tidb-data/prometheus-9090
log_dir: /u01/tidb/tidb-deploy/prometheus-9090/log
external_alertmanagers:
arch: amd64
os: linux
grafana_servers: - host: 10.20.0.1
ssh_port: 22
port: 3000
deploy_dir: /u01/tidb/tidb-deploy/grafana-3000
arch: amd64
os: linux
username: xxxxxxxx
password: xxxxxxxx
anonymous_enable: false
root_url: “”
domain: “”
alertmanager_servers: - host: 10.20.0.1
ssh_port: 22
web_port: 9093
cluster_port: 9094
deploy_dir: /u01/tidb/tidb-deploy/alertmanager-9093
data_dir: /u01/tidb/tidb-data/alertmanager-9093
log_dir: /u01/tidb/tidb-deploy/alertmanager-9093/log
arch: amd64
os: linux
store信息
{
“client-urls”: “http://0.0.0.0:2379”,
“peer-urls”: “http://0.0.0.0:2380”,
“advertise-client-urls”: “http://10.20.0.1:2379”,
“advertise-peer-urls”: “http://10.20.0.1:2380”,
“name”: “pd-10.20.0.1-2379”,
“data-dir”: “/u01/tidb/tidb-data/pd-2379”,
“force-new-cluster”: false,
“enable-grpc-gateway”: true,
“initial-cluster”: “pd-10.20.0.1-2379=http://10.20.0.1:2380”,
“initial-cluster-state”: “new”,
“initial-cluster-token”: “pd-cluster”,
“join”: “”,
“lease”: 3,
“log”: {
“level”: “”,
“format”: “text”,
“disable-timestamp”: false,
“file”: {
“filename”: “/u01/tidb/tidb-deploy/pd-2379/log/pd.log”,
“max-size”: 300,
“max-days”: 0,
“max-backups”: 0
},
“development”: false,
“disable-caller”: false,
“disable-stacktrace”: false,
“disable-error-verbose”: true,
“sampling”: null
},
“tso-save-interval”: “3s”,
“tso-update-physical-interval”: “50ms”,
“enable-local-tso”: false,
“metric”: {
“job”: “pd-10.20.0.1-2379”,
“address”: “”,
“interval”: “15s”
},
“schedule”: {
“max-snapshot-count”: 3,
“max-pending-peer-count”: 16,
“max-merge-region-size”: 20,
“max-merge-region-keys”: 200000,
“split-merge-interval”: “1h0m0s”,
“enable-one-way-merge”: “false”,
“enable-cross-table-merge”: “true”,
“patrol-region-interval”: “100ms”,
“max-store-down-time”: “30m0s”,
“leader-schedule-limit”: 4,
“leader-schedule-policy”: “count”,
“region-schedule-limit”: 2048,
“replica-schedule-limit”: 64,
“merge-schedule-limit”: 8,
“hot-region-schedule-limit”: 4,
“hot-region-cache-hits-threshold”: 3,
“store-limit”: {
“store-limit”: {
“1”: {
“add-peer”: 15,
“remove-peer”: 15
}
},
“tolerant-size-ratio”: 0,
“low-space-ratio”: 0.8,
“high-space-ratio”: 0.7,
“region-score-formula-version”: “v2”,
“scheduler-max-waiting-operator”: 5,
“enable-remove-down-replica”: “true”,
“enable-replace-offline-replica”: “true”,
“enable-make-up-replica”: “true”,
“enable-remove-extra-replica”: “true”,
“enable-location-replacement”: “true”,
“enable-debug-metrics”: “false”,
“enable-joint-consensus”: “true”,
“schedulers-v2”: [
{
“type”: “balance-region”,
“args”: null,
“disable”: false,
“args-payload”: “”
},
{
“type”: “balance-leader”,
“args”: null,
“disable”: false,
“args-payload”: “”
},
{
“type”: “hot-region”,
“args”: null,
“disable”: false,
“args-payload”: “”
},
{
“type”: “label”,
“args”: null,
“disable”: false,
“args-payload”: “”
}
],
“schedulers-payload”: {
“balance-hot-region-scheduler”: null,
“balance-leader-scheduler”: {
“name”: “balance-leader-scheduler”,
“ranges”: [
{
“end-key”: “”,
“start-key”: “”
}
]
},
“balance-region-scheduler”: {
“name”: “balance-region-scheduler”,
“ranges”: [
{
“end-key”: “”,
“start-key”: “”
}
]
},
“label-scheduler”: {
“name”: “label-scheduler”,
“ranges”: [
{
“end-key”: “”,
“start-key”: “”
}
]
}
},
“store-limit-mode”: “manual”
},
“replication”: {
“max-replicas”: 3,
“location-labels”: “”,
“strictly-match-label”: “false”,
“enable-placement-rules”: “true”,
“isolation-level”: “”
},
“pd-server”: {
“use-region-storage”: “true”,
“max-gap-reset-ts”: “24h0m0s”,
“key-type”: “table”,
“runtime-services”: “”,
“metric-storage”: “”,
“dashboard-address”: “http://10.20.0.1:2379”,
“flow-round-by-digit”: 3
},
“cluster-version”: “5.1.0”,
“labels”: {},
“quota-backend-bytes”: “8GiB”,
“auto-compaction-mode”: “periodic”,
“auto-compaction-retention-v2”: “1h”,
“TickInterval”: “500ms”,
“ElectionInterval”: “3s”,
“PreVote”: true,
“security”: {
“cacert-path”: “”,
“cert-path”: “”,
“key-path”: “”,
“cert-allowed-cn”: null,
“redact-info-log”: false,
“encryption”: {
“data-encryption-method”: “plaintext”,
“data-key-rotation-period”: “168h0m0s”,
“master-key”: {
“type”: “plaintext”,
“key-id”: “”,
“region”: “”,
“endpoint”: “”,
“path”: “”
}
}
},
“label-property”: {},
“WarningMsgs”: null,
“DisableStrictReconfigCheck”: false,
“HeartbeatStreamBindInterval”: “1m0s”,
“LeaderPriorityCheckInterval”: “1m0s”,
“dashboard”: {
“tidb-cacert-path”: “”,
“tidb-cert-path”: “”,
“tidb-key-path”: “”,
“public-path-prefix”: “”,
“internal-proxy”: false,
“enable-telemetry”: true,
“enable-experimental”: false
},
“replication-mode”: {
“replication-mode”: “majority”,
“dr-auto-sync”: {
“label-key”: “”,
“primary”: “”,
“dr”: “”,
“primary-replicas”: 0,
“dr-replicas”: 0,
“wait-store-timeout”: “1m0s”,
“wait-sync-timeout”: “1m0s”,
“wait-async-timeout”: “2m0s”
}
}
}
Display信息
[tidb@mgtk8suat01v ~]$ tiup cluster display tidb-test
Starting component cluster: /home/tidb/.tiup/components/cluster/v1.5.2/tiup-cluster display tidb-test
Cluster type: tidb
Cluster name: tidb-test
Cluster version: v5.1.0
Deploy user: tidb
SSH type: builtin
Dashboard URL: http://10.20.0.1:2379/dashboard
ID Role Host Ports OS/Arch Status Data Dir Deploy Dir
10.20.0.1:9093 alertmanager 10.20.0.1 9093/9094 linux/x86_64 Up /u01/tidb/tidb-data/alertmanager-9093 /u01/tidb/tidb-deploy/alertmanager-9093
10.20.0.1:3000 grafana 10.20.0.1 3000 linux/x86_64 Up - /u01/tidb/tidb-deploy/grafana-3000
10.20.0.1:2379 pd 10.20.0.1 2379/2380 linux/x86_64 Up|L|UI /u01/tidb/tidb-data/pd-2379 /u01/tidb/tidb-deploy/pd-2379
10.20.0.1:9090 prometheus 10.20.0.1 9090 linux/x86_64 Up /u01/tidb/tidb-data/prometheus-9090 /u01/tidb/tidb-deploy/prometheus-9090
10.20.0.1:4000 tidb 10.20.0.1 4000/10080 linux/x86_64 Up - /u01/tidb/tidb-deploy/tidb-4000
10.20.0.1:20160 tikv 10.20.0.1 20160/20180 linux/x86_64 Up /u01/tidb/tidb-data/tikv-20160 /u01/tidb/tidb-deploy/tikv-20160