v8.5.4版本用ticdc搭建主从集群不同步,这是为什么?

【TiDB 使用环境】测试
【TiDB 版本】
【操作系统】欧拉系统
【部署方式】tiup部署ticdc
【集群数据量】
【集群节点数】
【问题复现路径】做过哪些操作出现的问题
【遇到的问题:问题现象及影响】
【资源配置】进入到 TiDB Dashboard -集群信息 (Cluster Info) -主机(Hosts) 截图此页面
【复制黏贴 ERROR 报错的日志】
【其他附件:截图/日志/监控】
[tidb@tidb-1 ~]$ tiup cdc cli changefeed create --server=http://10.33.43.74:8300 \

–sink-uri=“mysql://root:51fD-40pu%5EGh%2A9A%2B7r@10.33.43.74:4000/”
–changefeed-id=“primary-to-secondary”
–start-ts=“463074583243915266”
Starting component cdc: /home/tidb/.tiup/components/cdc/v8.5.4/cdc cli changefeed create --server=http://10.33.43.74:8300 --sink-uri=mysql://root:xxxxx@10.33.43.74:4000/ --changefeed-id=primary-to-secondary --start-ts=463074583243915266
Create changefeed successfully!
ID: primary-to-secondary
Info: {“upstream_id”:7584353134813259657,“id”:“primary-to-secondary”,“keyspace”:“”,“sink_uri”:“mysql://root:xxxxx@10.33.43.74:4000/”,“create_time”:“2025-12-23T19:56:34.624788733+08:00”,“start_ts”:463074583243915266,“config”:{“memory_quota”:1073741824,“case_sensitive”:false,“force_replicate”:false,“ignore_ineligible_table”:false,“check_gc_safe_point”:true,“enable_sync_point”:false,“enable_table_monitor”:false,“bdr_mode”:false,“sync_point_interval”:600000000000,“sync_point_retention”:86400000000000,“filter”:{“rules”:[“.”]},“mounter”:{“worker_num”:16},“sink”:{“csv”:{“delimiter”:“,”,“quote”:“"”,“null”:“\N”,“include_commit_ts”:false,“binary_encoding_method”:“base64”,“output_old_value”:false,“output_handle_key”:false,“output_field_header”:false},“encoder_concurrency”:32,“terminator”:“\r\n”,“date_separator”:“day”,“enable_partition_separator”:true,“enable_kafka_sink_v2”:false,“only_output_updated_columns”:false,“delete_only_output_handle_key_columns”:false,“content_compatible”:false,“advance_timeout”:150,“send_bootstrap_interval_in_sec”:120,“send_bootstrap_in_msg_count”:10000,“send_bootstrap_to_all_partition”:true,“send-all-bootstrap-at-start”:false,“debezium_disable_schema”:false,“debezium”:{“output_old_value”:true},“open”:{“output_old_value”:true}},“consistent”:{“level”:“none”,“max_log_size”:64,“flush_interval”:2000,“meta_flush_interval”:200,“encoding_worker_num”:16,“flush_worker_num”:8,“use_file_backend”:false,“memory_usage”:{“memory_quota_percentage”:50}},“scheduler”:{“enable_table_across_nodes”:false,“region_threshold”:10000,“region_count_per_span”:0,“write_key_threshold”:0,“scheduling_task_count_per_node”:0,“enable_splittable_check”:false,“balance_score_threshold”:0,“min_traffic_percentage”:0,“max_traffic_percentage”:0},“integrity”:{“integrity_check_level”:“none”,“corruption_handle_level”:“warn”},“changefeed_error_stuck_duration”:1800000000000,“synced_status”:{“synced_check_interval”:300,“checkpoint_interval”:15}},“state”:“normal”,“creator_version”:“v8.5.4”,“resolved_ts”:463074583243915266,“checkpoint_ts”:463074583243915266,“checkpoint_time”:“2025-12-23 19:29:06.481”,“gid”:{“low”:0,“high”:0}}
[tidb@tidb-1 ~]$ cdc cli changefeed list --server=http://10.33.43.74:8300
[
{
“id”: “primary-to-secondary”,
“namespace”: “default”,
“summary”: {
“state”: “normal”,
“tso”: 463075076144365573,
“checkpoint”: “2025-12-23 20:00:26.747”,
“error”: null
}
}
]
[tidb@tidb-1 ~]$

3 个赞

两个tidb版本是不是一样的?

一样的,都是8.5.4,并且看changefeed状态啥的都没有报错。但是在主集群中删除数据或者删除数据等操作就是没同步到从集群

[2025/12/24 09:02:58.223 +08:00] [INFO] [replication_set.go:458] [“schedulerv3: replication state transition, poll”] [namespace=default] [changefeed=primary-to-secondary] [t
ableState="span:<{tableID:78291, startKey:7480000000000131ffd35f720000000000fa, endKey:7480000000000131ffd35f730000000000fa}> state:Absent checkpoint:<> stats:<> “] [capture
ID=d8d2f280-dd0d-4dda-98f5-e1e45e584daf] [old=Absent] [new=Prepare]
[2025/12/24 09:02:58.223 +08:00] [INFO] [replication_set.go:900] [“schedulerv3: replication state transition, add table”] [namespace=default] [changefeed=primary-to-secondar
y] [replicationSet=”{"Changefeed":{"Namespace":"default","ID":"primary-to-secondary"},"Span":"{tableID:78291, startKey:7480000000000131ffd35f720000000000fa, end
Key:7480000000000131ffd35f730000000000fa}","State":"Prepare","Primary":"","Captures":{"d8d2f280-dd0d-4dda-98f5-e1e45e584daf":"Secondary"},"Checkpoint":{"ch
eckpoint_ts":463087377290887178,"resolved_ts":463087377290887178},"Stats":{"stage_checkpoints":{"puller-egress":{"checkpoint_ts":463087377290887178,"resolved_ts
“:463087377290887178},"puller-ingress":{"checkpoint_ts":463087377290887178,"resolved_ts":463087377290887178},"sink":{"checkpoint_ts":463087377290887178,"resolved_
ts":463087377290887178},"sorter-egress":{"checkpoint_ts":463087377290887178,"resolved_ts":463087377290887178},"sorter-ingress":{"checkpoint_ts":463087377290887178
,"resolved_ts":463087377290887178}}}}”] [old=Absent] [new=Prepare]
看这个日志感觉是数据库操作同步到从集群到了prepare状态卡住了,没下文了

2 个赞

不会遇到BUG了吧

2 个赞

可能因为时钟不同步有关,等我重新搞一下

3 个赞

咋样,是由于时钟不一致导致的吗?

1 个赞

从日志看同步似乎卡在 Prepare 状态,检查 sink 端连接是否正常呗

1 个赞

表有主键吗?checkpointTs 不推进还是推进了没有数据写到下游

连接时正常的

修改的表数据是有主健的,怀疑过是主键问题,专门试着创建了个新库,发现也不同步。

设置了 filter?发下 changefeed 配置文件呢

sudo systemctl restart systemd-timesyncd 同步一下

还是不行,并且开了cdc新架构也不行

还是不行

相比之前的环境,主从集群同步了时间,使用了新框架的cdc,对上游数据库操作下游数据库还是不会同步,查看了cdc的作业,发现上下有集群也无延迟
[tidb@tidb-1 ~]$ tiup cdc cli changefeed query --server=http://10.33.43.74:8300 --changefeed-id=“primary-to-secondary”
Starting component cdc: /home/tidb/.tiup/components/cdc/v8.5.4/cdc cli changefeed query --server=http://10.33.43.74:8300 --changefeed-id=primary-to-secondary
{
“upstream_id”: 7584353134813259657,
“id”: “primary-to-secondary”,
“keyspace”: “default”,
“sink_uri”: “mysql://root:xxxxx@10.33.43.74:4000/”,
“config”: {
“memory_quota”: 1073741824,
“case_sensitive”: false,
“force_replicate”: false,
“ignore_ineligible_table”: false,
“check_gc_safe_point”: true,
“enable_sync_point”: false,
“enable_table_monitor”: false,
“bdr_mode”: false,
“sync_point_interval”: 600000000000,
“sync_point_retention”: 86400000000000,
“filter”: {
“rules”: [
.
]
},
“mounter”: {
“worker_num”: 16
},
“sink”: {
“csv”: {
“delimiter”: “,”,
“quote”: “"”,
“null”: “\N”,
“include_commit_ts”: false,
“binary_encoding_method”: “base64”,
“output_old_value”: false,
“output_handle_key”: false,
“output_field_header”: false
},
“encoder_concurrency”: 32,
“terminator”: “\r\n”,
“date_separator”: “day”,
“enable_partition_separator”: true,
“only_output_updated_columns”: false,
“delete_only_output_handle_key_columns”: false,
“content_compatible”: false,
“advance_timeout”: 150,
“send_bootstrap_interval_in_sec”: 120,
“send_bootstrap_in_msg_count”: 10000,
“send_bootstrap_to_all_partition”: true,
“send-all-bootstrap-at-start”: false,
“debezium_disable_schema”: false,
“debezium”: {
“output_old_value”: true
},
“open”: {
“output_old_value”: true
}
},
“consistent”: {
“level”: “none”,
“max_log_size”: 64,
“flush_interval”: 2000,
“meta_flush_interval”: 200,
“encoding_worker_num”: 16,
“flush_worker_num”: 8,
“use_file_backend”: false,
“memory_usage”: {
“memory_quota_percentage”: 50
}
},
“scheduler”: {
“enable_table_across_nodes”: false,
“region_threshold”: 10000,
“region_count_per_span”: 100,
“write_key_threshold”: 0,
“scheduling_task_count_per_node”: 20,
“enable_splittable_check”: false,
“balance_score_threshold”: 20,
“min_traffic_percentage”: 0.8,
“max_traffic_percentage”: 1.25
},
“integrity”: {
“integrity_check_level”: “none”,
“corruption_handle_level”: “warn”
},
“changefeed_error_stuck_duration”: 1800000000000,
“synced_status”: {
“synced_check_interval”: 300,
“checkpoint_interval”: 15
}
},
“create_time”: “2025-12-25 15:31:51.149”,
“start_ts”: 463115714360246279,
“resolved_ts”: 463117445524357134,
“target_ts”: 0,
“checkpoint_tso”: 463117445524357134,
“checkpoint_time”: “2025-12-25 16:54:13.105”,
“state”: “normal”,
“creator_version”: “v8.5.4”
}

从库时间一致、有主键,资源占用情况如何?

2 个赞

[2025/12/25 16:59:18.176 +08:00] [INFO] [schema_store.go:134] [“handle a ddl job”] [schemaID=304] [schemaName=test] [tableID=306] [tableName=addr_info] [type=“drop table”] [DDL=“/* ApplicationName=DBeaver 6.2.0 - Main */ drop table addr_info”] [schemaVersion=2259] [jobFinishTs=463117525381808136] [storeSchemaVersion=2256] [storeFinishedDDLTS=463075052547211278]
[2025/12/25 16:59:18.200 +08:00] [INFO] [utils.go:78] [“transform ddl query to result”] [charset=] [collate=] [result=“DROP TABLE addr_info”]
[2025/12/25 17:03:10.328 +08:00] [INFO] [persist_storage.go:621] [“gc is disabled”] [gcTs=463117339928035328]

2 个赞

gc明明是开启的,为啥会报gc不可用。

2 个赞

资源是空闲的

2 个赞