DM 数据同步报错,如何处理?

【TiDB 使用环境】测试
数据同步任务报错如下,如何修复?求大佬指点

"result": {
                        "isCanceled": false,
                        "errors": [
                            {
                                "ErrCode": 10006,
                                "ErrClass": "database",
                                "ErrScope": "not-set",
                                "ErrLevel": "high",
                                "Message": "startLocation: [position: (mysql-bin.012231, 201269362), gtid-set: 83ef3e40-f7f2-11ef-b39d-a088c260cf2a:1-519309589:519457506-3384465940], endLocation: [position: (mysql-bin.012231, 202131376), gtid-set: 83ef3e40-f7f2-11ef-b39d-a088c260cf2a:1-519309589:519457506-3384465940]: execute statement failed: commit",
                                "RawCause": "Error 8141 (HY000): assertion failed: key: 7480000000000001b95f69800000000000000703800000000000000d038000000000000000010031003300380037ff0032003600380031ff0039003800380000fd01741a000000000000f9010000000000000000f703800000000035624a, assertion: Exist, star_ts: 461032405724561427, existing start ts: 0, existing commit ts: 0",
                                "Workaround": ""
                            }
                        ],
                        "detail": null
                    },
 "sync": {
                        "totalEvents": "405727",
                        "totalTps": "10143",
                        "recentTps": "0",
                        "masterBinlog": "(mysql-bin.012238, 173661248)",
                        "masterBinlogGtid": "83ef3e40-f7f2-11ef-b39d-a088c260cf2a:1-3386987262",
                        "syncerBinlog": "(mysql-bin.012231, 191731262)",
                        "syncerBinlogGtid": "83ef3e40-f7f2-11ef-b39d-a088c260cf2a:1-519309589:5194575062",
                        "syncerBinlog": "(mysql-bin.012231, 191731262)",
                        "syncerBinlogGtid": "83ef3e40-f7f2-11ef-b39d-a088c260cf2a:1-519309589:519457506-3384457601",
                        "blockingDDLs": [
                        ],
                        "unresolvedGroups": [
                        ],
                        "synced": false,
                        "binlogType": "remote",
                        "secondsBehindMaster": "8050",
                        "blockDDLOwner": "",
                        "conflictMsg": "",
                        "totalRows": "405727",
                        "totalRps": "10143",
                        "recentRps": "0"
                    },

任务配置文件

# 任务名,多个同时运行的任务不能重名。
name: "dm-task-syc-1"
# 全量+增量 (all) 迁移模式。
task-mode: "all"
ignore-checking-items: ["all"]
# 下游储存 `meta` 信息的数据库
meta-schema: "dm_meta"
# 下游 TiDB 配置信息。
target-database:
  host: ".........."
  port: 6000
  user: "........"
  password: "............."
  session:
    sql_require_primary_key: OFF

mysql-instances:
  -
    source-id: "dm-source-1"
    block-allow-list:  "bw-rule-1"
    mydumper-config-name: "global"
    syncer-config-name: "global"
    filter-rules: ["filter-rule-1"]
    route-rules: ["route-rule-1"]

block-allow-list:
  bw-rule-1:
    do-dbs: ["db1"]
    do-tables:
    - db-name: "db1"
      tbl-name: "t1"
    - db-name: "db1"
      tbl-name: "t2"
    - db-name: "db1"
      tbl-name: "t3"
    - db-name: "db1"
      tbl-name: "t4"
    - db-name: "db1"
      tbl-name: "t5"
    - db-name: "db1"
      tbl-name: "t6"

filters:
  filter-rule-1:
    schema-pattern: "db1"
    table-pattern: "t1|t2|t3|t4|t5|t6"
    events: ["truncate table", "drop table"]
    action: Ignore

routes:
  route-rule-1:
    schema-pattern: "db1"
    target-schema: "new_db1"

syncers:
  global:
    safe-mode: true  # 启用安全模式

mydumpers:
  global:
    threads: 6
    chunk-filesize: 64    #单位为 MB


看起来是索引数据不一致的问题,可以根据文档进行解决
https://docs.pingcap.com/zh/tidb/stable/troubleshoot-data-inconsistency-errors/#error-8141

有没有永久的解决方法?不能每次遇到问题都先将参数改为OFF

https://github.com/pingcap/tidb/issues/57439
对应的表结构发下吧,可能是bug,如果是没解决的bug的话暂时只能绕过了

其实吧只要同步前校验源库和目标库的数据,过程中避免手动修改目标库数据,然后用DM监控,关注dm_syncer_error_count指标,及时发现异常

这种只能条跳过了。

没有遇到过这个问题。看了各位的回复,我也觉得是BUG。。

你是什么版本?

看上去的确像个bug啊

issues还没有被修复 :thinking:

此话题已在最后回复的 7 天后被自动关闭。不再允许新回复。