dumpling按条件备份大表数据 非常慢，好像卡住了没有进度

🪐 TiDB 技术问题
1

一个好的问题描述有利于社区小伙伴更快帮你定位到问题，高效解决你的问题

【TiDB 使用环境】生产环境
【TiDB 版本】v7.1.5
【部署方式】物理机部署
宽表数据，大概80字段，information_schema TABLE_ROWS: 64亿+数据量

我的dumpling条件数据量大概只有689万，执行dumpling后很慢，也没有啥进度。
我怀疑是我的命令不对？ 还是跟我的表有关 不适合用这个？
还有日志里[“estimate total rows”=6465665133] 这个64亿好像是我全量表数据，这个日志是啥意思？我的条件字段update_time是有索引的，为啥会打印全量数据呢

[tidb@b67 tidb-toolkit]$ ./dumpling -uroot -P 4000 -hb85 -p 'xxxx' -T bsppr.xpost --where "update_time < '2025-02-10 00:00:00'" -t 16  --no-schemas -o /tmp/dumpling-bsppr-xpost-test/
Release version: v7.1.5
Git commit hash: caa60c0917a886933a525d25e17057faac5b4da2
Git branch:      HEAD
Build timestamp: 2024-04-24 11:09:23Z
Go version:      go version go1.20.10 linux/amd64

[2026/07/10 09:39:56.959 +08:00] [INFO] [versions.go:54] ["Welcome to dumpling"] ["Release Version"=v7.1.5] ["Git Commit Hash"=caa60c0917a886933a525d25e17057faac5b4da2] ["Git Branch"=HEAD] ["Build timestamp"="2024-04-24 11:09:23"] ["Go Version"="go version go1.20.10 linux/amd64"]
[2026/07/10 09:39:56.972 +08:00] [INFO] [version.go:434] ["detect server version"] [type=TiDB] [version=7.1.5]
[2026/07/10 09:39:57.002 +08:00] [INFO] [client.go:311] ["[pd] create pd client with endpoints and keyspace"] [pd-address="[192.168.241.60:2379,192.168.241.59:2379,192.168.241.61:2379]"] [keyspace-id=0]
[2026/07/10 09:39:57.007 +08:00] [INFO] [pd_service_discovery.go:521] ["[pd] update member urls"] [old-urls="[http://192.168.241.60:2379,http://192.168.241.59:2379,http://192.168.241.61:2379]"] [new-urls="[http://192.168.241.59:2379,http://192.168.241.60:2379,http://192.168.241.61:2379]"]
[2026/07/10 09:39:57.007 +08:00] [INFO] [pd_service_discovery.go:546] ["[pd] switch leader"] [new-leader=http://192.168.241.59:2379] [old-leader=]
[2026/07/10 09:39:57.007 +08:00] [INFO] [pd_service_discovery.go:177] ["[pd] init cluster id"] [cluster-id=7210697100626091534]
[2026/07/10 09:39:57.008 +08:00] [INFO] [client.go:386] ["[pd] changing service mode"] [old-mode=UNKNOWN_SVC_MODE] [new-mode=PD_SVC_MODE]
[2026/07/10 09:39:57.008 +08:00] [INFO] [tso_client.go:231] ["[tso] switch dc tso allocator serving address"] [dc-location=global] [new-address=http://192.168.241.59:2379]
[2026/07/10 09:39:57.008 +08:00] [INFO] [tso_dispatcher.go:296] ["[tso] tso dispatcher created"] [dc-location=global]
[2026/07/10 09:39:57.008 +08:00] [INFO] [client.go:428] ["[pd] service mode changed"] [old-mode=PD_SVC_MODE] [new-mode=PD_SVC_MODE]
[2026/07/10 09:39:57.019 +08:00] [INFO] [dump.go:1519] ["generate dumpling gc safePoint id"] [id=dumpling_1783647597019778579]
[2026/07/10 09:39:57.066 +08:00] [INFO] [dump.go:151] ["begin to run Dump"] [conf="{\"s3\":{\"endpoint\":\"\",\"region\":\"\",\"storage-class\":\"\",\"sse\":\"\",\"sse-kms-key-id\":\"\",\"acl\":\"\",\"access-key\":\"\",\"secret-access-key\":\"\",\"session-token\":\"\",\"provider\":\"\",\"force-path-style\":true,\"use-accelerate-endpoint\":false,\"role-arn\":\"\",\"external-id\":\"\",\"object-lock-enabled\":false},\"gcs\":{\"endpoint\":\"\",\"storage-class\":\"\",\"predefined-acl\":\"\",\"credentials-file\":\"\"},\"azblob\":{\"endpoint\":\"\",\"account-name\":\"\",\"account-key\":\"\",\"access-tier\":\"\"},\"AllowCleartextPasswords\":false,\"SortByPk\":true,\"NoViews\":true,\"NoSequences\":true,\"NoHeader\":false,\"NoSchemas\":true,\"NoData\":false,\"CompleteInsert\":false,\"TransactionalConsistency\":true,\"EscapeBackslash\":true,\"DumpEmptyDatabase\":true,\"PosAfterConnect\":false,\"CompressType\":0,\"Host\":\"b85\",\"Port\":4000,\"Threads\":16,\"User\":\"root\",\"Security\":{\"CAPath\":\"\",\"CertPath\":\"\",\"KeyPath\":\"\"},\"LogLevel\":\"info\",\"LogFile\":\"\",\"LogFormat\":\"text\",\"OutputDirPath\":\"/tmp/dumpling-bsppr-xpost-test/\",\"StatusAddr\":\":8281\",\"Snapshot\":\"467572515662463073\",\"Consistency\":\"snapshot\",\"CsvNullValue\":\"\\\\N\",\"SQL\":\"\",\"CsvSeparator\":\",\",\"CsvDelimiter\":\"\\\"\",\"Databases\":[],\"Where\":\"update_time \\u003c '2025-02-10 00:00:00'\",\"FileType\":\"sql\",\"ServerInfo\":{\"ServerType\":3,\"ServerVersion\":\"7.1.5\",\"HasTiKV\":true},\"Rows\":0,\"ReadTimeout\":900000000000,\"TiDBMemQuotaQuery\":0,\"FileSize\":0,\"StatementSize\":1000000,\"SessionParams\":{\"tidb_snapshot\":\"467572515662463073\"},\"Tables\":{\"bsppr\":[{\"Name\":\"xpost\",\"AvgRowLength\":0,\"Type\":0}]},\"CollationCompatible\":\"loose\",\"IOTotalBytes\":null,\"Net\":\"\"}"]




[2026/07/10 09:41:57.450 +08:00] [INFO] [status.go:37] [progress] [tables="0/1 (0.0%)"] ["finished rows"=0] ["estimate total rows"=6465665133] ["finished size"=0B] ["average speed(MiB/s)"=0] ["recent speed bps"=0] ["chunks progress"=]
[2026/07/10 09:43:57.449 +08:00] [INFO] [status.go:37] [progress] [tables="0/1 (0.0%)"] ["finished rows"=0] ["estimate total rows"=6465665133] ["finished size"=0B] ["average speed(MiB/s)"=0] ["recent speed bps"=0] ["chunks progress"=]
[2026/07/10 09:45:57.449 +08:00] [INFO] [status.go:37] [progress] [tables="0/1 (0.0%)"] ["finished rows"=0] ["estimate total rows"=6465665133] ["finished size"=0B] ["average speed(MiB/s)"=0] ["recent speed bps"=0] ["chunks progress"=]
[2026/07/10 09:47:57.449 +08:00] [INFO] [status.go:37] [progress] [tables="0/1 (0.0%)"] ["finished rows"=0] ["estimate total rows"=6465665133] ["finished size"=0B] ["average speed(MiB/s)"=0] ["recent speed bps"=0] ["chunks progress"=]
2

dumpling 导出时建议加上 -F 256MiB -r 100000。

estimate total rows = 64 亿 是怎么来的

这是 Dumpling 启动时从 information_schema.TABLES.TABLE_ROWS 读取的整张表行数估算，跟 --where 完全无关，只是用来做进度分母和分块参考。所以它会把全量 64 亿当成总量

为什么卡住没进度

只加了 -t 16，没加 -r。

  • -r 控制「in-table concurrency」，默认是 0（关闭），意味着表内不切分片，只能顺序扫；

  • 你的 --where 用的是 update_time，不是主键列，Dumpling 又无法按 PK 区间切片（PK int/string 才会用 region 信息切）；

  • 结果：6.4 亿行的宽表（80 列），Dumpling 在等生成一个超大单 chunk，表面上像「卡死」。

1 个赞
3

多加几个参数，如下所示：

image
image1707×1109 147 KB

1 个赞
4

按你说的加上了-F 256MiB -r 100000 ，但感觉还是一样的效果

[2026/07/10 11:28:08.229 +08:00] [INFO] [versions.go:54] ["Welcome to dumpling"] ["Release Version"=v7.1.5] ["Git Commit Hash"=caa60c0917a886933a525d25e17057faac5b4da2] ["Git Branch"=HEAD] ["Build timestamp"="2024-04-24 11:09:23"] ["Go Version"="go version go1.20.10 linux/amd64"]
[2026/07/10 11:28:08.233 +08:00] [INFO] [version.go:434] ["detect server version"] [type=TiDB] [version=7.1.5]
[2026/07/10 11:28:08.245 +08:00] [INFO] [client.go:311] ["[pd] create pd client with endpoints and keyspace"] [pd-address="[192.168.241.60:2379,192.168.241.59:2379,192.168.241.61:2379]"] [keyspace-id=0]
[2026/07/10 11:28:08.250 +08:00] [INFO] [pd_service_discovery.go:521] ["[pd] update member urls"] [old-urls="[http://192.168.241.60:2379,http://192.168.241.59:2379,http://192.168.241.61:2379]"] [new-urls="[http://192.168.241.59:2379,http://192.168.241.60:2379,http://192.168.241.61:2379]"]
[2026/07/10 11:28:08.250 +08:00] [INFO] [pd_service_discovery.go:546] ["[pd] switch leader"] [new-leader=http://192.168.241.59:2379] [old-leader=]
[2026/07/10 11:28:08.250 +08:00] [INFO] [pd_service_discovery.go:177] ["[pd] init cluster id"] [cluster-id=7210697100626091534]
[2026/07/10 11:28:08.250 +08:00] [INFO] [client.go:386] ["[pd] changing service mode"] [old-mode=UNKNOWN_SVC_MODE] [new-mode=PD_SVC_MODE]
[2026/07/10 11:28:08.250 +08:00] [INFO] [tso_client.go:231] ["[tso] switch dc tso allocator serving address"] [dc-location=global] [new-address=http://192.168.241.59:2379]
[2026/07/10 11:28:08.251 +08:00] [INFO] [tso_dispatcher.go:296] ["[tso] tso dispatcher created"] [dc-location=global]
[2026/07/10 11:28:08.251 +08:00] [INFO] [client.go:428] ["[pd] service mode changed"] [old-mode=PD_SVC_MODE] [new-mode=PD_SVC_MODE]
[2026/07/10 11:28:08.252 +08:00] [INFO] [dump.go:1519] ["generate dumpling gc safePoint id"] [id=dumpling_1783654088252638471]
[2026/07/10 11:28:08.262 +08:00] [INFO] [dump.go:151] ["begin to run Dump"] [conf="{\"s3\":{\"endpoint\":\"\",\"region\":\"\",\"storage-class\":\"\",\"sse\":\"\",\"sse-kms-key-id\":\"\",\"acl\":\"\",\"access-key\":\"\",\"secret-access-key\":\"\",\"session-token\":\"\",\"provider\":\"\",\"force-path-style\":true,\"use-accelerate-endpoint\":false,\"role-arn\":\"\",\"external-id\":\"\",\"object-lock-enabled\":false},\"gcs\":{\"endpoint\":\"\",\"storage-class\":\"\",\"predefined-acl\":\"\",\"credentials-file\":\"\"},\"azblob\":{\"endpoint\":\"\",\"account-name\":\"\",\"account-key\":\"\",\"access-tier\":\"\"},\"AllowCleartextPasswords\":false,\"SortByPk\":true,\"NoViews\":true,\"NoSequences\":true,\"NoHeader\":false,\"NoSchemas\":true,\"NoData\":false,\"CompleteInsert\":false,\"TransactionalConsistency\":true,\"EscapeBackslash\":true,\"DumpEmptyDatabase\":true,\"PosAfterConnect\":false,\"CompressType\":0,\"Host\":\"b85\",\"Port\":4000,\"Threads\":16,\"User\":\"root\",\"Security\":{\"CAPath\":\"\",\"CertPath\":\"\",\"KeyPath\":\"\"},\"LogLevel\":\"info\",\"LogFile\":\"\",\"LogFormat\":\"text\",\"OutputDirPath\":\"/tmp/dumpling-bsppr-xpost-test/\",\"StatusAddr\":\":8281\",\"Snapshot\":\"467574217304440902\",\"Consistency\":\"snapshot\",\"CsvNullValue\":\"\\\\N\",\"SQL\":\"\",\"CsvSeparator\":\",\",\"CsvDelimiter\":\"\\\"\",\"Databases\":[],\"Where\":\"update_time \\u003c '2025-02-10 00:00:00'\",\"FileType\":\"sql\",\"ServerInfo\":{\"ServerType\":3,\"ServerVersion\":\"7.1.5\",\"HasTiKV\":true},\"Rows\":100000,\"ReadTimeout\":900000000000,\"TiDBMemQuotaQuery\":0,\"FileSize\":268435456,\"StatementSize\":1000000,\"SessionParams\":{\"tidb_snapshot\":\"467574217304440902\"},\"Tables\":{\"bsppr\":[{\"Name\":\"xpost\",\"AvgRowLength\":0,\"Type\":0}]},\"CollationCompatible\":\"loose\",\"IOTotalBytes\":null,\"Net\":\"\"}"]


[2026/07/10 11:30:08.379 +08:00] [INFO] [status.go:37] [progress] [tables="0/1 (0.0%)"] ["finished rows"=0] ["estimate total rows"=0] ["finished size"=0B] ["average speed(MiB/s)"=0] ["recent speed bps"=0] ["chunks progress"=]
[2026/07/10 11:32:08.378 +08:00] [INFO] [status.go:37] [progress] [tables="0/1 (0.0%)"] ["finished rows"=0] ["estimate total rows"=6467606429] ["finished size"=0B] ["average speed(MiB/s)"=0] ["recent speed bps"=0] ["chunks progress"=]
[2026/07/10 11:34:08.378 +08:00] [INFO] [status.go:37] [progress] [tables="0/1 (0.0%)"] ["finished rows"=0] ["estimate total rows"=6467606429] ["finished size"=0B] ["average speed(MiB/s)"=0] ["recent speed bps"=0] ["chunks progress"=]
[2026/07/10 11:36:08.379 +08:00] [INFO] [status.go:37] [progress] [tables="0/1 (0.0%)"] ["finished rows"=0] ["estimate total rows"=6467606429] ["finished size"=0B] ["average speed(MiB/s)"=0] ["recent speed bps"=0] ["chunks progress"=]
5

最后竟然报错了，不止是慢。

[tidb@b67 tidb-toolkit]$ ./dumpling -uroot -P 4000 -hb85 -p 'xxxx' -T bsppr.xpost --where "update_time < '2025-02-10 00:00:00'" -t 16  -o /tmp/dumpling-bsppr-xpost-test -r 100000 -F 256MiBRelease version: v7.1.5
Git commit hash: caa60c0917a886933a525d25e17057faac5b4da2
Git branch:      HEAD
Build timestamp: 2024-04-24 11:09:23Z
Go version:      go version go1.20.10 linux/amd64

[2026/07/10 13:20:46.505 +08:00] [INFO] [versions.go:54] ["Welcome to dumpling"] ["Release Version"=v7.1.5] ["Git Commit Hash"=caa60c0917a886933a525d25e17057faac5b4da2] ["Git Branch"=HEAD] ["Build timestamp"="2024-04-24 11:09:23"] ["Go Version"="go version go1.20.10 linux/amd64"]
[2026/07/10 13:20:46.510 +08:00] [INFO] [version.go:434] ["detect server version"] [type=TiDB] [version=7.1.5]
[2026/07/10 13:20:46.521 +08:00] [INFO] [client.go:311] ["[pd] create pd client with endpoints and keyspace"] [pd-address="[192.168.241.60:2379,192.168.241.59:2379,192.168.241.61:2379]"] [keyspace-id=0]
[2026/07/10 13:20:46.527 +08:00] [INFO] [pd_service_discovery.go:521] ["[pd] update member urls"] [old-urls="[http://192.168.241.60:2379,http://192.168.241.59:2379,http://192.168.241.61:2379]"] [new-urls="[http://192.168.241.59:2379,http://192.168.241.60:2379,http://192.168.241.61:2379]"]
[2026/07/10 13:20:46.527 +08:00] [INFO] [pd_service_discovery.go:546] ["[pd] switch leader"] [new-leader=http://192.168.241.59:2379] [old-leader=]
[2026/07/10 13:20:46.527 +08:00] [INFO] [pd_service_discovery.go:177] ["[pd] init cluster id"] [cluster-id=7210697100626091534]
[2026/07/10 13:20:46.528 +08:00] [INFO] [client.go:386] ["[pd] changing service mode"] [old-mode=UNKNOWN_SVC_MODE] [new-mode=PD_SVC_MODE]
[2026/07/10 13:20:46.528 +08:00] [INFO] [tso_client.go:231] ["[tso] switch dc tso allocator serving address"] [dc-location=global] [new-address=http://192.168.241.59:2379]
[2026/07/10 13:20:46.528 +08:00] [INFO] [tso_dispatcher.go:296] ["[tso] tso dispatcher created"] [dc-location=global]
[2026/07/10 13:20:46.528 +08:00] [INFO] [client.go:428] ["[pd] service mode changed"] [old-mode=PD_SVC_MODE] [new-mode=PD_SVC_MODE]
[2026/07/10 13:20:46.529 +08:00] [INFO] [dump.go:1519] ["generate dumpling gc safePoint id"] [id=dumpling_1783660846529742592]
[2026/07/10 13:20:46.544 +08:00] [INFO] [dump.go:151] ["begin to run Dump"] [conf="{\"s3\":{\"endpoint\":\"\",\"region\":\"\",\"storage-class\":\"\",\"sse\":\"\",\"sse-kms-key-id\":\"\",\"acl\":\"\",\"access-key\":\"\",\"secret-access-key\":\"\",\"session-token\":\"\",\"provider\":\"\",\"force-path-style\":true,\"use-accelerate-endpoint\":false,\"role-arn\":\"\",\"external-id\":\"\",\"object-lock-enabled\":false},\"gcs\":{\"endpoint\":\"\",\"storage-class\":\"\",\"predefined-acl\":\"\",\"credentials-file\":\"\"},\"azblob\":{\"endpoint\":\"\",\"account-name\":\"\",\"account-key\":\"\",\"access-tier\":\"\"},\"AllowCleartextPasswords\":false,\"SortByPk\":true,\"NoViews\":true,\"NoSequences\":true,\"NoHeader\":false,\"NoSchemas\":false,\"NoData\":false,\"CompleteInsert\":false,\"TransactionalConsistency\":true,\"EscapeBackslash\":true,\"DumpEmptyDatabase\":true,\"PosAfterConnect\":false,\"CompressType\":0,\"Host\":\"b85\",\"Port\":4000,\"Threads\":16,\"User\":\"root\",\"Security\":{\"CAPath\":\"\",\"CertPath\":\"\",\"KeyPath\":\"\"},\"LogLevel\":\"info\",\"LogFile\":\"\",\"LogFormat\":\"text\",\"OutputDirPath\":\"/tmp/dumpling-bsppr-xpost-test\",\"StatusAddr\":\":8281\",\"Snapshot\":\"467575988952236033\",\"Consistency\":\"snapshot\",\"CsvNullValue\":\"\\\\N\",\"SQL\":\"\",\"CsvSeparator\":\",\",\"CsvDelimiter\":\"\\\"\",\"Databases\":[],\"Where\":\"update_time \\u003c '2025-02-10 00:00:00'\",\"FileType\":\"sql\",\"ServerInfo\":{\"ServerType\":3,\"ServerVersion\":\"7.1.5\",\"HasTiKV\":true},\"Rows\":100000,\"ReadTimeout\":900000000000,\"TiDBMemQuotaQuery\":0,\"FileSize\":268435456,\"StatementSize\":1000000,\"SessionParams\":{\"tidb_snapshot\":\"467575988952236033\"},\"Tables\":{\"bsppr\":[{\"Name\":\"xpost\",\"AvgRowLength\":0,\"Type\":0}]},\"CollationCompatible\":\"loose\",\"IOTotalBytes\":null,\"Net\":\"\"}"]
[2026/07/10 13:22:46.642 +08:00] [INFO] [status.go:37] [progress] [tables="0/1 (0.0%)"] ["finished rows"=0] ["estimate total rows"=6469461330] ["finished size"=0B] ["average speed(MiB/s)"=0] ["recent speed bps"=0] ["chunks progress"=]
[2026/07/10 13:24:46.642 +08:00] [INFO] [status.go:37] [progress] [tables="0/1 (0.0%)"] ["finished rows"=0] ["estimate total rows"=6469461330] ["finished size"=0B] ["average speed(MiB/s)"=0] ["recent speed bps"=0] ["chunks progress"=]
[2026/07/10 13:26:46.642 +08:00] [INFO] [status.go:37] [progress] [tables="0/1 (0.0%)"] ["finished rows"=0] ["estimate total rows"=6469461330] ["finished size"=0B] ["average speed(MiB/s)"=0] ["recent speed bps"=0] ["chunks progress"=]
[2026/07/10 13:28:46.642 +08:00] [INFO] [status.go:37] [progress] [tables="0/1 (0.0%)"] ["finished rows"=0] ["estimate total rows"=6469461330] ["finished size"=0B] ["average speed(MiB/s)"=0] ["recent speed bps"=0] ["chunks progress"=]
[2026/07/10 13:30:46.642 +08:00] [INFO] [status.go:37] [progress] [tables="0/1 (0.0%)"] ["finished rows"=0] ["estimate total rows"=6469461330] ["finished size"=0B] ["average speed(MiB/s)"=0] ["recent speed bps"=0] ["chunks progress"=]
[2026/07/10 13:32:46.642 +08:00] [INFO] [status.go:37] [progress] [tables="0/1 (0.0%)"] ["finished rows"=0] ["estimate total rows"=6469461330] ["finished size"=0B] ["average speed(MiB/s)"=0] ["recent speed bps"=0] ["chunks progress"=]
[2026/07/10 13:34:46.642 +08:00] [INFO] [status.go:37] [progress] [tables="0/1 (0.0%)"] ["finished rows"=0] ["estimate total rows"=6469461330] ["finished size"=0B] ["average speed(MiB/s)"=0] ["recent speed bps"=0] ["chunks progress"=]
[2026/07/10 13:36:46.642 +08:00] [INFO] [status.go:37] [progress] [tables="0/1 (0.0%)"] ["finished rows"=0] ["estimate total rows"=6469461330] ["finished size"=0B] ["average speed(MiB/s)"=0] ["recent speed bps"=0] ["chunks progress"=]
[mysql] 2026/07/10 13:37:46 packets.go:37: read tcp 192.168.241.67:58932->192.168.241.85:4000: i/o timeout
[2026/07/10 13:37:46.567 +08:00] [INFO] [conn.go:44] ["cannot execute query"] [retryTime=1] [sql="SELECT `postid` FROM `bsppr`.`xpost` TABLESAMPLE REGIONS() ORDER BY `postid`"] [args=null] [error="sql: SELECT `postid` FROM `bsppr`.`xpost` TABLESAMPLE REGIONS() ORDER BY `postid`, args: []: invalid connection"] [errorVerbose="invalid connection\nsql: SELECT `postid` FROM `bsppr`.`xpost` TABLESAMPLE REGIONS() ORDER BY `postid`, args: []\ngithub.com/pingcap/tidb/dumpling/export.simpleQueryWithArgs\n\tgithub.com/pingcap/tidb/dumpling/export/sql.go:1153\ngithub.com/pingcap/tidb/dumpling/export.(*BaseConn).QuerySQL.func1\n\tgithub.com/pingcap/tidb/dumpling/export/conn.go:42\ngithub.com/pingcap/tidb/br/pkg/utils.WithRetry.func1\n\tgithub.com/pingcap/tidb/br/pkg/utils/retry.go:215\ngithub.com/pingcap/tidb/br/pkg/utils.WithRetryV2[...]\n\tgithub.com/pingcap/tidb/br/pkg/utils/retry.go:233\ngithub.com/pingcap/tidb/br/pkg/utils.WithRetry\n\tgithub.com/pingcap/tidb/br/pkg/utils/retry.go:214\ngithub.com/pingcap/tidb/dumpling/export.(*BaseConn).QuerySQL\n\tgithub.com/pingcap/tidb/dumpling/export/conn.go:34\ngithub.com/pingcap/tidb/dumpling/export.selectTiDBTableSample\n\tgithub.com/pingcap/tidb/dumpling/export/dump.go:980\ngithub.com/pingcap/tidb/dumpling/export.(*Dumper).concurrentDumpTiDBTables\n\tgithub.com/pingcap/tidb/dumpling/export/dump.go:880\ngithub.com/pingcap/tidb/dumpling/export.(*Dumper).concurrentDumpTable\n\tgithub.com/pingcap/tidb/dumpling/export/dump.go:737\ngithub.com/pingcap/tidb/dumpling/export.(*Dumper).dumpTableData\n\tgithub.com/pingcap/tidb/dumpling/export/dump.go:627\ngithub.com/pingcap/tidb/dumpling/export.(*Dumper).dumpDatabases\n\tgithub.com/pingcap/tidb/dumpling/export/dump.go:478\ngithub.com/pingcap/tidb/dumpling/export.(*Dumper).Dump\n\tgithub.com/pingcap/tidb/dumpling/export/dump.go:315\nmain.main\n\t./main.go:75\nruntime.main\n\truntime/proc.go:250\nruntime.goexit\n\truntime/asm_amd64.s:1598"]
6

关键问题在于 --where 里的过滤没有让 dumpling 高效分块。dumpling 默认按主键（或 _tidb_rowid）做区间分块并发导出，你加了 update_time < ‘xxx’ 这种非主键条件后，它没法用 update_time 索引来切分块，只能对全表做区间扫描再逐行判断 where，所以日志里 estimate total rows 显示的是全量 64 亿——这个值是 dumpling 估算的整表行数（用于分块规划），不是你条件命中的行数。这就是慢的根因：等于全表扫 64 亿行去筛你那 689 万。优化方向：1）如果导出目的是按 update_time 归档，考虑用主键范围配合，或者先确认表是否按 update_time 做了分区，用分区裁剪导出对应分区；2）加大 -t（并发）意义不大，因为瓶颈在全表扫描；3）对这种大宽表 + 条件导出，更适合直接用 SQL（SELECT … WHERE update_time < xxx，走 update_time 索引）配合 SELECT INTO OUTFILE 或分批导出，让优化器用上索引；4）也可以用 --sql 参数给 dumpling 传一条能走索引的自定义查询。核心是让扫描走 update_time 索引而不是全表分块。

1 个赞
7

感谢各位， 找到问题了，是我的条件没走索引， 通过 --sql " force index" 速度就很快了，700万数据2分钟就跑完了。
就很奇怪， 明明是加了索引，为啥找不到索引，好像我们项目上也遇到过这问题

8

这问题我遇到过。dumpling加–where条件时，它会先做一次全表扫描估算总行数（就是日志里那个64亿），然后再根据条件过滤。所以刚开始会卡在扫描阶段，不是真的卡住。

9

此话题已在最后回复的 7 天后被自动关闭。不再允许新回复。