TIDB升级到8.5.6后,应用端语句执行变慢

【TiDB 使用环境】生产环境
【TiDB 版本】v8.5.6
【部署方式】虚拟机部署
【遇到的问题:问题现象及影响】数据库版本从v8.5.1升级到8.5.6后,有一个语句在应用端执行需要10秒–300秒,在数据库端执行只需要100毫秒,升级之前在应用端执行也是300毫秒以内。这是什么原因导致的?
看了执行计划的区别主要是在pipeline_queue_wait,fetch and wait,min_local_stream,max_local_stream在执行慢的时候比较大。
看了CPU,内存,IO都不高。
【执行语句如下】

select count(1) from (
                SELECT 
                T1.ID, 
                CASE IFNULL(T1.ReviewSources,'')  WHEN 'ReviewComplaint' THEN 'review客诉' WHEN 'MailComplaint' THEN '邮件客诉' WHEN 'ReturnComplaint' THEN '退货客诉' ELSE '' END AS ReviewSources, /*评论来源*/
                IFNULL(T2.FValue,'') AS Site, /*站点*/
                IFNULL(T1.SKU,'') AS SKU, /*SKU*/
                IFNULL(T1.ASIN,'') AS ASIN, /*ASIN*/
                IFNULL(T6.Name,'') AS ProductName, /*产品名称*/
                IFNULL(T6.ProductImageFrontViewUrl,'') AS ProductImageFrontViewUrl, /*产品图*/
                IFNULL(T6.LargeCode,'') AS LargeCode, /*大编码*/
                IFNULL(T3.CategoryName,'') AS FirstCategoryName,/*一级类目名称*/
                IFNULL(T4.CategoryName,'') AS SecondCategoryName,/*二级类目名称*/
                IFNULL(T5.CategoryName,'') AS ThirdCategoryName,/*三级类目名称*/
                IFNULL(T6.ThirdCategoryID,0) AS ThirdCategoryID,/*三级类目内码*/
                IFNULL(T7.CQELabel,'') AS CQELabel,/*CQE标签*/
                '' AS KeyTypes,/*重点商品类别*/
                IFNULL(T1.UserName,'') AS UserName,/*用户名称*/
                CASE 
                WHEN IFNULL(T1.Score,'0')='0' OR IFNULL(T1.Score,'0')='' THEN '0'
                ELSE IFNULL(T1.Score,'0')
                END AS Score,/*评分*/ 
                IFNULL(T1.Title,'') AS Title,/*标题*/ 
                DATE_FORMAT(T1.OrderDate, '%Y-%m-%d') AS OrderDate,  /*订单时间*/
                DATE_FORMAT(T1.PaymentDate, '%Y-%m-%d %H:%i:%s') AS PaymentDate,  /*付款时间*/
                IFNULL(T1.OrderNum,'') AS OrderNum,/*订单单号*/ 
                IFNULL(T1.CommentDetails,'') AS CommentDetails,/*评论详情正文*/ 
                IFNULL(T1.ChineseCommentDetails,'') AS ChineseCommentDetails,/*中文正文*/  
                CASE 
                WHEN T1.IsContactBuyer=0 THEN '否'
                WHEN T1.IsContactBuyer=1 THEN '是' 
                WHEN T1.IsContactBuyer IS NULL THEN '' 
                END AS IsContactBuyer, /*是否联系买家*/
                CASE 
                WHEN T1.IsBuyerReply=0 THEN '否'
                WHEN T1.IsBuyerReply=1 THEN '是' 
                WHEN T1.IsBuyerReply IS NULL THEN '' 
                END AS IsBuyerReply, /*买家是否回复*/
                DATE_FORMAT(T1.CommentDate, '%Y-%m-%d') AS CommentDate,  /*评论/退货日期*/
                IFNULL(T1.CommentSourceCountry,'') AS CommentSourceCountry,  /*评论来源国家*/
                IFNULL(T1.CommentLikesCount,'') AS CommentLikesCount, /*评论点赞数*/
                IFNULL(T1.AuthenticatedUser,'') AS AuthenticatedUser, /*验证用户*/
                '' AS StarRating, /*星级*/
                0 AS CommentsQty, /*评论总数*/
                IFNULL(T1.CommentAddress,'') AS CommentAddress, /*评论地址*/
                CASE WHEN T1.IsClassify=1 THEN '是' ELSE '否' END AS IsClassify, /*是否分类*/
                CASE WHEN T1.IsAIMark=1 THEN '是' ELSE '否' END AS IsAIMark, /*是否AI打标*/
                CASE IFNULL(T1.CommentLevel,'')  WHEN 'Negative' THEN '差评' WHEN 'Praise' THEN '好评' ELSE '' END AS CommentLevel, /*评论级别*/
                IFNULL(T1.ClassifyBy,'') AS ClassifyBy, /*分类人*/
                IFNULL(T1.CreateBy,'') AS CreateBy,  /*创建人*/
                DATE_FORMAT(T1.CreateDate, '%Y-%m-%d %H:%i:%s') AS CreateDate,  /*创建时间*/
                IFNULL(T1.LastBy,'') AS LastBy,  /*修改人*/
                DATE_FORMAT(T1.LastDate, '%Y-%m-%d %H:%i:%s') AS LastDate  /*修改时间*/
                FROM O_AmazonComments T1
                 LEFT JOIN (SELECT T2.FKey,T2.FValue  FROM O_DataDictionary T1
                  LEFT JOIN O_DataDictionary T2 ON T2.ParentKey =T1.FKey 
                 WHERE T1.ParentKey='Amazon') T2 ON UPPER(T1.Site)=UPPER(T2.FKey)
                 LEFT JOIN O_SKUList T6 ON T1.SKU=T6.Number 
                 LEFT JOIN O_Category T3 ON T6.FirstCategoryID =T3.CategoryId 
                 LEFT JOIN O_Category T4 ON T6.SecondCategoryID =T4.CategoryId 
                 LEFT JOIN O_Category T5 ON T6.ThirdCategoryID =T5.CategoryId 
                 LEFT JOIN (SELECT DISTINCT
                T1.CQELabel,
                T2.Number
                FROM O_CQELabel T1
                LEFT JOIN O_SKUList T2 ON T2.ID=T1.SKUId 
                WHERE T1.ForbiddenStatus=0) T7 ON T1.SKU=T7.Number 
                WHERE 1=1
             AND T1.CommentDate >= '2026-03-01 00:00:00' AND T1.CommentDate < '2026-05-19 00:00:00') t

慢的执行计划如下


| id                                  | estRows  | estCost      | actRows | task         | access object                         | execution info                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | operator info                                                                                                                                                                                                        | memory   | disk     |
| HashAgg_43                          | 1.00     | 16781718.70  | 1       | root         |                                       | time:4m27.7s, loops:2, partial_worker:{wall_time:4m27.671134087s, concurrency:5, task_num:23, tot_wait:22m18.35316709s, tot_exec:2.085199ms, tot_time:22m18.355522246s, max:4m27.671113734s, p95:4m27.671113734s}, final_worker:{wall_time:4m27.671192876s, concurrency:5, task_num:9, tot_wait:8.784µs, tot_exec:3.053µs, tot_time:22m18.355694523s, max:4m27.671147819s, p95:4m27.671147819s}                                                                                                                                                                                                                                                                                                                                           | funcs:count(1)->Column#156                                                                                                                                                                                           | 311.8 KB | 0 Bytes  |
| └─HashJoin_45                       | 12409.31 | 16532521.65  | 21694   | root         |                                       | time:4m27.7s, loops:24, build_hash_table:{total:23.5ms, fetch:23.1ms, build:350.3µs}, probe:{concurrency:5, total:22m18.4s, max:4m27.7s, probe:3ms, fetch and wait:22m18.4s}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | left outer join, equal:[eq(MB.O_amazoncomments.sku, MB.O_skulist.number)]                                                                                                                      | 172.9 KB | 0 Bytes  |
|   ├─HashAgg_95(Build)               | 2284.62  | 1230246.64   | 2143    | root         |                                       | time:23.1ms, loops:6, partial_worker:{wall_time:23.0274ms, concurrency:5, task_num:5, tot_wait:112.789731ms, tot_exec:1.001199ms, tot_time:114.770574ms, max:22.958401ms, p95:22.958401ms}, final_worker:{wall_time:23.142505ms, concurrency:5, task_num:25, tot_wait:10.503µs, tot_exec:204.811µs, tot_time:115.417983ms, max:23.112896ms, p95:23.112896ms}                                                                                                                                                                                                                                                                                                                                                                              | group by:MB.O_cqelabel.cqelabel, MB.O_skulist.number, funcs:firstrow(MB.O_skulist.number)->MB.O_skulist.number                                                           | 567.1 KB | 0 Bytes  |
|   │ └─HashJoin_108                  | 2284.62  | 1058089.14   | 2143    | root         |                                       | time:22.8ms, loops:6, build_hash_table:{total:5.95ms, fetch:5.68ms, build:262.5µs}, probe:{concurrency:5, total:112.3ms, max:22.7ms, probe:2.12ms, fetch and wait:110.1ms}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                | inner join, equal:[eq(MB.O_cqelabel.skuid, MB.O_skulist.id)]                                                                                                                                   | 254.5 KB | 0 Bytes  |
|   │   ├─TableReader_111(Build)      | 2284.62  | 86814.91     | 2143    | root         |                                       | time:5.8ms, loops:4, cop_task: {num: 4, max: 1.99ms, min: 1.09ms, avg: 1.47ms, p95: 1.99ms, max_proc_keys: 992, p95_proc_keys: 992, tot_proc: 1.93ms, tot_wait: 1.07ms, copr_cache_hit_ratio: 0.00, build_task_duration: 2.17µs, max_distsql_concurrency: 1}, rpc_info:{Cop:{num_rpc:4, total_time:5.82ms}}                                                                                                                                                                                                                                                                                                                                                                                                                               | data:Selection_110                                                                                                                                                                                                   | 51.3 KB  | N/A      |
|   │   │ └─Selection_110             | 2284.62  | 853215.57    | 2143    | cop[tikv]    |                                       | tikv_task:{proc max:1ms, min:0s, avg: 250µs, p80:1ms, p95:1ms, iters:17, tasks:4}, scan_detail: {total_process_keys: 2287, total_process_keys_size: 227552, total_keys: 2291, get_snapshot_time: 846.4µs, rocksdb: {key_skipped_count: 2287, block: {cache_hit_count: 52}}}, time_detail: {total_process_time: 1.93ms, total_wait_time: 1.07ms, total_kv_read_wall_time: 1ms, tikv_wall_time: 3.74ms}                                                                                                                                                                                                                                                                                                                                     | eq(MB.O_cqelabel.forbiddenstatus, 0)                                                                                                                                                                      | N/A      | N/A      |
|   │   │   └─TableFullScan_109       | 2287.00  | 739094.27    | 2287    | cop[tikv]    | table:T1                              | tikv_task:{proc max:1ms, min:0s, avg: 250µs, p80:1ms, p95:1ms, iters:17, tasks:4}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         | keep order:false                                                                                                                                                                                                     | N/A      | N/A      |
|   │   └─IndexReader_113(Probe)      | 17853.00 | 362748.71    | 17853   | root         |                                       | time:19.6ms, loops:19, cop_task: {num: 7, max: 6.62ms, min: 419.6µs, avg: 3.13ms, p95: 6.62ms, max_proc_keys: 6013, p95_proc_keys: 6013, tot_proc: 14.6ms, tot_wait: 1.34ms, copr_cache_hit_ratio: 0.14, build_task_duration: 1.2µs, max_distsql_concurrency: 1}, rpc_info:{Cop:{num_rpc:7, total_time:21.8ms}}                                                                                                                                                                                                                                                                                                                                                                                                                           | index:IndexFullScan_112                                                                                                                                                                                              | 233.5 KB | N/A      |
|   │     └─IndexFullScan_112         | 17853.00 | 3482192.42   | 17853   | cop[tikv]    | table:T2, index:AK_Key_Number(Number) | tikv_task:{proc max:4ms, min:1ms, avg: 2.29ms, p80:4ms, p95:4ms, iters:44, tasks:7}, scan_detail: {total_process_keys: 16861, total_process_keys_size: 1008117, total_keys: 16867, get_snapshot_time: 917.7µs, rocksdb: {delete_skipped_count: 50592, key_skipped_count: 16861, block: {cache_hit_count: 142}}}, time_detail: {total_process_time: 14.6ms, total_suspend_time: 73.5µs, total_wait_time: 1.34ms, total_kv_read_wall_time: 14ms, tikv_wall_time: 17.4ms}                                                                                                                                                                                                                                                                    | keep order:false                                                                                                                                                                                                     | N/A      | N/A      |
|   └─HashJoin_47(Probe)              | 12409.31 | 14772079.52  | 21694   | root         |                                       | time:4m27.7s, loops:25, build_hash_table:{total:2.51ms, fetch:2.5ms, build:13.8µs}, probe:{concurrency:5, total:22m18.4s, max:4m27.7s, probe:7.71ms, fetch and wait:22m18.3s}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             | left outer join, equal:[eq(Column#157, Column#158)]                                                                                                                                                                  | 108.6 KB | 0 Bytes  |
|     ├─Projection_86(Build)          | 123.00   | 27751.52     | 27      | root         |                                       | time:2.5ms, loops:3, Concurrency:OFF                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      | upper(MB.O_datadictionary.fkey)->Column#158                                                                                                                                                               | 16.2 KB  | N/A      |
|     │ └─HashJoin_88                 | 123.00   | 26523.98     | 27      | root         |                                       | time:2.47ms, loops:3, build_hash_table:{total:1.64ms, fetch:1.62ms, build:19.8µs}, probe:{concurrency:5, total:11.9ms, max:2.41ms, probe:41.8µs, fetch and wait:11.9ms}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   | left outer join, equal:[eq(MB.O_datadictionary.fkey, MB.O_datadictionary.parentkey)]                                                                                                           | 108.6 KB | 0 Bytes  |
|     │   ├─TableReader_91(Build)     | 39.00    | 7534.34      | 26      | root         |                                       | time:1.61ms, loops:2, cop_task: {num: 1, max: 1.84ms, proc_keys: 246, tot_proc: 446.2µs, tot_wait: 647.8µs, copr_cache_hit_ratio: 0.00, build_task_duration: 4.12µs, max_distsql_concurrency: 1}, rpc_info:{Cop:{num_rpc:1, total_time:1.82ms}}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           | data:Selection_90                                                                                                                                                                                                    | 1.33 KB  | N/A      |
|     │   │ └─Selection_90            | 39.00    | 108740.15    | 26      | cop[tikv]    |                                       | tikv_task:{time:0s, loops:4}, scan_detail: {total_process_keys: 246, total_process_keys_size: 31116, total_keys: 247, get_snapshot_time: 545.8µs, rocksdb: {key_skipped_count: 246, block: {cache_hit_count: 13}}}, time_detail: {total_process_time: 446.2µs, total_wait_time: 647.8µs, tikv_wall_time: 1.42ms}                                                                                                                                                                                                                                                                                                                                                                                                                          | eq(MB.O_datadictionary.parentkey, "Amazon")                                                                                                                                                               | N/A      | N/A      |
|     │   │   └─TableFullScan_89      | 246.00   | 96464.75     | 246     | cop[tikv]    | table:T1                              | tikv_task:{time:0s, loops:4}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              | keep order:false                                                                                                                                                                                                     | N/A      | N/A      |
|     │   └─TableReader_94(Probe)     | 225.00   | 8893.54      | 175     | root         |                                       | time:2.34ms, loops:2, cop_task: {num: 2, max: 1.81ms, min: 745µs, avg: 1.28ms, p95: 1.81ms, max_proc_keys: 224, p95_proc_keys: 224, tot_proc: 615.9µs, tot_wait: 734µs, copr_cache_hit_ratio: 0.00, build_task_duration: 1.16µs, max_distsql_concurrency: 1}, rpc_info:{Cop:{num_rpc:2, total_time:2.52ms}}                                                                                                                                                                                                                                                                                                                                                                                                                               | data:Selection_93                                                                                                                                                                                                    | 6.36 KB  | N/A      |
|     │     └─Selection_93            | 225.00   | 108740.15    | 175     | cop[tikv]    |                                       | tikv_task:{proc max:0s, min:0s, avg: 0s, p80:0s, p95:0s, iters:4, tasks:2}, scan_detail: {total_process_keys: 246, total_process_keys_size: 31116, total_keys: 248, get_snapshot_time: 573.5µs, rocksdb: {key_skipped_count: 246, block: {cache_hit_count: 25}}}, time_detail: {total_process_time: 615.9µs, total_wait_time: 734µs, tikv_wall_time: 1.84ms}                                                                                                                                                                                                                                                                                                                                                                              | not(isnull(MB.O_datadictionary.parentkey))                                                                                                                                                                | N/A      | N/A      |
|     │       └─TableFullScan_92      | 246.00   | 96464.75     | 246     | cop[tikv]    | table:T2                              | tikv_task:{proc max:0s, min:0s, avg: 0s, p80:0s, p95:0s, iters:4, tasks:2}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                | keep order:false                                                                                                                                                                                                     | N/A      | N/A      |
|     └─HashJoin_50(Probe)            | 12409.31 | 14481242.26  | 21694   | root         |                                       | time:4m27.7s, loops:31, build_hash_table:{total:4m27.7s, fetch:4m27.7s, build:2.24ms}, probe:{concurrency:5, total:22m18.4s, max:4m27.7s, probe:10.7ms, fetch and wait:22m18.3s}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          | left outer join, equal:[eq(MB.O_amazoncomments.sku, MB.O_skulist.number)]                                                                                                                      | 1.09 MB  | 0 Bytes  |
|       ├─Projection_51(Build)        | 12409.31 | 10638965.11  | 21694   | root         |                                       | time:4m27.7s, loops:24, Concurrency:5                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     | MB.O_amazoncomments.sku, upper(MB.O_amazoncomments.site)->Column#157                                                                                                                           | 333.9 KB | N/A      |
|       │ └─TableReader_59            | 12409.31 | 10513881.79  | 21694   | root         | partition:P_LT_2026-06-01 00:00:00    | time:4m27.7s, loops:24, cop_task: {num: 10, max: 0s, min: 0s, avg: 0s, p95: 0s, copr_cache_hit_ratio: 0.00}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               | MppVersion: 2, data:ExchangeSender_58                                                                                                                                                                                | 534.8 KB | N/A      |
|       │   └─ExchangeSender_58       | 12409.31 | 156701385.35 | 21694   | mpp[tiflash] |                                       | tiflash_task:{time:4m27.7s, loops:33, threads:48}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         | ExchangeType: PassThrough                                                                                                                                                                                            | N/A      | N/A      |
|       │     └─TableFullScan_56      | 12978.91 | 145205178.95 | 21694   | mpp[tiflash] | table:T1                              | tiflash_task:{time:4m27.7s, loops:33, threads:48}, tiflash_wait: {pipeline_queue_wait: 81494ms}, tiflash_scan:{mvcc_input_rows:41274, mvcc_input_bytes:1485864, mvcc_output_rows:32006, lm_skip_rows:0, local_regions:1, remote_regions:0, tot_learner_read:1ms, region_balance:{instance_num: 1, max/min: 1/1=1.000000}, delta_rows:12700, delta_bytes:8519561, segments:1, stale_read_regions:0, tot_build_snapshot:0ms, tot_build_bitmap:22ms, tot_build_inputstream:22ms, min_local_stream:267658ms, max_local_stream:267659ms, dtfile:{data_scanned_rows:30262, data_skipped_rows:0, mvcc_scanned_rows:28574, mvcc_skipped_rows:1688, lm_filter_scanned_rows:30262, lm_filter_skipped_rows:0, tot_rs_index_check:0ms, tot_read:2ms}} | pushed down filter:ge(MB.O_amazoncomments.commentdate, 2026-05-07 00:00:00.000000), lt(MB.O_amazoncomments.commentdate, 2026-05-20 00:00:00.000000), keep order:false, PartitionTableScan:true | N/A      | N/A      |
|       └─HashJoin_68(Probe)          | 17853.00 | 1918379.85   | 17853   | root         |                                       | time:6.69ms, loops:19, build_hash_table:{total:1.71ms, fetch:1.68ms, build:37.1µs}, probe:{concurrency:5, total:22m18.3s, max:4m27.7s, probe:3.51ms, fetch and wait:22m18.3s}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             | left outer join, equal:[eq(MB.O_skulist.thirdcategoryid, MB.O_category.categoryid)]                                                                                                            | 86.3 KB  | 0 Bytes  |
|         ├─TableReader_85(Build)     | 114.00   | 3236.62      | 114     | root         |                                       | time:1.67ms, loops:2, cop_task: {num: 1, max: 2.06ms, proc_keys: 114, tot_proc: 450.9µs, tot_wait: 842.9µs, copr_cache_hit_ratio: 0.00, build_task_duration: 693ns, max_distsql_concurrency: 1}, rpc_info:{Cop:{num_rpc:1, total_time:2.04ms}}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                            | data:TableFullScan_84                                                                                                                                                                                                | 1.18 KB  | N/A      |
|         │ └─TableFullScan_84        | 114.00   | 41326.19     | 114     | cop[tikv]    | table:T5                              | tikv_task:{time:0s, loops:3}, scan_detail: {total_process_keys: 114, total_process_keys_size: 11426, total_keys: 799, get_snapshot_time: 740µs, rocksdb: {key_skipped_count: 798, block: {cache_hit_count: 13}}}, time_detail: {total_process_time: 450.9µs, total_wait_time: 842.9µs, tikv_wall_time: 1.6ms}                                                                                                                                                                                                                                                                                                                                                                                                                             | keep order:false                                                                                                                                                                                                     | N/A      | N/A      |
|         └─HashJoin_70(Probe)        | 17853.00 | 1545558.36   | 17853   | root         |                                       | time:28.6ms, loops:19, build_hash_table:{total:1.61ms, fetch:1.57ms, build:37.7µs}, probe:{concurrency:5, total:22m18.3s, max:4m27.7s, probe:3.77ms, fetch and wait:22m18.3s}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             | left outer join, equal:[eq(MB.O_skulist.secondcategoryid, MB.O_category.categoryid)]                                                                                                           | 86.3 KB  | 0 Bytes  |
|           ├─TableReader_83(Build)   | 114.00   | 3236.62      | 114     | root         |                                       | time:1.56ms, loops:2, cop_task: {num: 1, max: 1.99ms, proc_keys: 114, tot_proc: 511.5µs, tot_wait: 731.7µs, copr_cache_hit_ratio: 0.00, build_task_duration: 1.03µs, max_distsql_concurrency: 1}, rpc_info:{Cop:{num_rpc:1, total_time:1.98ms}}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           | data:TableFullScan_82                                                                                                                                                                                                | 1.19 KB  | N/A      |
|           │ └─TableFullScan_82      | 114.00   | 41326.19     | 114     | cop[tikv]    | table:T4                              | tikv_task:{time:1ms, loops:3}, scan_detail: {total_process_keys: 114, total_process_keys_size: 11426, total_keys: 799, get_snapshot_time: 691µs, rocksdb: {key_skipped_count: 798, block: {cache_hit_count: 13}}}, time_detail: {total_process_time: 511.5µs, total_wait_time: 731.7µs, total_kv_read_wall_time: 1ms, tikv_wall_time: 1.48ms}                                                                                                                                                                                                                                                                                                                                                                                             | keep order:false                                                                                                                                                                                                     | N/A      | N/A      |
|           └─HashJoin_72(Probe)      | 17853.00 | 1172736.86   | 17853   | root         |                                       | time:40.8ms, loops:19, build_hash_table:{total:1.68ms, fetch:1.65ms, build:35.2µs}, probe:{concurrency:5, total:17m50.7s, max:4m27.7s, probe:5.08ms, fetch and wait:17m50.7s}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                             | left outer join, equal:[eq(MB.O_skulist.firstcategoryid, MB.O_category.categoryid)]                                                                                                            | 86.3 KB  | 0 Bytes  |
|             ├─TableReader_81(Build) | 114.00   | 3236.62      | 114     | root         |                                       | time:1.63ms, loops:2, cop_task: {num: 1, max: 2.15ms, proc_keys: 114, tot_proc: 694.7µs, tot_wait: 815.6µs, copr_cache_hit_ratio: 0.00, build_task_duration: 7.75µs, max_distsql_concurrency: 1}, rpc_info:{Cop:{num_rpc:1, total_time:2.13ms}}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           | data:TableFullScan_80                                                                                                                                                                                                | 1.19 KB  | N/A      |
|             │ └─TableFullScan_80    | 114.00   | 41326.19     | 114     | cop[tikv]    | table:T3                              | tikv_task:{time:1ms, loops:3}, scan_detail: {total_process_keys: 114, total_process_keys_size: 11426, total_keys: 799, get_snapshot_time: 710.9µs, rocksdb: {key_skipped_count: 798, block: {cache_hit_count: 13}}}, time_detail: {total_process_time: 694.7µs, total_wait_time: 815.6µs, total_kv_read_wall_time: 1ms, tikv_wall_time: 1.75ms}                                                                                                                                                                                                                                                                                                                                                                                           | keep order:false                                                                                                                                                                                                     | N/A      | N/A      |
|             └─TableReader_76(Probe) | 17853.00 | 799915.37    | 17853   | root         |                                       | time:40ms, loops:19, cop_task: {num: 7, max: 11.7ms, min: 601µs, avg: 5.61ms, p95: 11.7ms, max_proc_keys: 6013, p95_proc_keys: 6013, tot_proc: 31.9ms, tot_wait: 975.2µs, copr_cache_hit_ratio: 0.14, build_task_duration: 19.5µs, max_distsql_concurrency: 1}, rpc_info:{Cop:{num_rpc:7, total_time:39.1ms}}                                                                                                                                                                                                                                                                                                                                                                                                                             | data:Selection_75                                                                                                                                                                                                    | 406.1 KB | N/A      |
|               └─Selection_75        | 17853.00 | 7777360.07   | 17853   | cop[tikv]    |                                       | tikv_task:{proc max:10ms, min:1ms, avg: 4.29ms, p80:7ms, p95:10ms, iters:44, tasks:7}, scan_detail: {total_process_keys: 17373, total_process_keys_size: 4051699, total_keys: 69492, get_snapshot_time: 517.4µs, rocksdb: {key_skipped_count: 90503, block: {cache_hit_count: 665}}}, time_detail: {total_process_time: 31.9ms, total_suspend_time: 139.1µs, total_wait_time: 975.2µs, total_kv_read_wall_time: 27ms, tikv_wall_time: 34.1ms}                                                                                                                                                                                                                                                                                             | not(isnull(MB.O_skulist.number))                                                                                                                                                                          | N/A      | N/A      |
|                 └─TableFullScan_74  | 17853.00 | 6886495.37   | 17853   | cop[tikv]    | table:T6                              | tikv_task:{proc max:9ms, min:1ms, avg: 4.14ms, p80:7ms, p95:9ms, iters:44, tasks:7}                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                       | keep order:false                                                                                                                                                                                                     | N/A      | N/A      |

快的执行计划如下:

id                                 |estRows |actRows|task        |access object                        |execution info                                                                                                                                                                                                                                                 |operator info                                                                                                                                                                                                       |memory  |disk   |
-----------------------------------+--------+-------+------------+-------------------------------------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--------+-------+
HashAgg_43                         |1.00    |1      |root        |                                     |time:22.3ms, loops:2, RU:90.35, partial_worker:{wall_time:22.288882ms, concurrency:5, task_num:23, tot_wait:109.732792ms, tot_exec:986.034µs, tot_time:111.304532ms, max:22.267728ms, p95:22.267728ms}, final_worker:{wall_time:22.314409ms, concurrency:5, tas|funcs:count(1)->Column#156                                                                                                                                                                                          |320.2 KB|0 Bytes|
└─HashJoin_45                      |12372.01|21286  |root        |                                     |time:22.2ms, loops:24, build_hash_table:{total:9.27ms, fetch:8.86ms, build:407.3µs}, probe:{concurrency:5, total:109.9ms, max:22ms, probe:2.95ms, fetch and wait:106.9ms}                                                                                      |left outer join, equal:[eq(MB.O_amazoncomments.sku, MB.O_skulist.number)]                                                                                                                     |172.9 KB|0 Bytes|
  ├─HashAgg_95(Build)              |2284.62 |2143   |root        |                                     |time:8.84ms, loops:6, partial_worker:{wall_time:8.756654ms, concurrency:5, task_num:5, tot_wait:40.936083ms, tot_exec:1.29927ms, tot_time:43.176707ms, max:8.64087ms, p95:8.64087ms}, final_worker:{wall_time:8.87724ms, concurrency:5, task_num:25, tot_wait:6|group by:MB.O_cqelabel.cqelabel, MB.O_skulist.number, funcs:firstrow(MB.O_skulist.number)->MB.O_skulist.number                                                          |574.6 KB|0 Bytes|
  │ └─HashJoin_108                 |2284.62 |2143   |root        |                                     |time:8.28ms, loops:6, build_hash_table:{total:5.7ms, fetch:5.39ms, build:317.8µs}, probe:{concurrency:5, total:40.6ms, max:8.23ms, probe:2.57ms, fetch and wait:38ms}                                                                                          |inner join, equal:[eq(MB.O_cqelabel.skuid, MB.O_skulist.id)]                                                                                                                                  |259.4 KB|0 Bytes|
  │   ├─TableReader_111(Build)     |2284.62 |2143   |root        |                                     |time:5.5ms, loops:4, cop_task: {num: 4, max: 1.66ms, min: 1.21ms, avg: 1.4ms, p95: 1.66ms, max_proc_keys: 992, p95_proc_keys: 992, tot_proc: 2.28ms, tot_wait: 889.2µs, copr_cache_hit_ratio: 0.00, build_task_duration: 1.21µs, max_distsql_concurrency: 1}, r|data:Selection_110                                                                                                                                                                                                  |51.3 KB |N/A    |
  │   │ └─Selection_110            |2284.62 |2143   |cop[tikv]   |                                     |tikv_task:{proc max:1ms, min:0s, avg: 250µs, p80:1ms, p95:1ms, iters:17, tasks:4}, scan_detail: {total_process_keys: 2287, total_process_keys_size: 227552, total_keys: 2291, get_snapshot_time: 712.7µs, rocksdb: {key_skipped_count: 2287, block: {cache_hit_|eq(MB.O_cqelabel.forbiddenstatus, 0)                                                                                                                                                                     |N/A     |N/A    |
  │   │   └─TableFullScan_109      |2287.00 |2287   |cop[tikv]   |table:T1                             |tikv_task:{proc max:1ms, min:0s, avg: 250µs, p80:1ms, p95:1ms, iters:17, tasks:4}                                                                                                                                                                              |keep order:false                                                                                                                                                                                                    |N/A     |N/A    |
  │   └─IndexReader_113(Probe)     |17853.00|17853  |root        |                                     |time:5.11ms, loops:19, cop_task: {num: 7, max: 1.73ms, min: 459.4µs, avg: 791.2µs, p95: 1.73ms, max_proc_keys: 480, p95_proc_keys: 480, tot_proc: 1.02ms, tot_wait: 1.28ms, copr_cache_hit_ratio: 0.71, build_task_duration: 3.16µs, max_distsql_concurrency: 1|index:IndexFullScan_112                                                                                                                                                                                             |233.5 KB|N/A    |
  │     └─IndexFullScan_112        |17853.00|17853  |cop[tikv]   |table:T2, index:AK_Key_Number(Number)|tikv_task:{proc max:5ms, min:1ms, avg: 2.14ms, p80:4ms, p95:5ms, iters:44, tasks:7}, scan_detail: {total_process_keys: 704, total_process_keys_size: 42181, total_keys: 706, get_snapshot_time: 756.4µs, rocksdb: {delete_skipped_count: 2118, key_skipped_coun|keep order:false                                                                                                                                                                                                    |N/A     |N/A    |
  └─HashJoin_47(Probe)             |12372.01|21286  |root        |                                     |time:21.8ms, loops:24, build_hash_table:{total:2.77ms, fetch:2.76ms, build:14.9µs}, probe:{concurrency:5, total:109ms, max:21.8ms, probe:6.48ms, fetch and wait:102.5ms}                                                                                       |left outer join, equal:[eq(Column#157, Column#158)]                                                                                                                                                                 |121.4 KB|0 Bytes|
    ├─Projection_86(Build)         |123.00  |27     |root        |                                     |time:2.74ms, loops:3, Concurrency:OFF                                                                                                                                                                                                                          |upper(MB.O_datadictionary.fkey)->Column#158                                                                                                                                                              |16.2 KB |N/A    |
    │ └─HashJoin_88                |123.00  |27     |root        |                                     |time:2.71ms, loops:3, build_hash_table:{total:1.64ms, fetch:1.62ms, build:17.8µs}, probe:{concurrency:5, total:12.7ms, max:2.6ms, probe:48.8µs, fetch and wait:12.7ms}                                                                                         |left outer join, equal:[eq(MB.O_datadictionary.fkey, MB.O_datadictionary.parentkey)]                                                                                                          |108.6 KB|0 Bytes|
    │   ├─TableReader_91(Build)    |39.00   |26     |root        |                                     |time:1.61ms, loops:2, cop_task: {num: 1, max: 1.96ms, proc_keys: 246, tot_proc: 546.1µs, tot_wait: 709.9µs, copr_cache_hit_ratio: 0.00, build_task_duration: 1.82µs, max_distsql_concurrency: 1}, rpc_info:{Cop:{num_rpc:1, total_time:1.94ms}}                |data:Selection_90                                                                                                                                                                                                   |1.34 KB |N/A    |
    │   │ └─Selection_90           |39.00   |26     |cop[tikv]   |                                     |tikv_task:{time:1ms, loops:4}, scan_detail: {total_process_keys: 246, total_process_keys_size: 31116, total_keys: 247, get_snapshot_time: 588µs, rocksdb: {key_skipped_count: 246, block: {cache_hit_count: 14}}}, time_detail: {total_process_time: 546.1µs, t|eq(MB.O_datadictionary.parentkey, "Amazon")                                                                                                                                                              |N/A     |N/A    |
    │   │   └─TableFullScan_89     |246.00  |246    |cop[tikv]   |table:T1                             |tikv_task:{time:1ms, loops:4}                                                                                                                                                                                                                                  |keep order:false                                                                                                                                                                                                    |N/A     |N/A    |
    │   └─TableReader_94(Probe)    |225.00  |175    |root        |                                     |time:2.53ms, loops:2, cop_task: {num: 2, max: 1.99ms, min: 884.1µs, avg: 1.43ms, p95: 1.99ms, max_proc_keys: 224, p95_proc_keys: 224, tot_proc: 659.8µs, tot_wait: 775.9µs, copr_cache_hit_ratio: 0.00, build_task_duration: 798ns, max_distsql_concurrency: 1}|data:Selection_93                                                                                                                                                                                                   |6.36 KB |N/A    |
    │     └─Selection_93           |225.00  |175    |cop[tikv]   |                                     |tikv_task:{proc max:1ms, min:0s, avg: 500µs, p80:1ms, p95:1ms, iters:4, tasks:2}, scan_detail: {total_process_keys: 246, total_process_keys_size: 31116, total_keys: 248, get_snapshot_time: 611µs, rocksdb: {key_skipped_count: 246, block: {cache_hit_count: |not(isnull(MB.O_datadictionary.parentkey))                                                                                                                                                               |N/A     |N/A    |
    │       └─TableFullScan_92     |246.00  |246    |cop[tikv]   |table:T2                             |tikv_task:{proc max:1ms, min:0s, avg: 500µs, p80:1ms, p95:1ms, iters:4, tasks:2}                                                                                                                                                                               |keep order:false                                                                                                                                                                                                    |N/A     |N/A    |
    └─HashJoin_50(Probe)           |12372.01|21286  |root        |                                     |time:20.6ms, loops:30, build_hash_table:{total:17.7ms, fetch:15.4ms, build:2.3ms}, probe:{concurrency:5, total:106.4ms, max:21.5ms, probe:9.59ms, fetch and wait:96.8ms}                                                                                       |left outer join, equal:[eq(MB.O_amazoncomments.sku, MB.O_skulist.number)]                                                                                                                     |1.12 MB |0 Bytes|
      ├─Projection_51(Build)       |12372.01|21286  |root        |                                     |time:16.1ms, loops:24, Concurrency:5                                                                                                                                                                                                                           |MB.O_amazoncomments.sku, upper(MB.O_amazoncomments.site)->Column#157                                                                                                                          |377.5 KB|N/A    |
      │ └─TableReader_59           |12372.01|21286  |root        |partition:P_LT_2026-06-01 00:00:00   |time:15.8ms, loops:24, cop_task: {num: 20, max: 0s, min: 0s, avg: 0s, p95: 0s, copr_cache_hit_ratio: 0.00}                                                                                                                                                     |MppVersion: 2, data:ExchangeSender_58                                                                                                                                                                               |534.8 KB|N/A    |
      │   └─ExchangeSender_58      |12372.01|21286  |mpp[tiflash]|                                     |tiflash_task:{time:14ms, loops:29, threads:48}                                                                                                                                                                                                                 |ExchangeType: PassThrough                                                                                                                                                                                           |N/A     |N/A    |
      │     └─TableFullScan_56     |12940.00|21286  |mpp[tiflash]|table:T1                             |tiflash_task:{time:14ms, loops:29, threads:48}, tiflash_scan:{mvcc_input_rows:41274, mvcc_input_bytes:1485864, mvcc_output_rows:32006, lm_skip_rows:0, local_regions:1, remote_regions:0, tot_learner_read:1ms, region_balance:{instance_num: 1, max/min: 1/1=1|pushed down filter:ge(MB.O_amazoncomments.commentdate, 2026-05-07 00:00:00.000000), lt(MB.O_amazoncomments.commentdate, 2026-05-20 00:00:00.000000), keep order:false, PartitionTableScan:true|N/A     |N/A    |
      └─HashJoin_68(Probe)         |17853.00|17853  |root        |                                     |time:5.87ms, loops:19, build_hash_table:{total:1.19ms, fetch:1.14ms, build:47.6µs}, probe:{concurrency:5, total:102.3ms, max:20.9ms, probe:3.56ms, fetch and wait:98.8ms}                                                                                      |left outer join, equal:[eq(MB.O_skulist.thirdcategoryid, MB.O_category.categoryid)]                                                                                                           |86.3 KB |0 Bytes|
        ├─TableReader_85(Build)    |114.00  |114    |root        |                                     |time:1.13ms, loops:2, cop_task: {num: 1, max: 1.59ms, proc_keys: 114, tot_proc: 479.9µs, tot_wait: 498.1µs, copr_cache_hit_ratio: 0.00, build_task_duration: 747ns, max_distsql_concurrency: 1}, rpc_info:{Cop:{num_rpc:1, total_time:1.57ms}}                 |data:TableFullScan_84                                                                                                                                                                                               |1.18 KB |N/A    |
        │ └─TableFullScan_84       |114.00  |114    |cop[tikv]   |table:T5                             |tikv_task:{time:0s, loops:3}, scan_detail: {total_process_keys: 114, total_process_keys_size: 11426, total_keys: 799, get_snapshot_time: 433.9µs, rocksdb: {key_skipped_count: 798, block: {cache_hit_count: 14}}}, time_detail: {total_process_time: 479.9µs, |keep order:false                                                                                                                                                                                                    |N/A     |N/A    |
        └─HashJoin_70(Probe)       |17853.00|17853  |root        |                                     |time:17.6ms, loops:19, build_hash_table:{total:1.04ms, fetch:988.3µs, build:48.5µs}, probe:{concurrency:5, total:98.8ms, max:20ms, probe:4ms, fetch and wait:94.8ms}                                                                                           |left outer join, equal:[eq(MB.O_skulist.secondcategoryid, MB.O_category.categoryid)]                                                                                                          |86.3 KB |0 Bytes|
          ├─TableReader_83(Build)  |114.00  |114    |root        |                                     |time:976.4µs, loops:2, cop_task: {num: 1, max: 1.62ms, proc_keys: 114, tot_proc: 441.1µs, tot_wait: 548µs, copr_cache_hit_ratio: 0.00, build_task_duration: 878ns, max_distsql_concurrency: 1}, rpc_info:{Cop:{num_rpc:1, total_time:1.6ms}}                   |data:TableFullScan_82                                                                                                                                                                                               |1.18 KB |N/A    |
          │ └─TableFullScan_82     |114.00  |114    |cop[tikv]   |table:T4                             |tikv_task:{time:0s, loops:3}, scan_detail: {total_process_keys: 114, total_process_keys_size: 11426, total_keys: 799, get_snapshot_time: 452.7µs, rocksdb: {key_skipped_count: 798, block: {cache_hit_count: 14}}}, time_detail: {total_process_time: 441.1µs, |keep order:false                                                                                                                                                                                                    |N/A     |N/A    |
          └─HashJoin_72(Probe)     |17853.00|17853  |root        |                                     |time:18.6ms, loops:19, build_hash_table:{total:1.09ms, fetch:1.04ms, build:43.4µs}, probe:{concurrency:5, total:94.1ms, max:19.1ms, probe:5.08ms, fetch and wait:89ms}                                                                                         |left outer join, equal:[eq(MB.O_skulist.firstcategoryid, MB.O_category.categoryid)]                                                                                                           |84.3 KB |0 Bytes|
            ├─TableReader_81(Build)|114.00  |114    |root        |                                     |time:1.03ms, loops:2, cop_task: {num: 1, max: 1.82ms, proc_keys: 114, tot_proc: 650µs, tot_wait: 476.4µs, copr_cache_hit_ratio: 0.00, build_task_duration: 7.95µs, max_distsql_concurrency: 1}, rpc_info:{Cop:{num_rpc:1, total_time:1.79ms}}                  |data:TableFullScan_80                                                                                                                                                                                               |1.18 KB |N/A    |
            │ └─TableFullScan_80   |114.00  |114    |cop[tikv]   |table:T3                             |tikv_task:{time:0s, loops:3}, scan_detail: {total_process_keys: 114, total_process_keys_size: 11426, total_keys: 799, get_snapshot_time: 413.9µs, rocksdb: {key_skipped_count: 798, block: {cache_hit_count: 14}}}, time_detail: {total_process_time: 650µs, to|keep order:false                                                                                                                                                                                                    |N/A     |N/A    |
            └─TableReader_76(Probe)|17853.00|17853  |root        |                                     |time:17.1ms, loops:19, cop_task: {num: 7, max: 10.8ms, min: 466.4µs, avg: 2.35ms, p95: 10.8ms, max_proc_keys: 5088, p95_proc_keys: 5088, tot_proc: 10.9ms, tot_wait: 648.9µs, copr_cache_hit_ratio: 0.57, build_task_duration: 18.2µs, max_distsql_concurrency:|data:Selection_75                                                                                                                                                                                                   |406.0 KB|N/A    |
              └─Selection_75       |17853.00|17853  |cop[tikv]   |                                     |tikv_task:{proc max:11ms, min:1ms, avg: 5.14ms, p80:9ms, p95:11ms, iters:44, tasks:7}, scan_detail: {total_process_keys: 5792, total_process_keys_size: 1376911, total_keys: 23171, get_snapshot_time: 194.7µs, rocksdb: {key_skipped_count: 31108, block: {cac|not(isnull(MB.O_skulist.number))                                                                                                                                                                         |N/A     |N/A    |
                └─TableFullScan_74 |17853.00|17853  |cop[tikv]   |table:T6                             |tikv_task:{proc max:10ms, min:1ms, avg: 4.71ms, p80:8ms, p95:10ms, iters:44, tasks:7}                                                                                                                                                                          |keep order:false                                                                                                                                                                                                    |N/A     |N/A    |
1 个赞

新的分布式执行框架(DXF)导致的等待
关闭pipeline试试

1 个赞

客户端执行只需要算出子查询最终会返回多少行数字。应用端执行可能处理的数据比较大。组装 计数 传输的销售比较大

1 个赞

换版本试试呢

如果结果集数据不大,且每次应用端执行都比数据库端执行时间长很多,有理由怀疑升级后jdbc相关参数的问题。

1 个赞

客户端的语句在服务端还没有到达解析步骤之间就卡锁了吧

pipeline_queue_wait时间偏高
通过会话级变量关闭 pipeline 功能,直接验证是否为 DXF 框架导致的等待
set @@tidb_enable_pipeline = OFF;

升级后,统计信息重新搜集没

1 个赞

关闭pipeline这个是会增加延迟是吧?
要怎么样关闭啊?AI查了一下没找到方法

1 个赞

从执行计划看,应该是还没到应用端就慢了吧

1 个赞

已经是最新版本了

set @@tidb_enable_pipeline = OFF;
这个设置会报错:SQL 错误 [1193] [HY000]: Unknown system variable ‘tidb_enable_pipeline’
最新版的怎么设置才对?

更新过统计信息了,两个执行计划的预估行和实际返回行都差不多的

这个怎么样可以确定呢?

结果集大概有十多、二十万行数据吧,但最终是做count的,就返回一个值

1 个赞

是不是这个原因呢?
流水线 DML(bulk)会大量占用 TiDB 后台资源(gRPC、chunk、write 线程),导致应用端的【读请求】被排队、阻塞、等待资源,所以应用返回慢;
但数据库直连不走应用连接池,不受排队影响,所以查得快。

应该不是这个原因,看样子是卡在tiflash了,好像是tiflash处理不过来,但tiflash的CPU和内存、IO的使用率都不高。我现在把所有相关的表都不推tiflash了,应用端就没有再出现执行很慢的情况了。新版本是不是在tiflash这块做了什么变更?

tidb_dml_type这个参数没有额外配置,用的默认值是standard

查一下表的健康度,analyze收集一下统计信息看看