v7.5.7版本集群查询语句包含or id is not null等条件走全表扫描

【TiDB 使用环境】生产环境
【TiDB 版本】v7.5.7
【遇到的问题:问题现象及影响】
研发存在抽数业务,需要先获取id的最大值和最小值。由于抽数平台限制,默认所有的SQL都会自带id is not null条件。
表中存在相关索引 KEY idx_created_at (created_at), KEY idx_modified_at (modified_at)
如果把id is not null去掉,可以用到created_at和modified_at索引。
如果把or去掉,modified_at 或created_at去掉一个,也可以走正常的索引。
对于现有SQL,如何调整可以命中时间索引,而不是走全表扫描。


| id                       | estRows      | estCost         | actRows   | task      | access object                | execution info                                                                                                                                                                                                                                                                                                                                                                                                 | operator info                                                                                                                                                                                                                                                                                      | memory    | disk  |
| HashAgg_13               | 1.00         | 23965603503.70  | 1         | root      |                              | time:2m47.2s, loops:2, partial_worker:{wall_time:2m47.192706716s, concurrency:5, task_num:1, tot_wait:13m55.963418873s, tot_exec:3.839µs, tot_time:13m55.963427182s, max:2m47.19268925s, p95:2m47.19268925s}, final_worker:{wall_time:2m47.192733223s, concurrency:5, task_num:1, tot_wait:13m55.963523287s, tot_exec:16.025µs, tot_time:13m55.963541185s, max:2m47.192715165s, p95:2m47.192715165s}           | funcs:min(Column#20)->Column#18, funcs:max(Column#21)->Column#19                                                                                                                                                                                                                                   | 13.6 KB   | N/A   |
| └─TableReader_14         | 1.00         | 23965601966.14  | 9         | root      |                              | time:2m47.2s, loops:2, cop_task: {num: 2683, max: 0s, min: 0s, avg: 931.9ms, p95: 1.52s, tot_proc: 24m43.6s, tot_wait: 16.6s, copr_cache_hit_ratio: 0.00, build_task_duration: 25.4ms, max_distsql_concurrency: 15}, rpc_info:{Cop:{num_rpc:2683, total_time:41m40.2s}}                                                                                                                                        | data:HashAgg_6                                                                                                                                                                                                                                                                                     | 475 Bytes | N/A   |
|   └─HashAgg_6            | 1.00         | 359484029428.67 | 9         | cop[tikv] |                              | tikv_task:{proc max:1.75s, min:64ms, avg: 920.6ms, p80:1.34s, p95:1.5s, iters:846873, tasks:2683}, scan_detail: {total_process_keys: 865834068, total_process_keys_size: 210525438840, total_keys: 865836826, get_snapshot_time: 15.5s, rocksdb: {delete_skipped_count: 30161072, key_skipped_count: 982776463, block: {cache_hit_count: 36361, read_count: 6941199, read_byte: 19.6 GB, read_time: 1m59.8s}}} | funcs:min(dw_merchant_log.merchant_operation_log.id)->Column#20, funcs:max(dw_merchant_log.merchant_operation_log.id)->Column#21                                                                                                                                                                   | N/A       | N/A   |
|     └─Selection_12       | 690816538.40 | 338800980761.35 | 2897200   | cop[tikv] |                              | tikv_task:{proc max:1.75s, min:64ms, avg: 919.7ms, p80:1.34s, p95:1.5s, iters:846873, tasks:2683}                                                                                                                                                                                                                                                                                                              | or(and(ge(dw_merchant_log.merchant_operation_log.created_at, 2025-12-07), le(dw_merchant_log.merchant_operation_log.created_at, 2025-12-11)), and(ge(dw_merchant_log.merchant_operation_log.modified_at, 2025-12-07), and(le(dw_merchant_log.merchant_operation_log.modified_at, 2025-12-11), 1))) | N/A       | N/A   |
|       └─TableFullScan_11 | 863520673.00 | 295711299178.65 | 865834068 | cop[tikv] | table:merchant_operation_log | tikv_task:{proc max:1.66s, min:54ms, avg: 847ms, p80:1.26s, p95:1.41s, iters:846873, tasks:2683}                                                                                                                                                                                                                                                                                                               | keep order:false                                                                                                                                                                                                                                                                                   | N/A       | N/A   |

上传个 plan replyer 看看呢。https://docs.pingcap.com/zh/tidb/stable/sql-plan-replayer/

看一下表结构

https://docs.pingcap.com/zh/tidb/stable/optimizer-hints/#use_index_merget1_name-idx1_name--idx2_name- 用这个 hint 试试

plan_replayer_12111524.zip (193.0 KB)

plan replyer 需要安装吗? 还是直接可以用

可以贴一下建表的sql?

把or改成union 试试呢,

站个位置等分析结果,最近也在头疼表扫描偏高的问题
现在能采取的手段要么是查询改写,要么是查询绑定,都是事后的手工干预,后者还可能有后遗症

用hint吧

要解决 SQL 因 id is not null 强制拼接、OR 条件导致无法命中时间索引的问题,核心思路是拆解 OR 条件 + 规避 id 条件对索引的干扰 ,同时利用现有 created_at/modified_at 索引

测试结果:

explain format="verbose"
 SELECT
   MIN(id),
   MAX(id)
 FROM
   `merchant_operation_log`
 WHERE
   (
     (
       created_at BETWEEN date_sub(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 3 DAY
       )
       AND date_add(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 1 DAY
       )
     )
     OR (
       modified_at BETWEEN date_sub(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 3 DAY
       )
       AND date_add(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 1 DAY
       )
     )
     AND id IS NOT NULL
   );
+------------------------------+--------------+-----------------+-----------+------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id                           | estRows      | estCost         | task      | access object                | operator info                                                                                                                                                                                                                                                                                      |
+------------------------------+--------------+-----------------+-----------+------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| HashAgg_13                   | 1.00         | 24029006145.31  | root      |                              | funcs:min(Column#20)->Column#18, funcs:max(Column#21)->Column#19                                                                                                                                                                                                                                   |
| └─TableReader_14             | 1.00         | 24029004607.75  | root      |                              | data:HashAgg_6                                                                                                                                                                                                                                                                                     |
|   └─HashAgg_6                | 1.00         | 360435069052.86 | cop[tikv] |                              | funcs:min(dw_merchant_log.merchant_operation_log.id)->Column#20, funcs:max(dw_merchant_log.merchant_operation_log.id)->Column#21                                                                                                                                                                   |
|     └─Selection_12           | 692744456.80 | 339694298508.65 | cop[tikv] |                              | or(and(ge(dw_merchant_log.merchant_operation_log.created_at, 2025-12-07), le(dw_merchant_log.merchant_operation_log.created_at, 2025-12-11)), and(ge(dw_merchant_log.merchant_operation_log.modified_at, 2025-12-07), and(le(dw_merchant_log.merchant_operation_log.modified_at, 2025-12-11), 1))) |
|       └─TableFullScan_11     | 865930571.00 | 296484363015.75 | cop[tikv] | table:merchant_operation_log | keep order:false                                                                                                                                                                                                                                                                                   |
+------------------------------+--------------+-----------------+-----------+------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
5 rows in set, 5 warnings (0.01 sec)

(root@127.0.0.1) [(none)]>show warnings;
+-------+------+----------------------------------------------------------------------------------------------------------------------------------------------------+
| Level | Code | Message                                                                                                                                            |
+-------+------+----------------------------------------------------------------------------------------------------------------------------------------------------+
| Note  | 1105 | [merchant_operation_log,idx_created_at] remain after pruning paths for merchant_operation_log given Prop{SortItems: [], TaskTp: copSingleReadTask} |
| Note  | 1105 | [merchant_operation_log,idx_created_at] remain after pruning paths for merchant_operation_log given Prop{SortItems: [], TaskTp: copMultiReadTask}  |
| Note  | 1105 | [merchant_operation_log,idx_created_at] remain after pruning paths for merchant_operation_log given Prop{SortItems: [], TaskTp: rootTask}          |
| Note  | 1105 | [merchant_operation_log,idx_created_at] remain after pruning paths for merchant_operation_log given Prop{SortItems: [], TaskTp: copSingleReadTask} |
| Note  | 1105 | [merchant_operation_log,idx_created_at] remain after pruning paths for merchant_operation_log given Prop{SortItems: [], TaskTp: rootTask}          |
+-------+------+----------------------------------------------------------------------------------------------------------------------------------------------------+
5 rows in set (0.00 sec)


explain format="verbose"
 SELECT /*+ USE_INDEX_MERGE(merchant_operation_log,idx_create_time,idx_modify_time) */
   MIN(id),
   MAX(id)
 FROM
   `merchant_operation_log`
 WHERE
   (
     (
       created_at BETWEEN date_sub(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 3 DAY
       )
       AND date_add(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 1 DAY
       )
     )
     OR (
       modified_at BETWEEN date_sub(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 3 DAY
       )
       AND date_add(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 1 DAY
       )
     )
     AND id IS NOT NULL
   );
+------------------------------+--------------+-----------------+-----------+------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id                           | estRows      | estCost         | task      | access object                | operator info                                                                                                                                                                                                                                                                                      |
+------------------------------+--------------+-----------------+-----------+------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| HashAgg_13                   | 1.00         | 24029006145.31  | root      |                              | funcs:min(Column#20)->Column#18, funcs:max(Column#21)->Column#19                                                                                                                                                                                                                                   |
| └─TableReader_14             | 1.00         | 24029004607.75  | root      |                              | data:HashAgg_6                                                                                                                                                                                                                                                                                     |
|   └─HashAgg_6                | 1.00         | 360435069052.86 | cop[tikv] |                              | funcs:min(dw_merchant_log.merchant_operation_log.id)->Column#20, funcs:max(dw_merchant_log.merchant_operation_log.id)->Column#21                                                                                                                                                                   |
|     └─Selection_12           | 692744456.80 | 339694298508.65 | cop[tikv] |                              | or(and(ge(dw_merchant_log.merchant_operation_log.created_at, 2025-12-07), le(dw_merchant_log.merchant_operation_log.created_at, 2025-12-11)), and(ge(dw_merchant_log.merchant_operation_log.modified_at, 2025-12-07), and(le(dw_merchant_log.merchant_operation_log.modified_at, 2025-12-11), 1))) |
|       └─TableFullScan_11     | 865930571.00 | 296484363015.75 | cop[tikv] | table:merchant_operation_log | keep order:false                                                                                                                                                                                                                                                                                   |
+------------------------------+--------------+-----------------+-----------+------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
5 rows in set, 6 warnings (0.00 sec)

show warnings;
+---------+------+----------------------------------------------------------------------------------------------------------------------------------------------------+
| Level   | Code | Message                                                                                                                                            |
+---------+------+----------------------------------------------------------------------------------------------------------------------------------------------------+
| Warning | 1105 | IndexMerge is inapplicable                                                                                                                         |
| Note    | 1105 | [merchant_operation_log,idx_created_at] remain after pruning paths for merchant_operation_log given Prop{SortItems: [], TaskTp: copSingleReadTask} |
| Note    | 1105 | [merchant_operation_log,idx_created_at] remain after pruning paths for merchant_operation_log given Prop{SortItems: [], TaskTp: copMultiReadTask}  |
| Note    | 1105 | [merchant_operation_log,idx_created_at] remain after pruning paths for merchant_operation_log given Prop{SortItems: [], TaskTp: rootTask}          |
| Note    | 1105 | [merchant_operation_log,idx_created_at] remain after pruning paths for merchant_operation_log given Prop{SortItems: [], TaskTp: copSingleReadTask} |
| Note    | 1105 | [merchant_operation_log,idx_created_at] remain after pruning paths for merchant_operation_log given Prop{SortItems: [], TaskTp: rootTask}          |
+---------+------+----------------------------------------------------------------------------------------------------------------------------------------------------+
6 rows in set (0.00 sec)


explain format="verbose"
 SELECT
   MIN(id),
   MAX(id)
 FROM
   `merchant_operation_log`
 WHERE
   (
     (
       created_at BETWEEN date_sub(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 3 DAY
       )
       AND date_add(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 1 DAY
       )
     )
     OR (
       modified_at BETWEEN date_sub(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 3 DAY
       )
       AND date_add(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 1 DAY
       )
     )
     -- AND id IS NOT NULL
   );

+----------------------------------+------------+---------------+-----------+------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------+
| id                               | estRows    | estCost       | task      | access object                                                    | operator info                                                                                                                    |
+----------------------------------+------------+---------------+-----------+------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------+
| HashAgg_7                        | 1.00       | 438186542.08  | root      |                                                                  | funcs:min(dw_merchant_log.merchant_operation_log.id)->Column#18, funcs:max(dw_merchant_log.merchant_operation_log.id)->Column#19 |
| └─IndexMerge_14                  | 5008717.95 | 288224018.98  | root      |                                                                  | type: union                                                                                                                      |
|   ├─IndexRangeScan_11(Build)     | 2507990.92 | 510376151.72  | cop[tikv] | table:merchant_operation_log, index:idx_created_at(created_at)   | range:[2025-12-07 00:00:00,2025-12-11 00:00:00], keep order:false                                                                |
|   ├─IndexRangeScan_12(Build)     | 2507990.92 | 510376151.72  | cop[tikv] | table:merchant_operation_log, index:idx_modified_at(modified_at) | range:[2025-12-07 00:00:00,2025-12-11 00:00:00], keep order:false                                                                |
|   └─TableRowIDScan_13(Probe)     | 5008717.95 | 1714925654.83 | cop[tikv] | table:merchant_operation_log                                     | keep order:false                                                                                                                 |
+----------------------------------+------------+---------------+-----------+------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------+
5 rows in set (0.01 sec)

explain format="verbose"
 SELECT
   MIN(id),
   MAX(id)
 FROM
   `merchant_operation_log`
 WHERE
   (
     (
       created_at BETWEEN date_sub(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 3 DAY
       )
       AND date_add(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 1 DAY
       )
     )
     OR (
       modified_at BETWEEN date_sub(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 3 DAY
       )
       AND date_add(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 1 DAY
       )
     )    
   )AND id IS NOT NULL;
+----------------------------------+------------+---------------+-----------+------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------+
| id                               | estRows    | estCost       | task      | access object                                                    | operator info                                                                                                                    |
+----------------------------------+------------+---------------+-----------+------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------+
| HashAgg_7                        | 1.00       | 438186542.08  | root      |                                                                  | funcs:min(dw_merchant_log.merchant_operation_log.id)->Column#18, funcs:max(dw_merchant_log.merchant_operation_log.id)->Column#19 |
| └─IndexMerge_14                  | 5008717.95 | 288224018.98  | root      |                                                                  | type: union                                                                                                                      |
|   ├─IndexRangeScan_11(Build)     | 2507990.92 | 510376151.72  | cop[tikv] | table:merchant_operation_log, index:idx_created_at(created_at)   | range:[2025-12-07 00:00:00,2025-12-11 00:00:00], keep order:false                                                                |
|   ├─IndexRangeScan_12(Build)     | 2507990.92 | 510376151.72  | cop[tikv] | table:merchant_operation_log, index:idx_modified_at(modified_at) | range:[2025-12-07 00:00:00,2025-12-11 00:00:00], keep order:false                                                                |
|   └─TableRowIDScan_13(Probe)     | 5008717.95 | 1714925654.83 | cop[tikv] | table:merchant_operation_log                                     | keep order:false                                                                                                                 |
+----------------------------------+------------+---------------+-----------+------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------+
5 rows in set (0.01 sec)

可以看到 hint 的时候:

Warning | 1105 | IndexMerge is inapplicable

优化器认为这个查询不适用 Index Merge

-- 括号内的 AND id IS NOT NULL
or(
  and(ge(...created_at...), le(...created_at...)),
  and(ge(...modified_at...), and(le(...modified_at...), 1))  -- 注意这里多了 and(..., 1)
)

这里的 and(..., 1) 实际上是 id IS NOT NULL 的谓词表达式(因为 id IS NOT NULL 总是为真,被转换为常量 1)。

Index Merge 不生效的根本原因是

原因 说明
运算符优先级 括号内的 AND id IS NOT NULL 只绑定到 modified_at 条件
非对称的 OR 分支 左分支和右分支的过滤逻辑不一致
破坏 Index Merge 模型 Index Merge 要求各分支独立且对称,不能有分支特定的额外过滤
优化器判断不适用 Warning “IndexMerge is inapplicable” 明确说明无法使用

对比 MySQL:

explain format='json'
 SELECT
   MIN(id),
   MAX(id)
 FROM
   `merchant_operation_log`
 WHERE
   (
     (
       created_at BETWEEN date_sub(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 3 DAY
       )
       AND date_add(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 1 DAY
       )
     )
     OR (
       modified_at BETWEEN date_sub(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 3 DAY
       )
       AND date_add(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 1 DAY
       )
     )
     AND id IS NOT NULL
   );
 
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| EXPLAIN                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| {
  "query_block": {
    "select_id": 1,
    "cost_info": {
      "query_cost": "0.35"
    },
    "table": {
      "table_name": "merchant_operation_log",
      "access_type": "ALL",
      "possible_keys": [
        "idx_created_at",
        "idx_modified_at"
      ],
      "rows_examined_per_scan": 1,
      "rows_produced_per_join": 1,
      "filtered": "100.00",
      "cost_info": {
        "read_cost": "0.25",
        "eval_cost": "0.10",
        "prefix_cost": "0.35",
        "data_read_per_join": "1K"
      },
      "used_columns": [
        "id",
        "created_at",
        "modified_at"
      ],
      "attached_condition": "((`dw_merchant_log`.`merchant_operation_log`.`created_at` between <cache>((str_to_date('20251210','%Y%m%d') - interval 3 day)) and <cache>((str_to_date('20251210','%Y%m%d') + interval 1 day))) or (`dw_merchant_log`.`merchant_operation_log`.`modified_at` between <cache>((str_to_date('20251210','%Y%m%d') - interval 3 day)) and <cache>((str_to_date('20251210','%Y%m%d') + interval 1 day))))"
    }
  }
} |
+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set, 1 warning (0.00 sec)

show warnings;
+-------+------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Level | Code | Message                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               |
+-------+------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Note  | 1003 | /* select#1 */ select min(`dw_merchant_log`.`merchant_operation_log`.`id`) AS `MIN(id)`,max(`dw_merchant_log`.`merchant_operation_log`.`id`) AS `MAX(id)` from `dw_merchant_log`.`merchant_operation_log` where ((`dw_merchant_log`.`merchant_operation_log`.`created_at` between <cache>((str_to_date('20251210','%Y%m%d') - interval 3 day)) and <cache>((str_to_date('20251210','%Y%m%d') + interval 1 day))) or (`dw_merchant_log`.`merchant_operation_log`.`modified_at` between <cache>((str_to_date('20251210','%Y%m%d') - interval 3 day)) and <cache>((str_to_date('20251210','%Y%m%d') + interval 1 day)))) |
+-------+------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

attached_condition 内容里面可以看到,id is not null 已经消除了(id 是主键,这个条件为 true)。

测试了下,tidb 即使修改条件对称也不行:

explain format="verbose"
 SELECT /*+ USE_INDEX_MERGE(merchant_operation_log,idx_create_time,idx_modify_time) */
   MIN(id),
   MAX(id)
 FROM
   `merchant_operation_log`
 WHERE
   (
     (
       created_at BETWEEN date_sub(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 3 DAY
       )
       AND date_add(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 1 DAY
       )
     ) AND id IS NOT NULL
     OR (
       modified_at BETWEEN date_sub(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 3 DAY
       )
       AND date_add(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 1 DAY
       )
     )
     AND id IS NOT NULL
   );

+------------------------------+--------------+-----------------+-----------+------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id                           | estRows      | estCost         | task      | access object                | operator info                                                                                                                                                                                                                                                                                              |
+------------------------------+--------------+-----------------+-----------+------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| HashAgg_13                   | 1.00         | 24033576299.40  | root      |                              | funcs:min(Column#20)->Column#18, funcs:max(Column#21)->Column#19                                                                                                                                                                                                                                           |
| └─TableReader_14             | 1.00         | 24033574761.84  | root      |                              | data:HashAgg_6                                                                                                                                                                                                                                                                                             |
|   └─HashAgg_6                | 1.00         | 360503621364.20 | cop[tikv] |                              | funcs:min(dw_merchant_log.merchant_operation_log.id)->Column#20, funcs:max(dw_merchant_log.merchant_operation_log.id)->Column#21                                                                                                                                                                           |
|     └─Selection_12           | 692884818.40 | 339758648393.68 | cop[tikv] |                              | or(and(ge(dw_merchant_log.merchant_operation_log.created_at, 2025-12-07), and(le(dw_merchant_log.merchant_operation_log.created_at, 2025-12-11), 1)), and(ge(dw_merchant_log.merchant_operation_log.modified_at, 2025-12-07), and(le(dw_merchant_log.merchant_operation_log.modified_at, 2025-12-11), 1))) |
|       └─TableFullScan_11     | 866106023.00 | 296539957845.98 | cop[tikv] | table:merchant_operation_log | keep order:false                                                                                                                                                                                                                                                                                           |
+------------------------------+--------------+-----------------+-----------+------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
5 rows in set, 6 warnings (0.00 sec)

+---------+------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Level   | Code | Message                                                                                                                                                            |
+---------+------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Warning | 1105 | IndexMerge is inapplicable                                                                                                                                         |
| Note    | 1105 | [merchant_operation_log,idx_created_at,idx_modified_at] remain after pruning paths for merchant_operation_log given Prop{SortItems: [], TaskTp: copSingleReadTask} |
| Note    | 1105 | [merchant_operation_log,idx_created_at,idx_modified_at] remain after pruning paths for merchant_operation_log given Prop{SortItems: [], TaskTp: copMultiReadTask}  |
| Note    | 1105 | [merchant_operation_log,idx_created_at,idx_modified_at] remain after pruning paths for merchant_operation_log given Prop{SortItems: [], TaskTp: rootTask}          |
| Note    | 1105 | [merchant_operation_log,idx_created_at,idx_modified_at] remain after pruning paths for merchant_operation_log given Prop{SortItems: [], TaskTp: copSingleReadTask} |
| Note    | 1105 | [merchant_operation_log,idx_created_at,idx_modified_at] remain after pruning paths for merchant_operation_log given Prop{SortItems: [], TaskTp: rootTask}          |
+---------+------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------+
6 rows in set (0.00 sec)

想到一个 workround:

"where": "((created_at BETWEEN '2025-12-07' AND '2025-12-11') OR (modified_at BETWEEN '2025-12-07' AND '2025-12-11'))",

如果写的配置文件,还可以将条件多加个小括号解决。

explain format="verbose"
 SELECT
   MIN(id),
   MAX(id)
 FROM
   `merchant_operation_log`
 WHERE
   ((
     (
       created_at BETWEEN date_sub(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 3 DAY
       )
       AND date_add(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 1 DAY
       )
     )
     OR (
       modified_at BETWEEN date_sub(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 3 DAY
       )
       AND date_add(
         str_to_date('20251210', '%Y%m%d'),
         INTERVAL 1 DAY
       )
     ))
     AND id IS NOT NULL
   );

+----------------------------------+------------+---------------+-----------+------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------+
| id                               | estRows    | estCost       | task      | access object                                                    | operator info                                                                                                                    |
+----------------------------------+------------+---------------+-----------+------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------+
| HashAgg_7                        | 1.00       | 438273598.97  | root      |                                                                  | funcs:min(dw_merchant_log.merchant_operation_log.id)->Column#18, funcs:max(dw_merchant_log.merchant_operation_log.id)->Column#19 |
| └─IndexMerge_18                  | 5009732.80 | 288280691.26  | root      |                                                                  | type: union                                                                                                                      |
|   ├─IndexRangeScan_15(Build)     | 2508499.08 | 510479562.45  | cop[tikv] | table:merchant_operation_log, index:idx_created_at(created_at)   | range:[2025-12-07 00:00:00,2025-12-11 00:00:00], keep order:false                                                                |
|   ├─IndexRangeScan_16(Build)     | 2508499.08 | 510479562.45  | cop[tikv] | table:merchant_operation_log, index:idx_modified_at(modified_at) | range:[2025-12-07 00:00:00,2025-12-11 00:00:00], keep order:false                                                                |
|   └─TableRowIDScan_17(Probe)     | 5009732.80 | 1715247226.60 | cop[tikv] | table:merchant_operation_log                                     | keep order:false                                                                                                                 |
+----------------------------------+------------+---------------+-----------+------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------+
5 rows in set, 5 warnings (0.00 sec)

1 个赞

update 下 tidb index merge 失效原因:
对 tidb 来说原 query where 条件是 A or B and C 这种结构。IndexMerge 要么只能做 And, 要么只能做 OR, 是不能同时做 and 和 or 的。所以才导致了这个问题。
和对不对称关系不大

1 个赞

给 SQL 条件加外层括号统一绑定 id 条件,让 IndexMerge 生效,避开全表扫描

1 个赞

可以试试外层加括号

2 个赞

1、尝试给条件加外层括号统一绑定 id
2、 IndexMerge 生效一下,避开全表扫描

or col is not null 这种条件实在是少见,这种情况很耗费资源

1 个赞