京公网安备11010502063487号热 key 是「一个 key 的访问量占了整个实例流量的一大块」。它不占内存,但会让单个分片、单个线程先到极限。怎么发现它?这篇构造一个 30% 流量落在同一个 key 上的负载,用 OBJECT FREQ 和 redis-cli --hotkeys 把它找出来,并看它在分片场景下的后果。
实际次数: {'hot:1': 149963, 'hot:2': 50128, 'k:35089': 14}50 万次 GET:30% 打 hot:1,10% 打 hot:2,其余均匀分到 10 万个普通 key 上。实例设成 allkeys-lfu(LFU 淘汰策略会记录每个 key 的访问频率)。
hot:1 OBJECT FREQ = 194 真实访问次数 149963 hot:2 OBJECT FREQ = 105 真实访问次数 50128 k:1 OBJECT FREQ = 6 真实访问次数 4 k:2 OBJECT FREQ = 6 真实访问次数 3 k:3 OBJECT FREQ = 6 真实访问次数 2
LFU 的计数器是对数的、概率性递增,范围 0–255:hot:1 是 194,hot:2 是 105,普通 key 约 6。访问次数相差数万倍,计数只差三十倍左右——所以它适合「找出明显更热的」,不适合精确计数。
# Scanning the entire keyspace to find hot keys as well as [00.00%] Hot key "k:85084" found so far with counter 7 [00.00%] Hot key "k:76747" found so far with counter 6 [00.00%] Hot key "k:69649" found so far with counter 6 [00.00%] Hot key "k:8888" found so far with counter 6 [00.00%] Hot key "k:11104" found so far with counter 6 [00.00%] Hot key "k:46990" found so far with counter 6 [00.00%] Hot key "k:52392" found so far with counter 6 [00.00%] Hot key "k:91808" found so far with counter 6 [00.00%] Hot key "k:39302" found so far with counter 6 [00.00%] Hot key "k:35212" found so far with counter 6 [00.01%] Hot key "k:1062" found so far with counter 6 [00.01%] Hot key "k:69411" found so far with counter 6 [00.01%] Hot key "k:37608" found so far with counter 6 [00.01%] Hot key "k:56283" found so far with counter 6 [00.01%] Hot key "k:50279" found so far with counter 6 [00.01%] Hot key "k:22400" found so far with counter 6 [00.02%] Hot key "k:89848" found so far with counter 7 [00.03%] Hot key "k:54358" found so far with counter 7 [00.03%] Hot key "k:44313" found so far with counter 7 [00.05%] Hot key "k:28557" found so far with counter 7 [00.06%] Hot key "k:29260" found so far with counter 7 [00.06%] Hot key "k:45305" found so far with counter 7 [00.07%] Hot key "k:10553" found so far with counter 7 [00.07%] Hot key "k:31380" found so far with counter 8 [00.09%] Hot key "k:59622" found so far with counter 7 [00.09%] Hot key "k:19348" found so far with counter 7 [00.10%] Hot key "k:83707" found so far with counter 7 [00.10%] Hot key "k:46997" found so far with counter 7 [00.10%] Hot key "k:37186" found so far with counter 7 [00.11%] Hot key "k:67322" found so far with counter 7 [00.11%] Hot key "k:18717" found so far with counter 8 [00.29%] Hot key "k:68490" found so far with counter 8 [00.33%] Hot key "k:3855" found so far with counter 8 [00.39%] Hot key "k:97013" found so far with counter 8 [00.46%] Hot key "k:69346" found so far with counter 8 [00.62%] Hot key "k:24973" found so far with counter 8 [00.92%] Hot key "k:50506" found so far with counter 8 [01.03%] Hot key "k:94465" found so far with counter 8 [01.23%] Hot key "k:11850" found so far with counter 8 [01.33%] Hot key "k:80544" found so far with counter 8 [01.39%] Hot key "k:15672" found so far with counter 8 [01.47%] Hot key "k:3467" found so far with counter 8 [01.63%] Hot key "k:14170" found so far with counter 8 [01.81%] Hot key "k:91248" found so far with counter 8 [02.09%] Hot key "k:91279" found so far with counter 8 [05.75%] Hot key "k:56849" found so far with counter 9 [05.81%] Hot key "k:90967" found so far with counter 9 [06.40%] Hot key "k:21469" found so far with counter 9 [07.68%] Hot key "k:11436" found so far with counter 9 [09.15%] Hot key "k:62086" found so far with counter 9 [09.71%] Hot key "hot:2" found so far with counter 105 [23.36%] Hot key "k:54416" found so far with counter 9 [26.57%] Hot key "k:97386" found so far with counter 9 [34.30%] Hot key "k:20394" found so far with counter 9 [39.11%] Hot key "k:12561" found so far with counter 9 [39.79%] Hot key "k:95985" found so far with counter 9 [39.93%] Hot key "k:31213" found so far with counter 9 [42.61%] Hot key "k:70558" found so far with counter 9 [43.85%] Hot key "k:21898" found so far with counter 9 [44.92%] Hot key "k:4079" found so far with counter 9 [49.75%] Hot key "k:62280" found so far with counter 9 [76.49%] Hot key "hot:1" found so far with counter 194 -------- summary ------- Sampled 100002 keys in the keyspace! hot key found with counter: 194 keyname: "hot:1" hot key found with counter: 105 keyname: "hot:2" hot key found with counter: 9 keyname: "k:56849" hot key found with counter: 9 keyname: "k:90967" hot key found with counter: 9 keyname: "k:21469" hot key found with counter: 9 keyname: "k:11436" hot key found with counter: 9 keyname: "k:62086" hot key found with counter: 9 keyname: "k:54416" hot key found with counter: 9 keyname: "k:97386" hot key found with counter: 9 keyname: "k:20394" hot key found with counter: 9 keyname: "k:12561" hot key found with counter: 9 keyname: "k:95985" hot key found with counter: 9 keyname: "k:31213" hot key found with counter: 9 keyname: "k:70558" hot key found with counter: 9 keyname: "k:21898" hot key found with counter: 9 keyname: "k:4079" (耗时 2.4 s)
扫全库读每个 key 的 OBJECT FREQ,按计数从高到低打印(默认最多 16 个)。第一名是 hot:1(194),第二是 hot:2(105),后面全是计数 9 的普通 key。约 2.4 s 扫完 10 万个 key(-i 0.01)。
['', 'Error: ERR An LFU maxmemory policy is not selected, access frequency not tracked. Please note that when switching between policies at runtime LRU and LFU data will take some time to adjust.'] OBJECT FREQ hot:1(policy 非 LFU 时)→ ERR An LFU maxmemory policy is not selected, access frequency not tracked. Please note that when switching between policies at runtime LRU and LFU data will take some time to adjust.
没有启用 LFU 策略时,--hotkeys 和 OBJECT FREQ 都返回 ERR An LFU maxmemory policy is not selected, access frequency not tracked;错误里还提醒:运行中切换策略后,LRU/LFU 数据需要一段时间才会调整。所以要用这个工具,通常要事先就用 LFU 策略,或者临时切过去等一会儿(临时切换对淘汰行为的影响本文没有测)。
各分片的请求数: {0: 100329, 1: 249930, 2: 149741} 热 key hot:1 所在分片 = 1 ,hot:2 所在分片 = 2把这 50 万次请求按 CRC16 分到 3 个分片(这是我用槽位公式算的,不是在真实集群上跑的):20% / 50% / 30%。hot:1 所在的分片承担了约一半的请求。增加分片数不会分摊一个 key 的流量;常见办法是在客户端做本地缓存,或把热 key 拆成多份(带后缀)分散读——这两条是通用做法,本文没有实测。