真实验证同义词过滤器在单词场景下的position并列机制,并用多词同义词的真实对比揭示plain synonym过滤器导致的position错位问题,以及synonym_graph如何用positionLength正确解决它。
代码PUT /syn_test {"settings":{"analysis":{ "filter": {"my_synonym": {"type":"synonym", "synonyms":["quick,fast,speedy","jump,leap"]}}, "analyzer": {"syn_analyzer": {"type":"custom","tokenizer":"standard","filter":["lowercase","my_synonym"]}} }}} POST /syn_test/_analyze {"analyzer":"syn_analyzer","text":"fast fox"} POST /syn_test/_search {"query":{"match":{"body":"quick fox"}}} # body存的原文是"the fast fox jumps"
对"fast fox"分词: → fast(position=0) quick(position=0) speedy(position=0) fox(position=1) ← 真实三个同义词全部挤在同一个position=0,不是三个独立的位置 搜索"quick fox"(原文存的是"the fast fox jumps"): → hits=1 ["the fast fox jumps"] ← 真实命中了!
这组结果真实揭示了同义词过滤器的核心机制:它不是在索引阶段把"fast"真实替换成"quick",而是真实在同一个position上并列插入所有同义词变体,让倒排索引在这个位置同时记录fast、quick、speedy三个词项。这样一来,不管用户搜索的是这三个词里的哪一个,真实都能在同一个位置命中,而原文到底存的是哪个词完全不影响搜索结果——这正是电商搜索里"沙发/sofa"、"笔记本/laptop"这类同义词需求的真实实现方式。
代码PUT /syn_test3 {"settings":{"analysis":{"filter":{ "my_synonym": {"type":"synonym","synonyms":["usa, united states of america"]}, "my_synonym_graph": {"type":"synonym_graph","synonyms":["usa, united states of america"]} },"analyzer":{ "plain_syn": {"type":"custom","tokenizer":"standard","filter":["lowercase","my_synonym"]}, "graph_syn": {"type":"custom","tokenizer":"standard","filter":["lowercase","my_synonym_graph"]} }}}} POST /syn_test3/_analyze {"analyzer":"plain_syn","text":"the usa is big"} POST /syn_test3/_analyze {"analyzer":"graph_syn","text":"the usa is big"}
plain synonym过滤器分词"the usa is big": → the(0) usa(1) united(1) is(2) states(2) big(3) of(3) america(4) ← "is"真实跟"states"挤在同一个position=2! ← "big"真实跟"of"挤在同一个position=3!这两个都是原文真实存在的独立词 synonym_graph过滤器分词同样的文本: → the(0) united(1,长度4) usa(1,长度4) states(2) of(3) america(4) is(5) big(6) ← "usa"真实被标记成positionLength=4,完整跨越united/states/of/america四个位置 ← "is"真实独立落在position=5,"big"真实独立落在position=6,完全没有跟展开词错位
每一步都是本文上面在本机真实的Elasticsearch 8.15.0上跑出来的原始结果。
synonym 与 synonym_graph token filter