京公网安备11010502063487号镜像队列(ha-mode)曾是 RabbitMQ 高可用的标配,quorum 队列取代了它。到了 4.3.6 镜像到底还剩什么?杀掉队列所在的节点,两种队列分别怎么表现?三副本写入的吞吐代价有多大?这篇起了一个真实的三节点集群来验证。
Disk Nodes
n1@localhost
n2@localhost
n3@localhost
Running Nodes
n1@localhost
n2@localhost
n3@localhost
Versions
n1@localhost: RabbitMQ 4.3.6 on Erlang 28.5.0.7
n2@localhost: RabbitMQ 4.3.6 on Erlang 28.5.0.7
n3@localhost: RabbitMQ 4.3.6 on Erlang 28.5.0.7集群共享元数据(用户、vhost、exchange、队列定义、绑定),而经典队列的消息只存在它所在的节点。所以“节点挂了”对单副本的经典队列是实打实的不可用。
name type leader members online messages cq classic n1@localhost [n1@localhost] 1000 qq quorum n1@localhost [n1@localhost, n2@localhost, n3@localhost] [n1@localhost, n2@localhost, n3@localhost] 0
rabbitmqctl list_queues name type leader members online 能看到每个队列的类型、leader、成员和在线成员:经典队列只有一个成员,quorum 队列默认每个节点一个副本。
$ rabbitmqctl set_policy ha-all "^cq$" '{"ha-mode":"all","ha-sync-mode":"automatic"}' --apply-to queues
Error:
Validation failed
[{<<"ha-mode">>,<<"all">>},{<<"ha-sync-mode">>,<<"automatic">>}] are not recognised policy settings
list_policies: Listing policies for vhost "/" ...
策略生效后 cq 仍是: cq classic n1@localhost [n1@localhost]
classic_queue_mirroring 的状态: [('removed', 'denied')]
broker 日志里关于镜像的行: (无)在 4.3.6 里设置 ha-mode 策略直接被拒绝(not recognised policy settings),classic_queue_mirroring 的阶段是 removed。想要多副本就用 quorum 队列。
quorum 队列 qq: 杀掉 n1 后第一次成功发布(带 confirm)在 303 ms 之后 Listing queues for vhost / ... name type leader members online messages qq quorum n2@localhost [n2@localhost, n1@localhost, n3@localhost] [n2@localhost, n3@localhost] 1000 classic 队列 cq(它只在 n1 上,n1 已死): ChannelClosedByBroker: (404, "NOT_FOUND - failed to perform operation on queue 'cq' in vhost '/' due to timeout");这个调用等了 66.6 s 才失败 从 n2 把 qq 取空:共 1001 条(1000 条原有 + 1 条 kill 之后发的)
杀掉 leader 所在的节点后,quorum 队列在 303 ms 内选出新 leader 继续服务,数据没丢;经典队列要等客户端超时(66.6 s)才报 404。
name type leader members online messages cq classic n1@localhost [n1@localhost] 1000 qq quorum n2@localhost [n2@localhost, n1@localhost, n3@localhost] [n2@localhost, n1@localhost, n3@localhost] 0 从 cq 取一条: 'm0'(重启后 classic 队列的持久消息还在)
节点回来后经典队列的持久消息还在,quorum 队列的副本作为 follower 追上进度,leader 不会自动回到原来的节点。
bench.classic 371 ms 53882 条/秒 bench.quorum 372 ms 53750 条/秒
在本机回环上,三副本的 quorum 队列和单副本经典队列的发布吞吐几乎一样(53750 与 53882 条/秒)。真实网络、磁盘上的代价没有在这里测。