跳到主要内容
版本:2.0.7

发布和消费分区 topic

Doubao 中英对照 Publishing and Consuming Partitioned Topics

在以下示例中,我们发布到一个名为 hello-pulsar-partitioned 的 topic。它是一个分区 topic,对于本示例,我们假设该 topic 已经以三个分区创建。

@SpringBootApplication
public class PulsarBootPartitioned {

public static void main(String[] args) {
SpringApplication.run(PulsarBootPartitioned.class, "--spring.pulsar.producer.message-routing-mode=CustomPartition");
}

@Bean
public ApplicationRunner runner(PulsarTemplate<String> pulsarTemplate) {
pulsarTemplate.setDefaultTopicName("hello-pulsar-partitioned");
return args -> {
for (int i = 0; i < 10; i++) {
pulsarTemplate.sendAsync("hello john doe 0 ", new FooRouter());
pulsarTemplate.sendAsync("hello alice doe 1", new BarRouter());
pulsarTemplate.sendAsync("hello buzz doe 2", new BuzzRouter());
}
};
}

@PulsarListener(subscriptionName = "hello-pulsar-partitioned-subscription", topics = "hello-pulsar-partitioned")
public void listen(String message) {
System.out.println("Message Received: " + message);
}

static class FooRouter implements MessageRouter {

@Override
public int choosePartition(Message<?> msg, TopicMetadata metadata) {
return 0;
}
}

static class BarRouter implements MessageRouter {

@Override
public int choosePartition(Message<?> msg, TopicMetadata metadata) {
return 1;
}
}

static class BuzzRouter implements MessageRouter {

@Override
public int choosePartition(Message<?> msg, TopicMetadata metadata) {
return 2;
}
}

}

在前述示例中,我们发布到一个分区 topic,并希望把某段数据发布到特定分区。如果交给 Pulsar 默认方式,它会采用轮询的分区分配模式,而我们希望覆盖这一行为。为此,我们通过 send 方法提供一个消息路由器对象。考虑所实现的三个消息路由器:FooRouter 总是把数据发送到分区 0,BarRouter 发送到分区 1,BuzzRouter 发送到分区 2。还要注意,我们现在使用 PulsarTemplate 的 sendAsync 方法,它返回一个 CompletableFuture。运行应用程序时,我们还需要在生产者上将 messageRoutingMode 设置为 CustomPartition(spring.pulsar.producer.message-routing-mode)。

在消费者端,我们使用带独占订阅类型的 PulsarListener。这意味着来自所有分区的数据最终都进入同一个消费者,并且不保证顺序。

如果我们希望每个分区由一个独立的消费者消费,该怎么做?我们可以切换到 failover 订阅模式,并添加三个独立的消费者:

@PulsarListener(subscriptionName = "hello-pulsar-partitioned-subscription", topics = "hello-pulsar-partitioned", subscriptionType = SubscriptionType.Failover)
public void listen1(String foo) {
System.out.println("Message Received 1: " + foo);
}

@PulsarListener(subscriptionName = "hello-pulsar-partitioned-subscription", topics = "hello-pulsar-partitioned", subscriptionType = SubscriptionType.Failover)
public void listen2(String foo) {
System.out.println("Message Received 2: " + foo);
}

@PulsarListener(subscriptionName = "hello-pulsar-partitioned-subscription", topics = "hello-pulsar-partitioned", subscriptionType = SubscriptionType.Failover)
public void listen3(String foo) {
System.out.println("Message Received 3: " + foo);
}

采用这种方式时,单个分区总是由一个专用消费者消费。

类似地,如果你想使用 Pulsar 的共享消费者类型,可以使用 shared 订阅类型。不过,使用 shared 模式时,你会失去任何顺序保证,因为单个消费者可能在另一个消费者获得机会之前就收到来自所有分区的消息。

考虑以下示例:

@PulsarListener(subscriptionName = "hello-pulsar-shared-subscription", topics = "hello-pulsar-partitioned", subscriptionType = SubscriptionType.Shared)
public void listen1(String foo) {
System.out.println("Message Received 1: " + foo);
}

@PulsarListener(subscriptionName = "hello-pulsar-shared-subscription", topics = "hello-pulsar-partitioned", subscriptionType = SubscriptionType.Shared)
public void listen2(String foo) {
System.out.println("Message Received 2: " + foo);
}