kafka同消费者组多消费者锁争抢解决方案

背景

多个租户的消息推送到kafka，同一个消费者组的多个消费者消费。导致同一时间多个消费者消费到同一个租户的信息，造成数据库行级锁等待。

解决方案

把同一个租户的消息发送到kafka时，取模路由到同一个分区，这样同一个租户的消息只会被一个消费者消费，避免数据库多个事务中对同一个租户操作造成行级锁等待。

spring.cloud.stream.bindings.billChargeRecordVoiceRobotOutput.producer.partitionKeyExtractorName
=billVoiceRobotPartitionKeyExtractor

spring.cloud.stream.bindings.billChargeRecordVoiceRobotOutput.producer.partitionSelectorName
=billVoiceRobotPartitionSelector

// 提取message的key
public class BillVoiceRobotPartitionKeyExtractor implements PartitionKeyExtractorStrategy {

    private static final Logger logger = LoggerFactory.getLogger(BillVoiceRobotPartitionKeyExtractor.class);

    @Override
    public Object extractKey(Message<?> message) {
        Object payload = message.getPayload();

        if (payload instanceof String) {
            MealResourceVoiceRobotEventDto eventDto = JSON.parseObject(payload.toString(), MealResourceVoiceRobotEventDto.class);
            if (StringUtils.isBlank(eventDto.getTenantId()) || null == eventDto.getCallType()) {
                return null;
            }
            String str = eventDto.getTenantId()   eventDto.getCallType().toString();
            return str.hashCode();
        }
        return null;
    }
}

//根据key，选择路由到哪个partition
public class BillVoiceRobotPartitionSelectorClass implements PartitionSelectorStrategy {
    private static final Logger logger = LoggerFactory.getLogger(BillVoiceRobotPartitionSelectorClass.class);

    @Override
    public int selectPartition(Object key, int partitionCount) {
        //logger.info("消息载荷的key：{},partitionCount：{}", key, partitionCount);
        return key == null ? 0 : ((int)key % partitionCount);
    }
}

原理

kafka一个分区的数据只会被同一个消费者组的某一个消费者消费。

这篇好文章是转载于：学新通技术网