系列
開(kāi)篇
- 這個(gè)系列主要用以分析mqadmin常見(jiàn)的比較核心的幾個(gè)命令，主要包括訂閱分組和topic的創(chuàng)建和刪除、Topic的權(quán)限變更拴驮。
- 這篇文章主要是用來(lái)分析訂閱分組創(chuàng)建和刪除抖部。
訂閱分組的作用
public class PullMessageProcessor extends AsyncNettyRequestProcessor implements NettyRequestProcessor {
private RemotingCommand processRequest(final Channel channel, RemotingCommand request, boolean brokerAllowSuspend)
throws RemotingCommandException {
RemotingCommand response = RemotingCommand.createResponseCommand(PullMessageResponseHeader.class);
final PullMessageResponseHeader responseHeader = (PullMessageResponseHeader) response.readCustomHeader();
final PullMessageRequestHeader requestHeader =
(PullMessageRequestHeader) request.decodeCommandCustomHeader(PullMessageRequestHeader.class);
response.setOpaque(request.getOpaque());
// 查找SubscriptionGroupConfig，如果不存在就會(huì)直接返回錯(cuò)誤
SubscriptionGroupConfig subscriptionGroupConfig =
this.brokerController.getSubscriptionGroupManager().findSubscriptionGroupConfig(requestHeader.getConsumerGroup());
if (null == subscriptionGroupConfig) {
response.setCode(ResponseCode.SUBSCRIPTION_GROUP_NOT_EXIST);
response.setRemark(String.format("subscription group [%s] does not exist, %s", requestHeader.getConsumerGroup(), FAQUrl.suggestTodo(FAQUrl.SUBSCRIPTION_GROUP_NOT_EXIST)));
return response;
}
}
- SubscriptionGroupConfig可以理解為消費(fèi)分組和broker之間的契約，如果broker不存在消費(fèi)分組的訂閱關(guān)系，那么該消費(fèi)分組就無(wú)法從該broker消費(fèi)任何消息。
創(chuàng)建訂閱分組
updateSubGroup
usage: mqadmin updateSubGroup [-a <arg>] [-b <arg>] [-c <arg>] [-d <arg>] -g <arg> [-h] [-i <arg>] [-m <arg>]
[-n <arg>] [-q <arg>] [-r <arg>] [-s <arg>] [-w <arg>]
-a,--notifyConsumerIdsChanged <arg> notify consumerId changed
-b,--brokerAddr <arg> create subscription group to which broker
-c,--clusterName <arg> create subscription group to which cluster
-d,--consumeBroadcastEnable <arg> broadcast
-g,--groupName <arg> consumer group name
-h,--help Print help
-i,--brokerId <arg> consumer from which broker id
-m,--consumeFromMinEnable <arg> from min offset
-n,--namesrvAddr <arg> Name server address list, eg: 192.168.0.1:9876;192.168.0.2:9876
-q,--retryQueueNums <arg> retry queue nums
-r,--retryMaxTimes <arg> retry max times
-s,--consumeEnable <arg> consume enable
-w,--whichBrokerWhenConsumeSlowly <arg> which broker id when consume slowly
- updateSubGroup負(fù)責(zé)創(chuàng)建訂閱分組验懊。
- updateSubGroup的命令參數(shù)中通過(guò)--groupName來(lái)指定consumer group name。
UpdateSubGroupSubCommand
public class UpdateSubGroupSubCommand implements SubCommand {
@Override
public String commandName() {
return "updateSubGroup";
}
@Override
public String commandDesc() {
return "Update or create subscription group";
}
@Override
public Options buildCommandlineOptions(Options options) {
Option opt = new Option("b", "brokerAddr", true, "create subscription group to which broker");
opt.setRequired(false);
options.addOption(opt);
opt = new Option("c", "clusterName", true, "create subscription group to which cluster");
opt.setRequired(false);
options.addOption(opt);
opt = new Option("g", "groupName", true, "consumer group name");
opt.setRequired(true);
options.addOption(opt);
opt = new Option("s", "consumeEnable", true, "consume enable");
opt.setRequired(false);
options.addOption(opt);
opt = new Option("m", "consumeFromMinEnable", true, "from min offset");
opt.setRequired(false);
options.addOption(opt);
opt = new Option("d", "consumeBroadcastEnable", true, "broadcast");
opt.setRequired(false);
options.addOption(opt);
opt = new Option("q", "retryQueueNums", true, "retry queue nums");
opt.setRequired(false);
options.addOption(opt);
opt = new Option("r", "retryMaxTimes", true, "retry max times");
opt.setRequired(false);
options.addOption(opt);
opt = new Option("i", "brokerId", true, "consumer from which broker id");
opt.setRequired(false);
options.addOption(opt);
opt = new Option("w", "whichBrokerWhenConsumeSlowly", true, "which broker id when consume slowly");
opt.setRequired(false);
options.addOption(opt);
opt = new Option("a", "notifyConsumerIdsChanged", true, "notify consumerId changed");
opt.setRequired(false);
options.addOption(opt);
return options;
}
@Override
public void execute(final CommandLine commandLine, final Options options,
RPCHook rpcHook) throws SubCommandException {
DefaultMQAdminExt defaultMQAdminExt = new DefaultMQAdminExt(rpcHook);
defaultMQAdminExt.setInstanceName(Long.toString(System.currentTimeMillis()));
try {
SubscriptionGroupConfig subscriptionGroupConfig = new SubscriptionGroupConfig();
subscriptionGroupConfig.setConsumeBroadcastEnable(false);
subscriptionGroupConfig.setConsumeFromMinEnable(false);
// groupName
subscriptionGroupConfig.setGroupName(commandLine.getOptionValue('g').trim());
// consumeEnable
if (commandLine.hasOption('s')) {
subscriptionGroupConfig.setConsumeEnable(Boolean.parseBoolean(commandLine.getOptionValue('s')
.trim()));
}
// consumeFromMinEnable
if (commandLine.hasOption('m')) {
subscriptionGroupConfig.setConsumeFromMinEnable(Boolean.parseBoolean(commandLine
.getOptionValue('m').trim()));
}
// consumeBroadcastEnable
if (commandLine.hasOption('d')) {
subscriptionGroupConfig.setConsumeBroadcastEnable(Boolean.parseBoolean(commandLine
.getOptionValue('d').trim()));
}
// retryQueueNums
if (commandLine.hasOption('q')) {
subscriptionGroupConfig.setRetryQueueNums(Integer.parseInt(commandLine.getOptionValue('q')
.trim()));
}
// retryMaxTimes
if (commandLine.hasOption('r')) {
subscriptionGroupConfig.setRetryMaxTimes(Integer.parseInt(commandLine.getOptionValue('r')
.trim()));
}
// brokerId
if (commandLine.hasOption('i')) {
subscriptionGroupConfig.setBrokerId(Long.parseLong(commandLine.getOptionValue('i').trim()));
}
// whichBrokerWhenConsumeSlowly
if (commandLine.hasOption('w')) {
subscriptionGroupConfig.setWhichBrokerWhenConsumeSlowly(Long.parseLong(commandLine
.getOptionValue('w').trim()));
}
// notifyConsumerIdsChanged
if (commandLine.hasOption('a')) {
subscriptionGroupConfig.setNotifyConsumerIdsChangedEnable(Boolean.parseBoolean(commandLine
.getOptionValue('a').trim()));
}
if (commandLine.hasOption('b')) {
String addr = commandLine.getOptionValue('b').trim();
defaultMQAdminExt.start();
defaultMQAdminExt.createAndUpdateSubscriptionGroupConfig(addr, subscriptionGroupConfig);
System.out.printf("create subscription group to %s success.%n", addr);
System.out.printf("%s", subscriptionGroupConfig);
return;
} else if (commandLine.hasOption('c')) {
String clusterName = commandLine.getOptionValue('c').trim();
defaultMQAdminExt.start();
Set<String> masterSet =
CommandUtil.fetchMasterAddrByClusterName(defaultMQAdminExt, clusterName);
for (String addr : masterSet) {
try {
defaultMQAdminExt.createAndUpdateSubscriptionGroupConfig(addr, subscriptionGroupConfig);
System.out.printf("create subscription group to %s success.%n", addr);
} catch (Exception e) {
e.printStackTrace();
Thread.sleep(1000 * 1);
}
}
System.out.printf("%s", subscriptionGroupConfig);
return;
}
ServerUtil.printCommandLineHelp("mqadmin " + this.commandName(), options);
} catch (Exception e) {
throw new SubCommandException(this.getClass().getSimpleName() + " command failed", e);
} finally {
defaultMQAdminExt.shutdown();
}
}
}
- UpdateSubGroupSubCommand是處理updateSubGroup命令的函數(shù)尸变。
- 核心邏輯在于組裝subscriptionGroupConfig對(duì)象义图，subscriptionGroupConfig就是訂閱分組的變量。
- UpdateSubGroupSubCommand可以指定broker或者指定cluster召烂，指定broker就在指定的broker上創(chuàng)建訂閱分組碱工，指定cluster就在cluster下的所有broker上創(chuàng)建訂閱分組。
MQClientAPIImpl
public class MQClientAPIImpl {
public void createSubscriptionGroup(final String addr, final SubscriptionGroupConfig config,
final long timeoutMillis)
throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.UPDATE_AND_CREATE_SUBSCRIPTIONGROUP, null);
byte[] body = RemotingSerializable.encode(config);
request.setBody(body);
RemotingCommand response = this.remotingClient.invokeSync(MixAll.brokerVIPChannel(this.clientConfig.isVipChannelEnabled(), addr),
request, timeoutMillis);
assert response != null;
switch (response.getCode()) {
case ResponseCode.SUCCESS: {
return;
}
default:
break;
}
throw new MQClientException(response.getCode(), response.getRemark());
}
}
- 創(chuàng)建訂閱分組的RequestCode為UPDATE_AND_CREATE_SUBSCRIPTIONGROUP奏夫。
AdminBrokerProcessor
public class AdminBrokerProcessor extends AsyncNettyRequestProcessor implements NettyRequestProcessor {
private RemotingCommand updateAndCreateSubscriptionGroup(ChannelHandlerContext ctx, RemotingCommand request)
throws RemotingCommandException {
final RemotingCommand response = RemotingCommand.createResponseCommand(null);
log.info("updateAndCreateSubscriptionGroup called by {}", RemotingHelper.parseChannelRemoteAddr(ctx.channel()));
SubscriptionGroupConfig config = RemotingSerializable.decode(request.getBody(), SubscriptionGroupConfig.class);
if (config != null) {
this.brokerController.getSubscriptionGroupManager().updateSubscriptionGroupConfig(config);
}
response.setCode(ResponseCode.SUCCESS);
response.setRemark(null);
return response;
}
- updateAndCreateSubscriptionGroup在broker負(fù)責(zé)處理消費(fèi)分組的創(chuàng)建邏輯怕篷。
- SubscriptionGroupManager負(fù)責(zé)保存訂閱分組。
SubscriptionGroupManager
public class SubscriptionGroupManager extends ConfigManager {
private static final InternalLogger log = InternalLoggerFactory.getLogger(LoggerName.BROKER_LOGGER_NAME);
private final ConcurrentMap<String, SubscriptionGroupConfig> subscriptionGroupTable =
new ConcurrentHashMap<String, SubscriptionGroupConfig>(1024);
private final DataVersion dataVersion = new DataVersion();
public void updateSubscriptionGroupConfig(final SubscriptionGroupConfig config) {
SubscriptionGroupConfig old = this.subscriptionGroupTable.put(config.getGroupName(), config);
if (old != null) {
log.info("update subscription group config, old: {} new: {}", old, config);
} else {
log.info("create new subscription group, {}", config);
}
this.dataVersion.nextVersion();
this.persist();
}
}
- SubscriptionGroupManager的subscriptionGroupTable負(fù)責(zé)保存訂閱分組酗昼。
刪除訂閱分組
deleteSubGroup
usage: mqadmin deleteSubGroup [-b <arg>] [-c <arg>] -g <arg> [-h] [-n <arg>]
-b,--brokerAddr <arg> delete subscription group from which broker
-c,--clusterName <arg> delete subscription group from which cluster
-g,--groupName <arg> subscription group name
-h,--help Print help
-n,--namesrvAddr <arg> Name server address list, eg: 192.168.0.1:9876;192.168.0.2:9876
- deleteSubGroup負(fù)責(zé)刪除訂閱分組廊谓。
- deleteSubGroup命令參數(shù)中通過(guò)--groupName來(lái)指定consumer group name。
DeleteSubscriptionGroupCommand
public class DeleteSubscriptionGroupCommand implements SubCommand {
@Override
public String commandName() {
return "deleteSubGroup";
}
@Override
public String commandDesc() {
return "Delete subscription group from broker.";
}
@Override
public Options buildCommandlineOptions(Options options) {
Option opt = new Option("b", "brokerAddr", true, "delete subscription group from which broker");
opt.setRequired(false);
options.addOption(opt);
opt = new Option("c", "clusterName", true, "delete subscription group from which cluster");
opt.setRequired(false);
options.addOption(opt);
opt = new Option("g", "groupName", true, "subscription group name");
opt.setRequired(true);
options.addOption(opt);
return options;
}
@Override
public void execute(CommandLine commandLine, Options options, RPCHook rpcHook) throws SubCommandException {
DefaultMQAdminExt adminExt = new DefaultMQAdminExt(rpcHook);
adminExt.setInstanceName(Long.toString(System.currentTimeMillis()));
try {
// groupName
String groupName = commandLine.getOptionValue('g').trim();
if (commandLine.hasOption('b')) {
String addr = commandLine.getOptionValue('b').trim();
adminExt.start();
adminExt.deleteSubscriptionGroup(addr, groupName);
System.out.printf("delete subscription group [%s] from broker [%s] success.%n", groupName,
addr);
return;
} else if (commandLine.hasOption('c')) {
String clusterName = commandLine.getOptionValue('c').trim();
adminExt.start();
Set<String> masterSet = CommandUtil.fetchMasterAddrByClusterName(adminExt, clusterName);
for (String master : masterSet) {
adminExt.deleteSubscriptionGroup(master, groupName);
System.out.printf(
"delete subscription group [%s] from broker [%s] in cluster [%s] success.%n",
groupName, master, clusterName);
}
try {
DeleteTopicSubCommand.deleteTopic(adminExt, clusterName, MixAll.RETRY_GROUP_TOPIC_PREFIX
+ groupName);
DeleteTopicSubCommand.deleteTopic(adminExt, clusterName, MixAll.DLQ_GROUP_TOPIC_PREFIX
+ groupName);
} catch (Exception e) {
e.printStackTrace();
}
return;
}
ServerUtil.printCommandLineHelp("mqadmin " + this.commandName(), options);
} catch (Exception e) {
throw new SubCommandException(this.getClass().getSimpleName() + " command failed", e);
} finally {
adminExt.shutdown();
}
}
}
- DeleteSubscriptionGroupCommand是處理deleteSubGroup命令的函數(shù)麻削。
- 核心邏輯在于往指定的broker刪除指定的consumer groupName蒸痹。
- DeleteSubscriptionGroupCommand可以指定broker或者指定cluster，指定broker就在指定的broker上刪除訂閱分組呛哟，指定cluster就在cluster下的所有broker上刪除訂閱分組叠荠。
- 刪除集群下訂閱分組即消費(fèi)分組，同時(shí)需要?jiǎng)h除消費(fèi)分組的重試Topic和死信Topic扫责。
deleteSubscriptionGroup
public class MQClientAPIImpl {
public void deleteSubscriptionGroup(final String addr, final String groupName, final long timeoutMillis)
throws RemotingException, MQBrokerException, InterruptedException, MQClientException {
DeleteSubscriptionGroupRequestHeader requestHeader = new DeleteSubscriptionGroupRequestHeader();
requestHeader.setGroupName(groupName);
RemotingCommand request = RemotingCommand.createRequestCommand(RequestCode.DELETE_SUBSCRIPTIONGROUP, requestHeader);
RemotingCommand response = this.remotingClient.invokeSync(MixAll.brokerVIPChannel(this.clientConfig.isVipChannelEnabled(), addr),
request, timeoutMillis);
assert response != null;
switch (response.getCode()) {
case ResponseCode.SUCCESS: {
return;
}
default:
break;
}
throw new MQClientException(response.getCode(), response.getRemark());
}
}
- 刪除訂閱分組的RequestCode為DELETE_SUBSCRIPTIONGROUP榛鼎。
AdminBrokerProcessor
public class AdminBrokerProcessor extends AsyncNettyRequestProcessor implements NettyRequestProcessor {
private RemotingCommand deleteSubscriptionGroup(ChannelHandlerContext ctx,
RemotingCommand request) throws RemotingCommandException {
final RemotingCommand response = RemotingCommand.createResponseCommand(null);
DeleteSubscriptionGroupRequestHeader requestHeader =
(DeleteSubscriptionGroupRequestHeader) request.decodeCommandCustomHeader(DeleteSubscriptionGroupRequestHeader.class);
log.info("deleteSubscriptionGroup called by {}", RemotingHelper.parseChannelRemoteAddr(ctx.channel()));
this.brokerController.getSubscriptionGroupManager().deleteSubscriptionGroupConfig(requestHeader.getGroupName());
if (this.brokerController.getBrokerConfig().isAutoDeleteUnusedStats()) {
this.brokerController.getBrokerStatsManager().onGroupDeleted(requestHeader.getGroupName());
}
response.setCode(ResponseCode.SUCCESS);
response.setRemark(null);
return response;
}
}
- deleteSubscriptionGroup在broker負(fù)責(zé)處理訂閱分組即消費(fèi)分組的刪除邏輯。
- SubscriptionGroupManager負(fù)責(zé)刪除訂閱分組。
SubscriptionGroupManager
public class SubscriptionGroupManager extends ConfigManager {
private static final InternalLogger log = InternalLoggerFactory.getLogger(LoggerName.BROKER_LOGGER_NAME);
private final ConcurrentMap<String, SubscriptionGroupConfig> subscriptionGroupTable =
new ConcurrentHashMap<String, SubscriptionGroupConfig>(1024);
private final DataVersion dataVersion = new DataVersion();
private transient BrokerController brokerController;
public void deleteSubscriptionGroupConfig(final String groupName) {
SubscriptionGroupConfig old = this.subscriptionGroupTable.remove(groupName);
if (old != null) {
log.info("delete subscription group OK, subscription group:{}", old);
this.dataVersion.nextVersion();
this.persist();
} else {
log.warn("delete subscription group failed, subscription groupName: {} not exist", groupName);
}
}
}
- 從SubscriptionGroupManager的subscriptionGroupTable刪除訂閱分組者娱。
