上一篇寫(xiě)了《【分布式日志系統(tǒng)】springboot+zipkin+dubbo實(shí)現(xiàn)鏈路跟蹤(上)》《【分布式日志系統(tǒng)】springboot+zipkin+dubbo實(shí)現(xiàn)鏈路跟蹤(中)》,有興趣的小伙伴可以往回翻翻,這一篇解決如何跟蹤笑陈。
一、基本操作
- pom添加依賴(lài)(常規(guī)操作)刺洒;
- 定義公共跟蹤配置類(lèi)
- dubbo服務(wù)端和調(diào)用端集成相應(yīng)配置(主要是yaml中配置);
- 驗(yàn)證挪鹏。
二袁辈、環(huán)境介紹
提示:可能不同的框架版本會(huì)導(dǎo)致有些地方不生效(如sleuth不支持apache版的dubbo)汗洒,大家在集成的過(guò)程中有問(wèn)題可以私信我挖滤,共同探討崩溪。
主框架版本如下:
springboot 2.6.6、
dubbo 2.7.12斩松、
zipkin 2.16.3伶唯、
brave 5.13.3
nacos-discovery-spring-boot-starter 0.2.10、
nacos-config-spring-boot-starter 0.2.10惧盹、
三 工程結(jié)構(gòu)
- basic-platform:父工程乳幸,管理框架版本等基礎(chǔ)依賴(lài);
- basic-common:基礎(chǔ)工具包钧椰,“定義公共跟蹤基礎(chǔ)配置類(lèi)”等均在此工程粹断,其他工程均依賴(lài)該工程;
- basic-portal:門(mén)戶(hù)的父工程(pom工程)演侯,其下包含三個(gè)子工程:
- server:前后端分離對(duì)應(yīng)的后端服務(wù) 姿染;
- portal-api:定義dubbo服務(wù)的接口背亥;
- portal-api-impl:定義portal-api對(duì)應(yīng)的接口實(shí)現(xiàn)秒际;
- basic-foundation:基礎(chǔ)的父工程(pom工程),其下包含兩個(gè)子工程:
- foundation-api:定義dubbo服務(wù)的接口狡汉;
- foundation-api-impl:定義 foundation-api對(duì)應(yīng)的接口實(shí)現(xiàn)娄徊;
四、pom依賴(lài)
主要的代碼均在basic-common的pom中盾戴,添加如下依賴(lài):
<dependency>
<groupId>org.apache.dubbo</groupId>
<artifactId>dubbo</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>io.zipkin.brave</groupId>
<artifactId>brave-instrumentation-dubbo</artifactId>
</dependency>
<dependency>
<groupId>io.zipkin.brave</groupId>
<artifactId>brave-spring-beans</artifactId>
</dependency>
<dependency>
<groupId>io.zipkin.brave</groupId>
<artifactId>brave-context-slf4j</artifactId>
</dependency>
<!-- tracing -->
<dependency>
<groupId>io.zipkin.reporter2</groupId>
<artifactId>zipkin-sender-okhttp3</artifactId>
</dependency>
<!-- tracing & mvc-->
<dependency>
<groupId>io.zipkin.brave</groupId>
<artifactId>brave-instrumentation-spring-webmvc</artifactId>
</dependency>
<!-- tracing & http-->
<dependency>
<groupId>io.zipkin.brave</groupId>
<artifactId>brave-instrumentation-httpclient</artifactId>
</dependency>
<dependency>
<groupId>io.zipkin.reporter2</groupId>
<artifactId>zipkin-sender-okhttp3</artifactId>
</dependency>
另外寄锐,需要定義一個(gè)配置類(lèi),來(lái)處理RPC的tracing尖啡,要不會(huì)不生效橄仆。
package com.wd.basic.common.support.trace;
import brave.CurrentSpanCustomizer;
import brave.SpanCustomizer;
import brave.Tracing;
import brave.context.slf4j.MDCScopeDecorator;
import brave.http.HttpTracing;
import brave.propagation.B3Propagation;
import brave.propagation.ExtraFieldPropagation;
import brave.propagation.ThreadLocalCurrentTraceContext;
import brave.rpc.RpcTracing;
import brave.sampler.Sampler;
import brave.servlet.TracingFilter;
import brave.spring.webmvc.SpanCustomizingAsyncHandlerInterceptor;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.condition.ConditionalOnBean;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Import;
import org.springframework.web.servlet.config.annotation.InterceptorRegistry;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurer;
import zipkin2.Span;
import zipkin2.codec.Encoding;
import zipkin2.reporter.AsyncReporter;
import zipkin2.reporter.Sender;
import zipkin2.reporter.okhttp3.OkHttpSender;
import javax.servlet.Filter;
@Configuration
@Import(SpanCustomizingAsyncHandlerInterceptor.class)
public class TracingConfig implements WebMvcConfigurer {
@Bean
Sender sender(@Value("${zipkin.base.url}") String url) {
return OkHttpSender.newBuilder()
.encoding(Encoding.PROTO3)
.endpoint(url)
.build();
}
/**
* Configuration for how to buffer spans into messages for Zipkin
*/
@Bean
@ConditionalOnBean(Sender.class)
AsyncReporter<Span> spanReporter(Sender sender) {
AsyncReporter.Builder builder = AsyncReporter.builder(sender);
builder.queuedMaxSpans(50000);
builder.queuedMaxBytes(104857600);
return builder.build();
}
/**
* Controls aspects of tracing such as the service name that shows up in the UI
*/
@Bean
Tracing tracing(@Value("${dubbo.application.name}") String applicationName, @Value("${zipkin.enable:false}") Boolean enable, @Autowired(required = false) AsyncReporter spanReporter) {
Tracing.Builder builder = Tracing.newBuilder()
.localServiceName(applicationName)
.propagationFactory(ExtraFieldPropagation.newFactory(B3Propagation.FACTORY, "user-name"))
.currentTraceContext(ThreadLocalCurrentTraceContext.newBuilder()
// puts trace IDs into logs
.addScopeDecorator(MDCScopeDecorator.create())
.build()
);
if (enable) {
builder.spanReporter(spanReporter);
builder.sampler(Sampler.ALWAYS_SAMPLE);
} else {
builder.sampler(Sampler.NEVER_SAMPLE);
}
return builder.build();
}
@Bean
SpanCustomizer spanCustomizer(Tracing tracing) {
return CurrentSpanCustomizer.create(tracing);
}
/**
* Decides how to name and tag spans. By default they are named the same as the http method
*/
@Bean
HttpTracing httpTracing(Tracing tracing) {
return HttpTracing.create(tracing);
}
@Bean
RpcTracing rpcTracing(Tracing tracing) {
return RpcTracing.create(tracing);
}
/**
* Creates server spans for http requests
*/
@Bean
Filter tracingFilter(HttpTracing httpTracing) {
return TracingFilter.create(httpTracing);
}
@Autowired
SpanCustomizingAsyncHandlerInterceptor webMvcTracingCustomizer;
/**
* Decorates server spans with application-defined web tags
*/
@Override
public void addInterceptors(InterceptorRegistry registry) {
registry.addInterceptor(webMvcTracingCustomizer);
}
}
五、其他工程添加配置
前提:所有工程均依賴(lài)basic-common衅斩。
dubbo工程的provider和consumer均添加如下配置(請(qǐng)注意必須添加dubbo.consumer.filter和dubbo.provider.filter)
dubbo:
application:
name: foundationRpc #服務(wù)名稱(chēng)
consumer:
timeout: 3000 #消費(fèi)超時(shí)時(shí)間
retries: 1 #重試次數(shù)
filter: tracing # 請(qǐng)注意必須添加該配置
check: false
provider:
filter: tracing # 請(qǐng)注意必須添加該配置
protocol:
name: dubbo
zipkin:
enable: true
base:
url: http://127.0.0.1:9411/api/v2/spans
六盆顾、測(cè)試
設(shè)計(jì)如下調(diào)用關(guān)系:
server定義http入口:
@GetMapping("tracing")
public String exec(){
log.info("come in tracing");
return rpcDemoService.tracing();
}
rpcDemoService為server中定義的service,實(shí)現(xiàn)如下:
package com.wd.basic.portal.server.modular.demo.service.impl;
import cn.hutool.core.util.StrUtil;
import com.wd.basic.foundation.rpc.test.service.DemoRpcService;
import com.wd.basic.portal.rpc.IDemoService;
import com.wd.basic.portal.rpc.ITraceDemoService;
import com.wd.basic.portal.server.modular.demo.service.RpcDemoService;
import lombok.extern.slf4j.Slf4j;
import org.apache.dubbo.config.annotation.DubboReference;
import org.springframework.stereotype.Service;
/**
* rpc演示服務(wù)impl
*
* @author 小塵哥
* @date 2022/04/14
*/
@Slf4j
@Service
public class RpcDemoServiceImpl implements RpcDemoService {
@DubboReference(interfaceClass = IDemoService.class, version = "2.0")
private IDemoService demoService2;
@DubboReference(interfaceClass = IDemoService.class, version = "1.0")
private IDemoService demoService1;
@DubboReference(interfaceClass = ITraceDemoService.class, version = "1.0")
private ITraceDemoService traceDemoService;
@DubboReference(interfaceClass = DemoRpcService.class, version = "1.0")
private DemoRpcService demoRpcService;
@Override
public String tracing() {
log.info("come in exec service");
String rpc2 = demoService2.testRpc();
return StrUtil.join(",", rpc2);
}
}
IDemoService 定義在portal-api中畏梆,實(shí)現(xiàn)如下
package com.wd.basic.portal.rpc.impl;
import com.wd.basic.foundation.rpc.test.service.DemoRpcService;
import com.wd.basic.portal.rpc.IDemoService;
import lombok.extern.slf4j.Slf4j;
import org.apache.dubbo.config.annotation.DubboReference;
import org.apache.dubbo.config.annotation.DubboService;
/**
* 演示服務(wù)impl
*
* @author 小塵哥
* @date 2022/03/16
*/
@Slf4j
@DubboService(interfaceClass = IDemoService.class,version = "2.0")
public class DemoServiceV2Impl implements IDemoService {
@DubboReference(interfaceClass = DemoRpcService.class, version = "1.0")
private DemoRpcService demoRpcService;
@Override
public String testRpc() {
log.info("來(lái)自統(tǒng)一門(mén)戶(hù) RPC 測(cè)試");
demoRpcService.testRpc();
return "調(diào)用到了rpc服務(wù) version 2.0.0";
}
}
DemoRpcService 定義在foundation-api中您宪,實(shí)現(xiàn)如下:
package com.wd.basic.foundation.rpc.test.service.impl;
import com.wd.basic.foundation.rpc.test.service.DemoRpcService;
import lombok.extern.slf4j.Slf4j;
import org.apache.dubbo.config.annotation.DubboService;
/**
* test Rpc服務(wù)
* @author 上官婉兒
*/
@Slf4j
@DubboService(interfaceClass = DemoRpcService.class,version = "1.0")
public class DemoRpcServiceImpl implements DemoRpcService {
@Override
public String testRpc() {
log.info("來(lái)自基礎(chǔ)服務(wù) RPC 測(cè)試");
return "RPC服務(wù)! from foundation";
}
}
七奠涌、日志配置
臨門(mén)一腳了宪巨,以上都配置好了,但是哪里能看到效果呢溜畅?肯定是需要從日志來(lái)跟蹤捏卓,我們采用springboot推薦的logback來(lái)記錄日志,請(qǐng)注意日志記錄格式慈格,添加了部分內(nèi)容“【%X{traceId},%X{spanId},%X{parentId}】”怠晴,完整配置如下:
<?xml version="1.0" encoding="UTF-8"?>
<!-- 日志級(jí)別從低到高分為T(mén)RACE < DEBUG < INFO < WARN < ERROR < FATAL贿肩,如果設(shè)置為WARN,則低于WARN的信息都不會(huì)輸出 -->
<!-- scan:當(dāng)此屬性設(shè)置為true時(shí)龄寞,配置文件如果發(fā)生改變汰规,將會(huì)被重新加載,默認(rèn)值為true -->
<!-- scanPeriod:設(shè)置監(jiān)測(cè)配置文件是否有修改的時(shí)間間隔物邑,如果沒(méi)有給出時(shí)間單位溜哮,默認(rèn)單位是毫秒。當(dāng)scan為true時(shí)色解,此屬性生效茂嗓。默認(rèn)的時(shí)間間隔為1分鐘。 -->
<!-- debug:當(dāng)此屬性設(shè)置為true時(shí)科阎,將打印出logback內(nèi)部日志信息述吸,實(shí)時(shí)查看logback運(yùn)行狀態(tài)。默認(rèn)值為false锣笨。 -->
<configuration scan="true" scanPeriod="10 seconds">
<!-- <include resource="org/springframework/boot/logging/logback/base.xml"
/> -->
<contextName>Logback For Basic Platform</contextName>
<!-- name的值是變量的名稱(chēng)蝌矛,value的值時(shí)變量定義的值。通過(guò)定義的值會(huì)被插入到logger上下文中错英。定義變量后入撒,可以使“${}”來(lái)使用變量。 -->
<!-- 定義日志文件 輸入位置 -->
<springProperty scope="context" name="logDir" source="custom.log.dir" defaultValue="D:/logback"/>
<!-- 定義日志文件 輸出級(jí)別 -->
<springProperty scope="context" name="logLevel" source="custom.log.level" defaultValue="info"/>
<!-- 日志最大的歷史 30天 -->
<springProperty scope="context" name="logMaxHistory" source="custom.log.max-history" defaultValue="180"/>
<!-- 控制臺(tái)輸出日志 -->
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} 【%X{traceId},%X{spanId},%X{parentId}】 [%thread] %-5level %logger-%msg%n %ex{5}</pattern>
<charset class="java.nio.charset.Charset">UTF-8</charset>
</encoder>
</appender>
<!--文件日志椭岩, 按照每天生成日志文件 -->
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<!--日志文件輸出的文件名-->
<fileNamePattern>${logDir}/%d{yyyy-MM-dd}/pf.%i.log.zip</fileNamePattern>
<maxFileSize>50MB</maxFileSize> <!-- 日志文件過(guò)大會(huì)使的編輯器打開(kāi)非常慢茅逮,因此設(shè)置日志最大50MB -->
<!--日志文件保留天數(shù)-->
<maxHistory>${logMaxHistory}</maxHistory>
<totalSizeCap>10GB</totalSizeCap> <!-- 總?cè)罩敬笮?-->
</rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<!--格式化輸出:%d表示日期,%thread表示線(xiàn)程名判哥,%-5level:級(jí)別從左顯示5個(gè)字符寬度%msg:日志消息献雅,%n是換行符-->
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} 【%X{traceId},%X{spanId},%X{parentId}】 [%thread] %-5level %logger-%msg%n %ex{5}</pattern>
</encoder>
</appender>
<!-- 異步輸出 -->
<appender name="dayLogAsyncAppender" class="ch.qos.logback.classic.AsyncAppender">
<includeCallerData>true</includeCallerData>
<!-- 不丟失日志.默認(rèn)的,如果隊(duì)列的80%已滿(mǎn),則會(huì)丟棄TRACT、DEBUG塌计、INFO級(jí)別的日志 -->
<discardingThreshold>0</discardingThreshold>
<!-- 更改默認(rèn)的隊(duì)列的深度,該值會(huì)影響性能.默認(rèn)值為256 -->
<queueSize>512</queueSize>
<appender-ref ref="FILE"/>
</appender>
<!--專(zhuān)為 spring 定制
-->
<logger name="org.springframework" level="${logLevel}"/>
<!-- root級(jí)別 DEBUG -->
<root level="${logLevel}">
<!-- 控制臺(tái)輸出 -->
<appender-ref ref="STDOUT" />
<!-- 文件輸出 -->
<appender-ref ref="dayLogAsyncAppender" />
</root>
</configuration>
八挺身、驗(yàn)證
來(lái)自server的日志
2022-04-26 16:44:25.817 【2926aa52265ad112,2926aa52265ad112,】 [http-nio-8080-exec-1] INFO com.wd.basic.portal.server.modular.demo.controller.RpcDemoController-come in tracing
2022-04-26 16:44:25.826 【2926aa52265ad112,2926aa52265ad112,】 [http-nio-8080-exec-1] INFO com.wd.basic.portal.server.modular.demo.service.impl.RpcDemoServiceImpl-come in exec service
來(lái)自portal-api的日志
2022-04-26 16:44:25.882 【2926aa52265ad112,6a4da6311934ab65,2926aa52265ad112】 [DubboServerHandler-10.1.252.224:20881-thread-5] INFO com.wd.basic.portal.rpc.impl.DemoServiceV2Impl-來(lái)自統(tǒng)一門(mén)戶(hù) RPC 測(cè)試
來(lái)自foundation-api的日志
2022-04-26 16:44:25.926 【2926aa52265ad112,53dbfb344223ef50,6a4da6311934ab65】 [DubboServerHandler-10.1.252.224:20880-thread-3] INFO com.wd.basic.foundation.rpc.test.service.impl.DemoRpcServiceImpl-來(lái)自基礎(chǔ)服務(wù) RPC 測(cè)試
九、日志分析
提取三部分日志中括號(hào)中的內(nèi)容:
統(tǒng)一門(mén)戶(hù):traceId = 2926aa52265ad112,spanId=2926aa52265ad112夺荒,parentId=’’;
統(tǒng)一門(mén)戶(hù)RPC服務(wù):traceId = 2926aa52265ad112,spanId=6a4da6311934ab65瞒渠,parentId=2926aa52265ad112;
基礎(chǔ)服務(wù)RPC服務(wù):traceId = 2926aa52265ad112,spanId=53dbfb344223ef50,parentId=6a4da6311934ab65;
說(shuō)明:
- 對(duì)于server技扼,作為請(qǐng)求入口伍玖,因此traceId和spanId相同,無(wú)上游服務(wù)剿吻,因此parentId為空窍箍。
- 對(duì)于portal-api服務(wù),parentId=統(tǒng)一門(mén)戶(hù)請(qǐng)求入口的spanId,因此它是統(tǒng)一門(mén)戶(hù)的下游服務(wù)椰棘。
- 對(duì)于foundation-api服務(wù)纺棺,parentId=同一門(mén)戶(hù)RPC服務(wù)的spanId,因此它是統(tǒng)一門(mén)戶(hù)RPC服務(wù)的下游服務(wù)邪狞。
綜上祷蝌,和測(cè)試開(kāi)始時(shí)的設(shè)計(jì)邏輯一致,由traceId串聯(lián)整個(gè)請(qǐng)求過(guò)程帆卓,由spanId和parentId構(gòu)建上下游調(diào)用關(guān)系巨朦,完成dubbo服務(wù)之間調(diào)用的鏈路跟蹤。