緩存的一般思路
下面是我理解的網(wǎng)絡請求框架的緩存基本實現(xiàn)搜贤。大致的過程是有緩存用緩存的數(shù)據(jù)，沒緩存發(fā)起http請求取數(shù)據(jù)些举，得到最新數(shù)據(jù)后存到緩存里馒吴。
image.png
那么Okhttp怎么實現(xiàn)緩存的，我們從Okhttp發(fā)起一次請求的全過程中來看緩存是怎么實現(xiàn)的
Okhttp請求過程源碼分析
最簡單的使用（以下代碼都是okhttp3.8.0為基礎）：
Response response = client.newCall(request).execute()
追蹤到Call接口的實現(xiàn)類RealCall的方法execute
@Override public Response execute() throws IOException {
synchronized (this) {
// 判斷是否在執(zhí)行脖母，是則拋出異常
if (executed) throw new IllegalStateException("Already Executed");
executed = true;
}
// 初始化跟蹤stack trace的對象士鸥，用來做日志，所以可以忽略先
captureCallStackTrace();
try {
//將異步的請求丟到異步的雙端隊列(Deque<RealCall> runningSyncCalls)中等待處理谆级，這里可以先忽略烤礁，直接看同步的結果
client.dispatcher().executed(this);
//獲取Response
Response result = getResponseWithInterceptorChain();
if (result == null) throw new IOException("Canceled");
return result;
} finally {
client.dispatcher().finished(this);
}
}
很明顯，獲取response在getResponseWithInterceptorChain這個方法里肥照。這里代碼很簡單脚仔，就是初始化一個interceptor列表，然后調用RealInterceptorChain的proceed函數(shù)建峭。
Response getResponseWithInterceptorChain() throws IOException {
// Build a full stack of interceptors.
List<Interceptor> interceptors = new ArrayList<>();
interceptors.addAll(client.interceptors());
interceptors.add(retryAndFollowUpInterceptor);
interceptors.add(new BridgeInterceptor(client.cookieJar()));
interceptors.add(new CacheInterceptor(client.internalCache()));
interceptors.add(new ConnectInterceptor(client));
if (!forWebSocket) {
interceptors.addAll(client.networkInterceptors());
}
interceptors.add(new CallServerInterceptor(forWebSocket));
Interceptor.Chain chain = new RealInterceptorChain(
interceptors, null, null, null, 0, originalRequest);
return chain.proceed(originalRequest);
}
interceptors（攔截器）
那么這些interceptor有什么用呢玻侥，我們挑幾個重要的一一看一下，記住這些interceptors的add順序很重要
client.interceptors()：
依次追蹤到interceptors的賦值的地方
public Builder addInterceptor(Interceptor interceptor) {
interceptors.add(interceptor);
return this;
}
public List<Interceptor> interceptors() {
return interceptors;
}
這個是不是很熟悉亿蒸，這個就是我們利用OkHttpClient.Builder builder構造okhttpClient的地方傳入的interceptor凑兰，也就是常說的application interceptor
CacheInterceptor：看名字很明顯是用來做緩存的
ConnectInterceptor：用來建立http連接
client.networkInterceptors()：同client.interceptors()，是我們創(chuàng)建okhttpclient時傳入的networkInterceptor
CallServerInterceptor：向server發(fā)請求的
RealInterceptorChain.proceed(request)
追蹤到下面的方法边锁，這時候傳入的streamAllocation姑食，httpCodec，connection都是null茅坛，index=0
public Response proceed(Request request, StreamAllocation streamAllocation, HttpCodec httpCodec,
RealConnection connection) throws IOException {
// 判斷index是否越界
if (index >= interceptors.size()) throw new AssertionError();
calls++;
// If we already have a stream, confirm that the incoming request will use it.
if (this.httpCodec != null && !this.connection.supportsUrl(request.url())) {
throw new IllegalStateException("network interceptor " + interceptors.get(index - 1)
+ " must retain the same host and port");
}
// If we already have a stream, confirm that this is the only call to chain.proceed().
if (this.httpCodec != null && calls > 1) {
throw new IllegalStateException("network interceptor " + interceptors.get(index - 1)
+ " must call proceed() exactly once");
}
// Call the next interceptor in the chain.
// 創(chuàng)建一個新的RealInterceptorChain音半，除了index+1，其他的參數(shù)都和上一個RealInterceptorChain保持不變
RealInterceptorChain next = new RealInterceptorChain(
interceptors, streamAllocation, httpCodec, connection, index + 1, request);
// 獲取當前index的Interceptor
Interceptor interceptor = interceptors.get(index);
// 執(zhí)行當前Interceptor的intercept方法贡蓖，傳入的參數(shù)為下一個RealInterceptorChain
Response response = interceptor.intercept(next);
// Confirm that the next interceptor made its required call to chain.proceed().
if (httpCodec != null && index + 1 < interceptors.size() && next.calls != 1) {
throw new IllegalStateException("network interceptor " + interceptor
+ " must call proceed() exactly once");
}
// Confirm that the intercepted response isn't null.
if (response == null) {
throw new NullPointerException("interceptor " + interceptor + " returned null");
}
return response;
}
所以RealInterceptorChain.proceed的大致過程如下
- 獲取下一個RealInterceptorChain next
- 調用當前的interceptor的intercept方法曹鸠，傳入?yún)?shù)為next
所以我們要追蹤interceptor的intercept方法,下面我以我項目里的一個做統(tǒng)計的intercptor為例來分析
public class NetStatisticsInterceptor implements Interceptor {
@Override
public Response intercept(Chain chain) throws IOException {
Request request = chain.request();
HttpUrl httpUrl = request.url();
HttpUrl.Builder urlBuilder = httpUrl.newBuilder();
if (httpUrl.queryParameter("app_version") == null) {
urlBuilder.addQueryParameter("app_version", BaseConfig.versionName);
}
// 用chain.request()構造一個新的傳入統(tǒng)計參數(shù)的request，作為參數(shù)調用chain.proceed
return chain.proceed(request.newBuilder().url(urlBuilder.build()).build());
}
}
這里對舊的oldRequest做了一堆處理斥铺，加入了一些通用的統(tǒng)計參數(shù)彻桃，包裝成生成了一個新的newRequest，然后調用chain.proceed方法晾蜘，這里又會重新調用RealInterceptorChain.proceed的方法邻眷，只是參數(shù)index+1了眠屎，request為重新包裝后的request了(其他的參數(shù)也可能變了，取決于Interceptor怎么寫)肆饶。接著又會走到RealInterceptorChain.proceed代碼里改衩，走下一個Interceptor的流程。
可以得出如下結論：
- 只要Interceptor的intercept方法調用了chain.proceed(request),就會調用Interceptor列表里的下一個Interceptor驯镊；反之可以不調用chain.proceed來打斷這個請求鏈
- 我們自定義的application interceptor和network interceptor時葫督，都必須返回chain.proceed得到的結果；否則就會打斷okhttp內(nèi)部的請求鏈
- 寫application interceptor時阿宅，在調用chain.proceed(request)之前包裝request
- 寫network interceptor時,在調用chain.proceed(request)之后得到的response包裝response
看到這里的代碼設計候衍，是不是和職責鏈模式很相似，唯一不同的是okhttp利用index自增的方式來實現(xiàn)每個攔截器的傳遞洒放。這里我必須感嘆下蛉鹿，代碼設計的真的很巧妙，還有就是設計模式這東西平惩看不出有啥用妖异，到實際碰到了真的很棒。
了解完這些攔截器怎么運行的领追，接下來具體看看各個攔截器是怎么把請求給串聯(lián)起來的他膳。
應用攔截器（client.interceptors())
這個我們常說的application interceptor因為在攔截器list的最前面，所以最先執(zhí)行绒窑，一般用于給request做一些簡單的包裝棕孙，例如添加參數(shù)，修改header等
CacheInterceptor
直接看intercept方法
@Override public Response intercept(Chain chain) throws IOException {
// 根據(jù)url獲取本地緩存
Response cacheCandidate = cache != null
? cache.get(chain.request())
: null;
long now = System.currentTimeMillis();
// 用當前時間now些膨、當前的請求request蟀俊、本地緩存cacheCandidate來構造CacheStrategy對象
// 調用strategy對象的get方法去判斷本地緩存cacheCandidate是否可用
CacheStrategy strategy = new CacheStrategy.Factory(now, chain.request(), cacheCandidate).get();
Request networkRequest = strategy.networkRequest;
Response cacheResponse = strategy.cacheResponse;
if (cache != null) {
cache.trackResponse(strategy);
}
if (cacheCandidate != null && cacheResponse == null) {
closeQuietly(cacheCandidate.body()); // The cache candidate wasn't applicable. Close it.
}
// If we're forbidden from using the network and the cache is insufficient, fail.
if (networkRequest == null && cacheResponse == null) {
return new Response.Builder()
.request(chain.request())
.protocol(Protocol.HTTP_1_1)
.code(504)
.message("Unsatisfiable Request (only-if-cached)")
.body(Util.EMPTY_RESPONSE)
.sentRequestAtMillis(-1L)
.receivedResponseAtMillis(System.currentTimeMillis())
.build();
}
// If we don't need the network, we're done.如果networkRequest為null就表示走本地緩存
if (networkRequest == null) {
return cacheResponse.newBuilder()
.cacheResponse(stripBody(cacheResponse))
.build();
}
// 走后面的interceptor鏈去取網(wǎng)絡數(shù)據(jù)得到networkResponse
Response networkResponse = null;
try {
networkResponse = chain.proceed(networkRequest);
} finally {
// If we're crashing on I/O or otherwise, don't leak the cache body.
if (networkResponse == null && cacheCandidate != null) {
closeQuietly(cacheCandidate.body());
}
}
// If we have a cache response too, then we're doing a conditional get.
if (cacheResponse != null) {
if (networkResponse.code() == HTTP_NOT_MODIFIED) {
Response response = cacheResponse.newBuilder()
.headers(combine(cacheResponse.headers(), networkResponse.headers()))
.sentRequestAtMillis(networkResponse.sentRequestAtMillis())
.receivedResponseAtMillis(networkResponse.receivedResponseAtMillis())
.cacheResponse(stripBody(cacheResponse))
.networkResponse(stripBody(networkResponse))
.build();
networkResponse.body().close();
// Update the cache after combining headers but before stripping the
// Content-Encoding header (as performed by initContentStream()).
cache.trackConditionalCacheHit();
cache.update(cacheResponse, response);
return response;
} else {
closeQuietly(cacheResponse.body());
}
}
// 用networkResponse、cacheResponse構造新的response
Response response = networkResponse.newBuilder()
.cacheResponse(stripBody(cacheResponse))
.networkResponse(stripBody(networkResponse))
.build();
if (cache != null) {
if (HttpHeaders.hasBody(response) && CacheStrategy.isCacheable(response, networkRequest)) {
// 如果response符合緩存的策略需要緩存订雾，則put到cache中
// Offer this request to the cache.
// 這里追蹤到put中肢预，可以發(fā)現(xiàn)只有method為GET才會add到cache中，所以okhttp是只支持get請求的緩存的洼哎；且key為response.request().url()
CacheRequest cacheRequest = cache.put(response);
return cacheWritingResponse(cacheRequest, response);
}
if (HttpMethod.invalidatesCache(networkRequest.method())) {
try {
cache.remove(networkRequest);
} catch (IOException ignored) {
// The cache cannot be written.
}
}
}
return response;
}
大致流程如下
- 獲取本地緩存cacheCandidate
- 如果本地緩存可用則直接返回cacheCandidate烫映，從而打斷interceptor鏈
- 走剩下的interceptor獲取networkResponse
- networkResponse、cacheResponse構造新的response
-
根據(jù)新的response里的header定制緩存策略噩峦，存入緩存中
image.png
CacheStrategy
從上面的代碼來看锭沟，主要的緩存策略都是在這個類里實現(xiàn)。
我們關注這兩個變量,networkRequest為null就不走網(wǎng)絡取數(shù)據(jù)识补，cacheResponse為null則不用緩存
/** The request to send on the network, or null if this call doesn't use the network. */
public final @Nullable Request networkRequest;
/** The cached response to return or validate; or null if this call doesn't use a cache. */
public final @Nullable Response cacheResponse;
public CacheStrategy get() {
CacheStrategy candidate = getCandidate();
if (candidate.networkRequest != null && request.cacheControl().onlyIfCached()) {
// We're forbidden from using the network and the cache is insufficient.
return new CacheStrategy(null, null);
}
return candidate;
}
追蹤到public CacheStrategy get()方法
private CacheStrategy getCandidate() {
// No cached response.
if (cacheResponse == null) {
return new CacheStrategy(request, null);
}
// Drop the cached response if it's missing a required handshake.
// 請求為https且緩存沒有TLS握手
if (request.isHttps() && cacheResponse.handshake() == null) {
return new CacheStrategy(request, null);
}
// If this response shouldn't have been stored, it should never be used
// as a response source. This check should be redundant as long as the
// persistence store is well-behaved and the rules are constant.
// 跟進緩存Response的code族淮，response和request的cache-control的noStore字段判斷是否需要緩存
if (!isCacheable(cacheResponse, request)) {
return new CacheStrategy(request, null);
}
CacheControl requestCaching = request.cacheControl();
// 請求的header不要緩存
if (requestCaching.noCache() || hasConditions(request)) {
return new CacheStrategy(request, null);
}
long ageMillis = cacheResponseAge();
long freshMillis = computeFreshnessLifetime();
if (requestCaching.maxAgeSeconds() != -1) {
freshMillis = Math.min(freshMillis, SECONDS.toMillis(requestCaching.maxAgeSeconds()));
}
long minFreshMillis = 0;
if (requestCaching.minFreshSeconds() != -1) {
minFreshMillis = SECONDS.toMillis(requestCaching.minFreshSeconds());
}
long maxStaleMillis = 0;
CacheControl responseCaching = cacheResponse.cacheControl();
if (!responseCaching.mustRevalidate() && requestCaching.maxStaleSeconds() != -1) {
maxStaleMillis = SECONDS.toMillis(requestCaching.maxStaleSeconds());
}
if (!responseCaching.noCache() && ageMillis + minFreshMillis < freshMillis + maxStaleMillis) {
Response.Builder builder = cacheResponse.newBuilder();
if (ageMillis + minFreshMillis >= freshMillis) {
builder.addHeader("Warning", "110 HttpURLConnection \"Response is stale\"");
}
long oneDayMillis = 24 * 60 * 60 * 1000L;
if (ageMillis > oneDayMillis && isFreshnessLifetimeHeuristic()) {
builder.addHeader("Warning", "113 HttpURLConnection \"Heuristic expiration\"");
}
return new CacheStrategy(null, builder.build());
}
// Find a condition to add to the request. If the condition is satisfied, the response body
// will not be transmitted.
String conditionName;
String conditionValue;
if (etag != null) {
conditionName = "If-None-Match";
conditionValue = etag;
} else if (lastModified != null) {
conditionName = "If-Modified-Since";
conditionValue = lastModifiedString;
} else if (servedDate != null) {
conditionName = "If-Modified-Since";
conditionValue = servedDateString;
} else {
return new CacheStrategy(request, null); // No condition! Make a regular request.
}
Headers.Builder conditionalRequestHeaders = request.headers().newBuilder();
Internal.instance.addLenient(conditionalRequestHeaders, conditionName, conditionValue);
Request conditionalRequest = request.newBuilder()
.headers(conditionalRequestHeaders.build())
.build();
return new CacheStrategy(conditionalRequest, cacheResponse);
}
看這個方法大部分都是返回CacheStrategy(request, null)也就是走網(wǎng)絡，那么我們直接看唯一的返回緩存的代碼：return new CacheStrategy(null, builder.build());什么條件呢？
// ageMillis是response的maxAge時間和當前時間算出來的cache的有效時間瞧筛。。导盅。较幌。具體我也沒看明白哈
//response不是no-cache且(ageMillis+request的min-fresh時間)<(request的max-age時間+request的max-stale)
!responseCaching.noCache() && ageMillis + minFreshMillis < freshMillis + maxStaleMillis
總結下
- request的header有only-if-cached：啥緩存都不用
- 沒有緩存：當然不用緩存
- request為https且緩存丟失了TLS握手：不用緩存
- request或者response的header有no-store：不用緩存
- response除了200一些的status code以外：不用緩存
- 滿足這個條件ageMillis + minFreshMillis < freshMillis + maxStaleMillis的request和response：用緩存
- 其他的一些情況（我看暈了）
- 總之就是根據(jù)request和response的header的cache-control來做緩存，我們可以嚴格按照http協(xié)議的來做緩存策略白翻，而不用去看okhttp協(xié)議怎么實現(xiàn)的（嗯乍炉，okhttp應該是嚴格按照http協(xié)議來寫的吧？）
ConnectInterceptor,CallServerInterceptor
ConnectInterceptor
Opens a connection to the target server and proceeds to the next interceptor
關鍵類：StreamAllocation
CallServerInterceptor
This is the last interceptor in the chain. It makes a network call to the server.
最佳實踐
服務端控制緩存
- 客戶端請求時滤馍，header傳入想要的緩存時間策略岛琼，例如
@Headers("Cache-Control: no-cache")// 不要緩存
@Headers("Cache-Control: public, max-age=604800")//緩存時間為604800秒
- 服務端指定緩存策略词裤，返回相應的response Cache-Control
然而很不幸惕澎，大部分的服務端都沒有返回Cache-Control來控制緩存，所以就有了下面的辦法
客戶端控制緩存時間
- 客戶端傳入header
@Headers("Cache-Control: public, max-age=30")//緩存時間為30秒 - 添加networkInterceptor
@Override public Response intercept(Chain chain) throws IOException { Request request = chain.request(); Response originalResponse = chain.proceed(request); if (TextUtils.isEmpty(originalResponse.header("Cache-Control"))) { // 這里把request傳入的header傳遞給response return originalResponse.newBuilder().header("Cache-Control", request.header("Cache-Control")).build(); } return originalResponse; }
客戶端控制緩存時間崎页，同時要求無網(wǎng)絡的時候使用緩存
有網(wǎng)絡的時候同上阁苞；無網(wǎng)絡的時候困檩，如果超過一天則顯示error，沒超過一天用緩存
- 客戶端傳入header同上
- networkInterceptor同上
- 添加applicationInterceptor,傳入一個max-age為無限大數(shù)的header就能強制用緩存了那槽，或者設置Cache-Control的FORCE_CACHE
@Override public Response intercept(Chain chain) throws IOException { Request request = chain.request(); CacheControl cacheControl = request.cacheControl(); boolean noCache = cacheControl.noCache() || cacheControl.noStore() || cacheControl.maxAgeSeconds() == 0; // 如果header強制要求不用緩存就不走這個邏輯 if (!noCache && !NetworkUtils.isNetworkAvailable(context)) { Request.Builder builder = request.newBuilder(); //if network not available, load in cache CacheControl newCacheControl = new CacheControl.Builder() .maxAge(Integer.MAX_VALUE, TimeUnit.SECONDS).build(); request = builder.cacheControl(newCacheControl).build(); return chain.proceed(request); } return chain.proceed(request); }
客戶端控制緩存時間悼沿，同時要求無網(wǎng)絡的時候使用緩存，并且這個緩存超過一天就失效了
基本同上骚灸，但是傳入的header不是maxAge而是max-stale糟趾，設置緩存過期后還能可用的時間為一天即可
CacheControl newCacheControl = new CacheControl.Builder().maxStale(ONE_DAY, TimeUnit.SECONDS).build();
客戶端控制緩存時間,request的時間和response的時間不同
前面的幾個方式，response的header實際上是從request取出來的甚牲，也就是說我們的response的header時間和request的header時間是一樣的义郑。但是如果要不一樣的情況怎么辦呢？舉個我項目里的例子
- 發(fā)送A請求（緩存為30分鐘）鳖藕，發(fā)現(xiàn)這個商品沒有買
- 花錢把這個商品買了魔慷，再次請求A請求刷新頁面
- 因為A請求有30分鐘緩存沒有刷新數(shù)據(jù)；于是乎我修改了request的header為不使用緩存(也就是age為0),這時數(shù)據(jù)刷新了
- 幾分鐘后著恩，我下次進來這個頁面院尔，再次請求A（因為之前age為0，所以并沒有緩存）喉誊，我又發(fā)了次請求（實際我期望的是使用緩存的）
實際上我希望的是在步驟3里發(fā)送A請求時邀摆，request的header為age=0，response的age=30min伍茄，那么怎么實現(xiàn)呢栋盹，所以提供了下面的方法
首先提供了一個工具類，用來存放header的時間和生成header敷矫。這里用ThreadLocal變量存放了response的時間
public final class NetAccessStrategy {
private NetAccessStrategy() {
}
private static final ThreadLocal<Integer> localCacheTime = new ThreadLocal<>();
public static void setThreadLocalCacheTime(int cacheTime) {
localCacheTime.set(cacheTime);
}
public static int getThreadLocalCacheTime() {
Integer time = localCacheTime.get();
localCacheTime.remove();
if (time == null) {
return 0;
}
return time;
}
public static final String NET_REQUEST = "net-";
/**
* @param requestCacheTime 本地緩存在超過這個時間后失效
* @param localCacheTime 本地緩存的時間
* @return
*/
public static String getRequestNetHeader(int requestCacheTime, int localCacheTime) {
return NET_REQUEST + requestCacheTime + "-" + +localCacheTime;
}
public static int[] getRequestCacheTime(String netHeader) {
int index1 = netHeader.indexOf("-", 1);
int index2 = netHeader.indexOf("-", index1 + 1);
int time1 = -1;
int time2 = -1;
if (index1 != -1 && index2 != -1) {
try {
time1 = Integer.parseInt(netHeader.substring(index1 + 1, index2));
} catch (NumberFormatException ignored) {
}
try {
time2 = Integer.parseInt(netHeader.substring(index2 + 1));
} catch (NumberFormatException ignored) {
}
}
return new int[]{time1, time2};
}
}
在application Interceptor里加上
// 如果發(fā)現(xiàn)net-開頭的自定義header時
if (header.startsWith(NetAccessStrategy.NET_REQUEST)) {
Request.Builder builder = request.newBuilder();
// 解析得到request和response的時間
int[] timeArray = NetAccessStrategy.getRequestCacheTime(header);
// 傳入request的age時間
CacheControl cacheControl = new CacheControl.Builder().maxAge(timeArray[0], TimeUnit.SECONDS).build();
// 存入response的時間
NetAccessStrategy.setThreadLocalCacheTime(timeArray[1]);
builder.cacheControl(cacheControl);
return chain.proceed(builder.build());
}
在network Interceptor里加上
Response originalResponse = chain.proceed(request);
int time = NetAccessStrategy.getThreadLocalCacheTime();
if (time > 0) {
// 取出response time例获，如果大于0則放到header里
return originalResponse.newBuilder().header("Cache-Control", "public, max-age=" + time)
.build();
}
return originalResponse.newBuilder().header("Cache-Control", request.header("Cache-Control"))
.build();
發(fā)請求時加上header汉额，這樣就能實現(xiàn)強制刷新，且緩存為300秒的功能了
NetAccessStrategy.getRequestNetHeader(0, 300)
我的主頁
PS：https://github.com/nppp1990榨汤，平常隨便寫寫蠕搜，還有kotlin的demo
