原理

Cache 仿佛一直以來都是提高性能, 特別是I/O密集型服務提高性能的法寶

參見下面一組數字, 比較不同媒介的存取速度: 緩存 > 內存 > 網絡 > 磁盤
(來自 Google Fellow Jeff Dean 的 PPT )

• 每個人都應知道的一組數字 :

從上面的數據, 我們知道數據緩存在 CPU cache 和內存中存取是最快的, 磁盤最慢, 順序為

1. 本地內存中的緩存數據
2. 本地磁盤中的數據
3. 遠程快速節(jié)點內存中的緩存數據
4. 遠程快速節(jié)點磁盤中的數據
5. 遠程慢速節(jié)點內存中的緩存數據
6. 遠程慢速節(jié)點磁盤中的數據

第2, 第3取決于網絡和磁盤的存取速度, 順序可能會對調

遠程節(jié)點分兩種：
一種是在低延遲網絡中的遠程節(jié)點濒持，或者說同一個局域網或城域網中的比較近的節(jié)點补鼻，又或雖然相隔距離遠但是通過專用高速網絡互連的節(jié)點粤剧，RTT一般在50ms 以內

一種是在高延遲網絡中的遠程節(jié)點，比如跨地區(qū)终吼，跨網絡，RTT大于100ms的網絡，有時候，同一個城市不同的網絡運營商之前的網絡速度可能比不同城市同一個運營商之前速度更慢

Cache 的種類

• 本地 Cache: 放在本地內存中的緩存, 典型代表就是內存中的 map
• 遠程 Cache: 放在遠程服務器內存中的緩存, 比如 memcached, redis cache
• 分布式 Cache: 它其實有兩種, 一種全部是遠程分布式 cache , 還有一種是本地和遠程cache 有同步機制, 存取都是在本地內存中

其實緩存無處不在, CPU 有L1/L2 緩存, 硬盤有讀寫緩存, 這里僅提及微服務常用的緩存

對于Cache 我們最關心的就是 Cache 的使用效率, 也就是命中率, 提高命中率的關鍵因素是

1. Cache key size 緩存的鍵值數量
2. Cache capacity 緩存的容量
3. Cache lifetime 緩存的生命周期

Cache 不可能無限增長, 不可能永遠有效, 所以對于 Cache 的清除策略和失效策略要細細考量.
對于放在 Cache 中的數據也最好是讀寫比較高的, 即讀得多, 寫得少, 不會頻繁地更新.

Cache的常用操作通常有

• get： 獲取
• put： 設置
• remove： 移除
• clear： 清除
• expire: 過期

對于Cluster Cache來說洼专，讀操作（get）肯定是本地方法，只需要從本臺計算機內存中獲取數據孵构。

Remove/clear 這兩個操作屁商，一般是本地方法加上遠程方法，需要和Cluster其他計算機進行同步颈墅。

Put這個寫方法蜡镶，可以是本地操作，也可以是遠程操作的恤筛。 Remote Put方法的場景是這樣官还，一臺計算機把數據放到Cache里面，這個數據就會被傳播到Cluster其他計算機上毒坛。這個做法的好處是Cluster各臺計算機的Cache數據可以及時得到補充妻枕，壞處是傳播的數據量比較大，這個代價比較大粘驰。
本地 Local Put方法的場景是這樣，一臺計算機把數據放到Cache里面述么，這個數據不會被傳播到Cluster其他計算機上蝌数。這個做法的好處是不需要傳播數據，壞處是Cluster各臺計算機的Cache數據不能及時得到補充度秘，這個不是很明顯的問題顶伞，從Cache中得不到數據，從數據庫獲取數據是很正常的現象剑梳。 Local Put比起Remote Put的優(yōu)勢很明顯唆貌，所以，通常的Cluster Cache都采用Local Put的策略垢乙。各Cache一般都提供了Local Put的配置選項锨咙，如果你沒有看到這個支持，那么請換一個Cache追逮。

通常有很多不同的 cache 實現酪刀，例如

• Guava Cache
• EhCache
• Memcached
• Redis

這里我們先重點講講 Guava Cache

最簡單的cache 實現莫過于在內存中維護一張 map 了, 按 key 檢索或存儲 , 不過內存畢竟有限, 要自己實現 Cache 和各種清除和失效策略, Guava Cache 是不錯的選擇, 可以很方便地設置最大容量, 清除和失效策略等等.

它的主要特性有:

• 自動加載數據項到緩存中
• 當最大限度到達時應用最近最少策略驅逐數據項
• 從上次的訪問或修改時間開始計算的數據項過期
• 數據項的鍵自動封裝于弱引用中
• 數據項的值自動封裝于弱引用或軟引用中
• 當數據項被驅逐或清除時通知
• 聚合緩存訪問統(tǒng)計

我們來舉一個天氣預報數據的緩存例子, 天氣預報是那種比較適合緩存的數據, 它在一定的時間范圍內變化不大, 讀寫比很高.

各大廠商紛紛推出各種 open API , 免費的大多有調用次數限制，例如百度的天氣API, 一天之內的限制是 5000條/天钮孵，并發(fā)上限是 200QPS骂倘， 一般使用用也夠了，可是如果不是一般情況巴席，使用緩存就可以輕松搞定

百度應用控制臺 http://lbs.baidu.com/apiconsole/key#/home

baidu application console

接口示例

接口參數說明

返回結果

完整代碼參見 https://github.com/walterfan/helloworld/tree/master/hellocache

主要界面如下：

主要代碼如下：

• WeatherApplication

package com.github.walterfan.hellocache;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class WeatherApplication {

    public static void main(String[] args) {
        SpringApplication.run(WeatherApplication.class, args);
    }
}

• WeatherController

package com.github.walterfan.hellocache;

import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Controller;
import org.springframework.stereotype.Service;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.servlet.ModelAndView;

import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.time.Instant;
import java.util.Map;
import java.util.Optional;

/**
 * Created by yafan on 8/10/2017.
 */
@Slf4j
@Controller
public class WeatherController {
    @Autowired
    private WeatherService weatherService;
    // inject via application.properties
    @Value("${hello.message:test}")
    private String message = "Hello World";

    @RequestMapping("/")
    public String index(Model model) {
        model.addAttribute("cityName", "hefei");
        model.addAttribute("currentTime", Instant.now().toString());
        return "index";
    }



    @GetMapping("/weather")
    protected String queryWeather(Model model, HttpServletRequest req, HttpServletResponse resp) {


        String cityName = req.getParameter("cityName");
        model.addAttribute("cityName", cityName);
        Optional<CityWeather> cityWeather =  weatherService.getWeather(cityName);
        if(cityWeather.isPresent()) {
            log.info("weather: {}", cityWeather.get());
            model.addAttribute("cityWeather", cityWeather.get());
            model.addAttribute("weatherString", cityWeather.map(x->x.toString()).orElse(""));
        }
        return "index";
    }



}

• WeatherService.java

package com.github.walterfan.hellocache;

import com.codahale.metrics.MetricRegistry;
import com.google.common.cache.LoadingCache;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import java.util.Optional;
import java.util.concurrent.ExecutionException;


@Slf4j
@Service
public class WeatherService {
    @Autowired
    private LoadingCache<String, CityWeather> cityWeatherCache;

    @Autowired
    private MetricRegistry metricRegistry;


    public Optional<CityWeather> getWeather(String city) {
        try {
            return Optional.ofNullable(cityWeatherCache.get(city));
        } catch (ExecutionException e) {
            log.error("getWeather error", e);
            return Optional.empty();
        }
    }
}

• WeatherCacheConfig.java 構建 Weather Cache 和相關依賴

package com.github.walterfan.hellocache;


import com.codahale.metrics.MetricRegistry;
import com.fasterxml.jackson.databind.ObjectMapper;

import com.google.common.cache.Cache;
import com.google.common.cache.CacheBuilder;
import com.google.common.cache.LoadingCache;
import com.google.common.collect.ImmutableList;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.EnableAspectJAutoProxy;
import org.springframework.context.annotation.Lazy;
import org.springframework.core.env.Environment;
import org.springframework.http.MediaType;
import org.springframework.http.converter.HttpMessageConverter;
import org.springframework.http.converter.json.MappingJackson2HttpMessageConverter;
import org.springframework.web.client.RestTemplate;

import java.util.List;
import java.util.concurrent.TimeUnit;

/**
 * Created by yafan on 14/10/2017.
 */
@EnableAspectJAutoProxy
@ComponentScan
@Configuration
@Slf4j
public class WeatherCacheConfig {//implements EnvironmentAware

    @Autowired
    private Environment environment;

    @Bean
    public WeatherCacheLoader weatherCacheLoader() {
        return new WeatherCacheLoader();
    }

    @Bean
    public RestTemplate restTemplate() {
        RestTemplate restTemplate = new RestTemplate();

        List<HttpMessageConverter<?>> converters = restTemplate.getMessageConverters();
        for (HttpMessageConverter<?> converter : converters) {
            if (converter instanceof MappingJackson2HttpMessageConverter) {
                MappingJackson2HttpMessageConverter jsonConverter = (MappingJackson2HttpMessageConverter) converter;
                jsonConverter.setObjectMapper(new ObjectMapper());
                jsonConverter.setSupportedMediaTypes(ImmutableList.of(
                        new MediaType("application", "json", MappingJackson2HttpMessageConverter.DEFAULT_CHARSET),
                        new MediaType("text", "javascript", MappingJackson2HttpMessageConverter.DEFAULT_CHARSET)));
            }
        }
        return restTemplate;
    }

    @Bean
    public String appToken() {
        return this.environment.getProperty("BAIDU_AK");
    }


    @Bean
    public LoadingCache<String, CityWeather> cityWeatherCache() {
        LoadingCache<String, CityWeather> cache = CacheBuilder.newBuilder()
                .recordStats()
                .maximumSize(1000)
                .expireAfterWrite(60, TimeUnit.MINUTES)
                .build(weatherCacheLoader());

        recordCacheMetrics("cityWeatherCache", cache);
        return cache;
    }

    public void recordCacheMetrics(String cacheName, Cache cache) {
        MetricRegistry metricRegistry = metricRegistry();
        metricRegistry.gauge(generateMetricsKeyForCache(cacheName, "hitCount"), () -> () -> cache.stats().hitCount());
        metricRegistry.gauge(generateMetricsKeyForCache(cacheName, "hitRate"), () -> () -> cache.stats().hitRate());

        metricRegistry.gauge(generateMetricsKeyForCache(cacheName, "missCount"), () -> () -> cache.stats().missCount());
        metricRegistry.gauge(generateMetricsKeyForCache(cacheName, "missRate"), () -> () -> cache.stats().missRate());

        metricRegistry.gauge(generateMetricsKeyForCache(cacheName, "requestCount"), () -> () -> cache.stats().requestCount());

        metricRegistry.gauge(generateMetricsKeyForCache(cacheName, "loadCount"), () -> () -> cache.stats().loadCount());
        metricRegistry.gauge(generateMetricsKeyForCache(cacheName, "loadSuccessCount"), () -> () -> cache.stats().loadSuccessCount());
        metricRegistry.gauge(generateMetricsKeyForCache(cacheName, "loadExceptionCount"), () -> () -> cache.stats().loadExceptionCount());
    }

    public String generateMetricsKeyForCache(String cacheName, String keyName) {
        String metricKey = MetricRegistry.name("cache", cacheName, keyName);
        log.info("metric key generated for cache: {}", metricKey);
        return metricKey;
    }

    @Bean
    public DurationTimerAspect durationTimerAspect() {
        return new DurationTimerAspect();
    }

    @Bean
    @Lazy
    public MetricRegistry metricRegistry() {
        return new MetricRegistry();
    }
}

• WeatherCacheLoader 從遠程的 Baidu API 獲取天氣預報數據放入緩存中
  同時把 main 函數放在這里, 也就是構建 Spring Application Context, 并從cache 中獲取天氣預報數據
  第一次花的時間比較長, 用了352 毫秒, 之后從cache 中獲取數據, 都只了幾十到幾百微秒
  注意對于百度天氣API, 你需要自己申請一個apptoken, 我申請了一個, 5000次/天的調用都免費的, 由于我用了一小時的緩存, 所以無需付出任何費用

package com.github.walterfan.hellocache;

import com.codahale.metrics.Histogram;
import com.codahale.metrics.MetricRegistry;
import com.codahale.metrics.Snapshot;


import com.google.common.cache.CacheLoader;
import com.google.common.cache.LoadingCache;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Component;
import org.springframework.web.client.RestTemplate;
import org.springframework.web.util.UriComponentsBuilder;


import java.util.Map;
import java.util.SortedMap;
import java.util.concurrent.ExecutionException;

/**
 * Created by yafan on 11/10/2017.
 *
 * refer to api http://lbsyun.baidu.com/index.php?title=car/api/weather
 */

@Slf4j
@Component
public class WeatherCacheLoader extends CacheLoader<String, CityWeather> {


    @Autowired
    private RestTemplate restTemplate;

    @Autowired
    private String appToken;

    @Autowired
    private LoadingCache<String, CityWeather> cityWeatherCache;

    @Autowired
    private MetricRegistry metricRegistry;

    @DurationTimer(name="getCityWeather")
    public CityWeather getCityWeather(String city) throws ExecutionException {
        return this.cityWeatherCache.get(city);
    }


    @Override
    public CityWeather load(String city) throws Exception {

        String url = "http://api.map.baidu.com/telematics/v3/weather";
        UriComponentsBuilder builder = UriComponentsBuilder.fromHttpUrl(url)
                .queryParam("location", city)
                .queryParam("output", "json")
                .queryParam("ak", appToken);

        ResponseEntity<CityWeather> resp = restTemplate.getForEntity(builder.toUriString(), CityWeather.class);

        log.debug("response status: " + resp.getStatusCode());
        return resp.getBody();
    }

    public static void main(String[] args) throws ExecutionException {
        try (AnnotationConfigApplicationContext context = new AnnotationConfigApplicationContext(WeatherCacheConfig.class)) {
            WeatherCacheLoader helloCacheLoder = (WeatherCacheLoader) context.getBean("WeatherCacheLoader");

            CityWeather cityWeather = helloCacheLoder.getCityWeather("hefei");
            for(int i=0;i<10;++i) {
                cityWeather = helloCacheLoder.getCityWeather("hefei");
            }

            log.info("----- weather -----");
            log.info(cityWeather.toString());

            MetricRegistry metricRegistry = (MetricRegistry) context.getBean("metricRegistry");


            SortedMap<String, Histogram> histograms =  metricRegistry.getHistograms();

            for(Map.Entry<String, Histogram> entry: histograms.entrySet()) {
                Snapshot snapshot = entry.getValue().getSnapshot();
                log.info("{}: size={},values: {}",  entry.getKey(), snapshot.size(), snapshot.getValues());
                log.info(" max={}, min={}, mean={}, median={}",
                        snapshot.getMax(), snapshot.getMin(), snapshot.getMean(), snapshot.getMedian());
            }
        }
    }


}

• output

13:02:03.273 [main] INFO com.github.walterfan.hello.DurationTimerAspect - Duration of getCityWeather: 352404 MICROSECONDS, threshold: 0 MICROSECONDS
13:02:03.274 [main] INFO com.github.walterfan.hello.DurationTimerAspect - Duration of getCityWeather: 251 MICROSECONDS, threshold: 0 MICROSECONDS
13:02:03.274 [main] INFO com.github.walterfan.hello.DurationTimerAspect - Duration of getCityWeather: 127 MICROSECONDS, threshold: 0 MICROSECONDS
13:02:03.274 [main] INFO com.github.walterfan.hello.DurationTimerAspect - Duration of getCityWeather: 87 MICROSECONDS, threshold: 0 MICROSECONDS
...
13:02:03.309 [main] INFO com.github.walterfan.hello.HelloCacheLoader - getCityWeather: size=11, count=[34, 36, 40, 45, 60, 63, 85, 87, 127, 251, 352404],values: {}
13:02:03.309 [main] INFO com.github.walterfan.hello.HelloCacheLoader -  max=352404, min=34, mean=32112.0, median=63.0

這里順手寫了兩個 Metrics 相關的輔助類, 利用 AOP 和 Metrics 來記錄調用時間

• class DurationTimer

ackage com.github.walterfan.hello;

import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
import java.util.concurrent.TimeUnit;

/**
 * Created by yafan on 15/10/2017.
 */
@Target({ElementType.METHOD})
@Retention(value = RetentionPolicy.RUNTIME)
@Documented
public @interface DurationTimer {

    String name() default "";

    long logThreshold() default 0;

    //default timeunit μs
    TimeUnit thresholdTimeUnit() default TimeUnit.MICROSECONDS;
}

• class DurationTimerAspect

package com.github.walterfan.hello;

import com.codahale.metrics.MetricRegistry;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.aop.aspectj.annotation.AspectJProxyFactory;
import org.springframework.beans.factory.annotation.Autowired;

import java.util.concurrent.TimeUnit;

/**
 * Created by yafan on 15/10/2017.
 */
@Aspect
public class DurationTimerAspect {


    private final Logger logger = LoggerFactory.getLogger(getClass());

    @Autowired
    private MetricRegistry metricRegistry;

    public <T> T proxy(T o) {
        final AspectJProxyFactory factory = new AspectJProxyFactory(o);
        factory.setProxyTargetClass(true);
        factory.addAspect(this);
        return factory.getProxy();
    }

    @Around("@annotation( durationAnnotation ) ")
    public Object measureTimeRequest(final ProceedingJoinPoint pjp, DurationTimer durationAnnotation) throws Throwable {
        final long start = System.nanoTime();
        final Object retVal = pjp.proceed();

        String timerName = durationAnnotation.name();
        if("".equals(timerName)) {
            timerName = pjp.getSignature().toShortString();
        }
        TimeUnit timeUnit = durationAnnotation.thresholdTimeUnit();
        long threshold = durationAnnotation.logThreshold();
        //System.out.println("timerName=" + timerName);
        try {
            long difference = timeUnit.convert(System.nanoTime() - start, TimeUnit.NANOSECONDS);

            if(difference > threshold) {
                metricRegistry.histogram(timerName).update(difference);
                logger.info("Duration of {}: {} {}, threshold: {} {}", timerName, difference, timeUnit.name(), threshold, timeUnit.name());
            }

        } catch (Exception ex) {
            logger.error("Cannot measure api timing.... :" + ex.getMessage(), ex);
        }
        return retVal;
    }

}

相關的 DTO 如下

package com.github.walterfan.dto;

import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;

import java.util.Date;
import java.util.List;

/**
 * Created by yafan on 14/10/2017.
 */
@JsonIgnoreProperties(ignoreUnknown = true)
public class CityWeather  {
    private int error;

    private String status;

    private Date date;

    private List<WeatherResult> results;

    public int getError() {
        return error;
    }

    public void setError(int error) {
        this.error = error;
    }

    public String getStatus() {
        return status;
    }

    public void setStatus(String status) {
        this.status = status;
    }

    public Date getDate() {
        return date;
    }

    public void setDate(Date date) {
        this.date = date;
    }

    public List<WeatherResult> getResults() {
        return results;
    }

    public void setResults(List<WeatherResult> results) {
        this.results = results;
    }

    @Override
    public String toString() {
        try {
            return new ObjectMapper().writerWithDefaultPrettyPrinter().writeValueAsString(this);
        } catch (JsonProcessingException e) {
            e.printStackTrace();
        }
        return null;
    }
}


//---------------------
package com.github.walterfan.dto;

/**
 * Created by yafan on 14/10/2017.
 */
public class WeatherData {

    private String date;
    private String dayPictureUrl;
    private String nightPictureUrl;
    private String weather;
    private String wind;
    private String temperature;

    public String getDate() {
        return date;
    }

    public void setDate(String date) {
        this.date = date;
    }

    public String getDayPictureUrl() {
        return dayPictureUrl;
    }

    public void setDayPictureUrl(String dayPictureUrl) {
        this.dayPictureUrl = dayPictureUrl;
    }

    public String getNightPictureUrl() {
        return nightPictureUrl;
    }

    public void setNightPictureUrl(String nightPictureUrl) {
        this.nightPictureUrl = nightPictureUrl;
    }

    public String getWeather() {
        return weather;
    }

    public void setWeather(String weather) {
        this.weather = weather;
    }

    public String getWind() {
        return wind;
    }

    public void setWind(String wind) {
        this.wind = wind;
    }

    public String getTemperature() {
        return temperature;
    }

    public void setTemperature(String temperature) {
        this.temperature = temperature;
    }
}
//------------------
package com.github.walterfan.dto;

/**
 * Created by yafan on 14/10/2017.
 */
public class WeatherIndex {
    private String title;
    private String zs;
    private String tipt;
    private String des;

    public String getTitle() {
        return title;
    }

    public void setTitle(String title) {
        this.title = title;
    }

    public String getZs() {
        return zs;
    }

    public void setZs(String zs) {
        this.zs = zs;
    }

    public String getTipt() {
        return tipt;
    }

    public void setTipt(String tipt) {
        this.tipt = tipt;
    }

    public String getDes() {
        return des;
    }

    public void setDes(String des) {
        this.des = des;
    }
}
//---------------------
package com.github.walterfan.dto;

import java.util.List;

/**
 * Created by yafan on 14/10/2017.
 */
public class WeatherResult {
    private String currentCity;
    private String pm25;

    private List<WeatherIndex> index;

    private List<WeatherData> weather_data;

    public String getCurrentCity() {
        return currentCity;
    }

    public void setCurrentCity(String currentCity) {
        this.currentCity = currentCity;
    }

    public String getPm25() {
        return pm25;
    }

    public void setPm25(String pm25) {
        this.pm25 = pm25;
    }

    public List<WeatherIndex> getIndex() {
        return index;
    }

    public void setIndex(List<WeatherIndex> index) {
        this.index = index;
    }

    public List<WeatherData> getWeather_data() {
        return weather_data;
    }

    public void setWeather_data(List<WeatherData> weather_data) {
        this.weather_data = weather_data;
    }
}

參考資料

• https://github.com/google/guava/wiki/CachesExplained