本文檔第一部分介紹一下 Envoy v2 API叠洗，第二部分給出了一個(gè) Java 編寫(xiě)的簡(jiǎn)陋 EDS Server 示例。

Envoy v2 API

Envoy 的 API 有 v1 和 v2 兩個(gè)版本恰梢，目前主流版本是 v2。它具有以下特點(diǎn)：

• 通過(guò) gRPC 流式傳輸對(duì) xDS API 的更新，這減少了資源的需求并且可以降低更新延遲。
• 一種新的 REST-JSON API祭钉。其中 JSON/YAML 格式是通過(guò) proto3 規(guī)范的 JSON 映射機(jī)制派生出來(lái)的瞄沙。
• 通過(guò)文件系統(tǒng)己沛、REST-JSON 或 gRPC 端點(diǎn)傳遞更新。
• 通過(guò)擴(kuò)展端點(diǎn)分配 API 進(jìn)行高級(jí)負(fù)載均衡距境，并向管理服務(wù)器報(bào)告負(fù)載以及資源的利用率申尼。
• 當(dāng)需要更強(qiáng)的一致性和排序?qū)傩詴r(shí)，Envoy v2 API 仍然可以保持基準(zhǔn)最終一致性模型垫桂。

Envoy 的 Bootstrap 配置可以采用 完全靜態(tài)师幕、部分靜態(tài)或者完全動(dòng)態(tài)。

靜態(tài)配置

下面提供了一個(gè)完全靜態(tài)引導(dǎo)配置的例子：

admin:
  access_log_path: /tmp/admin_access.log
  address:
    socket_address: { address: 127.0.0.1, port_value: 9901 }

static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address: { address: 127.0.0.1, port_value: 10000 }
    filter_chains:
    - filters:
      - name: envoy.http_connection_manager
        config:
          stat_prefix: ingress_http
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service
              domains: ["example.com"]
              routes:
              - match: { prefix: "/" }
                route: { cluster: some_service }
          http_filters:
          - name: envoy.router
  clusters:
  - name: some_service
    connect_timeout: 0.25s
    type: STATIC
    lb_policy: ROUND_ROBIN
    hosts: [{ socket_address: { address: 127.0.0.2, port_value: 1234 }}]

在上面的例子中诬滩，我們配置了一個(gè) envoy實(shí)例霹粥，監(jiān)聽(tīng) 127.0.0.1:10000，支持 http 協(xié)議訪問(wèn)疼鸟，http 訪問(wèn)域名為：http://example.com后控。接收到的所有http流量，轉(zhuǎn)發(fā)給 127.0.0.2:1234 的服務(wù)空镜。這個(gè)例子中 some_service 這個(gè)cluster中 hosts 是固定的(127.0.0.2:1234)浩淘，不利于擴(kuò)展。

EDS為動(dòng)態(tài)的配置

將配置 envoy 的配置調(diào)整如下吴攒。

admin:
  access_log_path: /tmp/admin_access.log
  address:
    socket_address: { address: 127.0.0.1, port_value: 9901 }

static_resources:
  listeners:
  - name: listener_0
    address:
      socket_address: { address: 127.0.0.1, port_value: 10000 }
    filter_chains:
    - filters:
      - name: envoy.http_connection_manager
        config:
          stat_prefix: ingress_http
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service
              domains: ["example.com"]
              routes:
              - match: { prefix: "/" }
                route: { cluster: some_service }
          http_filters:
          - name: envoy.router
  clusters:
  - name: some_service
    connect_timeout: 0.25s
    lb_policy: ROUND_ROBIN
    type: EDS
    eds_cluster_config:
      eds_config:
        api_config_source:
          api_type: GRPC
          cluster_names: [xds_cluster]
  - name: xds_cluster
    connect_timeout: 0.25s
    type: STATIC
    lb_policy: ROUND_ROBIN
    http2_protocol_options: {}
    hosts: [{ socket_address: { address: 127.0.0.3, port_value: 5678 }}]

這樣就實(shí)現(xiàn) some_service 這個(gè) cluster 的 hosts 的動(dòng)態(tài)配置了张抄。新的配置中，some_service 這個(gè) cluster 的 hosts 是 EDS(Endpoint Discovery Service) 的返回值決定的洼怔，就是說(shuō) EDS 會(huì)返回 some_service 這個(gè) cluster 的 hosts 的列表署惯。新配置中，EDS 服務(wù)的地址定義在 xds_cluster 這個(gè) cluster中镣隶。

EDS 服務(wù)的地址是：127.0.0.3:5678极谊，會(huì)返回 proto3 編碼的響應(yīng)格式如下：

version_info: "0"
resources:
- "@type": type.googleapis.com/envoy.api.v2.ClusterLoadAssignment
  cluster_name: some_service
  endpoints:
  - lb_endpoints:
    - endpoint:
        address:
          socket_address:
            address: 127.0.0.2
            port_value: 1234

基于xDS的動(dòng)態(tài)配置

下面提供了完全動(dòng)態(tài)的 bootstrap 配置，其中屬于管理服務(wù)器的所有資源都是通過(guò) xDS 發(fā)現(xiàn)的矾缓。

admin:
  access_log_path: /tmp/admin_access.log
  address:
    socket_address: { address: 127.0.0.1, port_value: 9901 }

dynamic_resources:
  lds_config:
    api_config_source:
      api_type: GRPC
      cluster_names: [xds_cluster]
  cds_config:
    api_config_source:
      api_type: GRPC
      cluster_names: [xds_cluster]

static_resources:
  clusters:
  - name: xds_cluster
    connect_timeout: 0.25s
    type: STATIC
    lb_policy: ROUND_ROBIN
    http2_protocol_options: {}
    hosts: [{ socket_address: { address: 127.0.0.3, port_value: 5678 }}]

這里我們假設(shè) 127.0.0.3:5678 提供完整的 xDS怀酷。

LDS服務(wù)的響應(yīng)格式如下：

version_info: "0"
resources:
- "@type": type.googleapis.com/envoy.api.v2.Listener
  name: listener_0
  address:
    socket_address:
      address: 127.0.0.1
      port_value: 10000
  filter_chains:
  - filters:
    - name: envoy.http_connection_manager
      config:
        stat_prefix: ingress_http
        codec_type: AUTO
        rds:
          route_config_name: local_route
          config_source:
            api_config_source:
              api_type: GRPC
              cluster_names: [xds_cluster]
        http_filters:
        - name: envoy.router

RDS 服務(wù)的響應(yīng)格式如下:

version_info: "0"
resources:
- "@type": type.googleapis.com/envoy.api.v2.RouteConfiguration
  name: local_route
  virtual_hosts:
  - name: local_service
    domains: ["*"]
    routes:
    - match: { prefix: "/" }
      route: { cluster: some_service }

CDS 服務(wù)的響應(yīng)如下:

version_info: "0"
resources:
- "@type": type.googleapis.com/envoy.api.v2.Cluster
  name: some_service
  connect_timeout: 0.25s
  lb_policy: ROUND_ROBIN
  type: EDS
  eds_cluster_config:
    eds_config:
      api_config_source:
        api_type: GRPC
        cluster_names: [xds_cluster]

EDS 服務(wù)的響應(yīng)如下:

version_info: "0"
resources:
- "@type": type.googleapis.com/envoy.api.v2.ClusterLoadAssignment
  cluster_name: some_service
  endpoints:
  - lb_endpoints:
    - endpoint:
        address:
          socket_address:
            address: 127.0.0.2
            port_value: 1234

Envoy EDS實(shí)戰(zhàn)示例

下面給出一個(gè)使用 Java 自行實(shí)現(xiàn) EDS服務(wù)的示例，這里我們使用了 REST 接口嗜闻。

實(shí)現(xiàn) EDS-Server

首先創(chuàng)建一個(gè) SpringBoot Web項(xiàng)目蜕依。實(shí)現(xiàn) EDS 比較復(fù)雜的部分就是響應(yīng)報(bào)文和請(qǐng)求報(bào)文的示例實(shí)現(xiàn)。

先看下請(qǐng)求報(bào)文(省略了setter 和 getter 部分)。

public class DiscoveryRequest {

    private DiscoveryNode node;
    @JsonProperty("resource_names")
    private List<String> resourceNames;

    @Override
    public String toString() {
        return "node = { " + this.node + "}, resource_names = " + this.resourceNames;
    }
}

public class DiscoveryNode {
    @JsonProperty("build_version")
    private String buildVersion;
    private String cluster;
    private String id;   

    @Override
    public String toString() {
        return "build_version = " + this.buildVersion + ", cluster = " + this.cluster +
                ", id = " + this.id;
    }
}

這樣生成的請(qǐng)求報(bào)文類(lèi)格式類(lèi)似下面這個(gè)樣子：

{
    "node": {
        "cluster": "myCluster",
        "id": "test-id",
        "build_version": "44f8c365a1f1798f0af776f6aa64279dc68f5666/1.12.1/Clean/RELEASE/BoringSSL"
    },
    "resource_names": [
        "myservice"
    ]
}

請(qǐng)求報(bào)文格式比較簡(jiǎn)單样眠，下面看下格式較為復(fù)雜的返回報(bào)文（依舊省略 setter 和 getter）友瘤。

public class DiscoveryResponse {

    @JsonProperty("version_info")
    private String versionInfo;
    private List<ResponseResource> resources = new ArrayList<>();

}

public class ResponseResource {

    @JsonProperty("@type")
    private String type;
    @JsonProperty("cluster_name")
    private String clusterName;
    @JsonProperty("endpoints")
    private List<EndPoints> endPoints = new ArrayList<>();

}

public class EndPoints {

    @JsonProperty("lb_endpoints")
    private List<LbEndPoint> lbEndPoints;

public class LbEndPoint {

    @JsonProperty("endpoint")
    private EndPoint endPonit;

public class EndPoint {

    private DiscoveryAddress address;

}

public class DiscoveryAddress {

    @JsonProperty("socket_address")
    private SocketAddress socketAddress;

    public DiscoveryAddress() {

    }

    public DiscoveryAddress(SocketAddress socketAddress) {
        this.socketAddress = socketAddress;
    }

}

public class SocketAddress {

    private String address;
    @JsonProperty("port_value")
    private int port;

    public SocketAddress() {

    }

    public SocketAddress(String address, int port) {
        this.address = address;
        this.port = port;
    }

}

生成的返回報(bào)文格式如下：

{
    "resources": [
        {
            "endpoints": [
                {
                    "lb_endpoints": [
                        {
                            "endpoint": {
                                "address": {
                                    "socket_address": {
                                        "address": "tcp://127.0.0.1",
                                        "port": 8888
                                    }
                                }
                            }
                        }
                    ]
                }
            ],
            "@type": "type.googleapis.com/envoy.api.v2.ClusterLoadAssignment",
            "cluster_name": "myService"
        }
    ],
    "version_info": "v1"
}

這里注意請(qǐng)求和返回報(bào)文的 json 格式一定不能錯(cuò)，否則無(wú)法和 envoy 進(jìn)行交互檐束。

再實(shí)現(xiàn) host 實(shí)例用來(lái)保存服務(wù)地址辫秧。

public class DiscoveryHost {

    // IP地址
    @JsonProperty("ip_address")
    private String ipAddress;
    // 端口
    private int port;
    private DiscoveryTags tags;

    @Override
    public String toString() {
        return "ip_address = " + this.ipAddress + ", port = " + this.port
                + ", tags = [" + this.tags + "]";
    }
}

public class DiscoveryHosts {

    private List<DiscoveryHost> hosts = new ArrayList<>();
    private String service;

    @Override
    public String toString() {
        return "service = " + this.service + ",hosts = " + this.hosts;
    }
}

/**
 * 服務(wù)發(fā)現(xiàn)中的元數(shù)據(jù)信息
 */
public class DiscoveryTags {

    // 分區(qū)
    @JsonProperty("az")
    private String zone;
    // 是否灰度
    private boolean cannary;
    // 權(quán)重
    @JsonProperty("load_balancing_weight")
    private int loadBalancingWeight;

    @Override
    public String toString() {
        return "az = " + this.zone + ", cannary = " + this.cannary +
                ", load_balancing_weight = " + this.loadBalancingWeight;
    }
}

這樣所有的對(duì)象都生成完畢了，下面看下對(duì)外暴露的接口被丧，包括服務(wù)發(fā)現(xiàn)的接口和操作服務(wù)的接口盟戏。

@RestController
public class DiscoveryController {

    private static final Logger LOGGER = LoggerFactory.getLogger(DiscoveryController.class);

    private int version = 1;
    private Map<String, List<DiscoveryHost>> serviceHosts = new HashMap<>();

    /**
     * envoy eds服務(wù)接口
     * @param request
     * @return
     */
    @PostMapping("/v2/discovery:endpoints")
    public DiscoveryResponse discoveryEndpoints(@RequestBody DiscoveryRequest request) {
        LOGGER.info("Dicovery Request: {}", request);
        // 讀取輸入
        DiscoveryNode node = request.getNode();
        String id = node.getId();
        String cluster = node.getCluster();
        List<String> resourceNames = request.getResourceNames();

        //構(gòu)建返回
        DiscoveryResponse response = new DiscoveryResponse();
        response.setVersionInfo("v" + version);
        ResponseResource resource = new ResponseResource();
        List<EndPoints> endPointsList = new ArrayList<>();
        List<LbEndPoint> lbEndPoints = new ArrayList<>();
        for(String resourceName : resourceNames) {
            if(serviceHosts.containsKey(resourceName)) {
                List<DiscoveryHost> hosts =  serviceHosts.get(resourceName);
                for(DiscoveryHost host : hosts) {
                    EndPoint endPoint = new EndPoint();
                    endPoint.setAddress(new DiscoveryAddress(new SocketAddress(host.getIpAddress(), host.getPort())));
                    LbEndPoint lbEndPoint = new LbEndPoint();
                    lbEndPoint.setEndPonit(endPoint);
                    lbEndPoints.add(lbEndPoint);
                }
            }

            resource.setType("type.googleapis.com/envoy.api.v2.ClusterLoadAssignment");
            resource.setClusterName(resourceName);
            if(lbEndPoints.size() > 0) {
                EndPoints endPoints = new EndPoints();
                endPoints.setLbEndPoints(lbEndPoints);
                endPointsList.add(endPoints);
            }
            resource.setEndPoints(endPointsList);
            response.getResources().add(resource);
            return response;
        }
        return null;
    }

    /**
     * 獲取當(dāng)前所有服務(wù)
     * @return
     */
    @GetMapping("/edsservice")
    public List<DiscoveryHosts> getAllServices() {
        List<DiscoveryHosts> hostsList = new ArrayList<>();
        for(Map.Entry<String, List<DiscoveryHost>> entry : this.serviceHosts.entrySet()) {
            DiscoveryHosts hosts = new DiscoveryHosts();
            hosts.setService(entry.getKey());
            hosts.setHosts(entry.getValue());
            hostsList.add(hosts);
        }
        return hostsList;
    }

    /**
     * 獲取某個(gè)服務(wù)
     * @param serviceName
     * @return
     */
    @GetMapping("/edsservice/{serviceName}")
    public DiscoveryHosts getHosts(@PathVariable("serviceName") String serviceName) {
        LOGGER.info("getHostsByServiceName: service={}", serviceName);
        DiscoveryHosts hostsList = new DiscoveryHosts();
        hostsList.setHosts(this.serviceHosts.get(serviceName));
        return hostsList;
    }

    /**
     * 添加某個(gè)服務(wù)
     * @param serviceName
     * @param hosts
     */
    @PostMapping("/edsservice/{serviceName}")
    public void addHosts(@PathVariable("serviceName") String serviceName,
                         @RequestBody DiscoveryHosts hosts) {
        LOGGER.info("addHost: service={}, body={}", serviceName, hosts);
        if (!this.serviceHosts.containsKey(serviceName)) {
            this.serviceHosts.put(serviceName, hosts.getHosts());
            this.version++;
        }

    }

    /**
     * 刪除某個(gè)服務(wù)
     * @param serviceName
     */
    @DeleteMapping("/edsservice/{serviceName}")
    public void removeHost(@PathVariable("serviceName") String serviceName) {
        LOGGER.info("removeHost: service={}", serviceName);
        this.serviceHosts.remove(serviceName);
    }

    /**
     * 修改某個(gè)服務(wù)
     * @param serviceName
     * @param hosts
     */
    @PutMapping("/edsservice/{serviceName}")
    public void updateHost(@PathVariable("serviceName") String serviceName,
                           @RequestBody DiscoveryHosts hosts) {
        LOGGER.info("updateHost: service={}, body={}", serviceName, hosts);
        if(this.serviceHosts.containsKey(serviceName)) {
            this.serviceHosts.put(serviceName, hosts.getHosts());
            this.version++;
        }
    }

}

這樣整個(gè) EDS Server 的代碼就開(kāi)發(fā)完畢了。

實(shí)現(xiàn)上游服務(wù)

下面實(shí)現(xiàn)用來(lái)接收 Enovy 發(fā)出的請(qǐng)求的上游服務(wù)甥桂，依舊創(chuàng)建一個(gè) SpringBoot Web 項(xiàng)目柿究。

添加2個(gè)簡(jiǎn)單的接口。

@RestController
public class UpstreamController {

    private static final String uuid = UUID.randomUUID().toString();

    @RequestMapping("/")
    public String index() {
        return "Hello, My UUID is " + uuid;
    }

    @RequestMapping("/healthz")
    public String health() {
        return "OK";
    }
}

添加配置文件 application.properties 黄选。

server.port=${PORT:8081}

上游服務(wù)編寫(xiě)完畢蝇摸，整體比較簡(jiǎn)單。

創(chuàng)建鏡像

創(chuàng)建 SpringBoot 鏡像的方法 Envoy 入門(mén)實(shí)戰(zhàn) 和部署一個(gè)SpringBoot應(yīng)用
都已經(jīng)介紹過(guò)办陷，這里不再贅述貌夕。這里創(chuàng)建一個(gè) EDS服務(wù)鏡像，2個(gè)上游服務(wù)鏡像民镜，上游服務(wù)鏡像使用的端口分別是 8081 和 8082啡专。

然后創(chuàng)建一個(gè) envoy 鏡像，envoy 鏡像使用的配置文件 envoy-config.yaml 如下殃恒。

admin:
  access_log_path: /tmp/admin_access.log
  address:
    socket_address:
      address: 127.0.0.1
      port_value: 9000

node:
  cluster: mycluster
  id: testid

static_resources:
  listeners:
  - name: listener_0

    address:
      socket_address: { address: 0.0.0.0, port_value: 10000 }

    filter_chains:
    - filters:
      - name: envoy.http_connection_manager
        config:
          stat_prefix: ingress_http
          codec_type: AUTO
          route_config:
            name: local_route
            virtual_hosts:
            - name: local_service
              domains: ["*"]
              routes:
              - match: { prefix: "/" }
                route: { cluster: service_backend }
          http_filters:
          - name: envoy.router

  clusters:
  - name: service_backend
    type: EDS
    connect_timeout: 0.25s
    eds_cluster_config:
      service_name: myservice
      eds_config:
        api_config_source:
          api_type: REST
          cluster_names: [edscluster]
          refresh_delay: 5s
  - name: edscluster
    type: STATIC
    connect_timeout: 0.25s
    hosts: [{ socket_address: { address: 127.0.0.1, port_value: 8080 }}]

其中需要注意的是由于 tomcat 不支持 URL 中存在"_"植旧，所以這里 cluster 名稱(chēng)使用了 edscluster。

envoy 鏡像的 Dockerfile 內(nèi)容如下：

FROM envoyproxy/envoy-alpine:latest
COPY envoy-config.yaml /etc/envoy/envoy.yaml

創(chuàng)建 envoy 鏡像离唐。

docker build -t envoytest2:v1.0.0 .

這樣我們就擁有了如下鏡像病附。

$ docker images
REPOSITORY                      TAG                 IMAGE ID            CREATED             SIZE
envoy-discovery                 1.0-SNAPSHOT        ab2722bc242c        44 hours ago        122MB
envoytest2                      v1.0.0              669ef4043e24        45 hours ago        40.9MB
upstream-service                1.0-SNAPSHOT        b2e02f942699        45 hours ago        122MB
upstream-service2               1.0-SNAPSHOT        1eb7d9738765        45 hours ago        122MB

啟動(dòng)環(huán)境

下面依次啟動(dòng)鏡像。

$ docker run -p 8081:8081 upstream-service:1.0-SNAPSHOT
$ docker run -p 8082:8082 upstream-service2:1.0-SNAPSHOT
$ docker run -p 10000:10000 --name envoytest2 envoytest2:v1.0.0
$ docker run --network=container:envoytest envoy-discovery:1.0-SNAPSHOT

現(xiàn)在在瀏覽器中中訪問(wèn) http://192.168.99.100:10000/ （其中192.168.99.100是 Docker 虛擬機(jī)的地址）會(huì)看到如下結(jié)果亥鬓。

找不到上游服務(wù).png

使用 Postman 向 http://192.168.99.100:8080/edsservice/myservice 發(fā)送如下 POST 請(qǐng)求完沪，將上游服務(wù)注冊(cè)到 Eds 服務(wù)之中。

{
  "hosts": [
    {
      "ip_address": "192.168.99.100",
      "port": 8081,
      "tags": {
        "server01": "8081",
        "canary": false,
        "load_balancing_weight": 50
      }
    },
    {
      "ip_address": "192.168.99.100",
      "port": 8082,
      "tags": {
        "server01": "8082",
        "canary": false,
        "load_balancing_weight": 50
      }
    }
  ]
}

然后再次訪問(wèn) http://192.168.99.100:10000/ 嵌戈，會(huì)看到上游服務(wù)的頁(yè)面覆积。

發(fā)現(xiàn)上游服務(wù)1.png

再次訪問(wèn)，可以看到另一個(gè)上游服務(wù)的頁(yè)面熟呛。

發(fā)現(xiàn)上游服務(wù)2.png

這是因?yàn)閮蓚€(gè)上游服務(wù)的權(quán)重一樣宽档， envoy 收到請(qǐng)求后會(huì)輪流定向到兩個(gè)上游服務(wù)。