etcd
etcd是一個高可用的分布式鍵值(key-value)數(shù)據(jù)庫枪狂。內(nèi)部采用raft協(xié)議作為一致性算法危喉，基于Go語言實現(xiàn)。etcd數(shù)據(jù)庫與redis類似州疾，其獨特性在于：
- 分布式部署辜限，擴展性強，且數(shù)據(jù)和事務(wù)保持一致
- 提供
watch接口严蓖，可監(jiān)聽多個鍵的變化 - 對于單個鍵而言薄嫡，每次更新其值都會保留上一個版本，可以對鍵進行版本回溯
-
ttl使用租約實現(xiàn)
etcd更強調(diào)的是各個節(jié)點之間的通信颗胡，同步毫深，確保各個節(jié)點上數(shù)據(jù)和事務(wù)的一致性，使得服務(wù)更穩(wěn)定毒姨，本身單節(jié)點的寫入能力并不強哑蔫。
redis更像是內(nèi)存型緩存，雖然也有cluster做主從同步和讀寫分離，但節(jié)點間的一致性主要強調(diào)的是數(shù)據(jù)闸迷，并不在乎事務(wù)嵌纲，因此讀寫能力很強，qps甚至可以達到10萬+
安裝
$ mkdir etcd
$ cd etcd
$ curl -L https://github.com/etcd-io/etcd/releases/download/v3.3.13/etcd-v3.3.13-linux-amd64.tar.gz -o
$ ./etcd-v3.3.13-linux-amd64.tar.gz
$ tar xzvf etcd-v3.3.13-linux-amd64.tar.gz --strip-components=1
$ ./etcd -version
$ ./etcdctl -version
本地多成員集群
針對單機用戶腥沽，開啟多進程逮走，模擬多機器集群(本次模擬開啟三個etcd集群)
1. 安裝go
2. 安裝goreman（進程管理工具）
$ go get github.com/mattn/goreman
3. 查看gopath
$ go env
GOPATH="/home/apple/go"
所以goreman運行路徑為home/apple/go/bin/goreman
4. 編寫goroman配置文件
goroman配置文件默認名稱為Procfile,可以更換，但啟動時今阳，需要通過-c指定配置文件
goroman管理進程的配置文件由 進程名:執(zhí)行命令組成
$ vim Procfile
編輯內(nèi)容如下
etcd1: ./etcd --name infra1 --listen-client-urls http://127.0.0.1:2379 --advertise-client-urls http://127.0.0.1:2379 --listen-peer-urls http://127.0.0.1:12380 --initial-advertise-peer-urls http://127.0.0.1:12 380 --initial-cluster-token etcd-cluster-1 --initial-cluster 'infra1=http://127.0.0.1:12380,infra2=http://127.0.0.1:22380,infra3=http://127.0.0.1:32380' --initial-cluster-state new --enable-pprof
etcd2: ./etcd --name infra2 --listen-client-urls http://127.0.0.1:22379 --advertise-client-urls http://127.0.0.1:22379 --listen-peer-urls http://127.0.0.1:22380 --initial-advertise-peer-urls http://127.0.0.1: 22380 --initial-cluster-token etcd-cluster-1 --initial-cluster 'infra1=http://127.0.0.1:12380,infra2=http://127.0.0.1:22380,infra3=http://127.0.0.1:32380' --initial-cluster-state new --enable-pprof
etcd3: ./etcd --name infra3 --listen-client-urls http://127.0.0.1:32379 --advertise-client-urls http://127.0.0.1:32379 --listen-peer-urls http://127.0.0.1:32380 --initial-advertise-peer-urls http://127.0.0.1: 32380 --initial-cluster-token etcd-cluster-1 --initial-cluster 'infra1=http://127.0.0.1:12380,infra2=http://127.0.0.1:22380,infra3=http://127.0.0.1:32380' --initial-cluster-state new --enable-pprof
5. 運行
$ /home/apple/go/bin/goreman -f Procfile start
此時etcd集群被開啟
6.查看集群列表
此時代表集群正確安裝并啟動
$ export ETCDCTL_API=3
$ ./etcdctl member list
8211f1d0f64f3269: name=infra1 peerURLs=http://127.0.0.1:12380 clientURLs=http://127.0.0.1:2379 isLeader=false
91bc3c398fb3c146: name=infra2 peerURLs=http://127.0.0.1:22380 clientURLs=http://127.0.0.1:22379 isLeader=true
fd422379fda50e48: name=infra3 peerURLs=http://127.0.0.1:32380 clientURLs=http://127.0.0.1:32379 isLeader=false
etcd基本使用
API地址https://godoc.org/github.com/coreos/etcd/client
官方包提供了對于etcd所有操作的API
創(chuàng)建用于操作etcd鍵值的KeysAPI對象kv
var cli client.Client
var kv client.KeysAPI
func handleError(e error, msg string) {
if e != nil {
log.Fatal(e)
}
if msg != "" {
log.Println(msg)
}
}
func init(){
cfg := client.Config{
Endpoints: []string{"http://127.0.0.1:2379", "http://127.0.0.1:22379", "http://127.0.0.1:32379"},
}
var e error
cli, e = client.New(cfg)
handleError(e, "")
kv = client.NewKeysAPI(cli)
}
設(shè)置鍵
func setVal(kv client.KeysAPI, key string, val string) {
log.Printf("設(shè)置鍵%s值%s\n", key, val)
_, e := kv.Set(context.Background(), key, val, nil)
if e != nil {
log.Println(e)
}
}
獲取鍵值
func getVal(kv client.KeysAPI, key string) string {
// 獲取鍵
log.Printf("開始獲取鍵%s \n", key)
resp, e := kv.Get(context.TODO(), key, nil)
handleError(e, "")
index, value := resp.Index, resp.Node.Value
log.Printf("獲取當前版本:%d 值:%s", index, resp.Node.Value)
return string(value)
}
創(chuàng)建文件夾
etcd的鍵值對存儲可以理解為文件存儲在目錄中言沐，鍵為目錄，值為文件
創(chuàng)建文件夾的目的為：使用etcd提供的watch方法可以監(jiān)控整個文件夾中鍵（即文件）的變化
func mkdir(kv client.KeysAPI, dir string) {
o := client.SetOptions{Dir: true}
_, e := kv.Set(context.Background(), dir, "", &o)
handleError(e, "創(chuàng)建完成")
}
調(diào)用
func TestSetVal(t *testing.T) {
setVal(kv, "/nginx/foo", "bar")
}
func TestGetVal(t *testing.T) {
getVal(kv, "/nginx/foo")
}
func TestMkdir(t *testing.T) {
mkdir(kv, "/nginx")
}
confd
輕量級的配置管理工具酣栈，主要有兩個目的
- 讀取etcd保存的配置信息险胰，同步到本地配置文件中，并保證本地配置文件是最新的
- 同步配置文件之后可以指定命令使配置生效
安裝
https://github.com/kelseyhightower/confd/blob/master/docs/installation.md
使用
把核心信息矿筝，比如upstream起便、server_name等存儲在etcd中，使用confd來自動生成nginx配置文件,并reload使配置生效
upstream www_test {
server 196.75.121.112:443; （動態(tài)生成）
}
server {
listen 443 ssl; （動態(tài)生成）
server_name www.test.com; (動態(tài)生成)
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;;
ssl_certificate /home/build/openresty/nginx/cert/dealssl/www.bestenover.com.crt; (動態(tài)生成)
location / {
proxy_pass https://www_test; (動態(tài)生成)
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_redirect off;
}
}
要實現(xiàn)動態(tài)配置首先要把核心信息存儲到etcd中
func TestNginxMkdir(t *testing.T) {
mkdir(kv, "/nginx")
mkdir(kv, "/nginx/https")
mkdir(kv, "/nginx/http")
mkdir(kv, "/nginx/ssl")
mkdir(kv, "/nginx/https/www")
mkdir(kv, "/nginx/https/www/server")
mkdir(kv, "/nginx/https/www/upstream")
mkdir(kv, "/nginx/https/www/server/location")
}
confd注冊監(jiān)控etcd的key為/nginx/窖维，只要發(fā)生變化就通知confd根據(jù)模板生成配置榆综。confd默認的配置路徑為/etc/confd/，創(chuàng)建conf.d和template兩個目錄铸史，分別存放配置資源和配置模板鼻疮。
nginx的配置資源如下所示：test.conf.toml
[template]
src = "test.conf.tmpl"
dest = "/tmp/test.conf"
keys = [
"/nginx",
]
check_cmd = "echo a"
reload_cmd = "echo b"
nginx的配置模板如下所示：test.conf.tmpl
upstream www_{{getv "/nginx/https/www/server/server_name"}} {
{{range getvs "/nginx/https/www/upstream/*"}}server {{.}};
{{end}}
}
server {
server_name {{getv "/nginx/https/www/server/server_name"}}:443;
ssl on
ssl_certificate {{getv "/nginx/https/www/server/ssl_certificate"}};
ssl_certificate_key {{getv "/nginx/https/www/server/ssl_certificate_key"}};
location / {
proxy_pass http://www_{{getv "/nginx/https/www/server/server_name"}};
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_redirect off;
}
}
開啟confd : confd -watch -backend etcd -node http://127.0.0.1:2379
設(shè)置內(nèi)容
func TestNginxSetVal(t *testing.T) {
setVal(kv, "/nginx/https/www/server/server_name", "test.com")
setVal(kv, "/nginx/https/www/server/ssl_certificate", "client.crt")
setVal(kv, "/nginx/https/www/server/ssl_certificate_key", "client.key")
setVal(kv, "/nginx/https/www/upstream/server1", "192.168.4.2:443")
setVal(kv, "/nginx/https/www/upstream/server2", "192.168.5.2:443")
}
生成結(jié)果
1 upstream www_test.com {
2 server 192.168.4.2:443;
3 server 192.168.5.2:443;
4
5 }
6
7 server {
8 server_name test.com:443;
9 ssl on
10 ssl_certificate client.crt;
11 ssl_certificate_key client.key;
12 location / {
13 proxy_pass http://www_test.com;
14 proxy_set_header Host $host;
15 proxy_set_header X-Real-IP $remote_addr;
16 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
17 proxy_set_header X-Forwarded-Proto https;
18 proxy_redirect off;
19 }
20 }
