1 在DVR環(huán)境中各組件的連接情況如圖
2 帶floatingip,從外部訪問虛機兑巾，ping floatingip 192.168.100.190
此時數(shù)據(jù)流如圖所示
2.1 獲取MAC
外網(wǎng)中的機器首先要通過 ARP 獲取虛機浮動 IP 對應(yīng)的 MAC 地址产上。浮動 IP 并沒有配置在 fip 的端口上，因此 fip 無法直接響應(yīng) ARP 請求，那怎么辦呢？Neutron 在 fip NS 的 fg 端口上配置了 arp proxy队贱，這樣欣除，fip 既可以響應(yīng)它自己的 interface 上的 IP 地址的 ARP 請求住拭，也可以響應(yīng)能通過它路由到的 IP 地址的 ARP 請求,
fip netns上配置的proxy_arp
[root@compute ~]# ip netns exec fip-82d87e5f-167e-4525-90b2-c8f340630a1e sysctl net.ipv4.conf.fg-42d94dcb-8b.proxy_arp
net.ipv4.conf.fg-42d94dcb-8b.proxy_arp = 1
fip netns 收到 ARP 請求后，將其 fg interface 的 MAC 地址返回,外網(wǎng)中的機器獲取到虛機浮動 IP 的 MAC 地址后历帚，發(fā)出 ICMP 網(wǎng)絡(luò)包
2.2 fip netns 路由到下qrouter netns
網(wǎng)絡(luò)包經(jīng)過 br-ex滔岳，被 fip 的 fg 端口收到，查路由表挽牢，命中第一條路由谱煤，從其 fpr interface 發(fā)出，到達 169.254.106.114.
ip netns exec fip-82d87e5f-167e-4525-90b2-c8f340630a1e ip route
169.254.106.114/31 dev fpr-b2ce2e13-1 proto kernel scope link src 169.254.106.115
192.168.100.0/24 dev fg-42d94dcb-8b proto kernel scope link src 192.168.100.184
192.168.100.190 via 169.254.106.114 dev fpr-b2ce2e13-1
fpr interface 和rfp interface 是一堆veth pair , fpr interface 發(fā)出的包被 veth 另一端的 qrouter 的 rfp-b2ce2e13-1 interface 收到禽拔。
可以看到qrouter netns 的 rfp-b2ce2e13-1口地址是169.254.106.114/31
[root@compute ~]# ip netns exec qrouter-b2ce2e13-127d-491a-a669-ff07263e3751 ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: rfp-b2ce2e13-1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
link/ether b2:e7:86:35:3d:ac brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 169.254.106.114/31 scope global rfp-b2ce2e13-1
valid_lft forever preferred_lft forever
inet6 fe80::b0e7:86ff:fe35:3dac/64 scope link
valid_lft forever preferred_lft forever
134: qr-d2b8d53f-64: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UNKNOWN qlen 1000
link/ether fa:16:3e:02:97:ab brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 brd 172.17.255.255 scope global qr-d2b8d53f-64
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe02:97ab/64 scope link
valid_lft forever preferred_lft forever
2.3 DNAT
在 qrouter 上刘离，首先做 DNAT
-A neutron-l3-agent-PREROUTING -d 192.168.100.190/32 -i rfp-b2ce2e13-1 -j DNAT --to-destination 172.17.0.7
DNAT后室叉，dest ip 改為172.17.0.7
2.4 再次路由
查 qrouter 的 main 路由表，命中第二條硫惕，從qr-d2b8d53f-64 發(fā)出
ip netns exec qrouter-b2ce2e13-127d-491a-a669-ff07263e3751 ip route
169.254.106.114/31 dev rfp-b2ce2e13-1 proto kernel scope link src 169.254.106.114
172.17.0.0/16 dev qr-d2b8d53f-64 proto kernel scope link src 172.17.0.1
數(shù)據(jù)包經(jīng)qr-d2b8d53f-64,發(fā)到br-int 到達虛機
3 帶floatingip從虛機訪問外網(wǎng)茧痕，從172.17.0.7 ping 8.8.8.8
3.1路由
進入qroute netns,查看路由規(guī)則走route 16表
[root@compute ~]# ip netns exec qrouter-b2ce2e13-127d-491a-a669-ff07263e3751 ip rule
0: from all lookup local
32766: from all lookup main
32767: from all lookup default
57483: from 172.17.0.7 lookup 16
2886795265: from 172.17.0.1/16 lookup 2886795265
[root@compute ~]# ip netns exec qrouter-b2ce2e13-127d-491a-a669-ff07263e3751 ip route list table 16
default via 169.254.106.115 dev rfp-b2ce2e13-1
3.2 SNAT
路由后會通過netfilter的POSTROUTING鏈中進行SNAT，將源IP改為192.168.100.190
[root@compute ~]# ip netns exec qrouter-b2ce2e13-127d-491a-a669-ff07263e3751 iptables -t nat -S
-P PREROUTING ACCEPT
-P INPUT ACCEPT
-P OUTPUT ACCEPT
-P POSTROUTING ACCEPT
-N neutron-l3-agent-OUTPUT
-N neutron-l3-agent-POSTROUTING
-N neutron-l3-agent-PREROUTING
-N neutron-l3-agent-float-snat
-N neutron-l3-agent-snat
-N neutron-postrouting-bottom
-A PREROUTING -j neutron-l3-agent-PREROUTING
-A OUTPUT -j neutron-l3-agent-OUTPUT
-A POSTROUTING -j neutron-l3-agent-POSTROUTING
-A POSTROUTING -j neutron-postrouting-bottom
-A neutron-l3-agent-POSTROUTING ! -i rfp-b2ce2e13-1 ! -o rfp-b2ce2e13-1 -m conntrack ! --ctstate DNAT -j ACCEPT
-A neutron-l3-agent-PREROUTING -d 169.254.169.254/32 -i qr-+ -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 9697
-A neutron-l3-agent-PREROUTING -d 192.168.100.190/32 -i rfp-b2ce2e13-1 -j DNAT --to-destination 172.17.0.7
-A neutron-l3-agent-float-snat -s 172.17.0.7/32 -j SNAT --to-source 192.168.100.190
-A neutron-l3-agent-snat -j neutron-l3-agent-float-snat
-A neutron-postrouting-bottom -m comment --comment "Perform source NAT on outgoing traffic." -j neutron-l3-agent-snat
從rfp-b2ce2e13-1口將包發(fā)到169.254.106.115恼除，rfp-b2ce2e13-1和fip netns的fpr-b2ce2e13-1踪旷，是一對veth pair，fpr-b2ce2e13-1收到數(shù)據(jù)包
[root@compute ~]# ip netns exec fip-82d87e5f-167e-4525-90b2-c8f340630a1e ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: fpr-b2ce2e13-1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
link/ether aa:f7:5f:31:67:db brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 169.254.106.115/31 scope global fpr-b2ce2e13-1
valid_lft forever preferred_lft forever
inet6 fe80::a8f7:5fff:fe31:67db/64 scope link
valid_lft forever preferred_lft forever
135: fg-42d94dcb-8b: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
link/ether fa:16:3e:c0:dc:94 brd ff:ff:ff:ff:ff:ff
inet 192.168.100.184/24 brd 192.168.100.255 scope global fg-42d94dcb-8b
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fec0:dc94/64 scope link
valid_lft forever preferred_lft forever
3.3 數(shù)據(jù)包在fip netns 中再次路由
包通過fpr-b2ce2e13-1發(fā)到169.254.106.115缚柳，后再在fip netns中查路由規(guī)則埃脏，路由
[root@compute ~]# ip netns exec fip-82d87e5f-167e-4525-90b2-c8f340630a1e ip rule
0: from all lookup local
32766: from all lookup main
32767: from all lookup default
2852022899: from all iif fpr-b2ce2e13-1 lookup 2852022899
[root@compute ~]# ip netns exec fip-82d87e5f-167e-4525-90b2-c8f340630a1e ip route list table 2852022899
default via 192.168.100.1 dev fg-42d94dcb-8b
從fg-42d94dcb-8b 口將包發(fā)到192.168.100.1，再有192.168.100.1繼續(xù)路由轉(zhuǎn)發(fā)
4 不帶浮動IP秋忙，虛機訪問外網(wǎng)
4.1 計算節(jié)點qrouter 路由
查看路由規(guī)則
ip netns exec qrouter-b2ce2e13-127d-491a-a669-ff07263e3751 ip rule
0: from all lookup local
32766: from all lookup main
32767: from all lookup default
2886795265: from 172.17.0.1/16 lookup 2886795265
查看路由表2886795265
ip netns exec qrouter-b2ce2e13-127d-491a-a669-ff07263e3751 ip route list table 2886795265
default via 172.17.0.10 dev qr-d2b8d53f-64
只有一條策略彩掐，經(jīng)過qr口，發(fā)到下一個路由172.17.0.10
172.17.0.10 路由是在network節(jié)點的snat netns sg口
[root@network ~]# ip netns exec snat-b2ce2e13-127d-491a-a669-ff07263e3751 ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
93: sg-935eff5f-b8: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1400 qdisc noqueue state UNKNOWN qlen 1000
link/ether fa:16:3e:bf:6d:b5 brd ff:ff:ff:ff:ff:ff
inet 172.17.0.10/16 brd 172.17.255.255 scope global sg-935eff5f-b8
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:febf:6db5/64 scope link
valid_lft forever preferred_lft forever
94: qg-2b2169ee-a5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN qlen 1000
link/ether fa:16:3e:9d:5c:bd brd ff:ff:ff:ff:ff:ff
inet 192.168.100.187/24 brd 192.168.100.255 scope global qg-2b2169ee-a5
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe9d:5cbd/64 scope link
valid_lft forever preferred_lft forever
4.2 SNAT
network 節(jié)點的snat netns 做SNAT
ip netns exec snat-b2ce2e13-127d-491a-a669-ff07263e3751 iptables -t nat -S
-A neutron-l3-agent-snat -o qg-2b2169ee-a5 -j SNAT --to-source 192.168.100.187
4.3 snat netns路由
ip netns exec snat-b2ce2e13-127d-491a-a669-ff07263e3751 ip rule
0: from all lookup local
32766: from all lookup main
32767: from all lookup default
沒有額外策略灰追，走main表
ip netns exec snat-b2ce2e13-127d-491a-a669-ff07263e3751 ip route list table main
default via 192.168.100.1 dev qg-2b2169ee-a5
172.17.0.0/16 dev sg-935eff5f-b8 proto kernel scope link src 172.17.0.10
192.168.100.0/24 dev qg-2b2169ee-a5 proto kernel scope link src 192.168.100.187
根據(jù)目的地址堵幽，命中第一條，包從qg-2b2169ee-a5口出去弹澎，再從192.168.100.1網(wǎng)關(guān)上繼續(xù)路由
