本文將介紹在Openstack創(chuàng)建網(wǎng)絡(luò)和啟動實(shí)例過程中其網(wǎng)絡(luò)拓?fù)涞淖兓糜诹私釴eutron的網(wǎng)絡(luò)拓?fù)浜蜋C(jī)制。
0. 準(zhǔn)備
參考《使用vagrant和virtualbox搭建openstack集群》安裝一臺單機(jī)Openstack,先不要增加額外的計(jì)算節(jié)點(diǎn)虽界,這樣所有的網(wǎng)絡(luò)拓?fù)渥兓芍苯釉贏llinone節(jié)點(diǎn)查看。
另外需要準(zhǔn)備一個(gè)鏡像,用于創(chuàng)建openstack實(shí)例供實(shí)驗(yàn)使用枷餐,在packstack的配置參數(shù)中哨坪,提供了一個(gè)demo鏡像的下載地址且轨,但是上述文章中沒有安裝demo,因此需要手動下載該鏡像乏矾,并在Openstack Dashboard中上傳屋剑。
$ grep CONFIG_PROVISION_IMAGE allinone
CONFIG_PROVISION_IMAGE_NAME=cirros
CONFIG_PROVISION_IMAGE_URL=http://download.cirros-cloud.net/0.3.5/cirros-0.3.5-x86_64-disk.img
CONFIG_PROVISION_IMAGE_FORMAT=qcow2
CONFIG_PROVISION_IMAGE_PROPERTIES=
CONFIG_PROVISION_IMAGE_SSH_USER=cirros
上傳鏡像的菜單路徑為:Project -> Compute -> Images -> Create Image润匙。
Packstack安裝默認(rèn)的底層網(wǎng)絡(luò)驅(qū)動是OVSSwitch,而租戶網(wǎng)絡(luò)類型為vxlan唉匾。在一步一步創(chuàng)建網(wǎng)絡(luò)孕讳,啟動實(shí)例的過程中,可以使用以下命令在網(wǎng)絡(luò)節(jié)點(diǎn)(即Allinone節(jié)點(diǎn)os-ctl1)上查看每個(gè)操作之后網(wǎng)絡(luò)拓?fù)涞淖兓闆r:
- ovs-vsctl show, 查看ovs網(wǎng)橋
- brctl show巍膘,查看linux網(wǎng)橋
- ip netns厂财,查看網(wǎng)絡(luò)命名空間
- ip address,查看root命名空間內(nèi)的設(shè)備和地址
- ip netns exec <namespace> ip address峡懈,查看指定命名空間內(nèi)的設(shè)備和地址
1. 從無到有
初始化完裝完Openstack后璃饱,在網(wǎng)絡(luò)節(jié)點(diǎn)即os-ctl1中運(yùn)行上述命令:
# ovs-vsctl show
ef2ce1ab-4571-4e55-84b8-50163f9c0e0a
Manager "ptcp:6640:127.0.0.1"
is_connected: true
Bridge br-tun
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port br-tun
Interface br-tun
type: internal
Port patch-int
Interface patch-int
type: patch
options: {peer=patch-tun}
Bridge br-ex
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port phy-br-ex
Interface phy-br-ex
type: patch
options: {peer=int-br-ex}
Port br-ex
Interface br-ex
type: internal
Bridge br-int
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port int-br-ex
Interface int-br-ex
type: patch
options: {peer=phy-br-ex}
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port br-int
Interface br-int
type: internal
ovs_version: "2.9.0"
# brctl show
bridge name bridge id STP enabled interfaces
# ip netns
# ip address | grep ^[0-9]
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
4: ovs-system: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
5: br-ex: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
6: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
7: br-tun: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000
上述結(jié)果顯示,Neutron默認(rèn)在網(wǎng)絡(luò)節(jié)點(diǎn)上創(chuàng)建了3個(gè)OVS網(wǎng)橋肪康,分別為br-tun荚恶,br-ex,br-int梅鹦,其中br-int分別和br-tun裆甩,br-ex互連,拓?fù)鋱D如下:
2. 創(chuàng)建租戶私有網(wǎng)絡(luò)
在Openstack Dashborad上創(chuàng)建一個(gè)租戶網(wǎng)絡(luò)齐唆,菜單路徑為Project -> Network -> Networks -> Create Network嗤栓。填入如下參數(shù):
- Network Name:test-net
- Subnet Name:test-net-subnet
- Network Address:172.16.2.0/24
- Gateway IP:172.16.2.1
- Enable DHCP: true
創(chuàng)建完畢后,網(wǎng)絡(luò)節(jié)點(diǎn)拓?fù)渥兓缦拢?/p>
# ovs-vsctl show
...
Bridge br-int
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port int-br-ex
Interface int-br-ex
type: patch
options: {peer=phy-br-ex}
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port br-int
Interface br-int
type: internal
Port "tape68eed07-f5"
tag: 2
Interface "tape68eed07-f5"
type: internal
...
# ip netns
qdhcp-d18f42ff-a688-4c57-acde-6299326022dc
# ip netns exec qdhcp-d18f42ff-a688-4c57-acde-6299326022dc ip address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
9: tape68eed07-f5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN qlen 1000
link/ether fa:16:3e:89:e5:fa brd ff:ff:ff:ff:ff:ff
inet 172.16.2.2/24 brd 172.16.2.255 scope global tape68eed07-f5
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fe89:e5fa/64 scope link
valid_lft forever preferred_lft forever
上述結(jié)果顯示箍邮,由于在創(chuàng)建subnet的時(shí)候勾選了啟用DHCP茉帅,因此Neutron為DHCP服務(wù)創(chuàng)建了一個(gè)命名空間,并在其中創(chuàng)建一個(gè)TAP設(shè)備tape68eed07-f5作為DHCP的網(wǎng)口锭弊,該網(wǎng)口被加入到br-int網(wǎng)橋中堪澎,該DHCP服務(wù)的地址為172.16.2.2。拓?fù)鋱D如下:
同時(shí)味滞,在dashboard中樱蛤,可以看到新創(chuàng)建的network自帶了一個(gè)port钮呀,這個(gè)port即為dhcp接入的端口。
3. 啟動實(shí)例并將其接入網(wǎng)絡(luò)
在Openstack Dashborad上啟動一個(gè)nova實(shí)例昨凡,菜單路徑為Project -> Compute -> Instances -> Launch Instance爽醋。填入如下參數(shù):
- Instance Name: test-vm1
- Source中選擇之前上傳的鏡像,由于是實(shí)驗(yàn)環(huán)境便脊,可將“Delete Volume on Instance Delete”設(shè)為Yes蚂四,避免浪費(fèi)空間。
- Flavor: m1.tiny
- Networks: 上面創(chuàng)建的test-net
實(shí)例啟動成功后哪痰,網(wǎng)絡(luò)節(jié)點(diǎn)拓?fù)渥兓缦拢?/p>
# ovs-vsctl show
...
Bridge br-int
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port "qvo5d07f509-f1"
tag: 2
Interface "qvo5d07f509-f1"
Port int-br-ex
Interface int-br-ex
type: patch
options: {peer=phy-br-ex}
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port br-int
Interface br-int
type: internal
Port "tape68eed07-f5"
tag: 2
Interface "tape68eed07-f5"
type: internal
...
# brctl show
bridge name bridge id STP enabled interfaces
qbr5d07f509-f1 8000.2eeb31a8ff84 no qvb5d07f509-f1
tap5d07f509-f1
# ip address | grep ^[0-9]
...
10: qbr5d07f509-f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP qlen 1000
11: qvo5d07f509-f1@qvb5d07f509-f1: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1450 qdisc noqueue master ovs-system state UP qlen 1000
12: qvb5d07f509-f1@qvo5d07f509-f1: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1450 qdisc noqueue master qbr5d07f509-f1 state UP qlen 1000
13: tap5d07f509-f1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast master qbr5d07f509-f1 state UNKNOWN qlen 1000
上述結(jié)果顯示,Neutron為新實(shí)例創(chuàng)建了4個(gè)設(shè)備晌杰,其中一個(gè)是Linux網(wǎng)橋qbr5d07f509-f1,該網(wǎng)橋接入了兩個(gè)設(shè)備:
- qvb5d07f509-f1乎莉,這個(gè)設(shè)備和qvo5d07f509-f1是一對VETH設(shè)備,而qvo5d07f509-f1被加入到了OVS網(wǎng)橋br-int中惋啃,因此該設(shè)備用于新建的Linux網(wǎng)橋和br-int的連接。
- tap5d07f509-f1边灭,這個(gè)TAP設(shè)備就是供新實(shí)例接入網(wǎng)絡(luò)的網(wǎng)口。
拓?fù)鋱D如下:
進(jìn)到創(chuàng)建的實(shí)例中查看網(wǎng)絡(luò)配置:
# sudo virsh list
Id Name State
----------------------------------------------------
1 instance-00000001 running
# virsh console 1
Connected to domain instance-00000001
Escape character is ^]
login as 'cirros' user. default password: 'cubswin:)'. use 'sudo' for root.
cirros login: cirros
Password:
$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc pfifo_fast qlen 1000
link/ether fa:16:3e:85:a9:84 brd ff:ff:ff:ff:ff:ff
inet 172.16.2.7/24 brd 172.16.2.255 scope global eth0
inet6 fe80::f816:3eff:fe85:a984/64 scope link
valid_lft forever preferred_lft forever
$ ping 172.16.2.2
PING 172.16.2.2 (172.16.2.2): 56 data bytes
64 bytes from 172.16.2.2: seq=0 ttl=64 time=3.599 ms
?
--- 172.16.2.2 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 3.599/3.599/3.599 ms
$ ping 172.16.2.1
PING 172.16.2.1 (172.16.2.1): 56 data bytes
?
--- 172.16.2.1 ping statistics ---
2 packets transmitted, 0 packets received, 100% packet loss
可以看到称簿,DHCP為該實(shí)例分配了地址172.16.2.7,并且可以ping通DHCP服務(wù)器172.16.2.2憨降,但是無法ping通網(wǎng)關(guān)172.16.2.1,這是因?yàn)槟壳暗膶?shí)驗(yàn)僅僅局限在子網(wǎng)內(nèi)部该酗,網(wǎng)關(guān)設(shè)備還沒有創(chuàng)建授药。
4. 創(chuàng)建Router為網(wǎng)絡(luò)互通做準(zhǔn)備
在Openstack Dashborad上創(chuàng)建Router,菜單路徑為Project -> Network -> Routers -> Create Router呜魄。填入如下參數(shù):
- Router Name: router01
- 創(chuàng)建完畢后悔叽,進(jìn)入router為其添加Interface, 將目前僅有的test-net-subnet加入到router中
添加Interface后,根據(jù)Interface所在子網(wǎng)爵嗅,Neutron自動為其分配IP 172.16.2.1娇澎,并且這個(gè)子網(wǎng)內(nèi)的實(shí)例test-vm1可以通過該ip訪問router,這個(gè)router即為這個(gè)子網(wǎng)的網(wǎng)關(guān)睹晒。
此時(shí)趟庄,網(wǎng)絡(luò)節(jié)點(diǎn)拓?fù)渥兓缦拢?/p>
# ovs-vsctl show
...
Bridge br-int
Controller "tcp:127.0.0.1:6633"
is_connected: true
fail_mode: secure
Port "qvo5d07f509-f1"
tag: 2
Interface "qvo5d07f509-f1"
Port int-br-ex
Interface int-br-ex
type: patch
options: {peer=phy-br-ex}
Port patch-tun
Interface patch-tun
type: patch
options: {peer=patch-int}
Port br-int
Interface br-int
type: internal
Port "tape68eed07-f5"
tag: 2
Interface "tape68eed07-f5"
type: internal
Port "qr-9516320e-52"
tag: 2
Interface "qr-9516320e-52"
type: internal
...
# ip netns
qrouter-7d360e8e-e392-4def-816b-f92f7e5f9bc5
qdhcp-d18f42ff-a688-4c57-acde-6299326022dc
# sudo ip netns exec qrouter-7d360e8e-e392-4def-816b-f92f7e5f9bc5 ip address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
14: qr-9516320e-52: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN qlen 1000
link/ether fa:16:3e:f6:ec:e8 brd ff:ff:ff:ff:ff:ff
inet 172.16.2.1/24 brd 172.16.2.255 scope global qr-9516320e-52
valid_lft forever preferred_lft forever
inet6 fe80::f816:3eff:fef6:ece8/64 scope link
valid_lft forever preferred_lft forever
上述結(jié)果顯示括细,Neutron為router01創(chuàng)建了一個(gè)新的命名空間,并在其中創(chuàng)建了一個(gè)TAP設(shè)備qr-9516320e-52戚啥,該設(shè)備被加入到br-int網(wǎng)橋中勒极,地址為172.16.2.1。拓?fù)鋱D如下:
重新進(jìn)入虛機(jī)虑鼎,發(fā)現(xiàn)可以ping通網(wǎng)關(guān)172.16.2.1了。
# virsh console 1
Connected to domain instance-00000001
Escape character is ^]
login as 'cirros' user. default password: 'cubswin:)'. use 'sudo' for root.
cirros login: cirros
Password:
$ ping 172.16.2.1
PING 172.16.2.1 (172.16.2.1): 56 data bytes
64 bytes from 172.16.2.1: seq=0 ttl=64 time=4.134 ms
64 bytes from 172.16.2.1: seq=1 ttl=64 time=0.695 ms
?
--- 172.16.2.1 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0.695/2.414/4.134 ms
最后键痛,再回到創(chuàng)建的test-net網(wǎng)絡(luò)中查看其詳細(xì)信息炫彩,發(fā)現(xiàn)此時(shí)已經(jīng)有了3個(gè)port,分別接入了dhcp絮短,router01江兢,test-vm1。
[圖片上傳失敗...(image-b9b2c6-1536040869064)]
問題:
- 此時(shí)br-int無法從dhcp服務(wù)器上獲取到ip地址丁频,手動配置一個(gè)同網(wǎng)段的地址也無法ping通網(wǎng)關(guān)杉允,為什么?
- 什么時(shí)候創(chuàng)建TAP設(shè)備席里,什么時(shí)候創(chuàng)建VETH設(shè)備叔磷,兩者區(qū)別是什么?