發(fā)往Docker容器內(nèi)的數(shù)據(jù)包源地址被修改的研究

引言

此文檔是對(duì)發(fā)往Docker容器內(nèi)的數(shù)據(jù)包源地址被修改的研究做的實(shí)踐記錄。

實(shí)踐環(huán)境一

Centos7.2+Docker1.12.6

關(guān)閉firewalld：
systemctl stop firewalld

設(shè)置selinux為Permissive模式：
setenforce 0

/etc/docker/daemon.json內(nèi)容：
{
  "userland-proxy": true
}

創(chuàng)建名為python的容器：
docker run -itd --name python --network bridge -p 8000:9999/udp --entrypoint bash python:2.7.15

容器內(nèi)接收udp數(shù)據(jù)包的程序內(nèi)容為：
#!/usr/bin/env python
# -*- coding: utf-8 -*-

import socket
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
# 綁定端口:
s.bind(('0.0.0.0', 9999))
print 'Bind UDP on 9999...'

while 1:
    # 接收數(shù)據(jù):
    data, addr = s.recvfrom(1024)
    print 'addr: {0}, data: {1}'.format(addr, data)

路由信息：

網(wǎng)橋docker0設(shè)備信息：

一搏讶、通過另一臺(tái)主機(jī)向當(dāng)前主機(jī)發(fā)送數(shù)據(jù)包
1.192.168.84.75主機(jī)上運(yùn)行接收udp數(shù)據(jù)包的程序系吩，運(yùn)行配置基于實(shí)踐環(huán)境，在192.168.84.79主機(jī)上通過 echo mac | nc -s 192.168.84.79 -p 42731 -u 192.168.84.75 8000（產(chǎn)生數(shù)據(jù)包的源地址科盛、源端口與目的地址、目的端口為：192.168.84.79.42731 > 192.168.84.75.8000）向192.168.84.75發(fā)送數(shù)據(jù)包榨崩。

對(duì)192.168.84.79主機(jī)的網(wǎng)卡進(jìn)行抓包處理：
tcpdump -i eno16777984 udp port 8000 -nn -vv
tcpdump: listening on eno16777984, link-type EN10MB (Ethernet), capture size 262144 bytes
13:33:15.227775 IP (tos 0x0, ttl 64, id 58746, offset 0, flags [DF], proto UDP (17), length 32)
    192.168.84.79.42731 > 192.168.84.75.8000: [bad udp cksum 0x2a09 -> 0x3f53!] UDP, length 4

分別對(duì)192.168.84.75主機(jī)的lo胳施、物理網(wǎng)卡（eno16777984）焦辅、docker0以及容器的網(wǎng)卡進(jìn)行抓包處理：
tcpdump -i lo -nn -vv
tcpdump: listening on lo, link-type EN10MB (Ethernet), capture size 262144 bytes
無內(nèi)容

tcpdump -i eno16777984 udp port 8000 -nn -vv
tcpdump: listening on eno16777984, link-type EN10MB (Ethernet), capture size 262144 bytes
13:33:25.690285 IP (tos 0x0, ttl 64, id 58746, offset 0, flags [DF], proto UDP (17), length 32)
    192.168.84.79.42731 > 192.168.84.75.8000: [udp sum ok] UDP, length 4

tcpdump -i docker0 -nn -vv
tcpdump: listening on docker0, link-type EN10MB (Ethernet), capture size 262144 bytes
13:33:25.690319 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
13:33:25.690354 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28
13:33:25.690362 IP (tos 0x0, ttl 63, id 58746, offset 0, flags [DF], proto UDP (17), length 32)
    192.168.84.79.42731 > 172.17.0.2.9999: [udp sum ok] UDP, length 4

tcpdump -i vethc4d778f -nn -vv
tcpdump: listening on vethc4d778f, link-type EN10MB (Ethernet), capture size 262144 bytes
13:33:25.690332 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
13:33:25.690354 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28
13:33:25.690364 IP (tos 0x0, ttl 63, id 58746, offset 0, flags [DF], proto UDP (17), length 32)
    192.168.84.79.42731 > 172.17.0.2.9999: [udp sum ok] UDP, length 4

由上述的時(shí)間可知數(shù)據(jù)包先經(jīng)過84.79主機(jī)的網(wǎng)卡，然后數(shù)據(jù)包被發(fā)往84.75主機(jī)，在84.75主機(jī)上經(jīng)過的設(shè)備依次為：主機(jī)網(wǎng)卡（eno16777984 ）---> docker0 ---> 容器網(wǎng)卡（vethc4d778f）

數(shù)據(jù)包發(fā)送前與發(fā)送后84.75主機(jī)的nat表的對(duì)比：
發(fā)送前執(zhí)行 iptables -t nat -nvL > pre
發(fā)送后執(zhí)行 iptables -t nat -nvL > now
然后通過diff命令對(duì)比兩個(gè)文件的內(nèi)容:

分析：84.79主機(jī)發(fā)送的數(shù)據(jù)包源地址、源端口與目的地址班巩、目的端口分別為：192.168.84.79.42731 > 192.168.84.75.8000。數(shù)據(jù)包從主機(jī)網(wǎng)卡eno16777984流出后，首先經(jīng)過PREROUTING鏈单匣，此時(shí)匹配到動(dòng)作DOCKER，DOCKER鏈的規(guī)則是：-A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER，即當(dāng)數(shù)據(jù)包的地址類型為本地地址LOCAL時(shí)鲸伴，將執(zhí)行DOCKER鏈的規(guī)則匹配。LOCAL本地地址不僅指127.0.0.1，也包括本機(jī)IP裹唆。因此數(shù)據(jù)包（192.168.84.79.42731 > 192.168.84.75.8000）能夠匹配到DOCKER劳坑。DOCKER鏈的第一條規(guī)則：-A DOCKER -i docker0 -j RETURN，表示流入docker0的設(shè)備的數(shù)據(jù)包將返回到DOCKER鏈的調(diào)用處；第二條規(guī)則：-A DOCKER ! -i docker0 -p udp -m udp --dport 8000 -j DNAT --to-destination 172.17.0.2:9999存和，表示不是流入docker0設(shè)備、udp協(xié)議、目的端口是8000的數(shù)據(jù)包將其目的地址修改為172.17.0.2宪祥，目的端口修改為9999。數(shù)據(jù)包滿足第二條規(guī)則此時(shí)數(shù)據(jù)包的源地址、源端口與目的地址野芒、目的端口變?yōu)椋?92.168.84.79.42731 > 172.17.0.2.9999。數(shù)據(jù)包經(jīng)過路由判斷目的地址（172.17.0.2）不是發(fā)往本機(jī)胀滚，于是經(jīng)過FORWARD到達(dá)POSTROUTING鏈顷编，POSTROUTING鏈的第一條規(guī)則：-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE双肤，表示對(duì)不是從docker0網(wǎng)卡流出且源地址是172.17網(wǎng)段的數(shù)據(jù)包進(jìn)行源地址偽裝。數(shù)據(jù)包不滿足這條匹配規(guī)則。第二條規(guī)則：-A POSTROUTING -s 172.17.0.2/32 -d 172.17.0.2/32 -p udp -m udp --dport 9999 -j MASQUERADE缩赛，表示對(duì)源地址、目的地址均為172.17.0.2/32旨袒、udp協(xié)議、目的端口為9999的數(shù)據(jù)包進(jìn)行源地址偽裝，同樣數(shù)據(jù)包也不滿足第二條規(guī)則隧魄。此時(shí)數(shù)據(jù)包的源地址嘱么、端口與目的地址几迄、端口沒有發(fā)生變化依然是：192.168.84.79.42731 > 172.17.0.2.9999木羹。數(shù)據(jù)包從POSTROUTING鏈出來后根據(jù)路由規(guī)則（172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0）會(huì)被發(fā)往docker0設(shè)備，發(fā)往docker0設(shè)備的數(shù)據(jù)包源地址、端口與目的地址苍在、端口同樣沒有發(fā)生變化（通過上述對(duì)docker0的抓包分析也可知）结啼。docker0設(shè)備通過ARP找到ip地址為172.17.0.2的設(shè)備vethc4d778f朴译，進(jìn)而將數(shù)據(jù)包發(fā)往vethc4d778f此設(shè)備属铁，vethc4d778f設(shè)備接收的數(shù)據(jù)包的源地址眠寿、源端口與目的地址、目的端口依然為：192.168.84.79.42731 > 172.17.0.2.9999焦蘑。因此程序接收到的數(shù)據(jù)包的源地址為：192.168.84.79盯拱，程序的輸出結(jié)果也可證實(shí)：addr: ('192.168.84.79', 42731), data: mac

在84.79主機(jī)上執(zhí)行 echo mac | nc -s 127.0.0.1 -p 42731 -u 192.168.84.75 8000 例嘱，報(bào)錯(cuò)誤信息Ncat: Invalid argument.

二狡逢、本機(jī)上通過網(wǎng)卡ip地址發(fā)送數(shù)據(jù)包

在192.168.84.75主機(jī)上執(zhí)行 echo mac | nc -s 192.168.84.75 -p 34191 -u 192.168.84.75 8000
分別lo、物理網(wǎng)卡（eno16777984）拼卵、docker0以及容器的網(wǎng)卡進(jìn)行抓包處理：

tcpdump -i lo -nn -vv 
tcpdump: listening on lo, link-type EN10MB (Ethernet), capture size 262144 bytes
無內(nèi)容

tcpdump -i eno16777984 udp port 8000 -nn -vv 
tcpdump: listening on eno16777984, link-type EN10MB (Ethernet), capture size 262144 bytes
無內(nèi)容

tcpdump -i docker0 -nn -vv
tcpdump: listening on docker0, link-type EN10MB (Ethernet), capture size 262144 bytes
13:37:07.072740 IP (tos 0x0, ttl 64, id 64295, offset 0, flags [DF], proto UDP (17), length 32)
    192.168.84.75.34191 > 172.17.0.2.9999: [bad udp cksum 0xc124 -> 0xc1c4!] UDP, length 4
13:37:12.075322 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
13:37:12.075355 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

tcpdump -i vethc4d778f -nn -vv
tcpdump: listening on vethc4d778f, link-type EN10MB (Ethernet), capture size 262144 bytes
13:37:07.072758 IP (tos 0x0, ttl 64, id 64295, offset 0, flags [DF], proto UDP (17), length 32)
    192.168.84.75.34191 > 172.17.0.2.9999: [bad udp cksum 0xc124 -> 0xc1c4!] UDP, length 4
13:37:12.075332 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
13:37:12.075355 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

可知數(shù)據(jù)包僅經(jīng)過docker0然后去往容器設(shè)備vethc4d778f奢浑，并沒有經(jīng)過綁定192.168.84.75 IP的網(wǎng)卡eno16777984

數(shù)據(jù)包發(fā)送前與發(fā)送后84.75主機(jī)的nat表的對(duì)比：

分析：echo mac | nc -s 192.168.84.75 -p 34191 -u 192.168.84.75 8000發(fā)出的數(shù)據(jù)包源地址、源端口與目的地址腋腮、目的端口為：192.168.84.75.34191 > 192.168.84.75.8000雀彼。經(jīng)過實(shí)踐（實(shí)踐本地訪問的數(shù)據(jù)包經(jīng)過的鏈與設(shè)備的次序一文）可知壤蚜，在本機(jī)上通過ip地址訪問發(fā)出的數(shù)據(jù)包僅經(jīng)過OUTPUT與POSTROUTING鏈。數(shù)據(jù)包從本地進(jìn)程發(fā)出后徊哑，經(jīng)過OUTPUT鏈袜刷，匹配到DOCKER鏈的第二條規(guī)則，即將數(shù)據(jù)包的目的地址莺丑、目的端口更改為172.17.0.2:9999著蟹，因此經(jīng)過OUTPUT鏈后的數(shù)據(jù)包源地址、源端口與目的地址窒盐、目的端口變?yōu)椋?92.168.84.75.34191 > 172.17.0.2.9999，緊接著經(jīng)過POSTROUTING鏈钢拧，數(shù)據(jù)包僅被POSTROUTING鏈的默認(rèn)規(guī)則匹配蟹漓，而沒有被更改源地址的兩條規(guī)則匹配到，因此從POSTROUTING鏈出來的數(shù)據(jù)包源地址源内、源端口與目的地址葡粒、目的端口依然為：192.168.84.75.34191 > 172.17.0.2.9999。根據(jù)路由規(guī)則膜钓，數(shù)據(jù)包會(huì)被發(fā)往docker0設(shè)備嗽交，然后被發(fā)往容器設(shè)備vethc4d778f，最后程序收到的數(shù)據(jù)包的源ip為192.168.84.75颂斜，結(jié)果可證實(shí)：addr: ('192.168.84.75', 34191), data: mac夫壁。

echo mac | nc -s 127.0.0.1 -p 34191 -u 192.168.84.75 8000 Ncat: Invalid argument.

三、本機(jī)上通過127.0.0.1地址發(fā)送數(shù)據(jù)包

在192.168.84.75主機(jī)上執(zhí)行 echo mac | nc -s 127.0.0.1 -p 33089 -u 127.0.0.1 8000
分別對(duì)主機(jī)的lo沃疮、物理網(wǎng)卡（eno16777984）盒让、docker0以及容器的網(wǎng)卡進(jìn)行抓包處理：

tcpdump -i lo -nn -vv
tcpdump: listening on lo, link-type EN10MB (Ethernet), capture size 262144 bytes
13:41:30.463055 IP (tos 0x0, ttl 64, id 54997, offset 0, flags [DF], proto UDP (17), length 32)
    127.0.0.1.33089 > 127.0.0.1.8000: [bad udp cksum 0xfe1f -> 0x90e6!] UDP, length 4

tcpdump -i eno16777984 udp port 8000 -nn -vv
tcpdump: listening on eno16777984, link-type EN10MB (Ethernet), capture size 262144 bytes
無內(nèi)容

tcpdump -i docker0 -nn -vv
tcpdump: listening on docker0, link-type EN10MB (Ethernet), capture size 262144 bytes
13:41:30.463419 IP (tos 0x0, ttl 64, id 35536, offset 0, flags [DF], proto UDP (17), length 32)
    172.17.0.1.33987 > 172.17.0.2.9999: [bad udp cksum 0x5843 -> 0x2b72!] UDP, length 4
13:41:35.467315 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
13:41:35.467348 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

tcpdump -i vethc4d778f -nn -vv
tcpdump: listening on vethc4d778f, link-type EN10MB (Ethernet), capture size 262144 bytes
13:41:30.463430 IP (tos 0x0, ttl 64, id 35536, offset 0, flags [DF], proto UDP (17), length 32)
    172.17.0.1.33987 > 172.17.0.2.9999: [bad udp cksum 0x5843 -> 0x2b72!] UDP, length 4
13:41:35.467325 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
13:41:35.467348 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

可知數(shù)據(jù)包在主機(jī)上經(jīng)過的設(shè)備次序?yàn)椋簂o ---> docker0 ---> vethc4d778f

數(shù)據(jù)包發(fā)送前與發(fā)送后主機(jī)nat表的對(duì)比如下：

分析：這種訪問方式數(shù)據(jù)包經(jīng)過的設(shè)備與鏈的次序是：OUTPUT ---> POSTROUTING ---> lo設(shè)備 ---> PREROUTING ---> INPUT ---> 用戶空間（docker-proxy）---> OUTPUT ---> POSTROUTING。本地監(jiān)聽8000端口的進(jìn)程是docker-proxy(/usr/bin/docker-proxy -proto udp -host-ip 0.0.0.0 -host-port 8000 -container-ip 172.17.0.2 -container-port 9999)司蔬，查看如下：

從lo設(shè)備流出的數(shù)據(jù)包（127.0.0.1.33089 > 127.0.0.1.8000）邑茄，經(jīng)過PREROUTING鏈與INPUT鏈后被本地8000進(jìn)程docker-proxy處理，處理后數(shù)據(jù)包的源地址俊啼、源端口與目的地址肺缕、目的端口變?yōu)椋?172.17.0.1.33987 > 172.17.0.2.9999；然后數(shù)據(jù)包經(jīng)過docker0后發(fā)往容器設(shè)備vethc4d778f授帕，最后被容器內(nèi)的程序接收到同木，程序的輸出為：addr: ('172.17.0.1', 33987), data: mac

echo mac | nc -s 192.168.84.75 -p 33089 -u 127.0.0.1 8000（程序能夠收到數(shù)據(jù)包）
lo設(shè)備、docker0與容器網(wǎng)卡抓包結(jié)果：

tcpdump -i lo -nn -vv
tcpdump: listening on lo, link-type EN10MB (Ethernet), capture size 262144 bytes
13:46:14.238701 IP (tos 0x0, ttl 64, id 54998, offset 0, flags [DF], proto UDP (17), length 32)
    192.168.84.75.33089 > 127.0.0.1.8000: [bad udp cksum 0x9412 -> 0xfaf3!] UDP, length 4

tcpdump -i docker0 -nn -vv
tcpdump: listening on docker0, link-type EN10MB (Ethernet), capture size 262144 bytes
13:46:14.238980 IP (tos 0x0, ttl 64, id 35537, offset 0, flags [DF], proto UDP (17), length 32)
    172.17.0.1.36234 > 172.17.0.2.9999: [bad udp cksum 0x5843 -> 0x22ab!] UDP, length 4
13:46:19.243339 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
13:46:19.243377 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

tcpdump -i vethc4d778f -nn -vv
tcpdump: listening on vethc4d778f, link-type EN10MB (Ethernet), capture size 262144 bytes
13:46:14.238992 IP (tos 0x0, ttl 64, id 35537, offset 0, flags [DF], proto UDP (17), length 32)
    172.17.0.1.36234 > 172.17.0.2.9999: [bad udp cksum 0x5843 -> 0x22ab!] UDP, length 4
13:46:19.243350 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
13:46:19.243377 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

程序輸出結(jié)果：addr: ('172.17.0.1', 36234), data: mac

四跛十、通過容器ip地址發(fā)送數(shù)據(jù)包

192.168.84.75主機(jī)上容器python的ip地址為172.17.0.2泉手，可通過下面方式獲得：

在主機(jī)上執(zhí)行 echo mac | nc -s 172.17.0.1 -p 42883 -u 172.17.0.2 9999分別對(duì)192.168.84.75主機(jī)的lo、物理網(wǎng)卡（eno16777984）偶器、docker0以及容器的網(wǎng)卡進(jìn)行抓包處理：

tcpdump -i lo -nn -vv
tcpdump: listening on lo, link-type EN10MB (Ethernet), capture size 262144 bytes
無內(nèi)容

tcpdump -i eno16777984 udp port 9999 -nn -vv
tcpdump: listening on eno16777984, link-type EN10MB (Ethernet), capture size 262144 bytes
無內(nèi)容

tcpdump -i docker0 -nn -vv
tcpdump: listening on docker0, link-type EN10MB (Ethernet), capture size 262144 bytes
13:49:05.123460 IP (tos 0x0, ttl 64, id 35538, offset 0, flags [DF], proto UDP (17), length 32)
    172.17.0.1.42883 > 172.17.0.2.9999: [bad udp cksum 0x5843 -> 0x08b2!] UDP, length 4
13:49:10.139337 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
13:49:10.139376 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

tcpdump -i vethc4d778f -nn -vv
tcpdump: listening on vethc4d778f, link-type EN10MB (Ethernet), capture size 262144 bytes
13:49:05.123479 IP (tos 0x0, ttl 64, id 35538, offset 0, flags [DF], proto UDP (17), length 32)
    172.17.0.1.42883 > 172.17.0.2.9999: [bad udp cksum 0x5843 -> 0x08b2!] UDP, length 4
13:49:10.139347 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
13:49:10.139376 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

數(shù)據(jù)包經(jīng)過的設(shè)備依次為：docker0 ---> vethc4d778f

數(shù)據(jù)包發(fā)送前與發(fā)送后主機(jī)nat對(duì)比：

分析： echo mac | nc -s 172.17.0.1 -p 42883 -u 172.17.0.2 9999 產(chǎn)生的數(shù)據(jù)包的源地址斩萌、源端口與目的地址缝裤、目的端口為：172.17.0.1.42883 > 172.17.0.2.9999。實(shí)踐發(fā)現(xiàn)颊郎，數(shù)據(jù)包在用戶空間內(nèi)產(chǎn)生憋飞，然后從OUTPUT鏈流向POSTROUTING鏈。緊接著數(shù)據(jù)包經(jīng)過docker0姆吭，經(jīng)過容器設(shè)備vethc4d778f被程序接收榛做，程序的輸出結(jié)果為：addr: ('172.17.0.1', 42883), data: mac

echo mac | nc -s 172.17.0.2 -p 42883 -u 172.17.0.2 9999
libnsock mksock_bind_addr(): Bind to 172.17.0.2:42883 failed (IOD #1): Cannot assign requested address (99)

docker0與容器網(wǎng)卡抓包結(jié)果：
tcpdump -i docker0 -nn -vv（數(shù)據(jù)包的源地址、源端口已經(jīng)被修改）
tcpdump: listening on docker0, link-type EN10MB (Ethernet), capture size 262144 bytes
13:51:26.388949 IP (tos 0x0, ttl 64, id 35539, offset 0, flags [DF], proto UDP (17), length 32)
    172.17.0.1.54670 > 172.17.0.2.9999: [bad udp cksum 0x5843 -> 0xdaa6!] UDP, length 4
13:51:31.403315 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
13:51:31.403347 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

tcpdump -i vethc4d778f -nn -vv
tcpdump: listening on vethc4d778f, link-type EN10MB (Ethernet), capture size 262144 bytes
13:51:26.388966 IP (tos 0x0, ttl 64, id 35539, offset 0, flags [DF], proto UDP (17), length 32)
    172.17.0.1.54670 > 172.17.0.2.9999: [bad udp cksum 0x5843 -> 0xdaa6!] UDP, length 4
13:51:31.403325 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
13:51:31.403347 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

程序結(jié)果：addr: ('172.17.0.1', 54670), data: mac

echo mac | nc -s 127.0.0.1 -p 42883 -u 172.17.0.2 9999 Ncat: Invalid argument.

實(shí)踐環(huán)境二

Centos7.2 + Docker1.12.6

關(guān)閉firewalld：
systemctl stop firewalld

設(shè)置selinux為Permissive模式：
setenforce 0

/etc/docker/daemon.json內(nèi)容：
{
  "userland-proxy": false
}

創(chuàng)建名為python的容器：
docker run -itd --name python --network bridge -p 8000:9999/udp --entrypoint bash python:2.7.15

容器內(nèi)接收udp數(shù)據(jù)包的程序內(nèi)容為：
#!/usr/bin/env python
# -*- coding: utf-8 -*-

import socket
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
# 綁定端口:
s.bind(('0.0.0.0', 9999))
print 'Bind UDP on 9999...'

while 1:
    # 接收數(shù)據(jù):
    data, addr = s.recvfrom(1024)
    print 'addr: {0}, data: {1}'.format(addr, data)

路由信息：

網(wǎng)橋docker0設(shè)備信息：

對(duì)dockerd進(jìn)程添加 userland-proxy=false 的配置后内狸，基于上述命令創(chuàng)建完容器與之前對(duì)比存在以下三點(diǎn)不同：

docker-proxy進(jìn)程消失
dockerd進(jìn)程監(jiān)聽8000端口
nat表的POSTROUTING鏈多了一條源地址偽裝的規(guī)則
-A POSTROUTING -o docker0 -m addrtype --src-type LOCAL -j MASQUERADE
（將源地址類型為LOCAL且從docker0設(shè)備流出的數(shù)據(jù)包的源地址偽裝成docker0設(shè)備的地址）

一检眯、通過另一臺(tái)主機(jī)向當(dāng)前主機(jī)發(fā)送數(shù)據(jù)包

192.168.84.75主機(jī)上運(yùn)行接收udp數(shù)據(jù)包的程序，運(yùn)行配置基于實(shí)踐環(huán)境昆淡，在192.168.84.79主機(jī)上通過 echo mac | nc -s 192.168.84.79 -p 55902 -u 192.168.84.75 8000 發(fā)送數(shù)據(jù)包锰瘸。

對(duì)192.168.84.79主機(jī)的網(wǎng)卡進(jìn)行抓包處理：
tcpdump -i eno16777984 udp port 8000 -nn -vv
tcpdump: listening on eno16777984, link-type EN10MB (Ethernet), capture size 262144 bytes
13:57:35.975290 IP (tos 0x0, ttl 64, id 58747, offset 0, flags [DF], proto UDP (17), length 32)
    192.168.84.79.55902 > 192.168.84.75.8000: [bad udp cksum 0x2a09 -> 0x0be0!] UDP, length 4

分別對(duì)192.168.84.75主機(jī)的lo、物理網(wǎng)卡（eno16777984）昂灵、docker0以及容器的網(wǎng)卡進(jìn)行抓包處理：
tcpdump -i lo -nn -vv 
tcpdump: listening on lo, link-type EN10MB (Ethernet), capture size 262144 bytes
無內(nèi)容

tcpdump -i eno16777984 udp port 8000 -nn -vv
tcpdump: listening on eno16777984, link-type EN10MB (Ethernet), capture size 262144 bytes
13:57:46.437799 IP (tos 0x0, ttl 64, id 58747, offset 0, flags [DF], proto UDP (17), length 32)
    192.168.84.79.55902 > 192.168.84.75.8000: [udp sum ok] UDP, length 4

tcpdump -i docker0 -nn -vv
tcpdump: listening on docker0, link-type EN10MB (Ethernet), capture size 262144 bytes
13:57:46.437850 IP (tos 0x0, ttl 63, id 58747, offset 0, flags [DF], proto UDP (17), length 32)
    192.168.84.79.55902 > 172.17.0.2.9999: [udp sum ok] UDP, length 4
13:57:51.451317 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
13:57:51.451355 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

tcpdump -i vethcbcc4e9 -nn -vv
tcpdump: listening on vethcbcc4e9, link-type EN10MB (Ethernet), capture size 262144 bytes
13:57:46.437863 IP (tos 0x0, ttl 63, id 58747, offset 0, flags [DF], proto UDP (17), length 32)
    192.168.84.79.55902 > 172.17.0.2.9999: [udp sum ok] UDP, length 4
13:57:51.451327 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
13:57:51.451355 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

數(shù)據(jù)包流經(jīng)設(shè)備的順序依次為：eno16777984（84.79） ---> eno16777984（84.75） ---> docker0 ---> vethcbcc4e9

數(shù)據(jù)包發(fā)送前與發(fā)送后主機(jī)nat表對(duì)比：

分析：84.75主機(jī)eno16777984網(wǎng)卡接收的數(shù)據(jù)包的源地址避凝、源端口與目的地址、目的端口為：192.168.84.79.55902 > 192.168.84.75.8000眨补，此時(shí)數(shù)據(jù)包經(jīng)過PREROUTING鏈被DOCKER鏈的-A DOCKER -p udp -m udp --dport 8000 -j DNAT --to-destination 172.17.0.2:9999規(guī)則匹配管削，數(shù)據(jù)包的源地址、源端口與目的地址撑螺、目的端口變?yōu)椋?92.168.84.79.55902 > 172.17.0.2.9999含思，緊接著數(shù)據(jù)包經(jīng)過路由判斷，判斷其不是發(fā)往本機(jī)的包甘晤，則通過FORWARD到達(dá)POSTROUTING鏈茸俭。POSTROUTING鏈此時(shí)有3條規(guī)則，第一條規(guī)則-A POSTROUTING -o docker0 -m addrtype --src-type LOCAL -j MASQUERADE安皱，表示將源地址類型為LOCAL且從docker0設(shè)備流出的數(shù)據(jù)包的源地址修改為docker0設(shè)備的地址调鬓，這一條不滿足條件；第二條規(guī)則-A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE酌伊，表示對(duì)源地址為172.17網(wǎng)段且不是從docker0設(shè)備流出的包進(jìn)行源地址偽裝腾窝，這一條不滿足條件。第三條規(guī)則-A POSTROUTING -s 172.17.0.2/32 -d 172.17.0.2/32 -p udp -m udp --dport 9999 -j MASQUERADE居砖，表示對(duì)源地址與目的地址均為172.17.0.2且目的端口為udp協(xié)議的9999的數(shù)據(jù)包進(jìn)行源地址偽裝虹脯，第三條同樣不滿足條件。POSTROUTING鏈出來的數(shù)據(jù)包源地址奏候、源端口與目的地址循集、目的端口為：192.168.84.79.55902 > 172.17.0.2.9999，接著數(shù)據(jù)包被發(fā)往docker0蔗草，然后發(fā)往容器設(shè)備vethcbcc4e9咒彤，最后被程序接收疆柔，程序的輸出結(jié)果為：addr: addr: ('192.168.84.79', 55902), data: mac

192.168.84.79主機(jī)上執(zhí)行 echo mac | nc -s 127.0.0.1 -p 55902 -u 192.168.84.75 8000 Ncat: Invalid argument.

二、本機(jī)上通過網(wǎng)卡ip地址發(fā)送數(shù)據(jù)包

在192.168.84.75主機(jī)上執(zhí)行 echo mac | nc -s 192.168.84.75 -p 41003 -u 192.168.84.75 8000
分別lo設(shè)備镶柱、物理網(wǎng)卡（eno16777984）旷档、docker0以及容器網(wǎng)卡進(jìn)行抓包處理：

tcpdump -i lo -nn -vv 
tcpdump: listening on lo, link-type EN10MB (Ethernet), capture size 262144 bytes

tcpdump -i eno16777984 udp port 8000 -nn -vv 
tcpdump: listening on eno16777984, link-type EN10MB (Ethernet), capture size 262144 bytes

tcpdump -i docker0 -nn -vv
tcpdump: listening on docker0, link-type EN10MB (Ethernet), capture size 262144 bytes
14:03:41.089945 IP (tos 0x0, ttl 64, id 64296, offset 0, flags [DF], proto UDP (17), length 32)
    172.17.0.1.41003 > 172.17.0.2.9999: [bad udp cksum 0x5843 -> 0x100a!] UDP, length 4
14:03:46.091328 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
14:03:46.091387 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

tcpdump -i vethcbcc4e9 -nn -vv
tcpdump: listening on vethcbcc4e9, link-type EN10MB (Ethernet), capture size 262144 bytes
14:03:41.089961 IP (tos 0x0, ttl 64, id 64296, offset 0, flags [DF], proto UDP (17), length 32)
    172.17.0.1.41003 > 172.17.0.2.9999: [bad udp cksum 0x5843 -> 0x100a!] UDP, length 4
14:03:46.091343 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
14:03:46.091387 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

可知僵蛛，數(shù)據(jù)包僅經(jīng)過docker0與容器設(shè)備vethcbcc4e9

數(shù)據(jù)包發(fā)送前與發(fā)送后主機(jī)nat表對(duì)比：

分析：echo mac | nc -s 192.168.84.75 -p 41003 -u 192.168.84.75 8000 發(fā)出的數(shù)據(jù)包的源地址兄渺、源端口與目的地址、目的端口為：192.168.84.75.41003 > 192.168.84.75.8000刹碾，內(nèi)核將數(shù)據(jù)包發(fā)送給本地進(jìn)程dockerd處理故觅，dockerd只負(fù)責(zé)接收而并沒有做任何的處理厂庇，此時(shí)數(shù)據(jù)包經(jīng)過OUTPUT鏈被DOCKER鏈的規(guī)則 -A DOCKER -p udp -m udp --dport 8000 -j DNAT --to-destination 172.17.0.2:9999（將udp協(xié)議、目的端口為8000的數(shù)據(jù)包的目的地址與端口修改為172.17.0.2.9999）匹配输吏，此時(shí)數(shù)據(jù)包的源地址权旷、源端口與目的地址、目的端口變?yōu)椋?92.168.84.75.41003 > 172.17.0.2.9999评也。緊接著數(shù)據(jù)包經(jīng)過POSTROUTING鏈炼杖，被規(guī)則 -A POSTROUTING -o docker0 -m addrtype --src-type LOCAL -j MASQUERADE匹配灭返，這條規(guī)則表示從docker0設(shè)備流出且源地址類型為LOCAL的數(shù)據(jù)包將其源地址修改為docker0的地址（這里是172.17.0.1）盗迟，為什么數(shù)據(jù)包（192.168.84.75.41003 > 172.17.0.2.9999）能夠匹配到這條規(guī)則呢，不妨分析一下熙含，首先數(shù)據(jù)包的源地址為192.168.84.75符合 -m addrtype --src-type LOCAL定義罚缕；為什么數(shù)據(jù)包能夠滿足 -o docker0的定義呢，數(shù)據(jù)包分明還沒有經(jīng)過docker0設(shè)備怎静，此處為何就匹配到了從docker0設(shè)備流出的規(guī)則呢邮弹？按照自身理解，數(shù)據(jù)包將會(huì)流入docker0設(shè)備蚓聘，當(dāng)然也會(huì)從docker0設(shè)備流出腌乡，但流入docker0設(shè)備在前從docker0設(shè)備流出在后，因此這里想要將-A POSTROUTING -o docker0 -m addrtype --src-type LOCAL -j MASQUERADE規(guī)則換成-A POSTROUTING -i docker0 -m addrtype --src-type LOCAL -j MASQUERADE夜牡，結(jié)果報(bào)出錯(cuò)誤“iptables v1.4.21: Can't use -i with POSTROUTING”与纽，即POSTROUTING鏈只能通過 -o 指定數(shù)據(jù)包流出的設(shè)備，只要數(shù)據(jù)包經(jīng)過的下一個(gè)設(shè)備會(huì)是docker0且會(huì)從docker0流出則此處會(huì)匹配到這條規(guī)則塘装，此時(shí)數(shù)據(jù)包的源地址急迂、源端口與目的地址、目的端口變?yōu)椋?72.17.0.1.41003 > 172.17.0.2.9999蹦肴。接著數(shù)據(jù)包經(jīng)過docker0設(shè)備發(fā)往容器設(shè)備vethcbcc4e9僚碎，進(jìn)而被程序接收，程序的輸出結(jié)果為：addr: ('172.17.0.1', 41003), data: mac

echo mac | nc -s 127.0.0.1 -p 41003 -u 192.168.84.75 8000

docker0與容器網(wǎng)卡抓包結(jié)果：
tcpdump -i docker0 -nn -vv（數(shù)據(jù)包的源地址已經(jīng)被修改）
tcpdump: listening on docker0, link-type EN10MB (Ethernet), capture size 262144 bytes
14:06:19.694983 IP (tos 0x0, ttl 64, id 64297, offset 0, flags [DF], proto UDP (17), length 32)
    172.17.0.1.41003 > 172.17.0.2.9999: [bad udp cksum 0x5843 -> 0x100a!] UDP, length 4
14:06:24.699338 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
14:06:24.699375 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

tcpdump -i vethcbcc4e9 -nn -vv
tcpdump: listening on vethcbcc4e9, link-type EN10MB (Ethernet), capture size 262144 bytes
14:06:19.695000 IP (tos 0x0, ttl 64, id 64297, offset 0, flags [DF], proto UDP (17), length 32)
    172.17.0.1.41003 > 172.17.0.2.9999: [bad udp cksum 0x5843 -> 0x100a!] UDP, length 4
14:06:24.699348 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
14:06:24.699375 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

程序結(jié)果：addr: ('172.17.0.1', 41003), data: mac

三阴幌、本機(jī)上通過127.0.0.1發(fā)送數(shù)據(jù)包

在192.168.84.75主機(jī)上執(zhí)行 echo mac | nc -s 127.0.0.1 -p 44818 -u 127.0.0.1 8000
分別主機(jī)的lo設(shè)備勺阐、物理網(wǎng)卡（eno16777984）卷中、docker0以及容器網(wǎng)卡進(jìn)行抓包處理：

tcpdump -i lo -nn -vv 
tcpdump: listening on lo, link-type EN10MB (Ethernet), capture size 262144 bytes

tcpdump -i eno16777984 udp port 8000 -nn -vv 
tcpdump: listening on eno16777984, link-type EN10MB (Ethernet), capture size 262144 bytes

tcpdump -i docker0 -nn -vv
tcpdump: listening on docker0, link-type EN10MB (Ethernet), capture size 262144 bytes
14:08:58.946048 IP (tos 0x0, ttl 64, id 54997, offset 0, flags [DF], proto UDP (17), length 32)
    172.17.0.1.44818 > 172.17.0.2.9999: [bad udp cksum 0x5843 -> 0x0123!] UDP, length 4
14:09:03.947336 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
14:09:03.947379 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

tcpdump -i vethcbcc4e9 -nn -vv
tcpdump: listening on vethcbcc4e9, link-type EN10MB (Ethernet), capture size 262144 bytes
14:08:58.946065 IP (tos 0x0, ttl 64, id 54997, offset 0, flags [DF], proto UDP (17), length 32)
    172.17.0.1.44818 > 172.17.0.2.9999: [bad udp cksum 0x5843 -> 0x0123!] UDP, length 4
14:09:03.947349 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
14:09:03.947379 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

數(shù)據(jù)包經(jīng)過的設(shè)備依次為：docker0 ---> vethcbcc4e9

數(shù)據(jù)包發(fā)送前與發(fā)送后nat表規(guī)則對(duì)比：

分析：echo mac | nc -s 127.0.0.1 -p 44818 -u 127.0.0.1 8000 產(chǎn)生的數(shù)據(jù)包的源地址、源端口與目的地址皆看、目的端口為：127.0.0.1.44818 > 127.0.0.1.8000仓坞。這種方式數(shù)據(jù)包僅經(jīng)過OUTPUT與POSTROUTING鏈，當(dāng)數(shù)據(jù)包經(jīng)過OUTPUT鏈時(shí)被DOCKER鏈的-A DOCKER -p udp -m udp --dport 8000 -j DNAT --to-destination 172.17.0.2:9999規(guī)則匹配腰吟，此時(shí)數(shù)據(jù)包的源地址无埃、源端口與目的地址、目的端口變?yōu)椋?27.0.0.1.44818 > 172.17.0.2.9999毛雇。緊接著數(shù)據(jù)包在經(jīng)過POSTROUTING鏈時(shí)被-A POSTROUTING -o docker0 -m addrtype --src-type LOCAL -j MASQUERADE規(guī)則匹配嫉称，此時(shí)數(shù)據(jù)包的源地址、端口與目的地址灵疮、端口變?yōu)椋?72.17.0.1.44818 > 172.17.0.2.9999织阅。數(shù)據(jù)包被發(fā)往docker0，然后發(fā)往容器設(shè)備vethcbcc4e9震捣，最后被程序接收荔棉，程序的輸出結(jié)果為：addr: ('172.17.0.1', 44818), data: mac

四、通過容器ip地址發(fā)送數(shù)據(jù)包

192.168.84.75主機(jī)上容器python的ip地址為172.17.0.2蒿赢，可通過下面方式獲得：

在主機(jī)上執(zhí)行 echo mac | nc -s 172.17.0.1 -p 54877 -u 172.17.0.2 9999润樱，分別對(duì)主機(jī)的lo、物理網(wǎng)卡（eno16777984）羡棵、docker0以及容器網(wǎng)卡進(jìn)行抓包處理：

tcpdump -i lo -nn -vv
tcpdump: listening on lo, link-type EN10MB (Ethernet), capture size 262144 bytes
無內(nèi)容

tcpdump -i eno16777984 udp port 9999 -nn -vv
tcpdump: listening on eno16777984, link-type EN10MB (Ethernet), capture size 262144 bytes
無內(nèi)容

tcpdump -i docker0 -nn -vv
tcpdump: listening on docker0, link-type EN10MB (Ethernet), capture size 262144 bytes
14:10:58.373599 IP (tos 0x0, ttl 64, id 35536, offset 0, flags [DF], proto UDP (17), length 32)
    172.17.0.1.54877 > 172.17.0.2.9999: [bad udp cksum 0x5843 -> 0xd9d7!] UDP, length 4
14:11:03.387315 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
14:11:03.387349 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

tcpdump -i vethcbcc4e9 -nn -vv
tcpdump: listening on vethcbcc4e9, link-type EN10MB (Ethernet), capture size 262144 bytes
14:10:58.373617 IP (tos 0x0, ttl 64, id 35536, offset 0, flags [DF], proto UDP (17), length 32)
    172.17.0.1.54877 > 172.17.0.2.9999: [bad udp cksum 0x5843 -> 0xd9d7!] UDP, length 4
14:11:03.387325 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
14:11:03.387349 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

數(shù)據(jù)包經(jīng)過的設(shè)備依次為：docker0 ---> vethcbcc4e9

數(shù)據(jù)包發(fā)送前與發(fā)送后主機(jī)nat對(duì)比：

分析：echo mac | nc -s 172.17.0.1 -p 54877 -u 172.17.0.2 9999 產(chǎn)生的數(shù)據(jù)包的源地址壹若、源端口與目的地址、目的端口為：172.17.0.1.54877 > 172.17.0.2.9999皂冰。實(shí)踐發(fā)現(xiàn)店展，數(shù)據(jù)包從用戶空間發(fā)出僅經(jīng)過OUTPUT鏈與POSTROUTING鏈。數(shù)據(jù)包從OUTPUT鏈流向POSTROUTING鏈時(shí)秃流，被-A POSTROUTING -o docker0 -m addrtype --src-type LOCAL -j MASQUERADE規(guī)則匹配赂蕴，將源地址類型為LOCAL、從docker0設(shè)備流出的數(shù)據(jù)包的源地址偽裝成docker0設(shè)備的地址172.17.0.1舶胀，此時(shí)數(shù)據(jù)包的源地址概说、源端口與目的地址、目的端口變?yōu)椋?72.17.0.1.54877 > 172.17.0.2.9999峻贮。緊接著數(shù)據(jù)包經(jīng)過docker0席怪，經(jīng)過容器設(shè)備vethcbcc4e9被程序接收，程序的輸出結(jié)果為：addr: ('172.17.0.1', 54877), data: mac

echo mac | nc -s 172.17.0.2 -p 54877 -u 172.17.0.2 9999
libnsock mksock_bind_addr(): Bind to 172.17.0.2:54877 failed (IOD #1): Cannot assign requested address (99)

docker0與容器網(wǎng)卡抓包結(jié)果：
tcpdump -i docker0 -nn -vv
tcpdump: listening on docker0, link-type EN10MB (Ethernet), capture size 262144 bytes
14:13:04.045159 IP (tos 0x0, ttl 64, id 35537, offset 0, flags [DF], proto UDP (17), length 32)
    172.17.0.1.35950 > 172.17.0.2.9999: [bad udp cksum 0x5843 -> 0x23c7!] UDP, length 4
14:13:09.051330 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
14:13:09.051367 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

tcpdump -i vethcbcc4e9 -nn -vv
tcpdump: listening on vethcbcc4e9, link-type EN10MB (Ethernet), capture size 262144 bytes
14:13:04.045176 IP (tos 0x0, ttl 64, id 35537, offset 0, flags [DF], proto UDP (17), length 32)
    172.17.0.1.35950 > 172.17.0.2.9999: [bad udp cksum 0x5843 -> 0x23c7!] UDP, length 4
14:13:09.051340 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
14:13:09.051367 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

程序結(jié)果：addr: ('172.17.0.1', 35950), data: mac

echo mac | nc -s 127.0.0.1 -p 54877 -u 172.17.0.2 9999

docker0與容器網(wǎng)卡抓包結(jié)果：
tcpdump -i docker0 -nn -vv（數(shù)據(jù)包的源地址被規(guī)則 -A POSTROUTING -o docker0 -m addrtype --src-type LOCAL -j MASQUERADE 修改為docker0地址172.17.0.1）
tcpdump: listening on docker0, link-type EN10MB (Ethernet), capture size 262144 bytes
14:15:08.717851 IP (tos 0x0, ttl 64, id 35538, offset 0, flags [DF], proto UDP (17), length 32)
    172.17.0.1.54877 > 172.17.0.2.9999: [bad udp cksum 0x5843 -> 0xd9d7!] UDP, length 4
14:15:13.723315 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
14:15:13.723350 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

tcpdump -i vethcbcc4e9 -nn -vv
tcpdump: listening on vethcbcc4e9, link-type EN10MB (Ethernet), capture size 262144 bytes
14:15:08.717868 IP (tos 0x0, ttl 64, id 35538, offset 0, flags [DF], proto UDP (17), length 32)
    172.17.0.1.54877 > 172.17.0.2.9999: [bad udp cksum 0x5843 -> 0xd9d7!] UDP, length 4
14:15:13.723326 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
14:15:13.723350 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

程序結(jié)果：addr: ('172.17.0.1', 54877), data: mac

去除POSTROUTING鏈的規(guī)則 -A POSTROUTING -o docker0 -m addrtype --src-type LOCAL -j MASQUERADE

執(zhí)行 echo mac | nc -s 127.0.0.1 -p 54877 -u 172.17.0.2 9999
docker0與容器網(wǎng)卡抓包結(jié)果：

tcpdump -i docker0 -nn -vv
tcpdump: listening on docker0, link-type EN10MB (Ethernet), capture size 262144 bytes
14:17:35.597021 IP (tos 0x0, ttl 64, id 35539, offset 0, flags [DF], proto UDP (17), length 32)
    127.0.0.1.54877 > 172.17.0.2.9999: [bad udp cksum 0x2b32 -> 0x06e9!] UDP, length 4
14:17:40.603314 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
14:17:40.603346 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

tcpdump -i vethcbcc4e9 -nn -vv
tcpdump: listening on vethcbcc4e9, link-type EN10MB (Ethernet), capture size 262144 bytes
14:17:35.597050 IP (tos 0x0, ttl 64, id 35539, offset 0, flags [DF], proto UDP (17), length 32)
    127.0.0.1.54877 > 172.17.0.2.9999: [bad udp cksum 0x2b32 -> 0x06e9!] UDP, length 4
14:17:40.603323 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
14:17:40.603346 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

程序結(jié)果：空纤控，即程序沒有接收到數(shù)據(jù)包

添加將udp協(xié)議挂捻、源地址類型為LOCAL的數(shù)據(jù)包的源地址修改為主機(jī)地址192.168.84.75

執(zhí)行 echo mac | nc -s 127.0.0.1 -p 54877 -u 172.17.0.2 9999
docker0與容器網(wǎng)卡veth1a4e47a抓包結(jié)果：

tcpdump -i docker0 -nn -vv
tcpdump: listening on docker0, link-type EN10MB (Ethernet), capture size 262144 bytes
14:20:10.098644 IP (tos 0x0, ttl 64, id 35540, offset 0, flags [DF], proto UDP (17), length 32)
    192.168.84.75.54877 > 172.17.0.2.9999: [bad udp cksum 0xc124 -> 0x70f6!] UDP, length 4
14:20:15.115338 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
14:20:15.115377 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

tcpdump -i vethcbcc4e9 -nn -vv
tcpdump: listening on vethcbcc4e9, link-type EN10MB (Ethernet), capture size 262144 bytes
14:20:10.098661 IP (tos 0x0, ttl 64, id 35540, offset 0, flags [DF], proto UDP (17), length 32)
    192.168.84.75.54877 > 172.17.0.2.9999: [bad udp cksum 0xc124 -> 0x70f6!] UDP, length 4
14:20:15.115350 ARP, Ethernet (len 6), IPv4 (len 4), Request who-has 172.17.0.2 tell 172.17.0.1, length 28
14:20:15.115377 ARP, Ethernet (len 6), IPv4 (len 4), Reply 172.17.0.2 is-at 02:42:ac:11:00:02, length 28

程序結(jié)果：addr: ('192.168.84.75', 54877), data: mac

總結(jié)

Centos7中更改數(shù)據(jù)包源地址的操作即SNAT可以在INPUT鏈與POSTROUTING鏈兩個(gè)地方操作。Docker在INPUT鏈中沒有定義SNAT規(guī)則船万，全部的SNAT操作均定義在POSTROUTING鏈中刻撒。所以對(duì)Docker數(shù)據(jù)包的SNAT的研究只需關(guān)注POSTROUTING鏈即可骨田。我們自己產(chǎn)品對(duì)docker的使用屬于上述實(shí)驗(yàn)一的范圍，即是將userland設(shè)置為true声怔，包括以下幾種情況：

如果機(jī)器的iptables規(guī)則沒有做額外的配置态贤，通過外主機(jī)向本機(jī)訪問或在本機(jī)上通過ip地址訪問，數(shù)據(jù)包的源地址不會(huì)被修改
本機(jī)上通過127.0.0.1訪問的數(shù)據(jù)包的源地址會(huì)被docker-proxy進(jìn)程修改醋火，因此本機(jī)訪問若想要數(shù)據(jù)包的源地址不被修改悠汽，需使用本機(jī)ip地址訪問
通過上面實(shí)踐可發(fā)現(xiàn)，源地址為127.0.0.1的數(shù)據(jù)包在多數(shù)場景下不能成功發(fā)送出去芥驳，能夠發(fā)送出去且能被容器內(nèi)程序接收到的數(shù)據(jù)包在容器網(wǎng)卡或docker0那一層其源地址127.0.0.1已經(jīng)被修改了柿冲；即容器內(nèi)程序接收到的數(shù)據(jù)包的源地址不可能是127.0.0.1。源地址為127.0.0.1的數(shù)據(jù)包在傳輸過程中意義不大兆旬，因?yàn)闊o法根據(jù)源地址127.0.0.1找到發(fā)包的設(shè)備
數(shù)據(jù)包在主機(jī)與主機(jī)或主機(jī)與容器之間傳輸時(shí)假抄，數(shù)據(jù)包的源地址多數(shù)情況下會(huì)發(fā)生改變，如果某功能是以數(shù)據(jù)包源IP地址作為來源主機(jī)的判斷是不可靠的丽猬；因?yàn)榭赡苋〉臄?shù)據(jù)包是處于中間狀態(tài)的包宿饱，這類包的源IP地址可能已被修改，這樣就會(huì)導(dǎo)致來源主機(jī)判斷的錯(cuò)誤脚祟。

最后編輯于：2019.01.25 09:23:21

?著作權(quán)歸作者所有,轉(zhuǎn)載或內(nèi)容合作請(qǐng)聯(lián)系作者

人面猴
序言：七十年代末谬以，一起剝皮案震驚了整個(gè)濱河市，隨后出現(xiàn)的幾起案子愚铡，更是在濱河造成了極大的恐慌蛉签，老刑警劉巖胡陪，帶你破解...
沈念sama閱讀 212,454評(píng)論 6贊 493
死咒
序言：濱河連續(xù)發(fā)生了三起死亡事件沥寥，死亡現(xiàn)場離奇詭異，居然都是意外死亡柠座，警方通過查閱死者的電腦和手機(jī)邑雅，發(fā)現(xiàn)死者居然都...
沈念sama閱讀 90,553評(píng)論 3贊 385
救了他兩次的神仙讓他今天三更去死
文/潘曉璐我一進(jìn)店門，熙熙樓的掌柜王于貴愁眉苦臉地迎上來妈经，“玉大人淮野，你說我怎么就攤上這事〈蹬荩” “怎么了骤星？”我有些...
開封第一講書人閱讀 157,921評(píng)論 0贊 348
道士緝兇錄：失蹤的賣姜人
文/不壞的土叔我叫張陵，是天一觀的道長爆哑。經(jīng)常有香客問我洞难，道長，這世上最難降的妖魔是什么揭朝？我笑而不...
開封第一講書人閱讀 56,648評(píng)論 1贊 284
?港島之戀（遺憾婚禮）
正文為了忘掉前任队贱，我火速辦了婚禮色冀，結(jié)果婚禮上，老公的妹妹穿的比我還像新娘柱嫌。我一直安慰自己锋恬，他們只是感情好，可當(dāng)我...
茶點(diǎn)故事閱讀 65,770評(píng)論 6贊 386
惡毒庶女頂嫁案：這布局不是一般人想出來的
文/花漫我一把揭開白布编丘。她就那樣靜靜地躺著与学，像睡著了一般。火紅的嫁衣襯著肌膚如雪嘉抓。梳的紋絲不亂的頭發(fā)上癣防，一...
開封第一講書人閱讀 49,950評(píng)論 1贊 291
城市分裂傳說
那天，我揣著相機(jī)與錄音掌眠，去河邊找鬼蕾盯。笑死，一個(gè)胖子當(dāng)著我的面吹牛蓝丙，可吹牛的內(nèi)容都是我干的级遭。我是一名探鬼主播，決...
沈念sama閱讀 39,090評(píng)論 3贊 410
雙鴛鴦連環(huán)套：你想象不到人心有多黑
文/蒼蘭香墨我猛地睜開眼渺尘，長吁一口氣：“原來是場噩夢啊……” “哼挫鸽！你這毒婦竟也來了？” 一聲冷哼從身側(cè)響起鸥跟，我...
開封第一講書人閱讀 37,817評(píng)論 0贊 268
萬榮殺人案實(shí)錄
序言：老撾萬榮一對(duì)情侶失蹤丢郊，失蹤者是張志新（化名）和其女友劉穎，沒想到半個(gè)月后医咨，有當(dāng)?shù)厝嗽跇淞掷锇l(fā)現(xiàn)了一具尸體枫匾，經(jīng)...
沈念sama閱讀 44,275評(píng)論 1贊 303
?護(hù)林員之死
正文獨(dú)居荒郊野嶺守林人離奇死亡，尸身上長有42處帶血的膿包…… 初始之章·張勛以下內(nèi)容為張勛視角年9月15日...
茶點(diǎn)故事閱讀 36,592評(píng)論 2贊 327
?白月光啟示錄
正文我和宋清朗相戀三年拟淮，在試婚紗的時(shí)候發(fā)現(xiàn)自己被綠了干茉。大學(xué)時(shí)的朋友給我發(fā)了我未婚夫和他白月光在一起吃飯的照片。...
茶點(diǎn)故事閱讀 38,724評(píng)論 1贊 341
活死人
序言：一個(gè)原本活蹦亂跳的男人離奇死亡很泊，死狀恐怖角虫，靈堂內(nèi)的尸體忽然破棺而出，到底是詐尸還是另有隱情委造，我是刑警寧澤戳鹅，帶...
沈念sama閱讀 34,409評(píng)論 4贊 333
?日本核電站爆炸內(nèi)幕
正文年R本政府宣布，位于F島的核電站昏兆，受9級(jí)特大地震影響枫虏，放射性物質(zhì)發(fā)生泄漏。R本人自食惡果不足惜，卻給世界環(huán)境...
茶點(diǎn)故事閱讀 40,052評(píng)論 3贊 316
男人毒藥：我在死后第九天來索命
文/蒙蒙一模软、第九天我趴在偏房一處隱蔽的房頂上張望伟骨。院中可真熱鬧，春花似錦燃异、人聲如沸携狭。這莊子的主人今日做“春日...
開封第一講書人閱讀 30,815評(píng)論 0贊 21
一樁弒父案回俐，背后竟有這般陰謀
文/蒼蘭香墨我抬頭看了看天上的太陽逛腿。三九已至，卻和暖如春仅颇，著一層夾襖步出監(jiān)牢的瞬間单默，已是汗流浹背。一陣腳步聲響...
開封第一講書人閱讀 32,043評(píng)論 1贊 266
情欲美人皮
我被黑心中介騙來泰國打工忘瓦，沒想到剛下飛機(jī)就差點(diǎn)兒被人妖公主榨干…… 1. 我叫王不留搁廓，地道東北人。一個(gè)月前我還...
沈念sama閱讀 46,503評(píng)論 2贊 361
代替公主和親
正文我出身青樓耕皮，卻偏偏與公主長得像境蜕，于是被迫代替她去往敵國和親。傳聞我的和親對(duì)象是個(gè)殘疾皇子凌停，可洞房花燭夜當(dāng)晚...
茶點(diǎn)故事閱讀 43,627評(píng)論 2贊 350

發(fā)往Docker容器內(nèi)的數(shù)據(jù)包源地址被修改的研究

引言

實(shí)踐環(huán)境一

實(shí)踐環(huán)境二

總結(jié)

推薦閱讀更多精彩內(nèi)容