問題是這樣的:我有一個比較大的docker鏡像文件, 在執(zhí)行 load 時提示“no space left on device”的錯誤惹骂,而實際使用的
/var
目錄空間是 60GB 充足的魂拦,經(jīng)過升級內(nèi)核測試和docker服務(wù)端啟動參數(shù)調(diào)整毛仪,終于解決了這個問題。下面就是我要分享給大家的處理過程芯勘。希望你不要遇到這樣的問題哈(鏡像文件太大真的不可思議!!!)箱靴。
系統(tǒng)運(yùn)行環(huán)境:
主機(jī)CPU架構(gòu) :X86_64
操作系統(tǒng)版本: RHEL 6.5
Kernel內(nèi)核版本:2.6.32-431.el6.x86_64
Docker 版本: docker-engine-1.7.1-1.el6.x86_64 graphdriver=devicemapper version=1.7.1
問題說明
- 本地有一個15GB的tensorflow鏡像tar文件包, load 到服務(wù)端的時候報錯導(dǎo)致無法成功加載這個鏡像tar包文件荷愕,具體錯誤信息如下:
$ cat /var/log/docker
time="2017-08-16T17:26:19.036130760+08:00" level=info msg="Loading containers: done."
time="2017-08-16T17:26:19.036175169+08:00" level=info msg="Daemon has completed initialization"
time="2017-08-16T17:26:19.036239870+08:00" level=info msg="Docker daemon" commit=786b29d execdriver=native-0.2 graphdriver=devicemapper version=1.7.1
time="2017-08-16T17:26:51.929543993+08:00" level=info msg="POST /v1.19/images/load"
time="2017-08-16T17:31:03.197653849+08:00" level=error msg="Handler for POST /images/load returned error: ApplyLayer exit status 1 stdout: stderr: write /root/.cache/bazel/_bazel_root/f8d1071c69ea316497c31e40fe01608c/execroot/serving/bazel-out/local-fastbuild/bin/tensorflow_serving/servables/tensorflow/simple_servers_test: no space left on device"
time="2017-08-16T17:31:03.197895155+08:00" level=error msg="HTTP Error" err="ApplyLayer exit status 1 stdout: stderr: write /root/.cache/bazel/_bazel_root/f8d1071c69ea316497c31e40fe01608c/execroot/serving/bazel-out/local-fastbuild/bin/tensorflow_serving/servables/tensorflow/simple_servers_test: no space left on device" statusCode=500
time="2017-08-16T17:38:30.359166697+08:00" level=info msg="Processing signal 'terminated'"
\nThu Aug 17 09:40:43 CST 2017\n
解決方案
- 調(diào)整docker服務(wù)端啟動參數(shù)(調(diào)整含義可以看man docker文檔解釋)衡怀,具體操作如下:
vi /etc/sysconfig/docker
## 將 other_args = "" 更改為如下內(nèi)容
other_args="-s=devicemapper --storage-opt dm.basesize=60G --storage-opt dm.loopmetadatasize=10G"
- 查看調(diào)整后的結(jié)果
work@host01:/docker/work>
work@host01:/docker/work>ps -ef | grep docker|grep -v grep
/usr/bin/docker -d -s=devicemapper --storage-opt dm.basesize=60G --storage-opt dm.loopmetadatasize=10G
work@host01:/docker/work>
work@host01:/docker/work>docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
tf/master 1.4 db45227e3209 2 weeks ago 15.04 GB