參考文獻(xiàn)
serving-ml-quickly-with-tensorflow-serving-and-docker
tensorflow-serving-of-multiple-ml-models-simultaneously-to-a-rest-api-python-client
本文先部署單個(gè)模型，然后部署多個(gè)模型以及同一模型的多個(gè)版本
各組件版本信息：
tensorflow/serving:2.2.0-rc2
Docker version 19.03.8
部署單個(gè)模型
- 從這個(gè)鏈接 下載模型文件并解壓。
解壓后的目錄 ：
└─resnet_v2_fp32_savedmodel_NHWC_jpg
└─1538687457（版本號）
│ saved_model.pb（模型pb文件）
│
└─variables
variables.data-00000-of-00001
variables.index
- 啟動(dòng)tensorflow serving：
docker run --rm -p 8501:8501 --name tfserving_resnet --mount type=bind,source=E:/dev/tensorflow/tensorflow-serving/model-files/resnet_v2_fp32_savedmodel_NHWC_jpg,target=/models/resnet -e MODEL_NAME=resnet -t tensorflow/serving:2.2.0-rc2 &
然后就可以發(fā)起http調(diào)用了褐耳。
調(diào)用方式有g(shù)RPC和restful兩種柳譬，詳見這個(gè)鏈接
部署多個(gè)模型
- 先準(zhǔn)備好2個(gè)模型文件棒厘。這里為了方便蕊蝗，我直接把上一個(gè)模型復(fù)制一份：
├─resnet_v2_fp32_savedmodel_NHWC_jpg
│ ├─1538687457
│ └─variables
│
└─resnet_v2_fp32_savedmodel_NHWC_jpg_2
└─1538687457
└─variables
2.編寫model_config.config盯捌，將2個(gè)模型命名為r1和r2烁巫，同時(shí)base_path也變了：
model_config_list: {
config: {
name: "r1",
base_path: "/models/resnet1",
model_platform: "tensorflow"
},
config: {
name: "r2",
base_path: "/models/resnet2",
model_platform: "tensorflow"
}
}
- 類似部署單個(gè)模型署隘，通過docker啟動(dòng)多個(gè)模型
docker run --rm -p 8500:8500 -p 8501:8501
--mount type=bind,source=E:/dev/tensorflow/tensorflow-serving/model-files/resnet_v2_fp32_savedmodel_NHWC_jpg,target=/models/resnet1
--mount type=bind,source=E:/dev/tensorflow/tensorflow-serving/model-files/resnet_v2_fp32_savedmodel_NHWC_jpg_2,target=/models/resnet2
--mount type=bind,source=E:/dev/tensorflow/tensorflow-serving/model-files/model_config.config,target=/models/model_config.config
-t tensorflow/serving:2.2.0-rc2 --model_config_file=/models/model_config.config
通過日志看到2個(gè)模型都已經(jīng)啟動(dòng)成功：
Successfully loaded servable version {name: r2 version: 1538687457}
Successfully loaded servable version {name: r1 version: 1538687457}
4、測試調(diào)用
模型1：
http://localhost:8501/v1/models/r1:predict
http://localhost:8501/v1/models/r1/metadata
模型2：
http://localhost:8501/v1/models/r2:predict
http://localhost:8501/v1/models/r2/metadata
同一模型的多個(gè)版本
同樣亚隙，簡單處理磁餐，把模型r1復(fù)制一份，然后版本號加1：
├─resnet_v2_fp32_savedmodel_NHWC_jpg
│ ├─1538687457
│ │ └─variables
│ └─1538687458（版本號加1）
│ └─variables
└─resnet_v2_fp32_savedmodel_NHWC_jpg_2
└─1538687457
└─variables
此時(shí)阿弃，tensorflow-serving自動(dòng)發(fā)現(xiàn)了1538687458這個(gè)版本诊霹，觸發(fā)加載，同時(shí)unload舊版本：
2020-05-23 06:51:49.453468: I tensorflow_serving/core/loader_harness.cc:87] Successfully loaded servable version {name: r1 version: 1538687458}
2020-05-23 06:51:49.453587: I tensorflow_serving/core/loader_harness.cc:138] Quiescing servable version {name: r1 version: 1538687457}
2020-05-23 06:51:49.453628: I tensorflow_serving/core/loader_harness.cc:145] Done quiescing servable version {name: r1 version: 1538687457}
2020-05-23 06:51:49.453891: I tensorflow_serving/core/loader_harness.cc:120] Unloading servable version {name: r1 version: 1538687457}
2020-05-23 06:51:49.488530: I ./tensorflow_serving/core/simple_loader.h:363] Calling MallocExtension_ReleaseToSystem() after servable unload with 123549680
2020-05-23 06:51:49.488595: I tensorflow_serving/core/loader_harness.cc:128] Done unloading servable version {name: r1 version: 1538687457}
通過查看r1的metadata發(fā)現(xiàn)版本號已經(jīng)更新：
http://localhost:8501/v1/models/r1/metadata 輸出：
{"name": "r1", "signature_name": "", "version": "1538687458"}......
總結(jié)
通過上面的實(shí)驗(yàn)渣淳，我們發(fā)現(xiàn)脾还，tensorflow-serving支持部署多個(gè)模型的多個(gè)版本，并且模型有了新版本后入愧，還能自動(dòng)加載最新版本鄙漏，因此在生產(chǎn)環(huán)境部署和更新模型都很方便。
