客戶端調(diào)用的兩種方式
gRPC
參考文獻(xiàn)
如何使用tensorflow-serving部署模型-segmentation
英文版
以語義分割為例荧嵌，調(diào)用代碼還是比較簡單的口糕，核心是三個(gè)變量托呕，圖像充甚、高度乞巧、寬度涨椒，要以tensor的形式傳入：
# define the image url to be sent to the model for
predictionimage_url = "https://www.publicdomainpictures.net/pictures/60000/nahled/bird-1382034603Euc.jpg"
response = requests.get(image_url)
image = np.array(Image.open(StringIO(response.content)))
height = image.shape[0]
width = image.shape[1]
print("Image shape:", image.shape)
# create the RPC stub
#***IP和端口***
channel = implementations.insecure_channel(host, int(port))
stub = prediction_service_pb2.beta_create_PredictionService_stub(channel)
# create the request object and set the name and signature_name params
request = predict_pb2.PredictRequest()
request.model_spec.name = 'deeplab'
request.model_spec.signature_name = 'predict_images'
# fill in the request object with the necessary data
request.inputs['images'].CopyFrom(
tf.contrib.util.make_tensor_proto(image.astype(dtype=np.float32), shape=[1, height, width, 3]))
request.inputs['height'].CopyFrom(tf.contrib.util.make_tensor_proto(height, shape=[1]))
request.inputs['width'].CopyFrom(tf.contrib.util.make_tensor_proto(width, shape=[1]))
# sync requests
result_future = stub.Predict(request, 30.)
# For async requests
#result_future = stub.Predict.future(request, 10.)
# result_future = result_future.result()
restful
參考文獻(xiàn)
使用docker部署tensorflow-serving
文獻(xiàn)中客戶端調(diào)用的完整的源碼在這里
restful代碼非常簡單，我覺得這種方式好：
SERVER_URL = 'http://localhost:8501/v1/models/resnet:predict'
# Download the image
dl_request = requests.get(IMAGE_URL, stream=True)
dl_request.raise_for_status()
# Compose a JSON Predict request (send JPEG image in base64).
jpeg_bytes = base64.b64encode(dl_request.content).decode('utf-8')
predict_request = '{"instances" : [{"b64": "%s"}]}' % jpeg_bytes
response = requests.post(SERVER_URL, data=predict_request)
response.raise_for_status()
prediction = response.json()['predictions'][0]
print(prediction['classes'])
#這里不太好的地方是绽媒，輸出的分類是一個(gè)編號（數(shù)字）蚕冬，不是具體的label如cat/dog這種，可讀性較差是辕，而且不知道去哪里找這個(gè)標(biāo)簽
restful還有其他一些比較有用的api囤热，如
- 查看模型狀態(tài)
model_metadata = requests.get('http://localhost:8501/v1/models/resnet', stream=True)
model_metadata.raise_for_status()
print(model_metadata.json())
#輸出
#{'model_version_status': [{'version': '1538687457', 'state': 'AVAILABLE', 'status': {'error_code': 'OK', 'error_message': ''}}]}
- 查看模型metadata
#注意跟上面相比，多了個(gè)metadata
model_metadata = requests.get('http://localhost:8501/v1/models/resnet/metadata', stream=True)
model_metadata.raise_for_status()
print(json.dumps(model_metadata.json()))
metadata示例
{
"model_spec": {
"name": "resnet",
"signature_name": "",
"version": "1538687457"
},
"metadata": {
"signature_def": {
"signature_def": {
"serving_default": {
"inputs": {
"image_bytes": {
"dtype": "DT_STRING",
"tensor_shape": {
"dim": [
{
"size": "-1",
"name": ""
}
],
"unknown_rank": false
},
"name": "input_tensor:0"
}
},
"outputs": {
"probabilities": {
"dtype": "DT_FLOAT",
"tensor_shape": {
"dim": [
{
"size": "-1",
"name": ""
},
{
"size": "1001",
"name": ""
}
],
"unknown_rank": false
},
"name": "softmax_tensor:0"
},
"classes": {
"dtype": "DT_INT64",
"tensor_shape": {
"dim": [
{
"size": "-1",
"name": ""
}
],
"unknown_rank": false
},
"name": "ArgMax:0"
}
},
"method_name": "tensorflow/serving/predict"
},
"predict": {
"inputs": {
"image_bytes": {
"dtype": "DT_STRING",
"tensor_shape": {
"dim": [
{
"size": "-1",
"name": ""
}
],
"unknown_rank": false
},
"name": "input_tensor:0"
}
},
"outputs": {
"probabilities": {
"dtype": "DT_FLOAT",
"tensor_shape": {
"dim": [
{
"size": "-1",
"name": ""
},
{
"size": "1001",
"name": ""
}
],
"unknown_rank": false
},
"name": "softmax_tensor:0"
},
"classes": {
"dtype": "DT_INT64",
"tensor_shape": {
"dim": [
{
"size": "-1",
"name": ""
}
],
"unknown_rank": false
},
"name": "ArgMax:0"
}
},
"method_name": "tensorflow/serving/predict"
}
}
}
}
}
