1. TensorFlow Lite 簡介

????????TensorFlow Lite是TensorFlow針對移動和嵌入式設備的輕量級解決方案莺葫。它使設備上的機器學習預測具有低延遲和小的二進制大小。 TensorFlow Lite還支持硬件加速Android神經(jīng)網(wǎng)絡API（SDK27以上）偏螺。

Github項目地址

文檔地址

2. TensorFlow Lite 架構

????????下圖顯示了TensorFlow Lite的架構設計：

img

????????首先痪宰，將TensorFlow模型(.pb)轉換為TensorFlow Lite文件格式(.tflite)叼架，這需要使用TensorFlow Lite轉換器畔裕。然后，您可以在移動應用程序中部署轉換后的模型文件乖订。

????????部署TensorFlow Lite模型文件使用：

• Java API：圍繞Android上C++ API的便捷包裝扮饶。
• C++ API：加載TensorFlow Lite模型文件并調(diào)用解釋器。 Android和iOS都提供相同的庫乍构。
• 解釋器：使用一組內(nèi)核來執(zhí)行模型甜无，解釋器支持選擇性內(nèi)核加載。沒有內(nèi)核哥遮，只有100KB岂丘；加載了所有內(nèi)核，300KB眠饮。這比TensorFlow Mobile要求的1.5M的顯著減少奥帘。
• 在選定的Android設備上，解釋器將使用Android神經(jīng)網(wǎng)絡API進行硬件加速仪召，如果沒有可用的寨蹋，則默認為CPU執(zhí)行。

3. 遇到的問題

• 在用toco轉換模型時扔茅，其轉換工具要求固定模型的輸入大小已旧，而風格遷移模型要求輸入圖像的大小是任意的，輸出圖像的大小與輸入圖像相同召娜，TensorFlow Lite不能滿足這一點需求运褪。
• TensorFlow Lite只包含了一小部分必須的TensorFlow運算符，在風格遷移模型中用到的許多操作都沒有實現(xiàn)好萤晴，只能自己寫C++代碼去自定義運算符吐句，難度過大。

4. 查閱的資料

4.1 Feature request : Allow variable (None) input_shape for Tflite toco #21440

gargn commented on 17 Aug

@Kayoku It's currently not feasible to get TOCO to support this. A suggested work around is to put an arbitrary size into TOCO and use ResizeInputTensor when calling the TFLite interpreter.

miaout17 commented on 25 Aug

I skimmed over the paper and I think it should handle variable input size as you said.

Assuming the network architecture can really handle arbitrary input size (I skimmed over the paper and I think it does), could you try this and let us know if it works:

• When converting, use an arbitrary input size like (1, 512, 512, 3).
• When using the interpreter, call interpreter->ResizeInputTensor to resize the input tensor before calling interpreter->Invoke.

Theoretically it should do the trick. Let us know if it works for your case.

4.2 TensorFlow Lite Guide

Each output should be an array or multi-dimensional array of the supported primitive types, or a ByteBuffer of the appropriate size. Note that some models have dynamic outputs, where the shape of output tensors can vary depending on the input. There's no straightforward way of handling this with the existing Java inference API, but planned extensions will make this possible.

5. 模型轉換

The model generated (or downloaded) in the previous step is a standard Tensorflow model and you should now have a .pb or .pbtxt tf.GraphDef file. Models generated with transfer learning (re-training) or custom models must be converted—but, we must first freeze the graph to convert the model to the Tensorflow Lite format. This process uses several model formats:

• tf.GraphDef (.pb) —A protobuf that represents the TensorFlow training or computation graph. It contains operators, tensors, and variables definitions.
• CheckPoint (.ckpt) —Serialized variables from a TensorFlow graph. Since this does not contain a graph structure, it cannot be interpreted by itself.
• FrozenGraphDef —A subclass of GraphDef that does not contain variables. A GraphDef can be converted to a FrozenGraphDef by taking a CheckPoint and a GraphDef, and converting each variable into a constant using the value retrieved from the CheckPoint.
• SavedModel —A GraphDef and CheckPoint with a signature that labels input and output arguments to a model. A GraphDef and CheckPoint can be extracted from a SavedModel.
• TensorFlow Lite model (.tflite) —A serialized FlatBuffer that contains TensorFlow Lite operators and tensors for the TensorFlow Lite interpreter, similar to a FrozenGraphDef.

After a TensorFlow model is trained, the TensorFlow Lite converter uses that model to generate a TensorFlow Lite FlatBuffer file (.tflite). The converter supports as input: SavedModels, frozen graphs (models generated byfreeze_graph.py), and tf.keras models. The TensorFlow Lite FlatBuffer file is deployed to a client device (generally a mobile or embedded device), and the TensorFlow Lite interpreter uses the compressed model for on-device inference. This conversion process is shown in the diagram below:

TFLite converter workflow