Deploying models to production with TensorFlow model server

栏目: IT技术 · 发布时间: 3年前

内容简介:So of course before starting you need to install TF Serving. The code examples for this blog in the repo have the installation commands typed out for you. Installing it is pretty straightforward, find installation stepsSo, you already have a model and the

TF Serving in Practice

So of course before starting you need to install TF Serving. The code examples for this blog in the repo have the installation commands typed out for you. Installing it is pretty straightforward, find installation steps here .

So, you already have a model and the first thing you would do is simply save it in a format usable by TF Serving. The third line here directory_path shows your model where to save the model and the other two lines just pass the inputs and the outputs-

tf.saved_model.simple_save( keras.backend.get_session(), directory_path, inputs = {'input_image': model.input}, outputs = {i.name: i for i in model.outputs})

If you navigate to the path where you saved this model you would see a directory structure like this, also I made a directory 1 which is my model version we will see how TF server helps us manage and deploy these versions. And also note your model is saved in a .pb extension.

Saved model directory

There is another great interface called the saved model CLI which I find pretty useful. This gives you a lot of useful information about your saved model like operation signatures and input-output shapes.

!saved_model_cli show --dir [DIR] --all

Here is sample output showing the information this tool provides-

Saved Model CLI Output
  • Starting the model server

So here is how you would then start the model server let us break this down-

os.environ["MODEL_DIR"] = MODEL_DIR%%bash --bgnohup tensorflow_model_server \ --rest_api_port = 8501 \ --model_name = test \ --model_base_path="${MODEL_DIR}" >server.log 2>&1

So the third line here tells it to use the tensorflow model server, of course you would not include the bash magic cell while implementing it in practice i.e. the code line %bash --bg but as Iassume most of you might use Colab I have added that as Colab doesn’t provide you a direct terminal.

The fourth line here specifies the port on which you want to run the TF Model Server and is pretty straightforward too.

A thing to noteid that the --model_name will also appear in the URL on which you will be serving your models, so if you have multiple models at action managing your serving model URLs also becomes a lot easier.

The last line here specifies that you want to enable logging and sometimes logs are just so helpful while debugging. I have personally used them quite a lot to figure out errors easily.


以上就是本文的全部内容,希望对大家的学习有所帮助,也希望大家多多支持 码农网

查看所有标签

猜你喜欢:

本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们

Java Web入门经典

Java Web入门经典

王国辉、陈英 / 机械工业出版社 / 2013-6 / 69.00元

《Java Web入门经典》以初学者为核心,全面介绍了JavaWeb开发中常用的各种技术。内容排列上由浅入深,让读者循序渐进掌握编程技术;在内容讲解上结合丰富的图解和形象的比喻,帮助读者理解“晦涩难懂”的技术;在内容形式上附有大量的提示、技巧、说明等栏目,夯实读者编程技术,丰富编程经验。全书共分4篇19章,其中,第一篇为“起步篇”,主要包括开启JavaWeb之门、不可不知的客户端应用技术、驾驭Ja......一起来看看 《Java Web入门经典》 这本书的介绍吧!

HTML 压缩/解压工具
HTML 压缩/解压工具

在线压缩/解压 HTML 代码

JSON 在线解析
JSON 在线解析

在线 JSON 格式化工具

HEX CMYK 转换工具
HEX CMYK 转换工具

HEX CMYK 互转工具