TensorFlow serving model deployment and service _tensorflow

This article reproduced from: https://zhuanlan.zhihu.com/p/23361413, the original title: TensorFlow Serving Taste Fresh

In the 2016, machine learning became more popular in the post-war era of Alpha go and Li Shishi. Google also launched the TensorFlow serving this year and added a fire.
TensorFlow serving is a high-performance open Source library for machine learning model serving. It can deploy the trained machine learning model to the line, using GRPC as an interface to accept external calls. More to the front of the eye, it supports model thermal update and automatic model versioning management. This means that once you deploy TensorFlow serving, you no longer need to worry about online services, just care about your offline model training.

Today I'm going to bring you a simple Linear regression model with TensorFlow serving.

The following demo runs above the Ubuntu 16.04 LTS.

TensorFlow serving is in a fast iterative period. If the content of this article is inconsistent with the Official document, please refer to the official documentation. Environment

TensorFlow serving currently relies on Google's Open-source compilation tool Bazel. Bazel is an open source version of Google's internal compiler tool, which is basically consistent in functionality and performance blaze. Specific installation can refer to the official documentation. You will also need to install GRPC (Google has an open source version of an internal tool).

Then please refer to the official Installation guide for completion. It is noteworthy that the final Bazel build will take approximately 30 minutes and occupy approximately 5-10g space (depending on machine performance). With the use of-c opt to a certain extent to speed up the build.
Model Training

Next we use TensorFlow to write a simple test using the Linear regression model. Data, I use the sine function to generate 1000 points, trying to fit in a straight line.

Sample data is generated as follows:

Y_data = x_data + * Np.sin (X_DATA/10)

Reshape data

X_data = Np.reshape (X_data, (N_samples, 1))
Y_data = Np.reshape (Y_data, (N_samples, 1))

Then use a simple y = wx + B To do a training, using the Adam algorithm. The following parameters are simply adjusted:

Sample = 1000, Learning_rate = 0.01, Batch_size = +, N_steps = 500

y = Tf.placeholder (Tf.float32, Shape= (batch_size, 1))

Do training

With Tf.variable_scope (' Test '):
    w = tf.get_variable (' Weights ', (1, 1), Initializer=tf.random_normal_initializer ( )
    B = tf.get_variable (' Bias ', (1,), Initializer=tf.constant_initializer (0))

    y_pred = Tf.matmul (x, W) + b
    Loss = Tf.reduce_sum (y-y_pred) * * 2/n_samples)

    opt = Tf.train.AdamOptimizer (learning_rate=learning_rate). Minimize (loss) with

    TF. Session () as Sess:
        Sess.run (Tf.initialize_all_variables ()) to

        _ in range (n_steps):
            indices = Np.random.choice (N_samples, batch_size)
            x_batch = x_data[indices]
            y_batch = y_data[indices]
            _, Loss_ val = Sess.run ([opt, loss], Feed_dict={x:x_batch, y:y_batch}) print W.eval

        () print B.eval
        () print Loss_
        Val

Roughly the loss convergence around 15.8. The precision should be sufficient, after all, just a simple test model. Model Export

The following is the focus of this article: Export the model.

Tf.train.Saver

Used to save and restore variable. It is very convenient to save the variables of the current model or to pour in the previously trained variables. One of the simplest uses:

Saver-tf.train.saver ()

Save the variables to disk.

Saver.save (Sess, "/tmp/test.ckpt") Restore variables from disk.

Saver.restore (Sess, "/tmp/test.ckpt")

Tf.contrib.session_bundle.exporter.Exporter

The export model also requires the assistance of this exporter. The embarrassing is that this exporter is too new, there is no API documentation support, can only refer to the Github code implementation.

The basic way to use exporter is

Incoming saver constructs an instance

Graph and input/output that invoke the INIT definition model

Exporting to a file using export

Model_exporter = exporter. Exporter (Saver)
model_exporter.init (
    sess.graph.as_graph_def (),
    named_graph_signatures={
        ' Inputs ': Exporter.generic_signature ({' X ': x}),
        ' outputs ': exporter.generic_signature ({' Y ': y_pred})})
Model_exporter.export (Flags.work_dir,         
                      tf.constant (flags.export_version), Sess)

Done. Compile. We have successfully exported a model that can be deployed on TensorFlow serving. It takes an X value and then returns a Y-value. The exported folder is named version, containing the meta file for deployment, the model checkpoint file, and the serialized model graph:

Model deployment

The deployment is simple and requires only the following two steps:

$  

We saw TensorFlow serving successfully loaded the model we just exported. And still trying to poll the new model:
Client

Next, we'll write a simple Client to invoke the Model we've deployed. Here we need to use the TensorFlow serving predict API and Grpc Implementations.insecure_channel to construct a request. It is particularly important to note that the input signature and data must match the previous export model. In this case, the name is called X, the float32 type, and the size is [100, 1] Tensor.

From Grpc.beta import implementations
import NumPy as NP
import TensorFlow as TF from

Tensorflow_ Serving.apis import PREDICT_PB2 from
tensorflow_serving.apis import PREDICTION_SERVICE_PB2

Tf.app.flags.DEFINE_string (' server ', ' localhost:9000 ', '
                           predictionservice host:port ')
flags = Tf.app.flags.FLAGS

n_samples =

host, port = FLAGS.server.split (': ')
channel = Implementations.insecure_channel (host, int (port))
stub = Prediction_service_pb2.beta_create_ Predictionservice_stub (channel)

Generate test Data

X_data = Np.arange (N_samples, Step=1, Dtype=np.float32)
X_data = Np.reshape (X_data, (N_samples, 1)) Send request

Request = Predict_pb2. Predictrequest ()
Request.model_spec.name = ' Test '
request.inputs[' x ']. CopyFrom (Tf.contrib.util.make_tensor_proto (X_data, shape=[100, 1))
result = Stub. Predict (Request, 10.0) # secs Timeout

Don't forget to configure the Bazel build file:

Finally compiled to run, you can see the online forecast results.

Extended

TensorFlow encapsulates a number of commonly used models to become estimator, helping users avoid the long, error-prone algorithm implementation section. For example, the above examples can be completely replaced with linearregressor. It takes a few lines of code to simply invoke the Fit () function to easily get a convergent model. The only disadvantage is that the current and TensorFlow serving is not 100% compatible. While Google is still doing its best to perfect tensorflow serving, it will take a while for the perfect distance to be perfected.

If you want to use the convenient and fast estimator, but also want to deploy online. Of course, there is a way, the author studied a bit, the implementation of a estimator training data, export the model and then deploy the online method. Finally, we use this online-deployed model to implement a system that evaluates the value of the House on the line.