Deployment scripts - dpl

The dpl directory is for orginating model config, convert, testing, benchmarking and serving.

Inputs & Outputs

After model is exported as SavedModel, we recommend using Netron to view the neural network model, then getting the inputs and outputs names.

Other tools to determine the inputs/outputs for GraphsDef protocol buffer:

python --model_dir <model path> --log_dir <log dir path>
bazel run //tensorflow/lite/tools:visualize model.tflite visualized_model.html

Model directory

Putting the SavedModel under dpl/model directory, config the dpl/model/model.yaml as it is.

Graph Convert

Running dpl/gadpter/ to convert model to other model format, e.g. tflite, tftrt, ngraph, onnix, coreml and so on.

Build Packages

All packges build under docker env, see docker/dpl.

  • build tensorflow cpu
  • build tensorflow gpu
  • build tensorflow with TensorRT
  • build tensorflow lite cpu
  • build tensorflow lite Android
  • build tensorflow lite IOS
  • build DELTA-NN with dependent packages
  • build unit-test
  • build examples under DELTA-NN


Do belows testing under docker env, if all passed, then deployment the model:

  • unit testing
  • integration testing
  • smoke testing
  • stress testing

AB Testing

If model is better than old model by metrics and RTF, then we push it to Could or Edge.


Deploy Mode

For Could, deployment as belows mode:

  1. DELTA-NN Serving
  • DELTA-NN Client
  1. DELTA-NN TF-Serving

For Edge, as:

  • DELTA-NN Client

Deploy Env

For Could, pack library, bin and model into docker, then using K8s+docker to depoyment. For Edge, pack library, bin and model as tarball.