DELTA-NN architecture

  1. features
  2. compile
  3. package
  • TF-Serving
  • Serving
  • Embedding
  • Client

Features

  • tiny size
  • pack all by docker
  • supporting custom-op
  • exporting only C api
  • compatibility TF-Serving and Serving RESTful API
  • compatibility with Could and Edge usage
  • supporting multi graphs inference
  • supporting multi modal application, e.g KWS(Edge) + ASR(Could)

Compile

How to compile delta-nn.

TF-Serving

Suport Custom-Op

Compile TF-Serving with custom Ops.

Serving

Using go to wraper DELTA-NN supporting HTTP/HTTPS protocal.

Support Engines

  • TF
  • TFTRT

REST API

Compatibility with TF-Serving RESTful API.

Data Exchange

Using json to exchange data with Client and Serving, since it support CBOR and BSON specification.

Embedding

Support Engines

  • TFLite

Core

DELTA-NN using TFLite as backend engine.

Client

Support Engines

  • TF
  • TFTRT
  • TFLite

HTTP/HTTPS Client

DELTA-NN using mbedtls as SSL/TLS library.

It's an OpenSSL alternative libraray, which has many features:

  • Fully featured SSL/TLS and cryptography library
  • Easy integration with a small memory footprint
  • Easy to understand and use with a clean API
  • Easy to reduce and expand the code
  • Easy to build with no external dependencies
  • Extremely portable