Let us know what is TensorFlow Lite Task Library

TensorFlow Lite is a framework of software packages that enables ML training locally on the hardware. This on-device processing and computing allow developers to run their models on targeted hardware. The hardware includes development boards, hardware modules, and embedded and IoT devices.

TensorFlow Lite Task Library contains a useful and powerful set of interfaces. That helps us handle most of the pre-processing and post-processing logic for running TensorFlow Lite models on mobile devices.

TensorFlow Lite Task Library is being widely used by Google products. It supports some of the classic machine learning tasks such as Image Classification and Segmentation, Object Detection, and Natural Language Processing.

Uses of TensorFlow Lite Task Library

1. Well-defined APIs
2. Complex but common data processing
3. High-performance gain
4. Extensibility and Customization

Supported ML tasks by TensorFlow Lite Task Library

I. Vision APIs

1. ImageClassifier
   • What an image represents is called Image classification.
   • We train the image classifier models with various images which makes it possible to recognize different image classes.
   • For instance, if we train our model with different types of flowers. Like, roses, tulips, and orchids, the model will be able to recognize them.
   • Use the Task Library ImageClassifierAPI to deploy custom image classifiers or pretrained ones into your mobile apps.
2. ObjectDetector
   • Identifying objects in a given image or video stream and their position can be done with the help of object detection models.
   • For example, a model might be trained with images containing various pieces of fruit, a label that specifies the class of fruit they represent (e.g. an apple, a banana, or a strawberry), and data specifying where each object appears in the image.
   • Use the Task Library ObjectDetectorAPI to deploy custom object detectors or pretrained ones into your mobile apps.
3. ImageSegmenter
   • To predict each pixel of an image with a particular class.
   • Use the Task Library ImageSegmenterAPI to deploy custom image segmenters or pretrained ones into your mobile apps.
4. ImageSearcher
   • Searching for similar images in an image database by a search query into a high dimensional vector.
   • Use the Task Library ImageSearcher API to deploy your custom image searcher into your mobile apps.
5. ImageEmbedder
   • This allows transferring an image into a high-dimensional feature vector representing the semantic meaning of an image.
   • Use the Task Library ImageEmbedder API to deploy your custom image embedder into your mobile apps.

II. Natural Language (NL) APIs

1. NLClassifier
   • This API classifies input text into different categories and is a versatile and configurable API that can handle most text classification and models.
2. BertNLClassifier
   • This API is very much similar to the NLClassifier.
   • Specially designed for Bert-related ML models which support Wordpiece and Sentencepiece tokenizations.
3. BertQuestionAnswerer
   • This API loads a Bert model and answers all the questions based on the content of the passage.
4. TextSearcher
   • This API allows searching for a similar text in the corpus.
5. TextEmbedders
   • This allows transferring text into a high-dimensional feature vector representing the semantic meaning of a text.

III. Audio APIs

AudioClassifier

• This API can be used for the classification of different sound types.
• For example, it can identify the bird species by their song.

IV. Custom APIs

Extend Task API infrastructure and build customized API.

Conclusion

In this blog, we learned about various APIs supported by TensorFlow Lite Task Library. And, some of its major uses of it.

References

1. https://www.tensorflow.org/lite/inference_with_metadata/task_library/overview
2. https://www.tensorflow.org/