Artificial Intelligence in Enterprise Applications with TensorFlow and Joget DX

The AI focus in Joget DX is to simplify the integration of pre-trained AI models into end user applications. As rationalized in the previous article, the training of AI models are best left to machine learning experts so once a trained model is available, the goal is to make it as accessible as possible to app designers.

With the bundled TensorFlow AI plugin, you essentially:

1. Upload a pre-trained TensorFlow model exported in protobuf (.pb) format
2. Configure the inputs and outputs
3. Configure optional post processing

The following sections showcases how a sample app on Joget DX incorporates some well known models for several common AI use cases:

1. Image Classification
2. Object Detection
3. Face Recognition
4. Audio Classification
5. Text Sentiment Analysis

Sample No-Code AI Apps on Joget DX

Image Classification

Inception v3 is a widely-used image recognition model that has been shown to attain greater than 78.1% accuracy on the ImageNet dataset. ImageNet is a dataset containing for image classification containing than 14 million labeled images. In the sample app, a simple decision process is designed to demonstrate the use of the TensorFlow AI Decision plugin as shown below:

The TensorFlow plugin is then configured by:

1. Uploading the pre-trained model in protobuf (.pb) format
2. Configuring the image with appropriate dimensions as the input
3. Configuring where to store the output result
4. Configuring additional post processing to match the output result to a specific label

Object Detection

SSD MobileNet v1 is a model for object detection trained using the COCO dataset. COCO is a large-scale dataset for object detection that contains 1.5 million object instances. The model can detect and identify multiple objects in a single image, as shown in the image below.

The TensorFlow plugin is configured by:

1. Uploading the pre-trained model in protobuf (.pb) format
2. Configuring the image with appropriate dimensions as the input
3. Configuring where to store the output results
4. Configuring additional post processing to match the output result to specific labels

Face Recognition

FaceNet is face recognition model introduced by Google that converts a face image into 128 measurements called an “embedding”. One embedding is then compared to another in order determine whether or not a face matches. This sample is based on the sample at https://github.com/davidsandberg/facenet.

The TensorFlow plugin is configured by:

1. Uploading the pre-trained model in protobuf (.pb) format
2. Configuring a couple of images with appropriate dimensions as the input
3. Configuring where to store the output results
4. Configuring additional post processing to compare the “distance” between the embeddings to determine a match

Download the model from https://github.com/davidsandberg/facenet/wiki/Validate-on-lfw. More information on face recognition can be found at https://medium.com/@ageitgey/machine-learning-is-fun-part-4-modern-face-recognition-with-deep-learning-c3cffc121d78

 

Audio Classification

ResNet stands for residual network, and as the name implies, it utilizes residual learning to preserve good results in neural network layers. ResNet demonstrates good results for image recognition, but also shows promise for audio classification e.g. determining the musical genre for an audio sample. This sample is based on https://github.com/chen0040/java-tensorflow-samples/tree/master/audio-classifier

The TensorFlow plugin is configured by:

1. Uploading the pre-trained model in protobuf (.pb) format
2. Configuring the audio sample with appropriate dimensions as the input
3. Configuring where to store the output result
4. Configuring additional post processing to match the output result to a specific label

Download the model from https://github.com/chen0040/java-tensorflow-samples/tree/master/audio-classifier/src/main/resources/tf_models. More information on audio classification can be found at https://medium.com/@CVxTz/audio-classification-a-convolutional-neural-network-approach-b0a4fce8f6c

Text Sentiment Analysis

CNN for Sentence Classification is a model for text classification, for example to determine whether customer feedback is positive or negative. This sample is based on the sample at https://github.com/ivancruzbht/tf_android

The TensorFlow plugin is then configured by:

1. Uploading the pre-trained model in protobuf (.pb) format
2. Configuring the text with related dictionary as the input
3. Configuring where to store the output result

Download the model from https://github.com/ivancruzbht/tf_android/tree/master/app/src/main/assets. More information on text classification can be found at https://medium.com/jatana/report-on-text-classification-using-cnn-rnn-han-f0e887214d5f