App Building— Gradio: Text-to-Text Translator w\ flan-t5-large
In this series, App Building with HuggyMonkey, we’ll embark on a journey to explore the world of building Applications and Tools using AI/ML models. Whether you’re a seasoned developer or a newbie, hopefully, these articles will provide valuable insights, tips, hands-on examples, and most of all inspiration for your next project.
Building A Text-to-Text Translator App In 5 Mins
We are going to build a Text-to-Text (TTT) translator app using Gradio and the flan-t5-large model for translation. The app will be able to translate among 50+ languages with varying degrees of accuracy.
Main Resources
Gradio — is an open-source Python library that is used to build machine learning and data science demos and web applications.
Transformers — provide APIs to download and train state-of-the-art pre-trained models that support a wide range of common tasks such as natural language processing, computer vision, and audio.
google/flan-t5-large — fine-tuned from the T5 pre-trained model, this model is capable of text-to-text tasks including translation, question/answering, and reasoning. The model is of size 783M Params and has an apache-2.0 license.
App’s Code
- Install Required Libraries
pip install transformers
pip install sentencepiece
pip install gradio
- Imports
from transformers import T5ForConditionalGeneration, AutoTokenizer
import gradio as gr
- Download the model and a corresponding tokenizer using the Transformer’s library.
model_name = "google/flan-t5-large"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = T5ForConditionalGeneration.from_pretrained(model_name)
- Function for using the model to do translation
def translate(input_text, src_lang, to_lang):
prompt = f"Translate {src_lang} to {to_lang}: {input_text}"
input_ids = tokenizer(prompt, return_tensors="pt").input_ids
outputs = model.generate(input_ids, max_new_tokens=1000)
model_translation = tokenizer.decode(outputs[0])
final_translation = model_translation[5:-4]
return final_translation
Prompt Construction: Construct the translation prompt to pass to the model. The flan-t5-large model is capable of several tasks and therefore we need to instruct it to perform the translation tasks.
Construct the prompt using f-strings to include information about the source language (src_lang
), target language (to_lang
), and the input text (input_text
). These variables are provided by Gradio while the app is being used.
Tokenization: The tokenizer
is used to convert the prompt into tokenized input IDs. The return_tensors="pt"
argument indicates that the output should be in PyTorch format.
Model Inference: The model (model
)(google/flan-t5-large) is then used to generate the translations based on the tokenized input. The max_new_tokens=1000
argument specifies the maximum number of tokens in the generated output.
Decoding: The generated output from the model is then decoded using the tokenizer to obtain a human-readable translation (model_translation
).
Post-processing: This step removes extra special tokens added by the model. In the case of google/flan-t5-large, the special token “<pad>” is added to the beginning of the sentence, and the special token “</s>” is added to the end. The special tokens are removed by slicing the model_translation
string to exclude the first 5 characters and the last 4 characters.
Output: The final processed translation (final_translation)
is returned from the function.
- Gradio User Interface
languages = [
'English', 'Spanish', 'Japanese', 'Persian', 'Hindi', 'French', 'Chinese',
'Bengali', 'Gujarati', 'German', 'Telugu', 'Italian', 'Arabic', 'Polish',
'Tamil', 'Marathi', 'Malayalam', 'Oriya', 'Panjabi', 'Portuguese', 'Urdu',
'Galician', 'Hebrew', 'Korean', 'Catalan', 'Thai', 'Dutch', 'Indonesian',
'Vietnamese', 'Bulgarian', 'Filipino', 'Central Khmer', 'Lao', 'Turkish',
'Russian', 'Croatian', 'Swedish', 'Yoruba', 'Kurdish', 'Burmese', 'Malay',
'Czech', 'Finnish', 'Somali', 'Tagalog', 'Swahili', 'Sinhala', 'Kannada',
'Zhuang', 'Igbo', 'Xhosa', 'Romanian', 'Haitian', 'Estonian', 'Slovak',
'Lithuanian', 'Greek', 'Nepali', 'Assamese', 'Norwegian'
]
desc = "<h1><em>tTranslaterR</em> is a translation app powered by AI models" \
" capable of translating text between 50+ languages</h1>"
translator = gr.Interface(fn=translate,
inputs=[gr.Textbox(label="Input Text", placeholder="Input Text To Be Translated"),
gr.Dropdown(label="From",
choices=languages,
value="English",),
gr.Dropdown(label="To",
choices=languages,
value="German")],
outputs=gr.Textbox(label="Translation"),
title="tTranslatorR",
description=desc
)
translator.launch()
- Final Code
from transformers import T5ForConditionalGeneration, AutoTokenizer
import gradio as gr
model_name = "google/flan-t5-large"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = T5ForConditionalGeneration.from_pretrained(model_name)
def translate(input_text, src_lang, to_lang):
prompt = f"Translate {src_lang} to {to_lang}: {input_text}"
input_ids = tokenizer(prompt, return_tensors="pt").input_ids
outputs = model.generate(input_ids, max_new_tokens=1000)
model_translation = tokenizer.decode(outputs[0])
final_translation = model_translation[5:-4]
return final_translation
languages = [
'English', 'Spanish', 'Japanese', 'Persian', 'Hindi', 'French', 'Chinese',
'Bengali', 'Gujarati', 'German', 'Telugu', 'Italian', 'Arabic', 'Polish',
'Tamil', 'Marathi', 'Malayalam', 'Oriya', 'Panjabi', 'Portuguese', 'Urdu',
'Galician', 'Hebrew', 'Korean', 'Catalan', 'Thai', 'Dutch', 'Indonesian',
'Vietnamese', 'Bulgarian', 'Filipino', 'Central Khmer', 'Lao', 'Turkish',
'Russian', 'Croatian', 'Swedish', 'Yoruba', 'Kurdish', 'Burmese', 'Malay',
'Czech', 'Finnish', 'Somali', 'Tagalog', 'Swahili', 'Sinhala', 'Kannada',
'Zhuang', 'Igbo', 'Xhosa', 'Romanian', 'Haitian', 'Estonian', 'Slovak',
'Lithuanian', 'Greek', 'Nepali', 'Assamese', 'Norwegian'
]
desc = "<h1><em>tTranslaterR</em> is a translation app powered by AI models" \
" capable of translating text between 50+ languages</h1>"
translator = gr.Interface(fn=translate,
inputs=[gr.Textbox(label="Input Text", placeholder="Input Text To Be Translated"),
gr.Dropdown(label="From",
choices=languages,
value="English",),
gr.Dropdown(label="To",
choices=languages,
value="German")],
outputs=gr.Textbox(label="Translation"),
title="tTranslatorR",
description=desc
)
translator.launch()
The code above builds a Gradio app powered by the google/flan-t5-large model which can translate between 50+ languages with varying degrees of accuracy.
You can test out using a larger more accurate flan-t5 model such as “google/flan-t5-xl” (2.85B params) or “google/flan-t5-xxl” (11.3B params) by simply swapping the model_name.
You can also explore other models capable of translation tasks on Hugging Face.
You can test out a similar app hosted on Hugging Face Spaces here