App Building— Gradio: Text-to-Text Translator w\ flan-t5-large

In this series, App Building with HuggyMonkey, we’ll embark on a journey to explore the world of building Applications and Tools using AI/ML models. Whether you’re a seasoned developer or a newbie, hopefully, these articles will provide valuable insights, tips, hands-on examples, and most of all inspiration for your next project.

Building A Text-to-Text Translator App In 5 Mins

We are going to build a Text-to-Text (TTT) translator app using Gradio and the flan-t5-large model for translation. The app will be able to translate among 50+ languages with varying degrees of accuracy.

Image adapted from https://huggingface.co/spaces/HuggyMonkey/tTranslatorR_v0.1

Main Resources

Gradio — is an open-source Python library that is used to build machine learning and data science demos and web applications.

Transformers — provide APIs to download and train state-of-the-art pre-trained models that support a wide range of common tasks such as natural language processing, computer vision, and audio.

google/flan-t5-large — fine-tuned from the T5 pre-trained model, this model is capable of text-to-text tasks including translation, question/answering, and reasoning. The model is of size 783M Params and has an apache-2.0 license.

App’s Code

• Install Required Libraries

pip install transformers
pip install sentencepiece
pip install gradio

• Imports

from transformers import T5ForConditionalGeneration, AutoTokenizer
import gradio as gr

• Download the model and a corresponding tokenizer using the Transformer’s library.

model_name = "google/flan-t5-large"

tokenizer = AutoTokenizer.from_pretrained(model_name)

model = T5ForConditionalGeneration.from_pretrained(model_name)

• Function for using the model to do translation

def translate(input_text, src_lang, to_lang):
    prompt = f"Translate {src_lang} to {to_lang}: {input_text}"
    input_ids = tokenizer(prompt, return_tensors="pt").input_ids
    outputs = model.generate(input_ids, max_new_tokens=1000)
    model_translation = tokenizer.decode(outputs[0])
    final_translation = model_translation[5:-4]
    return final_translation

Prompt Construction: Construct the translation prompt to pass to the model. The flan-t5-large model is capable of several tasks and therefore we need to instruct it to perform the translation tasks.

Construct the prompt using f-strings to include information about the source language (src_lang), target language (to_lang), and the input text (input_text). These variables are provided by Gradio while the app is being used.

Tokenization: The tokenizer is used to convert the prompt into tokenized input IDs. The return_tensors="pt" argument indicates that the output should be in PyTorch format.

Model Inference: The model (model)(google/flan-t5-large) is then used to generate the translations based on the tokenized input. The max_new_tokens=1000 argument specifies the maximum number of tokens in the generated output.

Decoding: The generated output from the model is then decoded using the tokenizer to obtain a human-readable translation (model_translation).

Post-processing: This step removes extra special tokens added by the model. In the case of google/flan-t5-large, the special token “<pad>” is added to the beginning of the sentence, and the special token “</s>” is added to the end. The special tokens are removed by slicing the model_translation string to exclude the first 5 characters and the last 4 characters.

Output: The final processed translation (final_translation)is returned from the function.

• Gradio User Interface

languages = [
    'English', 'Spanish', 'Japanese', 'Persian', 'Hindi', 'French', 'Chinese',
    'Bengali', 'Gujarati', 'German', 'Telugu', 'Italian', 'Arabic', 'Polish',
    'Tamil', 'Marathi', 'Malayalam', 'Oriya', 'Panjabi', 'Portuguese', 'Urdu',
    'Galician', 'Hebrew', 'Korean', 'Catalan', 'Thai', 'Dutch', 'Indonesian',
    'Vietnamese', 'Bulgarian', 'Filipino', 'Central Khmer', 'Lao', 'Turkish',
    'Russian', 'Croatian', 'Swedish', 'Yoruba', 'Kurdish', 'Burmese', 'Malay',
    'Czech', 'Finnish', 'Somali', 'Tagalog', 'Swahili', 'Sinhala', 'Kannada',
    'Zhuang', 'Igbo', 'Xhosa', 'Romanian', 'Haitian', 'Estonian', 'Slovak',
    'Lithuanian', 'Greek', 'Nepali', 'Assamese', 'Norwegian'
]

desc = "<h1><em>tTranslaterR</em> is a translation app powered by AI models" \
       " capable of translating text between 50+ languages</h1>"

translator = gr.Interface(fn=translate,
                          inputs=[gr.Textbox(label="Input Text", placeholder="Input Text To Be Translated"),
                                  gr.Dropdown(label="From",
                                              choices=languages,
                                              value="English",),
                                  gr.Dropdown(label="To",
                                              choices=languages,
                                              value="German")],
                          outputs=gr.Textbox(label="Translation"),
                          title="tTranslatorR",
                          description=desc
                          )

translator.launch()

• Final Code

from transformers import T5ForConditionalGeneration, AutoTokenizer
import gradio as gr

model_name = "google/flan-t5-large"

tokenizer = AutoTokenizer.from_pretrained(model_name)

model = T5ForConditionalGeneration.from_pretrained(model_name)

def translate(input_text, src_lang, to_lang):
    prompt = f"Translate {src_lang} to {to_lang}: {input_text}"
    input_ids = tokenizer(prompt, return_tensors="pt").input_ids
    outputs = model.generate(input_ids, max_new_tokens=1000)
    model_translation = tokenizer.decode(outputs[0])
    final_translation = model_translation[5:-4]
    return final_translation

languages = [
    'English', 'Spanish', 'Japanese', 'Persian', 'Hindi', 'French', 'Chinese',
    'Bengali', 'Gujarati', 'German', 'Telugu', 'Italian', 'Arabic', 'Polish',
    'Tamil', 'Marathi', 'Malayalam', 'Oriya', 'Panjabi', 'Portuguese', 'Urdu',
    'Galician', 'Hebrew', 'Korean', 'Catalan', 'Thai', 'Dutch', 'Indonesian',
    'Vietnamese', 'Bulgarian', 'Filipino', 'Central Khmer', 'Lao', 'Turkish',
    'Russian', 'Croatian', 'Swedish', 'Yoruba', 'Kurdish', 'Burmese', 'Malay',
    'Czech', 'Finnish', 'Somali', 'Tagalog', 'Swahili', 'Sinhala', 'Kannada',
    'Zhuang', 'Igbo', 'Xhosa', 'Romanian', 'Haitian', 'Estonian', 'Slovak',
    'Lithuanian', 'Greek', 'Nepali', 'Assamese', 'Norwegian'
]

desc = "<h1><em>tTranslaterR</em> is a translation app powered by AI models" \
       " capable of translating text between 50+ languages</h1>"

translator = gr.Interface(fn=translate,
                          inputs=[gr.Textbox(label="Input Text", placeholder="Input Text To Be Translated"),
                                  gr.Dropdown(label="From",
                                              choices=languages,
                                              value="English",),
                                  gr.Dropdown(label="To",
                                              choices=languages,
                                              value="German")],
                          outputs=gr.Textbox(label="Translation"),
                          title="tTranslatorR",
                          description=desc
                          )

translator.launch()

The code above builds a Gradio app powered by the google/flan-t5-large model which can translate between 50+ languages with varying degrees of accuracy.

You can test out using a larger more accurate flan-t5 model such as “google/flan-t5-xl” (2.85B params) or “google/flan-t5-xxl” (11.3B params) by simply swapping the model_name.

You can also explore other models capable of translation tasks on Hugging Face.

You can test out a similar app hosted on Hugging Face Spaces here