IBM: Tuning studio and classification for Thai and English language

Harnessing IBM’s Prompt Tuning Studio for Enhanced Model Performance

In the realm of AI and machine learning, the pursuit of efficiency and accuracy never ceases. IBM introduces an easy-to-use Prompt Tuning Studio. This tool is a gateway to optimizing AI models, especially smaller models like google’s Flan, with significant benefits including lower costs and increased accuracy. Here an experiment is conducted with 500 samples of customer review in both english and Thai and its corresponding sentiment (whether it is positive or not).

The Power of Prompt Tuning

Prompt tuning has emerged as an alternative approach to full fine-tuning AI models by using “soft prompts” that are integrated with the input data (does not change the underlying model). By leveraging this technique, especially on models like Flan-t5-xl, performance can see substantial improvements. The evidence is clear in the performance leap from an accuracy of 0.94 to an impressive 0.99 for English language tasks, using only 400 data points for tuning.

Cost-Effectiveness and Accuracy

One of the most compelling benefits of prompt tuning is its cost-effectiveness. Traditionally, training or fine-tuning AI models, especially for language tasks, could be resource-intensive. Prompt tuning, however, requires fewer examples and computational power.

The way accuracy will be measured is (for simplicity because we only have two classes)

Accuracy = #correctly classified sentiment/ #Total number of records

Below is the result for the prompt-tuned flan model:

Thai Language Classification

Beyond just the English language. The introduction of the Mixtral model quantized by IBM shows promise, with accuracies reaching up to 0.91. Without prompt tuning. So it is interesting to see how other models compared to the IBM-mixtral and whether prompt tuning helps or not in Thai language.

Reflecting on Results and Future Prospects

While initial prompt tuning results for the Thai language show a modest improvement from 0.51 to 0.54 accuracy, the significance lies in the proof of concept. This incremental improvement underscores a vital truth: the potential for growth and optimization is vast. As more models can undergo prompt tuning in prompt studio, we stand on the brink of a new horizon where performance can only ascend.

Pre-Requisites and Execution

To leverage IBM’s studio, ensure you have an IBM cloud account and watsonx resources provisioned. For detailed instruction — follow this
https://github.ibm.com/Kandanai-Leenutaphong/MODEL-TUNING-CG-CLASSIFICATION