Offline AI:Ollama+Qwen2.5+llama3.2 (30 Languages-Excluding Common): Chat with Ollama/Documents/Devices in Hindi, English, Bengali, French, German, Italian, Portuguese, Spanish, Thai, Arabic, , Chinese, Croatian, Czech, Danish, Dutch, Estonian, Finnish, Greek, Hungarian, Indonesian, Italian, Japanese, Khmer, Korean, Latvian, Lithuanian, Norwegian, Polish, Russian, Swedish-Part 02.
Learn to Connect Ollama with LLAMA3.2+Qwen2.5 or chat with Ollama/Documents- PDF, CSV, Word Document, EverNote, Email, EPub, HTML File, Markdown, Outlook Message, Open Document Text, PowerPoint Document, Text file or chat with Devices-Mac/Windows/Linux/Android Mobiles/Tablet.
Ollama: Large Language Model Runner
https://hub.docker.com/r/ollama/ollama
llama3.2
The Meta Llama 3.2 collection of multilingual large language models (LLMs) is a collection of pretrained and instruction-tuned generative models in 1B and 3B sizes (text in/text out). The Llama 3.2 instruction-tuned text only models are optimized for multilingual dialogue use cases, including agentic retrieval and summarization tasks. They outperform many of the available open source and closed chat models on common industry benchmarks.
Supported Languages: English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai are officially supported. Llama 3.2 has been trained on a broader collection of languages than these 8 supported languages.
Qwen2.5
Qwen2.5 is the latest series of Qwen large language models. For Qwen2.5, a range of base language models and instruction-tuned models are released, with sizes ranging from 0.5 to 72 billion parameters. Qwen2.5 introduces the following improvements over Qwen2:
- It possesses significantly more knowledge and has greatly enhanced capabilities in coding and mathematics, due to specialized expert models in these domains.
- It demonstrates significant advancements in instruction following, long-text generation (over 8K tokens), understanding structured data (e.g., tables), and generating structured outputs, especially in JSON format. It is also more resilient to diverse system prompts, improving role-play and condition-setting for chatbots.
- It supports long contexts of up to 128K tokens and can generate up to 8K tokens.
- It offers multilingual support for over 29 languages, including Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, Arabic, and more.
Please note: all models except the 3B and 72B are released under the Apache 2.0 license, while the 3B and 72B models are under the Qwen license.
You can refer to Part 01 of this article.
Let’s get started.
Step 01: Now You need to Have Ollama up and running on Mac/Windows/Linux/Android Mobiles/Tablet, You can refer below articles to Install Ollama. Process of Installation is more or less same on all device except Windows where you have Ollama Installer.
Step 02: Now Once you have Ollama installed on any above mentioned device with atleast 8GB RAM then run below command to pull and run llama3.2, I will suggest that based on your device RAM pickup the model. In my example I have pulled llama3.2:3b-instruct-q8_0 (3.4 gb) on Mac Mini with fast response (videos are 2x speed in this article)
ollama run llama3.2:3b-instruct-q8_0
Step 03: Now ask your question in any of these languages English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai
There is possibility to get some wrong answer, you need to ask for correction like shown in below screen shots.
Now if you see till above screenshot it has given answer it the respective language ( I cannot verify of language accuracy in above screenshots)
but after that for languages it has replied in english as shown in below screenshot.
I asked to check the missing answer and provide answer in respective language.
Here is the screenshot for corrected reply.
Here is the brief of training data and proficiency level as replied by llama3.2
Based on speed and response requirement, I am using above model in Mac/Windows/Linux/Android with 8GB RAM+8GMVRAM and 1B Parameter model in 4GB Android and IPAD (via H2O Personal GPT App).
Step 04: Now for Qwen2.5, I am using qwen2.5:7b-instruct in this example. You can pull and run this model with below command, Kindly note that this model is 4.7 GB in size so kindly use relevant tag from Ollama website based on device RAM.
ollama run qwen2.5:7b-instruct
Step 05: Now you can ask questions in these many languages.
Step 06: Now to talk to Documents PDF, CSV, Word Document, EverNote, Email, EPub, HTML File, Markdown, Outlook Message, Open Document Text, PowerPoint Document, Text file in 30+ languages complete the setup as mentioned in below article but make sure that instead of llama3 change the llm to llama3.2 or qwen2.5 based on your needs in script file, Once you are done try chat in above mentioned languages.
Step 06: Now to chat with your devices and get your work done you need to have open-interpreter, you can refer to below article for setting up open-interpreter, Kindly make sure you choose llama3.2 or qwen2.5 while selecting Ollama Models as per your language needs.
Stay tuned for more updates in this article.
