Neural Chatbot on Intel Developer Cloud
A Fast On-Ramp to Chatbots with No Coding Required
Some of my Intel colleagues recently posted an article on easy creation of high-performance chatbots:
My aim is to demonstrate a fast on-ramp to playing with this chatbot based on the 7-billion parameter version of the Mistral foundation model. The neural-chat-7b model was fine-tuned on Intel Gaudi2 AI accelerators using several Intel AI software libraries. I am going to show you how to play with this model with prebaked code on the Intel Developer Cloud.
If you follow these instructions, you can register for a standard free account on Intel Developer Cloud and start playing with a Jupyter Notebook preconfigured for various large language models (LLMs) such as:
- Writer/camel-5b-hf
- openlm-research/open_llama_3b_v2
- Intel/neural-chat-7b-v3–1
- HuggingFaceH4/zephyr-7b-beta
- tiiuae/falcon-7b
You can do all this without writing a single line of code. Simply type out a prompt and start playing. Powered by Intel Data Center GPU Max 1100s, this notebook provides a hands-on experience that doesn’t require deep technical knowledge.
Whether you’re a student, writer, educator, or just curious about AI, this guide is designed for you. LLMs have a wide range of applications, but experimenting with them can also be fun. Here, we’ll use some simple pretrained models to explore text generation interactively. Ready to try it out? Let’s set up our environment and start exploring the world of text generation with LLMs!
The short summary of your on-ramp instructions are listed here and shown in the screenshots below:
- Go to the Intel Developer Cloud.
- Click the “Get Started” button.
- Subscribe to the “Standard — Free” service tier and complete your cloud registration.
- To start up a free Jupyter Notebook session that uses the latest Intel CPUs and GPUs, click the “Training and Workshops” icon.
- Click “Simple LLM Inference: Playing with Language Models.” This will launch a prepopulated Jupyter Notebook (simple_llm_inference.ipynb).
- Run all cells.
- Select the model and options to try.
- Select the options and text prompt you want to try out. For options, I suggest choosing a model such as Intel using “with_context” and a prompt similar to “what are the specifications for an NBA basketball, court dimensions, and basket height.”
Interactive Prompt
The interactive nature of the notebook makes it easy to enter a prompt and generate a response:
Generative AI Accuracy Reminder
Chatbots such as these are known to produce erroneous results at times and should not be relied upon for accurate or trustworthy answers. For example, when I ran the above notebook with the “Without Context” option and the prompt “what are the specifications for an NBA basketball, court dimensions, and basket height,” the models returned nonsensical answers, i.e.:
“Bot: An NBA basketball has a diameter of 12 feet (3.7 meters), a height of 3 feet (0.91 meters) above the ground, and a width of 9 feet (2.7 meters)”
The bot generated the following response when I used the “With Context” option:
“Bot: . The specifications for an official NBA basketball include a circumference of 29.5 inches (75 cm) and a weight ranging from 20 to 22 ounces (567 to 624 grams). The height of the basketball hoop’s rim is 10 feet (3.05 meters) from the ground.. Regarding the court dimensions, it measures 94 feet (28.65 meters) in length and 50 feet (15.24 meters) in width.”
While the last response is not absurdly wrong, to me, a layman, it should be double-checked with official NBA specs to ensure the accuracy!
Call to Action
Try out this no-code approach to experimenting with chatbot models using the standard free Intel Developer Cloud account and the ready-made Jupyter Notebook. Try your hand with different 7B parameter models and prompts of your own.
