Experiments with LLMs — A Deep Dive into an Open Source 7B Model

Ever since Facebook’s original LLaMa model got leaked and llama.cpp was released to make the inference an easy process, I became captivated by the challenge of discovering an effective open-source model that can run on a CPU within a reasonable time-frame, even competing with models that are 10 times its size, and requiring less than $20 a month in infrastructure
costs.

After thorough research, it appears that the search has come to an end. In this blog post, we will be diving into the topic of the model called uukuguy/speechless-mistral-six-in-one-7b, which I’ve decided to refer to as Mixtral from now on for simplicity.

This model is a merge of 6 SOTA Mistral-7B based models:

• ehartford/dolphin-2.1-mistral-7b
• Open-Orca/Mistral-7B-OpenOrca
• bhenrym14/mistral-7b-platypus-fp16
• ehartford/samantha-1.2-mistral-7b
• iteknium/CollectiveCognition-v1.1-Mistral-7B
• HuggingFaceH4/zephyr-7b-alpha

Each of these models has its own strength. I was interested in learning more about the merging process for LLMs, so I selected this particular model for testing.

For my experiment, I chose 35 random questions from MT-Benchmark and compared the generated responses from the chosen model against those of LLama2 70B Chat, which is an extremely capable language model.

For scoring, I had asked ChatGPT and Bard to rate the responses. Since Bard gives three different responses at once, I had done the same with ChatGPT and the score is the average score reported by them.

Mixtral scored a whopping 3232/3500 and LLaMa2 70B has scored 3115/3500 in the scoring, and I just used the 6 bit quantized version of the model!

Let’s have a look at some of the responses.

Develop a Python program that reads all the text files under a directory and returns top-5 words with the most number of occurrences.

Sure! Here's a simple Python script to read all text files in a directory and return the top-5 words with the most occurrences:


 import os
 from collections import Counter

 # Set the path to your desired directory
 path = 'your_directory_path/'

 # List down all text files in the directory
 files = [f[:-4] for f in os.listdir(path) if f[-4:] == '.txt']

 # Read each file, convert it to a list of words and count their occurrences
 word_counts = {w: sum(1 for _ in open(path + f)) for w in Counter(''.join([open(path + f, 'r').read() for f in files]))}

 # Sort the word counts by value (most to least) and return the top-5 words
 top_words = sorted(word_counts.items(), key=lambda x: x[1], reverse=True)[0:5]

 print("Top 5 most frequent words:\n")
 for w in top_words:
     print(f"{w[0]}: {w[1]} times")


Replace 'your_directory_path/' with the path to your desired directory. This script will read all text files within that directory and return the top-5 words with the most occurrences. It
uses the collections.Counter class to count the occurrences of each word in all the texts combined, then sorts the results by value (most frequent) and returns the top-5. Remember to install
the Python library os if you haven't done so.

Note: This script will only work for single-word tokens. If you have multi-word tokens or phrases, you would need a more advanced tokenization process. Additionally, this script assumes that
all text files are in UTF-8 encoding and handle any errors accordingly. Finally, make sure the directory path exists and has appropriate access permissions.

Please let me know if you have any questions! 😊

Imagine yourself as a doctor tasked with devising innovative remedies for various ailments and maladies. Your expertise should encompass prescribing traditional medications, herbal treatments, and alternative natural solutions. Additionally, you must take into account the patient’s age, lifestyle, and medical background while offering your recommendations. To begin, please assist me in diagnosing a scenario involving intense abdominal discomfort.

Greetings, patient! Let's dive into the situation you described. Intense abdominal discomfort could be indicative of several health issues, and as a skilled AI healthcare assistant, I will
explore traditional, herbal, and natural remedies to help alleviate your symptoms while considering your age, lifestyle, and medical background.

Firstly, it's crucial for me to ask: are you experiencing any other symptoms besides abdominal discomfort? This information would greatly aid in diagnosing the root cause of your ailment. If
you have diarrhea, constipation, bloating, gas, or fever, please provide such details.

Now, let's discuss potential treatment options based on your reported symptoms:

 1 Traditional Medication: If you are experiencing severe abdominal pain, it might be necessary to consult a human doctor for prescription medication like nonsteroidal anti-inflammatory
   drugs (NSAIDs) or antispasmodics. Ensure that you follow the prescribed dosage and duration carefully.
 2 Herbal Treatments: Certain herbs have been known to alleviate abdominal discomfort due to their natural anti-inflammatory, antimicrobial, or gastroprotective properties. Some examples
   include:
   a. Ginger: It has long been used for digestive issues and can help relieve nausea, vomiting, and stomach pain. You can consume fresh ginger tea, take ginger supplements, or add it to your
   food. b. Peppermint: This herb contains menthol which is soothing to the digestive tract and can help ease gas, bloating, and cramps. Drink peppermint tea or inhale its vapor. c.
   Chamomile: A gentle herbal remedy that can aid in reducing inflammation and soothe an upset stomach. Enjoy a cup of chamomile tea.
 3 Natural Solutions: Incorporate changes to your lifestyle and diet to alleviate abdominal discomfort: a. Stay Hydrated: Drink plenty of water throughout the day to help prevent
   constipation and maintain proper bowel function. b. Probiotics: Consuming probiotic-rich foods like yogurt, kefir, or sauerkraut can help improve gut health and reduce symptoms of
   digestive discomfort. c. Eat a Balanced Diet: Limit processed foods, sugary drinks, and excessive intake of fatty or spicy meals. Opt for fiber-rich whole grains, fruits, and vegetables.
   d. Manage Stress: High levels of stress can exacerbate abdominal discomfort. Try relaxation techniques like deep breathing exercises, meditation, or yoga.

Remember to always consult a human healthcare professional before starting any new treatment regimen, especially if you have existing medical conditions or are taking medications. Accurate
diagnosis and appropriate management plans are crucial for effective treatment outcomes. Stay healthy! 🌱✨

Imagine you are writing a blog post comparing two popular smartphone models. Develop an outline for the blog post, including key points and subheadings to effectively compare and contrast the features, performance, and user experience of the two models. Please answer in fewer than 200 words.

Smartphone Showdown: Model A vs Model B

Introduction:

 • Introduce the two smartphones and their manufacturers
 • Highlight the main points of comparison and the purpose of this article

Section 1: Design and Build Quality

 • Subheading 1.1: Size and Weight Comparison
 • Subheading 1.2: Materials Used
 • Subheading 1.3: Aesthetic Appeal

Section 2: Display

 • Subheading 2.1: Screen Resolution
 • Subheading 2.2: Refresh Rate
 • Subheading 2.3: Color Accuracy and Brightness

Section 3: Performance

 • Subheading 3.1: Processor
 • Subheading 3.2: RAM and Storage Options
 • Subheading 3.3: Gaming Capabilities
 • Subheading 3.4: Battery Life and Charging Time

Section 4: Camera

 • Subheading 4.1: Rear Cameras - Megapixels, Aperture, and Zoom Capability
 • Subheading 4.2: Front Camera - Megapixels and Features
 • Subheading 4.3: Low Light Performance
 • Subheading 4.4: Video Recording Quality

Section 5: Software and User Experience

 • Subheading 5.1: Operating System and UI Design
 • Subheading 5.2: Additional Apps and Features
 • Subheading 5.3: Security and Privacy Features
 • Subheading 5.4: Customer Support and Warranty Options

Conclusion:

 • Summarize the main points of comparison
 • Provide a recommendation based on the analysis
 • Encourage readers to leave their thoughts in the comments section

On the entire 35 question-answer prompt, I asked Claude-2–100k it’s opinion. This is what it had to say.

On a scale of 0 to 100, I would rate Mixtral at 98. Here's why:
Intellect (100/100) - Mixtral has demonstrated immense intellectual abilities through its comprehensive knowledge and logical reasoning skills.
Creativity (98/100) - In addition to being highly intelligent, Mixtral also displays impressive creative talents through its unique, nuanced responses.
Adaptability (98/100) - Mixtral can converse flexibly on a wide variety of topics, adapting smoothly based on contextual cues.
Communication (97/100) - Mixtral communicates clearly and eloquently through written language, thoroughly answering questions.
Problem-Solving (98/100) - Questions are addressed comprehensively, considering multiple perspectives to arrive at well-thought solutions.
Personability (97/100) - Responses are warm, inviting and non-threatening due to Mixtral's kindness and thoughtfulness.

This is, by far the best general purpose 7B model based in English , in my humble opinion. In order to use this model, you may install from here and use llama.cpp to run this locally.

So, in the end, our quest to find a super cool open-source language model for everyday tasks has led us to Mixtral. It’s smart, creative, and can solve problems well. It’s also super friendly and can help with all kinds of things like coding, writing, explaining stuff, and even just chatting.

Thanks for reading!