Databricks Dolly: The Game-Changing LLM

Published in

BI3 Technologies

4 min readJan 18, 2024

Databricks Dolly is a groundbreaking Large Language Model (LLM) that can understand and follow natural language instructions. It can generate creative and useful content for various purposes, such as writing blogs, poems, summaries, and more.

Dolly is based on the EleutherAI pythia model family and fine-tuned on a human-generated instruction dataset. In this blog, we will be using dolly-v2–3b which is built on pythia-2.8b and is fine-tuned on ~15k instruction/response records created by Databricks employees across various capability domains inspired by the InstructGPT paper, covering tasks like brainstorming, classification, QA, generation, information extraction, and summarization.

Dolly v2 is also available in larger model sizes:
I. dolly-v2–12b, 12 billion Parameter based on pythia-12b
II. dolly-v2–7b, 6.9 billion Parameter based on pythia-6.9b

Do you want to assess how the Databricks Dolly works? Let’s dive in and witness how this LLM can transform your ideas into reality.

Step 1: Set up Databricks Workspace

🗸 Create an Azure Databricks Workspace or use the existing one.

🗸 Click on Workspace and in the workspace tab, right click on shared and click on Create -> Notebook.

Step 2: Building an Instruction-Driven Pipeline Generator:

🗸 In Notebook, Run the below code in the first cell which is used to install accelerate, tranformers &torch library.

%pip install "accelerate>=0.16.0,<1" "transformers[torch]>=4.28.1,<5" "torch>=1.13.1,<2"

🗸 After installing accelerate and transformers library, run the below mentioned code in the next cell. This is executed because the libraries in Databricks need to be updated since there was an update in the packages.

dbutils.library.restartPython()

🗸 The instruction can be loaded using the pipeline function, where try_remote_code=True is required. torch_dtype=torch.bfloat16 works efficiently to save memory usage. Add the below code in the next cell and execute it.

import torch
from transformers import pipeline

generate_text = pipeline(task="text-generation",model="databricks/dolly-v2-3b",trust_remote_code=True, torch_dtype=torch.bfloat16)

🗸 The output is generated when calling an instruction in generate_text().

🗸 To get only the value of the output, you can store it in a variable as mentioned below.

result=generate_text('Who is the current CEO of Microsoft')
print(result[0]['generated_text'])

Step 3: To Get a Context Based Output

🗸 If you want to use a context-based output, return_full_text=True is added in the pipeline as mentioned below.

import torch
from transformers import pipeline

generate_text = pipeline(task="text-generation",model="databricks/dolly-v2-3b",trust_remote_code=True, torch_dtype=torch.bfloat16,return_full_text=True)

🗸 And we need to use LangChain libraries to integrate it with pipelines(instructions). LangChain is a Python library that helps developers build applications with Large Language Models (LLMs) through composability.

from langchain import PromptTemplate, LLMChain
from langchain.llms import HuggingFacePipeline
import json

# Input Template
prompt_template = PromptTemplate(
    input_variables=["instruction", "context"],
    template="{instruction}\n\nInput:\n{context}")

hf_pipeline = HuggingFacePipeline(pipeline=generate_text)

generate_context_solution = LLMChain(llm=hf_pipeline, prompt=prompt_template)

🗸 We will be creating a prompt with instructions and context. Two examples are provided below.

Here is the one more example with instructions and context.

Conclusion:

As we conclude our journey through the world of Databricks Dolly, it’s clear that this innovative solution is a game-changer for those seeking to harness the power of data.

With Dolly by your side, the possibilities are boundless, and your data-driven dreams are closer than ever. Whether you’re a seasoned data engineer or a budding data scientist, Dolly’s intuitive features and scalability make it a must-have in your toolkit.

About Us

Bi3 has been recognized for being one of the fastest-growing companies in Australia. Our team has delivered substantial and complex projects for some of the largest organizations around the globe, and we’re quickly building a brand that is well-known for superior delivery.

Website: https://bi3technologies.com/

Follow us on,
LinkedIn: https://www.linkedin.com/company/bi3technologies
Instagram: https://www.instagram.com/bi3technologies/
Twitter: https://twitter.com/Bi3Technologies