How to build a free LLM cybersecurity lab with Google Colab and Ollama

Learn how to set up a free cybersecurity lab using Large Language Models (LLMs), Google Colab, and Ollama. Explore practical use cases for AI in cybersecurity.

8 min readJul 16, 2024

Introduction

Hello everyone! I’m excited to kickstart my series about “How Large Language Models (LLMs) are revolutionizing cybersecurity”. In this post, we’ll set up a free working environment to explore practical applications of LLMs in cybersecurity. By the end, you’ll have a lab ready to experiment.

The LLM equation

Understanding the core components of working with LLMs is crucial:

Inference = Hardware (GPU) + Model + Inference Library + UI (CLI, API, IHM)

Understanding this equation is crucial because it highlights the four main components you need to work with LLMs:

Hardware: Typically a GPU for efficient processing.
Model: The trained AI system (e.g., GPT-4, llama3, mistral, gemma2).
Inference library: The code that will load and use the model (like llama.cpp which is what ollama is using in the backend).
User Interface: How you interact with the model (CLI, API, GUI).

Choosing the right model: Leaderboards

To help you select the most suitable LLM for your cybersecurity tasks, check out these leaderboards:

LMSYS Chatbot Arena

Open LLM Leaderboard

LMSYS covers all the best LLMs in the world and uses an ELO system to rank them, while OpenLLM focuses on open-source LLMs. These resources provide performance comparisons across various models, helping you make an informed decision based on your specific needs.

Keep in mind that new models emerge frequently, and their performance continues to advance. Also, these leaderboards represent general knowledge of each LLM, but for specific use cases, you may need a particular type of LLM.

The power of open-source LLMs

In this series, we will only use open-source LLMs. Using open-source LLMs comes with several advantages, especially in the cybersecurity field:

Free to use, with some license restrictions.
Can be implemented on-premises, even in air-gapped networks.
Customizable for specific cybersecurity tasks.

This flexibility is crucial for security operations that require complete control over their data and models.

Setting up your lab

Google Colab: Your free GPU playground

We’ll be using Google Colab for our lab environment. To get started, open this notebook: https://colab.research.google.com/drive/1qcHPOn3D5INFhDY586T_JCpK_1AW0Tib#scrollTo=KvOUX8xdzmCC

Google Colab offers several benefits for our experimental setup:

Jupyter notebook environment
Access to T4 GPU
Perfect for experimentation and learning

However, note that while excellent for our lab purposes, Colab has limitations for production use, such as limited session durations and inconsistent GPU availability.

For production environments, especially in cybersecurity, consider setting up your own infrastructure or using dedicated cloud services that offer more control, reliability, and security features.

Ollama: Your local LLM manager

In all of the serie, we will use Ollama to manage all the LLM stuff:

Download and manage models easily
Use with command line
Use with API

GitHub - ollama/ollama: Get up and running with Llama 3, Mistral, Gemma 2, and other large language…

Get up and running with Llama 3, Mistral, Gemma 2, and other large language models. - ollama/ollama

github.com

Here’s how to install it in your Colab environment:

!curl https://ollama.ai/install.sh | sh

After installation, we need to start Ollama in a separate thread to be able to interract with it after.

import os
import asyncio
import threading

async def start_ollama_serve():
    await run_process(['ollama', 'serve'],
                      stdout=open(os.devnull, 'w'),
                      stderr=open(os.devnull, 'w'))

# Create a new event loop and start Ollama
new_loop = asyncio.new_event_loop()
thread = threading.Thread(target=run_async_in_thread, args=(new_loop, start_ollama_serve()))
thread.start()

# Wait for Ollama to load
import time
time.sleep(5)

Practical use cases

Now, let’s explore two practical use cases that demonstrate the power of LLMs in cybersecurity contexts.

Use case 1: Generating malware information cards

In this use case, we use LLMs to generate concise “malware cards” based on malware names. Here’s how we define our Malware class:

from pydantic import BaseModel
from datetime import datetime

class Malware(BaseModel):
    name: str
    first_seen: datetime
    language: str
    architecture: str
    developer: str
    description: str

We then use the instructor library to enforce this structure when querying our LLM:

def get_malware_description(malware_name: str) -> Malware:
    client = instructor.from_openai(
        OpenAI(
            base_url="http://127.0.0.1:11434/v1",
            api_key="ollama",  # required, but unused
        ),
        mode=instructor.Mode.JSON,
    )

    resp = client.chat.completions.create(
        model=MODEL,
        messages=[
            {
                "role": "user",
                "content": malware_name,
            }
        ],
        response_model=Malware,
    )
    return respp

Here’s an example output for WannaCry:

{
  "name": "WannaCry",
  "first_seen": "2017-05-12T00:00:00Z",
  "language": "C/C++",
  "architecture": "x86, x64",
  "developer": "Lazarus Group (believed)",
  "description": "A ransomware worm that exploits the EternalBlue vulnerability to spread rapidly across networks. It encrypts files and demands ransom for decryption."
}

Use case 2: Building a weekly cybersecurity news digest

Our second use case demonstrates how to leverage LLMs to create a personalized, AI-curated weekly news digest. The process involves:

Fetching articles from RSS feeds
Ranking and summarizing articles using an LLM
Generating a comprehensive digest

Here’s a snippet of how we fetch RSS entries:

import feedparser
import uuid

RSS_FEEDS = [
    "https://feeds.feedburner.com/TheHackersNews",
    "https://www.bleepingcomputer.com/feed/",
]

def fetch_rss_entries(url):
    feed = feedparser.parse(url)
    return [Article(id=uuid.uuid4().hex[:8],
                   title=entry.title,
                   link=entry.link,
                   published=datetime(*entry.published_parsed[:6]))
            for entry in feed.entries]

articles = []
for rss_feed in RSS_FEEDS:
    articles.extend(fetch_rss_entries(rss_feed))

We then use our LLM to rank and summarize the articles:

def rank_cyber_news(articles: List[Article]) -> List[RankedCyberNews]:
    client = instructor.from_openai(
        OpenAI(
            base_url="http://127.0.0.1:11434/v1",
            api_key="ollama",  # required, but unused
        ),
        mode=instructor.Mode.JSON,
    )

    article_data = "\n".join([f"{a.id} - {a.title}" for a in articles])
    prompt = f"""As a senior cybersecurity analyst, analyze the following articles and identify the 5 most significant cybersecurity news items. For each news item:

    1. Provide a concise title (max 100 characters).
    2. Write a brief summary (max 250 characters).
    3. Assign an impact score (1-10) based on its potential consequences for individuals, organizations, or global cybersecurity.
    4. List related article IDs with a relevance score (1-10) for each.

    Consider factors such as:
    - The scale and severity of any breaches or vulnerabilities
    - The prominence of affected organizations or systems
    - The novelty or sophistication of attack methods
    - Potential long-term implications for cybersecurity practices

    Articles to analyze:
    {article_data}

    Provide your analysis in a structured format suitable for JSON parsing.
    """

    resp = client.chat.completions.create(
        model=MODEL,
        messages=[
            {
                "role": "user",
                "content": prompt,
            }
        ],
        response_model=List[RankedCyberNews],
    )
    return resp

This process helps security professionals save time and improve situational awareness by providing a curated summary of the most important cybersecurity news.

Bonus: Your weekly cybersecurity podcast

As an extension of our news digest, we’ve implemented a feature to convert the digest into a podcast script and then into audio using text-to-speech technology (MeloTTS).

Here’s how we generate the podcast script:

def build_podcast(full_text: str) -> str:
    client = OpenAI(
        base_url="http://127.0.0.1:11434/v1",
        api_key="ollama",  # required, but unused
    )
    prompt = f"""
    You are an engaging cybersecurity podcast host. Create a script for a 5-minute daily news digest based on the provided articles. Your task:

    1. Start with a brief, catchy introduction.
    2. Summarize 2-3 key cybersecurity stories from the given content.
    3. For each story, provide:
       - A concise overview of the incident or development
       - Its potential impact on individuals or organizations
       - Quick, actionable advice for listeners (if applicable)
    4. Use a conversational tone, as if speaking directly to the audience.
    5. Conclude with a brief sign-off, encouraging listeners to stay vigilant.

    Guidelines:
    - Keep the total length around 600-800 words.
    - Use simple language and avoid technical jargon when possible.
    - Include relevant statistics or numbers to add context.
    - Add brief transitions between stories to maintain flow.

    Content to summarize:

    {full_text}

    Remember, create only the podcast script without any formatting characters or annotations.
    """
    resp = client.chat.completions.create(
        model=MODEL,
        messages=[
            {
                "role": "user",
                "content": prompt,
            }
        ]
    )
    return resp.choices[0].message.content

Here is a sample of the generated script:

Welcome back to Cyber Security Weekly, your one-stop shop for all things digital danger. This week, we're diving into a major international takedown of cybercriminal groups using a popular cybersecurity tool for malicious purposes. It's a stark reminder that even the most legitimate tools can fall into the wrong hands.
Authorities from around the world, led by the UK's National Crime Agency, have shut down a vast network of servers and websites associated with Cobalt Strike. This red-teaming framework is designed to help security professionals identify vulnerabilities in systems,
but illicit versions have been exploited by hackers for years. The operation focused on older, unlicensed versions of Cobalt Strike, taking down hundreds of IP addresses across 27 countries. Cyber experts say this action significantly hampers the ability of cybercriminals to conduct their operations.
Think of Cobalt Strike as a Swiss army knife in the hands of bad actors. It's incredibly versatile, used for everything from ransomware attacks and espionage to establishing persistent backdoors into victim networks. This takedown sends a powerful message – even sophisticated cybercriminals aren't immune to international law enforcement.
But that's not all happening in the world of cybercrime this week. Spanish and Portuguese authorities have cracked down on a brazen vishing scheme targeting elderly citizens, netting €2. 5 million. Criminals posing as bank employees tricked vulnerable individuals into revealing personal information,
leading to widespread financial losses. The criminals used an elaborate money laundering operation to disguise their illicit gains. And just last week, INTERPOL launched Operation First Light, a massive global effort targeting online scams and organized crime networks.
The operation resulted in the seizure of millions of dollars worth of assets and the arrest of thousands of suspects across continents. These stories highlight the ever-evolving nature of cybercrime and the need for constant vigilance from individuals, businesses,
and governments alike. Stay tuned next week as we delve deeper into specific tactics used by hackers and explore ways to protect yourself online. Until then, stay safe out there.

We then use MeloTTS to convert this script into audio:

from melo.api import TTS

speed = 1.1
device = 'auto' # Will automatically use GPU if available

model = TTS(language='EN', device=device)
speaker_ids = model.hps.data.spk2id

output_path = 'en-default.wav'
model.tts_to_file(podcast, speaker_ids['EN-Default'], output_path, speed=speed)

Ethical considerations

While powerful, LLMs in cybersecurity require responsible use:

Verify outputs with authoritative sources
Be mindful of potential biases
Ensure data compliance with privacy laws

Remember: LLMs are tools to enhance cybersecurity decision-making.

Conclusion

Setting up this free LLM lab using Google Colab and Ollama is just the beginning of our series exploring the convergence of AI and cybersecurity. In future posts, we’ll delve into more advanced applications and uncover new use-cases.

Stay curious. Stay secure. Keep pushing boundaries.

FAQs

1. What are the benefits of using LLMs in cybersecurity?

Rapid information synthesis
Automated threat intelligence
Natural language processing for log analysis
Assistance in security policy analysis
Generation of readable reports from technical data

2. Can I use this lab setup for professional work?

Not recommended due to Colab limitations. Use a dedicated environment for professional tasks.

3. How often should I update the LLM models?

Check monthly, update quarterly or when significant improvements are released.

4. Are there risks with open-source LLMs for security tasks?

Yes, including potential outdated information and biases. Always verify outputs with authoritative sources.

5. What skills do I need to start with LLMs in cybersecurity?

Basic Python, cybersecurity fundamentals, NLP basics, and critical thinking skills.

6. LLMs vs. traditional rule-based systems in cybersecurity?

LLMs excel at unstructured data and adaptability. Rule-based systems are better for well-defined, structured problems. Many solutions use a hybrid approach.