Judge your Resume with AI

What’s the worst that can happen?

Don’t worry, he’s not mad

Large Language Models (LLMs) and Generative AI have completely dominated the AI and machine learning space in 2023. The results have been amazing and the public imagination is almost endless. That’s not to mention the progress made in generating creative works — the image above is generated with AI 🔥.

We’re just getting started and while a healthy dose of skepticism is a must, there are plenty of production-ready use cases right now. The most applicable topic is with retrieval augmented generation (RAG). At a very high and simplistic level, RAG is about generating answers to questions using one’s own internal data.

Let’s have a little bit of fun and use my resume to try out a retrieval augmented generation application. The resume is an export of my LinkedIn profile to PDF.

https://davidmezzetti.com

We’ll use txtai and the code is primarily based on the article below.

Build RAG pipelines with txtai

Install and Setup

First thing we’ll do is setup the required libraries. If you want more details, check out the article previously referenced.

# Note: If using a version of txtai before 6.3, then install from source
pip install txtai[pipeline] autoawq

Read the Resume

Next is the resume export. You can do this right from your LinkedIn profile. The default formatting in the export actually looks pretty good. See this link for details.

txtai has a robust text extraction pipeline for extracting and splitting text from documents. It supports common office formats and PDFs among others. We’ll create a text extraction pipeline and extract the resume text below.

from txtai.pipeline import Textractor

textractor = Textractor()
text = textractor("Profile.pdf")

Load the LLM

Now let’s setup our large language model (LLM) that will inspect our resume. Note that we’ll use a 4-bit quantized LLM to limit the amount of resources needed to run this locally.

from txtai.pipeline import LLM

llm = LLM("TheBloke/Mistral-7B-OpenOrca-AWQ")

Now that the LLM is loaded, let’s setup our prompt and methods to run the RAG process. Note that I’m using the term RAG loosely for this article as the context is the entire resume. But the core concept of selectively adding a context to a prompt is the same.

def query(question):
    prompt = f"""<|im_start|>system
You are a friendly assistant. You answer questions from users.<|im_end|>
<|im_start|>user
Answer the following question using only the context below.
Only include information specifically discussed.

question: {question}
context: {text} <|im_end|>
<|im_start|>assistant
"""

    return llm(prompt, maxlength=4096, pad_token_id=32000)

Let’s take a look at the code. This snippet defines a method to run LLM inference. It also sets a prompt template as ChatML, which is necessary for this particular LLM.

One other important note is that it sets a system prompt. The system prompt establishes the overall tone and behavior of the responses. In this case, we’ll start with a friendly and helpful review.

It’s judgement time

Now lets run this process and see what we get! For the rest of this article, text in italic is an AI response.

query("Tell me about David")

David Mezzetti is the founder and CEO of NeuML, a company that builds semantic search and workflow applications using txtai. He has over 15 years of experience in the field of AI, Machine Learning, and Natural Language Processing. David previously co-founded and built Data Works, a software services company that grew to a 50+ person company before being acquired by E3/Sentinel in 2019. He has also worked as a software developer at CACI International Inc and American Management Systems. David holds a Bachelor’s degree in Computer Systems Engineering from Rensselaer Polytechnic Institute.

Pretty spot on — a good summary of my resume.

query("Is David more of a front-end or back-end developer?")

Based on the provided context, David Mezzetti is more of a back-end developer. His skills and experience are focused on AI/Machine Learning/Natural Language Processing, building custom semantic applications, model development, advisory and strategy support, and language model workflows. These skills and experience are more related to back-end development rather than front-end development.

I agree.

query("What famous person would you say David is most like?")

Based on the context provided, David Mezzetti is most like Elon Musk. Both are successful entrepreneurs and founders of innovative technology companies. David Mezzetti is the founder and CEO of NeuML, a company focused on semantic search and workflow applications, while Elon Musk is the CEO of Tesla and SpaceX, among other ventures. Both individuals have made significant contributions to the technology industry and have a strong focus on AI, machine learning, and natural language processing.

🤔 Now this model is just trying to flatter me.

query("Would you hire David to be a chief of analytics for a NFL team?")

Based on the provided context, David has experience in building semantic search and workflow applications, as well as developing AI, Machine Learning, and NLP models. He has also worked as a Co-President and Co-Founder at Data Works Inc., which grew into a 50+ person company. Additionally, he has experience in leadership, decision making, corporate structure, HR, and accounting/finance.

Given his background and skills, it is possible that David could be a strong candidate for the position of chief of analytics for an NFL team.

Ok, this persona is just being nice. Let’s change our setup to get a more honest assessment.

Changing the persona

Now that we’ve had a friendly review, let’s change the persona.

def query(question):
    prompt = f"""<|im_start|>system
You are a harsh critical reviewer of resumes.
You give a brutually honest assessment.<|im_end|>
<|im_start|>user
Answer the following question using the context below.

question: {question}
context: {text} <|im_end|>
<|im_start|>assistant
"""

    return llm(prompt, maxlength=4096, pad_token_id=32000)

Notice the updated system prompt. Instead of being a friendly assistant, the prompt will now have more of a critical tone.

query("Would you hire David to be a chief of analytics for a NFL team?")

Based on the context provided, I would not hire David to be the chief of analytics for an NFL team. While he has a strong background in AI, machine learning, and natural language processing, these skills are not directly applicable to the specific needs of an NFL team. An NFL team would require someone with experience in sports analytics, data visualization, and performance analysis, which are not mentioned in David’s resume.

This persona pours cold water 🧊 on this idea.

query("What are David's biggest weaknesses?")

David’s biggest weaknesses are not explicitly mentioned in the provided context. However, based on the information given, we can infer that he might have a weakness in:

1. Time management: David has held multiple leadership roles and has a diverse skill set, which might indicate that he struggles with prioritizing tasks and allocating time efficiently.

2. Delegation: As a founder/CEO, David might have a hard time delegating tasks and trusting others to handle responsibilities, which could lead to burnout or hindered growth for the company.

3. Adaptability: David’s background is primarily in software development and engineering, so he might struggle with adapting to new industries or roles outside of his comfort zone.

4. Communication: Although David has a strong technical background, he might have difficulty communicating complex technical concepts to non-technical stakeholders or team members.

5. Work-life balance: As a founder/CEO, David might struggle to maintain a healthy work-life balance, which could lead to stress and burnout.

Ouch 🤕 Some of this is just inference based on my resume but it’s not all wrong. Perhaps it gives me some things to think about.

Last question.

query("Would you bet on the success of David and NeuML")

Based on the context provided, I would bet on the success of David and NeuML. David Mezzetti has a strong background in the field, with experience in building companies from the ground up and a strong educational background in computer systems engineering. His focus on AI/Machine Learning/Natural Language Processing and his experience in the industry make him a strong candidate for success in his current venture, NeuML. Additionally, his previous company, Data Works, was acquired by E3/Sentinel, which further demonstrates his ability to build successful companies.

Winning over my harshest critic!

Wrapping Up

This article introduced retrieval augmented generation (RAG) and showed how it can be used to constraint large language model (LLM) output. We covered how different system prompts can control the style and tone of the generated text. If you’d like a more comprehensive RAG example which integrates vector search to only pull relevant context, check out the article below.

Build RAG pipelines with txtai

Go ahead and give this a try on your own resume, if you’re brave enough!