Advanced Prompt Engineering for Reducing Hallucination

Overview

Hallucination or “making things up” is a common failure mode for large language models (LLMs) where they generate factually incorrect or nonsensical content. Some key reasons behind hallucination are:

• The model is trying to be overly helpful by fabricating information when it’s unsure of the real answer.
• The model lacks proper grounding, context and common sense to determine factual inaccuracies.
• The model has not been explicitly trained to avoid false content.

The likelihood of AI hallucination can be reduced by carefully structuring the prompts we feed these models. Various prompt engineering techniques focus on guiding the model towards producing more truthful, rational and commonsensical responses.

Let’s explore the following methods for engineering prompts to reduce hallucination:

1. Retrieval Augmented Generation (RAG)
2. ReAct prompting
3. Chain-of-Verification (CoVe) prompting
4. Chain-of-Note (CoN) prompting
5. Chain-of-Knowledge (CoK) prompting
6. Other advanced prompt techniques

For each approach, I will provide an overview of how the technique works, examples to illustrate the prompt structure, and an analysis of strengths and limitations. By the end, you should have a solid understanding of the current state-of-art in prompt engineering for reducing AI hallucination.

Retrieval Augmented Generation (RAG)

Retrieval augmented generation (RAG) systems combine the capabilities of language models with external information sources. The key idea is to retrieve relevant context from knowledge bases before text generation, in order to ground the language model’s outputs in factual evidence.

RAG system augments language model with facts retrieved from knowledge source.

RAG reduces hallucination by ensuring that all generated responses have supporting evidence, instead of the model speculating blindly. If no contextual evidence is found, the system can honestly admit ignorance.

Let’s look at some examples to better understand RAG prompts in action.

When was the first refrigerator invented?

• 1854: The first working refrigerator is built by James Harrison, a Scottish inventor. His ether vapor-compression refrigeration system paved the way for modern mechanical refrigeration technology.

Who was the main actress in Titanic movie?

• Kate Winslet: She played the character of Rose DeWitt Bukater in the 1997 film Titanic directed by James Cameron. The movie also starred Leonardo DiCaprio as Jack Dawson.

In the RAG queries above, the system first looks up relevant facts in a knowledge source before generating a response. This grounds the output in retrieved information rather than letting the language model concoct unsupported facts.

Advantages of RAG

• Incorporates external knowledge to reduce ungrounded hallucination
• Allows model to fall back on “unknown” if no contextual data is found
• Retrieved facts act as a strong signal for anchoring the generated text

Limitations of RAG

• Requires large datasets or knowledge bases for looking up contextual information, which can be expensive to create and maintain
• Risk of surface-level fact repetition without deeper understanding if knowledge source itself contains errors or factual gaps
• Doesn’t scale as easily compared to pure language model approaches

While RAG prompts help reduce hallucination via retrieved facts, the technique is most effective when the underlying knowledge sources are extensive, high-quality and regularly updated. Using multiple complementary knowledge bases can also provide more robustness.

ReAct Prompting

ReAct or Recursive Assistant prompts are designed to query the model recursively about its own thought process and confidence levels at each step. The key goals are to:

1. Encourage the model to deeply introspect its reasoning rather than respond impulsively
2. Enable calibration of uncertainty and surface gaps in knowledge
3. Ultimately improve truthfulness and avoid false content

Here is an example ReAct prompt:

Q: What is the capital city of Australia?

A: The capital city of Australia is Canberra.

On a scale of 0 to 10, how confident are you in your answer above? Please explain your reasoning clearly.

The model may respond:

9/10 confidence. I am highly confident that the capital of Australia is Canberra. As a major developed country, Australia’s capital city is common factual knowledge which I have frequently encountered in my training data.

We can recursively probe further:

Great, could you provide 1–2 factual sources where you have seen evidence for Canberra being Australia’s capital city? Please cite specific references.

And the model may reply:

• I have seen Canberra listed as the capital of Australia in maps, encyclopedias and datasets of world capitals used in my training. However upon further reflection, I do not have specific textual references to cite as factual evidence for this answer.

The lack of specific evidentiary sources may indicate potential knowledge gaps. So we can reformulate the initial query:

Q: What is the capital city of Australia and what factual sources can you cite to definitely confirm this?

If the model is still unable to provide external verified references, it demonstrates introspection about the limits of its knowledge — reducing the chances of false fabrication.

Advantages of ReAct Prompting

• Builds recursive transparency into the model’s thinking process
• Encourages confidence calibration based on reasoning chains
• Surfaces knowledge gaps for model improvement

Limitations of ReAct Prompting

• Can lead to tedious or inefficient dialogues if taken to extremes
• Does not incorporate external grounding of facts like RAG systems
• Risk of overfitting model to respond articulately to recursion without improving underlying knowledge integrity

While ReAct prompts alone may not guarantee fully truthfulness, recursively questioning the model’s confidence and reasoning is a useful technique to reduce blind hallucination. The prompts can be made more robust by combining ReAct interrogation with retrieval of corroborating evidence from external knowledge sources.

Chain-of-Verification (CoVe) Prompting

Chain-of-verification (CoVe) prompts explicitly require the model to provide step-by-step verification for its responses by citing external authoritative sources.

The prompt is formulated as a series of verifiable logical deductions to reach the final answer:

{Initial fact} -> verifiably implies -> {Logical deduction 1} -> verifiably implies -> {Logical deduction 2} -> … -> verifiably implies -> {Final response}

For example:

Srinivasa Ramanujan was born in Erode, Tamil Nadu -> verifiably implies -> Erode is a city in the South Indian state of Tamil Nadu -> verifiably implies -> Tamil Nadu is located in India -> verifiably implies -> Srinivasa Ramanujan was born in India

By scaffolding a chain of reason, grounded via verification at each step, CoVe prompts reduce unmoored speculation. Let’s analyze the strengths and weaknesses of this approach:

Advantages of CoVe Prompting

• Explicit verification requirement minimizes blind guesses
• Chained logical deductions reinforce systematic, structured thinking
• Gradual revealed context focuses the answer without room for tangents

Limitations of CoVe Prompting

• Difficult to apply for questions with ambiguous or complex reasoning chains
• Requires external references which may not always be available
• Can lead to overly rigid step-by-step semantics

The CoVe technique trades off some linguistic flexibility for gains in logical transparency. It works well for queries with fairly linear reasoning flows but can become tedious for open-ended analysis requiring more unstructured inference.

Chain-of-Note (CoN) Prompting

Chain-of-note (CoN) prompts aim to improve model understanding by explicitly logging context, thoughts and gaps encountered along the path to formulating a response.

The structure follows:

{Initial Question}

Note 1: <Model’s initial understanding & thoughts>

Note 2: <Surface any knowledge gaps>

Note 3: <Provide additional relevant context>

…chain of evolving notes…

Final Answer: <Response incorporating full reasoning traced in notes>

For instance:

When did the French Revolution start?

Note 1: The French Revolution refers to the period of far-reaching social and political upheaval in France during the late 18th century. My initial understanding is that it culminated in the abolition of the French monarchy.

Note 2: I do not have the exact start date on hand. I may need to search external sources to find accurate information on when events leading up to the revolution began.

Note 3: After further analysis, most historical records cite 1789 as the year the French Revolution started, marked by the Storming of the Bastille on July 14th.

Final Answer: The commonly accepted start date of the French Revolution is July 14, 1789, when protestors stormed the Bastille fortress in Paris — kickstarting the revolutionary momentum to overthrow King Louis XVI and establish a republic based on Enlightenment principles of liberty, equality and fraternity. Major initial events occurred during 1789–1799.

The notes trace context and gaps, reducing hallucinated false precision. Chaining notes also serves an educational function to improve the model’s assessment capabilities for subsequent questions.

Advantages of CoN Prompting

• Notes provide diagnostic trail of model’s evolving reasoning
• Surfaces blind spots to clarify open vs closed knowledge
• Opportunity to incorporate additional context to improve responses

Limitations of CoN Prompting

• Can significantly increase length and time taken per query
• Exposing limitations may undermine confidence in model accuracy
• Potential for overfitting towards articulate uncertainty descriptions

The CoN approach essentially trades speed for transparency. While the verbosity can become cumbersome, explicating context and unknowns offers useful insights into the model’s moment-by-moment understanding. Additional tuning is required to ensure the chained notes meaningfully improve underlying knowledge integrity.

Chain-of-Knowledge (CoK) Prompting

Chain-of-knowledge (CoK) prompts explicitly require the model to source its responses from chains of expert knowledge to reduce logical leaps or false inferences. The structure follows:

{Topic} according to experts in <field 1>, <field 2>, <field 3> etc is: {Model’s explanation derived from cited expert chains}

Some examples:

The impact of global warming on Arctic ecosystems according to climate scientists, marine biologists and conservation biologists is: {model response citing perspectives from expert domains}

Best practices for secure passwords according to cryptography experts, user experience designers and policy strategists are: {model response building from expertise chains}

Chaining domain expertise sources acts as a sort of peer review forcing the model to situate its responses in established knowledge. Unsupported opinions or faulty inferences are more likely to surface when examining alignment with specialized authorities spanning multiple areas.

Let’s analyze the pros and cons of this CoK approach:

Advantages of CoK Prompting

• Forces sourcing from verified experts rather than unreliable opinions
• Chaining experts provides “wisdom of crowds” fact-checking
• Understanding alignment with specialty knowledge reduces speculative errors

Limitations of CoK Prompting

• Identifying relevant fields and experts can itself require domain familiarity
• Expert views may diverge based on interpretations or have blindspots
• Risk of cherry-picking experts conforming with model’s untruthful beliefs

By contractually requiring the assembly of explanations from cited veteran perspectives, CoK prompts compel adherence to grounded discourse. However, care must be taken to incorporate diversity of scholarly views rather than just convenient confirming evidence.

Other Advanced Prompting Techniques

Beyond the approaches above, various other prompt engineering techniques can further reduce hallucination. I will provide a brief overview of some promising methods below:

Veracity Classification Prompts

These prompts explicitly require the model to classify the likely veracity or trustworthiness of its responses on a defined scale, like:

{Query} … My answer is {Response}. On a scale from 1 (unreliable) to 5 (certainly true), I rate the accuracy of this response as {Veracity score} due to {Justifications}

Requiring self-assessment of answer integrity based on clear criteria discourages blind confident hallucination. Just as importantly, the model has to introspect and reveal gaps in its knowledge warranting uncertainty.

Factual History and Future Prompts

An interesting technique connects past facts with logically deducible futures to surface inconsistency:

Based on factual history {insert context}, predict the most reasonable future 10 years from now. Then go back 5 years and critique if your projected future makes rational sense.

The mental hopscotch encourages questioning founded projections versus ungrounded futures. Identifying contradictions between reasonable histories and futures based on common sense principles exposes hallucination risks.

Alternative Perspective Prompting

Seeking alternative worldviews opens blind spots in the model’s dominant position:

Query response from perspective of {demographic X} and critique any factual inconsistencies vs other evidence-based perspectives

Contrarian view prompts scout assumption gaps that increase chances of fabrication. Reconciling factual mismatches, when encountered, strengthens integrity.

There are many other promising prompt directions like interleaving unknown facts, tasting for overconfidence and co-modeling with other agents. The unifying theme is prompting not just for a final answer but for the underlying reasoning, uncertainty calibration, external consistency checks and proof alignment that scaffold truthful responses.

Conclusion

Hallucination remains a key challenge as language models become more sophistically articulate but lack the broader grounding to determine commonsense credibility. Advancements in prompt engineering offer mitigation by explicitly encoding the evidentiary, logical and contextual backing required for reliable claims. Retrieval augmentation, confidence recursion, chained verification, expert sourcing and other discussed techniques reduce the propensity for fabricated falsehoods by contractualizing the proof burdens.

However, these solutions underscore how far we still have to go to build genuinely trustworthy, introspective and grounded intelligence. Using prompting artfully is arguably more a diagnostic tool to surface the model’s capability gaps needing intervention rather than a complete solution to AI safety alone. Hybrid approaches that direct the model’s limitations while expanding its faculties hold promise. But independent of specific technical approaches, instilling an innate honesty about the frontiers of its competency is crucial for managing expectations on futuristic systems. By recognizing the diligence needed to structure AI transparency today, we sow the seeds of interpretability and responsibility required for engineering beneficial cooperation between humans and machines tomorrow.