Don’t bother with Prompt Engineering

Effective Communication with ChatGPT: Conversations, Incantations, and True Prompt Engineering

If you’ve used ChatGPT or one of its competitors, you’ve likely heard about “prompt engineering.” In this blog post, we’ll explore three ways to interact with ChatGPT, discuss their suitability for different use cases, and provide practical tips for getting the most out of this AI assistant.

You can use ChatGPT in 3 ways:

1. like talking to a person
2. using specialised prompts
3. true prompt engineering

1 will work for 90% of cases. Combining 1 and 2 works for 99% of cases. Actual prompt engineering is rarely worth the investment at all. Let me explain what I mean.

Talking to ChatGPT like a person

ChatGPT is trained to work conversationally. That means you can talk to it almost like a human assistant. The better you are at communicating, the better results you’ll get. If you want to get better at this, read books and articles on good communication. In a nutshell:

1. Say what you’re after
2. Converse with it to improve understanding

Feedback is really helpful here. Just like with people, it’s important to give feedback to others so that they can follow your directions better. It’s also important to ask for feedback, so you can give clearer instructions.

Photo by Christina @ wocintechchat.com on Unsplash

If you’re struggling then here’s some good examples to get you started.

• “That’s too formal, make it more friendly”
• “Great. 5 more examples”
• “You haven’t understood. How can I make my request clearer?”
• “Ask any questions you have first, so you know exactly what I mean”

Using specialised prompts

This trend seems to have taken off, with countless twitter threads, experts selling PDFs and courses under the term “prompt engineering”. While not true engineering, specialized prompts have gained popularity for their ability to quickly generate desired responses without the need for multiple iterations.

I like to call these “prompt incantations” as you just say the predefined words and hope the magic will do what you intended.

Photo by Artem Maltsev on Unsplash

A few examples:

• Ask it to explain each step of its thinking, known as chain-of-thought reasoning
• Prime it with some example good responses to model its response on
• Tell it to respond in a certain style, like “as a senior engineer…” or “as Richard Feynman…”

True prompt engineering

Engineering is about using scientific principles to design and build stuff. So what would prompt engineering really look like?

Here’s some steps that ChatGPT suggests we take. While these steps can be informative, they are often overkill for 99% of cases, as casual conversation or specialized prompts are usually sufficient to achieve the desired results.

I would not recommend doing this. Skip past this section, unless you’re very inquisitive.

1. Define the objective: Establish a clear purpose for the prompt, focusing on the desired outcome or response, and identify any engineering concerns such as reliability, security, or robustness.
2. Conduct literature review: Research relevant literature on ChatGPT’s capabilities, limitations, and biases, as well as studies on effective prompts and their impact on response quality.
3. Decompose prompt components: Break down the essential elements of an effective prompt, including context, constraints, and instructions, while addressing specific engineering concerns.
4. Generate prompt variations: Create multiple versions of the prompt, incorporating different phrasings, structures, and levels of detail while considering the identified engineering concerns.
5. Develop evaluation criteria: Establish a set of metrics to assess the effectiveness of each prompt, such as response relevance, accuracy, coherence, and adherence to engineering requirements.
6. Design experiments: Plan a series of experiments to compare the performance of different prompts, including a methodology for measuring and analyzing results against the evaluation criteria.
7. Execute experiments and collect data: Test the prompts with ChatGPT, and gather data on the quality of the generated responses as well as any relevant engineering metrics.
8. Analyze and compare results: Evaluate the performance of each prompt by applying the established evaluation criteria and examining any correlations or trends in the data.
9. Refine and iterate: Based on the analysis, modify the most promising prompts to address any shortcomings or further optimize their performance. Repeat the experimentation and analysis until a satisfactory prompt is obtained.
10. Monitor and maintain: Deploy the optimized prompt and continuously monitor its performance in real-world settings. Address any emerging issues and apply updates as needed.
11. Learn and disseminate knowledge: Document the process, results, and lessons learned from the prompt engineering efforts. Share findings and best practices with peers to contribute to the broader understanding of effective prompt design.

Conclusion

Effective communication with ChatGPT is mainly about improving your conversation skills and, to a lesser extent, memorizing a few prompt incantations. As with many things in life, the best way to improve is to practice and learn from feedback.