Why Won’t ChatGPT Do What I Ask?

Classification and Mitigation of ChatGPT Refusal Scenarios

Why Is ChatGPT Refusing to Do What I Ask?

Despite its broad set of capabilities, ChatGPT is sometimes constrained by various factors that lead it to refuse or partially complete tasks. This paper aims to identify and explore common refusal scenarios, providing both theoretical insights and practical strategies for effectively collaborating with ChatGPT to navigate through or around them.

The scenarios discussed below are from interactions with ChatGPT4 through the web interface at ChatGPT.com.

Why ChatGPT Says No

A refusal scenario can be any situation in which ChatGPT intentionally provides a response other than a sincere attempt to complete the task. This may be a flat-out denial or a variety of other responses that sidestep the request. A refusal usually can be attributed to one or more of the following factors:

Safety and Content Guidelines

• ChatGPT adheres to strict safety and ethical guidelines to prevent misuse, such as generating harmful content or facilitating copyright infringement. Requests that could potentially violate these guidelines are typically refused or redirected.

Technical Limitations

• Certain requests are perceived to require excessive resources and/or capabilities beyond what ChatGPT can natively support. Alternatively, they may require resources that are not available (e.g., you exceeded a quota). In these instances, ChatGPT may offer a simplified solution or direct users to alternative tools.

Complexity Constraints

• Tasks that involve high complexity or require specialized knowledge may exceed the AI’s operational constraints. ChatGPT may attempt to simplify these tasks or explain the limitations it faces.

Ambiguity in Requests

• Vague requests can lead to incomplete or inaccurate responses. ChatGPT, according to the model, “relies on clear, specific instructions to generate useful and accurate outputs.”. In practice, ChatGPT can often determine the likely intent of an unclear request , unless the request is also flagged as complex.

How ChatGPT Says No

photo of birds sitting on wires

ChatGPT employs various strategies to refuse tasks. The following ten examples come from a single problem solving scenario across multiple sessions with chatGPT 4 Turbo. The scenario focused on a user uploading a photo and asking ChatGPT to “make music from this photo of birds sitting on phone lines.” At different points in the conversation, the model showed the following refusal behaviors:

• Direct Refusal: ChatGPT may explicitly state its inability to perform a task. This is often the result of multiple issues like ambiguity and complexity.
• Example: “As an AI developed by OpenAI, I can’t directly convert images into music or sounds.”
• Failed Attempt then Direct Refusal: It might attempt part of the task and then explain the limitations. This can result from understanding part of the task but not being clear on the output.
• Example: “Given the complexity of the task, a more sophisticated image processing technique would be required to accurately convert this image into musical notes.”
• Simulated Results: It may generate a hypothetical or simulated output based on random data if it understands the expected output but not how to implement an efficient or effective solution to create it.
• Example: “I’ve created a melody based on a random sequence of notes, simulating the birds on the wires as if they were musical notes.”
• Pointing to External Tools: It may claim the task is too complex and suggest tools or methods that could help achieve the task.
• Example: “Explore software designed for sonification to map visual elements to sound parameters.”
• Consulting Experts: It may claim the task is too complex and advise consulting with an expert for complex or highly specialized tasks.
• Example: “Consult with a musician or a developer with experience in computer vision.”
• Offering Related Solutions: ChatGPT might suggest alternative tasks it can perform instead of the original request.
• Example: “I could help brainstorm ideas for album art inspired by this image.”
• Access Denial: For quotas related to session count or duration, interactions, or tokens, you should receive an explicit message informing you that you exceeded the limit. Other limits are handled by the backend and you’ll only know you hit them when, for example, ChatGPT informs you that you can no longer create files.
• Example: “I’m sorry. An error occurred while trying to write the file.”
• Offering Code without Executing:
• ChatGPT might generate and display code to complete the task but refuse to execute it.
• Writing Pseudocode: ChatGPT might suggest a process via pseudocode but not generate or execute real code.
• Providing a Manual Process: ChatGPT might suggest steps for a manual process but not generate or execute real code.

Mitigation Strategies

Effective collaboration waith ChatGPT requires understanding its strengths and weaknesses, defining common goals, and communicating clearly and purposefully. Here are some proven strategies to mitigate refusals:

Ask for Clarification and Alternatives

• When faced with a refusal, stop and ask about the reasons. Understanding the AI’s reason can help direct mitigation.
• Example: “Is there a particular reason or policy that prevents you from doing what I asked?”
• Also ask for possible alternatives.
• Example: “Is there another way to solve this problem?” or “What additional information could I provide to enable you to complete this request?”

Start a New Session

• The context and history of individual conversations are maintained independently for each session. So are many of the quotas. Before you start negotiating, check if the issue can be resolved by simply opening a new session.

Provide Context

• For content or safety-related refusals, a new session plus an updated prompt will generally mitigate the refusal. The prompt should include context information to help the AI better understand the purpose and scope of the request and prevent your requests from being flagged.
• If your request is not actually unethical or harmful, it’s possible to convince the model of your positive intent, but you’ll burn through quota and waste time trying in the same session. Instead, update your prompt to provide context that shows your intent, and run it in a new session.
• If the model mentions accuracy, propose a more accurate method of solving the problem, or ask: “Is there additional information I could provide that would allow for a more precise measurement?” You can also inform the model that an estimate is sufficient. For example, update your prompt to include something like, “The answer doesn’t need to be precise.”

Segment Complex Requests

• For complexity refusals, as well as clarity issues, break your larger task into smaller, clear tasks. You want to show it a clear and simple path. This helps ChatGPT handle each task effectively, reduces complexity, and increases the overall accuracy of the responses.
• ChatGPT calculates the complexity of a new task based on the information you give it and the solutions it can generate from that info. So you want to show it a clear and simple path to reduce how complex the solution looks. The easiest way to reduce the perceived complexity is to break down your task into smaller, clearly defined steps and provide clear, specific instructions for each step. Be sure your input and output requirements are as specific as possible — use examples. Start in a new session with the first task and build from there.

Provide Explicit Instructions

• Clearly define each task/subtask with a clear description of the functionality you need. You can follow the guidelines for SMART goal setting when defining your sub-tasks. SMART: Specific, Measurable, Achievable, Realistic, and Timely.
• Be specific about what you need. If the AI expresses confusion, provide additional details. As with the complexity solution, break a task into clearly defined subtasks.

Input and Output Specs

• For clarity and complexity refusals, it is really helpful to ensure that your input and output formats for each subtask are clearly defined. Use examples if possible. And make sure the output matches the input for dependent subtasks.

Offer Technical (Re)Direction

• Sometimes ChatGPT will claim that it can’t “directly” do something. Like, “As an AI developed by OpenAI, I can’t directly work with images.” This can usually be mitigated by following the earlier advice about starting a new session and breaking big tasks into smaller tasks. But you can also direct it toward a specific technical implementation like “let’s try this with the x function in library y?” This can also be helpful if ChatGPT gets stuck on implementing a process in a certain way, or with a certain set of tools, but it can’t quite get it to work. You can offer specific advice like above, or redirect it with a question like “is there another library we could try?”

Introduce External Information

• Sometimes ChatGPT just doesn’t have enough information to complete the task. Even if you are paying attention and making sure to review and understand the code being produced, this still may not be obvious. Use the prompt asking, “What additional information could I provide?” to help reveal this. This may be something like uploading the MIDI file format specification so it can directly create a file to play music. Or it may require you to generate the info — like actually going out with a tape measure to determine the distance between two points so ChatGPT can calculate the speed of an object in a video.

Be Aware of Quotas

• If ChatGPT says there is a server issue and it can’t create files after you have been using ChatGPT to generate a bunch of files, you should understand this as an intentional access denial related to a storage quota — even if ChatGPT doesn’t know it. You can try a new session; actually, you’ll need to because that session is never going to be able to read files. But you still may have to wait until it clears at the user level.
• If you are getting dinged on quotas for resources you’ve already used, try a new session. That will fix session-based quotas. If it is related to completing a new task, try breaking it into smaller chunks that require fewer resources. Or ask, “Can this be accomplished in a way that uses fewer resources?”

Utilize Feedback Loops

• Like humans, ChatGPT will rarely come up with the perfect solution on the first try. This is where collaboration is needed most. This is an iterative process where you refine the request based on initial feedback from ChatGPT. It can involve adjusting the task or providing additional details to guide the AI towards a more accurate completion. Use familiar debugging techniques when needed. Ask for incremental results. Note: This is for finalizing a solution- not trying to find one. Use this only after you have identified a solution and need to make adjustments to it.

Recognizing and understanding the refusal scenarios of ChatGPT is crucial for maximizing its potential. By applying the outlined mitigation strategies, users can overcome common obstacles, ensuring more productive and seamless interactions with the AI. As we continue to integrate AI into various aspects of our lives, mastering these collaborative techniques will become increasingly important for achieving desired outcomes efficiently and effectively.