How To Train Your Agent

Well, everyone tells you that agents will do things autonomously, but the reality is that most agents are not that autonomous and need careful planning and implementation. Most agentic demos that I’ve seen are exactly what the name says, DEMOS, not products.

Everything works in demos, but the real-world complexity and scale break these systems. In reality, these autonomous system are quite similar to classic software stack with intelligence in parts. We will try to understand about how to train your agent through one very interesting project called ART·E.

Table Of Contents

• Let’s Define Our Task
• Build Prompted Agent First
• How Well ART·E Performed?
• Hard Problems of RL
• Additional RL Objectives
• Conclusion

Let’s Define Our Task

OpenAI’s version of Deep Research showed how effective reinforcement learning (RL) can be for teaching an agent a specific task. Compared to previous research agents, it was a major step forward in effectiveness. With this new project, “ART·E”, AI researchers applied this training recipe to a new, realistic task: answering natural-language questions by searching an an email inbox.

The team produced a model that is faster, cheaper, and more accurate than o3 on this task. The task was realistic and useful, while still being narrow enough to see quick improvement through RL.