Environment-Grounded Automated Prompt Optimization for {LLM} Game Agents

{LLM} agents in interactive environments are highly sensitive to their prompts, yet prompt engineering remains a manual, task-specific process. We introduce an automated prompt optimization framework for {LLM} agents that decomposes the observation-to-action pipeline into a goal-conditioned descriptor agent and an action selection agent, and iteratively refines each module’s prompt through an {LLM}-driven evolutionary loop guided by environment returns. We propose a behavior analyzer to attribute episode outcomes to specific prompt components, and a mutator to propose targeted revisions to the prompt, before validating them through environment rollouts. We evaluate on all five {BabyAI} tasks in the {BALROG} benchmark, comparing our pipeline against {BALROG}’s {RobustCoTAgent} under both plain and guided prompt initializations. Optimization improves performance consistently across tasks and conditions, without requiring updates to the model weights. On {PutNext}, a multi-step coordination task where the {RobustCoTAgent} achieves 0\% success, our framework reaches up to 72.5\% success rate using the same underlying {LLM} with optimized prompts. These results suggest that a multi-agent framework, combined with automatic prompt optimization, enhances {LLMs} without the need for fine-tuning or extensive human supervision.