PreFlect is a prospective reflection framework for LLM agents that shifts error detection from post-hoc correction to pre-execution foresight. It criticizes and refines agent plans before any action is taken, using a set of distilled planning errors—recurrent failure and success patterns extracted from historical agent trajectories—as experiential anchors. A dynamic re-planning mechanism complements this by adaptively updating the plan during execution when the original plan becomes infeasible. Evaluated on GAIA and SimpleQA benchmarks, PreFlect significantly improves overall agent utility, outperforming strong reflection-based baselines and several more complex agent architectures. It also transfers across different agent frameworks (Smolagents and OWL).
Key Points
Shifts reflection from post-hoc correction to pre-execution foresight, decoupling error detection from irreversible outcomes.
Distills three domain-agnostic planning errors from offline trajectory analysis: insufficient constraint verification, ineffective tool selection, and shallow content verification.
Uses a reflector agent that cross-references generated plans against these error types to detect vulnerabilities and revise plans before execution.
Adds a dynamic re-planning trigger that continuously monitors execution and activates a new planning-and-reflection phase when progress stalls or feasibility conditions are violated.
Achieves state-of-the-art results on GAIA (58.18% with GPT-4.1, 59.39% with Gemini-2.5-pro), outperforming baselines by 17.14% and 11.68% respectively, and improves SimpleQA Correct metric by 12.79% on average.
Transfers successfully across agent architectures: adding PreFlect improves Smolagents from 46.06% to 58.18% and OWL from 50.30% to 60.61% on GAIA.
Ablations show that removing planning errors drops Level 3 performance from 38.46% to 23.08%, and removing dynamic re-planning reduces total score from 58.18% to 51.52%.
Concepts
Prospective Reflection: The process of evaluating and revising an agent’s plan before any action is taken. It uses distilled planning errors as grounded reference priors to detect vulnerabilities (e.g., hazardous tool usage) and replace the plan with a revised one.
Planning Errors: A taxonomy of recurrent failure and success patterns extracted offline from mixed-outcome trajectories. The three core types are insufficient constraint verification, ineffective tool selection, and shallow content verification. Each error type includes contrastive examples (success vs. failure).
Dynamic Re-Planning: An execution-time mechanism that monitors the agent’s trajectory. When repeated failures or feasibility violations occur, it triggers a new planning and prospective reflection phase conditioned on the execution history, generating an updated, validated plan before further actions.
Details
Architecture
PreFlect integrates a prospective reflection loop into the planning stage of an agent. After initial plan generation, a reflector (a separate instance of the same backbone LLM) examines the plan using the distilled planning errors. If a flaw is detected, the plan is revised iteratively until a validated plan is produced. This plan then enters execution, during which a dynamic re-planning monitor watches for stagnation or deviation. On trigger, a new planning and reflection phase is appended, reusing the same modules. All newly generated plans are prospectively validated before further actions.
Rendering diagram…
Planning Error Distillation
Planning errors are derived from an offline process using HotpotQA (50 hard-level training samples) and MuSiQue (50 training samples), disjoint from evaluation benchmarks. For each task, three trajectories are sampled with a base agent (Smolagents + GPT-4.1), yielding 300 trajectories. Only tasks with both successful and failed trajectories are retained (32 tasks: 18 HotpotQA, 14 MuSiQue). An LLM then performs comparative diagnostic analysis on these mixed trajectories, identifying critical errors that arise from planning deficiencies. 68 diagnoses are generated and aggregated by an LLM-based aggregator that iteratively merges similar entries, creates new categories, or discards redundancies. After manual refinement, three domain-agnostic error types remain:
Insufficient constraint verification: The plan identifies a potential answer but fails to verify it against all specific constraints (e.g., timeframes, subsets, exclusions). Example: identifying Richard Griffiths for a task requiring an actor in Sleepy Hollow and all six final Harry Potter films, without confirming his presence in every film.
Ineffective tool selection: Persisting with a tool despite known failures (e.g., using visit_webpage on a site that returns 403) without switching to alternatives like inspect_file_as_text.
Shallow content verification: Relying solely on search snippets or broad summaries without visiting actual pages, or extracting data without verifying definitions (geographic scope, units).
Reflection and Revision
The reflector executes a multi-turn conversation. In the first turn, it receives the task, plan, available tools, and a summary of past actions. It produces an analytical diagnosis using the error taxonomy. If a critical error is detected, a second turn revises the plan using the success/failure patterns from the planning errors collection. Core principles include pragmatism (flag at most two most critical errors), root‑cause focus, trajectory awareness, and constraint awareness of tool/environment limitations. The output is a JSON object with analysis result (PASS/FAIL) and detected errors.
Dynamic Re-Planning
The agent explicitly reasons about the current execution state and identifies why the existing plan is no longer effective before invoking re plan(). The call passes a string describing the difficulties (e.g., "Tool Failure: Cannot access primary source (Tesla IR page returns 403 Forbidden)."). After re-planning, the agent generates an updated strategy and resumes execution. At least two extra planning steps are enforced; if no updated planning step occurs between steps 1–6 or 7–12, the agent is forced to update the plan at steps 6 and 12.
Experimental Setup
PreFlect is built on Smolagents (CodeAgent class) with a maximum of 20 action steps. Tools include DuckDuckGo web search, a webpage visitor (using requests and markdownify), and modality-specific inspectors (text, visual, audio) powered by Gemini-3-flash-preview for multi-modal processing. All methods use the same minimal tool set and inference parameters.
Baselines: Reflexion, Self-Refine, ReAct, Smolagents with Open Deep Research (HF-ODR), AutoAgent, Magnetic-1, HAL, OWL, OAgent. Costs are calculated at $2/M input tokens and $8/M output tokens.
Metrics: GAIA uses pass@1; SimpleQA reports Correct, Incorrect, Not Attempted, and Correct Given Attempted counts. An LLM-as-a-judge (GPT-4o) determines correctness for SimpleQA.
Results
Benchmark
Backbone
PreFlect
Best Baseline
Improvement
GAIA total
GPT-4.1
58.18%
50.30% (HF-ODR)
+7.88%
GAIA total
Gemini-2.5-pro
59.39%
53.94% (OWL)
+5.45%
GAIA Level 3
GPT-4.1
38.46%
26.92% (OWL)
+11.54%
SimpleQA Correct
GPT-4.1
–
–
+12.79% avg
PreFlect also outperforms complex multi-agent frameworks: with GPT-4.1 it reaches 58.18% vs. AutoAgent 55.15%, OWL 53.33%, and others.
Qualitative example (GAIA Level 3): Task to find symbol meaning on Eva Draconis’s site. The agent initially hits 404/403 errors. At step 15 it re-plans to use the Wayback Machine, but the plan incorrectly relies on visual inspector for archived images. Prospective reflection detects ineffective tool selection (visit_webpage cannot retrieve images), and revises the plan to source-code excavation using visit_webpage on archived homepage snapshots. The agent successfully extracts a hidden textual anchor leading to a symbols glossary where symbol 10 reads "War is not here this is a land of peace."
Ablation and Analysis
With GPT-4.1 on GAIA (165 tasks):
Full PreFlect: 58.18% total.
Removing planning errors (PE): 50.30% total, Level 3 drops from 38.46% to 23.08%.
Error distribution among flagged plans (74.44% defect rate): insufficient constraint verification 64.92%, ineffective tool selection 32.84%, shallow content verification 2.24%.
Cost Analysis
Method
GAIA Score
Cost
PreFlect (GPT-4.1)
58.18%
$54.48
Smolagents baseline
46.06%
$46.31
HAL
49.70%
$74.19
HF-ODR
50.30%
$109.88
Transferability
PreFlect improves Smolagents from 46.06% to 58.18% total on GAIA (Level 1: 56.60→71.70, Level 2: 47.67→55.81, Level 3: 19.23→38.46) and OWL from 50.30% to 60.61% (Level 1: 60.38→73.58, Level 2: 51.16→58.14, Level 3: 26.92→42.31). This demonstrates that PreFlect is orthogonal to agent architectures and can be integrated as a reflection module.
Implementation Details
Base framework: Smolagents (Roucher et al., 2025) CodeAgent.
Planning prompt follows Smolagents default.
Reflector uses the same backbone LLM (GPT-4.1 or Gemini-2.5-pro) with separate prompts.
For re-planning, at least two extra planning steps are enforced; forced updates at steps 6 and 12 if no voluntary re-planning occurred.
Tools: web search (DuckDuckGo), visit_webpage (requests + markdownify), text inspector (handles .pdf, .docx, .pptx, .xlsx, .html), visual inspector (base64-encoded images via VLM), audio inspector (OpenAI Whisper-1). All multi-modal tools use Gemini-3-flash-preview uniformly.
Planning errors constructed using Gemini-3-pro-preview for long-context ability. The set of 32 tasks (18 HotpotQA, 14 MuSiQue) yielded 68 diagnoses → 3 error types after aggregation and manual filtering.
Related Work
PreFlect builds on prior work in LLM agents (ReAct, AutoAgent, OWL, Alita, Smolagents) and self-reflection (Reflexion, Self-Refine, AR, MIRROR, SAMULE). It distinguishes itself by focusing on prospective rather than retrospective reflection and by using empirically distilled planning errors as a reusable knowledge base. The framework is openly transferable to architectures such as Smolagents and OWL.