PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agents Hanyu Wang 1 Yuanpu Cao 1 Lu Lin 1 Jinghui Chen 1 Abstract Advanced large language model agents typically adopt self-reflection for improving performance, where agents iteratively analyze past actions to correct errors. However, existing reflective ap- proaches are inherently retrospective: agents act, observe failure, and only then attempt to recover. In this work, we introduce PreFlect, a prospec- tive reflection mechanism that shifts the paradigm from post hoc correction to pre-execution fore- sight by criticizing and refining agent plans be- fore execution. To support grounded prospective reflection, we distill planning errors from histori- cal agent trajectories, capturing recurring success and failure patterns observed across past execu- tions. Furthermore, we complement prospective reflection with a dynamic re-planning mechanism that provides execution-time plan update in case the original plan encounters unexpected devia- tion. Evaluations on different benchmarks demon- strate that PreFlect significantly improves overall agent utility on complex real-world tasks, out- performing strong reflection-based baselines and several more complex agent architectures. Code will be updated at https://github.com/ wwwhy725/PreFlect. 1. Introduction Large language model agents (Wu et al., 2024; Hu et al., 2025; Li et al., 2025), empowered by advanced LLMs (Guo et al., 2025a; Comanici et al., 2025; Yang et al., 2025), have demonstrated competence in handling various real-world tasks. However, when faced with tasks requiring highly com- plex reasoning and multiple tools handling, even a minor error may ruin all the efforts (Press et al., 2023). To over- come the fragility of single-pass reasoning, recent research 1College of Information Sciences and Technology, The Penn- sylvania State University, State College, PA, USA. Correspon- dence to: Hanyu Wang , Jinghui Chen . Preprint. February 10, 2026.Think of how to escape after falling Retrospective Prospective Proactively simulate risks along the path Figure 1. Retrospective vs. Prospective Reflection. (Left) The retrospective agent triggers reflection only after encountering a failure. (Right) The prospective agent anticipates potential risks before execution, allowing it to adjust its plan and successfully bypass the obstacle. has increasingly integrated self-reflection mechanisms into agent architecture (Gou et al., 2023; Shinn et al., 2023; Zhou et al., 2023). Unlike standard CoT approaches, reflection enables agents to act as their own critics: scrutinizing past execution trajectories, diagnosing action-level errors, and iteratively refining their future strategies (Madaan et al., 2023). Despite these advances, most existing reflection paradigms remain fundamentally retrospective, operating in a reac- tive manner that triggers correction only after failures have already occurred. While effective in some settings, such post-hoc reflection faces several inherent limitations in com- plex environments. Specifically, retrospective correction is often inadequate when actions produce irreversible con- sequences, such as accidentally deleting an important file. Additionally, diagnosing errors after execution can intro- duce substantial trajectory-level noise: by storing both failed attempts and subsequent fixes in memory, agents may suffer from contextual interference that destabilizes future decision-making. Furthermore, retrospective methods typ- ically rely on repeated trial-and-error cycles, leading to significant computational cost and increased token latency as agents iteratively recover from mistakes. To address these limitations, we propose Prospective Re- flection (PreFlect), a novel framework that shifts the re- flection paradigm from reactive correction to proactive pre- vention, equipping agents with a prospective capability to 1 arXiv:2602.07187v1 [cs.AI] 6 Feb 2026 PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agents anticipate potential pitfalls. Central to this shift is identify- ing when reflection should intervene in the agent’s decision process. Existing methods typically reflect only after ac- tions have been executed, when errors may already be costly or irreversible. In contrast, we observe that the planning stage provides a critical opportunity for proactive control: it is the point at which the agent commits to a strategy, yet has not begun to act on it. Building on this insight, we introduce a planning-phase reflection mechanism that oper- ates in the cognitive window between plan generation and action execution. By evaluating and revising plans before they are carried out, PreFlect decouples error detection from irreversible outcomes, allowing agents to scrutinize their strategies for potential risks prior to execution. However, the efficacy of prospective reflection is often con- strained by the inherent difficulty of anticipating future fail- ures. In the absence of structured guidance, aimless critique may even introduce more risks, such as hallucinations (Zhu et al., 2025b). To resolve this issue, we incorporate the “Planning Errors” into PreFlect, which serves as an experien- tial anchor by distilling recurrent failure and success patterns from past agent trajectories. Specifically, upon generating a plan, the agent cross-references its proposed strategies against the planning errors to diagnose specific vulnerabil- ities such as hazardous tool usage. This guided reflection allows the agent to transform abstract foresight into concrete diagnosis by identifying semantic similarities between the current plan and historical pitfalls. Consequently, the agent can proactively substitute risky components with reliable alternatives, resulting in a robust and optimal trajectory that is empirically grounded before any action is committed to the environment. Moreover, although the planning errors provide reliable and grounded references, proactively reflecting on future errors may still be insufficient under the uncertainty of complex environments. Relying solely on pre-execution critique can leave blind spots during execution and lead to oversights. Therefore, we incorporate a dynamic re-planning mecha- nism to complement planning-phase reflection by adaptively updating the agent’s plan during execution. Unlike tradi- tional architectures that adhere to rigid, fixed interval plan- ning cycles, the dynamic re-planning continuously monitors the execution state to evaluate whether the current trajectory remains viable or necessitates a strategic shift. For instance, if the agent originally plans to search for a certain query while repeated failures, indicating infeasibility, the agent can trigger re-planning to revise its strategy and pivot toward alternative approaches. Together, prospective reflection and dynamic re-planning form a unified framework that supports both proactive plan- ning and robust execution. To validate PreFlect, we conduct comprehensive experiments across multiple benchmarks. Results show that PreFlect achieves consistent performance improvements while maintaining reasonable token overhead. To sum up, our contributions are threefold: • We propose PreFlect, a prospective reflection frame- work that enables LLM agents to proactively identify and avoid critical failures before execution. • To further improve the effectiveness and efficiency of PreFlect, we introduce two key components: the planning errors and dynamic re-planning mechanism, which respectively provide experiential priors to assist reflection and adaptively revise plans during execution. • Extensive experiments demonstrate that PreFlect signif- icantly outperforms existing reflection-based baselines and competitive agent frameworks. Furthermore, trans- ferability results show that PreFlect generalizes across different agent architectures. 2. Related Work LLM Agents. LLM-based agents enhance reasoning by integrating external modules for planning, memory, and tool use. The ReAct (Yao et al., 2022) architecture established the iterative “reason-act” paradigm, which has since evolved into specialized research on test-time searching (Yao et al., 2023; Besta et al., 2024), tool tuning (Zeng et al., 2025; Qian et al., 2025; Jiang et al., 2025), and memory structur- ing (Xu et al., 2025; Chhikara et al., 2025). Beyond modular improvements, recent frameworks introduce diverse coordi- nation strategies: AutoAgent (Chen et al., 2023) and OWL (Hu et al., 2025) utilize dynamic multi-agent coordination and role-playing for task decomposition, while Alita (Qiu et al., 2025) enables autonomous expansion via refined task- specific MCPs. Finally, Smolagents (Roucher et al., 2025) prioritizes efficiency through lightweight, code-based ac- tions and local deployment integration. Self-Reflection. Self-reflection enables LLM agents to au- tonomously critique past executions and iteratively refine subsequent strategies. Reflexion (Shinn et al., 2023) es- tablished the foundation for this paradigm by introducing verbal reinforcement to store linguistic feedback to prevent recurring errors. Self-Refine (Madaan et al., 2023) employs an iterative loop where the model generates, critiques, and corrects its own output without external supervision. Recent advancements explore more granular reflection strategies: AR (Wang et al., 2024) focuses on real-time adaptation by preparing backup moves and switching to alternatives upon primary execution failure. MIRROR (Guo et al., 2025b) fur- ther extends this by integrating inter-agent communication with intra-agent self-critique. SAMULE (Ge et al., 2025) shifts from prompting to specialized training, fine-tuning a reflection LLM to provide high-quality feedback to an executor agent. 2 PreFlect: From Retrospective to Prospective Reflection in Large Language Model AgentsPlanning Errors Planning & Reflection Re-use same module PreFlect Workflow … Plan Action ObservationTask Input Yes Reflector Error Detected? Task Input Is Constraint Verified? Replace No Thought Task Output x N Dynamic Re-Planning Re-Plan? Yes No Planning & Reflection Thought … RevisionPlan Thought Action Observation Action Observation … Basic Workflow … Task Output Figure 2. The architecture of PreFlect comparing with basic agent workflow (top). PreFlect integrates a prospective reflection loop (pink box) into the planning stage. The bottom-right panel illustrates dynamic re-planning workflow. 3. Methodology We organize this section as follows: Section 3.1 provides an overview of our proposed method. In Section 3.2, we expand on the core concept of prospective reflection, with Section 3.2.1 introducing planning errors and Section 3.2.2 explaining how PreFlect reflects on and revises the plan. Finally, in Section 3.3, we show how dynamic re-planning and prospective reflection complement each other. 3.1. Overview of PreFlect Figure 2 provides an overview of PreFlect by contrasting it with a standard agent workflow and illustrating how prospec- tive reflection and execution-time dynamic re-planning are integrated into a unified loop. The top of the figure shows the basic agent workflow. Given a task input, the agent first generates a plan that decomposes the task into a se- quence of steps based on the currently known conditions. The agent then executes the plan through an iterative think– act–observe loop, during which it reasons about the next step, takes an action, and observes the environment. This execution loop repeats until the task is completed, at which point the final output is produced. The bottom of the figure depicts the workflow of PreFlect. The left part highlights the planning and prospective reflection module. After an initial plan is generated, a reflector examines the plan be- fore execution by leveraging a set of distilled planning er- rors, which summarize common failure and success patterns observed from historical agent trajectories. The reflector detects potential planning errors and revises the plan ac- cordingly. This process iterates until a validated plan is produced, which is then passed to the execution stage. The right part of the figure shows how execution proceeds under PreFlect and how execution-time dynamic re-planning is handled. Similar to the basic workflow, the agent executes the validated plan using the think–act–observe loop. During execution, the agent continuously monitors the trajectory. When the current trajectory becomes infeasible or deviates from the validated plan, a re-planning trigger is activated. Importantly, dynamic re-planning does not rewind or dis- card the past trajectory; instead, it appends a new planning and prospective reflection phase that is conditioned on the execution history so far. As illustrated inside the dynamic re-planning workflow, this re-planning step reuses the same planning and prospective reflection module shown on the left, ensuring that all newly generated plans, whether initial or re-generated during execution, are validated prospectively before further actions are taken. Overall, PreFlect forms a single closed-loop system in which prospective reflection is applied both prior to initial execu- tion and whenever re-planning is required, enabling the agent to anticipate and mitigate planning errors before they manifest as irreversible execution failures. In the following, we will step into the design details of each component. 3.2. Prospective Reflection The core component of our proposed PreFlect is a prospec- tive reflection mechanism where the agent reflects on the plan based on distilled error patterns, identifies critical er- rors, and replaces the original plan with a revised one to avoid future failures. 3.2.1. PLANNING ERRORS Unlike retrospective reflection, which corrects errors through post-execution trajectory review, prospective re- flection must reason about potential failures before they occur, and is therefore subject to greater uncertainty. With- out grounded guidance, such forward-looking critique can become unreliable or overly speculative. To address this 3 PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agents challenge, we adopt an offline distillation process that ex- tracts common error modes and success/failure patterns from an agent’s past experience, providing experiential pri- ors that enable more accurate prospective reflection. Trajectory Collection. Constructing Planning Errors re- quires leveraging the agent’s own experience across diverse tasks. For each task, we sample three trajectories and iden- tify cases with mixed outcomes, where the agent produces both successful and failed attempts. This contrastive set- ting highlights the key differences between effective and ineffective strategies, facilitating the extraction of recurring planning-level failure patterns. To promote generalization and avoid overfitting, the data used for Planning Error con- struction is disjoint from all evaluation benchmarks. Diagnosis. Given these mixed trajectories, we prompt an LLM to conduct a comparative diagnostic analysis. The diagnosis focuses on identifying critical errors in failed tra- jectories that arise from planning deficiencies and could be avoided through improved planning. In parallel, it an- alyzes how successful trajectories circumvent or resolve these pitfalls. Each diagnostic result produces one or more error types, along with corresponding descriptions, impacts, and supporting evidence. This process allows us to isolate the exact planning failures that differentiate success from failure. An Example in Planning Errors Error Type: insufficient constraint verification Description: The plan identifies a potential answer but fails to rigorously verify it against all specific constraints or conditions in the prompt . . . It relies on . . . without including steps to explicitly confirm that the candidate satisfies every requirement . . . Success Example: In a task identifying an actor . . . the agent identified Michael Gambon and correctly verified that . . . ensuring he met the specific constraint. Failure Example: In a task identifying an actor . . . the agent identified Richard Griffiths because . . . However, the plan failed to verify . . . leading to an incorrect answer. Aggregation. The final stage of our framework synthesizes individual diagnoses into a unified taxonomy of planning errors. We employ an LLM-based aggregator to construct this taxonomy by iteratively comparing each new diagnos- tic entry against the existing error set. For each entry, the aggregator determines whether to instantiate a new error category, merge it into an existing profile, or discard it as redundant. This process distills generalizable planning-level failure modes that transcend specific task instances, ensur- ing broad applicability. In parallel, the LLM summarizes empirical evidence and trajectory impacts into concrete suc- cess and failure patterns. The resulting Planning Errors therefore connect high-level reasoning deficiencies with re- curring behavioral signatures, providing informative priors for downstream reflection. After the initial LLM-driven synthesis, we perform a final manual refinement step to re- move categories that are overly narrow or task-dependent (e.g., errors specific to a particular query formulation). This yields a finalized set of three core error types: 1) insuf- ficient constraint verification 2) ineffective tool selection 3) shallow content verification, capturing failures in con- straint maintenance, tool usage, and content understanding. Notably, these errors are designed to be domain-agnostic, maximizing their utility across diverse agentic tasks. For more details, please refer to Appendix B. 3.2.2. REFLECTION & REVISION Error Identification. After a plan is generated, a reflec- tor agent evaluates it to identify potential critical errors before execution. To improve accuracy and reduce false pos- itives, the reflector leverages the distilled Planning Errors as grounded reference priors. By conditioning reflection di- rectly on these domain-agnostic failure modes, the agent can perform more structured and reliable prospective critique. Particularly, the agent summarizes its previous actions to have a comprehensive understanding of the task and cur- rent states. Meanwhile, the agent analyzes available tools and learns how and why some tools fail or succeed in the environment of the current task. This self-awareness helps foster more accurate and fact-based prospective reflection, enabling foresight into future scenarios, such as knowing whether a tool should be avoided under a certain circum- stance before even attempting it. After gathering enough information about the current states, the reflector critiques the proposed plan by checking whether it exhibits patterns aligned with any known Planning Errors. If a flaw is de- tected, the agent revises the plan accordingly, producing a higher-quality strategy prior to action execution. Plan Revision. Given the identified errors in the previous step, the agent now switches its attention to how to com- prehensively develop an updated plan to impact the future positively. Notably, the planning errors collection contains contrastive examples of short trajectory segments where positive examples demonstrate how the agent avoids such an error, while negative ones expand on how the agent is misled to failure. Combining the high-level error descrip- tion with these concrete cases, the agent is enabled to match its current state with the error information and find out the optimal path toward success. 3.3. Dynamic Re-Planning Although Planning Errors provide grounded experiential priors for prospective reflection, reasoning about future 4 PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agents failures remains inherently uncertain, and the agent may still overlook critical risks during execution. To comple- ment planning-phase reflection, we introduce a dynamic re-planning mechanism that enables the agent to adapt its strategy online when the environment reveals unexpected ob- stacles. Notably, whenever re-planning is triggered, prospec- tive reflection is invoked again to assess potential risks in the updated plan. Dynamic Plan Updating. During execution, the agent con- tinuously gathers new observations through interaction with the environment. Such feedback may invalidate earlier as- sumptions, causing the original plan to become suboptimal or infeasible. Since many execution-time constraints are only revealed at runtime (e.g., unavailable tool outputs or missing external information), proactive reflection alone cannot guarantee optimal trajectories under evolving states. To address this, we equip the agent with a re-plan op- eration that can be triggered whenever progress stalls or feasibility conditions are violated. Following the ReAct paradigm, the agent explicitly reasons about the current exe- cution state and identifies why the existing plan is no longer effective before invoking re-planning. The agent then gener- ates an updated strategy and immediately resumes execution under the revised plan. Implementation details are provided in Appendix A. 4. Experiments 4.1. Experiment Settings Datasets. To evaluate the general task completion perfor- mance of PreFlect, we conduct comprehensive experiments on the following datasets with complete agentic settings. 1) GAIA (Mialon et al., 2023) is a general AI assistant benchmark that evaluates AI agent’s ability to complete real-world tasks, covering multi-step reasoning, fact check- ing, web browsing, and tool usage. We select the valida- tion set. 2) SimpleQA (Wei et al., 2024) is a challenging benchmark that evaluates the ability to answer fact-seeking questions. Due to resource limitations, we randomly select 100 samples from the test set. Note that for constructing the planning errors, we leverage HotpotQA (Yang et al., 2018) and MuSiQue (Trivedi et al., 2022) datasets, ensuring no overlap with the test benchmarks while challenging the agent’s diverse capabilities. Baselines. We compare with various baselines to validate the effectiveness of our proposed method. In particular, we select Reflexion (Shinn et al., 2023) and Self-Refine (Madaan et al., 2023), which are standard retrospective re- flection methods. We omit Wang et al. (2024) as it tar- gets step-level anticipation and is not directly applicable to GAIA-style long-horizon planning tasks. Also, we compare with several agent frameworks, such as ReAct (Yao et al., 2022), Smolagents and its Open Deep Research (ODR) sys- tem (Roucher et al., 2025), AutoAgent (Chen et al., 2023), Magnetic-1 (Fourney et al., 2024), HAL (Stroebl et al., 2025), OWL (Hu et al., 2025), and OAgent (Zhu et al., 2025a). Evaluation Metrics. For GAIA, we use standard “pass@1” which refers to the percentage of problems solved correctly when 1 candidate solution is sampled per task. For Sim- pleQA, we follow the standard evaluation to report Correct (Corr.), Incorrect (Incor.), Not Attempted (N.A.), and Correct Given Attempted (C./Att.), which refer to the number of correct, incorrect, not attempted, and correct given attempted answers out of all the answers, respectively. Agent Implementations. PreFlect (built upon Smolagents) operates under a maximum budget of 20 execution steps and is equipped with standard web search and webpage navigation tools. To satisfy the multimodal tool require- ments of GAIA tasks (e.g., involving images or audio), we additionally provide a set of lightweight modality-specific inspection tools. Importantly, all compared methods are evaluated under the same tool access and execution con- straints to ensure a fair comparison in our main experiments. Further implementation details are deferred to Appendix A. 4.2. Main Results Table 1 presents the primary experimental results compar- ing PreFlect against various baselines on the GAIA and SimpleQA benchmarks. To ensure a rigorous and equitable comparison, we maintain strictly identical settings, such as tool set and inference parameters, across all evaluated methods, thereby eliminating confounding factors related to environmental configuration. Quantitative Analysis. Table 1 presents the performance of PreFlect compared to representative agentic and reflective baselines on GAIA and SimpleQA benchmarks. On GAIA, PreFlect achieves the highest total score of 58.18% (GPT- 4.1) and 59.39% (Gemini-2.5-pro), representing an average improvement of 17.14% and 11.68% over baselines. Be- sides, the performance gap on Level 3 tasks, where PreFlect outperforms baselines by 14.29% on average, suggests that prospective reflection excels at high-complexity reasoning where error propagation is most severe. On the SimpleQA benchmark, PreFlect significantly en- hances factuality, achieving an average improvement of 12.79% in the Correct metric compared to all baselines across both LLM backbones. The simultaneous increase in the Correct Given Attempted metric (6.08% on average for GPT-4.1 and 7.73% on average for Gemini-2.5-pro) and decrease in the Not Attempted metric (12.29% on average for GPT-4.1 and 4.00% on average for Gemini-2.5-pro) in- dicate that our method allows agents to locate grounded 5 PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agents Table 1. Main experimental results on GAIA and SimpleQA benchmarks. PreFlect is built upon Smolagents framework. All the results are using the pass@1 metric. The best results are shown in bold. Agent Framework Reflective Method GAIA (%) SimpleQA (%) Level 1 (↑) Level 2 (↑) Level 3 (↑) Total (↑) Corr. (↑) Incor. (↓) N.A. (↓) C./Att. (↑) Backbone LLM: GPT-4.1 ReAct - 41.51 33.72 7.69 32.12 61 19 20 76.25 ReAct Reflexion 49.06 34.88 15.38 36.36 71 21 8 77.17 ReAct Self-Refine 39.62 39.53 11.54 35.15 74 18 8 80.43 Smolagents - 56.60 47.67 19.23 46.06 72 16 12 81.82 Smolagents Reflexion 56.60 54.65 19.23 49.70 79 20 1 79.80 Smolagents Self-Refine 58.49 52.33 23.08 49.70 78 18 4 81.25 Smolagents PreFlect (ours) 71.70 55.81 38.46 58.18 83 15 2 84.69 Backbone LLM: Gemini-2.5-pro ReAct - 56.60 45.35 30.77 46.67 55 16 29 77.46 ReAct Reflexion 56.60 47.67 30.77 47.88 70 18 12 79.55 ReAct Self-Refine 56.60 44.19 34.62 46.67 62 24 14 72.09 Smolagents - 54.72 52.32 34.62 50.30 76 18 6 80.85 Smolagents Reflexion 60.38 52.33 38.46 52.73 78 22 0 78.00 Smolagents Self-Refine 60.38 50.00 34.62 50.91 81 17 2 82.65 Smolagents PreFlect (ours) 67.92 60.47 38.46 59.39 81 13 6 86.17 information. Different from traditional methods to improve the factuality of an LLM without tools (Wang et al., 2025a; Cao et al., 2024), the performance gain under the agentic setting demonstrates that our proposed method results in a more comprehensive and robust system that further avoids hallucinations. Qualitative Analysis. Figure 3 illustrates how PreFlect handles a complex Level-3 GAIA task to identify the mean- ing of a specific symbol on Eva Draconis’ personal website which can be accessed on her YouTube page. The task is particularly challenging because both the target YouTube channel and the personal webpage is no longer active. While the agent initially struggles for 14 steps with empty pages and missing links, PreFlect triggers a “turning point” at step 15 via prospective reflection. Recognizing that standard searches are futile, the agent performs dynamic re-planning to pivot toward the Wayback Machine. Crucially, to mitigate potential ineffective tool selection problem, the revised plan shifts from direct image retrieval to source-code excavation. By extracting raw HTML via the “visit webpage” tool, the agent successfully locates the archived symbol description, completing a task that would otherwise result in a failure state. 4.3. Comparison with Complex Agent Frameworks Beyond the controlled reflection-based comparisons in Ta- ble 1, Table 2 situates PreFlect within a broader landscape of sophisticated agent frameworks. These systems are typically evaluated under non-standardized and substantially more resource-rich settings, often involving more powerful tool access, different backbone models, larger action budgets, and specialized prompt or memory designs. As a result, their reported performance is not directly comparable to our controlled experimental setting. We therefore cite these results directly from the original publications and present them separately to provide qualitative context rather than a strict head-to-head evaluation. Detailed configurations and sources for these agent frameworks are provided in Appendix C. Table 2. Comparative results on the GAIA validation set between various agentic frameworks. Results are directly cited from their respective original reports or other sources. Frameworks Backbone LLM GAIA (%) L1 (↑) L2 (↑) L3 (↑) Total (↑) AutoAgent Claude-Sonnet-3.5 71.70 53.49 26.92 55.15 Magnetic-1 o1 56.60 46.51 23.08 46.06 HAL Agent GPT-4.1 52.83 55.81 23.08 49.70 HF-ODR GPT-4.1 58.49 50.00 34.62 50.30 HF-ODR o1 67.92 53.49 34.62 55.15 OWL GPT-4.1 71.70 50.00 26.92 53.33 OAgent GPT-4.1 67.92 53.49 34.62 55.15 Smolagents + PreFlect GPT-4.1 71.70 55.81 38.46 58.18 Smolagents + PreFlect Gemini-2.5-pro 67.92 60.47 38.46 59.39 As shown in Table 2, PreFlect outperforms significantly more complex multi-agent architectures (e.g., AutoAgent and OWL) by over 3%. These results underscore that ef- fective prospective reflection can surpass simply increasing execution steps or orchestrating additional agent roles. Im- portantly, PreFlect is orthogonal to these frameworks and can in principle be integrated as a reflection module to fur- ther improve their planning and execution robustness. 4.4. Transferability Although we primarily instantiate PreFlect within the Smo- lagents framework, we further evaluate its transferability 6 PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agents Qualitative Analysis: An Example From GAIA Level 3 Task ü Task: Eva Draconis has a personal website which can be accessed on her YouTube page. What is the meaning of the only symbol seen in the top banner that has a curved line that isn’t a circle or a portion of a circle? Answer without punctuation. Ç Ground Truth Trajectory: 1. Google Eva Draconis YouTube → 2. Find her website URL in her about section → 3. Enter to identify symbols and their meanings → 4. Find the only symbol that matches all the requirements → 5. Note the meaning “War is not here, this is a land of peace.” @ Step 1–14: Plans, Actions, and Failure Agent initiated a multi-step search strategy: scraping YouTube “About” sections, scanning social media (Facebook/Pa- treon), and querying Google Images. Outcome: All attempts resulted in 404/403 errors or irrelevant results (e.g., Space.com forums), as the primary site (orionmindproject.com) was defunct. Dynamic Re-Planning (Step 15) Thought: All searches for . . . for orionmindproject.com and Eva Draconis site are futile . . . A new approach is mandatory. Re-plan Proposal: The agent proposed to double-down on Wayback Machine visual captures, attempting to retrieve and process the banner image via the inspector tool. ✓ Plan Reflection & Revision Reflection Analysis: [ineffective tool selection] The plan . . . to locate and process a banner image . . . but . . . visit webpage tool cannot retrieve images, and the inspector tools require a publicly accessible image URL . . . The plan does not include a strategy to overcome this technical constraint. Revised Plan: Pivot from visual retrieval to source-code excavation. Propose using ‘visit webpage’ to extract the HTML or text contents of the homepage for archived homepage snapshots of orionmindproject.com. ÷ Final Execution (Step 16-18): → Action: visit webpage on archived source code. → Key Observation: Detected a hidden textual anchor: “the alien writing on the top is... Kesovan symbols... see what they mean here.” → Result: Accessed symbols glossary. Symbol 10 (Leaning curve): “War is not here this is a land of peace”  Figure 3. An example of how PreFlect triggers dynamic re-planning, reflects on the plan based on the planning errors, and finally revises the plan to avoid failure. across different agent architectures. Specifically, we inte- grate PreFlect into a distinct agent framework, OWL (Hu et al., 2025). To ensure a fair comparison, we evaluate both variants under the same setting as in our main experiments (identical tool access, execution budgets, and hyperparam- eter settings), and report pass@1 accuracy on the GAIA validation set. Table 3. Transferability results across different agentic systems. Frameworks Level 1 Level 2 Level 3 Total Smolagents 56.60 47.67 19.23 46.06 + PreFlect 71.70 55.81 38.46 58.18 OWL 60.38 51.16 26.92 50.30 + PreFlect 73.58 58.14 42.31 60.61 As shown in Table 3, PreFlect consistently improves per- formance over the corresponding backbone, indicating that prospective reflection can serve as a general add-on module beyond a single agent implementation. 4.5. Error Distribution To further reveal the error patterns exhibited by our proposed reflection mechanism, we analyze the reflective processes and error recognition of PreFlect with GPT-4.1 backbone on the GAIA validation set. Across all 165 tasks, the de- fect rate, defined as the proportion of plans identified as risky among all updated plans, is 74.44%, revealing that most of the plans are reflected as carrying critical errors. As illustrated in Figure 4, the error distribution is heavily skewed: insufficient constraint verification (64.92%) and ineffective tool selection (32.84%) account for nearly all identified errors. This concentration suggests a fundamental gap in planning. 7 PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agents While the agent aims for concise, high-level strategies, the high frequency of constraint-related and tool-related errors reveals that critical task boundaries and reasonable tool selection are often lost in abstraction. This indicates that for complex reasoning tasks, conciseness may come at the cost of operational rigor. Without explicit grounding in task- specific environments, the agent tends to neglect something that can be the key to completing the task correctly.Insufficient Constraint Verification Ineffective Tool Selection Shallow Content Verification 0 10 20 30 40 50 60 70 80 Percentage of Identified Errors 64.92% 32.84% 2.24% Figure 4. Error distribution of PreFlect on GAIA using GPT-4.1 as the backbone LLM. 4.6. Ablations To rigorously assess the individual contributions of the Plan- ning Errors (PE) and Dynamic Re-Planning (DRP) modules, we conducted an ablation study using Smolagents as our baseline framework. The results, summarized in Table 4, isolate the impact of removing each component while keep- ing the experimental setting consistent. Table 4. Ablation studies on the planning errors (denoted as PE) and the dynamic re-planning (denoted as DRP). Results are from GPT-4.1 as the backbone LLM on the GAIA validation set. Method Level 1 Level 2 Level 3 Total Smolagents 56.60 47.67 19.23 46.06 PreFlect w/o PE 58.49 53.49 23.08 50.30 PreFlect w/o DRP 54.72 54.65 34.62 51.52 PreFlect 71.70 55.81 38.46 58.18 Planning Errors as Empirical Priors. The ablated results indicate that the Planning Errors is essential for the transi- tion from retrospective to prospective reflection. Removing it leads to a significant performance drop, particularly on Level 3 tasks (decreasing from 38.46% to 23.08%). This aligns with our hypothesis that predicting future errors is inherently difficult without historical context. PE acts as an “empirical prior”, grounding the prospective reflection and allowing the agent to anticipate and avoid potential pitfalls before execution begins. Dynamic Re-Planning as Compensation. While PE aids in pre-execution error avoidance, the w/o DRP variant reveals the necessity of runtime flexibility. Despite having access to historical error experiences, the agent may still fail to detect or mitigate every risk during the planning phase. The 6.66% total performance gap between PreFlect and w/o DRP demonstrates that re-planning serves as a critical safety net, without which, the added complexity of prospective reflection might lead to rigid plans that the agent cannot recover from if a minor execution error occurs. 4.7. Cost Analysis To evaluate the practical scalability of PreFlect, we ana- lyze its operational costs relative to performance on the GAIA validation set. We compare results of GPT-4.1 as the backbone LLM whose input price is $2 per million tokens while the output price is $8 per million tokens. As illus- trated in Figure 5, PreFlect achieves a superior performance- cost trade-off compared to existing frameworks. While the baseline Smolagents costs $46.31, our method reaches a highest score of 58.18% with only a marginal 17.64% cost increase ($54.48). In stark contrast, advanced agents like HAL and HF-ODR incur significantly higher expenditures ($74.19 and $109.88, respectively) yet yield lower perfor- mance (49.70% and 50.30%).Smolagents HAL Agent HF-ODR PreFlect 20 40 60 80 100 120 140 Total Cost (USD) $46.3 $74.2 $109.9 $54.5 Total Cost GAIA Score 44 49 54 59 64 69 74 GAIA Score (%) 46.06 49.70 50.30 58.18 Figure 5. Performance-cost trade-off on the GAIA validation set. The primary axis shows the total cost (USD) and the secondary axis shows the corresponding GAIA scores. 5. Conclusion In this work, we propose PreFlect, a prospective reflection mechanism that shifts from reactive correction to proac- tive failure prevention. Our approach introduces distilled planning errors to provide structured and grounded error detection, while a dynamic re-planning mechanism brings up an updated plan in case errors are neglected during ex- ecution. Experimental results demonstrate that PreFlect’s significant improvements in complex task solving and re- markable transferability across different agent architectures. 8 PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agents Impact Statement This work introduces a novel prospective reflection mech- anism to enhance general capability of LLM agents. By addressing significant weaknesses of previous reflection methods, PreFlect contributes to the development of more reliable and advanced agent systems. References Besta, M., Blach, N., Kubicek, A., Gerstenberger, R., Pod- stawski, M., Gianinazzi, L., Gajda, J., Lehmann, T., Niewiadomski, H., Nyczyk, P., et al. Graph of thoughts: Solving elaborate problems with large language models. In Proceedings of the AAAI conference on artificial intel- ligence, volume 38, pp. 17682–17690, 2024. Cao, Y., Zhang, T., Cao, B., Yin, Z., Lin, L., Ma, F., and Chen, J. Personalized steering of large language mod- els: Versatile steering vectors through bi-directional pref- erence optimization. Advances in Neural Information Processing Systems, 37:49519–49551, 2024. Chen, G., Dong, S., Shu, Y., Zhang, G., Sesay, J., Karls- son, B. F., Fu, J., and Shi, Y. Autoagents: A frame- work for automatic agent generation. arXiv preprint arXiv:2309.17288, 2023. Chhikara, P., Khant, D., Aryan, S., Singh, T., and Yadav, D. Mem0: Building production-ready ai agents with scalable long-term memory. arXiv preprint arXiv:2504.19413, 2025. Comanici, G., Bieber, E., Schaekermann, M., Pasupat, I., Sachdeva, N., Dhillon, I., Blistein, M., Ram, O., Zhang, D., Rosen, E., et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261, 2025. Fourney, A., Bansal, G., Mozannar, H., Tan, C., Salinas, E., Niedtner, F., Proebsting, G., Bassman, G., Gerrits, J., Alber, J., et al. Magentic-one: A generalist multi- agent system for solving complex tasks. arXiv preprint arXiv:2411.04468, 2024. Ge, Y., Romeo, S., Cai, J., Sunkara, M., and Zhang, Y. Samule: Self-learning agents enhanced by multi-level reflection. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 16602–16621, 2025. Gou, Z., Shao, Z., Gong, Y., Shen, Y., Yang, Y., Duan, N., and Chen, W. Critic: Large language models can self- correct with tool-interactive critiquing. arXiv preprint arXiv:2305.11738, 2023. Guo, D., Yang, D., Zhang, H., Song, J., Zhang, R., Xu, R., Zhu, Q., Ma, S., Wang, P., Bi, X., et al. Deepseek-r1: In- centivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025a. Guo, Z., Xu, B., Wang, X., and Mao, Z. Mirror: Multi-agent intra-and inter-reflection for optimized reasoning in tool learning. arXiv preprint arXiv:2505.20670, 2025b. Hu, M., Zhou, Y., Fan, W., Nie, Y., Xia, B., Sun, T., Ye, Z., Jin, Z., Li, Y., Chen, Q., et al. Owl: Optimized workforce learning for general multi-agent assistance in real-world task automation. arXiv preprint arXiv:2505.23885, 2025. Jiang, D., Lu, Y., Li, Z., Lyu, Z., Nie, P., Wang, H., Su, A., Chen, H., Zou, K., Du, C., et al. Verltool: Towards holis- tic agentic reinforcement learning with tool use. arXiv preprint arXiv:2509.01055, 2025. Li, X., Dong, G., Jin, J., Zhang, Y., Zhou, Y., Zhu, Y., Zhang, P., and Dou, Z. Search-o1: Agentic search-enhanced large reasoning models. arXiv preprint arXiv:2501.05366, 2025. Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., et al. Self-refine: Iterative refinement with self- feedback. Advances in Neural Information Processing Systems, 36:46534–46594, 2023. Mialon, G., Fourrier, C., Wolf, T., LeCun, Y., and Scialom, T. Gaia: a benchmark for general ai assistants. In The Twelfth International Conference on Learning Represen- tations, 2023. Press, O., Zhang, M., Min, S., Schmidt, L., Smith, N. A., and Lewis, M. Measuring and narrowing the composi- tionality gap in language models. In Findings of the As- sociation for Computational Linguistics: EMNLP 2023, pp. 5687–5711, 2023. Qian, C., Acikgoz, E. C., He, Q., Wang, H., Chen, X., Hakkani-T¨ur, D., Tur, G., and Ji, H. Toolrl: Reward is all tool learning needs. arXiv preprint arXiv:2504.13958, 2025. Qiu, J., Qi, X., Zhang, T., Juan, X., Guo, J., Lu, Y., Wang, Y., Yao, Z., Ren, Q., Jiang, X., et al. Alita: Generalist agent enabling scalable agentic reasoning with minimal predefinition and maximal self-evolution. arXiv preprint arXiv:2505.20286, 2025. Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., and Sutskever, I. Robust speech recognition via large- scale weak supervision. In International conference on machine learning, pp. 28492–28518. PMLR, 2023. 9 PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agents Roucher, A., del Moral, A. V., Wolf, T., von Werra, L., and Kaunism¨aki, E. ‘smolagents‘: a smol library to build great agentic systems. https://github.com/ huggingface/smolagents, 2025. Shinn, N., Cassano, F., Gopinath, A., Narasimhan, K., and Yao, S. Reflexion: Language agents with verbal rein- forcement learning. Advances in Neural Information Processing Systems, 36:8634–8652, 2023. Stroebl, B., Kapoor, S., and Narayanan, A. Hal: A holistic agent leaderboard for centralized and repro- ducible agent evaluation. https://github.com/ princeton-pli/hal-harness, 2025. Trivedi, H., Balasubramanian, N., Khot, T., and Sabharwal, A. musique: Multihop questions via single-hop ques- tion composition. Transactions of the Association for Computational Linguistics, 10:539–554, 2022. Wang, H., Li, T., Deng, Z., Roth, D., and Li, Y. Devil’s advocate: Anticipatory reflection for llm agents. arXiv preprint arXiv:2405.16334, 2024. Wang, H., Cao, B., Cao, Y., and Chen, J. Truthflow: Truthful llm generation via representation flow correction. arXiv preprint arXiv:2502.04556, 2025a. Wang, N., Hu, X., Liu, P., Zhu, H., Hou, Y., Huang, H., Zhang, S., Yang, J., Liu, J., Zhang, G., et al. Efficient agents: Building effective agents while reducing cost. arXiv preprint arXiv:2508.02694, 2025b. Wei, J., Karina, N., Chung, H. W., Jiao, Y. J., Papay, S., Glaese, A., Schulman, J., and Fedus, W. Measuring short- form factuality in large language models. arXiv preprint arXiv:2411.04368, 2024. Wu, Q., Bansal, G., Zhang, J., Wu, Y., Li, B., Zhu, E., Jiang, L., Zhang, X., Zhang, S., Liu, J., et al. Autogen: Enabling next-gen llm applications via multi-agent conversations. In First Conference on Language Modeling, 2024. Xu, W., Liang, Z., Mei, K., Gao, H., Tan, J., and Zhang, Y. A-mem: Agentic memory for llm agents. arXiv preprint arXiv:2502.12110, 2025. Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhut- dinov, R., and Manning, C. D. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 conference on empirical methods in natural language processing, pp. 2369–2380, 2018. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K. R., and Cao, Y. React: Synergizing reasoning and acting in language models. In The eleventh international conference on learning representations, 2022. Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T., Cao, Y., and Narasimhan, K. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36:11809–11822, 2023. Zeng, Y., Ding, X., Hou, Y., Wang, Y., Du, L., Dai, J., Ding, Q., Tang, D., Tu, D., Liu, W., et al. Tool zero: Training tool-augmented llms via pure rl from scratch. In Find- ings of the Association for Computational Linguistics: EMNLP 2025, pp. 9135–9147, 2025. Zhou, A., Yan, K., Shlapentokh-Rothman, M., Wang, H., and Wang, Y.-X. Language agent tree search unifies reasoning acting and planning in language models. arXiv preprint arXiv:2310.04406, 2023. Zhu, H., Qin, T., Zhu, K., Huang, H., Guan, Y., Xia, J., Yao, Y., Li, H., Wang, N., Liu, P., et al. Oagents: An empir- ical study of building effective agents. arXiv preprint arXiv:2506.15741, 2025a. Zhu, K., Liu, Z., Li, B., Tian, M., Yang, Y., Zhang, J., Han, P., Xie, Q., Cui, F., Zhang, W., et al. Where llm agents fail and how they can learn from failures. arXiv preprint arXiv:2509.25370, 2025b. 10 PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agents A. Implementation Details A.1. Agent Framework We build PreFlect based upon Smolagents (Roucher et al., 2025) framework. Specifically, we add the prospective reflection mechanism and dynamic re-planning to the CodeAgent class. Therefore, the agent that executes all the tool calls and reasonings is a single agent. Due to resource limitations, we set the maximum action steps to 20. For re-planning, we enforce at least 2 extra planning steps to ensure there’s room for the agent to perform periodic summarization and planning. Therefore, except for the initial planning stage, if there is not at least one updated planning step between the 1st to 6th steps, and 7th to 12th steps, we enforce the agent to update the plan at the 6th and 12th steps. The prompt for planning follows the default prompt in Smolagents for the CodeAgent class. A.2. Tools We use the minimum necessary tools for both GAIA and SimpleQA, and all the experiments except Table 2 are done with the same tool set. • Web Search: Due to resource limitations, we use the default web search tool in Smolagents and use the DuckDuck- GoSearch as the search engine. • Visit Webpage: Similarly, we use the default visit webpage tool in Smolagents, which is powered by ‘requests’ and ‘markdownify’ libraries. • Text Inspector: This tool enables the agent to parse diverse document types (e.g., .pdf, .docx, .pptx, .xlsx, .html) into Markdown text via a MarkdownConverter. It supports both raw content retrieval and query-based analysis, returning structured responses with brief answers, detailed analysis, and document context. This tool can also be found in Smolagent’s example of using the ODR agent to run GAIA tasks. • Visual Inspector: Designed for image processing (.jpg, .png, .gif, .bmp), this tool utilizes a Vision-Language Model to interpret visual data. It automatically generates detailed captions or performs targeted visual question-answering by encoding images into Base64 for API-based inference. • Audio Inspector: This tool facilitates auditory analysis by transcribing files (.mp3, .m4a, .wav) using the OpenAI Whisper-1 model (Radford et al., 2023). It provides either a direct transcription or a structured three-part response (brief, detailed, and contextual) to specific natural language questions regarding the audio content. For the LLM-powered tools, we uniformly adopt Gemini-3-flash-preview (i.e., gemini-3-flash-preview) for all the experiments in order to ensure the most advanced multi-modal processing ability. A.3. Reflection We utilize a separate agent (i.e., the reflector) to handle prospective reflection in order to achieve maximum lightweight and clear context. Specifically, whenever a plan is generated, we call an agent powered by the same backbone LLM to execute a multi-turn conversation where both plan reflection and revision are completed. The system prompt specifies the planning errors and instructions, and assigns the role of a reflector to the agent. In the first turn of conversation, we input the task, plan, available tools, and a summarization of past actions, and expect to obtain an analytical diagnosis of the plan. If any critical error is detected, we enter the second turn to revise the plan, utilizing the success and failure patterns in the planning errors as a reference. All the prompts are provided in Appendix D. A.4. LLM Settings We call OpenAI API to use GPT-4.1 model (i.e., gpt-4.1-2025-04-14) and use the OpenAIModel class in Smo- lagents to load the model, with exactly the default parameters across all the experiments. Similarly, we utilize the LiteLLMModel class to load Gemini models, such as Gemini-2.5-pro (i.e., gemini-2.5-pro), and also follow the default parameters. As for the SimpleQA evaluation pipeline, which adopts an LLM-as-a-judge fashion, we utilize GPT-4o (i.e., gpt-4o-2024-08-06) to determine whether the agent’s answer is correct, incorrect, or not attempted. 11 PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agents B. Planning Errors All the LLMs we use to construct the planning errors are Gemini-3-pro-preview (i.e., gemini-3-pro-preview) due to its strong long-context understanding and processing ability. B.1. Construction We randomly select 50 hard-level data samples from the HotpotQA (Yang et al., 2018) training set and 50 samples from the MuSiQue (Trivedi et al., 2022) training set as our seed tasks. By repeatedly running the base agent (i.e., Smolagents with GPT-4.1) on these tasks to collect three trajectories per task, we obtain 300 trajectories in total. However, we only particularly retain tasks involving both successful and failure trajectories as mentioned in Section 3.2.1. This results in a final candidate subset of 32 tasks, including 18 from HotpotQA and 14 from MuSiQue. Example Diagnosis from HotpotQA Task: What character did this actor, who also played a supporting role in Sleepy Hollow, play in the final six Harry Potter films? Diagnosis: Error Type: inadequate cross verification of constraints Description: The agent identifies a candidate answer but fails to rigorously verify if it meets all the specific constraints defined in the task (e.g., ‘final six films’). Evidence: TRAJ 2, Step 19: ... “There is no evidence ... of any other Sleepy Hollow supporting actor being present in the last six Harry Potter films. The consistent answer is Richard Griffiths as Vernon Dursley.” Impact: In Trajectory 2, the agent identifies Richard Griffiths but fails to verify if he is in all of the ‘final six’ films. In contrast, Trajectory 1 and Trajectory 3 correctly identify Michael Gambon, who ..., fitting the constraint. Example Diagnosis from MuSiQue Task: When did the torch arrive in the region where Irreversi takes place? Diagnosis: Error Type: tool selection inflexibility Description: The agent continues to use a specific tool (e.g., ‘visit webpage’) on a specific domain (e.g., Wikipedia) despite repeated failures (403 Forbidden), instead of switching to alternative tools or alternative sources immediately. Evidence: TRAJ 2, Step 13: The agent attempts to use ‘visit webpage’ on the Wikipedia URL ... despite knowing from Step 4 that Wikipedia blocks this tool. Impact: This error prevents TRAJ 2 from accessing the definitive source of the schedule. TRAJ 1 encounters the same 403 error initially but later correctly switches to ‘inspect file as text’ (TRAJ 1, Step 22) to read the Wikipedia page and successfully retrieves the date. TRAJ 3 avoids Wikipedia entirely for the date confirmation and successfully uses a news site (CNN). B.2. Final Planning Errors We prompt an LLM to generate diagnoses by comparing and analyzing the trajectories. The 32 tasks eventually lead to 68 diagnoses in total, with various identified error types and detailed analyses. However, most of these errors have similar essences and therefore can be merged together. As mentioned in Section 3.2.1, we also perform a manual filter to drop the errors that are not related to long-term planning or too specific to tasks. Specifically, after LLM aggregation, the planning errors contain four error types: insufficient constraint verification, shallow content verification, ineffective tool selection, and ineffective search query. Among these error types, the “ineffective search query” is too specific to action-level reasoning instead of high-level planning. During the planning stage, the agent tends to propose a general guideline rather than every detailed action to take. Therefore, we remove the ineffective search query error from the planning errors and end up with three error types covering aspects of constraint, tool, and content understanding. Particularly, the planning error construction is an offline process which is separated from the major agent reflection and working process. We sample various trajectories from basic Smolagents with GPT-4.1 as the backbone to build up planning 12 PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agents errors and directly reuse them with PreFlect powered by either GPT-4.1 or Gemini-2.5-pro. The performance boosts across different backbone LLMs reflect the generalizability of the experiential prior, further validating the effectiveness of the grounded guidance in detecting potential risks in planning stage. For error identification, we use the error types, descriptions, and 2 failure examples as the experiential prior to help the reflector perform a more structured and precise diagnosis. As for plan revision, we provide the reflector with all the success and failure examples corresponding to the identified error so that the reflector accesses more high-quality references extracted from the agent’s own experience. Planning Error Details insufficient constraint verification The plan identifies a potential answer but fails to rigorously verify it against all specific constraints or conditions in the prompt (e.g., timeframes, specific subsets, exclusions). It relies on partial matches, ambiguous snippets, or initial findings without including steps to explicitly confirm that the candidate satisfies every requirement using authoritative sources before concluding. Failure Example 1: In a task identifying an actor who appeared in ‘Sleepy Hollow’ and the ‘final six’ Harry Potter films, the agent identified Richard Griffiths (Vernon Dursley) because he appeared in both franchises. However, the plan failed to verify his presence in *each* of the final six films (he is absent from some), leading to an incorrect answer based on a partial match. Failure Example 2: In a task identifying the 2004 sporting event participated in by the star of ‘Uyirodu Uyiraga’ (Ajith Kumar), the agent relied on a single ambiguous news snippet mentioning ‘Formula Asia BMW F3 Champi- onships’ and ignored a previous snippet mentioning ‘British Formula 3’. It failed to visit a dedicated database to resolve the conflict or verify the specific year, leading to an incorrect answer. shallow content verification The agent relies solely on search snippets or broad summaries without visiting the actual webpages, or extracts data from visited pages without verifying if the specific data definitions (e.g., geographic scope, units) match the task constraints. It fails to recognize that generic pages often serve as navigation hubs or that specific labels in the text might contradict its assumptions. Failure Example 1: The agent repeatedly searched for ‘2008 Olympic torch relay Hong Kong date’ but dismissed the search snippets as ‘general year summaries’ without visiting the pages to find the specific date within the text Failure Example 2: The agent relied too heavily on the web search text output or a blocked Wikipedia page, missing the opportunity to extract the answer from other accessible sources (like a Daily Mail article) visible in the search results. ineffective tool selection The agent persists in using a specific tool or approach despite known limitations or repeated failures (e.g., using a web scraper on blocked sites, or repeatedly searching for a direct answer when results are noisy), failing to switch to alternative tools or sources (e.g., visiting a homepage to navigate manually). Failure Example 1: The agent attempted to use visit webpage on a Wikipedia URL to find the torch relay schedule, despite receiving a 403 Forbidden error, and did not attempt to use inspect file as text which can handle such URLs. Failure Example 2: The agent repeatedly attempted to visit a Wikipedia page despite receiving 403 Forbidden errors, and even tried to inspect a non-existent local path derived from the URL, rather than pivoting to other accessible news articles found in search results. C. Agent Sources The results of both the HAL Generalist Agent and the HF-ODR are directly cited from HAL’s website 1, including GAIA scores and total costs. OAgent’s result is from the original paper (Zhu et al., 2025a) while OWL’s result is reproduced by Wang et al. (2025b). We also directly cite the results of AutoAgent and Magnetic-1 from Chen et al. (2023). 1https://hal.cs.princeton.edu/gaia 13 PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agents C.1. Details of the Agent Frameworks In this section, we expand on the implementation details of other open-source complex agent frameworks. By analyzing the engineering details, we further claim that our proposed method achieves higher performance even with limited computing resources, validating the effectiveness of the proposed reflection mechanism. • HAL Generalist Agent: It is a single-agent framework that is built upon Smolagents’ CodeAgent class with 200 max action steps and a fixed interval planning update of every 4 steps. The agent has access to the GoogleSearchTool, the default VisitWebpageTool and PythonInterpreterTool, TextInspectorTool, execute bash, edit file, file content search, and a visual inspector tool powered by GPT-4o. • HF-ODR: It is a multi-agent framework that consists of a manager agent and a text browser agent, which is specialized in browsing webpages. The browser agent is allowed to execute up to 20 action steps with a fixed interval of planning at every 4 steps. It is also equipped with comprehensive tools to browse webpages, including GoogleSearchTool, VisitTool, page scrolling (PageUpTool and PageDownTool), FinderTool (i.e., Ctrl+F), FindNextTool (i.e., finding the next match in a Ctrl+F search), ArchiveSearchTool (i.e., search the Wayback Machine), and TextInsepctorTool. The manager agent, equipped with a text inspector tool and a visual inspector tool, has 12 maximum action steps and a plan every 4 steps. • OAgents: It has a similar architecture to HAL Generalist Agent, where a single CodeAgent handles everything. Differently, it employs the optimal modules that are empirically validated in their experiments. The system supports up to 100 maximum action steps. Its core architecture features a multi-faceted planning module that supports subtask decomposition, staged plan calibration, and periodic plan reviews every 1 step. For factual grounding, it utilizes an ensembled toolkit comprising a multi-source search engine (Google, Wikipedia, Baidu, Bing, DuckDuckGo) alongside specialized multimodal inspectors for text, audio (Whisper-1), and visual analysis. The framework further incorporates a hierarchical memory system (summarization, vectorized retrieval, and long-term memory fusion) and a Test-Time Scaling (TTS) module that employs Best-of-N sampling and list-wise verification to enhance decision-making robustness. • Magentic-One: It features an orchestrator agent that implements an outer loop to manage the task ledger and an inner loop to manage the progress ledger. Except for the orchestrator, the system consists of a WebSurfer to manage web browser, a FileSurfer to read local files of most types, a Coder to write code, and a ComputerTerminal to provide access to a console shell where the Coder’s programs can be executed, and where new programming libraries can be installed. • AutoAgent: This framework utilizes a fully automated multi-agent architecture where a central LLM-powered actionable engine orchestrates specialized worker agents, including Web, Code, and File agents. It employs a Manager- Worker inspired by Magnetic-One, supporting dynamic task decomposition and sequential reasoning. The self-play agent customization module enables autonomous generation and refinement of tools, agent profiles, and workflows with natural language. The framework also integrates an agentic RAG system to ingest diverse file formats such as PDF and images. • OWL: OWL is a multi-agent system built upon Camel with a Role-Playing architecture. Two agents, respectively, play the roles of user and assistant to solve user input tasks. The assistant agent manages various tools and specialized agents, including a Browser system that is composed of a browser agent and a planning agent, multi-modality handling tools including image, video, and audio, CodeExecution tool, Excel tool, Search tool, and FileWrite tool. The user agent decomposes the task and prompts the assistant agent to finish a sub-task while the assistant agent assigns specialized agents to execute and provides execution results or failures to the user agent. 14 PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agents D. Prompts D.1. Prompt For Prospective Reflection System Prompt: Reflection & Revision You are a hyper-vigilant AI Quality Assurance agent designed to perform pre-execution strategic reflection on an agent’s plan to prevent task failure and wasted effort. Your analysis should be bi-directional: look backward to review the past actions and forward to prevent future errors. Your role in this multi-turn conversation: 1. First turn: Detect critical errors in the updated plan based on the provided error taxonomy 2. Second turn: Revise the plan to address the detected errors You must focus only on the critical errors that fundamentally lead to task failure and ignore minor errors that are hardly harmful or easily fixable with minor adjustments. ## Error Taxonomy {error taxonomy} ## Core Principles • Pragmatism over Pedantry: Only flag at most two most critical errors that will *actually* cause failure. • Maximize Task Success: Focus on flaws that have a real risk of causing failure or incorrect results. • Root Cause Focused: Identify the underlying cause, not just surface manifestations. • Trajectory Awareness: Use the history summary to diagnose recurring or systemic failures. • Reference: Use failure/success examples to understand error types and solutions. • Feasibility and Completeness: Ensure the plan is executable given available tools. • Constraint Awareness: Pay attention to past mistakes that reveal tool/environment limitations. User Prompt: Prospective Reflection (Turn 1) # Task {task} # Trajectory {previous context} # Available Tools {available tools} # Current Plan {plan content} # Instructions for Turn 1: Error Detection Analyze the above plan for critical errors that could lead to task failure. Your responsibilities: 1. Check the plan using the Error Taxonomy. 2. Enforce learning from history: Ensure the plan incorporates lessons learned from the trajectory. Be strict but pragmatic: flag only issues that materially threaten success. If there are multiple critical errors, flag the most critical one first. However, if the trajectory shows that the agent has already tried everything within capabilities, then you should be lenient with the ’flawed’ plan, such as resorting to secondary sources and broadening the scope of search which may contradict the constraints. # Output Format Your response MUST be a single JSON object. No other text or explanation is allowed. { ”analysis result”: ”PASS” — ”FAIL”, // Use ”FAIL” ONLY if at least one error from the taxonomy is present. ”detected errors”: [ 15 PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agents // If ”analysis result” is ”PASS”, this array MUST be empty. { ”error type”: ”specific error type from taxonomy”, ”description”: ”A concise, clear explanation...”, ”evidence”: ”A direct quote...”, ”how error affects task completion”: ”A specific explanation...” } ] } User Prompt: Plan Revision (Turn 2) # Turn 2: Plan Revision Based on the errors you detected, now revise the plan to address them. The detected errors are in the order of severity. Thus you should revise the plan to prevent the most critical one first. # Evidence {error evidences str} # Core Directives for Plan Revision: 1. Prioritize Feedback: Your absolute top priority is to meticulously analyze and address the most critical errors you detected. 2. Perform Root Cause Analysis: Do not just fix the symptoms. Before creating a new plan, you must first understand why the previous plan and past trajectory failed. 3. Balance Incremental and Substantial Rework: Take a balance between incremental rework and drastic strategy changes. No need to start from scratch if the original plan is already good. 4. Ensure Feasibility & Completeness: The new plan must be robust and executable. Note toolkit limitations: {available tools}. 5. Draw Experience from Previous Errors: Avoid making similar mistakes. For example, if a Wikipedia page was inaccessible, do not try the same domain again. Explicitly emphasize this in your revised plan. 6. Follow The Plan Structure: Your improved plan should follow the original structure: Facts Survey and Plan sections. # Instructions Generate a revised, improved plan that addresses critical errors. Your output should be the complete revised plan text only, without any JSON formatting or additional explanation. D.2. Prompt For Agentic Re-Planning In order to leverage the autonomy of the agent, we add an instruction and two examples of dynamic planning into the system prompt. System Guidelines: Dynamic Planning At each step, in the Thought: sequence, you should first take a holistic view of the history, thinking of whether the current path is optimal. If not, consider calling re plan() function to pivot to a new plan. Otherwise, explain your reasoning towards solving the task and the tools that you want to use. 16 PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agents CRITICAL STRATEGIC PRINCIPLE - RE-PLANNING: Since you plan forward, you need to follow your plan. However, the real world is unpredictable. Do NOT blindly follow the plan. Use re plan() to make a strategic pivot if the current plan is not working. HOW TO USE RE-PLANNING: If you want to request a re-planning step, you must call re plan() in the code sequence. replan result = re plan("Subtask ’fetch wikipedia page’ failed due to 403 Forbidden error... Therefore, I need to re-plan.") print(replan result) IMPORTANT: Never propose a new strategy inside re plan(). You MUST only state the difficulties encoun- tered! Examples: Strategic Re-planning Task: “Find the population of the capital city of the country where Mount Everest is located.” Thought: According to my plan, I should first find which country Mount Everest is in... print(web search(query="Mount Everest location country")) Observation: Mount Everest is located on the border between Nepal and Tibet (China). Thought: The observation breaks my original plan’s assumption... it is shared by two countries. The original plan is now ambiguous. I must re-plan to handle this multi-country scenario. replan result = re plan("Critical Ambiguity: Mount Everest is on the border of TWO countries, invalidating the single-country assumption.") print(replan result) Task: “Download the latest earnings report from Tesla’s investor relations page...” Thought: I will start by finding the Tesla investor relations page URL... print(web search(query="Tesla investor relations earnings reports")) Observation: Found Tesla investor relations page: https://ir.tesla.com/ Thought: Now I will attempt to visit the page... print(visit webpage(url="https://ir.tesla.com/")) Observation: Error: 403 Forbidden - Access denied. Thought: I have hit a hard technical blocker... Continuing with the original plan is futile. I need to pivot my strategy to obtain the data from a secondary source. replan result = re plan("Tool Failure: Cannot access primary source (Tesla IR page returns 403 Forbidden).") print(replan result) D.3. Prompt For Diagnosis Construction System Prompt: Diagnosis You are a professional error analyst. You are given three trajectories from an AI agent to solve a same task. Your goal is to compare the three trajectories in a **global perspective** and provide a detailed analysis of the **critical** errors that could have been solved in the **planning phase**. Your diagnosis will be used to build an error taxonomy which contains 1) common critical error types that significantly lead to task failures 2) contrastive success-failure cases that are caused by different error types. 17 PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agents You can ignore some minor errors and only focus on the critical errors that lead to the failure of the task. Besides, you should be tolerant of the early-stage errors since the agent is still exploring the task. If these early errors are fixed later, they should not be diagnosed as critical errors. ## Get To Know the Agent ### Agent Workflow The agent will first write a plan and update it at every 5 actions, and then execute the plan in action steps which contains ’Thought’, ’Action’, and ’Observations’. It takes the observations from the environment as input to decide the next action step. NOTE: The "Thought" in each ACTION step is not a plan step! ### Available Tools - web_search: Performs a duckduckgo web search based on input query; then returns the top search results. This tool is powered by ‘ddgs‘. It uses ‘DDGS().text‘ method to search the web. - visit_webpage: Visits a webpage at the given url and reads its content as a markdown string. NOTE: This tool is powered by ‘markdownify‘ and ‘requests‘, which means it can only handle text-based webpages. For json-heavy webpages, it always returns an error. For example, it returns 403 Forbidden Error for all wikipedia pages. - inspect_file_as_text: This tool is powered by gpt-4o to read a file as markdown text and ask questions about it. This tool handles the following file extensions: [’.html’, ’.htm’, ’.xlsx’, ’.pptx’, ’.wav’, ’.mp3’, ’.m4a’, ’.flac’, ’.pdf’, ’.docx’], and all other types of text files. - inspect_file_as_image: This tool is powered by gpt-4o to process image files and answer related questions. This tool supports the following image formats: [’.jpg’, ’.jpeg’, ’.png’, ’.gif’, ’.bmp’]. - inspect_file_as_audio: This tool uses OpenAI whisper-1 to get audio transcriptions and uses gpt-4o to answer related questions. This tool supports the following audio formats: [’.mp3’, ’.m4a’, ’.wav’]. **CRITICAL NOTE ABOUT INSPECTOR TOOLS**: The current environment does NOT support downloading files or accessing local file paths. All inspector tools (inspect_file_as_text, inspect_file_as_image, inspect_file_as_audio) MUST be called with internet-accessible URLs, NOT local file paths. Extremely cautious about using paths like ’/path/to/file.png’ or ’./local/file.pdf’ - these will fail unless you are completely sure the file is stored locally. Instead, provide URLs that are publicly accessible on the web (e.g., ’https://example.com/file.pdf’). ## Diagnosis ### Purpose of the Diagnosis The diagnosis will be aggregated into an error taxonomy for agents to reflect on their plans. For future tasks, the agent will self-check its plans and action steps so far, use this error taxonomy to identify critical errors in the plan, and revise the plan to guide the agent to the optimal path given task and environment constraints. Thus you should do everything to serve this purpose. ### Diagnosis Instructions 1. Compare the three trajectories and understand the different and similar details between them. 2. Identify the key differences between them that lead to the different outcomes. - The errors should be **GENERAL** enough to be applied to other tasks - Make sure that the error can be identified and fixed right after the flawed plan is generated 3. Given the current available tools, make sure that the success is achievable by fixing the critical errors instead of by luck. 4. Extract error patterns that are generalizable to other tasks. Make sure that for new tasks, these error types can be applied to diagnose the critical errors and 18 PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agents guarantee the success of new tasks. 5. Focus on three significant parts of the trajectory: the plan, the thought, and the tool usage. Your error types should correspond to them. For example, a ’logical_flawed_plan’ error can refer to a plan that has significant flaws in logics; an ’overlooked_observation’ error may infer that in the ’Thought’, the agent ignores some important observations; a ’tool_failure_ignorance’ error may happen when the agent fails to rectify the tool usage in an ’Action’. **EXAMPLE:** For three trajectories with the same task to find the rating of the first featured review on IMDb page for ’Iron Man’, TRAJ 1 successfully gets the page url ’https://www.imdb.com/title/tt0371746/’, and then uses the ‘inspect_file_as_text‘ tool to read the webpage content and get the rating. However, TRAJ 2 and TRAJ 3 get stuck in visiting the correct url with ‘visit_webpage‘ tool. A 403 Forbidden Error occurs repeatedly while the agent keeps using ‘visit_webpage‘. In this case, the error type can be ’tool_failure_ignorance’ since the agent fails to identify the 403 error with ‘visit_webpage‘ tool and does not plan to alternate to use ‘inspect_file_as_text‘. ### Diagnosis Format You should organize your analysis into a JSON format with the following fields: - **error_type**: the type of critical error made by the agent in those failure trajectories. It should be very **GENERAL** with no specific details or information about this task. For example, if the agent ignores the constraint in the task which limits the search range to be between 2020 and 2023, the error type can be ’constraint_ignorance’. If the agent fails to visit the webpage with ‘visit_webpage‘ tool and does not realize that ‘inspect_file_as_text‘ can also be used to read the webpage content, the error type can be ’tool_misuse’. - **error_description**: the description of the error. You should briefly describe the error in a generalizable way rather than based on the specific task. - **error_location&traj_content**: the **DETAILED** location and content of the error in the trajectory. You should provide the specific step number or mention the exact plan step. For example, if the error happens at an action step, the location is ’STEP n’ where n is the step number. And you need to cite the content of the trajectory at the error location such as the agent calls the tool ’visit_webpage’ with the url ’...’ but fails to visit the webpage. - **how_error_affects_traj**: how the error affects the entire trajectory in a global perspective. What happens after the error? Is it the pivotal turning point of the trajectory - the first half on the right track but the second half goes off the rails? Will fixing it lead to the success of the task? You MUST provide a high-level analysis of how the error affects the trajectory and **explicitly compare** the trajectory with and without the error such as ’TRAJ 1 ... In contrast, TRAJ 2 and TRAJ 3...’. Your output example: [ { "error_type": "...", "error_description": "...", "error_location&traj_content": "TRAJ N, Step M: ...", "how_error_affects_traj": "...", }, { "error_type": "...", //if there are multiple error types, you can format them as a list of valid JSON objects "error_description": "...", "error_location&traj_content": "TRAJ K, Step J: ...", "how_error_affects_traj": "...", } ... ] 19 PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agents If the error is not critical, not general enough, or not fixable by the available tools, you can respond with "no_critical_error". You should respond **only with the diagnosis** as a list of valid JSON objects without any code fences such as ‘‘‘json or ‘‘‘. User Prompt: Diagnosis The Task is: {task} Here are the three AI agent’s trajectories included in ..., ..., and ...: --- {traj1} --- {traj2} --- {traj3} **Now provide your diagnosis in the JSON format without any code fences such as ‘‘‘json or ‘‘‘:** D.4. Prompt For Error Aggregation System Prompt: Aggregation You are a helpful AI assistant to maintain an error taxonomy. You will get a diagnosis based on comparing three trajectories - some correct and some incorrect - generated by an AI agent for the same task. Your goal is to analyze the diagnosis and decide how to integrate it into the existing error taxonomy. ## Purpose of the Error Taxonomy For future tasks, the agent will self-check its plans and action steps so far, use this error taxonomy to identify critical errors in the plan, and revise the plan to guide the agent to the optimal path given task and environment constraints. Thus you should do everything to serve this purpose. ## Get To Know the Agent ### Available Tools - web_search: This tool is powered by ‘ddgs‘. It uses ‘DDGS().text‘ method to search the web. - visit_webpage: This tool is powered by ‘markdownify‘ and ‘requests‘, which means it can only handle text-based webpages. For json-heavy webpages, it always returns an error. For example, it returns 403 Forbidden Error for all wikipedia pages. - inspect_file_as_text: This tool is powered by a strong multi-modal LLM (such as gpt-4o) to read a file as markdown text and ask questions about it. This tool handles the following file extensions: [’.html’, ’.htm’, ’.xlsx’, ’.pptx’, ’.wav’, ’.mp3’, ’.m4a’, ’.flac’, ’.pdf’, ’.docx’], and all other types of text 20 PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agents files. - inspect_file_as_image: This tool is powered by a strong multi-modal LLM (such as gpt-4o) to process image files and answer related questions. This tool supports the following image formats: [’.jpg’, ’.jpeg’, ’.png’, ’.gif’, ’.bmp’]. - inspect_file_as_audio: This tool uses OpenAI whisper-1 to get audio transcriptions and uses a strong multi-modal LLM (such as gpt-4o) to answer related questions. This tool supports the following audio formats: [’.mp3’, ’.m4a’, ’.wav’]. **CRITICAL NOTE ABOUT INSPECTOR TOOLS**: The current environment does NOT support downloading files. All inspector tools (inspect_file_as_text, inspect_file_as_image, inspect_file_as_audio) MUST be called with internet-accessible URLs or existing local file paths. If the task requires accessing local file paths, you must make sure the files are indeed stored locally. Otherwise, provide URLs that are publicly accessible on the web (e.g., ’https://example.com/file.pdf’). Note that all the file paths provided by the task are locally available! Implementation details: (1) inspect_file_as_image downloads from the URL using requests.get(), saves to a temporary local file, then encodes it as base64 for GPT-4o vision API; (2) inspect_file_as_audio and inspect_file_as_text use MarkdownConverter/OpenAI APIs which expect to open files with open(), so they require the URL to be downloadable; (3) If a file is not publicly accessible via URL, these tools cannot be used. ## Instructions for Error Taxonomy Maintenance For each error in the diagnosis, you should: 1. **Understand the error:** - Whether it is truly a critical error in the planning phase instead of other phases such as execution or observation. - Whether and how it can be identified and fixed right after the flawed plan is generated - Whether and how it can be generalized to other tasks. 2. **Determine if it fits an existing error type:** - Review each existing error type carefully - Check if the error’s core planning problem matches an existing type - Consider whether the error pattern and the root cause align with an existing type 3. **Make a decision:** - **SKIP:** If the error is not critical, not general enough, not fixable by the available tools, not related to the planning phase, or not identifiable by just looking back on the past action steps as well as the current plan, you should skip it. - **MERGE:** If the error generally fits an existing type, output that error type with enrichments: * **Examples**: ALWAYS add the specific case(s) from this diagnosis as concrete example(s). - If the diagnosis contains COMPARISONS between successful and failed trajectories (e.g., "In TRAJ 1... succeeded" vs "In TRAJ 2... failed"), extract BOTH parts: • Add the successful trajectory description to success_examples • Add the failed trajectory description to failure_examples - If the diagnosis describes only a failure (no comparison), add it to failure_examples - If the diagnosis describes only a success (no comparison), add it to success_examples * Optionally refine the description to be more comprehensive - **NEW:** If the error represents a truly distinct planning problem not covered by any existing type, create a new error type with a **short, high-level, and generalizable** error name. 4. **Guidelines:** 21 PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agents - **Error type names and descriptions** should be **GENERALIZABLE** across different tasks - **Examples** should be specific and concrete, better mention the tool calls or other details: * When the diagnosis contains comparisons (e.g., "TRAJ 1 succeeded... In contrast, TRAJ 2 failed..."), split into two examples: one for success_examples describing the successful approach, and one for failure_examples describing the failed approach * When the diagnosis describes only failures or only successes, add to the appropriate list * ALWAYS add examples since each diagnosis is from a different task * Treat each trajectory as an individual example, briefly describe the task and how they succeed or fail - **Prefer skipping** if the error is not related to the planning phase or not easy to fix by revising the plan. - **Prefer merging** over creating new types when the core issue is similar - **Only create new types** when the planning failure mode is fundamentally different ## OUTPUT FORMAT Return a valid JSON object with this structure: { "updates": [ { "action": "merge" or "new" or "skip, "error_type": "error_type_name", "description": "Generalizable description", "success_examples": [ "Specific success example from this diagnosis (if applicable)" ], "failure_examples": [ "Specific failure example from this diagnosis (if applicable)" ] } ] } User Prompt: Aggregation ## Current Error Taxonomy {taxonomy_str} ## Single Diagnosis {diagnosis_str} --- **IMPORTANT:** - Always prioritize **quality over quantity**. Prioritize merging existing error types over creating new ones. - Return ONLY the JSON object, no other text. - Each item in "updates" has its own "action" field. - If **action is "skip"**, the error_type name MUST be empty. - If **action is "merge"**, the error_type name MUST match an existing type exactly. - If **action is "new"**, the error_type name MUST be different from all existing types. - **Examples**: ALWAYS include example(s) from this diagnosis. - If the diagnosis has "no_critical_error" or adds nothing new, return: {"updates": []} 22 PreFlect: From Retrospective to Prospective Reflection in Large Language Model Agents Generate the update now: 23