The agent core loop is the fundamental while-loop pattern underlying all LLM-based agents: it calls an LLM, checks whether the response contains tool calls, executes them if present, and stops when a text-only response is received. This pattern, formalized by the ReAct paper (Yao et al., 2022), interleaves reasoning and acting and yields a 34% improvement on ALFWorld benchmarks compared to chain-of-thought alone. All major agent frameworks—OpenAI Agents SDK, Claude Agent SDK, smolagents, Vercel AI SDK, and LangGraph—implement variations of this loop, differing in termination signals, streaming, parallelism, and checkpointing. In production, the loop itself is simple; the challenges lie in context engineering, tool design, safety controls, and managing multi-agent orchestration. A minimal 30-line implementation can achieve 76.8% on SWE-bench Verified, demonstrating that complexity belongs in tools and context management, not the loop.
Key Points
The core loop is a while(true) that cycles through LLM call → tool execution → LLM call until a text-only response signals completion.
Tool calls are the continuation signal (“I need more information”); a text response is the termination signal (“I have what I need”).
Frameworks vary: OpenAI Agents SDK uses a discriminated union for outcome classification; Claude Agent SDK runs in a separate CLI process with NDJSON communication; smolagents uses code-as-action with a final_answer() call; Vercel AI SDK makes the loop opt-in with composable stop conditions; LangGraph replaces the while loop with a directed cyclic graph with checkpointing and parallelism.
Multi-agent patterns (pipeline, manager, handoffs, fan-out) add cost (roughly 15× tokens vs a standard chat) but improve performance by up to 90.2% on internal evaluations.
Context engineering—write, select, compress, isolate—is more impactful than prompt engineering; tool responses account for 67.6% of tokens in practice.
Tool design favors fewer, higher-level, task-oriented tools with poka-yoke interfaces and explicit namespacing.
Production safety requires defense in depth: max iterations, wall-clock timeout, token/cost budgets, loop fingerprint detection, and proper error classification.
Testing spans three layers: deterministic logic tests, integration tests with mock tools, and end-to-end evaluations with LLM-as-judge.
Common failures include infinite loops, context window overflow, tool confusion, error compounding, framework lock-in, and missing idempotency.
Concepts
ReAct (Reasoning + Acting): The interleaving of reasoning traces and tool actions proposed by Yao et al. (2022). Achieved a 34% improvement on ALFWorld benchmarks over chain-of-thought alone. The core loop embodies this pattern.
Termination Signals: In the universal loop, tool calls indicate the model wants more information (continue loop), while a pure text response indicates it has all it needs (stop). Variations exist: smolagents uses final_answer() exception; Vercel AI SDK uses stop conditions and a “done tool” pattern.
Code-as-Action: Instead of JSON tool calls, the model generates executable code. smolagents’ research shows roughly 30% fewer steps compared to JSON tool calls.
Agent vs Workflow: Anthropic distinguishes an agent (model decides control flow in an open-ended loop) from a workflow (developer defines a predetermined sequence).
Superstep Execution: LangGraph’s execution model, borrowed from Google’s Pregel, runs all scheduled nodes in parallel per tick, merges state, and writes checkpoints.
Context Engineering: Four strategies—write context (save info externally), select context (pull relevant info at the right time), compress context (reduce tokens without losing info), isolate context (sub-agents with fresh windows). The KV-cache hit rate is the single most important metric for cost efficiency (cached tokens cost $0.30/million vs $3/million uncached).
Poka-yoke Tool Interfaces: Designing tools to prevent errors by design, e.g., requiring absolute file paths eliminated an entire class of model errors on SWE-bench.
Early Stopping Generate: When max iterations is reached, append a prompt asking for the best answer and call the LLM one more time without tools to synthesize a response.
Details
Framework Implementations
The universal agent core loop is a while loop that calls an LLM, checks for tool calls, executes them, and repeats until a text response. Anthropic’s “Building Effective Agents” distinguishes an agent (model decides control flow in an open-ended loop) from a workflow (developer defines a predetermined sequence). Each framework layers its own design decisions on this foundation.
OpenAI Agents SDK: The core loop calls runSingleTurn() in a while(true). Each turn’s outcome is classified into a discriminated union with four branches, which constitutes the entire decision tree. The default max_turns is 10 (one turn equals one LLM invocation; tool execution does not count). Agent-to-agent handoff is implemented as a specialized tool call (transfer_to_<agent_name>), reusing existing tool infrastructure. Guardrails run at three points: input (first turn only, parallel with first LLM call), output (after final response), and tool (before and after each tool execution); each returns a tripwire_triggered boolean.
Claude Agent SDK: The agent loop runs inside a bundled Claude Code CLI binary, separate from the application process. Communication uses stdin/stdout with NDJSON. Three streaming granularities are available: final results, progress updates, or live token streaming. The permission system has three layers: allowed_tools (auto-approve), disallowed_tools (block, overrides allow), and permission_mode (fallback). Permissions can be scoped to individual command patterns (e.g., Bash(npm:*)). Tool permission denials are returned as tool results, enabling the agent to self-heal by attempting an alternate approach. Context management compacts automatically when nearing the context limit and emits a SystemMessage(subtype="compact_boundary"). Instructions that must survive compaction go in CLAUDE.md files, re-injected every request. Sub-agents (via the Task tool) get fresh context windows and return condensed summaries (typically 1,000–2,000 tokens from 10,000+ tokens of internal work). Every ResultMessage includes total_cost_usd, token usage, num_turns, and session_id; runs are resumable.
smolagents (HuggingFace): Uses code-as-action instead of JSON tool calls, based on the thesis that code languages best express computer actions. The research paper (“Executable Code Actions Elicit Better LLM Agents”) shows roughly 30% fewer steps compared to JSON tool calls. The loop accumulates typed steps (SystemPromptStep, TaskStep, ActionStep, PlanningStep) into an AgentMemory. Termination occurs when generated code calls final_answer(), which raises a FinalAnswerException. If max_steps is reached without final_answer(), the agent synthesizes a response from history (graceful degradation). Built-in planning steps at configurable intervals. Analysis of 15,724 traces showed first-call parsing errors dropped success rates from 51.3% to 42.3%, leading to a structured CodeAgent variant using JSON schema with “thoughts” and “code” fields for 100% parsing reliability.
Vercel AI SDK: TypeScript-first, designed for web developers. Agent is an interface, not a class; third parties can implement it (e.g., Temporal’s DurableAgent for workflows surviving process restarts). The default stop condition is stepCountIs(1) — no looping unless explicitly opted in. Stop conditions are composable (e.g., stopWhen: [stepCountIs(20), yourCustomCondition()]). The prepareStep hook runs before each LLM call and can dynamically change model, tools, messages, or tool choice per iteration. A “done tool” pattern forces toolChoice: 'required' and defines a tool without an execute function; calling it halts the loop (structured output termination signal).
LangGraph: Replaces the while loop with a directed cyclic graph. Primitives: State (TypedDict or Pydantic model), Nodes (Python functions transforming state), and Edges (routing functions deciding what runs next). The “loop” is a cycle: llm_call node → conditional edge (should_continue) routing to either tool_node or END. Execution model borrows from Google’s Pregel: supersteps where all scheduled nodes run in parallel per tick, state is merged, and a checkpoint is written. Checkpointing occurs at every node transition; available savers include InMemorySaver, SqliteSaver, PostgresSaver, and community implementations for Redis and Couchbase. This enables parallel branch execution, fault tolerance, interrupt/resume, human-in-the-loop approval, and time travel (load a prior checkpoint, modify state, fork execution). Retrying a long agent run from scratch is expensive; checkpointing avoids that. The trade-off: overkill for simple loops, valuable for durable, resumable, parallelizable workflows.
CrewAI uses deterministic orchestration (Flows with @start() and @listen() decorators) and autonomous reasoning (Crews) with a ReAct loop inside CrewAgentExecutor._invoke_loop(). AutoGen models everything as inter-agent conversation, with the loop being message exchange. Its v0.4 adopts an actor model, and its Magentic-One variant uses a dual-loop ledger planning system. Neither introduced patterns not seen in the other four frameworks.
Production Considerations
Single-agent vs Multi-agent
Anthropic internal token scaling data: a standard chat interaction costs 1× tokens, a single-agent loop costs roughly 4×, and a multi-agent system costs approximately 15×. Multi-agent outperformed single-agent by 90.2% on internal evaluations, but the 15× cost is real. Four multi-agent patterns in practice:
Pipeline: Agents run sequentially, passing output. Predictable, no parallelism. Good for “research → draft → review.”
Manager: One orchestrator delegates to specialists and synthesizes outputs. Clean separation; manager can become a bottleneck.
Handoffs: Agents transfer control directly (e.g., OpenAI Agents SDK’s transfer_to_<agent_name>). Decentralized, flexible, harder to debug.
Fan-out: Multiple agents work in parallel on independent sub-tasks, results merged. Best throughput; requires genuinely decomposable tasks.
For most practical tasks, a single agent with good tools suffices. Multi-agent is justified when genuine specialization is needed—different system prompts, tool sets, or models.
Context Engineering
The new frontier is context engineering, not prompt engineering. What the model sees at each loop iteration matters more than initial instructions. Anthropic’s context engineering guide (link) frames four strategies:
Write context: Save information outside the context window (scratchpads, memory files, progress notes). The model persists information it’ll need later to a tool-accessible location.
Select context: Pull relevant information in at the right time via tools (grep, glob, RAG, database queries). The model needs to find things when needed, not carry everything.
Compress context: Reduce token count without losing critical info. Claude Code auto-compacts after 95% usage. Tool result clearing (replacing old tool outputs with summaries) is “the safest, lightest-touch form of compaction.”
Isolate context: Sub-agents with fresh context windows tackle sub-tasks and return condensed summaries. A sub-agent might use 10,000+ tokens internally but return a 1,000-token summary.
The Manus team’s findings (link): tool responses account for 67.6% of total tokens; system prompt is only 3.4%. “Tools comprise nearly 80% of what the agent actually sees.” Optimizing system prompt is practically irrelevant compared to optimizing tool responses. Their single most important metric is the KV-cache hit rate: cached tokens cost $0.30 per million vs $3 per million uncached—a 10× difference. They never dynamically add or remove tools mid-iteration because it invalidates the cache; they use logit masking instead to preserve the prefix cache. Agents maintain todo.md files to “recite objectives into the end of context” to combat the lost-in-the-middle problem. They keep failed actions visible in context so the model sees what didn’t work and avoids repeating it (“erasing failure removes evidence”). They rebuilt their framework four times and called their optimization process “Stochastic Graduate Descent.”
Tool Design
Anthropic’s tool design guidance (link): “Few thoughtful tools targeting specific high-impact workflows.” Litmus test: “If a human engineer can’t definitively say which tool should be used in a given situation, an AI agent can’t be expected to do better.” Overlapping tools confuse the model. Build higher-level, task-oriented tools (e.g., schedule_event instead of list_users, list_events, create_event; search_logs with filtering instead of read_logs).
Practical details:
Object dispatch, not if/else chains.Object.fromEntries(tools.map(t => [t.name, t])) for lookup. Clean, extensible, O(1).
Tools return error strings, not exceptions. The model needs to see errors as text in its context to self-correct. Exceptions break the loop.
Poka-yoke tool interfaces. Anthropic found that requiring absolute file paths eliminated an entire class of model errors on SWE-bench. They spent more time optimizing tool design than the overall prompt.
Namespace when integrating.asana_projects_search, jira_search—prefixed names prevent collisions and improve selection reliability. Fifteen well-defined, distinct tools can work; fewer than ten overlapping ones fail.
Building from Scratch
Reading frameworks is useful; building a minimal agent yourself is more useful.
The minimal viable agent: A working agent in about 30 lines using the OpenAI-compatible API shape (adapted from Victor Dibia’s walkthrough). The most common beginner mistake: the assistant’s message containing tool_calls must be appended to history before appending tool results. The API requires tool result messages to reference existing tool_call_ids. Swapping the order yields a cryptic validation error. The Anthropic shape differs: tool results go in a { role: 'user', content: toolResults } message, and the stop condition checks response.stop_reason === 'end_turn' instead of checking for empty tool calls, but the loop structure is identical.