Context engineering workflow: A practical 9-step framework for reliable AI agents

An AI agent can retrieve the correct document and still produce the wrong answer. A context engineering workflow addresses this gap by managing what information reaches the model, when it arrives, and how it is organized.

Larger context windows do not eliminate irrelevant retrieval, outdated requirements, or missing evidence. Developers, technical founders, and AI product managers need a repeatable method for delivering the right information, in the right format, at the right step. The nine-step framework below covers selection, preparation, execution, validation, and selective persistence. Its purpose is not simply fewer tokens: it is better task completion, inspectable evidence, and controlled operating costs.

What is a context engineering workflow?

A context engineering workflow is a repeatable process for selecting, preparing, delivering, validating, and retaining information supplied to an AI agent at each task step. It coordinates instructions, evidence, task state, tool definitions and relevant memory so the model receives sufficient, authorized context for the current task.

Dynamic context management means changing that package as the task progresses. A vendor-research agent needs security documentation during security evaluation, not necessarily during pricing comparison.

Anthropic’s Effective context engineering for AI agents recommends curating context and retrieving information as needed. This is implementation guidance, not proof that one approach works for every model.

Context engineering vs. prompt engineering, RAG and workflow engineering:

For example, correct vendor documentation cannot rescue an outdated requirement or unclear comparison format. Workflow control governs execution; context engineering governs the information supplied. Better context improves the conditions for success, but cannot guarantee correct outputs.

What belongs in an AI agent’s context?

Seven categories may contribute to active context:

1. Instructions and behavioral constraints.
2. Current user input.
3. Active task state.
4. Retrieved evidence.
5. Tool definitions and outputs.
6. Relevant memory.
7. Response requirements and schemas.

Not every call needs all seven. Available information is not active context: source systems remain external and authoritative.

Authorization determines eligibility before relevance determines priority. Other checks include freshness and provenance-meaning where information originated and how it was obtained.

For one account brief, a CRM (customer relationship management system) may supply approved company fields-not the complete account history.

A simple context lifecycle

These are practical categories, not mandatory standards.

Just-in-time context loading retrieves information when a step needs it rather than preloading everything.

The minimum-sufficient-context principle

Supply enough authorized information to satisfy the task’s acceptance criteria, without unnecessary material. Token minimization is not the sole objective. Test removal of questionable items against representative tasks, preserving decisive evidence and constraints. Do not use the model’s self-reported confidence as the sole removal criterion.

Smaller context is not automatically better. Token efficiency matters only alongside successful outcomes.

How to build a context engineering workflow in 9 steps

1. Define the task and success condition.
2. Select eligible context sources.
3. Retrieve context just in time.
4. Filter, rank, and deduplicate.
5. Compress or structure the selected information.
6. Assemble the final context package.
7. Execute the model, tool, or subagent.
8. Validate the result against evidence and rules.
9. Persist useful state and discard noise.

Step 1 - Define the task and success condition

Action: Translate the request into testable requirements before retrieval. Establish whether current information, private data, or tools are necessary.

Output: A structured task specification:

Outcome: Compare three B2B software vendors.
Scope: Approved security and integration requirements.
Sources: Current official documentation; authorized requirements.
Tools: Read-only document retrieval.
Format: Comparison table with evidence references.
Complete when: Every criterion has evidence or is marked unknown.

Common failure: Retrieving first, then defining “good enough” around whatever appears.

Step 2 - Select eligible context sources

Action: Approve sources and tools by user, tenant, and task. Public documentation supports capability claims; authorized internal requirements define evaluation constraints.

Model Context Protocol (MCP) exposes tools and resources through a common protocol. Official MCP documentation describes authorization mechanisms, but applications must still enforce tool and data permissions.

Output: An approved source and tool shortlist.

Common failure: Exposing every connected database and write-capable tool to the research agent.

Step 3 - Retrieve context just in time

Action: Retrieve targeted passages using document identifiers, record keys, paths, or saved queries. Apply metadata and date filters.

Load vendor security documentation when evaluating security requirements-not the vendor’s entire website.

Output: A candidate context set with source metadata.

Common failure: Preloading full repositories, histories, or tool catalogs. On-demand retrieval can reduce unnecessary material, but extra calls may increase latency.

Step 4 - Filter, rank and deduplicate

Action: Hard-exclude unauthorized or otherwise ineligible content. Rank remaining evidence by relevance, authority, freshness, and specificity. Remove repeated passages while preserving contradictory evidence for inspection.

Output: A prioritized evidence set with source provenance and conflict flags.

Common failure: Ranking solely by semantic similarity. Similar wording does not establish usefulness or truth; selecting only favorable evidence hides uncertainty rather than resolving it.

Step 5 - Compress or structure the selected information

Action: Use extraction for exact passages, summarization for broader material, and structured fields for comparison. Preserve decisive values, restrictions, sources, dates, and uncertainty.

Output: Compact evidence blocks.

Common failure: Compressing this to “SSO supported,” which changes the purchasing implication.

Step 6 - Assemble the final context package

Action: Organize trusted instructions separately from retrieved data.

Instructions and constraints
Current task and success criteria
Relevant state or memory
Evidence with source identifiers
Permitted tools for this step
Required output schema

These are logical components. Use supported message roles and tool APIs, not necessarily one concatenated prompt.

Output: A context package for one execution step.

Common failure: Treating instructions inside retrieved documents as application instructions. Labels and delimiters organize content; they are not security boundaries. Prompt caching is an optional, provider-dependent optimization.

Step 7 - Execute the model, tool or subagent

Action: Route work deliberately:

• Deterministic function: Calculations and known rules.
• Tool: External retrieval or actions.
• Model: Synthesis and interpretation.
• Subagent: Bounded delegation with clear benefit.

Output: A draft, tool result, or bounded artifact. Subagents return findings, evidence, and artifact references-not full internal transcripts.

Common failure: Sending every exploration through the main agent. Subagents add coordination costs; separate context is not a permission sandbox. Enforce least privilege and consequential-action approval.

Step 8 - Validate the result against evidence and rules

Action: Check completion, constraints, citations, evidence support, authorization, and output schema.

• Missing evidence → Retrieve again.
• Conflict → Inspect sources or clarify.
• Invalid schema → Repair and recheck.
• Unsupported conclusion → Revise or abstain.

Output: An accepted result or bounded correction request.

Common failure: Accepting fluent prose as verified output. Set retry limits and escalation rules. Schema validity establishes structure, not factual correctness; citations establish traceability, not necessarily support.

Step 9 - Persist useful state and discard noise

Action: Retain approved decisions, unfinished work, useful preferences, and final artifacts selectively.

Output: Updated state or a durable artifact.

Common failure: Treating the conversation as permanent memory. Define expiration, refresh, correction, and deletion rules. Updating a source-of-truth system requires separate authorization.

How the 9 steps map to the WSCI mModel

LangChain’s Context Engineering for Agents describes Write, Select, Compress, and Isolate (WSCI) strategies.

Add quality, security and observability controls

Controls belong throughout the workflow. Evaluate task outcomes alongside cost and latency; a cheaper run that omits decisive evidence is not an improvement.

Pre-Injection security and provenance gate

Before material enters active context, check:

1. User and tenant authorization.
2. Source identity and provenance.
3. Task-specific freshness.
4. Embedded instructions in untrusted material.
5. Unnecessary sensitive data.
6. Attempts to override application rules.
7. Traceability to original evidence.

OWASP’s guidance on prompt injection supports defense in depth: enforce permissions outside the model, restrict tools, and require human-in-the-loop approval for consequential actions.

Context quality scorecard

Use representative tasks with predefined evaluation rules.

Get the next post in your inbox

New guides and AgentKit updates, no spam. Unsubscribe at any time.

Minimum observability requirements

Observability means recording enough execution information to investigate behavior:

• Run ID, task, and configuration version.
• Selected/rejected source IDs and reasons.
• Token counts, timings, and compression operations.
• Tool activity and validation results.
• Memory changes and final status.

Prefer identifiers and redacted metadata. Protect retained excerpts with access controls and expiration policies. Do not log credentials, unnecessary personal data, or private reasoning transcripts.

Example architecture: AI research agent

When implementing this architecture, AgentKit serves as an open scaffolding and development framework for building robust AI Engineering Systems (supporting environments like Claude Code, Cursor, Codex, DeepSeek Harness, etc.).

AgentKit provides pre-packaged abstractions for:

• Reusable Skill Definitions: Standardizing instructions across steps.
• MCP Integrations: Dynamic tool binding without polluting global prompt space.
• Subagent Orchestration: Isolated context boundaries for delegated tasks.
• Task State Artifacts: Structured persistence for Step 9 state retention.

Common context engineering dailures and an MVP checklist

Diagnose the failure before adding infrastructure. Incorrect answers can originate in context, model limitations, tool behavior, or unclear requirements.

Context failure diagnosis table

These labels are practical descriptions, not a standardized taxonomy.

Minimum viable context engineering checklist

Copy this checklist into your implementation plan:

• Define testable success criteria and explicit unknowns.
• Separate trusted instructions from retrieved data.
• Approve sources, tools, and permissions.
• Retrieve information just in time.
• Check eligibility and task-specific freshness.
• Deduplicate while preserving conflicts.
• Compress without losing restrictions or exact values.
• Retain source identifiers and evidence provenance.
• Apply controls to untrusted content.
• Validate outputs; cap retries and define escalation.
• Persist selectively with refresh, correction, and deletion policies.
• Measure outcomes, token use, cost, and latency.
• Add subagent isolation only for justified delegation.
• Compare changes against a simple baseline.

Test source selection, compression, and memory policies independently where possible. Model-reported confidence is not a calibrated probability unless separately evaluated.

Frequently asked questions

What is a context engineering workflow?

A context engineering workflow is a systematic process for curating, organizing, and delivering the most relevant information to an AI agent at each step of a task. It optimizes information flow using strategies like selection, compression, and sub-agent isolation to ensure high task accuracy without overloading context windows.

Context engineering vs. prompt engineering: What is the difference?

Prompt engineering focuses on crafting system prompts and instructions to shape LLM behavior. Context engineering manages the entire information payload-including instructions, retrieved evidence, memory, dynamic tool definitions, and output schemas-ensuring the agent has the exact right context at every execution step.

What causes context rot in AI agents?

Context rot occurs when an LLM's reasoning degrades as token length grows. As agents accumulate uncurated transcripts, tool outputs, and irrelevant documents, the model struggles with recall and becomes distracted, leading to hallucinations or missed constraints.

What are the main components of a context engineering workflow?

A complete workflow typically includes:

1. Task definition and success criteria
2. Authorization and security filtering
3. Just-in-time retrieval of source data
4. Context ranking and deduplication
5. Compression and structuring of evidence
6. Selective state persistence
7. Validation against task rules

How does the Model Context Protocol (MCP) assist in context management?

MCP provides a standardized protocol for agents to connect with external tools and resources. Instead of hardcoding all tools into a single prompt, MCP enables dynamic loading of specific tool schemas only when required by the current step.

Read more:

• Project-aware AI coding: A complete guide and evaluation
• AI generated code review checklist: A pre-merge guide
• AI coding rework: How to measure and reduce hidden debt

Conclusion

A context engineering workflow manages information across selection, preparation, execution, validation, and selective persistence. Its value comes from supplying sufficient evidence under clear permissions-not maximizing context volume or minimizing tokens at any cost.

Reliable AI agents also need application-level controls, traceable sources, bounded corrections, and explicit unknowns. Larger windows cannot replace these safeguards, and smaller packages are useful only when they preserve task success.

Start with one representative vendor-comparison task. Compare the workflow against a simple baseline using completion rate, evidence quality, review effort, cost, and latency. Add complexity only when evaluation supports it.

Copy the checklist above into your next implementation plan; use the optional AgentKit mapping only where reusable components fit your requirements.