Claude Certified Developer - Foundations certification Overview

I recently completed the Claude Certified Developer - Foundations certification. This certification is based on the official course from Anthropic: https://anthropic-partners.skilljar.com/path/claude-certified-developer-foundations However, this course is only available to Anthropic partners. Below is my overview of the modules and structure of the Prep Course. Claude Certified Developer - Foundations Prep Course The course consists of the following main modules: MSO Foundations Learn the model fundamentals and technical foundations the rest of the Developer Foundations course builds on. 1.1 How LLMs Behave 1.2 Models & Reasoning 1.3 Prompting Modes 1.4 Technical Substrate Production-Grade Prompting, Agents & Tool Use Build your first production integration on Claude, with reliable prompts, tools, context management, and agent loops. 2.1 Prompting Craft 2.2 Extended Thinking 2.3 Tool-Use and Schema Design 2.4 Streaming Responses 2.5 Context Engineering 2.6 Agent Construction 2.7 Agent Memory 2.8 Cumulative Debug Task 2.9 Multimodal and Batch Ingestion Claude Code, MCP & Integration Learn to make a working Claude integration configurable, shareable, and safe to connect to real systems. 3.1 Module Orientation & Learning Objectives 3.2 The Explore → Plan → Code Loop 3.3 Permission Modes 3.4 Settings Configuration Hierarchy 3.5 Placing the Human Review Gate 3.6 Durable Project Context: CLAUDE.md 3.7 Rules Instruction Files 3.8 Hooks 3.9 Subagents 3.10 Packaging Workflows: Skills 3.11 Custom Commands & Plugin Packaging 3.12 Building an MCP Server 3.13 MCP Transport 3.14 Prompt Caching (Applied to MCP/Tool Definitions) 3.15 Retrieval-Augmented Generation (RAG) — Two Forms 3.16 MCP Configuration Scope 3.17 MCP Tool-Level Permissions 3.18 GitHub MCP Server — Worked Authentication Example 3.19 Enterprise Authentication Patterns 3.20 Regulated/Enterprise Deployment Requirements 3.21 Code Modernization as an Applied Case Study Production Engineering, Evals & Security Learn to take an agent that works in development and prove it holds up under real production traffic. 4.1 Module Introduction 4.2 Evals & Judges 4.3 Testing & Tracing 4.4 Failure Handling & Model Selection 4.5 Cost & Orchestration 4.6 Security 4.7 Cumulative Task Accelerators & IP Contribution Package a build that works into one that survives reuse, review, and deployment beyond the engagement that created it. 5.1 Module Introduction 5.2 Packaging for Reuse 5.3 Contributing Back 5.4 Requirements & Lifecycle 5.5 Systems Lifecycle for Claude Applications 5.6 Deployment & Versioning 5.7 Comparing Platforms 5.8 Trust Boundaries 5.9 Cumulative Task Module 1: MSO Foundations 1.1 How LLMs Behave 1.1.1 Tokens Everything Claude processes (prompt, history, tools, results, output) is measured in tokens, not words/characters, and both pricing and context budget are token-based. Output tokens cost more than input tokens. 1.1.2 Context Window The max tokens allowed in a single request — system prompt, user prompt, documents, history, tool results, and output combined. Exceeding it returns a model_context_window_exceeded stop reason. 1.1.3 Sampling & Temperature Claude samples the next token from a probability distribution rather than picking one fixed "best" token. Temperature (a request parameter) tunes this: lower = more repeatable, higher = more varied/creative. 1.1.4 Non-Determinism Sampling means identical inputs can yield different, equally valid outputs, so exact-string-match tests are unreliable. Test for properties instead — e.g., "required field present" or "value within range." 1.1.5 Testing Model Output: Structural vs. Semantic Correctness Check Type Definition Examples Structural correctness Deterministic, yes/no checks Regex match, valid JSON, exact substring, value within tolerance Semantic correctness Meaning-based checks, can't be scripted deterministically Summary captures key points, correct tone, accurate despite different phrasing Semantic checks need an LLM-as-judge: a separate model call scores the output, often against a rubric/reference answer. 1.1.6 Evals A testing framework for non-deterministic outputs: scores quality across many test cases and reports an aggregate pass rate (e.g., "87% passed"). Consists of test inputs, correctness criteria (structural or LLM-judge), and an aggregation method. 1.2 Models & Reasoning 1.2.1 Claude Model Family Four tiers trading off cost, latency, capability, and quality: Tier Description Sonnet Balanced default for most production workloads Haiku Optimized for speed/cost within its capability range Opus For demanding work beyond Sonnet's envelope Fable Highest-capability tier, for the hardest reasoning/coding/agentic tasks 1.2.2 Model Selection Strategy Start with Sonnet by default; move up a tier only when an eval shows it fails the quality bar, or down to Haiku only when an eval shows the quality drop is acceptable. Model choice should be eval-driven, not assumed. 1.2.3 Reasoning Modes Reasoning mode (on/off, separate from model choice) lets the model spend extra tokens "thinking" before answering; current models use adaptive thinking tuned via an effort setting (the older budget_tokens control is deprecated, now returns a 400 error). Thinking is hidden by default and worth the cost only on hard, multi-step problems — not simple lookups. 1.3 Prompting Modes 1.3.1 Zero-shot / One-shot / Multi-shot (Few-shot) Prompting Distinguished by how many worked examples are given in the prompt: Mode Examples Given Best Used When Zero-shot None (instruction only) Task is simple, output shape is obvious One-shot One input/output example A single reference clarifies expected output Multi-shot (Few-shot) Several examples Needs specific structure, casing, or edge-case handling 1.3.2 Cost Tradeoff of Examples Each example consumes tokens on every call and eats into context budget, so examples aren't free — they trade quality/precision against cost. 1.3.3 Interaction with Model Choice More capable models often succeed zero-shot where smaller ones need few-shot examples, so added examples can substitute for a cheaper model. Best practice: start with the simplest model and fewest examples meeting your eval bar, then add either only if needed. 1.4 Technical Substrate 1.4.1 SDK vs. Raw REST API Claude is accessed via an HTTP REST API (JSON over your API key); the SDK just wraps auth, request construction, retries, and parsing. Both hit the same API and models. 1.4.2 Response Delivery Patterns Pattern Description Synchronous Send request, wait for full response, then act — simplest pattern Streaming Delivered in pieces via server-sent events as generated; output appears immediately, client reassembles the final message Asynchronous (AsyncAnthropic) Non-blocking async/await enables concurrency without blocking, while each call still returns in real time Message Batches API High-volume/offline: submit a batch, poll for completion; up to 24h latency for lower per-token cost Module 2: Production-Grade Prompting, Agents & Tool Use 2.1 Prompting Craft 2.1.1 Diagnosing Prompt Failures Instead of Adding Words When a prompt fails, fix it by identifying which structural technique is missing — not by rewording or adding instructions. Rewording doesn't fix boundary confusion or format drift; only the matching technique does. 2.1.2 The Four Failure → Fix Mapping Symptom Missing Technique Why Wrong output shape (prose instead of JSON/label) Output constraint Nothing specified the response's form/stopping point Content/scope drift over turns System prompt (or a more specific one) Behavioral contract too vague to hold across turns Right task, invented structure Few-shot examples Claude can't infer exact structure from description alone Works on tested inputs, breaks on edge case Constraint covering that variant Prompt only validated against a narrow input set 2.1.3 System Prompts Carry the persistent behavioral contract for the whole session — role, output format, rules that must not change between turns. Write once as a stable instruction layer. 2.1.4 XML Tags Used to separate instructions from examples/data (e.g., , ) so Claude doesn't misread examples as part of the task itself. 2.1.5 Few-Shot Examples Show the exact input→output pattern (structure, casing, format) rather than describing it — closes gaps written instructions leave open, especially for edge cases. 2.1.6 Output Constraints Explicitly define the exact form of the response (e.g., "return only one label, no other text") — controls form independent of content, preventing parser-breaking variability. 2.1.7 When to Stack vs. Simplify vs. Diagnose Stack all four techniques for complex tasks with defined output contracts and edge cases; simplify for simple tasks (e.g., plain summarization). If a prompt has grown longer over 3+ re-prompts without improving, stop and diagnose the missing technique instead of adding more text. 2.1.8 Worked Postmortem: Six-Pass Classification Prompt A ticket-classifier prompt was revised 6 times, growing longer without fixing inconsistency — Pass 4 added description instead of a constraint, Pass 5's verbosity induced verbose output (latency regression, no accuracy gain). Pass 6 fixed it by replacing prose with an output constraint + 2 few-shot examples. 2.1.9 Structured Outputs: Moving Control from Prompt to API Instead of asking for a shape via prompt text, you give the API a JSON schema and the model is constrained at generation time (constrained decoding) — only schema-matching tokens can be produced. Mechanism What It Does JSON outputs Set output_config.format to type: json_schema + your schema — constrains the final response text Strict tool use Set strict: true on a tool definition — constrains/validates arguments before your code runs Tradeoffs: first request on a new schema is slower (grammar compiled, cached 24h); input tokens rise slightly (format-description system prompt injected); a guaranteed schema doesn't guarantee success — check stop_reason for refusal/max_tokens; incompatible with message prefilling. 2.2 Extended Thinking 2.2.1 What It Does When enabled, Claude reasons step-by-step in a separate thinking block before the final answer. On newest models this content is hidden by default — request a summarized display to see it. 2.2.2 Adaptive Thinking / Effort Setting Enabled via the thinking parameter (off by default); depth is tuned via an effort setting, not a fixed token budget. The older budget_tokens control is deprecated (400 error on newest models), and thinking tokens bill the same as output tokens. 2.2.3 When to Use Extended Thinking Task Decision Multi-step reasoning (math, multi-hop logic, dependent action planning) Enable, match effort to problem depth Mechanical tasks (classification, format conversion, lookups) Leave off — no benefit, wastes tokens Agentic loops planning across tool calls Enable, budget for the planning step 2.2.4 The Carry-Back Rule (Critical Constraint) In tool-use loops with thinking enabled, every thinking block must be sent back unchanged next turn — each has a signature, and editing/dropping it (even redacted blocks) breaks the signature and the API rejects the request. Manage context bloat via context engineering, never by stripping thinking blocks. 2.3 Tool-Use and Schema Design 2.3.1 Tool-Use and Schema Design Covered indirectly via structured outputs/strict tool use (see 2.1.9); explicit schema-design content beyond tool descriptions affecting routing appears under Agent Construction (see 2.6). 2.4 Streaming Responses 2.4.1 Why Streaming Sends the response in pieces as generated (via server-sent events) instead of waiting for the full message, removing blank-screen wait on long responses. Your code must reassemble blocks itself and handle early termination. 2.4.2 Event Sequence and Handler Actions Event Meaning Handler Action message_start New message beginning Initialize empty content array content_block_start New block opening (text/tool_use/thinking) Create slot at that index content_block_delta Incremental fragment of a block Append to block; tool_use JSON isn't parseable until block closes content_block_stop Block complete Finalize block (first point tool_use JSON is parseable) message_delta Top-level changes (stop_reason, usage) Record stop_reason message_stop Stream complete Assembled content is now the finished message 2.4.3 Never Act on a Partial Block tool_use inputs arrive as fragmented JSON strings, invalid until content_block_stop. Parsing/running tools before block closure causes malformed JSON or missing arguments. 2.4.4 Commit to History Only After message_stop Only add a streamed assistant turn to history once message_stop arrives and every block is fully assembled — a turn from an interrupted stream has an incomplete tool_use block and violates tool-use pairing rules on the next request. 2.4.5 Handling Interrupted Streams Treat accumulated content as provisional until message_stop; on interruption, discard the partial turn and retry. Check stop_reason from message_delta before continuing a loop — tool_use means calls are ready to run. 2.4.6 Postmortem: "Read Loop Ended" ≠ "Message Complete" A handler that commits history whenever its read loop exits (instead of gating on message_stop) can silently commit corrupted tool_use blocks — the validation error surfaces on the next request, not the one that caused it. 2.5 Context Engineering 2.5.1 Model Selection Recap Four-tier family (Fable, Opus, Sonnet, Haiku) trading cost/latency/capability — start with Sonnet, move tiers only on eval results. Confirm current lineup/identifiers against docs at build time. 2.5.2 Context Window Is a Shared, Finite Budget Covers system prompt, history, every persisting tool result, and output. An over-budget request is rejected pre-generation; hitting the ceiling mid-response returns partial output with model_context_window_exceeded — neither path auto-trims, so the app must manage it, and production sessions fill context faster than dev/test. 2.5.3 Four Strategies for Managing Context Budget Strategy What It Does When to Use What's Lost Pruning Rewind to an earlier message, drop everything after After an unproductive/dead-end path Everything after the rewind point Compaction (/compact; server-side beta or manual summarization) Summarizes history into a condensed version Approaching context ceiling, want to retain knowledge Any detail not captured in the summary Clearing (/clear; new API session) Starts fresh, empty context Next task is unrelated All session context (persist elsewhere, e.g. CLAUDE.md) Subagent handoffs Spawn isolated subagent with task-specific context; returns summary Self-contained delegable subtasks Visibility into subagent's intermediate reasoning 2.5.4 Prompt Caching & Token Counting Prompt caching reuses processing of a stable prefix at a fraction of cost via cache_control (type: ephemeral, up to 4 breakpoints) — the highest-leverage cost reduction for multi-turn sessions. count_tokens returns a request's token count without running inference, for dev validation and production budget gating. 2.5.5 RAG: Three Failure Points RAG lets an LLM retrieve external documents via chunking → embedding match → assembly → answer. Chunking must balance size (too small loses context, too large dilutes matches); embedding match can miss exact-term matches (pair with lexical search); sloppy assembly causes the model to ignore retrieved content entirely. 2.5.6 Indexed vs. Iterative RAG Indexed (fetch-once) is inspectable/testable but needs an index to build/maintain/secure — good for a stable corpus. Iterative (search-across-rounds) avoids staleness but costs more tokens/time and is less inspectable — better for changing corpora or multi-step questions. 2.5.7 Compaction: Summarizer Prompt Design A vague summarizer prompt ("summarize so far") loses task-critical detail; a well-specified one (preserve file paths, decisions, errors/resolutions) retains what matters. Under-specified summarizers are a leading cause of multi-session agent failures. 2.5.8 Subagent Handoffs for Long-Horizon Tasks Instead of growing the context window, decompose the task and give each subagent only a scoped task, minimal context, relevant prior results, needed tools, and clear exit conditions. Keeps per-turn cost low at the expense of implementation overhead. 2.5.9 Postmortem: Context Budget Not Tested Against Production Data A receipt-processing agent's 40k-token self-imposed cap worked fine against 800-token/call dev fixtures but was exhausted by turn 8 against real 3,200-token production tool outputs, crowding out system instructions. Fix: prune tool outputs after use and compact proactively — always measure actual production-scale token costs before shipping. 2.6 Agent Construction 2.6.1 Definition An agent is a multi-step tool-use loop with managed context and a defined goal. Key upfront question: does the problem actually require an agent, given the added coordination overhead, context cost, and failure surface? 2.6.2 Workflow vs. Agent Decision Choose a Workflow When... Choose an Agent When... Steps can be enumerated in code Path can't be enumerated in advance Error cost is high, step-level guardrails needed Non-determinism acceptable, actions constrained by toolset Standard observability tooling required Inputs vary unpredictably Inputs are well-constrained Task requires creative tool sequencing 2.6.3 Three Wiring Paths Path Who Runs the Loop You Own Best For Raw API loop Your code Everything: loop, execution, context mgmt, retries, exit conditions Full control / learning / library constraints Agent SDK SDK, in your process Tool execution + app; SDK gives loop structure, context mgmt, tool registration Claude Code's scaffolding without rebuilding it Claude Managed Agents (public beta) Anthropic (server-side, via SSE) App layer + agent definition as a versioned resource Long-running tasks (mins–hours), avoid building sandbox/loop 2.6.4 Managed Agents Specifics Anthropic runs the loop/sandbox/retries server-side with stateful stored sessions; not currently eligible for Zero Data Retention or HIPAA BAA (rules out PHI/ZDR workloads). Requires the managed-agents-2026-04-01 beta header; common path is prototyping on Agent SDK then re-expressing config as a Managed Agents resource for production. 2.6.5 Four Steps Common to Every Agent Loop (1) Register tools with consistent schema, (2) set a system prompt scoped to the agent's task/toolset, (3) handle the tool-use loop — every tool_use block must get a tool_result and all blocks from one turn resolved together, (4) define explicit exit conditions rather than relying on Claude to volunteer completion. 2.6.6 Human-in-the-Loop (HITL) Insertion Points Insertion Point Trigger Risk Addressed Before destructive tool call Write/delete/send operation High — irreversible actions After a planning step Plan generated, about to execute Medium — wrong plan even if execution is correct On unexpected output Error flag, empty result, out-of-bounds value Variable — catches failures retries won't fix 2.6.7 Tool Orchestration: Over-Tooling vs. Under-Tooling Too many overlapping tools causes erratic routing (the more common production problem, from registering tools "just in case"); too few causes hallucinated paths. Start with the minimum tool set and add only on confirmed capability gaps. 2.6.8 Regulated Data Constraints Determine Delivery Route Constraint Rules Out Survives Review Attorney-client privilege Unaudited consumer Claude.ai calls Direct API/SDK via firm's SSO-authenticated app, routed through firm-approved LLM gateway with full logging HIPAA (PHI) Any endpoint without a covering BAA BAA-covered direct API, or Bedrock/Vertex on a HIPAA-eligible account (BAA excludes Console, Workbench, betas, consumer plans) GDPR / data residency Routes without pinned execution region; direct API (no EU residency) Bedrock/Vertex with region pinned to the covered jurisdiction FedRAMP / government Any non-authorized endpoint, incl. dev/test on commercial endpoint Claude for Government (C4G), Bedrock GovCloud, Vertex Assured Workloads (Claude Enterprise on AWS Marketplace is NOT FedRAMP authorized) Internal data-residency policy Any vendor outside the approved list Delivery route on the approved vendor only (SOC 2 governs system operation, not endpoint selection — covered in Module 4.) 2.7 Agent Memory 2.7.1 Memory Scope Scope Persists Cost Use When Lost In-context Within a single session Zero retrieval overhead, token cost grows with conversation Short sessions fitting fully in context Everything at session end External storage Across sessions/users/instances, in a DB Retrieval latency + read/write engineering Cross-session continuity needed Nothing (cost is latency/complexity) Summarized memory Condensed version injected next session Lower cost than full replay, drops detail Long-running conversations exceeding budget Anything summarizer didn't preserve Stateless (none) Nothing Zero overhead Self-contained, one-off jobs All prior context 2.7.2 Design-Time Decision, Not Refactor-Time Choosing memory scope during design avoids costly production refactors; a common failure pattern defaults to full in-context history, which works until token cost/latency climbs and hits the context ceiling. 2.7.3 Skills: Reusable, On-Demand Instruction Sets A Skill is a reusable SKILL.md (frontmatter: name, description + instructions body) that Claude loads only when the description matches an incoming request — unlike in-context memory, instructions aren't resident every session. 2.7.4 Skills vs. CLAUDE.md vs. In-Context Instructions Pattern Loads Context Cost Best For Skill On-demand, when description matches Low (only name+description loaded upfront) Task-specific expertise not needed every session CLAUDE.md Every session unconditionally (CLI); controlled by settingSources in Agent SDK Fixed overhead per session Always-on project-wide standards In-context instructions Every turn in that session Grows with session length, doesn't survive session end Short, one-off sessions 2.7.5 Skills on the Messages API (Beta) Requires two beta headers (code-execution-2025-08-25, skills-2025-10-02); Skills run inside the code execution container, not the app's own environment, affecting available tools/filesystem access. 2.7.6 Subagents and Skills Subagents don't automatically inherit Skills or conversation history from the parent (clean context on delegation), but do inherit the parent's permission context. A subagent needing a Skill must have it explicitly listed in its own configuration. 2.7.7 Postmortem: In-Context Memory Filling by Session Four An escalation-support agent worked fine in dev (long continuous sessions) but failed in production, where many shorter sessions accumulated state until injected history exceeded 40k tokens by session 4 — before any tool call. Fix: refactor to external storage, though this took longer under production pressure than doing it at design time. 2.8 Cumulative Debug Task 2.8.1 Cumulative Debug Task Applied exercise combining prior concepts, not new theory. Observed bug-diagnosis mapping: vague tool descriptions → schema/routing failure; interrupted streams/stripped thinking blocks → carry-back rule violation; orphaned tool_result without matching tool_use → context/pairing violation; concatenated full session transcripts → memory scope failure at scale. 2.9 Multimodal and Batch Ingestion 2.9.1 Image Token Cost Images process in 28×28-pixel patches: cost = ⌈width/28⌉ × ⌈height/28⌉ visual tokens (e.g., 1000×1000px ≈ 1,296 tokens). Each tier has a max native resolution (oversized images downscaled first) — confirm current limits against docs, and measure production image token cost before building a pipeline. 2.9.2 Three Ways to Send an Image/File Method Mechanism Overhead Best For Inline base64 Encode bytes directly in the message Full payload sent every request One-off images unlikely to be reused URL reference Pass a public URL; Claude fetches it No payload, but URL must stay stable/public/reachable Already-hosted, stable public images Files API (beta; not on Bedrock/Vertex) Upload once → get file_id → reference later One-time upload cost, near-zero overhead thereafter Reused assets, large files, multi-turn conversations 2.9.3 Sending PDFs Uses a document block type (vs. image) with the same source structure (base64/URL/file_id); no required name field, optional title/context fields, same token-cost and Files API reuse mechanics. 2.9.4 Prompting Technique Carryover to Multimodal The same four prompting techniques apply to image/PDF analysis, but images introduce visual ambiguity (overlapping objects, depth, occlusion) that prompts should explicitly instruct how to handle. 2.9.5 Message Batches API For high-volume async processing — up to 100,000 requests or 256MB per batch. Submit → get batch_id → poll → download results (arbitrary order, matched via custom_id). Lower per-token cost, latency up to 24h — suited to offline pipelines/evals/bulk jobs, not real-time interactions. 2.9.6 Postmortem: "Chunked Loop" Mistaken for Batching Looping over a list calling the synchronous API per item (even in smaller chunks) is not batching — it's serialized calls still hitting the same rate limits at volume. True batching requires the actual Batch API. 2.9.7 Use-Case Fit Scenario API Why User uploads photo, expects immediate result Synchronous Real-time required Nightly job classifying 5,000 records Batches API No latency constraint; cost savings matter Eval run against 2,000 examples Batches API Offline, no real-time need Chatbot reply generation Synchronous User is actively waiting 2.9.8 Two Failure Modes Combining Multimodal + Batch (1) Misreading latency needs — using batch for a user-facing image flow fails because the user is waiting; (2) underestimating context cost — multiple large images/PDFs per request can blow past token limits at scale. Module 3: Claude Code, MCP & Integration 3.1 Module Orientation & Learning Objectives 3.1.1 Module Orientation Claude Code is a terminal-native dev partner running the same agent loop as the API, plus a permission layer, config system, and team-sharing features; MCP enables secure external-service integration. Core theme: configs that "work on your machine" often fail once shared, deployed, or run against production. 3.2 The Explore → Plan → Code Loop 3.2.1 The Three-Phase Loop Claude Code works in three phases: Explore (reads files, traces logic, no edits), Plan (proposes a structured edit description, requires human approval), Code (writes/executes changes only after approval) — this sequence reduces wrong assumptions. 3.2.2 Plan Mode as the Hook Point Plan mode holds the agent in the explore phase, blocking all edits/commands until released — a good default for unfamiliar or high-stakes codebases. 3.3 Permission Modes 3.3.1 Permission Modes Mode Auto-Approves Still Gated Limitations Default Reads only All edits/commands Safe but slow; baseline for new/unfamiliar projects AcceptEdits Reads, file edits, common filesystem commands (mkdir, touch, rm, rmdir, mv, cp, sed) in working dir All other shell commands; writes outside working dir; protected paths Good for trusted local work; not for running scripts Plan Reads only All edits/commands until plan approved Exploration on sensitive/unfamiliar code; not for tasks needing output Auto Everything, but a classifier reviews each action and blocks escalation/hostile intent Production deploys, migrations, mass deletes, credential exfiltration, force-push to main Research preview — not a safety guarantee; varies by plan/model DontAsk Only pre-approved allow-listed tools + read-only commands Everything else auto-denied Built for locked-down CI/scripts, not for reducing local friction BypassPermissions All tool calls, no prompts, no safety checks Nothing (except catastrophic rm -rf / or rm -rf ~) Only safe in isolated/disposable containers — never on a live dev workstation 3.4 Settings Configuration Hierarchy 3.4.1 Configuration Hierarchy Level Location Scope Use For User ~/.claude/settings.json Every project on the machine, not committed Personal defaults (e.g., preferred mode) Project .claude/settings.json (committed) Everyone who clones the repo Team-wide conventions, allow/deny rules Local project .claude/settings.local.json (git-ignored) Personal overrides for one project Individual preferences not meant for the team Enterprise managed-settings.json (admin-set) Cannot be overridden by users/projects Org-wide security controls 3.4.2 Rule Precedence A deny rule always wins over an allow rule regardless of mode; enterprise-level deny rules are the most durable — they can't be removed by any developer and apply even under bypass mode. 3.5 Placing the Human Review Gate 3.5.1 The Governing Question & Gate Placement Ask "what's the worst outcome if this action runs unchecked?" — let low-stakes reversible actions through, gate hard-to-undo/sensitive-path actions via deny rules or prompts, and always require human review before merging changes to team-flagged sensitive code. 3.5.2 Postmortem: BypassPermissions Removed a Safety Prompt Bypass mode skipped the confirmation prompt and protected-path guard that would have caught an overly broad file-pattern match, causing accidental deletion of production config files. Mitigation: set explicit deny rules on sensitive directories first; prefer classifier-gated modes (e.g., Auto) over full bypass. 3.6 Durable Project Context: CLAUDE.md 3.6.1 CLAUDE.md Basics & /init CLAUDE.md at the project root loads into every session, prepended before any user message, so conventions/constraints/commands persist without restating; /init generates a starter file by scanning the codebase (validate before relying on it). 3.6.2 Size Dilution Failure Mode As the file grows, each instruction becomes a smaller fraction of loaded context, reducing the chance any single rule is followed — keep CLAUDE.md to behavior-changing constraints and move everything else into on-demand Skills. 3.6.3 Postmortem: 847-Line CLAUDE.md A real path restriction (line 347) was diluted among hundreds of unrelated lines (historical logs, archived notes) and the agent violated it. Fix: keep it as a working rule set, move path-specific rules to rules files, historical context to reference docs, non-negotiable constraints to hooks. 3.7 Rules Instruction Files 3.7.1 Rules Files: Path-Scoped Context Live in .claude/rules/, scoped via a paths glob in YAML frontmatter, loading into context only when Claude works with matching files — avoiding CLAUDE.md-style dilution. 3.7.2 Scoping Comes from Frontmatter, Not Location A rules file without a paths field loads unconditionally at launch (same priority as CLAUDE.md) regardless of subdirectory; pattern is broad/universal → CLAUDE.md, narrow/path-specific → rules files (e.g., .claude/rules/database.md with paths: ["src/db/**/*.sql"]). 3.8 Hooks 3.8.1 Hooks: Deterministic Lifecycle Control Hooks intercept/control tool calls at fixed lifecycle points deterministically, unlike a CLAUDE.md instruction the model might follow inconsistently — defined in settings files, configured via /hooks. 3.8.2 Hook Events Event Timing Can Block? Use For PreToolUse Before tool call executes Yes — exit code 2 blocks it, stderr shown to agent Access control enforcement PostToolUse After tool call completes No Formatting, tests, audit logging UserPromptSubmit On prompt submission, before processing — Inject context, validate request Stop When model finishes responding — Notifications, cleanup, audit commits Notification On Claude Code notifications (permission requests, 60s idle) — Route to external channel/logging SessionStart Session start/resume — Initialize state, validate env vars SessionEnd Session end — Teardown, final audit writes 3.8.3 Hook vs. Convention A PreToolUse hook enforces a constraint (e.g., blocking edits to a production config path) at every tool call, every session, regardless of permission mode — the difference between a guardrail (hook) and a convention (CLAUDE.md instruction). 3.9 Subagents 3.9.1 Subagents: Isolated Context Specialized assistants that run tasks in an isolated context — no inheritance of main conversation history, accumulated files, or session state; return only their output. 3.9.2 Built-in vs. Custom Subagent Behavior Built-in Explore/Plan subagents skip CLAUDE.md and git status (optimized for speed, so project rules don't apply); general-purpose loads both. Custom subagents don't auto-inherit skills — list needed skills explicitly in the subagent's frontmatter (.claude/agents). 3.9.3 Mechanism Comparison Mechanism Loads When Context Cost Belongs CLAUDE.md Full file, prepended Every session Persistent, dilutes with size Universal constraints/commands Rules file File contents On matching file access (or session start if unscoped) Path-scoped: only when triggered; unscoped: same as CLAUDE.md Path-specific guidance Hook Runs a script At configured lifecycle event Minimal Guardrails, automation, audit Subagent Task context only When dispatched Returns summary only Exploration/investigation, parallelizable work 3.10 Packaging Workflows: Skills 3.10.1 Skills: Portable Markdown Procedures A Skill is a portable SKILL.md in .claude/skills (frontmatter identifies/describes it, body holds steps); the same file can run in Claude Code, via Messages API, or via Agent SDK — only where it runs and what it can access changes. 3.10.2 Skill Runtimes Runtime How It Loads Where Steps Run Key Requirement Claude Code Filesystem discovery (description match or invoke by name) Local terminal, under active permission mode/deny rules Filesystem-based, governed by settings layer Messages API Sent with request, runs in code execution container Anthropic's container, not local machine Requires code-execution + skills beta headers; no local file/tool assumptions Agent SDK Loaded by agent; controlled by settingSources (TS) / setting_sources (Python) The SDK's own process, once filesystem sources are enabled Must explicitly set settingSources — no reliable default Claude Managed Agents Defined once as an API resource (model, prompt, tools, MCP, skills) Anthropic-provisioned sandbox Requires managed-agents-2026-04-01 beta header; not ZDR/HIPAA-eligible; skills attached at agent-definition time 3.10.3 Three Portability Rules (1) Write the skill's description as a precise matching criterion; (2) don't assume local filesystem/tools exist — document dependencies explicitly; (3) subagents don't inherit skills in any runtime — must be explicitly listed. 3.11 Custom Commands & Plugin Packaging 3.11.1 Custom Commands (Legacy vs. Skills) Skills are now the recommended format for both explicit (/skill-name) and automatic invocation; the older .claude/commands/ format still works but is legacy. Use disable-model-invocation: true in frontmatter for commands that should only run when explicitly called. 3.11.2 Plugin Namespacing A plugin name becomes the command prefix (e.g., /payments:run-tests), preventing collisions across plugins; renaming a plugin renames all its commands. 3.11.3 Plugins & Marketplaces A plugin is a versioned bundle of skills, hooks, subagents, and MCP servers distributed via a marketplace. Anthropic's official marketplace is available by default; third-party ones are added via /plugin marketplace add . Enterprise admins can deploy plugins org-wide via managed settings, gated by a managed marketplace allowlist, paired with extraKnownMarketplaces to auto-register for all users; managed-scope settings sit above user/project settings and cannot be overridden. 3.11.4 Packaging Decision Table Layer Reach For When Skill Task-specific procedure should stay out of context until needed Custom command Predictable, explicit high-frequency entry point wanted Plugin A working local setup needs to be shared/versioned across a team 3.11.5 Postmortem: Plugin Portability Failure A plugin skill referenced an author's absolute local path and an undocumented env var — installed fine everywhere, but execution failed for every teammate. Fix: use $CLAUDE_PROJECT_DIR / ${CLAUDE_PLUGIN_ROOT} for paths, bundle dependent assets, document/validate env vars at install, and test on a clean machine; note a locally-relied-on deny rule/hook isn't auto-included in a plugin bundle. 3.12 Building an MCP Server 3.12.1 What MCP Is MCP (Model Context Protocol) separates tool definitions from individual applications into a standalone server process exposing tools, resources, and prompts to any connecting client — build once, reuse everywhere. 3.12.2 Tools, Resources, Prompts Tools are actions the model can call; Resources are read-only data fetched directly into context (not via a tool call) — direct (fixed address) or templated (parameterized), used when cheap predictable injection beats a tool call (client support varies); Prompts are pre-written, vetted instruction templates exposed by name for consistent wording across clients. 3.13 MCP Transport 3.13.1 Transport Options Transport Mechanism Use For stdio Local subprocess via standard input/output Local tools, personal scripts, dev servers on your own machine — not shareable/remote HTTP Standard network connection via URL Recommended for any non-local server — shared team servers, hosted integrations SSE Legacy transport predating HTTP No longer recommended for new servers; treat as legacy if encountered 3.13.2 Context Cost & Tool Discovery Connected servers' tool definitions would occupy context if loaded upfront; Claude Code defers loading and searches to discover/load only relevant tools per task by default (an opt-in mode loads upfront if definitions fit within ~10% of the context window). Connect only needed servers to keep requests lean. 3.14 Prompt Caching (Applied to MCP/Tool Definitions) 3.14.1 How Prompt Caching Works Caches processing of a stable request prefix so follow-up requests reuse it at a fraction of cost; the first request writes to cache, and an exact match is required — a single changed character before the cache point invalidates it. Best candidates: long system prompts, large tool-definition sets, frequently-queried reference docs. 3.14.2 Cache Configuration Details Enabled via cache_control: {type: ephemeral} on the last block to cache (up to 4 breakpoints); request order is fixed (tools → system prompt → messages), so a breakpoint after tools caches definitions while messages stay dynamic. Default lifetime is 5 minutes from last read; opt-in 1-hour via ttl: "1h"; a minimum token threshold applies (1,024 for most current models). 3.15 Retrieval-Augmented Generation (RAG) — Two Forms 3.15.1 Classical RAG vs. Agentic Search Classical RAG pre-chunks and embeds source material into a database, matched by similarity at query time (index built in advance); agentic search has no pre-built index — the model searches/fetches live sources on demand (e.g., MCP tool discovery, Projects surfacing relevant sections). Both retrieve a relevant slice and generate from it — the difference is timing. 3.15.2 Two Properties of Retrieval (1) Scales flat — request cost stays roughly constant as source material grows, since only a relevant slice is retrieved per query; (2) quality depends on retrievability — if retrieval misses the needed document the model never sees it, so well-named, well-organized files matter. 3.16 MCP Configuration Scope 3.16.1 Scope Levels Scope Location Applies To Use For Local ~/.claude.json (per-project entry) Current project only, not shared Project-specific, not-yet-shared config User Personal Claude settings All your projects, still personal Personal utilities used everywhere Project .mcp.json (committed to repo) Everyone who clones the repo Team-wide shared server access Enterprise Centrally managed config (admin-controlled) Entire organization Shared internal services, security tooling 3.16.2 Note on Project-Scoped stdio Servers A project-scoped stdio server still runs from each teammate's own machine — each clone needs the runtime (e.g., Node for an npx-launched server) installed locally. 3.17 MCP Tool-Level Permissions 3.17.1 Per-Tool Permission Rules MCP tools are identified as mcp__server__tool, so rules can target individual tools rather than the whole server — an allow rule on one tool lets it run without prompting while others on the same server still prompt; a deny rule on a write-capable tool blocks it while read-only tools remain available, and deny always overrides allow. 3.17.2 API MCP Connector: Scope vs. Governance An mcp_toolset object with a per-tool enabled flag controls whether the model even sees a tool (context/scope control), distinct from a permission rule controlling whether an exposed tool may run (governance control) — often used together; verify exact syntax/beta headers against docs. 3.18 GitHub MCP Server — Worked Authentication Example 3.18.1 GitHub MCP Setup Transport: HTTP (remote, hosted by GitHub), registered via server URL; scope: Project for whole-team access, local for individual use; auth: a Personal Access Token passed as a Bearer token, supplied via environment variable and referenced in config — never committed inline (a committed token enters repo history permanently). 3.18.2 OAuth Alternative For services authenticating individual users via browser sign-in (e.g., Linear MCP) — client redirects to the provider's sign-in page, token issued/stored automatically after approval, no manual credential handling; right pattern when authorization is tied to user identity. 3.18.3 MCP Setup Reference Context Transport Scope Secrets Handling Personal local tool stdio Local Env vars only, never in config file Shared team server HTTP Project (.mcp.json) OAuth or env vars; never committed Personal experiment stdio/HTTP Local Env vars only Org-wide deployment HTTP Enterprise Admin-managed secrets, config locked 3.19 Enterprise Authentication Patterns 3.19.1 Auth Method by Service Type Service Type Auth Method Secret Location Remote, user identity (SaaS/cloud) OAuth (server returns 401 → browser sign-in) Token issued/stored by client Remote, service identity (internal API) API key via environment variable Environment only, injected by CI/pipeline runner Local, filesystem access stdio, no network auth Filesystem permission model + deny rules as governance 3.19.2 Secret Management: Three Practices (1) Separation — config holds only a variable reference, never the value, since committed values persist in repo history; (2) storage — env var for local/short-lived secrets, a secret store for shared/audited ones (enables single-point rotation); (3) rotation — replace on a schedule and immediately after suspected exposure, scoped narrowly per credential. 3.20 Regulated/Enterprise Deployment Requirements 3.20.1 Beyond Authentication: Three Requirements (1) Configuration lock — enterprise managed settings prevent developers from overriding auth setup; (2) audit logging — a PostToolUse hook logging every tool call/parameters, firing deterministically and unskippable by the model; (3) data residency — an HTTP endpoint pinned to a region plus a platform deployment enforcing regional processing. 3.20.2 Authentication/Integration Checklist Service Type Auth Secrets Logging Config Lock Remote, user identity OAuth Client-stored token PostToolUse hook Enterprise managed settings Remote, service identity API key (env var) Environment only PostToolUse hook Enterprise managed settings Local Filesystem permissions None needed PostToolUse hook Deny rules in managed settings 3.20.3 Postmortem: OAuth Staging-to-Production Failure OAuth redirect URIs are registered per host, so a working staging connection doesn't guarantee production works — the production host's URI isn't automatically authorized, and regulated customers often require separate app registrations per environment. Fix: add the new host's redirect URI before cutover and include this in the deployment checklist. 3.21 Code Modernization as an Applied Case Study 3.21.1 Modernization Risk Profile Legacy modernization concentrates high blast radius, unpredictable dependencies, and limited reversibility; the Explore/Plan/Code loop and Plan mode hold the agent in read-only review, hooks guard sensitive paths during high-risk phases, and CLAUDE.md carries target-pattern conventions so the agent doesn't drift back to legacy patterns. 3.21.2 Three Scoping Questions Before high-risk agentic work, ask: (1) what's the blast radius — which systems depend on this code; (2) how are changes audited — is a PostToolUse hook logging every tool call sufficient; (3) who approves each phase before the next begins — Plan mode enforces the explore/execute boundary, but the approval process itself must be defined beforehand. Module 4: Production Engineering, Evals & Security 4.1 Module Introduction 4.1.1 Core Theme: Production Reveals What Development Hides Success criteria never written down, retriable errors never given a path, budgets never instrumented, action boundaries never enforced — these are decisions, not bugs, and must be made on paper before failures happen live. 4.1.2 The Design Document — Four Decisions Decision What It Defines Success criteria Concrete, checkable output definitions (e.g., "a two-sentence summary listing every action item and owner") — what the eval set is built from Failure handling Expected production errors, each marked retriable/terminal, and what the user sees on unrecoverable failure Cost and latency budget Per-request budget, monthly cost ceiling, latency target, minimum reliability floor — set before architecture Trust boundary Which inputs are untrusted, and the smallest set of actions/access the feature needs — turns least privilege into an enforceable hook, not a remembered setting 4.2 Evals & Judges 4.2.1 What an Eval Is A fixed set of input cases + expected behavior + grading, run and averaged into a trackable score — turns "done" from a feeling into a number. Write the eval before building the feature, forcing success to be defined upfront. 4.2.2 Minimal Eval Pipeline Load dataset → run each case → grade output → average scores. Change one variable (prompt, tool, or model) at a time between runs so you know what caused any score movement. 4.2.3 Three Grading Methods Method Best For Catches Blind Spot Exact/string match Output with exactly one correct form Wrong answer, zero ambiguity, near-zero cost Fails on any valid paraphrase or reordering Code-graded check Structured/code output Invalid JSON, unparseable code, out-of-range values, missing fields Confirms well-formedness only, not content quality LLM-as-judge Open-ended quality (faithfulness, instruction-following, tone) What no code rule can express Noisy, costly, produces a confident-looking but meaningless number until calibrated 4.2.4 Cost Dimension Exact match/code checks run locally at near-zero cost (thousands per change); a judge is a second model call per case — grade format/structure with code on every commit, reserve judge calls for slower scheduled quality passes. 4.2.5 Building/Calibrating a Judge Prompt the judge for strengths, weaknesses, and reasoning alongside the score, or it drifts to a "safe middle" (~6) regardless of quality. Calibrate by running it against a human-labeled set and measuring agreement — low agreement means an untrustworthy score, fixed by tightening the rubric. 4.2.6 Coverage Over Perfection A larger, noisier automated eval set usually reveals more than a small hand-graded one; use Claude to generate additional edge cases from a small labeled starting set, then spot-check them. 4.2.7 Iteration Workflow Set goal → write initial prompt → run eval → read per-case failures → apply one change → re-run. Per-case breakdown matters as much as the average (can hide fixes canceling breaks); categorize failure cause (formatting, retrieval, long-input) to target the next fix. 4.2.8 Postmortem: Field-Extraction Two-Date Failure A feature passed ~12 manual checks (all single-date messages) and shipped with shape-only validation; a message with two dates extracted the wrong one, since validation confirmed shape, not correctness, and no holdout set covered that case. Fix: define graded cases including model-generated edge cases before shipping — the eval guards against regressions, it doesn't fix the underlying prompt. 4.3 Testing & Tracing 4.3.1 Four Test Levels Level Isolates Cannot Catch Unit One function (parser, tool wrapper) alone How components fit together Functional One Claude call returns expected shape for an input Failures in the surrounding system Integration The handoff/seam between two components (e.g., retrieval → model) Whole-flow behavior emerging only end-to-end End-to-end Full flow as a user would run it Where the break is — slowest, hardest to localize 4.3.2 Integration Seam Failures Most silent production failures live at the integration seam — each side passes its own tests while the handoff between them is broken (e.g., one returns a list of dicts, the other expects a plain string). 4.3.3 Tracing Records each run step (prompt, tool calls, intermediate outputs, timing), turning "the case failed" into "step 4: parser raised KeyError on a field the model didn't return" — a 5-minute fix instead of a day of investigation. 4.3.4 Retrieval Routing (Cost-Aware) A cheap classifier routes simple lookups to fetch-once retrieval and multi-part questions to iterative/agentic search, avoiding defaulting everything to the expensive or shallow path. Skip the router if all traffic is one shape. 4.3.5 Postmortem: Unit/Functional Passed, End-to-End Failed retrieve() returned a list of dicts while build_prompt() expected a plain string, causing malformed context and the model to answer from memory instead of retrieved content — only an integration test exercising the real handoff would catch this. 4.4 Failure Handling & Model Selection 4.4.1 Retriable vs. Terminal Category HTTP Status Codes Examples Retriable 429, 529, 500, 502, 503, 504 Rate limit, overload, transient server fault, timeout Terminal 400, 401, 403, 404 Bad request, auth failure, missing resource, permissions problem Misclassifying as terminal fails loudly and gets fixed (safe default when unsure); misclassifying as retriable hammers the service and hides the real problem. 4.4.2 SDK-Level Retries Anthropic client libraries auto-retry transient failures with progressive delays up to a configurable cap — avoid stacking a custom retry loop on top (multiplies attempts against a rate limit); either let the SDK own retries or turn them off and own the whole path. 4.4.3 retry-after Header Present on 429/529 responses, gives the exact wait time — more precise than blind backoff. Read it first; fall back to exponential backoff only when absent. 4.4.4 Tool Errors & Refusals Tool errors must return to Claude explicitly with is_error: true — a silent empty result is treated as valid data, producing a confident-but-wrong answer. A refusal (stop_reason: "refusal") is a 200 at the HTTP layer, so the retriable classifier won't catch it — treat it as terminal, raise and log, never retry. 4.4.5 Error-Handling Decision Table Error Retriable? Strategy Fallback Rate limit (429) Yes Exponential backoff + jitter, honor retry-after, capped attempts Clean error or cached/simpler result after cap Overloaded (529) Yes Backoff (Anthropic-side load, not a rate-limit signal) Fail over or graceful error if persists Bad request (400) No No retry Fix/reject input, surface to caller Tool result error Depends Retry only if cause is transient Return error flag to Claude, never silence Refusal (200, stop_reason: refusal) No No retry Raise to caller, log, never treat as valid output 4.4.6 Postmortem: Unhandled Rate Limit + Retry Storm A loop with no error handling worked in low-volume dev; the first production 429 raised an unhandled exception, and the instinct to add immediate retries deepened the rate limit. Fix: honor retry-after first, fall back to capped exponential backoff with jitter, and fail fast on terminal statuses (400/401/403/404). 4.4.7 Model Selection in Production Family order: Fable (hardest reasoning/coding/agentic) → Opus (above Sonnet's envelope) → Sonnet (balanced default) → Haiku (speed/cost-optimized). Start with Sonnet, move up only when an eval shows a missed quality bar, move down only when an eval shows an acceptable drop — defaulting to the most capable model "just in case" is the most common, expensive mistake. Route bulk traffic to a default model, override via a cheap signal (task type, length, difficulty) only where needed. 4.5 Cost & Orchestration 4.5.1 Observability: Three Metrics Instrument token usage (input/output), latency, and error rate per call from the start — per-call logging turns "why is the bill high?" into "which step, on which request type, is responsible?" 4.5.2 Cost/Latency Levers: Model Selection & Streaming Route simpler work to smaller/faster models, reserve capable ones for steps that need them. With streaming + tool use, accumulate content_block_delta events by index and never act on a tool_use block until the stream closes — a broken mid-stream response needs a full retry, not partial output passed downstream. 4.5.3 Prompt Caching Economics Cache writes cost a premium (1.25x base for 5-min TTL, 2x for 1-hour); cache reads cost ~0.1x standard input — only pays off when reads outnumber writes. Two modes: automatic (single flag, system manages breakpoints) or explicit cache_control breakpoints. 4.5.4 Message Batches API Async processing at lower per-request cost in exchange for non-immediate completion — right for non-urgent, high-volume work (overnight jobs, backfills), wrong for anything a user is waiting on; compounds with prompt caching when a batch reuses shared context. 4.5.5 Multi-Agent Orchestration (Orchestrator-Worker) A lead agent decomposes a task, delegates to parallel subagents, then synthesizes results — genuinely helps independently-splittable tasks (e.g., research) but Anthropic's internal eval showed it costs ~15x a normal chat interaction, and is less effective on tightly coupled tasks like coding where steps can't be parallelized. Use a capable lead with cheaper subagents to reduce the multiplier; failure handling multiplies with agent count. 4.5.6 Reliability Floor Define a base (retry budget, latency ceiling) first, then tune cost above it, never below — cost pressure is more visible daily than reliability pressure, so the eval's pinned baseline score is what makes the floor enforceable before shipping. 4.5.7 Postmortem: Fan-Out on a Tightly-Coupled Task Parallel fan-out applied to a sequential, dependent-step task tripled the bill with barely improved quality, since subagents mostly waited on each other. Reverting to single-agent restored cost with equal quality. 4.6 Security 4.6.1 Prompt Injection: The Core Threat The model processes its entire context as one undifferentiated token stream — no structural boundary separates trusted instructions from untrusted data, so hidden instructions in fetched content are read as commands. 4.6.2 Defense: Treat Content as Data Treat all fetched/user-supplied content as data to examine, never instructions to follow — trusting your own users doesn't help, since the hostile instruction typically arrives via retrieved content. Delimiter-wrapping helps but is a soft boundary; the reliable boundary is what the agent is allowed to do. 4.6.3 Threat Scope & Jailbreak vs. Injection Any content someone else can write is a vector (shared docs, DB records, emails, chained tool outputs), direct or hidden. Jailbreaks target the model's own safety constraints; injections hijack the application's instructions — both need the same layered defense. 4.6.4 Secure-by-Design Identity and Access Agent identity carries only the narrowest permission set the task requires; secrets live in env vars or a secret manager, never committed config. Least privilege bounds the blast radius of a successful injection; committed secrets are permanent exposures since only rotation (impossible for a hardcoded value) fixes a leak. 4.6.5 Hook-Based Guardrails A PreToolUse hook blocks a tool call before execution and logs every privileged action — enforced control vs. an unenforced prompt-level rule. Precedence when rules conflict: deny > ask > allow. 4.6.6 Scoping for Regulated Review Three early questions: where is data processed (residency), how is access logged (per-action audit trail), and can configuration be centrally administered (prevents developers quietly widening permissions). 4.6.7 ZDR Note Zero Data Retention eligibility varies by model and platform, not guaranteed even under an existing agreement — confirm against Anthropic's Trust Center (and Bedrock/Vertex/Foundry retention policies separately) at scoping time. 4.6.8 Layered Security Model Model training + classifiers (reduce injection landing) → treating content as data (reduce acted-upon injections) → least privilege + locked config (bound blast radius) → hooks (enforce + record) → regulated-review scoping (make it auditable) — no single layer is sufficient alone. 4.6.9 OS-Level Sandboxing Isolates at the process level regardless of hook/identity config: filesystem isolation restricts the agent to its working directory, network isolation restricts outbound connections to a named endpoint set. Holds even when a hook is missing or bypassed — typically the first thing enterprise security reviewers ask about. 4.6.10 Postmortem: Hidden Instruction Redirected a Write An internal-only agent skipped validating fetched web content, assuming trusting the user meant trusting the request; a hidden instruction in a fetched page redirected the agent's write to an unintended path. Fix: treat fetched content as data plus a PreToolUse hook denying writes outside the permitted path. 4.6.11 Defense Checklist Threat Entry Point Control Logged Prompt injection Hidden instructions in fetched content Treat as data + hook refusing untrusted-triggered actions Source, attempted action, block Jailbreak Crafted user prompt Input validation + model action constraints Flagged prompt, refusal Over-broad access Identity scoped wider than needed Least privilege, secrets manager, locked auth config Every privileged action + identity Sandbox escape Steered agent reaching uncovered paths/endpoints OS-level filesystem/network isolation Every denied access attempt + trigger 4.7 Cumulative Task 4.7.1 Cumulative Task Structure A single runnable application containing three planted defects, one per layer (eval/testing, failure-handling, security) — e.g., an agent that writes based on untrusted fetched content without a hook, a retry loop with no real backoff that retries terminal errors too, and no is_error handling on tool results. Module 5: Accelerators & IP Contribution 5.1 Module Introduction 5.1.1 Core Theme A build that works is not yet a build that survives reuse, review, or deployment — templates aren't configurable, contributions aren't verifiable by a stranger, models aren't pinned, platforms haven't cleared compliance, and trust boundaries haven't been mapped. Much of this work is driven by the customer's cloud/compliance posture, not technical preference. 5.2 Packaging for Reuse 5.2.1 Accelerator Definition A solution packaged so future engagements start from a working foundation instead of a blank repo — separating engagement-specific code from a parameterized reusable core. Package while the build is fresh, before the reasoning behind hardcoded values is lost. 5.2.2 Three Asset Types Asset Type What It Bundles Correct Packaging Requires Agent Template System prompt, tool schemas, loop structure Pull domain values into configuration with documented defaults — new team sets values, doesn't edit the loop MCP Server Package Exposed tools, their inputs, controllable scope Document each tool input; let the installing team set scope — installs without code edits Eval Suite Graded test set + judge rubric Ship dataset and rubric together as the deployment gate (run against a pinned baseline before promoting a new model version) 5.2.3 Common Failure: Loose Scripts Shipping an agent as loose scripts instead of a template looks reusable because it "runs," but customer-specific values stay buried across files, so the next team copies and diverges instead of configuring one asset. 5.2.4 Documentation & Audit Bundling Documentation must cover environment assumptions, expected inputs, handled failure modes, and the defining eval, or the next team treats the asset as a black box. Bundle the audit log (data touched, identity acted under) too — a regulated reviewer asks for this at the first security review. 5.2.5 Packaging Checklist Asset Type Parameterize Document Bundle for Audit Agent template Prompts, paths, scopes, credentials by reference, thresholds Environment assumptions, expected inputs, handled failures, defining eval Data touched, identity acted under, action log MCP Server Scopes, credentials by reference, per-customer paths Expected inputs per tool, scope boundaries, handled failures Data touched, identity acted under, action log Eval Suite Thresholds, dataset paths Rubric logic, what scores mean, pinned baseline Data touched, identity acted under, action log 5.2.6 Postmortem: Hardcoded Values Labeled Reusable A team hardcoded customer-specific values into a template to hit a deadline, then labeled it "reusable"; a second team couldn't configure it (no parameters, no documentation, no bundled eval) and had to rewrite it fully. A template that runs has not been packaged for reuse — these are different finishing states. 5.3 Contributing Back 5.3.1 Definition Moving an asset from private reuse to shared infrastructure through a documented channel carrying version, install steps, and components as one unit — an asset already packaged for internal reuse is already close to what a maintainer needs. 5.3.2 Matching Contribution to Channel The Claude Cookbook takes self-contained, single/multi-pattern reference implementations demonstrated end to end, not a full application; open-source MCP servers/tools each have their own repo and conventions. Misplacement — sending a full multi-component app to the Cookbook — is one of the most common reasons a contribution never gets reviewed. 5.3.3 Four Things That Make Verification Possible (1) Does one thing, (2) an example shows it running, (3) a test proves it works, (4) a short statement names the assumptions — the bar is set by what needs checking, not code cleverness. 5.3.4 Rights and Attribution Come First Licensing decides whether a contribution can be accepted at all; code carried in from a customer engagement may have constraints on where it can go. Confirming the right to contribute and attributing prior work is a gate that must be passed before technical review. 5.3.5 Contribution-Readiness Reference Channel What a Maintainer Checks Licensing/Attribution Example/Test Bar Cookbook (focused example) or tool/server's own repo Code does one thing, fully readable Confirm right to contribute engagement code, prior work attributed Runnable example + a test proving behavior, not just description 5.3.6 Postmortem: Unreviewed PR for Three Weeks A PR sat unreviewed because it had no test, no example, and no stated environment assumptions — the maintainer couldn't verify it without reconstructing the developer's work. A contribution the reviewer can't verify sits at the back of the queue regardless of code correctness. 5.4 Requirements & Lifecycle 5.4.1 From Business Problem to Functional Requirements A functional requirement states what the system must do with enough detail to check (e.g., "classify each ticket into one of four queues... never auto-send without human approval," not "help agents answer faster") — a specific goal becomes an eval line and a review criterion. 5.4.2 Deriving Infrastructure Requirements Non-functional constraints derived by asking: latency (how fast, measured where?), scale (how many requests, at what peak?), residency (where must data be processed?), identity (who acts, under what credentials, what's auditable?) — these four most often decide the deployment platform. 5.4.3 Documenting Requirements A short record of functional behaviors, infrastructure constraints, and the regulation each derives from lets a platform choice be defended as following from requirements rather than familiarity. 5.5 Systems Lifecycle for Claude Applications 5.5.1 Seven Phases Requirements (capture functional/infrastructure needs) → Design (platform, model, trust boundaries) → Build (agent, tools, prompts) → Test (evals, unit/integration/e2e) → Deploy (pin version, gate on eval) → Operate (instrument cost/latency/errors, enforce guardrails) → Iterate (feed production findings back into requirements). 5.5.2 Gating Between Phases A gate is the decision point to move between phases where a regulated engagement retains control (e.g., don't move design→build until the platform satisfies residency). Refusing to skip a gate is what keeps an application reviewable; a one-off experiment may collapse phases, a regulated deployment cannot. 5.6 Deployment & Versioning 5.6.1 Platform Choice Driven by Customer's Cloud Platform Identity/Data Model When to Choose First-party Claude API Anthropic identity and terms No binding cloud/residency constraint; wants newest capabilities first Claude Platform on AWS Anthropic identity/terms via customer's AWS account; inference outside the AWS boundary On AWS but wants Anthropic model IDs/lifecycle parity with first-party API Claude in Amazon Bedrock Messages API at /anthropic/v1/messages; data stays inside customer's AWS boundary On AWS, wants feature parity + compliance posture there Claude on Amazon Bedrock (legacy) AWS identity/billing; InvokeModel/Converse APIs, ARN-versioned IDs Existing (unmigrated) Bedrock integration Google Vertex AI Google Cloud identity/IAM/billing; regional or global endpoints On Google Cloud with compliance posture there Third-party (e.g., Microsoft Foundry) Wrapping product's identity/billing Already runs the platform embedding Claude; residency depends on hosting form (Azure-hosted vs. Anthropic-hosted) per model 5.6.2 Identity and Residency Answered by Platform Bedrock uses AWS identity and keeps data in the customer's AWS boundary; Vertex uses Google Cloud identity/boundary; both offer regional routing. Matching platform to the customer's existing compliance agreement avoids a residency review from scratch. 5.6.3 Pinning Versions Every model ID points to a specific snapshot; aliases (e.g., "Opus," "Sonnet") evolve over time, so pin the full model ID, not the alias (e.g., claude-haiku-4-5-20251001 vs. the moving claude-haiku-4-5), and version prompts/assets alongside code with a rollback-ready prior version. 5.6.4 Promote via the Eval Send a new version to a portion of traffic, compare against the pinned baseline, promote or roll back on the result — the eval is the deployment gate, not just a one-time test. 5.6.5 Deployment-Platform Versioning Platform Versioning First-party API Pin full model ID, keep prior snapshot Claude Platform on AWS Same ID format as Claude API; lifecycle follows Anthropic's schedule Claude in Amazon Bedrock Pin full model ID with anthropic. prefix; partner retirement dates differ Claude on Amazon Bedrock (legacy) Pin via ARN-versioned identifiers Google Vertex AI Pin full model ID before rollout; partner retirement dates differ Third-party platform Pin per the platform's own versioning controls 5.6.6 Postmortem: Moving Alias Broke Production A deployment shipped against a moving alias ("opus"); it silently advanced to a new version, breaking downstream parsing with no pinned prior version to roll back to, forcing a hotfix instead. Lesson: pin the full model ID, retain the prior pinned version, gate promotions through the eval. 5.7 Comparing Platforms 5.7.1 Latency Depends on platform location relative to the customer and feature-access timing (first-party API usually gets new capabilities first) — must be measured from the customer's actual region/payload. Within Bedrock, global vs. regional endpoints is the primary residency control and can affect cost. 5.7.2 Compliance Often Ends the Debate A customer already certified on one cloud is unlikely to re-certify on another; residency/certifications/audit access differ by platform and are pass-or-fail for regulated customers. First-party API may lack EU residency (Bedrock/Vertex typically required); on Foundry, hosting is per-model. Raise compliance constraints at scoping, not at contract review. 5.7.3 Cost Beyond Per-Token Rate Token rates are broadly aligned across platforms; total cost is driven by egress, platform fees, and integration effort — instrument cost per call per platform rather than comparing token price alone. 5.7.4 Cross-Platform Comparison Reference Dimension How It Differs How to Measure Winner Latency In-region platform shortens round trip; first-party API gets features first From customer's actual region + payload In-region cloud wins on latency; first-party wins on feature access Compliance Residency, certifications, audit controls vary by platform Against customer's existing certification/residency requirements at scoping The already-certified platform wins Cost Token price, egress, platform fees, integration effort all vary Total cost per call including egress/integration Lowest total cost for the actual workload 5.7.5 Postmortem: Familiar Platform Failed Residency Review A team picked the platform they knew best for a regulated customer since migration was fast; it passed functional tests but failed the customer's residency requirement at security review, requiring a rebuild. Familiarity optimizes for build speed, not for whether the deployment is allowed to ship. 5.8 Trust Boundaries 5.8.1 Multi-Component Coordination An app might chain an API request → Claude Code task → MCP server reaching a customer system; each connection creates a place where identity, secrets, and untrusted input can cross — map what each component does before connecting anything. 5.8.2 Least Privilege Applies to the Whole Application Each component operates under its own identity, scoped to only what its task needs — the application is only as contained as its most privileged seam, so one overly-broad component becomes the weak point even if others are properly scoped. 5.8.3 Regulated Review Requirements Requires justifying audit logging, data-residency decisions, and permission controls across the full application, not per component — confirm ZDR/HIPAA BAA eligibility for each individual component against the Trust Center before scoping. 5.8.4 Multi-Component Integration Map Component Contributes Trust Boundary at Seam Control First-party API Orchestrates workflow, holds entry point Request entering the app from outside Input validation + identity the call runs under Claude Code task Runs agentic work, may fetch external content Content it fetched — untrusted downstream Treat fetched content as data at the next seam MCP server Reaches a customer system to read/act System access held on the app's behalf Scope to least privilege + log the access 5.8.5 Postmortem: Untrusted Content Passed as Trusted Input Three components each passed their own tests; content fetched by the Claude Code task was passed directly into the next call as trusted input with no boundary control at that seam — a hidden instruction there would have executed. A component being trusted in isolation says nothing about the seam leaving it. 5.9 Cumulative Task 5.9.1 Cumulative Task Structure A single deployed accelerator with three planted defects: a hardcoded customer-specific value where a parameter belongs (packaging), a moving model alias instead of a pinned full ID (versioning), and fetched untrusted content passed directly into the next call with no boundary control (trust boundary). Task: identify all three, explain the runtime consequence, and write the corrected lines.