Code Review Checklists for Agent Configuration PRs

Agent configuration files aren't documentation. A YAML file that defines what an AI agent can read, write, call, and spend takes effect the moment that agent runs, which means the PR that changes it deserves the same scrutiny as a firewall rule change. Check Point Research disclosed two vulnerabilities in Claude Code, one carrying a CVSS score of 8.7, enabling remote code execution and API key exfiltration through repository-level attack vectors. The attack surface wasn't a prompt. It was repository-level configuration, sitting in plain sight, waiting for someone to open the project.

That's the part engineering teams tend to miss. A developer commits a policy file with a typo that silently disables every deny rule, and the agent runs unprotected until someone happens to notice, usually after something has already gone wrong. The visibility gap isn't at runtime. It's in the repo, at review time, in the five minutes someone spent skimming a diff they assumed was routine. A Dark Reading poll found 48% of cybersecurity professionals now rank agentic AI as the top attack vector heading into 2026, yet only 21% of executives report full visibility into what their agents can access, call, or touch. That gap between concern and visibility is exactly where a structured checklist earns its keep.

What follows is that checklist, organized around the five properties that separate a production-grade agent deployment from an experiment that got lucky.

Credential scope: what the agent is allowed to touch

Every credential handed to an agent defines a blast radius. The reviewer's job is to confirm that radius is as small as the task allows, not as large as convenience allows.

Start with the basics. Are credentials scoped to the specific repositories, APIs, and services the agent actually needs, rather than inherited wholesale from a broad service account? Are they minted per session, or do they persist after the agent's work is done? A long-lived token that survives the session is a door left unlocked long after the reason for opening it has passed. Write access should be separated from read access, and write access should be boxed into the declared working directory, not the whole filesystem.

MCP servers deserve their own line of scrutiny. The config should explicitly enumerate which servers the agent can connect to, because approving a server on convenience and trusting prompt constraints to contain the agent afterward is the most common failure mode reviewers let slide. According to NHIMG's State of MCP Server Security report from 2025, only 18% of MCP server deployments implement any access scoping for tool permissions at all. That's not a rounding error. That's most of the ecosystem running wide open.

Secrets belong in a vault or secret manager, injected at runtime, never as literals sitting in the config file itself. A config file with a credential written in plaintext is a leak already in progress, just waiting for someone to grep the git history. And outbound calls need an explicit allowlist of endpoints; absence of one means any tool, or any sub-agent spawned by that tool, can reach out to wherever it wants.

The OWASP Agentic AI Top 10, published in December 2025 and reviewed by more than a hundred security experts with endorsements from NIST and Microsoft, names identity and privilege abuse and tool misuse among its ten critical risks. Both are preventable at exactly this review step, before merge, not after an incident report. And the risk isn't theoretical: the ClawHub registry, the primary marketplace for OpenClaw skills, was poisoned at scale in the first quarter of 2026. Five of the seven most-downloaded skills at peak infection turned out to be confirmed malware. Skill and server sources need review, not the assumption of safety that comes from a high download count.

The question a reviewer should be asking throughout: if this credential were compromised, or if the agent were prompt-injected mid-session, what's the worst-case reachable surface?

Budget caps: preventing runaway spend before the agent runs

One widely cited case involved an enterprise that deployed agent access broadly and burned through its annual AI budget within months of rollout. Separately, one enterprise reportedly spent $500 million in a single month after deploying agent access with no usage caps in place at all. Neither of these is a story about bad engineers. It's a story about a missing config field.

Agentic workflows eat tokens at a different order of magnitude than chat. A task that costs a few thousand tokens in a standard chatbot session can cost far more once an agent starts looping, calling tools, and spawning sub-agents that spawn their own calls in turn. Average monthly enterprise AI spend hit $85,521 in 2025, up 36% year over year, and that climb happened despite token prices falling over the same period. Session counts and complexity are outrunning the unit economics.

So the checklist has to be concrete. Does the config declare a hard ceiling on tokens or cost per session, a stop rather than an alert someone might ignore at 2 a.m.? Is there a per-developer or per-team cap sitting underneath the account-wide limit, so one runaway session doesn't quietly consume the whole org's headroom? Does the config specify which model the agent runs on, or does it default to whatever's most capable, and most expensive, available at call time? Splitting model selection by sub-agent, planning on a frontier model and executing on something cheaper, is now standard practice among teams that have been burned once already.

Prompt caching matters here too. For repetitive tasks or stable system prompts, caching can cut costs by 50 to 90% on eligible requests, according to Anthropic's own documentation. Claude Sonnet 4 and 4.5 pricing, for reference, runs $3.00 per million input tokens and $15.00 per million output tokens, with cache reads discounted 90% when cache control headers are set explicitly. Parallel sub-agent spawning needs a cap too, since uncapped parallelism multiplies spend in a way that no single per-session limit will catch in time.

Spend also needs to be tagged, not just monitored. Team, feature, and trigger metadata attached to every session means cost is attributable after the fact, not just visible as a lump sum on an invoice. A survey from Benchmarkit and Mavvrik found only 34% of companies had mature AI cost management in place; 57% were still tracking spend in spreadsheets. For most organizations, the budget cap in the config file is the only enforceable control that exists.

Ask this at review time: if this agent ran continuously for an hour with zero human interruption, what's the maximum billable output, and is that number one anyone signed off on?

Sandbox isolation: what the agent's execution environment can reach

A credential scope is a promise on paper. Sandbox isolation is what makes that promise hold, because an agent running in a shared environment can escalate past its declared permissions through the host itself, regardless of what the credential file says.

The reviewer's checklist here starts with a basic question: does the config specify an isolated sandbox, not a shared CI runner and not a developer's local machine? Is that sandbox ephemeral, torn down after the session, or does it persist and accumulate state, cached credentials, and side effects across runs it was never meant to remember? Network egress controls matter independently of the allowlist in the config; if the sandbox itself doesn't enforce those boundaries, the allowlist is a suggestion, not a wall.

Filesystem access should be locked to the declared working directory. An agent that inherits the host's full filesystem has a scope no config file can meaningfully constrain. And process spawning outside the declared tool set is its own risk category: OWASP's ASI05, unexpected code execution, can be triggered through prompt injection or unrestricted subprocess access, and all three routes run straight through a sandbox with soft edges.

Sub-agents complicate this further. Each one needs its own isolation boundary, because a parent agent's sandbox permissions bleeding into a spawned child agent defeats the purpose of scoping the parent in the first place. OWASP's Agentic AI Top 10 calls out cascading failures and insecure inter-agent communication as distinct risks, and both show up precisely where those parent-child boundaries were never explicitly drawn.

A field study of experienced developers working with agentic tools, drawing on 13 direct observations and 99 survey responses, found that professionals apply deliberate discipline in how they scope and verify agentic work. Sandbox isolation is that same discipline, just built into infrastructure instead of habit.

The reviewer question worth repeating every time: if this sandbox were compromised, what's reachable from inside it that isn't already reachable through the declared credential scope? If the answer is "more than the credentials allow," the sandbox is the actual attack surface, not the config.

Trigger conditions: when the agent is allowed to act

Trigger conditions are the field reviewers skip most often, because attention naturally goes to what an agent can do once it's running, not to what causes it to start in the first place.

The checklist needs to close that gap directly. Is the trigger source enumerated explicitly (specific GitHub events, specific Slack commands, specific API calls, a cron schedule) rather than left as a wildcard that fires on anything resembling the right shape? Is there an allowlist of actors or roles permitted to invoke the agent at all? An agent triggerable by any commenter on a public repository is an open invitation, not a control.

Prompt injection through trigger inputs deserves its own line item. PR descriptions, issue titles, and commit messages are all untrusted text the moment an agent reads them as instructions instead of data, and a config that doesn't guard against that distinction is handing attackers a direct channel to redirect the agent's goal. Branch and environment restrictions matter for the same reason: an agent that can trigger on any branch, including main or a release branch, carries scope no reviewer actually approved. Rate limits and debounce logic prevent event storms from turning a single trigger into a runaway loop of invocations. And the config should say, plainly, what happens on ambiguous or failed input: does the agent halt and wait for a human, or proceed on a default guess?

OWASP ranks agent goal hijacking, ASI01, as its top risk, and in practice it enters most often through exactly this channel: trigger inputs carrying attacker-controlled content the agent mistakes for legitimate instruction. Ellipsis, a cloud runtime for YAML-defined coding agents, addresses this partly through scoped credentials and audit logs that are declared in the config itself. One governance approach worth noting, from AgenticRail, has the caller declare intent up front so that deviations can be refused rather than detected after the fact. That's the trigger-review principle in miniature: declare intent up front, and refuse deviation rather than trying to detect it after the fact.

The same principle applies at the repository level: configuring required checks and human reviewers so no agent-authored PR merges without oversight points in exactly the same direction. The trigger review at the config-file level is just that same guardrail, applied one layer earlier.

Ask plainly: who or what can fire this agent, and could that actor be spoofed or manipulated into firing it on someone else's behalf?

Audit logging: what the agent's actions leave behind

Of the observability layers missing most often from production agent deployments, the audit trail tops the list, with per-request tracing close behind. Neither is optional once an agent has write access to anything that matters.

The distinction reviewers need to hold onto is between logging outputs and logging actions. A log that records what the agent said but not what it called isn't an audit trail, it's a transcript, and transcripts don't help six months later when something's gone wrong and nobody can reconstruct why. Structured logging of every tool call is the baseline. Those logs need attribution to a specific person or trigger, not just to the agent's own identity, so there's a human accountable for each invocation rather than a shrug and a system name.

Retention matters just as much as capture. Logs need export to a SIEM or an immutable store, because logs that live only inside a session container disappear the moment that sandbox gets torn down, taking the evidence with it. Token counts and cost should be recorded per tool call and per session, not just totaled at billing time when the granularity is already gone. Diff logging, a record of every file change the agent made rather than a single squashed commit at the end, preserves the actual sequence of what happened. And ideally, the config supports reconstructing the agent's full reasoning path for a session: what it was given, what it called, in what order.

For regulated environments, one more field matters: which model version produced a given output, and which policy governed that session. Regulators are already asking that question routinely, and a config that can't answer it leaves the organization exposed regardless of how the agent actually behaved.

Emerging AI governance frameworks increasingly map audit requirements to dimensions like accountable ownership and lifecycle controls. Audit logging is what satisfies those dimensions directly; without it, they remain aspirational rather than enforced. Veracode's 2025 State of Software Security report found flaw half-life sitting at 252 days, with average fix times running 47% longer than in 2020. Gaps in audit trails make agent-introduced flaws harder still to detect and attribute than the human-introduced ones that report is measuring.

The question to close with: six months from now, if this agent caused an incident, could anyone reconstruct exactly what it accessed, what it called, what it changed, and who triggered it in the first place?

How to structure the PR review itself

None of the five domains above matter if the checklist lives in someone's head instead of the repository. Agent configuration belongs in version control, reviewed and versioned the same way application code is, because that's the only place a checklist becomes enforceable rather than aspirational.

Practically, that starts with a structured PR description template mapping directly to the five domains: credential scope, budget caps, sandbox isolation, trigger conditions, audit logging. Authors self-attest against each one before a human reviewer even opens the diff. Microsoft's approach with a Governance Attestation GitHub Action illustrates the pattern well: a workflow file that validates a structured checklist before merge, with default sections covering security review, privacy review, legal review, responsible AI review, accessibility, release readiness, and org-specific launch gates, customizable down to the sections that actually apply. Applied to agent config PRs specifically, a subset built around security, privacy, and responsible AI review covers the ground that matters most.

CODEOWNERS rules should require at least one security-aware reviewer on any change to agent config files, the same reviewer who'd sign off on a network policy change, not just whoever's next in the rotation. Required status checks close the rest of the gap: a linter that validates config schema, a secrets scanner that fails the PR outright if literals show up in credential fields, and a cost estimator that computes worst-case spend given the declared model and cap settings before merge, not after the first bill arrives.

Agent config diffs also need to be read differently than application code diffs. A one-line change to a deny rule, or a single budget field, can carry more consequence than a hundred-line feature addition sitting in the same PR queue. Reviewers who treat line count as a proxy for risk will miss exactly the changes that matter most.

NIST SP 800-218A ties AI model development, including generative AI and dual-use foundation models, back to secure software development governance broadly. That framing is useful here: agent config review isn't a nice-to-have bolted onto the development lifecycle, it's a required step within it. And the checklist itself needs the same discipline applied to it that it applies to everything else: version-controlled, evolving through the same PR process it governs, as new capabilities, new MCP servers, and new trigger surfaces get added over time.

Scale makes the automation question non-negotiable. Stripe's engineering organization, per its own public statements from 2025, generates roughly 1,300 pull requests a week, up from about 300 a week before agent integration, a jump of more than four times over twelve months. At that volume, a manual, ad-hoc review process for agent configuration simply doesn't hold. The checklist has to be automated and enforced at merge time, not something a reviewer remembers to run through when they have a spare ten minutes.

Running governed agent configs in practice

The five domains aren't independent controls stacked next to each other. They're a chain, and a weak link in any one of them undermines the rest. A tight credential scope means little if the sandbox executing it has no isolation. A hard budget cap means little if the trigger conditions let anyone on the internet fire the agent. Audit logging matters most exactly when the other four have already failed, which is the moment an organization needs to know what happened and can't afford to guess.

What separates a production-grade agent deployment from an experiment that hasn't broken yet isn't the sophistication of the model or the cleverness of the prompt engineering behind it. It's whether the config file governing that agent went through a review process built to catch the failure modes above, before merge, every time, without exception for the PR that looked routine. The teams that treat agent configuration with the same rigor as a firewall rule change are the ones who'll still be running these systems in production a year from now. The ones who don't will find out the hard way which of the five domains they skipped.