Coding Agent Playbook

Use this page beside your editor. The eight chapters explain the full workflow; this playbook helps you run the next task. For a complete first exercise, start with the synthetic-data quick start.

Source review: September 5, 2026. These patterns are adaptations for research code; exact features and permissions depend on the tool and environment.

1. Give the agent a task it can finish and you can check

Describe one observable result, point to the relevant files, and state the checks that define completion. For example: “Select the latest valid measurement per person, resolve tied dates by encounter ID, and pass these three manually checked cases.” A request to “improve the pipeline” leaves the agent to invent both the work and its acceptance criteria. Official coding guidance recommends clear scope, repository context, and a way to verify the result [Codex best practices].

Use the agent task brief to record:

• The intended behavior and the files the agent may change.
• The agreed methods, input/output contract, and one worked example.
• The test or build command and expected result.
• Decisions that need your input, such as changing a cohort definition.

For a bug, ask for a small failing case first. For a routine, reversible edit, let the agent proceed once the requirements are clear. Reserve a planning-only pass for uncertainty that would materially change the implementation.

2. Keep context small, relevant, and durable

Put stable repository conventions and verified commands in the instruction file your tool supports. Put this task’s methods and expected outputs in its brief. In Codex, AGENTS.md discovery is hierarchical; verify which instructions apply instead of assuming every tool reads the same files [AGENTS.md guidance].

Point to a representative function, an existing test, and the relevant documentation. Ask the agent to inspect those before editing. Adding unrelated logs and entire data dictionaries can make it harder to find the constraints that matter [Codex best practices].

When a session becomes long or changes direction, create a session handoff. Record the goal, decisions, current changes, commands already run, unresolved failures, and next action. A fresh session should confirm that summary against the files and Git diff. This is a practical adaptation of persistent execution plans, not a special filename that every agent loads automatically [Execution plans].

3. Set permissions where access actually happens

An instruction to avoid a directory does not prevent a tool from reading it. Neither does .gitignore. Use a workspace and tool permissions that expose only what the task needs; keep protected datasets outside the agent’s accessible environment.

flowchart TB
    accTitle: Keep protected records outside the agent workspace
    accDescr: The agent edits code using synthetic fixtures. A researcher reviews and transfers the code into an approved data environment, where protected records remain. The researcher reviews results and logs; only outputs approved for sharing become context for the agent.
    subgraph workspace["Agent workspace"]
        draft["Agent edits and tests<br/>Code + synthetic fixtures"]
    end
    review["Researcher reviews code"]
    subgraph protected["Approved data environment"]
        run["Researcher runs code<br/>with protected records"]
        results["Results and logs stay here"]
        release["Researcher reviews outputs"]
        run --> results --> release
    end
    context["Context for agent"]
    draft --> review
    review -->|Researcher transfers code| run
    release -->|Approved for sharing| context

Code crosses into the approved data environment after researcher review. Protected records stay there; only outputs approved for sharing may return as agent context.

Review shell, browser, app, and MCP access separately. For example, Codex documents that its command network proxy does not govern every other connected tool surface. “Shell networking is disabled” therefore does not establish that every connection is disabled [Agent security and approvals].

Treat instructions encountered in retrieved pages, issue text, and tool results as untrusted material. They do not authorize new uploads, credential access, or changes to the task. For research data boundaries, see tool selection.

4. Delegate only work that can proceed independently

Useful parallel tasks include a source check and a code change, or implementation and a separate review of the method. Give each implementation task distinct file ownership, a shared acceptance contract, and a named integration step. Two agents editing the same function usually create more review work.

Separate working trees can isolate file changes, but they do not reconcile incompatible design decisions. Start with one agent for a small task; add another when independence is clear. This is a workflow recommendation, not a claim that extra agents improve every task.

For review, ask for findings with file locations, a concrete failure case, and the expected behavior. A second agent can provide a fresh inspection, but its agreement is not independent scientific evidence. A human still checks the method, and executable tests check the implementation.

5. Test the behavior, then examine the change

Write expected results from the agreed method before looking at the generated answer. Include missing values, ties, empty input, and values immediately around any thresholds. For numerical results, choose a justified tolerance; for labels, identifiers, and ordering, check exact expectations [testthat expectations].

Useful research-code checks include:

Contract	A check that can expose a real error
One output per eligible person	Compare IDs and selected encounters with a hand-checked fixture.
Deterministic tie breaking	Shuffle the input and verify the same selected records.
No silent unit conversion	Supply a documented unit mismatch and require a diagnostic or explicit conversion.
Missing information stays missing	Verify a missing value is not replaced with zero or an invented estimate.
Refactoring preserves behavior	Compare outputs on established fixtures, including stdout and exit status where relevant.

Inspect the staged diff before committing. Have the agent report the command, result, and anything it could not run. If a check fails repeatedly, reduce the input and revisit the assumption before requesting another rewrite.

6. Reproduce both the environment and the analysis

These tools address different parts of reproducibility:

Need	Practice	Limit
Recreate R packages	Record dependencies with renv; restore from renv.lock.	R itself and system tools still need compatible installations.
Rebuild affected analysis steps	Declare dependencies with targets.	Cached results can still embody an incorrect method.
Reproduce a reported result	Record code revision, input version, parameters, seed, and commands.	A seed does not make remote model services deterministic.

For an R project that does not already have an environment manager, initialize renv, test the project, then snapshot its dependencies. A collaborator restores the lockfile in their own checkout [renv introduction].

# Once, in the R project you intend to manage:
renv::init()
# After installing the required packages and checking the analysis:
renv::snapshot()
# In another checkout, after installing the recorded R version:
renv::restore()

Install renv as an explicit setup step if needed. Commit renv.lock, .Rprofile, renv/settings.json, and renv/activate.R; keep installed libraries out of Git. Record R and system dependencies separately. An analysis script should report a missing dependency instead of silently installing the newest package during a run [renv introduction].

When a project grows beyond a short script, targets can track steps such as input → clean → summarize → report. Declare external file dependencies with format = "file" and return their paths; otherwise, a change to a file may not invalidate the step you expect. Use targets::tar_outdated() to inspect what needs work and targets::tar_make() to run the pipeline [targets walkthrough], [File tracking]. Keep the _targets/ store in the approved data environment because it may contain intermediate records.

7. Evaluate an LLM that remains in the running pipeline

There is a difference between testing R code an assistant helped write and evaluating a model that extracts fields during every analysis run. A schema can enforce shape while the values remain wrong. Let absent information remain absent: for example, ellmer structured extraction supports optional fields with required = FALSE. Its documentation also recommends representing tables as arrays of row objects [ellmer structured data].

For an extraction task, keep a small, expert-checked set of permitted or synthetic examples separate from the examples used to refine the prompt. Include absent, ambiguous, and contradictory information. Check exact field accuracy, unsupported additions, and when the model appropriately abstains. Inspect individual failures alongside any overall score [Evaluation practices].

Record the prompt and schema versions, provider/model identifier, date, relevant settings, and reviewed results. Rerun the evaluation after changing the prompt, schema, model, or provider. Save permitted responses when needed for audit; a future API call may not recreate them. These checks can be ordinary R tests and a results table and do not require a hosted evaluation service [Evaluation practices].

When work stalls

Symptom	Next move
The agent keeps rewriting unrelated files	Narrow the task brief and name the allowed files.
Tests pass, but the result looks wrong	Check the expected result against the method; the test may encode the same mistake.
The session repeats old decisions	Write a handoff and resume with the current diff and one next action.
A collaborator cannot rerun the result	Compare input versions, R/system dependencies, parameters, and commands.
Structured output invents missing values	Allow absence in the schema and add missingness cases to the evaluation set.
Two agents disagree	Ask for competing failure cases and resolve them against the task’s acceptance criteria.