All guides

Production AI Radar

How to build an agent tool harness with LangGraph approvals

Bounded agent graphs with tool allowlists and human-in-the-loop nodes — production agents that cannot freestyle into prod writes.

TrialLLMOps15 min
Agent harness · approvals

Hover a node · click to focus · ←/→ steps

Spec → allowlisted tools → model loop. Writes pause for human approval; MCP goes through an allowlist gateway.

When you need this

  • Agents call arbitrary tools with no allowlist
  • Need human approval before CRM/email side effects
  • Monolithic agent prompt is unmaintainable
  • Want spec-driven graphs instead of open-ended loops

Prerequisites

  • Tool inventory with risk tiers (read / write / external)
  • LangGraph (or equivalent) runtime
  • Approval UX (UI, Slack, or Temporal signal)

Tools

  • Assess for complex agents; keep simpler RAG as plain pipelines until graph complexity pays off.

  • Strong for prod agents; assess ops learning curve vs simple queues.

  • Deploy as single ingress before adding a second LLM vendor.

Steps

  1. 1

    Write the agent spec

    Goal, allowed tools, data scopes, success criteria, and stop conditions. Reject monolithic ‘do anything’ agents for prod.

  2. 2

    Implement the graph

    Nodes for plan → tool call → observe. Explicit edges; max iterations. Tool nodes only call allowlisted functions.

  3. 3

    Insert approval gates

    Interrupt before write/external tools. Persist state; resume on human decision. Log approver identity.

  4. 4

    Route model calls through the gateway

    All LLM calls via LiteLLM with team tags. Guardrails on untrusted tool outputs before they re-enter the prompt.

  5. 5

    Evaluate and harden

    Scenario tests for injection via tool results. Trace graphs in Phoenix/Langfuse. Optional Temporal wrapper for long-running jobs.

Adoption pitfalls

  • Open-ended tool registry in prod
  • Approvals that can be skipped via another code path
  • No budget on agent loops → runaway cost

Adoption checklist

  • Prod agents have a written spec
  • Write tools require human approval
  • Tool allowlist enforced in code
  • Max iteration / budget limits set
  • Injection tests for tool outputs in CI

SEER REAL assessment / sprint

Assessment flags monolithic agents and missing approvals. Sprint specs and ships one LangGraph harness with allowlisted tools and a write-approval gate.

Related radar blips