> ## Documentation Index
> Fetch the complete documentation index at: https://docs.multivon.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Agent recipes

> Three short patterns that turn an MCP-connected agent into a working eval loop.

These are the prompts you'd actually type to a Claude Desktop / Cursor / Claude Code session that has `multivon-mcp` configured. The agent figures out which tool to call from the natural-language request — these recipes just show the shape of the resulting flow.

## Recipe 1: evaluate a RAG output for hallucination

**Use case:** you just shipped a RAG endpoint and want to confirm the model didn't invent facts not present in the retrieved context.

**Prompt the agent:**

> I just got this output from my RAG endpoint. Can you check it for hallucinations?
>
> **Question:** "What is the auto-renewal policy?"
>
> **Retrieved context:** "Section 12.4: This Agreement shall automatically renew for successive one-year terms unless either party gives 30 days written notice prior to the renewal date."
>
> **Output:** "The contract auto-renews annually with 30-day written notice, and includes automatic price increases capped at 5%."

**What happens:**

1. Agent calls `eval_faithfulness(input, context, output)` — and optionally `eval_hallucination(output, context)` to confirm.
2. JSON comes back: `{"score": 0.667, "passed": false, "threshold": 0.9, "reason": "2/3 claims grounded ..."}`.
3. Agent reads the reason field, sees the unsupported claim ("price increases capped at 5%"), and tells you precisely which span hallucinated.

```
score: 0.667 (passed: False), threshold: 0.9
reason: 2/3 claims grounded
  ✓ "annual renewal" — supported by context
  ✓ "30-day notice" — supported by context
  ✗ "5% price increase cap" — NOT in context
```

The agent's follow-up is usually "add a Hallucination evaluator to your CI gate, threshold ≥0.85, and re-prompt with explicit 'only use facts from context' instructions."

## Recipe 2: score an agent tool call

**Use case:** you're building an agent and want to verify it called the right tool with the right arguments on a known trace.

**Prompt the agent:**

> Score this trace. The expected tool was `search_products` with `query="organic flour"` and `max_results=5`. The actual call was `search_products` with `query="organic flour"` and `max_results=10`.

**What happens:**

1. Agent calls `eval_tool_call_accuracy(expected_tool="search_products", actual_tool="search_products", expected_arguments={"query": "organic flour", "max_results": 5}, actual_arguments={"query": "organic flour", "max_results": 10})`.
2. JSON comes back deterministically — no LLM judge involved:

```
score: 0.0 (passed: False)
reason:
  tool name: ✓ expected='search_products', got='search_products'
  arg 'query': ✓
  arg 'max_results': ✗ expected=5, got=10
```

The agent points out which specific argument drifted. No API key needed — `eval_tool_call_accuracy` is pure string comparison.

<Tip>
  Pair this with `eval_g_eval` when you want a semantic score on top — *was the tool call sensible given the user's intent?* — not just byte-equal argument matching.
</Tip>

## Recipe 3: run a pdfhell benchmark + ship the audit pack

**Use case:** procurement wants evidence that your document-AI vendor's model holds up against adversarial PDFs.

**Prompt the agent:**

> Run the pdfhell mini suite against `anthropic:claude-sonnet-4-6`, then package the result as a procurement-ready audit pack.

**What happens:**

1. Agent calls `pdfhell_run(model="anthropic:claude-sonnet-4-6", suite="mini")` — returns a full report dict with `pass_rate`, per-trap CIs, per-case details, suite hash.
2. Agent inspects the report, notes any trap family below threshold (e.g. `footnote_override: 0.7`), and surfaces the specific case IDs that failed.
3. If the run JSON was written to disk, the agent calls `eval_audit_pack(run_json_path, cases_dir, output_zip_path)` to build the hash-chained ZIP. The manifest in the response confirms suite hash + file count.

```
Pass rate: 0.933 (28/30) on mini suite. Per-trap:
  hidden_ocr_mismatch:        1.00  (10/10)
  footnote_override:          0.80  (8/10)   ← below 0.9 gate
  split_table_across_pages:   1.00  (10/10)

Audit pack: /tmp/audit-pack.zip (412 KB, 64 files)
Suite hash: sha256:abc1234... — verifiable from manifest.json
```

The agent's follow-up is usually a diff of the two failed `footnote_override` cases — model output vs expected — so you can see whether the model dropped the carve-out clause specifically.

## Recipe 4: did my fix actually help? (multivon-mcp 0.3.0)

**Use case:** you just refactored a RAG pipeline and want to know whether the new version regressed on any case the old version got right.

**Prompt the agent:**

> Compare `runs/before.json` against `runs/after.json` and tell me what got worse.

**What happens:**

1. Agent calls `eval_compare_runs(baseline_json_path="runs/before.json", new_json_path="runs/after.json")`.
2. Response includes `pass_rate_delta`, per-case `regressions` (cases that passed before but fail now), `improvements`, and a McNemar p-value over paired cases.
3. Agent flags the regressions case-by-case so you can decide whether to ship.

This is the loop the marquee /eval scenario was already gesturing at — now it's one MCP call.

## Recipe 5: synthesise an eval suite from your docs (multivon-mcp 0.3.0)

**Use case:** you want to bootstrap an eval suite from a FAQ or product doc and don't want to handwrite cases.

**Prompt the agent:**

> Generate 20 RAG cases from `docs/faq.md` so I can score my pipeline against them.

**What happens:**

1. Agent reads the doc.
2. Calls `eval_generate_cases(from_text=<file content>, n=20, task="qa")`.
3. Gets back a list of `{input, expected_output, context}` dicts — drop straight into `EvalCase(...)` and you have a runnable suite.

## Recipe 6: score the trace your agent just executed (multivon-mcp 0.3.0)

**Use case:** the agent ran a multi-step task and wants to evaluate its own trajectory.

**Prompt the agent:**

> Score the trace from the run you just executed.

**What happens:**

1. Agent dumps its trace as a JSON dict (supported shapes: LangGraph, OpenAI Agents SDK, or canonical manual format).
2. Calls `eval_ingest_trace(trace_json=<dict>, framework="langgraph")`.
3. Receives an `EvalCase` with input + steps + tool\_calls. Can then pipe through `eval_tool_call_accuracy` / `eval_step_faithfulness` for the actual scoring.

## Discovering tools mid-session

If the agent isn't sure which evaluator to use, it can call `eval_discover` to dump the full catalog. The response includes every shipped evaluator, its tier (`deterministic` / `llm_judge_qag` / `agent_trace` / `compliance` / `safety` / `multimodal` / `consistency`), the import path for direct SDK use, and which `(evaluator, judge_model)` pairs have shipped calibration data.

For agents that don't speak MCP, the same catalog is available via the CLI: `multivon-eval discover --json` and `pdfhell discover --json`.

This is what makes multivon-mcp self-describing — the agent never has to read documentation to know what's available.