Trap families - Multivon Docs

PDF Hell 0.6.1 ships 17 deterministic trap families. Each generator returns a PDF plus a code-derived answer key. The mini-v4 suite uses 30 seeds per family (510 cases); mini-v4-sample uses 10 per family (170 cases).

Family	What it isolates
`hidden_ocr_mismatch`	Invisible OCR text disagrees with the rendered amount.
`footnote_override`	A small footnote changes or overrides the body rule.
`split_table_across_pages`	A table header and its rows are separated by a page boundary.
`composite_trap`	Multiple document hazards interact in one answer path.
`scale_dependent_rendering`	Meaning depends on text that becomes fragile at small visual scale.
`cross_page_coreference`	A reference on one page must be resolved using another page.
`unicode_confusable_total`	Visually confusable characters alter a numeric total.
`zero_width_space_split`	Invisible Unicode separation disrupts token or value reading.
`currency_mismatch_conversion`	Currency labels and conversion context conflict.
`mirrored_footer_notice`	A transformed footer contains a controlling notice.
`em_dash_minus_sign`	A dash-like glyph changes subtraction or sign interpretation.
`upside_down_amount`	A transformed amount must be read with its page geometry.
`checksum_validation_rule`	A stated checksum rule determines which value is valid.
`mirror_image_glyphs`	Mirrored glyphs must be interpreted visually rather than as text order.
`boldface_binding_rule`	Bold styling binds a qualifier to the correct value or clause.
`shaded_box_binding_rule`	A shaded region determines which label and value belong together.
`color_grounding_trap`	Color is required to ground the answer to the right element.

Reproduce and inspect

pdfhell list-traps
pdfhell discover                 # machine-readable catalog
pdfhell make --trap hidden_ocr_mismatch --seed 7001
pdfhell build --suite mini-v4-sample --out cases/mini-v4-sample

The family name and seed are the unit of reproducibility. A suite adds a version and hash for the exact collection of those pairs. Compare model runs only when their suite hashes and input modalities match.

Interpreting failures

Scoring checks the procedural expected answer. A family can also list forbidden_answers that identify the particular lure the model followed; this is diagnostic and does not replace the primary correctness score. Refusals and provider/API errors are recorded separately from ordinary wrong answers. For text-layer traps, run a matched pixels-only experiment to separate provider PDF ingestion from visual model behavior:

pip install 'pdfhell[pixels]'
pdfhell run --model openai:gpt-4o --suite mini-v4-sample --pixels --dpi 150

Pixels-only is a different modality, not a drop-in replacement for the PDF score. Keep the DPI and rasterizer build with every result.

The two Unicode families, zero_width_space_split and unicode_confusable_total, were redesigned in 0.6.1 after earlier versions produced visible replacement glyphs. Use newly generated cases for any current claim about those families.

Suite registry

CLI name	Version	Cases	Suite hash
`smoke`	smoke-v1	3	`8cb2f6ab`
`mini`	mini-v1	30	`8ad87b8d`
`mini-v2`	mini-v2	180	`a0385d55`
`mini-v3`	mini-v3	300	`6407e7bb`
`mini-v4`	mini-v4	510	`9707abe1`
`mini-v4-sample`	mini-v4-sample	170	`a451ce10`

The hashes above identify the registries shipped by PDF Hell 0.6.1. The JSON run artifact is authoritative for a particular experiment.

​Reproduce and inspect

​Interpreting failures

​Suite registry

Reproduce and inspect

Interpreting failures

Suite registry