PDF field detection: What broke across 23 real forms

The result

0.8506 micro F1, with a 0.7425 document average

Across 23 documents, the answer key contains 2,130 fields. The detector returned 1,912 predictions: 1,719 true positives, 193 false positives, and 411 false negatives. A match required at least 0.50 intersection over union (IoU) on the same page.

Micro F1 pools every field, so a 356-field document can move it sharply. Macro F1 gives each document equal weight. The 0.1081 gap between them is the warning label: performance varies more by document than the headline number suggests.

Method

What counted as a detected field

We drew a rectangle around each fillable region on the pages a user would complete. Instruction-only pages weren't annotated. The evaluator paired predictions and truth rectangles one to one on the same page, then counted a pair as a match when its IoU reached 0.50.

The score is geometry-only. Field type is ignored, and F1 is the harmonic mean of precision and recall. We omit type accuracy because eight documents still carry generic, unreviewed type labels.

Prior work exists and we didn't invent this problem. CommonForms is the large public dataset, and AutoFormBench (Northern Kentucky University, 2026) annotates 407 forms to compare OpenCV against four YOLO architectures. Ours is a much smaller corpus with hand-drawn rectangles, and its value is the per-document detail rather than scale.

The corpus is a committed regression set rather than an unseen test. 20 documents are development documents. The other three form a committed validation subset, which means they are visible to the test gate. A separate unseen holdout exists outside the repository, but it hasn't been scored for this article.

Several official PDFs were flattened before annotation. In normal use, pdf2fill reads existing AcroForm fields directly when they are present. This benchmark removes that easier path and measures only the fallback detector for digital pages made from text, lines, vector shapes, and embedded raster elements.

Failure map

A blank line can use six visual primitives

A person sees a place to type. A PDF may encode that place as underscores, a horizontal rule, a bordered rectangle, nine adjacent comb cells, a box nested inside a larger border, or a font glyph that only looks like a checkbox.

Rules and underscores were the easy half

The original detector was strongest on underscore runs and ordinary ruled blanks. Documents built around those marks commonly scored from the high 0.70s into the 0.90s. That success didn't transfer cleanly to bordered cells and combs.

Some checkboxes were hidden inside bigger contours

An early contour pass kept only the outside edge of a connected shape. Small checkboxes nested inside a form border disappeared before classification. Reading the contour hierarchy recovered 46 true positives on the development documents, with no new false positives, in a round that moved overall F1 from 0.7312 to 0.7510.

Higher resolution couldn't turn a diamond into a square

One 356-field patient intake document contained 251 checkbox annotations. The raster checkbox pass found none at 150, 225, 300, or 450 DPI. At 300 DPI, an isolated diamond covered 60.9% of its bounding box, below the square detector's 85% extent floor. Of the 251 checkbox annotations, 248 sat on lozenge characters in the text layer.

An allow-listed glyph pass later recovered all 251 on that targeted document, plus 32 more across two other documents. The full-corpus change added 283 true positives and four false positives, with no per-document F1 regression. That document was a development case, so it isn't independent evidence of generalization.

Comb fields forced an annotation decision

A row of nine character cells can be annotated as nine boxes or as one field with nine positions. The product needs the second version: one place to type the full value. We changed the answer-key convention to one field per contiguous comb run, even though the detector still emitted individual cells. That made the answer key more useful and made the weak score more visible.

Labels can live inside the box

Some intake forms print a caption inside the same bordered area where the answer belongs. A filter designed to reject boxes containing text then rejects the input itself. The geometry alone can't reliably tell whether the text is a label, a filled answer, or ordinary table content.

What we rejected

Two plausible fixes failed measurement

Rendering the 4-page patient document at higher DPI raised peak memory from 147 MB at 150 DPI to 582 MB at 450 DPI. Checkbox recall stayed at zero. The problem was shape, so extra pixels only made the same mistake more expensive.

We also widened the accepted band for long labelled rules. The broad variant found six more true fields and added 23 false positives, reducing overall F1 from 0.7510 to 0.7474 with regressions on five documents. A left-label-only variant added one true positive and one false positive, barely moving F1 to 0.7511. Both variants were dropped.

Answer-key defects

A correct box can fail against a wrong rectangle

On an anonymized bank direct-deposit form, nine drawn checkboxes measured 9.24 by 9.24 points. The hand-drawn truth rectangles measured 16.50 by 14.00 points, 170.6% more area. The actual vector box reached only 0.3696 IoU against that truth, so a geometrically correct detector couldn't cross the 0.50 match threshold.

We corrected those nine rectangles from the PDF's vector content, not from detector output. The unmodified engine's score stayed exactly the same because it had missed all nine boxes before and after. A second mechanical audit later snapped 34 noisy checkbox annotations on another development form. Its IoU-0.50 score stayed unchanged, while its stricter IoU-0.75 F1 moved from 0.38 to 0.78.

That audit matters because tuning against a hand-labelled corpus can optimize both the detector and the mistakes in its answer key. Corrections need an independent geometric source and a before-and-after score for the untouched engine.

Development sensitivity

The biggest win was also the most targeted

Patient intake form A contributes 356 annotated fields, 16.7% of the whole corpus. We diagnosed its checkbox glyphs directly and built the glyph pass against that failure. Its final score is 0.9944 F1.

Removing that one document leaves 22 documents and 1,774 annotated fields. Micro F1 falls from 0.8506 to 0.8196. We publish both figures because the lower one better shows how much the headline depends on a targeted development case.